Mobile Testing 简明教程

Mobile Testing - Application

移动应用程序测试的一个简单定义如下：“ Mobile application testing 是为手持移动设备开发的 application software 的功能性、可用性和一致性进行测试的过程。移动应用程序测试可以是自动化或手动类型的测试。”

A simple definition of mobile application testing would go like this “Mobile application testing is a process by which an application software developed for handheld mobile devices is tested for its functionality, usability, and consistency. Mobile application testing can be automated or manual type of testing.”

Note - 为了更好地理解，我们假设我们正在测试一个用于在线航班订票系统的移动应用程序。

Note − For better understanding, we shall assume that we are testing a mobile application for online flight ticket booking system.

Functional Testing

功能测试是确保任何应用程序按照定义要求工作最基本的测试。与其他基于用户界面的应用程序类似，移动应用程序需要用户在用户场景中进行大量的人机交互。

Functional testing is the most basic test for any application to ensure that it is working as per the defined requirements. Similar to other user-interface based applications, mobile applications require a number of human interactions in user scenarios.

示例测试场景 -

Example test scenarios −

  1. Verify that flight availability is displayed for a chosen source destination on a selected date only.

  2. Verify that past dates are not included in the search results.

Compatibility Testing

相容性测试是移动应用测试中涉及范围最广的部分。移动应用相容性测试的目的通常是为了确保应用的关键功能在特定设备上按预期运行。相容性本身只需要短短几分钟，还可以提前规划。

Compatibility testing has got the highest stack when it comes to mobile application testing. The purpose of a mobile app compatibility test, in general, is to ensure an app’s key functions behave as expected on a specific device. The compatibility itself should only take a few minutes, and can be planned well in advance.

决定应在哪些移动设备上执行相容性测试并非易事（因为无法对所有可用设备进行测试）。因此，应准备一个包含所有可能组合的测试矩阵，并让客户对其进行优先级排序。

It is not going to be an easy task, deciding on which mobile devices compatibility tests should be performed (as testing with all available devices is merely impossible). So prepare a test matrix with every possible combination and get it prioritized by the client.

示例测试场景 -

Example test scenarios −

  1. Verify that flight search is performed successfully with Android device.

  2. Verify that flight search is performed successfully for Apple iPad.

Localization Testing

如今，大多数应用都设计为在全球范围内使用，因此非常重要的是关注区域性问题，例如语言、时区等。当用户更改时区时验证应用功能非常重要。必须考虑到，有时西方设计可能不适用于东方国家受众，反之亦然。

Nowadays, most of the apps are designed for global use and it is very important to care about regional trails like languages, time zones, etc. It’s important to validate the app’s functionality when someone changes the time zone. It has to be taken into consideration that sometimes western designs may not work with the audience from eastern countries or vice-versa.

示例测试场景 -

Example test scenarios −

  1. Verify that there is no UI or data truncation issue when we use the mobile app with different languages (or say, non-English language).

  2. Verify that time zone changes are handled gracefully for your mobile application.

Laboratory Testing

实验室测试通常由网络运营商执行，通过模拟完整的无线网络来进行。执行此测试是为了找出移动应用在使用语音和/或数据连接执行某些功能时出现的任何故障。

Laboratory testing, usually carried out by network carriers, is done by simulating the complete wireless network. This test is performed to find out any glitches when a mobile application uses voice and/or data connection to perform some functions.

示例测试场景 -

Example test scenarios −

  1. Verify that there is no glitch while a customer is having voice chat with the support staff.

Performance Testing

移动性能测试涵盖客户端应用性能、服务器性能和网络性能。确保性能测试场景涵盖所有这些领域非常重要。借助性能测试工具，可以轻松识别现有网络、服务器和服务器端应用瓶颈（给定预定义的负载和事务组合）。

Mobile performance test covers client application performance, server performance, and network performance. It is important to make sure that the performance test scenarios cover all those areas. With the help of performance testing tools, it is not difficult to identify the existing networks, servers, and server-side application bottlenecks, given the predefined loads and transaction mix.

示例测试场景 -

Example test scenarios −

  1. Verify that flight available check takes only a reasonable amount of time.

  2. Verify that while flight availability is being checked, the mobile operates normally and does not hang.

Stress Testing

压力测试是必须的，可以找出功能和用户界面测试中可能被忽视的异常、挂起和死锁。以下是压力测试的一些标准 -

Stress testing is a must to find exceptions, hangs, and deadlocks that may go unnoticed during functional and user interface testing. Here is a list of some of the criteria for stress testing −

  1. Load your application with as much data as possible to try to reach its breaking point.

  2. Perform the same operations over and over again.

  3. Perform the repeated operations at varying speeds – very quickly or very slowly.

  4. Leave your application running for a long period of time, both interacting with the device and just letting it sit idle, or performing some automatic task that takes a long time, for example, a slideshow.

  5. Randomly send screen taps and keystrokes to your application.

  6. Have multiple applications running on your device so that you can switch between your application and other device applications often.

示例测试场景 -

Example test scenarios −

  1. Check 1000 users are accessing the mobile app to search for domestic flights.

  2. Check 1000 users are accessing the mobile app to search for international flights.

Security Testing

破解、认证和授权策略、数据安全、会话管理和其他安全标准的漏洞应作为移动应用程序安全测试的一部分来验证。应用程序应在通过网络认证用户时加密用户名和密码。

Vulnerabilities to hacking, authentication, and authorization policies, data security, session management and other security standards should be verified as a part of mobile app security testing. Applications should encrypt user name and passwords when authenticating the user over a network.

测试与安全相关的场景的一种方法是通过代理服务器（如 OWASP Zed Attack Proxy）路由手机数据，并寻找漏洞。

One way to test security related scenarios is to route your mobile’s data through a proxy server like OWASP Zed Attack Proxy and look for vulnerabilities.

示例测试场景 -

Example test scenarios −

  1. Verify that the application does not get operated with same user credentials on two different mobile devices.

  2. Verify that a session automatically gets expired if it remains inactive for more than 15 minutes.

Memory Leakage Testing

与其他计算机相比，移动设备的内存非常有限，而且移动操作系统默认会终止使用过多内存并造成不良用户体验的应用程序。

Mobile devices have very limited memory as compared to other computers, and mobile operating systems have a default behavior to terminate applications that are using excessive memory and causing a poor user experience.

内存测试对于移动应用程序非常重要，以确保每个应用程序在整个用户使用过程中保持优化的内存使用情况。建议我们在实际目标设备上进行内存测试，因为系统架构与模拟器与实际设备不同。

Memory testing is exceptionally important for mobile applications to ensure that each application maintains optimized memory usage throughout the user journey. It is recommended that we conduct memory testing on the actual target device, since the system architecture is different from an emulator to an actual device.

示例测试场景 -

Example test scenarios −

  1. Do flight availability checks ten times and note down increasing memory usage for each check.

  2. Keep the application running for ten minutes and observe if memory usage remains stable.

Power Consumption Testing

不同的移动设备中使用着几种类型的电池（即镍镉/锂离子/镍金属混合）。在关注功耗测试时，我们需要测量每个活动级别的电池状态。这将使我们更好地了解单个应用程序的功耗。

There are several types of batteries used in different mobile devices (i.e. nickel cadmium/ lithium ion/ Nickel metal hybrid). While we focus on power consumption testing, we are required to measure the state of the battery at each activity level. It will give us a better understanding of power consumption by an individual application.

功耗测试可以通过手动进行；市场上也有一些免费工具，如 Trepn Profiler、Power Tutor 和 Nokia Energy Profiler。这些应用程序可以显示智能手机或平板电脑上的实时功耗。

Power Consumption test can be done manually; also there are some free tools available in the market such as Trepn Profiler, Power Tutor, and Nokia Energy Profiler. These are applications which can display the real-time power consumption on a smartphone or tablet.

示例测试场景 -

Example test scenarios −

  1. Use the mobile app to search for flight availability, and check the power consumption remains minimal.

  2. Keep the mobile app in ideal condition; verify that there is no power consumption when there is no activity happening for the app.

Interrupt Testing

应用程序在运行时可能会遇到各种中断，如来电或网络覆盖中断和恢复。这可以再次区分为 −

An application, while functioning, may face several interruptions like incoming calls or network coverage outage and recovery. This can again be distinguished for −

  1. Incoming and Outgoing SMS and MMS

  2. Incoming and Outgoing calls

  3. Incoming Notifications

  4. Battery Removal

  5. Cable Insertion and Removal for data transfer

示例测试场景 -

Example test scenarios −

  1. Verify that flight availability check gets paused and resumed back after receiving an incoming call.

  2. Verify that the user can reject a call while using the App and again resume the same App thereafter.

Usability Testing

可用性测试根据以下三个目标人群标准评估应用程序：

Usability testing evaluates the application based on the following three criteria for the target audience −

  1. Efficiency − The accuracy and completeness with which specified users can achieve specified goals in a particular environment.

  2. Effectiveness − The resources expended in relation to the accuracy and completeness of goals achieved.

  3. Satisfaction − The comfort and acceptability of the work system to its users and other people affected by its use.

在应用程序设计早期进行可用性测试非常重要，不要仅在应用程序完成时才进行。可用性测试需要用户的大量参与，并且输出结果可能会影响应用程序设计，而在项目的后期阶段更改起来非常困难。

It is very important to get usability testing in place from the early stage of the application design, and it should not be done only when the application is completed. Usability testing requires heavy involvement of users, and the output might affect the application design, which is very difficult to change in the later stages of the project.

示例测试场景 -

Example test scenarios −

  1. Flight availability check should be on the home page.

  2. Sponsored advertisement should not be displayed in the middle of the content.

Installation Testing

安装测试验证安装过程顺利进行，用户无需面对任何困难。

Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty.

示例测试场景 -

Example test scenarios −

  1. Verify that the installation process is smooth and does not take long.

  2. Verify that the installation is successful through enterprise app store.

Uninstallation Testing

卸载测试的基础知识可以概括为一行“卸载应该一次性清除与 App 相关的数据”。

The basics of uninstallation testing can be summarized in one line as “Uninstallation should sweep out data related to the App in just one go”.

示例测试场景 -

Example test scenarios −

  1. Verify that all the application related files are removed successfully after uninstallation.

  2. If it is an App which stores media files (like Whatsapp or Facebook), retain the files even after the uninstallation of the App.

Updates Testing

我们应密切关注移动应用程序更新。用户经常抱怨更新后应用程序无法令人满意地工作。因此，在更新测试中，我们必须明确指出该 App 将按其以前的工作方式工作，这一点非常重要。简而言之，它不应破坏任何东西。移动应用程序更新可以通过两种方式进行： Automatic update 和 Manual update 。

We need to be very much cautious about mobile app updates. People frequently complain about applications not working satisfactorily after an update. So it is very important that under the update testing, we qualify that the App will work as it was working previously. In a nutshell, it should not break anything. Mobile application updates can take place in two ways − Automatic update and Manual update.

示例测试场景 -

Example test scenarios −

  1. Verify that the application works successfully after automatic update.

  2. Verify that the update progress is displayed properly.

Certification Testing

若要获取合规性证书，需要根据不同移动平台制定的准则对每个移动设备进行测试。

To get a certificate of compliance, each mobile device needs to be tested against the guidelines set by different mobile platforms.

示例测试场景 -

Example test scenarios −

  1. Verify that the application adheres to the policy of iOS phones when installed on iPhone.

  2. Verify that the application adheres to the policy of Android phones when installed on Android.