Mobile app developmentTesting Automation For Apps, Games And The Mobile WebPosted on May 31, 2021 by andra

Testing Automation For Apps, Games And The Mobile Web

Testing is crucial to ensuring success in the highly competitive landscape of mobile apps and games. But even poorly planned testing can take up 20 to 50% of your total development effort, in which case it would also account for the single biggest cost in your budget. To ensure that testing is extremely efficient, covering the breadth of today’s mobile ecosystems and device models, the best option is an online cloud-based service.

Mobile application ecosystems — let’s count Android and iOS 

They are unbelievably dynamic, but they also suffer from both software and hardware fragmentation. This is especially true for Android, but fragmentation also exists in the iOS ecosystem, as experienced with the rollout of iOS 8. As the latest version of iOS was released, many existing apps were made clumsy on updated devices.

Even the new iPhone 6 and iPhone 6 Plus have had not-so-typical issues for Apple devices. In addition, a significant proportion of users with older devices have very few options: essentially, buy new hardware (i.e. a new device) to get everything working well.

In the Android world, things are different. As OEMs launch new devices, software updates and customizations for their devices, application and game developers get serious headaches trying to keep their latest products up to snuff and fully compatible with all possible device variants. Making a certain app or game work only on high-end devices is out of the question, though. Why would a developer want to miss out on a significant chunk of potential users?

Professional automated testing software is a solution to a common problem: how to produce high-quality, robust and reliable software with the ever-growing complexity of technology and under massive competitive pressure. Automated software testing is a cost-effective solution to this problem. Not to mention, it provides three business benefits:

  • increased testing efficiency,
  • increased testing effectiveness,
  • faster time to market.

What You Will Learn

This article walks through a sample use case for test automation and provides a downloadable example to get you started. Also, we’ll focus on different aspects of mobile test automation and explain how this relatively new yet popular topic can help mobile app and game developers to build better, more robust products for consumers. With the advanced example later in the article, we’ll show how image recognition can be used to test mobile games; specifically, we’ll run Appium’s test automation framework against Supercell’s Clash of Clan game to illustrate how image recognition can be built into the test-automation process.

Test Automation Is Perfect For Mobile App Development

Developing mobile applications is very different from developing PC software or even embedded software. Mobile development is meant to be agile, and a lot of great tools and practices have been developed for that agility. However, doing something manually — such as testing an app — is never agile, which is why test automation has shown tremendous growth among app and game developers, speeding up their doings and yielding robust and better results.

To achieve compatibility between users, devices and the market, including test automation as a part of the agile development process is typical. Fortunately, a lot of tools are available, and test automation is a perfect fit for this process. For example, let’s say your typical development sprint is two weeks. You have daily standups and a lot of scrum activities, and you own internal policies that gear development to the final product. Test automation offers a significant value-add by enabling testing to be done in parallel — for example, as nightly test sessions. By the next morning, the tests will have been finalized and the results of the latest regression will be ready for review. Fixing an issue earlier will save a lot of time and get developers to finalize the product sooner; most importantly, it cumulates to better quality, with fewer bugs.

Things To Consider When Automating Mobile App Testing

Let’s look at how test automation (as opposed to manual testing) can improve the development process, add value and speed up development.

Test automation offers the possibility to test mobile apps instantly and effectively. Once tests have been automated, they can be executed quickly and repeatedly, again and again. In almost all cases, this is the most cost-effective method for regression testing of software products that have a long maintenance life. In fact, test automation of any mobile app is the best way to increase the effectiveness, efficiency and coverage of the testing process. The true benefit of automation comes not only from the repeatability of tests, but also from the ability to execute tests that probably could not even be performed manually.

COSTS AND ASSETS

Regardless of whether you go for manual or automated testing, you’ll need the following assets and resources (all of which cost money): time, people, infrastructure, tools and training. Depending on the size of the project and the application, test automation will quite obviously provide a good return on investment. For example, once test cases have been created, automated tests can be run over and over again at no additional cost, and these can be more rapidly completed than any manual tests. Automated software testing can reduce the time required to run repetitive tests from weeks to hours. This is a significant time-saving that translates directly into cost-saving.

INTEGRATED APP DEVELOPMENT AND TESTING CYCLE

Increasing efficiency and productivity with automation actually starts with adopting a new mindset. Software tests have to be repeated often during all development cycles to ensure the best possible quality of an application. Every time source code is modified, software tests should be repeated. For each release, your software should be tested on all supported variants of operating systems and all configurations of hardware. Manually repeating these tests would be costly and time-consuming. For example, running comprehensive tests manually on all variants of Android and on actual devices would take a lot of time.

TOOLS AND TECHNOLOGY: TEST AUTOMATION FRAMEWORKS

To get the most out of your efforts and maximize testing coverage, select the most robust and most cross-platform method. That automated methods can be used both to validate requirements and to reduce costs through automated test-case generation is well known. However, the full automatization of large software entities also comes with a cost that many companies haven’t been ready to pay for. Historically, one of the reasons has been a concern over the lack of adequate integration with well-established development life cycles.

TEST COVERAGE AND REUSABILITY: OPEN STANDARDS MEAN NO VENDOR LOCK-IN

Automated testing can increase the depth and scope of tests and significantly improve software quality. Lengthy and thorough tests — often not doable with manual testing — can be run automatically. Ideally, test cases should have full access to an application and test all aspects of it — memory contents, data tables, file contents and internal program states — to determine whether the product behaves as expected. Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing. Developers, freed from repetitive manual tests, will have more time to create new automated test cases and build more compelling features (and then more test cases).

IMPROVE EFFECTIVENESS AND FINALIZE SOONER

In a nutshell, professional automated testing software is a solution to a common problem: how to produce high-quality, robust and reliable software with the ever-growing complexity of technology and under massive competitive pressure. Automated testing improves business results in three ways: greater testing efficiency, greater testing effectiveness and a shorter time to market.

Different Ways To Automate Mobile Testing

In general, there are three ways to automate the testing of mobile apps.

HANDWRITTEN TEST SCRIPTS

Typically, this is the best choice when you know what you’re doing and when you have programming-capable people doing the test scripts. Plenty of options are available for test automation frameworks, tools and integration — both commercial and open-source offerings. Having your engineers write all of the test scripts will take up some time and tie up resources, but it will get you exactly what you want: well-structured, thorough scripts that test precisely the aspects of your app that you want to test.

RECORD-PLAYBACK APPROACH

This approach is less error-prone because nothing needs to be written in code, but it is typically more limited in functionality. Tests can be quickly recorded and then played back again and again against different OS versions and device configurations. These tests focus on user interactions and user-driven activities. Some things might fall beyond a test’s scope (such as integration with other technologies and compatibility with other software).

AUTOMATIC TEST EXERCISERS

Automatic test exercisers provide a great way to smoke-test applications. No specific tests are needed; rather, the focus is on testing user-interface logic (such as opening menus, clicking buttons, swiping and multi-gesture actions). Automatic test exercisers yield the least exact results but provide quick feedback on each iteration of an app.

Focus Areas In Testing For Mobile Apps and Games

USER INTERFACE AND FUNCTIONALITY

A user interface and its overall functionality will directly affect how successful your app or game will be. These two things, which encompass visual appeal and gameplay, are the most important things to get right — and you must ensure that device fragmentation doesn’t break any of these. Various things in the UI need to be tested:

  • UI layouts and elements. Games especially are typically targeted at a high number of different screen resolutions and screen types. Regression testing should be done each and every time the UI’s layout changes to ensure that the game works.
  • Menu structure and functions. Testing menu structures, functionality and behavior can be automated with instrumentation and the help of different test-automation frameworks.
  • Screen orientation. Surprisingly, so many apps and games out there get this wrong. If a screen’s orientation changes during an interaction, for example, what happens? What is supposed to happen? Does the app or game work well in both landscape and portrait modes?
  • Screen resolution. A lot of screen resolutions exist, especially on Android, and auto-scaling will usually help developers. However, test your game across these resolutions to ensure that the graphics do not stretch.

GRAPHICS PERFORMANCE

Performance needs to be consistent across all device variants among your users. Because of this, test on as many real devices as possible. To determine how well your app or game responds to various levels of usage, including performance and battery usage, consider creating tests that last for hours. To determine whether your game runs effectively under a heavy load for a long time, run load (or stress) tests. These performance tests will measure, for example, how responsive your game is on real devices.

Conclusion

Testing is crucial to ensuring success in the highly competitive landscape of mobile apps and games. But even poorly planned testing can take up 20 to 50% of your total development effort. In this case it would also account for the single biggest cost in your budget. To ensure that testing is extremely efficient, covering the breadth of today’s mobile ecosystems and device models, the best option is an online cloud-based service.

If you only start thinking about testing a few days before the app hits the market, it’s too late. You’ll need to test a wealth of elements, data and functionality from day one.

Tags: app testing, mobile app, mobile app development