Last year, 60% of consumers planned to spend more than 50% of Holiday shopping online. Now, are you ready for the holiday season this year? While it’s true that the time for a shopping spree is yet to come, it’s essential to get prepared on offering your customers exceptional shopping experience sooner than later. And a well-structured testing plan is what you need to get the holiday readiness – in this context, functional and compatibility testing, performance and load testing, and proactive end-to-end monitoring.

We’ll be covering all these three in our series on how to make the mobile shopping experience fun, convenient and safe for your users. Below we laid out a timeline for going through these stages to maximize your revenue potential. And the first blog post covers the first and most important stage of getting your mobile app ready for increased user interaction – Functional and compatibility testing.

In this stage, you will see all the groundwork for all other phases – its outputs and the actual test scripts that will be used in the next stages as well. Be cautious: cutting corners here will significantly limit the benefits you can reap from the following phases. Timeline-wise, you should start preparing your functional and device compatibility test suite about ten weeks before the peak time (the end of November till the end of December). In the period from Black Friday and Cyber Monday up to Christmas and New Year, app makers and their end-users will either feel thankful or frustrated with QA specialists.

That is why we have to leave enough time for several iterations and iron out the defects found at this stage while still having time to implement the next stages: performance & load testing and proactive end-to-end monitoring.

Preparing the Functional Test Automation Suite

A well-managed functional testing process will let you feel safe while users explore your apps actively during the holiday season. Each element of the process should be thoroughly taken into consideration, including:

• Identifying the proper functions your app is supposed to perform
• Preparing input and output data according to the specification requirements
• Executing test cases and verifying results.

We recommend building a comprehensive test automation suite suitable for both native and hybrid apps as well as the mobile version of your website. Design the suite so that it can be successfully executed against a wide variety of real devices. Nothing wastes more test engineering value than a test suite that produces different results on different devices. Also, don’t neglect usability at this stage. Even though functional and usability testing are usually separated in the mobile testing paradigm, a functional testing approach often supports the testing of the overall system components usability against the requirements. All e-сommerce companies should consider usability check as a critical element of user experience, especially before the holiday season.

The functional and compatibility testing stage can take anywhere from two to six weeks depending on desired functional and device coverage, the number of issues uncovered during this testing cycle and the release velocity to fix uncovered issues.

What Needs to be Covered by Your Test Automation Suite

Before you start preparing the actual steps in the functional test suite, give the conceptual basis a little thinking. It always helps to start with the right questions. This way, you can shift focus and consolidate the testing efforts towards a seamless holiday shopping experience. Consider these questions:

• Are you aware of the bottlenecks of the system or the app?
• Do you know the performance at normal and peak usage levels?
• Do you know which hardware will be used to optimize the business objectives?
• Do you have any statistics of the most common causes of issues with functionality?
• Who are the end customers?

The next step in preparation is to ensure that your functional test suite covers the most common use cases for your app or mobile web app. Typically, all of them are transformed into the following scenarios:

• Installation of the app
• Uninstallation of the app
• Upgrading from a previous version of the app to the latest one
• Making sure that the app resumes at the last operation in case of a crash or reboot
• Creating a new account (pay special attention to validating if all the mandatory fields are working as expected and are not displayed as optional)
• Making any payment gateway transactions in the app (using Visa, MasterCard, PayPal, etc.)
• Upgrading and downgrading the subscription level (if you operate on subscription levels)
• Ensuring that the developed APIs meet the requirements of performance, functionality, reliability and application security
• Ensuring that the API integrations with other services like Facebook, Twitter, Google, etc. work as expected
• Checking if a user can navigate through the most common navigation paths
• Checking if the application provides a user guide for those who are unfamiliar with the app
• Verifying that each view/page is visually correct
• Ensuring that all reporting that app does is working and sends the right data to the server-side
• Checking functionality under particular circumstances, like when the application goes into the background
• Testing conversational commerce elements (if your mobile apps support chatbots, online messaging apps, live support, voice assistants, etc., user satisfaction will depend directly on the way these elements are functioning)

When implementing the test suites, it is vital to make each test atomic so that they can be run individually. This will be very valuable in stage 2 and stage 3 when the same suites can be used in load and performance testing and proactive end-to-end monitoring. An approach like this eliminates the need to run all test cases in the same sequence every time.

The scenarios you keep in mind while preparing test cases can usually be covered by the frontmost types of functional and compatibility testing. So, be sure to include these types into your QA strategy underlying the holiday readiness:

• Unit testing
• Integration testing
• Smoke testing
• Interface testing
• System testing
• Acceptance testing
• Regression testing

Selecting Top Target Devices

So, you have your functional test suite in place and producing robust and repeatable results on a wide variety of real devices. Now, it’s high time to define the set of target devices on which everything must run like clockwork. To double-check that your current real device testing strategy has a great device coverage around the globe, you can download this free device sheet.

In any case, your checklist should contain the following:

1. The most popular Android & iOS devices from this and the previous year

This is a good starting point for building up your target device list. Device compatibility issues are more prevalent on Android devices due to many different manufacturers, form factors, OEM-specific customizations and different OS versions on the market. For iOS, it is equally important to cover the latest popular models as well as the blockbusters from the past few years. The lifetime of each iOS device is much longer than that of an Android device, so be aware of the large number of 2- or even 3-year-old iOS devices used.

2. The right mix of hardware and software

Tablets are used a lot during the holiday season shopping, so it’s important to ensure that your apps will work on devices with different screen sizes and OSs. Speaking of the latter, we recommend covering at least three of the latest major OS versions for both Android and iOS.

3. All expected Holiday Sales blockbuster devices

Numerous mobile devices become gifts, so remember to include the anticipated blockbusters for each holiday season as these will be the devices that are very likely to run your latest app and access your web site.

4. Different browsers on each device

Along with the leading Google Chrome browser that can boast a market share of 56.74% and Safari with 22%, we suggest that you also cover the UC Browser, Samsung Internet and Opera.

5. Consider using device cloud services

Modern mobile app testing platforms are really helpful in automating app tests on hundreds of pre-selected Android and iOS devices available 24/7 with unlimited access, usage and users. Ensuring different hardware and software combinations allows for the flexibility that is much appreciated when preparing for the hot holiday season.

6. Automation efforts

Test automation will help you reach an optimal level of test effectiveness, test efficiency and test coverage. The ultimate value of test automation comes when you stop conducting repeatable test runs and allow test execution to take place 24/7.

The Bottom Line

All your efforts at stage 1 of the holiday readiness should be about functional and compatibility testing. Doing that should result in your app’s or service’s excellent user experience, acceptance, scalability, responsiveness, stability and appropriate navigation. Now add the thoroughly checked device and OS compatibility to the list and praise yourself for a good job. Next – stage 2.

The post Stage 1 in Getting to Holiday Readiness – Functional and Compatibility Testing appeared first on Bitbar.