Different people have different expectations for the automated testing implementation. It often happens that after a while initial expectations don’t prove their value because considerable investments in Automation don’t make any profits. Let’s try to understand why it happens and how we can avoid common mistakes.

To be or not to be

You really need automated testing if:

• You have a long-term project;
• Your development team is split up, or you have more than two developers in a team. Developer has to be sure that his changes won’t break someone else’s code. Without auto-testing he’ll find it out at best in a day or two, at worst – from users;
• You support multiple product versions and release patches and service packs for each of them. It’s black and white, various configurations testing is a routine, and it definitely must be done away wit;
• You have agile with short iterations and frequent releases. You have very little time to run a manual regression within a sprint, but you need to know that everything is OK;If your project is not like the above mentioned, then you probably do not need automated testing.

Why do we need test automation?

We need any automation to make a person free from routine work. Test automation works in the same way. However, there is also a misconception that test automation should completely supersede manual testers and all the products should be tested with a script. But of course, it is a nonsense. There is no script yet that is able to test. Script can only repeat programmed actions and signal if something went wrong. Script is able to do it quickly and without human intervention.

This feature is used to get information about any change in the product’s quality that is being tested faster than a person can do it. In addition to speed, there are other requirements, which all together make up automation effectiveness: completeness of test coverage, results clarity and reliability, development and support cost, etc. The basic measures of efficiency are speed, test coverage and cost.

Why only UI-tests automation is a huge mistake?

There is a common mistake – decision to implement only UI-tests automation. This decision doesn’t seem so bad while making it. Sometimes it even works for a while. Sometimes you don’t need more, if the product is already at support stage without further developing. But as a rule, long-term projects, which are actively developing require another approach.

UI-test – is what testers do, it is a natural way of programs testing. Moreover, it is a simulation of how users will interact with an application. It may seem to be a perfect variant. But for every plus there is a minus: 

• UI-tests are brittle;
• UI-tests are slow.

UI-tests are brittle because they depend on the page layout. Changing the on-screen buttons arrangement or adding/removing any element may break tests. Automation tools can often change the test logic.

The more such tests you run, the more time you have to spend on support. As a result, you can’t entirely rely on results of these tests due to frequent false-positive response. Eventually, you’ll spend all the time rewriting scripts.

UI-tests are slow because an app’s interface is slow, it requires redesign, resources restart etc. Test script doesn’t work most of the time, it waits.

When UI-testing takes you two days, even on condition that independent test groups are running simultaneously on several servers, it is really hard to use such automation as an indicator of quality in daily practice.

What to do?

Stabilize.

In fact, I overstated the problem of brittle UI-tests intentionally, because it is easy to solve, but QA engineers hardly even try to do that.

The first thing to do in any case – is to ask developers not to forget to use unique attributes, so that automation tool could identify them. It should be clearly stated in the Development Guides and be one of the points in the definition of done for developers. Then even if UI changes a lot, you’ll have a chance to stay shaken but unhurt.

Speed up.

If UI-tests is quite easy to deal with, the problem of slow tests should be solved with all-out approach, because it affects the development process as a whole.

The first and the easiest way that can help here – is to deploy an application and run tests on a “fast” hardware, to avoid situations where the network latency affects interaction of test and application etc.

The second thing to do – is to implement the possibility to use independent and parallel test running from the very beginning. It helps to save time. However, there are also some limitations. You can’t run parallel tests endlessly.

The third and most radical thing to do – is to create fewer UI-tests. The less UI-tests you run – the quickly you get results.

Integrated approach

It is important to realize that unit tests test code, that means they give developers confidence that a part of their code works as intended, and, what is the most essential, that code doesn’t break a code logic of colleagues. It happens because colleague’s code is also covered with unit tests, and developer runs these tests before committing to the repository. 

UI-tests test the whole system, namely, that a user will use. It is critically important to have such tests available.

There is only one general recommendation: you need to have all kinds of automated tests at each stage as needed. Then you can get effective feedback.

Test Driven Development is not even a recommendation, it should come from a developer by default. Only in this case it’s possible to avoid challenges during refactoring and common development problems in large teams.

At the level of UI-testing it’s better to implement acceptance tests, so called Happy Path or End-To-End cases that are shown during the demo. It includes both web and mobile applications.

Try to use these recommendations and you’ll notice that nothing is as bad and hard to do as it looks.