Software systems are an integral part of life from business applications to consumer products. Quality has become a major paradigm these days. Software Testing itself is a vast area to be classified. Primarily, testing can be performed in two ways; either manually or automated testing. It is very crucial to decide when to choose manual or automation testing during software development to ensure quality meeting project constraints. Hence, latest deployment methods are required to provide well-functioning software in production as frequent as possible. This is where Continuous Integration and continuous deployment comes into use.

What is CI/CD?

It means continual integration of the code to the relevant branch of working project. It implies that “integration tests” are run at every code change while Continuous Delivery means that every change that passes tests are automatically deployed.

For instance, this technique can be implemented through agile model in which teams work on small increments and deploy it to live environment after each sprint.

Unfortunately, this approach introduces problems due to troublesome testing strategies followed in numerous companies.  Some organizations emphasize on automation and fully avoid manual testers. Others concentrate on automation testing but not allowing testers to get involved in test strategies.  Therefore, a moderate approach needs to be followed to take full advantage of automation testing.

Verification vs. Testing

The test strategies created in a CI-environment where focus is on automation, mostly guarantees checking or verification and negligible testing (as in exploration). Hence, testers could not find something out of the box. Good testing includes both.

In addition to this, focusing on automation checks fulfilling the acceptance criteria when developing a new feature will only enable you to reach a small part of the way towards good quality. It can seem easy to develop and check (not test) the functionality. In addition to this, it is thought to perform the same tests repetitively. But they are failed at finding one important scenario which could be the most valuable one. On the flip-side, if focus is on finding things which have not been considered by the team yet would ensure great quality.

Advantages of manual over automation

It is impossible to test each and every scenario using automated tool. When an application or product is arrived for testing; testing team starts planning its software testing life cycle activities. A number of test cases are created for the application beforehand to carry out the testing. Some test cases would be non-repetitive which might be complex and too costly to automate. There are other tests which are repetitive and are executed every time the build is received. It could be functionality that must be in a working condition at all times. Out of all those, the desired test cases which we want to automate are filtered.

Furthermore, only a fraction can be automated out of the planned ones because automation is time consuming and difficult task.  Apparently, it will not provide full assurance to the stakeholders in context of quality.

In addition to this, automation testing cannot replace human implications, intuitions and reasoning. Only a known set of test cases are automated in real life scenario. The most of test automation is performed on checking one view of the system. Testers have the tendency to design repeatable tests on items that are familiar.

This repeatable checking finds things that were conjectured to be known. These items can probably change as time passes with the system development. In this area specifying the expected outcome correctly is feasible.

Exploring beyond the specification will recognize things to be explored and learnt more about. This might include areas for research, particular risks or just a desired idea to understand. It is not possible to clearly state outcome before examining here. Moreover, it is unlikely to automate such things.

Testers have the tendency to discover new things while exploring/testing.

Occasionally, testers identify something surprisingly valuable. Similarly as mentioned before, such scenarios can also not be automated. Automation is majorly performed on the known things. Hence, it would not be ok to ignore things that are valuable and not known until enough time is spent on testing or exploring.