Now with these guideline the next question comes which should be considered to Pilot project?

Pilot A Test Automation Project

In pilot project we mostly need to automate the smoke testcases. Make it as easy as possible, around the GUI aspect- dropdowns,popups,menus,lists,etc(controls), then Edges(Certain,definite,number of edges).

Every edge connects two consecutive components.Every two nodes or controls of G joined by an unique path.
So control( C)=number of edges+1.
Total testcases could be covered=Controls-1.

In GUI we do not have statement coverage rather we have control coverage. Picking up easy staff to automate that provides value and quick feedback should be the first one to pick up.

This will give all the stakeholders a comfort layer to think more.
Then try to make wrappers around those. Once they are done, pick up few functional testcases. Somebody may ask “why we need to work with controls?”

The reason is simple. start with small step forward by targeting the only single important flow. Lean and mean approach to follow in pilot.But at the same time do not loose the bigger aspect of automation.

There are two aspects to it:

• • The manual tests tends to focus on the limited area and often to words business needs. They mostly miss to cover huge numbers of GUI components presents in an application.Automating these area will reduce the risk of undetected bugs around controls.Manual way of regression testing is a big failure due to various factors.More over, it is loss of time and man power.Introduction automation (if put correctly) will surely reduce the manual effort by 60%. That means manual testers or functional testers get more time to think and design new testcases.In today’s rapid speed development,agile is the way to go and TDD or BDD can be an efficient manner to support automation Testing.
• These become the basic building blocks for framework. They may be the low level functions or components that will be stitched together to make a business flow.

• Pilot A Test Automation Project

  The tests which are repeatable in nature ,lengthy and boring should be the first choice. This cases are the pain key areas of manual process. Once automated, they will bring more value add. Next comes the complex cases.Introducing automation for a quick win may destroy automation for a long term.

• So be very careful on this.The only success criteria for a successful automation in Agile is “One Goal for All”. Everybody in the team(Dev,testers,business analysts,owners etc have some role to play.).

  Agile way of developing software is not not the only criteria.Clear communication,impact area analysis ,Relationship between team members,building automation script criteria are few notable discussion area.

  In pilot project the tool sales person provide all necessary support. But while automating a piece , It is better  to take help from somebody who knows the tool or the language.Sales pitches are often misleading.

  Pilot A Test Automation Project

  There is a big myth around automation that test automation is real easy and simple. This is due to super cool sales pitch from the tool’s sales team.But all we need to understand all testers can not develop automation and it is as good as development task.
  The second big myth or misleading information is “commercial tools are way expensive”.

  To minimize the cost often organizations go for free tools and develop their own. Use scripting language like perl and ruby. Use shareware tools or do not go for automation at all. But in reality the cost of the tool is 3 to 10% of the total testing cost.

  So the pilot project will be more favorable to reputable and established creators/vendors as they are less vulnerable to fad driven ,24*7 support availability , But open source products should also be considered.

  Pilot A Test Automation Project

  Development team can play an instrumental role during pilot project. Also major success for automation depends on the relationship between developers and testers. Pilot Projects are right place to make this nexus.

  By educating development team about the automation requirements,this will reduce the rework viz regression development significantly.The cost of regression development is too high This can lead to low ROI.

• Pilot A Test Automation Project

  Testcases should be outside of the test tool. Use of Quality centre, OATS or some repository for this purpose. In this case even your pilot fails the testcases can re reused for other pilot projects. Only the controls and wrappers need to change.With every successful testcases,mock the testcases via automation as far as they are realistic.Remember tests may be run over several other platforms.
  While doing a pilot project, we are suppose to provide the best tool for our customer or for our organization. Test tool selection should be based on:

  • Robustness
  • Track record
  • Cost of the tool
  • Cost of implementation of scripts
  • Rolling out automation on the floor cost
  • Performance
  • Timely manner of execution
  • Less learning curve
  • Available technical support
  • features available etc.

  Pilot A Test Automation Project

  So create a score card and provide number may be 0-10 scale and then compare which suites to your need.Benefits of using commercial tools:

  1. Customer support base is huge. open source or in-house tool do not have support base or having less support base.So research,development and up gradation time is saved.
  2. Commercial tools are being constantly updated to cope up with current technology trend.So it can support multiple technologies where as the opensource tools will have less or very specific support to a particular technology.
  3. Commercial tools will have huge support like forums ,blogs and how to guide where as opensource tools will have less forum support.
  4. Plug and play easy frameworks are readily available for commercial tools
  5. Commercial tools can be easily integrated with Test management tools which makes reporting and execution easy.

• Pilot A Test Automation Project

  It is essential to keep the testers out of pilot as much as possible. Create distinct roles for automator and testers.
  Documentation is another important aspect to look for. Tools documentation helps the test engineers to understand better.It reduces the training timeline by expediting the knowledgebase. FAQ page could be another great idea to understand the common pitfalls and how to get rid of those.
  validate few data conditioning test while doing Pilot project.Date testing and screenshot capture testing should also be there in the pilot project checklist.
  What is very important while doing pilot is to maintain stakeholders updated , to meet their interest and support with report.

  The pilot report should be publish as early as possible. Frequency matters!!!. So be prepare for sending report.
  UI level tests are very unstable , fragile ,expensive to maintain. Hence automation must look out for an alternative to carry out in some other way. Like-API testing. While piloting a project,automators must look at the feasibility to introduce API testing.

Pilot A Test Automation Project From an automation stand’s point, unit testcases are very important assets. They are very easy to develop and can execute in few seconds. While piloting, we must target such testcases , which will give not only speed but also confidence. But smaller GUI automation success , expectation and enthusiasm become high(uninformed optimism) with this mostly the initial scope is forgotten. The new scope become larger,technologically more complex with the initial timeline so pilot project engineers need to be very careful about the expectation.
While piloting a project there might be one more scenario that is migration (from one old tool to new one). It is always better to pick up fresh or new flows for piloting rather than picking up faulty flow. I encountered such a situation. What I did , I did not pick up the flows which were faulty or were running fine with old tool.

Pilot A Test Automation Project

I supported all new features and testcases with new tool and slowly migrated the old flows to new one. It is always said that don’t change it until it is broken. I obeyed the same. My old testcases were running in old system where as the new testcases were started running in new tool.

For each approach we take to introduce automation needs an explanation to management. We need to be clear why I am choosing a tool, language,cost and most importantly the ROI. Always remember management support for automation is one of the prime factor for successful automation.A realistic objective , provided sufficient time and a appropriate team will surely achieve the desired ROI. Introducing automation to an organisation is great but always keep all the stakeholder updated about the fact that it is an automated way of testing not automatic testing. Scripts can only do any testing what is already coded but can not cover new functionality development