Scenario:

You are trying to Execute Tests using the “Run Functional Tests” task, in a vNext build. Under Test Selection, you have chosen “Test Plan”, and have provided the plan and suite details, as indicated below.

However, the test execution fails with the below message:

“The test Agent being used does not support the feature to execute tests by selecting Test Suite(s)”

Cause:

The reason for this should be a failure that is returned, when the “CheckTestAgentCompat” task runs prior to executing tests, to make sure that the test agent version being used supports test execution from Test Suite(s).

However, in one of the cases that I have worked on, we found that the test agent being used was of the latest version, but still we were hitting the above error message. Upon checking further, we were able to get to the root of this.

There was a certain sequence of actions performed on the test agent machine, that caused this issue. The tests were working fine and started to fail, after installing Visual Studio on the test agent box.

When test agent is installed, it creates two registry entries on the machine, testagentcore and testexeccore at (HKLMSOFTWAREWow6432NodeMicrosoftDevDivvstfServicing14.0).

If visual studio is installed after test agent is installed on the box, then it downgrades the version value for the testexeccore reg value, and this is the cause of the error.

Resolution:

The fix in this case, is to just do a repair for the existing test agent installation.

• Open control panel, and then do a repair for “Microsoft Agents for Visual Studio 2015 with Update 3”
• Once the repair is done, the machine should prompt for a restart – please reboot the machine.

After the repair, the reg key :

HKLMSOFTWAREWow6432NodeMicrosoftDevDivvstfServicing14.0testExecCore should be updated back to the actual value, and the tests should now run fine.

Content: Sreeraj Syama Rajendran
Review: Romit Gulati