Authored by Philip Froese [MSFT]

In the next release of the HLK, the Ucsi tests have been updated to run against the new Type-C SuperMUTT device instead of a partner system. This means the test setup will be simpler, the tests will run more quickly, and test content will be more thorough than in Windows 10 April 2018 Update HLK and earlier. The new preview tests are only in pre-release HLK builds 17676 and higher.

New Test Setup

The new test setup is simple: it requires a single Windows PC with Type-C, designated the System Under Test (SUT), and the Type-C SuperMUTT connected to one of the Type-C connectors on the system. See the image below.

Type-C SuperMUTT

The Type-C SuperMUTT is the newest edition to the Microsoft USB Test Tool (MUTT) family of devices. It incorporates the functionality of the legacy SuperMUTT device as well as Type-C and PD functionality.

The software and firmware utilities are available in the MUTT Package download: https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/mutt-software-package

And the tool is available for purchase through MCCI: http://www.mcci.com/mcci-v5/devtools/type-c-supermutt.html

Firmware version 45 or later, available in MUTT Package v2.6 and later, is required to run the UCSI HLK tests in the next release of the HLK.  See the TypeCSuperMUTT.pdf documentation in the MUTT Package download for more information.

New Test Names

The UCSI compliance tests have been replaced with a new version of the tests to run against the Type-C SuperMUTT. The new version of the test has the tag “[Type-C MUTT]” added to the test name. You will see both the old and the new versions of the test in the next release of the HLK pre-release kit for some time as we want to leave the old version in for at least a few weeks to enable cross-version validation if needed.

What is Different?

Beyond a new name and a simpler test setup, what else is new in the tests? For the most part, the tests cover the same features in the same manner as the original tests did. However, there are some cases where the new tests will be more rigorous.

Take USB Operation Role tests for example. Many desktop UCSI systems do not support being set into the UFP role, though they may resolve to that role initially when connected to a pure DFP (such as a charger). This behavior makes it difficult to test every permutation of DFP/UFP/DRP connections that a platform may have to support if it is tested against a functionally equivalent peer device. However, with the Type-C SuperMUTT the test can deterministically place the Type-C SuperMUTT into a specific, known port direction or configuration before connecting it to the SUT and is thus able to provide more complete test coverage of USB Operation Role features.

Thus, there may be some test cases in next release of the HLK that are more thorough and thus uncover new platform quirks that Windows 10 April 2018 Update HLK and earlier test content did not.

You will also notice that some tests have been removed. Some tests were determined to be redundant when run against the Type-C SuperMUTT and so were removed. Others targeted UCSI commands that have been removed in UCSI v1.1, and which Windows never supported, so were removed as obsolete.

USB Type-C UCSI Data and Power Role Swap Tests

Applicable Tests

• USB Type-C UCSI Data Role Swap
• USB Type-C UCSI Power Role Swap

Hardware Requirements

1. One UCSI compliant Windows system.
2. One Type-C MUTT device.

Test Setup

1. Install HLK client on the System Under Test (SUT).
2. Connect the Type-C MUTT to any USB Type-C port on the SUT.
3. Locate the device node in Device Manager (devmgmt.msc) named "UCSI USB Connector Manager". The node is under the "Universal Serial Bus controllers" category.
4. Right-click on the device, and select "Properties" and open the "Details" tab.
5. Select "Device Instance Path" from the drop-down and note the property value.
6. Open Registry Editor (regedit.exe).
7. Navigate to the device instance path under this key: HKEY_LOCAL_MACHINESystemCurrentControlSetEnumDevice Parameters
8. Create a DWORD value named "TestInterfaceEnabled" and set the value to 0x1.
9. Restart the device by selecting the "Disable" option on the device node in Device Manager, and then selecting "Enable". Alternatively, you can simply restart the PC.

Test Parameters

Troubleshooting

• Some UCSI commands are marked "optional" in the UCSI specification. However, Microsoft requires some optional commands to be implemented in order for the UCSI driver to function. The tests may fail if you've forgotten to implement one of these commands in your BIOS. You can find that list of commands here.
• If the test is not detecting the Type-C MUTT:
  • Check your cable connection to the Type-C MUTT.
  • Check in Device Manager. Has the Type-C MUTT enumerated?
    • Look for the "SuperMUTT" device with hardware ID USBVID_045E&PID_078F in the Device Manager.
  • Check in Device Manager. Does the "Device Status" of the UCSI device report any errors? If so:
    • Right-click on the UCSI device and disable it.
    • Start UCSI logging (see https://aka.ms/usbtrace )
    • Enable the UCSI device. Do whatever it takes to get the yellow bang to reproduce.
    • Stop logging.
    • View the resulting WPP logs and check for errors.

USB Type-C UCM Data and Power Role Swap Tests

Applicable Tests

• USB Type-C UCM Data Role Swap
• USB Type-C UCM Power Role Swap

Hardware Requirements

1. Onoe Windows system with a USB Type-C connector.
2. One Type-C MUTT device.

Test Setup

1. Install the HLK client on the System Under Test (SUT).
2. Connect the Type-C MUTT to any USB Type-C port on the SUT.
3. Ensure all other USB Type-C ports are disconnected.

Test Parameters

Troubleshooting

• "No USB Type-C Connectors found with partners attached".
  • Check your cable connection to the Type-C MUTT.
  • Check in Device Manager. Has the Type-C MUTT enumerated?
    • Look for the "SuperMUTT" device with hardware ID USBVID_045E&PID_078F in the Device Manager.
    • If your system is using UCSI, you can take UCSI logs during attach to investigate why the attach has not been reported to the OS.

UCSI Compliance tests

Applicable tests

This category of tests refers to all tests in the HLK with a name that begins with "UCSI" and is post-fixed with “[Type-C MUTT]”. The early RS5 pre-release HLK builds will include the original UCSI Compliance Tests as well, these tests have the same names, but without the “[Type-C MUTT]” suffix. Unless you are specifically running the old test for comparison, you should ignore the old versions of the tests.

UCSI Compliance Tests are meant to test the UCSI-capable Type-C system’s compliance to UCSI Specification.

Broad Categories of UCSI Compliance Tests

• UCSI Command Interface tests.
  • Tests all of the UCSI commands that are claimed to be supported by the SUT.
• USB Operation Mode tests.
  • Tests all of the USB Operation Modes that are claimed to be supported by the SUT on the given Connector.
• USB Operation Role tests.
  • Tests all of the USB Operation Roles and role swaps that are claimed to be supported by the SUT on the given Connector.
• Power Direction Role Tests.
  • Tests all of the Power Direction Roles that are claimed to be supported by the SUT on the given Connector. Performs role swaps.
• UCSI Notification tests.
  • Tests all of the UCSI notifications that are claimed to be supported by the System under Test on the given Connector.

Hardware Requirements

1. One UCSI compliant Windows system.
2. One USB Type-C MUTT device.

How to identify Connector 1

In the next release of the HLK, the tests can be run against any available connector on the SUT, but you will need to specify the connector that the Type-C MUTT is connected to via the UcsiConnectorNumber parameter. If you do not know the connector numbers on a multi-port system, the UCSI Connector One Identification [Type-C MUTT] test can help you identify which is connector 1. If the SUT has 3 or more connectors, and you don’t know their mapping, you will simply need to experiment by connecting the Type-C MUTT to an unknown port, setting UcsiConnectorNumber to a new value, and running a test to see if the device is found on that connector; repeat with a new connector number until successful. This should just take a few attempts to establish the connector number layout for any new system.

Test Setup

1. Install the HLK client on the System Under Test (SUT).
2. Connect the Type-C MUTT to any USB Type-C port on the SUT.
3. Record the connector number to which the Type-C MUTT is attached. You will supply this number when scheduling the tests.
4. Install TestUcsi.sys driver on the SUT. On a URS system, it is important that the Type-C MUTT is connected before the test driver is installed, otherwise the USB host stack may not get loaded.
   • In Device Manager, find the device "UCSI USB Connector Manager". This device will have the driver UcmUcsi.sys installed.
   • Right-click the device and select "Update Driver Software"
     • Enter the path to TestUcsi.inf
       • TestUcsi.inf and TestUcsi.sys are located in TestUcsi
     • Follow the prompts to complete the driver installation. This will replace UcmUcsi.sys with TestUcsi.sys.

Test Parameters

Test Execution

The Type-C MUTT allows the HLK tests to emulate and automate all scenarios in RS5 that previously may have required manual intervention. When scheduling the UCSI tests, only select the ones with the “[Type-C MUTT]” designation. You can select all of them at once, or only run a subset of them if you wish.

Test Cleanup

The UCSI Data and Power role swap tests require UcmUcsi.sys rather than TestUcsi.sys. If you plan to run the UCSI Data Role Swap or UCSI Power Role Swap tests in the HLK after this test, be sure to clean up your test environment after running the UCSI compliance tests. You can do this by replacing TestUcsi.sys with the original UcmUcsi.sys or by reinstalling Windows on both systems.

Troubleshooting

• Please review the HLK test log to determine why the test failed. In many cases, the test log will state common known causes of a given failure and how it may be resolved.
• Exame the driver logs per the guidance provided in the following blog post:
  • https://blogs.msdn.microsoft.com/usbcoreblog/2017/08/03/reading-driver-logs-in-usb-type-c-hlk-tests/
• If the test failure is unclear from both the test and driver logs or you believe it to be incorrect, and you are going to reach out to Microsoft for additional guidance, please prepare the following when reporting the issue to us. This will help us get to the bottom of your bug report as quickly as possible!
  • The HLKX package containing the failed test result. This will contain the failed test log as well as driver and Type-C MUTT firmware logs we may need to review.
  • An explanation of what diagnostic efforts you have already applied and why they were inconclusive:
    • Was the test log inconclusive or confusing? (We'd love to hear feedback in order to make them better!)
    • Do you believe the test result to be incorrect, and if so, why?
  • If you believe a test failure to be in error, do you have passing logs from the same test case in Windows 10 April 2018 Update HLK? If so, please provide them. Or do you have other evidence (e.g. driver traces, PD trace, etc.) that would demonstrate the PPM behaving properly in the scenario under test?