Setting up an environment to run USB Type-C system HLK tests


Authored by Michelle Bergeron [MSFT] and Makarand Sonare [MSFT]

[Last updated 4/24/2017]

There are new tests in the Hardware Lab Kit (HLK) that target systems with USB Type-C. These tests are available for the Windows Anniversary Update version of HLK and Windows 10. Some of the tests require extra configuration – here is a guide to help you set up your test environment for them.

USB Type-C UCSI Data and Power Role Swap tests

Applicable tests

Hardware Requirements

  1. Two UCSI 1.0 compliant Windows desktop systems.
  2. UcmUcsi.sys must be loaded as the UCSI controller driver.
  3. USB Type-C cable.

Note: If your Type-C system does not use UCSI, the test will detect this and allow your system to pass.

Test Setup

  1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.
  2. Install HLK client on the SUT.
  3. Connect the UCSI Type-C connector on the SUT to the UCSI Type-C connector on the Partner via the Type-C cable.
  4. Enable the UCSI test interface on the SUT by following steps 1-7 in the “How to Test UCSI” section of USB Type-C Connector System Software Interface (UCSI) driver.

Tracing Instructions

Test failures for the UCSI data and power role swap tests can be investigated by collecting logs as per the below instructions.

  1. Start WPP tracing before executing tests. Run the following from an elevated command prompt:
    logman start -ets ucmucsi -ct perf -p {EAD1EE75-4BFE-4E28-8AFA-E94B0A1BAF37} 0xffffffff 0xff -o c:\ucmucsi\ucmucsilogs.etl
    
  2. Stop WPP tracing after running the tests. Add the following commands to a .cmd file and run from an elevated command prompt once the test completes.
     logman stop ucmucsi -ets 
    
  3. You will be able to view the resulting logs in UcmUcsiLogs.etl.

Test Parameters

Parameter Name Parameter Description
SwapsToPerform Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested.
ValidateUsbFn If ValidateUsbFn = true, the test will validate function stack behavior.

Troubleshooting

  • Some UCSI commands are marked "optional" in the UCSI specification. However, Microsoft requires some optional commands to be implemented in order for the UCSI driver to function. The tests may fail if you've forgotten to implement one of these commands in your BIOS. You can find that list of commands here.
  • If the test is not detecting an attached partner
    • Check your cable connection to the attached partner
    • Check in Device Manager. Has the attached partner actually enumerated? If not...
      • Detach the partner
      • Start UCSI logging (instructions above)
      • Attach the partner.
      • Stop logging.
      • View the resulting WPP logs and check for errors.
    • Check in Device Manager. Is the UCSI device a yellow bang? If it is...
      • Right click on the UCSI device and disable it.
      • Start UCSI logging (instructions above)
      • Enable the UCSI device. Do whatever it takes to get the yellow bang to reproduce.
      • Stop logging.
      • View the resulting WPP logs and check for errors.

USB Type-C UCM Data and Power Role Swap tests

Applicable tests

  • USB Type-C UCM Data Role Swap
  • USB Type-C UCM Power Role Swap

Hardware Requirements

  1. Two Windows systems, each with a USB Type-C connector.
  2. The Type-C connector on the SUT must have a UcmCx client driver. For information about developing a UcmCx client driver, visit USB Type-C connector driver programming reference.
  3. USB Type-C cable.

Note: If your Type-C system implements UCSI, this test is still applicable because UcmUcsi.sys is a UcmCx client driver.

Test Setup

  1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.
  2. Install HLK client on the SUT.
  3. Connect the Type-C connector on the SUT to the Type-C connector on the Partner via the Type-C cable.

Test Parameters

Parameter Name Parameter Description
SwapsToPerform Number of data role swaps to perform. The minimum is 2 so that both host and function mode are tested.
ValidateUsbFn If ValidateUsbFn = true, the test will validate function stack behavior.

Troubleshooting

  • "No USB Type-C Connectors found with partners attached".
    • Check your cable connection to the partner.
    • Check in Device Manager. Has the partner device enumerated?
      • If your system is using UCSI, you can take UCSI logs during attach to investigate why the attach has not been reported to the OS. UCSI logging instructions are found above.

UCSI Compliance tests

Applicable tests

This category of tests refers to all tests in the HLK with a name that begins with "UCSI". UCSI Compliance Tests are meant to test the UCSI-capable Type C system’s compliance to UCSI Specification V1.0. These tests are marked as "manual" in the HLK. The test binaries are included in the HLK; however, you will need to run them yourself using the instructions below.

Broad Categories of UCSI Compliance Tests
  • UCSI Command Interface tests.
    • Tests all of the UCSI commands that are claimed to be supported by the SUT.
  • USB Operation Mode tests.
    • Tests all of the USB Operation Modes that are claimed to be supported by the SUT on the given Connector.
  • USB Operation Role tests.
    • Tests all of the USB Operation Roles and role swaps that are claimed to be supported by the SUT on the given Connector.
  • Power Direction Mode tests.
    • Tests all of the Power Direction Modes that are claimed to be supported by the SUT on the given Connector.
  • Power Direction Role tests.
    • Tests all of the Power Direction Roles that are claimed to be supported by the SUT on the given Connector. Performs role swaps.
  • UCSI Notification tests.
    • Tests all of the UCSI notifications that are claimed to be supported by the System Under Test on the given Connector.

Hardware Requirements

  1. Two UCSI 1.0 compliant Windows desktop systems.
  2. Connection Exerciser hardware (optional - unless you are running the test from HLK Studio. If you are running the test in HLK Studio, the Connection Exerciser is required.)
  3. USB Type-C PD capable cable.
  4. USB 2.0 and USB 3.0 device
  5. Debug accessory (if supported)
  6. Analog audio accessory (if supported)

How to Identify Connector 1

All of the UCSI compliance tests assume Partner system or Devices are connected on Connector 1. To identify Connector 1, perform the following steps:

  1. On the SUT, open an elevated command prompt.
  2. From the elevated command prompt on the SUT, run the following test to identify Connector 1:
    Te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::TestIdentifyConnectorOne /p:WaitTimeInMinutes=5

    You must specify the WaitTimeInMinutes parameter in order for the test to pass.

  3. Repeat Step 2 on the Partner system to identify Connector 1.

Test Setup

  1. Designate one system as the System Under Test (SUT) and the other as the "Partner" system.
  2. Install HLK client on both systems. You will only schedule tests on the SUT.
  3. Connect the Type-B port on the Connection Exerciser Arduino to the SUT on any port other than Connector 1
  4. Connect Connector 1 of the SUT to J1 connector on the Connection Exerciser directly.
  5. Connect Connector 1 of the Partner to J2 or J3 connector on Connection Exerciser via Type-C cable.
  6. Install TAEF on both systems
    1. For information about installing TAEF, refer to the TAEF Getting Started page.
    2. Set up Te.Service on the Partner system. Instructions to do that are here.
  7. Install the TestUcsi.sys driver on both systems.
    1. In Device Manager, find the device "UCSI USB Connector Manager". This device will have the driver UcmUcsi.sys installed.
    2. Right click the device and select "Update Driver Software"
      1. Enter the path to TestUcsi.inf
        1. TestUcsi.inf and TestUcsi.sys are located in \TestUcsi\<ARCHITECTURE>\
      2. Follow the prompts to complete the driver installation. This will replace UcmUcsi.sys with TestUcsi.sys.

Test Parameters

Parameter Name Parameter Description
IsRunningOnSystemUnderTest Identifies whether the test is running on the System Under Test or Partner machine.

Values:

true - If test is executed on SUT.

false - If test is executed on Partner.

PartnerMachineName Machine name of the partner machine that is connected to the SUT using a USB C-to-C cable. The test performs TAEF network remote execution to execute code on this partner machine.
WaitTimeInMinutes For tests that require manual intervention, this is the amount of time in minutes the test will wait to see the event that is expected to occur in response to the requested manual action.

Test Execution with Connection Exerciser

  1. Connect the SUT and Partner system via a Connection Exerciser and Type-C Cable on Connector 1.
  2. Refer to the Test Setup section for more information.
  3. On the SUT, run UcsiCommandInterfaceTests.
    te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::*
  4. On the SUT, run UcsiTests.
    te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTest::* /p: IsRunningOnSystemUnderTest=true /p:waittimeinminutes=1 /p:partnerMachine=<MACHINE_NAME>

Test Execution without Connection Exerciser

  1. Connect the SUT and Partner system via a Type-C Cable on Connector 1.
  2. On the SUT, run UcsiCommandInterfaceTests.
    te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiCommandInterfaceTests::*
  3. On the SUT, run UcsiTestsManual.
    te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTestsManual::* /p:IsSUT=true /p:waittimeinminutes=1 /p:partnerMachine=<MACHINE_NAME>
  4. Each test requires executing some TAEF tests on the Partner system as per the instructions given on the command prompt.

Example: When running UcsiTestsManual::TestDFPModeToUFP on the System Under Test:

CMD> te.exe UcsiComplianceTest.dll /name:TestUcsi::UcsiTestsManual::TestDFPModeToUFP /p:IsSUT=true /p:waittimeinminutes=1 /p:partnermachine=<MACHINE_NAME>
 Test Authoring and Execution Framework v5.6 for x64 
 Resetting the PPM 
 Enabling All Notifications 
 
 StartGroup: TestUcsi::UcsiTestsManual::TestDFPModeToUFP 
 Setting USB Operation Mode to Dfp on Connector 1 
1. On Partner Machine  
Run: CMD Prompt>Te.exe UcsicomplianceTest.dll /p:IsSUT=false /p:waitTimeInMinutes=<timeInMinutes> /name:TestUcsi::UcsiTestsManual::TestUFPModeOnPartner
 2. Connect the Partner 
 3. Waiting for 1 minutes 
  
           Waiting for:                   ConnectChange 

The above test sets Connector 1's USB Operation Mode to DFP on the SUT. It then requires you to run the following command on the Partner system:

Te.exe UcsicomplianceTest.dll /p:IsSUT=false /p:waitTimeInMinutes=<timeInMinutes> /name:TestUcsi::UcsiTestsManual::TestUFPModeOnPartner

Tracing Instructions

Test failures for the UCSI compliance tests can be investigated by collecting logs as per the below instructions.

  1. Start WPP tracing before executing tests. Add the following commands to a .cmd file and run from an elevated command prompt:
    logman start -ets testucsi -ct perf -p {C9ED7F4E-286B-4054-B74F-7DE43EFBCA2A} 0xffffffff 0xff -o c:\testucsi\testucsi.etl 
    logman start -ets testucsicontroller -ct perf -p {EBFD4AED-ED11-4F0C-8F37-9C93084D1530} 0xffffffff 0xff -o c:\testucsi\TestUcsiController.etl 
    logman start -ets ucsitest -ct perf -p {F8BADD54-3B0F-4B79-8389-D0ABB7AE242B} 0xffffffff 0xff -o c:\testucsi\ucsitest.etl
  2. Stop WPP tracing after running the tests. Add the following commands to a .cmd file and run from an elevated command prompt once the test completes.
     logman stop testucsi -ets 
     logman stop ucsitest -ets 
     logman stop testucsicontroller -ets
  3. You will be able to view the resulting logs in UcsiTest.etl, TestUcsi.etl, and TestUcsiController.etl.

Test Cleanup

The UCSI Data and Power role swap tests require UcmUcsi.sys rather than TestUcsi.sys. If you plan to run the UCSI Data Role Swap or UCSI Power Role Swap tests in the HLK after this test, be sure to clean up your test environment after running the UCSI compliance tests. You can do this by replacing TestUcsi.sys with the original UcmUcsi.sys or by reinstalling Windows on both systems.

Troubleshooting

  • "Unable to connect to TE.Service"
    • Verify that you've used the above steps to configure TAEF remote execution
    • Verify that the machines can ping each other
    • Instead of using the machine name, try using its IP address. You can get an IP address of a machine using the following command in Command Prompt:
      ping -4 <machine name>
  • No USB error notifications while using the UCSI test driver
    • This is by design - the UCSI test driver exists only for the purposes of testing your PPM and does not interact with the OS in the same way as the inbox UcmUcsi.sys. To verify user notifications, please reinstall UcmUcsi.sys.
  • One of my role swap tests failed because "UcsiConnectorPartnerTypePoweredCableWithUfp".
    • We're on it! There will be an errata for RS2 (we don't have a date for the errata release yet - sorry!) and we've already fixed it for RS3. In  the meantime, you may try to work around it by connecting a different partner and/or using a different USB cable.

Selecting the tests in HLK

The following steps illustrate where to find the tests in the HLK.

  1. Add the SUT as your test target
  2. Right click on the machine, click "Add/Modify features". Verify that System.Fundamentals.SystemUSB.USBC is selected (and System.Fundamentals.SystemUSB.USBTypeCUCSI if your system is UCSI).
  3. Select the "Tests" tab. Scroll down and see that the tests are available. In the example, the SUT uses UCSI.
Comments (13)

  1. Jegannathan M Ramanujam says:

    Hello,

    I've the latest WDK installed in my PC (10.1.10586.0). TAEF version is v5.3 and I'm unable locate UcsiComplianceTest.dll. How to obtain Type-C software test environment?

    Jegan

    1. Hi Jegan,

      You will need to install the Hardware Lab Kit onto a separate machine. The HLK (as opposed to the WDK) contains the test binaries.

      You can install the HLK here:
      https://go.microsoft.com/fwlink/p/?LinkId=733613

      Michelle Bergeron [MSFT]

  2. Jegannathan M Ramanujam says:

    Hello Michelle,

    I do have HLK server and a windows 10 x64 client PC. And, my HLK server isn't auto detecting the UCM test for my UCMCx Client driver installed in my client PC. I'm using latest and greatest version of HLK suite available in the MS web site. And, I was pointing to WDK for manually running TAEF and from there I can run the data role and power role swap test procedures on my UCMCx client driver.

    Questions:
    Is this article based on pre-release for upcoming tool set? or Is the tool set already available? If available, how to get to the tools to run the above manual procedures? I'm not successful to run this test yet.

    Cheers,
    Jegan

    1. USB Blog says:

      Hi Jegan,

      This is available in builds for the Windows Anniversary Update (also known as Redstone 1 or RS1). Your HLK build and your SUT must both be using Anniversary Update editions. This is good information for us to specify in this post and we'll update it accordingly.

      As for the server not auto-detecting the test on the SUT, check that UcmCx.sys is loaded on the SUT.

      Michelle Bergeron [MSFT]

      1. Jegannathan M Ramanujam says:

        Michelle,

        Now, I've the HLK Anniversary edition suite and my SUT & partner PC are updated with the windows 10 anniversary release (version 1607).

        We aren't using UcmUcsi.sys but we have developed a vendor UCMCx client driver. In this context, I've following questions.

        1.) Above test UCSI compliance test relies on TestUcsi.sys (replacing UcmUcsi.sys) and running the procedure that you described here. How to run these role swap test for Vendor developed UCMCx client driver?
        2.) And, My HLK controller project isn't automatically recognizing the Vendor developed UCMCx client driver as Type-C connector driver . Is this known behavior?

        Cheers,
        Jegan

  3. david says:

    I have confuse about the UCSI hlk testing now, it said MUTT Connection Exerciser type-c hardware is an optional for testing, but now when I see the help file in UCSI test in RS2 HLK, it always said this hardware should include in setup steps.
    so please help confirm whether MUTT Connection Exerciser type-c hardware is a must for UCSI HLK test.

  4. david says:

    One more question:
    UcsiComplianceTest.dll, is it also located in the \TestUcsi\\usb\UcsiCompliance\ in HLK server, with the UCSI test driver (testucsi.sys )together?
    or we have to got it from other place, such as connect have the ucsi HLK UCSI binaries...

    1. Prabhakarv says:

      Hi all,

      I have followed this setup (https://msdn.microsoft.com/en-us/library/windows/hardware/mt644884(v=vs.85).aspx) for HLK test for UCSI test in which i have used ucmusci.sys to be load testucsi.sys , we are modifiying the BIOS implementation of UCSI for a device in where ucmucsi.sys will be load for that device i.e inbox driver has been loaded for that.

      Test environment :
      1) one UCSI_1 capable system where installed HLK client and has been taken in HLK server machine
      2) UCSI_1 has been act as a SUT and connected to MUTT of J1 connection
      3) J2 and J3 where connecting the type c cable to other UCSI_2 capable system also connected to same netwrok domin and installed HLK client
      4) when i installed the HLK client on UCSI_2 system where the HLK server machine not taking the UCSI_1 system and disconnecting it
      5) how i can connect the partner system via the type c cable for UCSI_2 system
      6) In each UCSI test cases it where mentioned the MUTT device so could you please anyone suggest me how i can make use of HLK test setup for my chip.
      7) In the above link they were talking about the proxy controller ,could you please anyone suggest me in real environment and give me step it will be useful.

      Regards,
      Prabhakar V

      1. Prabhakarv says:

        Hi all,

        Anyone suggest me the setup how the other UCSI_2 Capable system as a partner to the HLK server.

        I have followed the setup in which MUTT device is connected as one UCSI_1 capbale system as a SUT connected to MUTT of J1 and J@ &J3 is connected to UCSI_2 capable system through USB type C cable.
        Loaded the testucsi.sys driver to both UCSI capable system which has inbox driver "UCSI USB Connection Manager" .

        Run the UCSI test in HLK server where the SUT of UCSI_1 capable system(Installed HLK client ) has been detected and i were struck in how i can connect the partner system of UCSI_2 to the HLK server machine.

        Could you please anyone suggest me how i can proceed ?

        Please help on this.

        Regards,
        Prabhakar V

  5. Luisdua says:

    I'm a little confused now seeing all of this. Is there any basic tutorial or an introductory thing through which I can better understand it? Thanks in advance!

    1. USB Blog says:

      If you are looking for info on HLK itself, Microsoft has some info here: https://msdn.microsoft.com/en-us/library/windows/hardware/dn915002(v=vs.85).aspx

      If you have any specific feedback about what's confusing about these tests in particular, we'd love to hear it so we can make it clearer for everyone.

      Michelle Bergeron [MSFT]

  6. CCChen says:

    Hello,
    We've a type-c device with developing fw for testing.
    It runs fine through Win10 RS2 preview (15025).
    But when we update to Win 10 RS2 1703,15063, the device shows not connected.
    Is there any policy changed when OS initializing type-c device?

    1. USB Blog says:

      Hi CCChen,
      There have not been any changes to our UCSI driver in this timeframe. It sounds like this might be a real issue with the UCSI firmware. We detect attach in the UCSI driver when the firmware sends a Connect Status change notification. Only after we receive this notification do we report the attach to the lower USB stack to being enumeration, etc.
      You can take the UCSI logs as detailed in this post and view them with TraceView. Make sure that the logs are detecting a Connect Status change and there aren't any errors.

Skip to main content