Changes to “System – Sleep and PNP (disable and enable) with IO Before and After (Certification)” test in February HCK QFE


In the February 2014 HCK for Windows 8.1 QFE, these changes are made to the System – Sleep and PNP (disable and enable) with IO Before and After test:

– We also released filter 4706 to remove the need to plug in a MUTT device.

  • The test run time is reduced by a factor of four.

These changes address the feedback we received about the test after we shipped Windows 8.1 HCK.

Existing filters & contingencies

As a result of reducing the test difficulty, all existing blanket errata filters and contingencies granted for this test will expire on March 31, 2014. Specifically, these filters (along with any blanket contingencies that were given) will expire on March 31, 2014:

  • 4663  System – Sleep and PNP (disable and enable) with IO Before and After (Certification) to be waived while investigation is underway.
  • 4664  System – Sleep and PNP (disable and enable) with IO Before and After (Certification) to be waived while investigation is underway – waiver for when logs are not generated.

Summary of changes

Driver verifier settings changes

Because of the driver verifier settings changes that we’ve made, you’ll notice a reduction in stress on the system when the test runs. This has been the case since the release of QFE 003.

Device configuration requirement change

We encourage you to plug in an optional MUTT/SuperMUTT device when running the test. We have released filter 4706 to filter out errors that occur because there is no MUTT device plugged in.

Recommendations

  • We recommend that you run the test with QFE 006 changes and report any test or operating system bugs to us.
  • The 0xC4 bug ensures that the test reports are legitimate WLAN driver bugs that must be fixed. Follow up with your IHV on any 0xC4 bug checks found. This can be done before or after certification. The IHV will need a copy of the .dmp file generated. If you need help with the investigation of these bug checks, you can contact driverqualitytools@microsoft.com
  • Running the test with the MUTT/SuperMUTT device helps you find bugs in the USB controller and/or other devices enumerated on the USB bus (like webcams). So we strongly recommend that you run the test with the MUTT device connected. Simply connect the MUTT before starting the test and follow up on any issues that you find.
Comments (10)

  1. Mike Cao says:

    Referring to your words "We recommend that you run the test with QFE 006 changes and report any test or operating system bugs to us " ,How Can I contact with Microsoft without CSS support ?

    Thanks,

    Mike

  2. Mark Hughes says:

    Hi Mike,

    Some Sysdev (Dashboard) accounts are set up with bug management services that allows bugs to be filed directly through the Sysdev site.   If your Sysdev account is not set up with bug management the only way to report a HCK issues is through the CSS team.

    BTW….Your account is set up with the bug management feature.    

    Thanks,

    Mark

  3. Mike Lu says:

    Are Fidelity Test and Firmware Post Time test supposed to be moved to the optional test?

    Why do I still find them as certification level in this QFE update?

  4. Summer Nie says:

    Hi, Mark:

    Is Mutt device supposed to be plugged in during "device status check"?

    As we are running QFE006, and it still failed on Super MUTT during device status check phase, so the main test will not process at all.

    Per the description, does it mean we could unplug the Super Mutt device from system once device status check was passing?  

    Thank you!

  5. paul_Reed55 says:

    Hi Mike Lu,

    When the requirement relaxations were finalized, we were approaching the holiday season in December and many teams were leaving for extended holiday.  Then when they returned there was a really tight deadline to make the February HCK QFE release.  Not all teams were able to do so.  However, we have published errata filters that will turn all failures into pass.  So if you do not wish to test for the quality bar, you can run the test and it will fail.  The errata will convert it to a pass.  

  6. KAENNGAI CHAN says:

    with IO Before and After test: You no longer need to configure all devices on the system before the test can run. – We also released…

    i like how Microsoft use the words and i know i will fail on test without test, let people have more talents have more chance for test

  7. HC Huang says:

    Hi Sir,

    I filed a bug on dashboard about this test.

    Could you help to take a look at the failure's hckx file?

    Bug ID is 605896.

    Thank you in advance.

    HC

  8. swiPetar says:

    I have the same problem as Summer Nie (Sun, Feb 16 2014 11:11 PM).

    Without the SuperMutt due to the Filter (4706), the test "Passes" (green check within a green circle)  DeviceStatusCheck, but then skips directly to RestoreMemoryTrimming and finishes the test with a fail (red X in a red circle).

    My understanding of the updated test QFE0006/Filter4706 was that the test should still run, it should just ignore the missing SuperMutt. Am I wrong?

  9. paul_Reed55 says:

    Hi swiPetar,

    Microsoft CSS would need to see the log file to see if there are any other points of failure in the log file which might be causing the test to still fail.  

  10. swiPetar says:

    Actually I figure this out late on Friday. The test case has a default behavior which is to "FailAndStop" if there are any errors in DeviceStatusCheck.

    Since I don't have a SuperMutt installed, it Fails and Stops. The default behavior should have been updated in the QFE that made the mutt a non-requirement.

    I manually changed the behavior to FailAndContinue (so the test continues, and the Failure is covered by Filter4706) by going into HCK Manager and editing the Tasks for Test Case 3005 and changing the "Failure Action" from "FailAndStop" to "FailAndContinue".