Testing with UAC – Overview



  Hi, I am Wei Wang, a Lead Software Design Engineer in Test on the User Account Control (UAC) team within the Windows Security organization.  As many of you experienced, User Account Control in Windows Vista fundamentally changes the way users interact with Windows, therefore, it has a great impact on how testing is performed on the Windows platform and the numerous supported applications.  The Windows test organization went through a great deal of work to augment its test infrastructure to be UAC compliant.  Many test teams spent a significant amount of time to redesign how their tests are structured in order to fully exercise their components in a UAC environment.


  There are three major buckets of change with respect to the differences between XP and Win2K3 to Vista.


 


1)      Moving away from testing as the built-in administrator account


2)      Test executable separation (test setup/clean-up vs. runtime test cases)


3)      Testing as user of different user groups (Users, Administrators, Network Operator, etc…)
 


  Historically the majority of testing was performed as the built-in administrator account because it was a lot easier to test when you are running with full administrator privilege.  Because most users run as a member of the local administrators group, it is effectively the same whether you test as the built-in administrator or a member of administrators group.  However, in Windows Vista with UAC enabled (as it is by default), this is no longer true since all users run with standard user rights by default even when being a member of the local administrators group.  Therefore, with this change, testing as the built-in administrator is not the same as testing as a member of the local administrators group.  This is a dramatic change from both the user and test perspective.  Instead, all tests should be executed under the default standard user account or in some cases as a user who is a member of local administrators group –user in Admin Approval Mode.


 


  Throughout our internal and external test evangelism effort, we (the UAC team) encountered many cases where setup and cleanup code, which often perform administrative tasks, are compiled into the same executable with the code that exercises all the end user test cases.  With the changes in UAC, a single test binary composed of setup/cleanup and runtime functionality no longer work the setup and cleanup code will by default fail to run due to the lack of administrative rights.  There are two workarounds to this issue.  The first one is to move the setup and cleanup code into a separate executable and to mark this executable with a manifest defining a run level equal to “requiredAdministrator”.  Please refer to the UAC developer whitepaper on how to mark applications with the applicable run level.  This solution will closely match the end user experience.  However, if for some reason, separating your test into separate binaries is not an acceptable solution; the second workaround is to mark the test executable as “requiredAdministrator” and run the setup and cleanup code with full administrator rights and to impersonate down to a standard user access token during the rest of the test case execution.  The problem with this testing method is that your impersonating token will not be identical to the filtered token on the system; therefore you can potentially hit issues when running as standard user without impersonation.  Additionally the impersonated token will not have the virtualized bit set, and thus registry and file virtualizations will not occur.


 


 


  Another major test change is really not caused by UAC, but manifests with UAC.  Some test teams continue to test as administrator as they did in the past.  The standard user account is either not tested, or only tested in a few test scenarios.  Historically this was understandable as most customers ran as administrators on XP and Win2K3.  However, this is no longer the case as all users, irrespective of privilege, will logon as a standard user.  It is essential to verify your APIs, components, and applications interact correctly and reasonably as standard user.  The basic rule is to review your scenarios, define the standard user and administrative behavior both positive and negative, then exercise each scenario under both user accounts and verify the outcome.  Following this simple methodology will give you complete functional test coverage.  Additionally you can reference the UAC Shell Guidance <coming in Beta2> for our recommended best practices for UI components to verify if your user experiences are consistent with our designs.


 


  User Account Control fundamentally changes how testing is performed.  We understand that this is a significant shift in how we’ve tested in the past, but UAC is an important effort to reduce the default attack surface of the Windows operating system.  We hope you will see the benefit of UAC and therefore invest in changing your test infrastructure to be UAC compliant.  This is only an overview of UAC’s impact on testing and we will follow up with a series of more in-depth articles on tools, test design, implementation tips, and additional guidelines to facilitate testing with UAC.


Comments (11)

  1. Wei,

    Nice article.  

    Small typo – I believe "requiredAdministrator" should read "requireAdministrator".

    Marcus

  2. yannan says:

    兄弟是叫王伟(音译)吧?看名字好像是华人.

    i think you r chinese:-)

  3. Peter says:

    The experience at my company shows that testing under Vista is much harder than testing anywhere else.  

    One thing that’s killing us now is the ‘Program Compatibility Assistant’ (I left a question on the last-but-one entry in this blog; so far there have been no answers).  In particular, whenever it runs we can only assume (because it tells us in plain English) that it’s done "something" to the system; this means that the box is now useless for QA.

    Which means reinstalling the OS!

    Are there any hints for learning what the PCA is doing so we can undo it?

    Thanks

    Peter

  4. Alex says:

    Hi Peter. Thanks for your question, I will track down an expert on the Program Compatibility Assistant to reply as soon as possible.

    – Alex

  5. Sathish says:

    Hi Peter, please see the response posted about the Program Compatibility Assistant on the other UAC thread at http://blogs.msdn.com/uac/archive/2006/02/22/537129.aspx

    thanks,

    Sathish (Microsot)

  6. Alexandra says:

    What about standard users not being able to write to files created by admins, even if the location is defined as common machine data, for all users, like the ProgramData folder ?

    Is there any way around that ?

    What is the reasoning behind, this location is recommended in various articles as the place to store all users data ???

    Thanks!

  7. rsclient says:

    Is there any reason that ‘Alexandra’s comment has not been answered?  I’ve noticed that polite questions on this blog tend not to get no response — shall I reword the question to be rude, just so we get an answer?

  8. rsclient says:

    What — are the questions here too hard to answer?  Perhaps you have a future flipping burgers — that might be easy enough for you.

  9. Aaron Margosis says:

    Alexandra – actually, this has been the case for a long time.  In Windows XP, when a user (any user) creates a file in the shared locations, that user becomes the owner of the file and retains full control.  Other non-admin users have read-only access to those files.  Admins, of course, can do whatever they want on the machine.

  10. Lanette C says:

    Thank you for a blog about testing in Vista. I am looking for some specific information about what testing impacts the "filtered token" that is generated when permissions are elevated to complete a task might have.

    We found on our initial test pass on Vista that even running "as Administrator" as part of the administrators group, some actions fail which work correctly if you are logged in as the administrator. We aren’t really sure why or what to expect in this situation, and more information would be helpful. Why is it different if you temporarily give access to run as the Admin?

    So far, the information that I am finding is either pretty high level, so great for a user, or intended for developers.  I understand that industry wide the movement is towards everyone coding, but having basic blackbox testing information is still really helpful, especially in the early stages of testing. I hope you can point me to some more information on testing with User Account Control, and thank you for including blogs about testing.

    It would be fantastic to see a blog about what happens to XP user accounts during an upgrade to Vista.  If you were a power user, what are you now? A user or a part of the administrators group? Also, what can the Guest do and is it different than XP?  If this information exists, could you please reply with a link? A table with what permissions each account type has, and how it related to account types in XP would be wonderful. Does such a table exist already?

    Also, is there a better place to ask Vista testing questions? I seem to keep coming up with more questions than answers.

  11. robert smith says:

    Bottom line:

    How do we shut this "Feature" Off.  How do we tell our users how to shut it off?  

    I mean OFF completely?  THen we’ll selectively tweak it back up to where it’s actuallly helpful.

    Point: We can’t get SQLServer 2005 SP1 to install because it says that the account isn’t in the Administrators group.  The Account IS in the Administrators group.. and we’ve forcibly upped the USERS grout to FULL Permissions across the board… AND we’ve disabled the secpol.msc ‘Run all applications in Admin Approval Mode’ and yet still all of the UAC popus every time we try anything and still MSIs tell us that we aren’t in Admin mode and just bail.

    Better start training your test teams to use the CSR tools because that’s where the big need is going to be on campus if this continues to RTM.

    Honestly, stop being testters and start thinking of your mother who’s sitting at her box wondering if Mac really is easier after all.

    Currently, UAC is not helpful to anyone, thought B2 would tone it down, made it worse.

    Just my (very) frustrated two cents.

    smith

    kirkland