Windows Vista Shim Internals Basics: How Shims Work to Address Compatibility Issues (and What are the Ramifications?)


I want to talk a little bit today about shims, specifically addressing how they work to address compatibility issues, and what the security ramifications are when you use a shim to address a compatibility issue.

Merriam Webster defines a shim as:

“a thin often tapered piece of material (as wood, metal, or stone) used to fill in space between things (as for support, leveling, or adjustment of fit)”

(Wow, I feel like an 8th grader beginning a term paper, using a definition like that!)

But the name shim comes exactly from this definition – we jam a thin piece of code between things – specifically, between the application’s code and Windows code.

This works specifically because we implement an Import Address Table (IAT) to link to DLLs – specifically, to Windows DLLs.

Rather than talk about this, I figured I would just show this to you in a debugger. I began with the following source code:

#include <windows.h>

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                   PSTR szCmdLine, int iCmdShow)
{
    MessageBox(NULL, TEXT(“Hello world.”), TEXT(“Hello”), MB_OK);
    return 0;
}

Nothing too fancy. In fact, it’s pretty much identical to what you’ll find at the beginning of Charles Petzold’s Programming Windows book. Let’s see how this looks in assembler:

0:000> uf ShimsAndIat!WinMain
ShimsAndIat!WinMain [c:…shimsandiatshim.c @ 5]:
    5 00401000 6a00                push    0
    6 00401002 68f4204000      push    offset ShimsAndIat!`string’ (004020f4)
    6 00401007 68fc204000      push    offset ShimsAndIat!`string’ (004020fc)
    6 0040100c 6a00                push    0
    6 0040100e ff15ac204000   call    dword ptr [ShimsAndIat!_imp__MessageBoxA (004020ac)]
    7 00401014 33c0                xor     eax,eax
    8 00401016 c21000             ret     10h

When we call into Windows to invoke the MessageBoxA function, we don’t jump there directly. We jump to a local symbol: _imp__MessageBoxA. If we then take a look at what we find at this address, we’ll find our pointer to the function of interest:

0:000> dds 4020ac
004020ac  76ff56df USER32!MessageBoxA

Because we are jumping to the location pointed to by this pointer, we can see how easy it is to inject some new code: we simply need to replace the contents of the pointer (at 0x004020ac) with a new value – one that points to our shim code! The application we are trying to fix thinks it is calling in to Windows, but the shim code runs instead, and this shim code may call into Windows.

Let’s take a look at a shimmed up application.

In this application, we are making a call to GetVersionEx. The application has a bug – it is checking for equality of the version of Windows to 5.1 (Windows XP). Here is the portion of the unassembled function doing this comparison:

0:000> uf DWM_Compositing_Rendering_Demo!WinMain
DWM_Compositing_Rendering_Demo!WinMain:

… (disassembly elided for clarity)

   10 0040103d ff1528204000    call    dword ptr [DWM_Compositing_Rendering_Demo!_imp__GetVersionExW (00402028)]
   11 00401043 837c240805       cmp     dword ptr [esp+8],5
   11 00401048 753e                 jne     DWM_Compositing_Rendering_Demo!WinMain+0x88 (00401088) 
   11 0040104a 837c240c01      cmp     dword ptr [esp+0Ch],1
   11 0040104f 7537                 jne     DWM_Compositing_Rendering_Demo!WinMain+0x88 (00401088)

… (disassembly elided for clarity)

But this application is shimmed. If we look at what _imp_GetVersionExW is pointing to, it’s not GetVersionEx. It’s:

0:000> dds 00402028
00402028  6cd54c66 AcLayers!NS_WinXPSP2VersionLie::APIHook_GetVersionExW

Yep – it’s been redirected to a version lie, which causes this check to succeed. (If only every developer’s keyboard included a > symbol…)

The shim itself doesn’t need to call into Windows, but most do. They have to in many cases to do anything interesting. Because, as you can see, from the perspective of Windows, shim code is no different than application code. It’s just arbitrary code that happens to be calling in to Windows to do some work for it. It can’t circumvent any protections, it can’t do anything less securely – shim code can only do what your application’s code could do. No more, no less.

That’s why there are no shims for “don’t give me a UAC prompt for this application” – because your application can’t do it either.

It also means that anything we can fix using a shim you could fix using code. After all, we did it with some code sitting between your application and Windows, you could just write similar code directly in your application and not have to use a shim at all.

Comments (47)

  1. Kalyan says:

    Its interesting and informative. just looking into what shim is…

  2. In an earlier post , I began having a conversation around how shims work. I wanted to supplement some

  3. Ben Cooke says:

    Has it been your experience that applications for which shims are provided tend not to be updated by the vendor?

    Presumably the compatibility team does report these issues to the original vendor if they are still around and contactable, but I wonder how many vendors simply think "Well, they’ve fixed it for us so we don’t need to do anything!" thus requiring these shims to stay present for longer than would otherwise be necessary.

  4. cjacks says:

    When we shim something, we shim it either for a specific version or up to and including a specific version. So, a vendor that has a compat issue isn’t going to get the shim on the next version unless they don’t update the version number.

    Existing copies of older versions don’t go away, so we want to keep shimming those. New versions won’t get the shim any longer, so this works out pretty well. Of course, we can’t force anybody to release a new version of something unless they have new features and value they want to add to justify the cost of developing a new version.

  5. Jonathan Wilson says:

    Is there any kind of way to find out if a specific app has any shims applied to it (either on XP or on Vista)?

    And if so, what those shims are doing?

  6. cjacks says:

    Using Compatibility Administrator from the Application Compatibility Toolkit, you can view all of the applications we shim in the system shim database. Just expand the Applications node in the tree on the left.

    You can also use the search functionality, point it at the directory containing the binary you are curious about, and it will determine which of the .exes will pick up which shims.

    As for what they shims are doing – it depends on the shim. In general, they’re keeping the application working…

  7. I was reading the ‘Basics on how shims work’, and all I could think about was the old linux/unix trick of using LD_PRELOAD to intercept library calls in dynamically linked applications (used to great effect to bypass timebombed applications). It…

  8. Sanjay says:

    How do we patch managed code like C#? Many of the .net framework BCL calls ultimately uses Win32 calls, if we need to patch such calls, can we apply shim to .net framework DLLs? Those are strongly named /signed.

  9. cjacks says:

    Yes, you can shim managed code – we grab it at the Win32 API level. But you apply it to exes, not to DLLs. There is no requirement for them to be strong named or signed.

  10. I’ve been talking a lot about shims lately here, but it’s been rather lopsidedly technical. Using shims

  11. I&#39;ve been talking a lot about shims lately here, but it&#39;s been rather lopsidedly technical. Using

  12. ananda84 says:

    Hi,

    Is it possible to check what type of shim is being applied programatically?

  13. cjacks says:

    ananda84,

    I’m not sure I understand your intent. If you want to see which shims will be applied, Compatibility Administrator’s query tool will help you. Point it at a directory, and we’ll tell you what’s being shimmed.

    If you want to know what shims are applied to the program you are actually writing, my advice would be not to do that. We’re shimming you up for a reason – if you try to foil us, we have to be sneakier. (See: ShimViaEAT.) But, for copy protection and cheater-protection (games), you can monitor the Import Address Table for changes. We see that quite a bit, typically in programs that are also actively avoiding debuggers (because our job is apparently too easy otherwise).

  14. Ganesh says:

    Hello again Chris,

    In very simple words,I am trying to code an app which will scan/query a list of applications and check if a "version lie" has been set.

    Thanks

    -Ganesh.

  15. cjacks says:

    Hi Ganesh,

    You don’t have to write that application. Compatibility Administrator already does that. Use the Query tool.

    Chris

  16. danielye says:

    I have a regular application…

  17. To each and every one of you who attended my TechEd session – thanks! There are so many interesting talks

  18. I had a number of great conversations at TechEd 2008 IT Professionals down in Orlando. I mentioned in

  19. Ashwini says:

    My query is regarding what should be done after I create a .sdb file using XPVersionlie shim.

    I am using XPVersionlie shim on LH08. I have also created a .sdb file using compatbility Administrator and stored it on my desktop.

    But now if I just run my application, the bug reappears, I know I am missing one step after creating the .sdb file and before running the exe, but not getting what step I am missing.

  20. cjacks says:

    Hi Ashwini,

    In CompatAdmin, you can right click on the database and select install. For wider distribution, call sdbinst.exe and point it at the SDB.

    Thanks,

    Chris

  21. Ravi says:

    Can i get information of functionality,design and working of shims and list of applications compatible with it.

    Thanks in advance.

  22. cjacks says:

    Hi Ravi,

    Can you be more specific? This blog post is an exploration of the design and working of shims. What is it that you feel is missing? I’d be happy to go into it.

    Thanks,

    Chris

  23. ravi says:

    Hi chris

    thanks for ur response,

    my search is how to design a shim or write a shim?

  24. ravi says:

    Hi chris,

    I want to be very clear, i am new to this technology,

    I just know what is shim,where it can be used and when it can be used,but i want to know how to write a shim. or design a shim? where can i get the material or liks relating to this topic.

    thanks in advance.

  25. Q says:

    Thanks for the great article, Chris! Any pointers on using shims on the Windows Mobile platform, or is it more or less the same as the example you gave for Windows Vista? -Q

  26. cjacks says:

    Ravi -

    You can’t write your own shims (although you can write your own API interception – what we do isn’t magic), but we write the shims we ship to be as general purpose as possible. I’d start with the documentation in the Application Compatibility Toolkit, and I write about them here all the time as well.

    Q -

    Honestly, I have no idea if Windows Mobile has shims or not – it’s a totally different code base which I don’t have access to. I’m not even sure how you could find out? Interesting…

    Chris

  27. ABC says:

    I am really very happy to see these blogs about SHIM. I have one doubt, what exactly the shim is doing here. If we apply shim to any issue, is it really changing the application code or it is bypassing the code?

    and anyone is having any idea about this folder "C:windowsAppPatchsysmain.sdb", please share your knowledge that would be really greatful.

    ABC

  28. cjacks says:

    Hi ABC,

    It’s injecting code between the app and Windows, so neither gets modified – this entry explains the technical details.

    sysmain.sdb is the shim database we ship with Windows. You can use CompatAdmin to see what’s in there.

    Chris

  29. ABC says:

    So if any application is made compatible with the shims are placed in this SYSMAIN.sdb, right?

    if we delete this SYSMAIN.sdb file from our local machine, we could see the compatible impact on applications.

  30. ABC says:

    I know these blogs are not related to SAT Tool. But I did not find any blogs which clear our doubts regarding this SAT Tool.

    But I have one doubt and it’s urgent to me. Hence I am posting here itself only….excuse me in this….

    Using SAT Tool we can find out the installation issues. But if we are running the .EXE files(legacy applications) tool is running and capturing the issues.

    But if go with the .MSI( msi installer applications), it is throwing an error saying, "set up did not run successfully"….

    please tell me, this SAT Tool is only to find the legacy application installation issues or will it work for MSI based app’s also?….

    I need this info. very urgently…..if it is possible pls reply ASAP.

    Thanks in advance ….

    -ABC

  31. cjacks says:

    There is no reason to mess around with sysmain.sdb. It’s protected by Windows Resource Protection anyway. Why are you so obsessed with messing around with it? if you create your own shim database, it’s a separate SDB. Sysmain is only updated via Windows Update (where it’s noted as a compatibility update – that’s what we’re doing is updating this).

    Are you migrating to 64-bit? If not, SAT is not very useful to you. I never really use it because it only catches 3 types of issues, 1 of those is auto-mitigated, and the others are only relevant if you’re going to 64-bit Windows, so I’m not an expert in that tool. I find other tools more useful for finding 16-bit binarys and 32-bit drivers that don’t require me to have to run the full install – you can detect that statically.

  32. ABC says:

    We are migrating to Vista,means it’s 64bit right?

    So this SAT Tool is required to identify the installation issues.

    and as you said there are other tools to identify the issues, if you don’t mind could you give me the tool names and whether these are Microsoft Tools are any other 3rd party tools?.

    pls I need your help in finding the instalation issues…

    -ABC

  33. cjacks says:

    There are 32-bit and 64-bit versions of Windows Vista.

    The tools I know about are ChangeBase AOK and App-DNA AppTitude. I have not used either personally, but I know folks at both companies personally and I’ve seen their stuff even though I don’t have either in my hot little hands yet.

  34. ABC says:

    ya, it’s clear. Even I heared about App-DNA.

    I created one DCP using the ACM tool and then after I ran the DCP, but I was getting FAILED and UNCOMPRESSED folders under ACT_LOG_SHARE instead getting the PROCESSED and UNCOMPRESSED.

    why I am getting the FAILED folder?..what might be the reasons for this. How can I find work around for this?….

    In advance thanks to you.

    -ABC

  35. ravi says:

    Hi chris,

    Thanks for ur response on my doubt on writng shims.

    can i have some sample applications two or three which used shims and how they over come the issue by shims

    thanks in advance.

  36. ravi says:

    Hi chris,

    Thanks for ur time, Actually i need a list of applications which can be shimmed and which has shims any staderd applications you know.

    Thanks in advance.

  37. ABC says:

    Hi Chris,

    I created one DCP using the ACM tool and then after I ran the DCP, but I was getting FAILED and UNCOMPRESSED folders under ACT_LOG_SHARE instead getting the PROCESSED and UNCOMPRESSED.

    why I am getting the FAILED folder?..what might be the reasons for this. How can I find work around for this?….

    Pls give me reply back……in most of the client machines I am seeing these folders, when I am creating the DCP.

    Thanks in advance…

    -ABC

  38. cjacks says:

    Ravi-

    CompatAdmin will show you commercial software that we have shimmed. If it’s not in there, we don’t know about it (or can’t fix it) and therefore I’d be hard pressed to point you to something I don’t know about. It still requires a human brain to map a problem in an app to a fix, which is why I write so much here.

    ABC -

    I don’t know why this is failing. I’d check the event log and see if it says anything there. You can also frequently try restarting the log processing service and then copying it from the failed folder back into the log processing share to have it try again.

    Chris

  39. ABC says:

    Hi Chrish,

    Thanks. Ya, even I tried so many times running the Log processing service. But when I ran the DCP, just for few seconds that service will be running and then it is getting tunred off…..I dont know what is happend….even I did R&D on this. But I could not.

    Please do you have any info. like where I can get the Trouble shooting material for this. Please I am helpless to my team now. I need it. I am completely depended on you..

  40. cjacks says:

    ABC-

    I wouldn’t get too dependent on me – I’m not an official (or even unofficial) support channel…

    I’d post your query on the ACT newsgroups. It’s microsoft.public.windows.app_compatibility.

    http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.deployment.app_compatibility&cat=en_US_BFF781FD-77D8-BFD2-4429-5A9D175157BF&lang=en&cr=US

    Chris

  41. ravi says:

    Hi chris,

    Thank you very much this is what i am looking into.

    I got what i want

  42. One of most common issues we face with PowerShell comes from users or ISVs misunderstanding PowerShell’s

  43. santhosh.kulandaiyan says:

    This article is a fantastic explanation of internals of Shims. Thanks Chris!

    So shims can not do suppress UAC, (AAM or OTS popups..). so then lets say that my application writes to C:program filesabc.txt which obviously needs admin credentials to succeed. And when i run this program as guest user, it would normally virtualize if the app is not UAC aware.

    okay, so now let me run it through ACT and SUA gives me the compatibility problems (because it is a legacy application – meaning, without the manifest). and then let me shim this application up. now because the application still wont be able to write to c:program filesabc.txt —> the virtualization will still happen.. Is that correct? The shim will not be able to fix this..(or would the shim automatically make the application UAC compliant?)

    Thanks

    Santhosh

  44. cjacks says:

    Hi Santhosh,

    If virtualization is already fixing it, I would just let it. It’s redirecting to a per-user area of the file system. You could also shim it with CorrectFilePaths to redirect to a per-user area of the file system, but if I had to choose between the one that requires me to work, and the one that I get for free, I’ll take the free one! :-)

    Thanks,

    Chris