Deferred Custom Actions and Permissions on Windows Vista with UAC Enabled


The first major obstacle to testing an application for compatibility on Windows Vista is to actually get that software installed, and there are still some challenges remaining in achieving this. One of the issues that has popped up is with deferred custom actions.

When you specify the type of a custom action in the MSI database, you are creating a bitmask. This isn’t always easy to see, since the Orca MSI editor provided with the Platform SDK presents this information in decimal rather than hexadecimal format, but it is true. To specify a deferred custom action, you specify msidbCustomActionTypeInScript, or 0x400, in the bitmask. So, if you wanted to do this with a DLL, you would logical or msidbCustomActionTypeInScript and msidbCustomActionTypeDll == 0x400 | 0x1 == 0x401 == 1025.

This is one of the custom action types that we are seeing that are causing problems as a result of UAC. With nothing more than 0x401, this cutom action will be run impersonating the logged in user, not in the system context. It is not using the elevated token, it is using the standard token, which has quite a few privileges and groups removed from it, including the local Administrators group! If your custom action requires elevated permissions to run, then it will fail, as will your installation.

However, you can specify that the custom acton will not use impersonation. By including in the bitmask msidbCustomActionNoImpersonate == 0x800, you will no longer be impersonating the currently logged in user (with their non-elevated token), and instead be running with system privileges just like the rest of the MSI installation. This avoids the issue of an access denied message being triggered by some action in the MSI (at the expense of removing the user impersonation). In the above example, we would use msidbCustomActionTypeInScript | msidbCustomActionTypeDll | msidbCustomActionNoImpersonate == 0x400 | 0x1 | 0x800 == 0xC01 == 3073.

For the setup developer, this means that custom action types that require impersonation of the user should be separated from custom action types that require elevated permissions – the impersonated user no longer has these permissions by default and this is no longer an appropriate assumption. For the administrator, it makes it possible to modify the custom action type (MSI is white box, and you can edit the contents of the MSI database using the Orca SDK tool) to hopefully end up with a successful setup.


Comments (12)

  1. joby says:

    How can I set the bitmask msidbCustomActionNoImpersonate in the MSI. Is it a property? If not which item in the table corresponds to this bitmask

    Rgds,

    Joby

  2. cjacks says:

    It’s the type column in the custom actions table – the Orca SDK tool presents it in decimal, so I just use calc.exe to convert to hex to I can see what is going on. Ran into this one again today! If you see a type value in the low 1000’s, then your spidey sense should be tingling…

  3. Marimuthu R says:

    Using Orca tool when i open my msi, when I click on the CustomAction at the left undet Tables, top the right I find a number of entries with titles Action, Type, Source and Target in this order.

    Could you please tell me where the above mentioned bitmask filed occures?

    I don’t find any fields as such.

  4. cjacks says:

    Marimuthu – see my response directly above your comment. 🙂

  5. bingen says:

    generally a 3073 in the type column will do fine for dll’s without impersonation. as a rule of thumb: substract 1024 and add 3072. or check the msi help… 😉

  6. Noam says:

    I created MSI installation in vs2005 with 2 OCX files and I added msidbCustomActionNoImpersonate action with type 3073.

    I run the MSI file on Vista and the OCX files were not register when the UAC is enabling. Can you help me?

    Noam

  7. cjacks says:

    Noam, you shouldn’t need to use custom actions to register COM components (such as OCX files) – this is built in to the Windows Installer framework. Use the RegisterClassInfo standard action, which references the Class table.

    I haven’t done this using VS2005 – I use WIX (Windows Installer XML) which provides you with much more flexibility. See: http://wix.sourceforge.net/

  8. Noam says:

    I browsed the web, but I can’t find the values I need to insert to the RegisterClassInfo action.

    I oppened the msi in ORCA, and in the Class table, there are a lot of parameters such as CLSID, Context, Component… and many more.

    Please help me.

    What should I write in the fields to register the OCX on Vista.

    Thanks,

    Noam

  9. Noam says:

    Hi,

    I openned the MSI in ORCA, but the ‘Class’ table is empty.

    In ‘Class’ table there are many columns, (such as CLSID, Context, Component and others).

    I understand that I need to insert the RegisterClassInfo action in there, but what are the other parameters values I need to insert in order to register the OCX files in Vista (wiithout disabling UAC)?

    I browsed the web, but couldn’t find any reference on it.

    Can you help me?

    Noam.

  10. cjacks says:

    Documentation on the class table is here:

    http://msdn2.microsoft.com/en-us/library/aa367861.aspx

    What tool are you using to generate your MSIs? If you’re using WIX, the most instructive tool I know of is dark.exe. It will reverse engineer the XML for any MSI you want. Look at how other people do things, and do similar things in your code. It’s very useful. If you’re hand-editing the tables (egad!) you can just look at other MSIs with Orca.

  11. Alder says:

    Chris Jackson, you’ve said:

    "Noam, you shouldn’t need to use custom actions to register COM components (such as OCX files) – this is built in to the Windows Installer framework. Use the RegisterClassInfo standard action, which references the Class table."

    But doesn’t it go against incapsulation paradigm? I have a .dll with COM component. If I have to use RegisterClassInfo, then I have to know the CLSID of factory of this COM. In case there will be addition of another .dll component and another factory to same dll, we will have to update installer manually, which is not error prone, while making the dll to register itself is.

  12. cjacks says:

    Alder-

    It all comes down to how you choose to "install" something. If you use the Windows Installer framework, your system changes are white box, and the framework can be responsible for adding or remvoing things. If you implement your "installer" in DllRegisterServer, you’re still writing stuff, just in a black box way in your own code.

    There are absolutely valid reasons for using either approach. My preference is to be white box, because then you can fiddle with that which you want to change, plus you don’t have to be responsible for cleaning up after yourself. (This is what I want to do – you take care of the reference count, the uninstall, etc.) But that doesn’t mean it’s the only, or even always the best, way to approach it.

    If dllregisterserver changes what it drops, then just calling it may be best. But you could also refactor to have one component per dll, and have these explicitly defined in the build process for both the app and the installer.

    We support you either way you want to do it. At some point, somebody has to know what you’re doing to complete the install.

    Chris

Skip to main content