What’s up with D3DCompiler_xx.DLL?


For most of the early evolution of the HLSL language, the compiler was part of the D3DX utility library (aka D3DX9). This compiler supported Shader Models 1.x, 2.0, and 3.0 for Direct3D 9 vertex shaders and pixel shaders. For Direct3D 10, the graphics team started a new compiler to support the demands of the new Direct3D 10 API with Shader Model 4.0 versions of vertex, pixel, and geometry shaders. Direct3D 10.x added Shader Model 4.1 variants. Direct3D 11 adds Shader Model 5.0 and two new classes of shaders for tessellation (hull & domain) shaders.

There have been two main consequences to this new HLSL compiler development effort.

First, the “new” HLSL compiler (or since we are talking about something that happened around 2006, the now “current” HLSL compiler) supports almost, but not quite all, of the HLSL shader profiles. It does support Shader Model 2.0 and 3.0, and even supports Vertex Shader Model 1.x shaders. It does not support Pixel Shader Model 1.x shaders. Partly it was a matter of diminishing engineering returns, and partly these very early pixel shader models were not “complete” in a computational sense and required immense special-case support to work. Further, the baseline requirement for the Windows Vista desktop and the WDDM driver model was Pixel Shader 2.0 support. Thus Pixel Shader Model 1.x was deprecated.

To ease the transition, we provide a “/LD” command-line switch for FXC.EXE, and the D3DXSHADER_USE_LEGACY_D3DX9_31_DLL flag for D3DXCompile* APIs  that easily allows developers to invoke the now ‘legacy’ compiler for those cases where they still had to compile Pixel Shader Model 1.x shaders. We also added some features to the “new” HLSL compiler that would silently promote all Pixel Shader 1.x profiles to Pixel Shader 2.0 profiles  to avoid the need to edit potentially hundreds of shader files just to deprecate these older models (this is part of what happens when using “/Gec” or the D3DXSHADER_ENABLE_BACKWARDS_COMPATIBILITY flag). So all fine and dandy, but this does tend to make a bit of a messy story for what DLLs do you actually need at runtime, and therefore have to deploy to your customers. If you use the ‘legacy’ compiler, you have to ship the D3DX9_31*.DLL in addition to whatever ‘current’ D3DX9 DLL you were making use of. Ideally of course you’d do all your HLSL shader compilation at build time and your customers would never need the compiler, but this is not so easy to achieve especially for User-Generated Content toolsets.

The second impact of this work was the realization that as we added more generations of ‘active’ Direct3D API, and their matching D3DX utility library, we were duplicating a lot of the same code and potentially having the same few megabytes of compiled code loaded several times in an application. Early releases of D3DX10 had a copy of the compiler, and the same code was inside D3DX9. As we were working on Direct3D 11, this means we might have yet another copy in the eventual D3DX11 DLL as well. Over the intervening few years, the graphics team has gradually dis-entangled the HLSL compiler from the rest of D3DX9. The D3DCompiler_xx.DLL was introduced to host the ‘shared’ HLSL compiler, and D3DX10 made use of it. As it took a while to get all the Effects 9 interactions with the HLSL codebase worked out, and all the shader reflection required to support it, for a while we continued to have the HLSL compiler in both the D3DX9 DLL and D3DCompiler, but starting in the 2009 releases we were able to get D3DX9, D3DX10, and D3DX11 all using D3DCompiler_*.DLL.

This opens up an interesting option for developers: Calling D3DCompiler_*.DLL directly and not using the D3DX utility library at all. We know that some customers only use it for HLSL support, and the majority of the functionality for it was in D3DCompiler_*.DLL. In the August 2009 release we added D3DCompiler.h to allow direct access to the DLL’s entry points for this purpose. D3DX still remains a convenient way to use it (the D3DCompiler.h only has “FromMemory” and not any “FromFile” versions), but they are no longer hosted in the same DLL. It should also be noted that the DirectX 11 Shader Reflection API is in fact hosted in D3DCompiler_xx.DLL (not D3D11.DLL as implied by the documentation).

In addition to all the graphics API shader models, there are also Effects library profiles (fx_2_0, fx_4_0, fx_4_1, and fx_5_0) for the different generations of the Effects library. Effects 9 runtime is part of the D3DX9 utility library. The Effects 10 runtime is provided in the OS (along with an OS copy of the HLSL Shader Model 4.0 compiler which matches the DirectX SDK December 2005 release). The Effects 11 runtime is provided as source in the DirectX SDK, and this relies on the D3DCompiler.h interfaces to perform shader reflection operations.

The D3DCompiler.h API as written was originally developed for Direct3D 10.x and extended for Direct3D 11. As such, it made use of the ID3D10Blob API and many of its related #defines were in the D3D10 headers using the “D3D10_” convention. This has the side-effect of DirectX11 code using some of these D3D10 symbols, which is a bit confusing. For the DirectX SDK (June 2010) release, we are addressing this by making a number of these “versionless” defines with aliases for older code: ID3DBlob, ID3DInclude, D3D_SHADER_MACRO, D3DCOMPILE_ to replace D3D10_SHADER_* and D3D10_EFFECT_* etc. There is a D3DCreateBlob() function now in D3DCompiler_xx.DLL as well to avoid having to use the D3D10 DLL’s D3D10CreateBlob() entry-point. Existing code should continue to compile, but these changes should make it easier to be “clean” in your use of DirectX 11 APIs.

Update: The most current version of D3DCompiler_xx.DLL is available in the Windows SDK.

Related: HLSL, FXC, and D3DCompile

Comments (9)

  1. Annonymous says:

    I have this problem:

    I am using visual studio 11 Beta (also has visual studio 2010), DirectX SDK (june 2010 iirc), and windows sdk 7, using windows 7.

    I am trying to use D3Dcompiler.h from the windows sdk, which should include the D3DCompileFromFile method, however visual studio keep using the D3DCompiler.h from the DirectX SDK path (june 2010 one), Even though I set the Toolset to Windows 7.1.

    How do I solve this?

  2. walbourn says:

    If you set to the "Platform Toolset" of "Windows 7.1 SDK" you are actually using the VS 2010 compiler and tools.

    If using VS 11, you should use the Windows 8.0 SDK and only use the DXSDK for the few things like D3DX that are legacy and no longer included. See <msdn.microsoft.com/…/ee663275.aspx> for details on how to do it. You should check the include paths you have set for your project under VC++ Directories.

  3. Kirill says:

    What is the right way to rpeort bugs against HLSL compiler ?

    I have reported one (against June 2010 version) but it was lost in ages witohut resolution…

    Thanks

  4. walbourn says:

    The DirectX SDK connect site used to be the place, but it's offline. I'll inquire.

    Did you try to repro the problem you found on June 2010 with the Windows 8.0 SDK version? Many bugs were fixed in the new version.

  5. I still manage to reproduce the problem with the new FXC.exe (9.30.9200.16384).

    It failed to compile my shader source, however previous version (9.24.950.2656) is able to.

    It seems to be a bug in the new compiler. I have the simple source to reproduce.

    What is the right way to rpeort bugs against HLSL compiler to get  the things addressed ?

    Thanks

  6. walbourn says:

    I looked at Kirill's repro. Basically his shader is very close to the instruction limits of ps_2_0. Older compilers happen to make it just fit, but all the compilers from 2009 and later can't quite make it fit. It compiles fine with ps_2_a. This means the shader should be simplified for ps_2_0, move to a higher shader model, or you have to stick with an old compiler to keep it "as is".

  7. Voodooman says:

    Wow, so stupid!

    So many games was no shipped with _46 and _47 dlls and yet Dxsetup not updated to include them, latest 8.1 SDK missing _46, 44 and 44 and 45 totally missing, and we have to manually search 46 and 47 dlls and apply them to game or system just because for some stupid reason MS cant add few extra dlls and update dxsetup! And its not game dev fault they don't ship dlls its yours, because earlier your had eula that forced redist installers instead direct dlls, and you don't make this information clear for developers about per application distribution. And why do you even bother them to do that? its so hard to push few extra libs via winodws update to ensure that every system has these dlls?

  8. walbourn says:

    Everything related to the legacy DirectX SDK including the DirectX End-User Runtime (aka DXSETUP) is deprecated, were last updated in April 2011, and won't be updated again. See Not So DirectSetup for details. I also cover all the D3DCompiler version issues in the linked blog post above

    Changing a decade+ of habit is certainly difficult. The existing DXSETUP exists as a fallback for broken installers for titles built with the legacy DXSDK which use #43 or earlier. That said, I'm sure there are recent titles that still use the legacy DirectX SDK but also use the Windows 8.x SDK with VS 2012/VS 2013 that didn't fully understand the new deployment model. If built with VS 2013/Windows 8.1 SDK and run on Windows 8.1 or later, it already 'just worked'. Presumably if they support Windows 7, they would have tried testing on Windows 7…

    #44 and #45 were beta DLLs so nobody should have shipped using them. Which apps did you find that use those?