HLSL, FXC, and D3DCompile


With the retirement of D3DX (See “Where is the DirectX SDK?”, “Where is the DirectX SDK (2013 Edition)?“, and “Where is the DirectX SDK (2015 Edition)?“), there is some confusion about how applications access the High-Level Shader Language (HLSL) compiler. As mentioned in an earlier post (See “What’s up with D3DCompiler_xx.DLL?”), the HLSL compilation functions, shader reflection, and some support functions for the compiler were pulled out of D3DX as of the DirectX SDK (August 2009) release. While many developers continue to use the D3DX functions to compile HLSL and Effects, they are just pass-through functions to the D3DCompile API which can be used directly instead. The latest version of the D3DCompile API includes some new functions as well such as D3DCompileFromFile and D3DCompile2.

Windows 8.1: The latest version of the D3DCompiler (#47) is now included with the Windows 8.1 OS and is available for runtime use by Windows Store apps. Runtime compilation of HLSL is supported by using async IO and D3DCompile, which is why D3DCompileFromFile is not available for submitted applications. There is a new HLSL linker option as well that provides some of this flexibility to compose shaders at runtime without recompiling from the HLSL source.

Visual Studio 2012 and HLSL

With Visual Studio 2012, HLSL and the FXC.EXE tool is now integrated into build environment and the Windows 8.0 SDK. If you add .hlsl and .fx extension files to your project, there are properties for controlling how the HLSL compiler is invoked as part of your build including target profile, entry point name, etc. The resulting compiled shader uses the default extension of .cso (Compiled Shader Object). The texture editor supports HLSL colorization, and the Visual Studio Graphics Diagnostics feature provides support for debugging HLSL as well.

There is one quirk of having HLSL integrated into MSBuild that requires some restructuring of existing HLSL source files. The MSBuild system assumes that each source files is built exactly once. If you need to build the same HLSL source multiple times, you’ll need individual .hlsl or .fx files for each instance you want to build added to your project. Judicious use of the #include directive should make it fairly easy to share the same source where needed, but this is a bit of a move away from the .fx file model where all variations of shaders were kept in a single file. You of course can still make use of some custom build script and integrate a custom build step for more sophisticated generation of shaders, but this “one .hlsl file per shader combination” model is going to be seen in a lot of samples.

Note: This automatic integration only works for C++ projects, not C# projects.

VS 2013: Visual Studio 2013 comes with Windows 8.1 SDK which includes the D3DCompiler_47.DLL. It works essentially the same as it did with VS 2012.

Build-time vs. Runtime Compilation

As we have recommended for many years, developers should compile their HLSL shaders at build-time rather than rely on runtime compilation. There is little benefit to doing runtime compilation for most scenarios, as vendor-specific micro-optimizations are done by the driver at runtime when converting the HLSL binary shader blobs to vendor-specific instructions as part of the shader object creation step. Developers don’t generally want the HLSL compiler results to change ‘in the field’ over time, so it makes more sense to do compilation at build-time. That is the primary usage scenario expected with the Windows 8.0 SDK and Visual Studio 11, and is the only supported scenario for Windows Store apps (a.k.a Metro style apps) in Windows 8. FXC and the MSBuild rules above are well suited to this usage.

For development purposes, it is often very convenient to use runtime compilation. Such applications typically keep a ‘shader cache’ file that is generated during execution, and a ‘final’ version of the shader cache is then shipped with the title. This scenario is supported by the D3DCompile API hosted in the D3DComplier_*.DLL. This is fully supported for Win32 desktop applications, and can even be used for Windows Store apps during development although not in deployment. In this case, the D3DCompiler_*.DLL from the Windows 8.0 SDK should be included side-by-side with the application itself (i.e. application local deployment). The d3dcomplier.h header and d3dcompiler.lib are part of the Windows 8.0 SDK include and lib paths, and the D3DCompiler_*.DLL itself is located in "%WindowsSdkDir%\redist\d3d". If your application makes use of the D3DCompile API, you will want to add a Custom Build Step for the project to make sure the right DLL is copied as part of your build.

Platform: Win32 (All Configurations)
Command Line: copy “$(WindowsSdkDir)redist\d3d\x86\D3DCompile*.DLL” “$(TargetDir)”
Outputs:$(TargetDir)D3DCompiler_46.DLL

Platform: x64 (All Configurations)
Command Line: copy “$(WindowsSdkDir)redist\d3d\x64\D3DCompile*.DLL” “$(TargetDir)”
Outputs:$(TargetDir)D3DCompiler_46.DLL

Platform: ARM (All Configurations)
Command Line: copy “$(WindowsSdkDir)redist\d3d\arm\D3DCompile*.DLL” “$(TargetDir)”
Outputs:$(TargetDir)D3DCompiler_46.DLL

Note this assumes you are using the Windows 8.0 SDK RTM

Redistribution

The legacy DirectX SDK versions of D3DCompile (#33 – #43) were deployed using the DirectSetup technology and installed to the %WINDIR%\System32 (and %WINDIR%\SysWow64) folders using the redistribution package (which requires administrator rights at installation time). With the Windows 8.0 SDK, the D3DCompile DLL is never installed to the %WINDIR% folders. Instead, you should copy it into your applications folder and deploy it ‘application local’ with your Win32 desktop application.

Windows 8.1: D3DCompile #47 is included with the Windows 8.1 OS so there is no need to redistribute it for Windows Store apps targeting Windows 8.1. There is also a copy in the Windows 8.1 SDK you can redistribute the same was as you could D3DCompiler #46 from the Windows 8.0 SDK.

Effects

The Effects 11 (aka FX11) shared source library requires the “Effect” Shader Type and the “Shader Model 5” profile. It also requires the D3DCompiler_*.DLL at runtime for the shader reflection APIs, which makes it unsuitable for Windows Store apps.

Note: The “fx” profiles (fx_2_0, fx_4_0, fx_4_1, and fx_5_0) are deprecated and may be removed from a future update of the HLSL compiler.

Platform Support

The shader binaries built by the current FXC.EXE and D3DCompile API using the Shader Model 2.0 and Shader Model 3.0 profiles will work on all Direct3D 9.0c compatible platforms including Windows XP. The actual D3DCompiler_*.DLL in the Windows 8.0 SDK itself only supports Windows Vista, Windows 7, and Windows 8. The same is true of the Visual Studio 11 compiler toolset, although the VS 11 IDE can build using the Visual Studio 2010 toolset which supports Windows XP Service Pack 3 via a Platform Toolset setting of "v100" assuming Visual Studio 2010 is installed side-by-side with Visual Studio 11. You can also install the Windows 7.1 SDK side-by-side and use the Platform Toolset setting of "Windows 7.1 SDK" which uses the VS 2010 RTM compiler toolset and headers that were included with the Windows 7.1 SDK.

Note: you can select a Platform Toolset of "v100-sdk80" with the Windows 8.0 SDK and Visual Studio 2010 installed to get the VS 2010 compiler toolset but using the newer Windows 8.0 SDK header set, but remember that version of D3DCompiler_*.DLL does not support runtime use or deployment on Windows XP.

For Shader Model 5 shaders, some HLSL language features require the DirectX 11.1 runtime. This uses the existing mechanism that indicates a shader requires the hardware support double-precision shaders for DirectX 11.0. This includes use of Feature Level 11.1 features (UAVs at every stage, 64 UAV slots), DirectX 11.1 runtime features (minimum-precision data-types), and new DirectX 11.1 optional hardware features (extended double-precision instructions, SAD4). A shader blob that indicates one of these requirements will fail to bind at runtime if the system doesn’t support them. This information is included in the HLSL disassembly output, and all require some explicit feature usage in the HLSL source.

Porting Notes

Here’s a handy table of equivalents for replacing legacy D3DX HLSL compiler related functions (see Living without D3DX for a complete listing).

D3DXCompileShaderFromFile
D3DX10CompileFromFile
D3DX11CompileFromFile
D3DCompileFromFile
D3DXCompileShader
D3D10CompileShader
D3DX10CompileFromMemory
D3DX11CompileFromMemory
D3DCompile
D3DXCompileShaderFromResource
D3DX10CompileFromResource
D3DX11CompileFromResource
No direct equivalent. Can use resource APIs and then D3DCompile above.
D3DXPreprocessShader
D3DXPreprocessShaderFromFile
D3DXPreprocessShaderFromResource
D3D10PreprocessShader
D3DX10PreprocessShaderFromFile
D3DX10PreprocessShaderFromMemory
D3DX10PreprocessShaderFromResource
D3DX11PreprocessShaderFromFile
D3DX11PreprocessShaderFromMemory
D3DX11PreprocessShaderFromResource
D3DPreprocess
D3DXDisassembleShader
D3D10DisassembleShader
D3DX10DisassembleShader
D3DDisassemble
D3D10ReflectShader
D3DX10ReflectShader
D3DReflect
D3D11Reflect
ID3DXBuffer
ID3D10Blob
ID3DBlob
D3DXCreateBuffer
D3D10CreateBlob
D3DCreateBlob
D3D10GetInputSignatureBlob
D3D10GetOutputSignatureBlob
D3D10GetInputAndOutputSignatureBlob
D3D10GetShaderDebugInfo
D3DGetBlobPart

Version History

  • D3DCompiler_47 – Windows 10 (inbox), Windows 10 SDK; Windows 8.1 (inbox), Windows 8.1 SDK, Visual Studio 2013
  • D3DCompiler_46 – Windows 8.0 SDK, Visual Studio 2012
  • D3DCompiler_43 – DirectX SDK (June 2010)
  • D3DCompiler_42 – DirectX SDK (February 2010)
  • D3DCompiler_41 – DirectX SDK (March 2009)
  • D3DCompiler_40 – DirectX SDK (November 2008)
  • D3DCompiler_39 – DirectX SDK (August 2008)
  • D3DCompiler_38 – DirectX SDK (June 2008)
  • D3DCompiler_37 – DirectX SDK (March 2008)
  • D3DCompiler_36 – DirectX SDK (November 2007)
  • D3DCompiler_35 – DirectX SDK (August 2007)
  • D3DCompiler_34 – DirectX SDK (June 2007)
  • D3DCompiler_33 – DirectX SDK (Aprli 2007)

Presentations

High Level Shader Language (HLSL) Update—Introducing Version 5.0 (Gamefest 2008)

Symbolic Differentiation in GPU Shaders

Related: Getting Started with Direct3D 11, What’s up with D3DCompiler_xx.DLL

Comments (21)

  1. Martins Mozeiko says:

    Is it legal to distribute D3DCompiler dll file together with application? I understand that previously it was not legal to distribute individual DirectX components like d3dx dll file. You could distribute small directx redistributable installer that installed this file. Has something changed in this aspect?

  2. walbourn says:

    The EULA for the Windows 8.0 SDK allows for the D3DCSX and D3DCompiler DLL to be redistributed application local.

    The DirectX SDK EULA for DirectSetup did not allow application-local deployment, only the use of the DirectSetup REDIST package. This package's last update contains the June 2010 version of D3DCompiler (#43). All newer versions of this DLL (#44 and beyond) are only available as part of the Windows 8.0 SDK.

    Note I'm not a lawyer and cannot give legal advice. You should seek your own legal council and determination based on the Windows 8.0 SDK's EULA.

  3. Eugene Golushkov says:

    Is there any chance that D3DReflect intefaces would be made available for Metro style applications? It`s not time/memory/CPU consuming functionality, and the only way to construct input layouts and discover constant buffer layouts for shader without hardcoding it into the application. It seems that D3DReflect interfaces were just unfortunate to be in the same dll as D3DCompile part, and were forbidden for Metro style applications only for this reason.

  4. Eugene Golushkov says:

    I use such string to copy d3dcompiler.dll

    robocopy "$(WindowsSdkDir)RedistD3D$(TargetedSDKArchitecture)" "$(ProjectDir)." d3dcompiler_44.dll

    rem suppress successful robocopy exit statuses, only report genuine errors (bitmask 16 and 8 settings)

    set/A ERRORLEVEL="%ERRORLEVEL% & 24"

    It`s valid for all architectures and does not copy dll second time.

  5. Isaac Lascasas says:

    How does the compiler know if it is a vs,gs or ps and for example the profile you want to compile to? If I add hlsl files to a project it asumes vs. Is there a way to let the compiler know from the filename or the code that information? Do I need to manually set all this options for every .hlsl file in the project?

  6. walbourn says:

    The target shader stage is part of the profile which you set as a per file project setting in VS2012.

  7. oscarbg says:

    So how is WebGL gonna we supported on WinRT if OpenGL isn't avaiable and GLSL->HLSL translators like ANGLE think would require dynamic runtime compilation i.e. shipping with d3dcompiler.dll..

    Also saying is not supported means will be blocked from posting store?

  8. walbourn says:

    D3DCompile*.DLL will not pass the Windows App Store Certification Toolkit (WACK) tests, so "not supported" does in fact mean "can't be deployed to Windows Store".

  9. Saad says:

    How do I add d3dcompiler_46.dll to my visual studio 2012 project . I am getting the following error without it.

    Effects11.lib(EffectAPI.obj) : error LNK2001: unresolved external symbol D3DCompileFromFile

  10. walbourn says:

    @Saad: D3DCompiler.lib from the Windows 8.x SDK. See the CodePlex docs.

  11. James says:

    What are the actual/legal differences between the DLL in the Redist folder and the ones in the bin folder? For example:

    C:Program Files (x86)Windows Kits8.0RedistD3Dx64d3dcompiler_46.dll

    vs.

    C:Program Files (x86)Windows Kits8.0binx64d3dcompiler_46.dll

  12. walbourn says:

    @James – in theory they are the same DLL, but only the files in the REDIST folder are actually covered by the EULA for Redistribution.

  13. James says:

    They actually aren't binary identical. The non-redist is 9.30.9200.20546 and three months newer than the redist version, 9.30.960.9200. If the newer DLL is somehow better, is it legal to distribute shaders compiled with that library?

  14. walbourn says:

    @James – It is likely it has been patched as part of a VS update. I'll ask about it. In any case, you can always redist the resulting shader binary files.

  15. walbourn says:

    The REDIST version I have on my system installed with the Windows 8.0 SDK RTM is 9.30.9200.20789, which exactly matches the BIN version. Perhaps you have something left-over from previews installed?

  16. James says:

    Well, I just looked again in the exact same paths, and now I have 9.30.9200.20789 in both locations (although they still aren't binary exact for some reason). I dunno what I could have done to have updated the files. The only things I can remember installing since are VS 2013 trial, 2013 Update 2 RC, and 2013 Update 2 RTM.

  17. walbourn says:

    The Windows 8.1 SDK EULA also allows local redist of the compiler dll per the REDIST.TXT.

  18. p1pkin says:

    is there a change log for D3Dcompiler ? I noticed d3dcompiler_47.dll was silently updated in Windows 10 SDK, and have some 8.0/8.1 bugs fixed (I noticed StructuredBuffer + SV_Depth output was fixed)

    but filename still the same – _47

  19. walbourn says:

    For Windows 10, the graphics team decided not to update the DLL name which is confusing. Since it is once again an "OS component" changing the name is very impactful and prevents many bug fixes from being applied to existing applications. The D3DCompiler_47.DLL in the Windows 10 OS and the Windows 10 SDK is newer (File version 10.0.10240.16384) than the Windows 8.1/Windows 8.1 SDK version of D3DCompiler_47.DLL (File version 6.3.9600.16384).

  20. bharat singh says:

    is there d3dcompiler_dll 46.dll

    for windows 7

  21. walbourn says:

    The D3DCompile_46.dll can be redistributed with a Windows desktop application that supports Windows Vista or later. You find the DLL in the Windows 8.0 SDK under the folder $(WindowsSdkDir)redistd3dx86 (32-bit apps) or $(WindowsSdkDir)redistd3dx64 (64-bit native apps). D3DCompile_47.dll is in the Windows 8.1 SDK in the same place which supports the same OSes.