Using automatic shader PDB resolution in PIX


Many features of PIX on Windows require access to shader debug information, but it can be a challenge to handle this efficiently. Shader debug information takes up a lot of space and isn’t necessary for executing shaders, so this data is often removed from the compiled shaders but doing so severely limits the help PIX on Windows can provide.

The Direct3D shader compilers allow you to strip debug data out of compiled shaders and place the debug data in a convenient location, typically outside of the disk image of your application. By following the guidelines below, you can set up your build system to produce debug data in a way that lets PIX discover and load shader symbols automatically. This allows you to create lean builds of your shaders without sacrificing the ability to inspect and debug your shaders.

There are a few steps to go through to enable this. The general process is outlined below and specific steps for DXBC and DXIL based shader follow.

First, the compiler must emit debug data. Your build system must then extract the debug information and store it in a file. The compiler can offer a suggested name for the external debug data. This name will also be available to PIX, and is the key by which PIX can automatically load the debug data.

The suggested name is in fact a hash of the shader binary and, optionally, the HLSL source code that contributed to it. You can pass flags to the Direct3D compilers to control whether or not source is included in the hash.

Lastly, inform PIX of the location(s) of these saved PDB files via the Symbol / PDB Options section of the Settings menu.

Whenever shaders are examined or shader debugging is launched, the debug data should be automatically located and loaded. Any failures in this process will be reported when accessing the shader in the Pipeline or Shader Debugger view.

Note that the name provided by the compiler can be replaced. If you supply your own name, ensure that it is unique, or PIX may load the wrong debug data file. User-supplied files can have embedded subdirectories, and PIX will attempt to find the debug data file in each subdirectory. For example, if you supply Symbols\Material137\PixelShader.PDB, PIX will search for that file appended to each of its PDB search paths, and then will try to find Material137\PixelShader.PDB, and finally PixelShader.PDB.

Absolute paths are also supported. The string encoding is expected to be UTF-8. The d3dcompiler_47.dll’s D3DSetBlobPart function can insert your new filename, and this function works for both DXBC and DXIL shaders. Details on that process are listed at the end of this section.

The process of removing debug data is different for DXBC versus DXIL shaders. Both processes are described in the following.

DXBC shaders (compiled with fxc or via d3dcompiler*.dll)

Fxc.exe offers command-line support for generating correctly-named PDBs directly. You need FXC from the Windows 10 Fall Creator’s Update SDK (version 16299) or later. Use the command line below:

fxc.exe /Zi /Zss /Fd .\

Those parameters are:
/Zi Turn on debug information.
/Zss Include source when generating hash for the PDB name (the alternative is /Zsb, which doesn’t include source code in the hash).
/Fd <outputdirectory>\ Name an output directory instead of a PDB name.

Note the trailing backslash on the argument for /Fd. This trailing backslash tells FXC to generate the hash-names for the PDB and place the resulting files in the specified directory.

If fxc.exe is not part of your tool chain, several APIs are defined in d3dcompiler.h that allow you to extract debug data from a compiled shader and place it into a file in a location convenient to your situation. These APIs are flat DLL exports from d3dcompiler*.dll.

First of all, compile your shaders with the following flags:

  • D3DCOMPILE_DEBUG: This flag causes the compiler to emit debug information into the output container.
  • D3DCOMPILE_DEBUG_NAME_FOR_SOURCE or D3DCOMPILE_DEBUG_NAME_FOR_BINARY: These flags cause the compiler to emit the suggested name for the debug data file. Note that FOR_SOURCE means that the suggested name will be a hash of both the HLSL source code and the resulting object code. FOR_BINARY means that only the binary will be considered.

To remove the debug data from the shader and name the PDB files do the following. Note that error handling has been omitted for clarity.

// Starting with a compiled shader in pShaderBytecode of length BytecodeLength,
// retrieve the debug info part of the shader:
ComPtr<ID3DBlob> pPDB;
D3DGetBlobPart(pShaderBytecode, BytecodeLength, D3D_BLOB_PDB, 0, pPDB.GetAddressOf());

// Now retrieve the suggested name for the debug data file:
ComPtr<ID3DBlob> pPDBName;
D3DGetBlobPart(pShaderBytecode, BytecodeLength, D3D_BLOB_DEBUG_NAME, 0, pPDBName.GetAddressOf());

// This struct represents the first four bytes of the name blob:
struct ShaderDebugName 
{
    uint16_t Flags;       // Reserved, must be set to zero.
    uint16_t NameLength;  // Length of the debug name, without null terminator.
                          // Followed by NameLength bytes of the UTF-8-encoded name.
                          // Followed by a null terminator.
                          // Followed by [0-3] zero bytes to align to a 4-byte boundary.
};

auto pDebugNameData = reinterpret_cast<const ShaderDebugName *>(pPDBName->GetBufferPointer());
auto pName = reinterpret_cast<const char *>( pDebugNameData) + 1);

// Now write the contents of the blob pPDB to a file named the value of pName
// Not illustrated here

// Now remove the debug info from the target shader, resulting in a smaller shader
// in your final application’s data:
ComPtr<ID3DBlob> pStripped;

D3DStripShader(pShaderBytecode, BytecodeLength, D3DCOMPILER_STRIP_DEBUG_INFO, pStripped.GetAddressOf());

// Finally, write the contents of pStripped as your final shader file.

DXIL Shaders (compiled via dxcompiler.dll)

You can discover the relevant functions for enabling this for DXIL shaders via a factory function exposed by dxcompiler.dll as illustrated below.

First, compile your shader with the following flags:

  • /Zi – include debug information.
  • /Zss or /Zsb – generate a suggested name for the debug data. Note that /Zss means that both the HLSL and shader binary will be hashed to generate the name. /Zsb means that the source will not be considered.

The DXIL compiler offers a one-step mechanism for compiling and extracting debug information. Use the IDxcCompiler2::CompileWithDebug method, defined in dxcapi.h. The last two arguments allow you to retrieve a pointer to the suggested debug data file name, and the debug data itself. Simply write the data to a file with the suggested name. Be sure to pass the /Qstrip_debug flag if you wish to produce a smaller shader without duplicated debug information.

If it is inconvenient to use the one-step method, the following manual process can be followed to remove debug data and name the LLD files. Note that error handling has been omitted for clarity.

#include "DxilContainer.h"
#include "DXCApi.h"

Using namespace hlsl;

// First create the factory interface through which other Dxc interfaces can be retrieved:
decltype(&DxcCreateInstance) pfnDxcCreateInstance = 
    (decltype(&DxcCreateInstance))GetProcAddress(dxcompiler_dll, "DxcCreateInstance");

// Build a reflector for the existing container
ComPtr<IDxcContainerReflection> pDxcContainerReflection;
(*pfnDxcCreateInstance)(CLSID_DxcContainerReflection, IID_PPV_ARGS(pDxcContainerReflection.GetAddressOf()));

// Use this or any other mechanism for creating a simple COM object that publishes an IDxcBlob
// that wraps your shader binary.
ComPtr<IDxcBlob> pContainer = QuickCom::Make<DxcBlob>(pShaderBytecode, BytecodeLength);

// Load your shader binary into the container reflector:
pDxcContainerReflection->Load(pContainer.Get());

// Find which part index contains the shader name and retrieve it:
UINT32 debugNameIndex;
pDxcContainerReflection->FindFirstPartKind(DFCC_ShaderDebugName, &debugNameIndex);
ComPtr<IDxcBlob> pPDBName;
pDxcContainerReflection->GetPartContent(debugNameIndex, pPDBName.GetAddressOf());

// Find which part index contains the debug data and retrieve it:
UINT32 debugPartIndex;
pDxcContainerReflection->FindFirstPartKind(DFCC_ShaderDebugInfoDXIL, &debugPartIndex);
ComPtr<IDxcBlob> pPDB;
pDxcContainerReflection->GetPartContent(debugPartIndex, pPDB.GetAddressOf());

// DxilShaderDebugName is defined in DxilContainer.h
pDebugNameData = reinterpret_cast<const DxilShaderDebugName *>(pPDBName->GetBufferPointer());
auto pName = reinterpret_cast<const char *>( pDebugNameData + 1);

// Now write the contents of pPDB to a file named by pName
// Not illustrated here

// Lastly, create a new shader container with the debug data removed
ComPtr<IDxcContainerBuilder> pDxcContainerBuilder;
(*pfnDxcCreateInstance)(CLSID_DxcContainerBuilder, IID_PPV_ARGS(pDxcContainerBuilder.GetAddressOf()));
pDxcContainerBuilder->Load(pContainer.Get());
pDxcContainerBuilder->RemovePart(DFCC_ShaderDebugInfoDXIL);
ComPtr<IDxcOperationResult> pStrippedResult;
pDxcContainerBuilder->SerializeContainer(pStrippedResult.GetAddressOf());
pStrippedResult->GetResult(pStripped.GetAddressOf());

// Use the contents of pStripped as your final shader

Supplying a Custom Debug File Name

Follow this process to supply your own debug file name in such a way that PIX can discover it in your application’s shader binaries. Error handling has been removed for clarity:

// Any valid UTF-8 path is allowed
const char pNewName[] = "MyOwnUniqueName.lld";

// Blobs are always a multiple of 4 bytes long. Since DxilShaderDebugName
// is itself 4 bytes, we pad the storage of the string (not the string itself)
// to 4 bytes also.
size_t lengthOfNameStorage = (_countof(pNewName) + 0x3) & ~0x3;

// See above for the definition of DxilShaderDebugName
size_t nameBlobPartSize = sizeof(DxilShaderDebugName) + lengthOfNameStorage;

auto pNameBlobContent = reinterpret_cast<DxilShaderDebugName*>(malloc(nameBlobPartSize));

// Ensure bytes after name are indeed zeroes:
ZeroMemory(pNameBlobContent, nameBlobPartSize);

pNameBlobContent->Flags = 0;
// declared length does not include the null terminator:
pNameBlobContent->NameLength = _countof(pNewName) - 1;
// but the null terminator is expected to be present:
memcpy(pNameBlobContent + 1, pNewName, _countof(pNewName));

ComPtr<ID3DBlob> pShaderWithNewName;
D3DSetBlobPart(pShaderBytecode, BytecodeLength, D3D_BLOB_DEBUG_NAME, 0, pNameBlobContent, 
               nameBlobPartSize, pShaderWithNewName.GetAddressOf());

// D3DSetBlobPart returns a complete new blob container:
pShaderBytecode = pShaderWithNewName->GetBufferPointer();
BytecodeLength = pShaderWithNewName->GetBufferSize();

free(pNameBlobContent);
Skip to main content