Long Paths in .NET, Part 3 of 3 Redux [Kim Hamilton]

  • Article

My original part 3 blog caused confusion, mostly because it didn’t tie together loose ends and explain that an immediate .NET “solution” is at best partial. To minimize confusion and answer questions in the comments, I decided to do a complete overhaul and link to the original (here).

Win32 file-naming conventions include the MAX_PATH (260 character) restriction. A subset of Win32 APIs allow you to work around the MAX_PATH restriction by adding the \\?\ prefix.

Solutions based on the \\?\ prefix

Recall that these are the problems with the \\?\ prefix:

  1. Other apps may not support this prefix, and may not be able to use these files
  2. Not all Win32 APIs allow this prefix, meaning that the file will not necessarily work with other .NET APIs

#2 isn’t obvious to many people so let’s focus on that.

.NET dependence on Win32

.NET relies heavily on Win32 and isn’t always able to provide clean workarounds to Win32 restrictions. If we introduce a concept that has partial Win32 support, we either:

  1. Open up a new set of potential failures, or
  2. Add kludges on top of existing functions to make things work

An example of (a) is: you move a file to a location enabled with the \\?\ prefix, and then pass that path to another .NET API, which results in a call to a Win32 API that doesn’t support \\?\. (e.g. LoadLibrary). This sequence of operations would fail.

An example of (b) is: we try to make things work for Win32 functions that don’t support \\?\. This would be a major undertaking of rebuilding Win32 functionality or re-building a file system in the .NET layer (for example, we associate with shorter paths that can be accessed). Either approach is extremely risky and error prone, and we don’t see this as an option at the moment.

.NET solutions based on \\?\, given Win32 restrictions

Currently, .NET doesn’t allow you to use the \\?\ prefix at all. Let’s consider how .NET might enable it:

  1. Supported by default: .NET adds the \\?\ prefix behind the scenes
    1. Because of limited support of the \\?\ prefix, this will at times force you into awkward coding practices, like having to explicitly check size to make sure the file names are supported by your other apps or other functions.
    2. Also, this would be a maintenance burden going forward if Win32 provides a solution that isn’t based around \\?\
  2. Supported on an opt-in basis: callers have to opt in, such as by adding the \\?\ prefix
    1. This is just like if we had never blocked out the option from Win32
    2. Because this turns off canonicalization, this would be associated with full trust demand, which seems reasonable for applications that want to use long paths
    3. Could add convenience API such as Path.ToLongPath(file), which would add the \\?\ prefix for you, before you pass to System.IO APIs.

Other Solutions: Softening Impact

We also considered off-by-one attempts to soften the MAX_PATH limit, such as auto-shrinking which is done in the Windows shell. It’s compelling because such a path can be supported by other Win32 APIs and the shell. However, based on feedback, we think this just makes the story more confusing. The lack of consistency makes it useful to only corner-case scenarios, and certainly apps wouldn’t want to depend on it.

Proposed “Solution” (Best attempt for now)

We won’t be able to provide seamless long path support throughout .NET until something is done on the Win32 side to broaden support of long paths throughout their APIs.

As a mitigation to at least provide parity with Win32, we propose to allow use of the \\?\ prefix (described in #2 above) to avoid the MAX_PATH limitation. The caller must explicitly use this prefix, at best aided with a helper API as follows:

 String longPathEnabledName = Path.ToLongPath(@"C:\veryLongPath...\veryLongName.txt");
FileStream fs = new FileStream(longPathEnabledName);

This enables use of long paths throughout the System.IO namespace. Support of long path files outside of System.IO will be limited by support of underlying Win32 APIs.

This is not a real solution, but this provides core file (reading, writing, etc) functionality for long path files, which is compelling for certain types of apps. Such apps are best described as closed-system, file reading/writing-focused apps.

Ideal Solution

Some possibilities include:

  1. A Win32 solution, supported throughout Win32 APIs for full parity
  2. Going forward, less dependence on Strings as paths. This was touched on in my comments to long paths part 1 and is a very large discussion, for some later time.