What’s so special about the number 64 when it comes to TLS slots?


Last time, we ended with the question, "What's so special about the number 64?" when discovering that a program crashed if it ever got a TLS slot index greater than or equal to 64.

Versions of Windows prior to Windows 2000 supported up to 64 TLS slots. This was codified in the constant

#define TLS_MINIMUM_AVAILABLE 64

but even back then, it was noted in the documentation that 64 was merely the minimum. According to my really ancient copy of the Win32 SDK documentation

The constant TLS_MINIMUM_AVAILABLE defines the minimum number of TLS indices available in each process. This minimum is guaranteed to be at least 64 for all systems.

(I found a site that has something very close to the original documentation.)

Somehow, this got misinterpreted as "The constant TLS_MINIMUM_AVAILABLE defines the maximum number of TLS indices available in each process."

One theory is that the code was originally written back in the days when the actual limit was indeed 64, and the code was written based on the implementation rather than the documentation. (Possibly because they either got a source code license or reverse-engineered the function and observed that Tls­Alloc always returned a value less than 64.)

I'm not entirely convinced of this theory because the maximum number of TLS slots increased in Windows 2000, but the program in question was released in 2007.

On the other hand, this theory could still be valid the program was using a library that was originally written pre-2000. Even though the program itself was written after 2000, parts of it were written before 2000, back when the TLS limit was 64.

Another theory is that somebody "heard from somewhere" that TLS slots will never go higher than 63, and they simply believed it.

Maybe you can come up with your own theory. Share it in the comments.

Comments (35)
  1. Damien says:

    “If I want my program to be able to run on all possible systems, I mustn’t use more than 64 TLS slots” – I.e. they’re ignoring the fact that the system will implicitly tell them via a failed call when they hit the *actual* limit.

  2. ElectronShepherd says:

    “The constant TLS_MINIMUM_AVAILABLE defines the minimum number of TLS indices available in each process. This minimum is guaranteed to be at least 64 for all systems.”

    I would read that as “if you want to guarantee your program works everywhere, don’t use more than 64 slots, since it might not work”.

    Since most people writing software do want it to run everywhere, that effectively makes 64 the maximum.

    Even now, the documentation for TlsGetValue ( https://msdn.microsoft.com/en-us/library/windows/desktop/ms686812.aspx ) says

    “In particular, it succeeds if dwTlsIndex is in the range 0 through (TLS_MINIMUM_AVAILABLE– 1).”

    which implies it will fail with a dwTlsIndex of more than TLS_MINIMUM_AVAILABLE.

    1. Joel Schultz says:

      @ElectronShepherd

      “In particular, it succeeds if dwTlsIndex is in the range 0 through (TLS_MINIMUM_AVAILABLE– 1).”

      No conclusions can be drawn from that sentence about whether the function succeeds or fails when dwTlsIndex >= TLS_MINIMUM_AVAILABLE. All that sentence says says is that you get an automatic “success” if it is less than that.

    2. You are taking sentences out of context. The paragraph is talking about how TlsGetValue does minimal parameter validation. In particular, values in the range 0 .. (TLS_MINIMUM_AVAILABLE-1) are always considered as valid parameters, even if the index is invalid.

    3. Darran Rowe says:

      This is always one of the big problems when reading documentation quickly. It is easy enough to miss really subtle things, especially when you have some preconceived notion stuck in your mind.
      This is why I always tend to read the documentation multiple times and slowly to try and make sure that I have it right. I also don’t rely on just one set of documentation if I have multiple sources available. Like for example, it is stated elsewhere in the MSDN that the maximum TLS slots is actually 1088. https://msdn.microsoft.com/en-us/library/windows/desktop/ms686749(v=vs.85).aspx

  3. morlamweb says:

    I get the sense that the implementer of the TLS_MINIMUM_AVAILABLE constant tried to head off the app compat issues by the name of the constant itself: “minimum” implies that the value could change in some future version of the OS.

  4. French Guy says:

    Maybe the code did use TLS_MINIMUM_AVAILABLE and its value was 64 at the time it was compiled, on the machine it was compiled. This is, of course, the scenario that is most generous with the programmer.

    1. cheong00 says:

      I think it’s possible. if (tlsIndex < TLS_MINIMUM_AVAILABLE) { … } doesn't sound too bad to me.

      The developer might have expected the value in header be updated when compile symbol for target OS is changed (like when #define _WIN32_WINNT_WINXP is used)

      1. Darran Rowe says:

        That does sound bad to me though.
        The function does actually notify you when it fails. It tells you by returning TLS_OUT_OF_INDEXES if there are no more slots available. So instead of if (tlsindex < TLS_MINIMUM_AVAILABLE) you should be using if (tlsindex != TLS_OUT_OF_INDEXES). This has the benefit of working in all situations and is actually shorter than checking against minimum available.
        There is another problem though. you cannot get any reasonable information out of the value of the tls index itself. The documentation for TlsAlloc states "The value of the TLS index should be treated as an opaque value; do not assume that it is an index into a zero-based array." So from that I would draw the conclusion that you couldn't even guarantee the order that the indices come out is sequential. You couldn't even guarantee that the values passed out are in the range [0-64) or[0-1088).
        Of course, this is a more recent addition, but all it does is document that the form that the index takes is undocumented.

      2. Ivan K says:

        Well TLS_MINIMUM_AVALABLE is still defined as 0x40 in the “8.1” Windows SDK headers on my machine, so 64 will still be baked into that code on recompile.
        At any rate, there is a blog post I found from 2007 called “Thread Local Storage, part 2: Explicit TLS” which includes the number 64 in what seems to be an example (or maybe hand-decompiled?) implementation of TlsGetValue() – I’m not sure because I only spent a couple minutes googling and wondering about reasons. I won’t bother posting the url, but a search should find it.

        1. cheong00 says:

          Well, I said “The developer might have expected…” doesn’t mean it really happened.

          And btw, what’s the use of including it in the header files if it’s not really updated? Is there any place that this constant has any real use for developers?

          1. Ivan K says:

            Yeah sorry about my choice of wording. I tried to combine two answers in one reply. And you’re right of course. I think most of that was my surprise that it hadn’t changed. Anyway, I think the headers were processor specific so it makes sense to expect that it might change.

  5. Joshua says:

    There’s only one thing I’m not getting. The DLL appears to have died in the application’s own installer (notice how it isn’t a MSI installer); what could have changed so that it’s using up so many TLS slots now?

    1. Perhaps Windows started using one or more slots? Or if the program was linked against mscvrt.dll (which it shouldn’t be, in 2007, but it wouldn’t surprise me) that might have started using some?

  6. Glassware says:

    Raymond – about your comment – “I’m not entirely convinced of this theory because the maximum number of TLS slots increased in Windows 2000, but the program in question was released in 2007.”

    That’s very true, but remember, in normal operation this bug is harmless. If someone wrote a shared code library looking for TLS slots, and it interpreted a number > 64 as a failure, any code that used it would generally work just fine. The only error was a false negative – the code thinks the use case where TlsAlloc returns a value greater than 64 was a failure, but it was a success. So if a program written post-2000 used this code, it would just sometimes think they had run out of TLS slots.

    The only problem only caused an app compatibility crash when there were multiple failures – code using the > 64 assumption, plus other code that assumed that the first batch of code always succeeded.

    Next question: What was the resolution to this? Did you add a compatibility shim in TlsAlloc for this app to always return 0-63, or was there a more clever solution that tricked the code into accepting numbers > 64?

    1. The f_314272e0 function allocates a TLS slot, tries to initialize it, and when initialization fails, it forgets to free the TLS slot. It wasn’t shown in the article, but the calling code checks whether TLS slot was allocated (by performing the erroneous comparison against zero). Therefore, the bug is all in the helper library, where its error handling code was apparently never tested.

      1. Glassware says:

        I’m impressed – that’s an amazing amount of debugging. I once had to do a 6809 disassembly project many decades ago and I recall it took me a month to figure out the purposes behind the various bits of code. May I ask how long this overall research took?

  7. Myria says:

    Is the limit on the number of loaded modules (.exe, .dlls) that use __declspec(thread) still 64, or is that limit gone now, too?

  8. Alex Cohn says:

    It’s probably a coincedence, but the article http://www.nynaeve.net/?p=181, also dated 2007, remembers an (outdated by then) limit of 64 TLS slots:

    > The TlsSlots array in the TEB is a part of every thread, which gives each thread a guaranteed set of 64 thread local storage indexes. Later on, Microsoft decided that 64 was not enough TLS slots to go around and added the TlsExpansionSlots array, for an additional 1024 TLS slots. The TlsExpansionSlots array is demand-allocated in TlsAlloc if the initial set of 64 slots is exceeded.

  9. Perhaps elsewhere in the program (or in other programs used by the same library) they were using a 5-bit field to store the indexes?

    1. Uh, make that a 6-bit field. D’oh!

  10. xcomcmdr says:

    I don’t know what happened, but at least all the recent posts are harder to read thanks to some obsolete 80 colums limit applied to the entire text.

    It wasn’t like that yesterday.

    Can it be fixed, pretty please ?

    1. Ray Koopa says:

      And I thought,
      due to those many linebreaks,
      that Raymond decided,
      in a moment of great ideas,
      to write poems,
      rather than blog articles.

      1. Yuri Khan says:

        I think
        something has changed
        in the source-to-HTML formatter.
        Where the old one removed line breaks,
        the new one inserts a hard line break.

        It’s surprising
        how not all blog software
        has embraced Markdown yet.

        1. Klimax says:

          Because not everybody likes Markdown.

        2. Karellen says:

          Because raw HTML is not hard to learn or write, has only one (extended over the years, but backwards-compatible) syntax, and is generally more useful to know.

          I’ve never seen the point of Markdown. Why not just use HTML? It really is not that complicated, especially if you start simple with, e.g. , and extend from there.

          1. Karellen says:

            Sorry, that was meant to read “<em>is not hard to learn or write</em>” and “…with, e.g. <h1>, and…”

            (Last time I posted a comment where it was an issue, “<” was auto-escaped and worked as a less-than sign. Looks like you can now post (some) HTML in your comments…)

          2. Yuri Khan says:

            You illustrate the point perfectly.

            I’d be all for HTML if (1) it worked consistently everywhere, (2) everyone would bother to learn it, and (3) each HTML comment were validated. However, it doesn’t, they don’t, and they aren’t.

            Some blogs accept a limited subset of HTML. Some accept only plain text. Some accept a subset of HTML but also turn newlines into <br>s. Some (like this one) change the rules during upgrades, without bothering to convert or special-case the existing content.

            Markdown is not ideal, but it has sensible newline handling, and it is simple enough that most ordinary people won’t botch it.

      2. Klimax says:

        Interestingly, article in RSS feed is formatted correctly.

  11. What’s with all those manual line breaks?

    1. John Elliott says:

      It does make the articles read a little like The Story Of Mel.

      1. Ray Koopa says:

        It sure has an artistic touch!

        1. Raymond can’t do anything wrong in the fans’ eyes, eh? Not that I am complaining. I myself was a fan of Mark R. until he joined Microsoft and became a novelist, writing novels that are more boring than a Windows Installer log.

          1. Darran Rowe says:

            It is because he didn’t do anything here.
            When the post was first posted (Monday 13th June), it displayed fine. It was only later (I noticed it myself today, Tuesday 14th June) that the post was messed up. Today’s post is messed up too. If you read the comments for that, he did mention that the blog software was adding in extra tags which was messing up the formatting.
            If there is any potential for “doing something wrong” here, you could see it as posting on the MSDN blogs?

          2. xcomcmdr says:

            > Raymond can’t do anything wrong in the fans’ eyes, eh?

            Well, so far the only PITA around here is you.

Comments are closed.

Skip to main content