Deftly solving compatibility problems by withholding information


One of the continuing compatibility problems that plagued Direct3D was the way it reported texture formats. Historically, the way an application checked which texture formats were available was by calling Enum­Z­Buffer­Formats and passing a callback function which is called once for each supported format. The application's callback made some sort of decision based on the information it received. The problem was that any time a new format was added, a bunch of programs ended up not working. Either the new format confused them, or the change in the order of the formats violated some assumption. For example, they may have assumed that if a video card supports R8G8B8 format, then it will always be the first one in the list.

For a time, the compatibility strategy was to try to detect what flavor of Direct3D the application was designed for and manipulating the list of supported formats in a way to keep that application happy, say by ordering the formats in a particular way or suppressing some formats from the list.

In Direct3D 8, a new direction was taken: Ask, don't tell.

Instead of the application being told what formats are available, the application asks, "Do you support format X?" and Direct3D answers "Yes" or "No." This solves both types of problems: Applications never saw a format they didn't expect, because if they didn't expect it, they would never ask for it in the first place. And applications always saw the supported formats in the order they requested, because the application chose what order they asked for them.

The main casualties of the new design were diagnostic programs which listed technical details of your video card. They no longer were able to get a list of all supported formats; instead, they had to have a table of all the formats and ask for them one at a time.

There was one company that objected to this new design because they wanted their program to support all texture formats, even the ones that didn't exist at the time the program was written. This is just another variation of Sure, we do that. "Oh look, this video card supports pixel format 826. I'm going to use it! Just one question, though. What's pixel format 826?"

They broke their problem into two parts and were asking for help with the first part, unaware that even if the managed to solve that part, they were stuck with the impossible second part!

Comments (21)
  1. Joshua Ganes says:

    This solution sounds like a very reasonable strategy. Any new format will require some programming effort to handle. Telling an application about it is rather pointless. I suppose one possible scenario where the old interface may be useful would be in an application that can call home and look for automatic updates when it encounters an unknown format.

  2. Barbie says:

    Oh, there are potential use cases, like an end-user-provided texture that they would _somehow_ tag as the newfangled format (streaming blissfully through the game with no validation whatsoever). Not very common for sure. Really, if your game only supports the list of formats that were exposed at the time, that's still plenty (and your typical end-user has _no_ idea what a DXT3 is anyways)

    [… and of course that "tag" would be… a pixel format! So you could pass it to CheckDeviceFormat. In other words, "You can still do that." -Raymond]
  3. Silly says:

    Was going to say something like "game studios must want users to play <2011 game> in 2026 without any datedness or emulation", But more likely game studios just want their software to run now on all modern hardware despite what D3D reports… they hope to sort out the hardware themselves.

  4. Thorsten says:

    One could call the changed pattern IoIoC: Inversion-of-Inversion-of-Control.

  5. JM says:

    @Thorsten: by double negation, that's just "Control".

  6. Joshua says:

    I actually did run into a special case once where I could support image formats that I had never heard of before so long as GDI+ did. Depending on what their program was doing (e.g. copying buffers around and using DirectX functions to convert them) it might be able to support such things.

  7. 640k says:

    As long as the "format X" is a reasonable short integer, you can still try everyone.

    These problems are common in handshaking algorithms between hardware/apis and apps, when determining the best common format. Most developers get such logic wrong because:

    1. Long, long list which boils down to:

    2. They don't care.

  8. tobi says:

    Returning all available formats at once in a buffer and shuffling that buffer randomly would have solved some of the (forseeable!) compatibility problems.

    Analogy: NTFS has the same problem now. It can never stop returning sorted file names. It would have been better to randomly shuffle the first few items so applications could never have assumed sort order.

  9. Anonymous Coward says:

    It's also, perhaps, an instance of giving people the smallest amount of information they need to act upon. Give them extra irrelevant information, like the old system, and people will inevitable act on that instead.

  10. blah says:

    +1 to tobi. The D3D debug runtime takes care to scramble buffers and whatnot. It should have done the analogue here.

  11. dbacher says:

    There's also a performance advantage here, that's not immediately apparent.

    DX6-era video cards used fixed-function circuits built in to the chips for most of their texture processing.

    DX8-era video cards would have started to use programs to process the textures.  The change in the API call would enable them to load just those programs the application asked about, prior to receiving a texture, which would likely be an improvement in memory usage and performance on the video card, as well.

  12. alexcohn says:

    This reminds my the COM strategy: all you are guaranteed to have is IUnknown, and from there you start querying for the interfaces you understand.

  13. Tony Williams says:

    Alex: this is exactly why QueryInterface is the way it is.

    Joshua: your example is a kind of opaque pass-through where the intermediary doesn't know the format but both ends need to. Someone needs to do the checking, but without understanding the format itself. That's why we created interface IDs. There are methods in COM interfaces that do this kind of thing.

  14. 640k says:

    By enumerating CLSIDs in registry it should be easy to query for all supported interfaces.

    [Who says that every interface must be recorded in the registry? -Raymond]
  15. Gabe says:

    Tim: The real reason you would want to ask a COM object what interfaces it has (instead of calling QI with every one you know) is that you know lots more interfaces than an object is likely to have. If you know how to handle 1000 different interfaces, do you really want to QI each object 1000 times? No, it would be better to just ask each object which interfaces it can give you, and look up their GUIDs in a dictionary to see if you support them.

    Fortunately this scenario doesn't apply to texture formats because the author of a DX client will know what formats they have textures in and in what order they would prefer them.

    [Oh, you mean IMulti­QI::Query­Multiple­Interfaces? -Raymond]
  16. Tim says:

    @640k: Why go through the trouble of enumerating CLSIDs in the registry? You could just as well query for ALL possible interfaces (GUIDs). Either way you are left with a second, impossible to answer problem: What are you going to do with your shiny new void* (besides casting it to IUnknown* and calling Release() on it)?

  17. Tim says:

    Gabe, I suppose I wasn't clear enough in my previous post. The point I was trying to make is that if you ask for a random interface (which is what taking class IDs from the registry boils down to) there is nothing you can do with that interface.

  18. Gabe says:

    Tim: If you ask for a random interface (say, by enumerating CLSIDs in the registry), it is useless. Asking for an interface you don't understand is useful only for proxies (like what IMulti­QI::Query­Multiple­Interfaces does).

    The scenario I was contriving was one where you have a large number of interfaces that you understand and don't want to QI for them all. If you understand 1000 interfaces and 1000 objects, that's 1M calls to QI (or 1000 calls to Query­Multiple­Interfaces which results in 1M calls to QI). If you could enumerate the interfaces each object supported, you'd only have to make 1000 calls for 1000 objects.

    Of course, this is contrived and I can't think of an actual time this would be needed for COM. I was just trying to demonstrate that there are times when the old way (enumeration) is better than the new way (asking for just what you understand), even in situations like this where some getting back items that you don't understand would be useless.

    [Either way, you need to make 1M GUID comparisons. The comparison happens either in QI or in your own code that studies the results of GetInterfaces. And where are you going to put the results of your 1M comparisons? In a table of some sort, keyed by object and interface. In other words, you've reimplemented object->QI(interface) with TableLookup(object, interface). Not sure what you gained here. -Raymond]
  19. 640k says:

    [Who says that every interface must be recorded in the registry? -Raymond]

    COM.

    [Um, no. There are plenty of interfaces not in the registry. IRunnableTask ({85788d00-6807-11d0-b810-00c04fd706ec}), for example. -Raymond]
  20. 640k says:

    [Um, no. There are plenty of interfaces not in the registry. IRunnableTask ({85788d00-6807-11d0-b810-00c04fd706ec}), for example. -Raymond]

    Then it's not COM. It's "registry-less COM".

    [I will not get dragged into a game of no true Scotsman. -Raymond]
  21. 640k says:

    [I will not get dragged into a game of no true Scotsman. -Raymond]

    Please read the COM spec. It's more than vtables, you know.

Comments are closed.