It's only temporary

NT has a whole lot of really cool features that aren’t always obvious without REALLY looking closely at the documentation.

One of my favorite is what I call “temporary” temporary files.

A “temporary” temporary file is one whose storage is never written to disk (in the absence of memory pressure).  It behaves just like a file does (because it’s a file) but the cache manager disables the lazy writer on the file, and the filesystem knows to never commit the pages containing the file’s metadata to disk.

To create a “temporary” file, you call CreateFile specifying FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE in the dwFlagsAndAttributes attribute.  This combination of bits acts as a hint to the filesystem that the file data should never be flushed to disk.  In other words, such a file can be created, written to, and read from without the system ever touching the disk.

Why would you ever want such a file?

Well, consider the case of a web server that’s creating content for client applications.  In this case, it’s possible that halfway through rendering the content to be transmitted to the client, you encounter an error.  The problem is that you’ve already sent the 200 OK response to the client, so the client thinks there are no errors.  To fix this, you can render the content to a “temporary” temporary file, and if there is an error rendering the content, you can response with an appropriate error code if the rendering fails.  If the rendering succeeds, you can use TransmitFile (if your server is written to raw sockets) or HttpSendHttpResponse (if your server is written to the HTTP API) to send the response data directly from the file.

There are lots of other reasons for using this mechanism – for example, Exchange 5.5’s POP, IMAP and NNTP servers used this technique to render RFC822 message content.  The Exchange server would render the email message to a temporary file, and then use TransmitFile to send the response to the client.  We used this for two reasons – we wanted to be able to respond to render failures above, but also because we needed to deal with potentially huge email messages.

Which is the other huge advantage of “temporary” temporary files over in-memory buffers - their size is bounded by the available disk space, NOT by available RAM.  So while an in-memory render might fail on a 1G file (because you couldn’t allocate a gigabyte of contiguous memory), it will work just fine to a “temporary” temporary file (assuming you have a gigabyte of disk space free on the rendering drive).  If you exceed available memory, the memory manager will flush the file data to disk. This causes a performance hit, but your operation will succeed instead of failing.