Playsound is failing on Vista! What's wrong?

Recently BillP, the author of the antispyware application WinPatrol asked on the MSDN forums about a problem he was having with his application.

His app called PlaySound(MAKEINTRESOURCE(IDR_WOOF), hInst, SND_RESOURCE | SND_SYNC | SND_NOWAIT) but it was failing (returning false).

He was wondering if this might be a bug in Vista's playsound implementation - a reasonable assumption given that his application worked just fine on previous versions of Windows.

 

I knew that we were passing all the PlaySound regression tests, and there are a number of elements of Windows that use SND_RESOURCE (the pearl startup sound is one of them), so the SND_RESOURCE functionality wasn't broken.  I was puzzled for a bit and then I realized that there WAS one change to PlaySound in Vista (other than some general code cleanup, the addition of support for the SND_SENTRY and SND_SYSTEM flags and support for accessibility events on PlaySound).

For Vista, I tightened up the validation logic that's used when checking files before the PlaySound call.  Among other things, we check to make sure that:

a) The cbSize field in the "fmt " tag in the WAV file is less than 1K in length and
b) The WAVEFORMATEX in the "fmt " tag in the WAV file fits inside the "fmt " tag (done by checking that the waveformatex->cbSize+sizeof(WAVEFORMATEX) is less than the size of the "fmt " chunk[1]).

I downloaded BillP's app, and checked the resources in the file.  And in sure enough, the WAVEFORMATEX in the file had a length of 0x38 when it should have been 0.  Once I patched his binary to change the 0x38 to a 0, his application stared barking away.

At this point there are two options:  (1) BillP can fix his application to correct the corrupted resource or (2) Microsoft can change the PlaySound API to allow this kind of corruption (either to allow a bogus cbSize or to edit the app's WAVEFORMATEX to "fix" the bogus cbSize).  Changing PlaySound is not trivial at this point, and it's not clear what the right fix is - the error might be in the size of the "fmt " chunk, which means that the information in the cbSize field might be accurate.  In addition, the downstream audio rendering APIs are likely to choke on these malformed structures anyway. 

 

It's this kind of subtle breaking change that makes modifying any of the older Windows APIs such a nightmare. 

 

[1] We only check if the "fmt " chunk size is greater than sizeof(WAVEFORMAT) - if the "fmt " chunk is sizeof(WAVEFORMAT) than we assume that this structure is a WAVEFORMAT structure, which doesn't have a cbSize field.