Debugging walkthrough: Access violation on nonsense instruction, episode 3


A colleague of mine asked for help debugging a strange failure. Execution halted on what appeared to be a nonsense instruction.

eax=022b13a0 ebx=00000000 ecx=02570df4 edx=769f4544 esi=02570dec edi=05579748
eip=76c49131 esp=05cce038 ebp=05cce07c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
KERNELBASE!GetFileAttributesExW+0x2:
76c49131 ec              in      al,dx

This is clearly an invalid instruction. But observe that the offset is +2, which is normally the start of the function, because the first two bytes of Windows operating system functions are a mov edi, edi instruction. Therefore, the function is corrupted. Lets look back two bytes to see if it gives any clues.

0:006> u 76c49131-2
KERNELBASE!GetFileAttributesExW:
76c4912f e95aecebf3      jmp     IoLog!Mine_GetFileAttributesExW (6ab07d8e)

Oh look, somebody is doing API patching (already unsupported) and they did a bad job. They tried to patch code while a thread was in the middle of executing it, resulting in a garbage instruction.

This is a bug in IoLog. The great thing about API patching is that when you screw up, it looks like an OS bug. That way, nobody ever files bugs against you!

(In this case, IoLog is a diagnostic tool which is logging file I/O performed by an application which is being instrumented.)

My colleague replied, "Thanks. Looks like a missing lock in IoLog. It doesn't surprise me that API patching isn't supported..."

Comments (23)
  1. NotThisMind says:

    Sorry for 'spamming', for a non-assembly programmer and mostly non-debugger, how can you tell that's API patching? I know i could google, but if someone is kind to explain here...

  2. skSdnW says:

    @NotThisMind: It is jumping from a low-level usermode dll to a function in another dll that is not a part of Windows. For file APIs it generally goes kernel32 -> kernelbase -> ntdll -> kernelmode, this is just something you learn over time by debugging different issues.

  3. Pete says:

    My guess is because the function the crash happens in is called GetFileAttributesEx and the first instruction is

    jmp     IoLog!Mine_GetFileAttributesExW

    (Note the names: GetFileAttributesEx jumping to Mine_GetFileAttributesExW)

    Like Raymond said, normally the first two bytes of a Windows function are a mov edi, edi instruction, but in this case it's a jump to a different function with a "Mine_" name. Looks like IoLog overwrote the function with a jump to their own version.

  4. genumi says:

    I only know enough to be dangerous, and I had the opposite problem:

    I guessed someone was patching, but only after Raymond said the instruction was 'clearly' invalid. That isn't clear to me (see above knowledge assessment)

    Couldn't it possibly be someone trying something very clever?

  5. Joshua says:

    @genumi: You can't access IO ports from usermode. Besides, it's a system call wrapper. It's gonna call another function so the first thing it needs to do is set up its stack frame.

  6. Ben Voigt says:

    @genumi, the instruction is clearly invalid because it is performing direct hardware access, and in Windows, direct hardware access should only happen in kernel mode (besides the fact that this particular function has no business doing hardware access, which it should be delegating to the storage stack, in order to support all manner of different filesystems and busses, including network shares).

  7. Jeff says:

    You mean using an invalid instruction and catching the resulting exception to do something funky? I really don't see the point in doing that. I don't know how you would inject the exception frame in every application that would indirectly execute that invalid instruction.

  8. rsola says:

    I was a bit confused by the title "Access violation on nonsense instruction", until I realized it belongs to a series of articles. Attempting to execute IN and OUT instructions in user mode will raise a STATUS_PRIVILEGED_INSTRUCTION exception, unless a driver messes with the I/O permission bitmap (x86 specific, 32-bit only).

  9. Kemp says:

    I am having great difficulty parsing this sentence: "But observe that the offset is +2, which is normally the start of the function, because the first two bytes of Windows operating system functions are a mov edi, edi instruction". I can't figure out what the "because" is referring to ("[x is happening/we have made deduction x] because the first two bytes of [...]"), it doesn't seem to follow from the start of the sentence. Also, possibly due to my lack of debugger experience, I can't see where the +2 offset is shown here (maybe that's causing my parsing error?).

    Anyone able to help out a noob?

  10. Sean Liming says:

    If I am reading this correctly, using windbg one can go back two bytes using the u command to see what really caused the error. The error is from an application called IoLog. - Is that correct?

  11. Katie says:

    @Kemp

    KERNELBASE!GetFileAttributesExW+0x2 shows that it is 2 bytes into the GetFileAttributesExW function. This is normally the real start of the function, because the first 2 bytes are taken up by "mov edi,edi" for reasons explained in the linked article. Because the offset is +2, it is fairly safe to assume that you're looking at an actual instruction in the function, not some constant value or anything else where you might expect it to be an invalid instruction.

  12. Gabe says:

    I don't understand "Looks like a missing lock in IoLog". How could locking have helped? Surely no lock could have prevented a thread from executing GetFileAttributesExW while IoLog was doing its patching, unless IoLog did it at process startup while the LoaderLock was held (meaning you'd have to link against IoLog rather than loading it at runtime).

    Isn't the correct thing to do write the JMP Mine_GetFileAttributesExW at GetFileAttributesExW-5 and then write JMP $-5 at GetFileAttributesExW+0?

  13. Katie says:

    @Sean Liming

    I thought your name looked familiar - I used your books at my previous employer while porting an embedded system from XP to WES7.

    "u" is for unassemble. You can see he specifically gives the address where it broke with a -2 to go back two bytes, which he knows is the beginning of the function. Unassembling from there he saw that the "ec" byte that was interpreted as a bad instruction was actually the third byte of a 5-byte instruction encoding a jump to a function in IoLog. The linked article gives a good explanation of how it should be done. But it still isn't supported. :-P

  14. Joshua says:

    @Gabe: As far as correct can be defined for this procedure that is how it's done.

  15. Henke37 says:

    The correct solution is explained in the linked article: put the real jump in the designated hotpatch area and replace the leading nop with a short jump.

  16. Kemp says:

    @Katie

    Thanks for that, the text makes a lot more sense now. Somehow I missed the +2 in the one place it actually made sense to look :)

  17. Sean Liming says:

    @Katie

    I hope the books helped

    Thanks I was having a hard time understanding the article. Is this a tool in development?

  18. boogaloo says:

    "I don't understand "Looks like a missing lock in IoLog". How could locking have helped?"

    @Gabe It can't, his colleague didn't understand and Raymond either didn't continue the story with how he explained it to him or he didn't explain it to him at the time and he has just now pointed him to this article.

  19. FCotrina says:

    oh, I just wish that Windows on ARM had the same possibility of API patching. DLLs and EXEs in Windows Phone are just too tightly packed, leaving no space for any Detour-ing.

    Would anyone happen to know the reason? I mean, this "move edi, edi" is added by the x86 compiler, and it's absent in the ARM compiler ?

    Given that BigWindows and WoA are using same sources for the kernel and OS, it wouldn't be extra effort, I believe.

  20. Chris says:

    @FCotrina  what environment are you using that you could actually patch the API in Windows on ARM?

  21. boogaloo says:

    @FCotrina It just makes detour-ing more complex, relocating the original code to new ram.

  22. voo says:

    @FCotrina Read the earlier article where Raymond spends half of his time explaining that detouring OS functions is an OS feature. Meaning: Whether Windows on ARM or anywhere else uses or doesn't use detour shouldn't bother you at all, because you should not be doing anything with those points at all.

    So why did the Windows on ARM guys decide that this internal feature wasn't worth it? Probably because the only legitimate reason to use it is never used in the first place (MS has't hotpatched the kernel for a long time) and it's now just another backwards compatibility constraint on Intel Windows because some programmers ignored the whole "for internal use only" part and just went ahead with it anyhow.

    Really a win-win situation for them: Less effort and less things that people can screw up for them.

  23. Bruce Dawson says:

    Note that you can detect image patching with the windbg !chkimg command. It compares the memory saved in a crash dump to the bytes that should be there (as retrieved from a symbol server). This only works, of course, if the patched memory is in the crash dump. Luckily most crash dumps save 256 bytes of memory around EIP (either on all threads or just the crashing thread, I'm not sure) so if the crash happens near where the patching occurred, !chkimg will detect it.

    If the crash happens in a child function (due to the patching sending execution down a bad path) then !chkimg will not help, and in fact windbg will show the code bytes from symbol server, which makes things look really mysterious.

Comments are closed.

Skip to main content