How dangerous are kernel mode rootkits?

This post was originally going to be a comment in response to rootbeer's question in a comment to my last blog entry, but it became such a long reply that I decided to make it a post.  The question was:

“How dangerous are kernel-level rootkits today?”

I’m not sure how much you know about rootkits, but I’ll assume you know at least the ideas behind how they work.  If not, there are plenty of resources that explain how they work online.  Basically a rootkit is the name for a mechanism used to try to hide the existence of code, files, processes, etc. from the rest of the system.  Technically the code hidden inside a rootkit may not have been written to be malicious (in fact it may try to provide a service to the user).  However, most people in the operating systems community feel that rootkit methods should never be used.  The reason for this is that by hiding files, etc. rootkits negatively impact a user’s ability to control and manage their own system.  From now on when I talk about rootkits I will be discussing rootkits that hide malicious code.

Rootkits can operate in user mode or kernel mode; more recent and more powerful rootkits operate in kernel mode where they are completely able to hide themselves from user mode.  As viruses have become more sophisticated over the years anti-viruses have had to become more intelligent to detect and destroy viruses.  The same thing is happening currently with rootkits.  As rootkits get better and better at hiding themselves from the system anti-rootkits have to adapt to these changes in the rootkits they try to detect.

The good news is that (as far as I know) as of now no one has come up with a way to completely hide a rootkit from the system.  In trying to hide themselves, rootkits have to interpose at some layer within the operating system so that they can hide whatever they are trying to hide from the rest of the system.  The problem (for rootkit writers) is that it is extremely, extremely difficult to fully hide their interposition mechanism from the system.

For instance, earlier kernel mode rootkits hid system resources by hooking the system call table.  This allowed them to hide files from the rest of the system, but all it took to discover the rootkit was looking at the system call table for abnormal entries (its not quite this simple because some drivers, like the one with regmon, hooks the system service table for completely different reasons).  The latest kernel mode rootkits (that I am aware of) interpose on the page fault handler so that they can hide the malicious code unless it is trying to be executed.  However, this requires them to fail access to their code in nonpaged pool.  This is something that should never occur (think page fault in nonpaged pool blue screens) and hence it can be detected.  Also, so far they have not been able to hide their interposed page fault handler and so that can be detected also.

After that long winded discussion there are basically two things to remember.  First of all, a kernel mode rootkit only gets installed if it is able to find an existing vulnerability to take advantage of (which there are).  And second, up until now, and in the foreseeable future, rootkits will always have some component that can be detected.

Just like viruses and all code exploits, the race will be between hackers trying to find more sophisticated ways to infiltrate the system and hide their code and the good guys finding ways to detect and remove rootkits.  I don’t think the situation is hopeless and I have a suspicion that years from now we will discuss rootkits the same way we discuss viruses today.

I will end this by saying I am not on the security team and by no means am I a security expert.  So, if I got something wrong (it is 4 am after all), something needs to be clarified, or you want to add something to this discussion feel free to leave a comment.