Identifying Global Atom Table Leaks

Hi, it's the Debug Ninja back again with another debugging adventure.  Recently I have encountered several instances where processes fail to initialize, and a review of available resources showed that there was no obvious resource exhaustion.  A more in depth review found that there were no available string atoms in the global atom table.

 

Global atoms are organized on a per-session basis.  If atoms cannot be allocated in session 0, services may fail to start or processes launched by various services may fail to start.  However, a user logged in to a different session will not experience any such failures.

 

String atoms are numbered from 0xC000 through 0xFFFF, providing a maximum of 0x4000 atoms per session.  For more information on atoms, and atom tables, see https://technet.microsoft.com/en-us/query/ms649053.

 

When there are no more string atoms available, calls to APIs that allocate string atoms will fail.  Because atoms are often allocated at process or dll init time, the most common symptom is that processes fail to initialize.  The process may cleanly exit without an error.  You are likely experiencing this problem if you debug your application and find that the failure originates from an API that allocates string atoms such as RegisterClass, RegisterClassEx, GlobalAddAtom, or AddAtom.

 

To determine if the global string atom table is full you will need to perform a kernel debug.  This can be a live debug or a post-mortem debug using a dump.

 

First identify the session where the failures have occurred and set the process context to a process in this session.  In my example, w3wp.exe was launching a process and this process failed to initialize.

 

2: kd> !process 0 0 w3wp.exe

PROCESS fffffa8005083060

    SessionId: 0  Cid: 1668    Peb: fffdf000  ParentCid: 08ec

    DirBase: 8a2df000  ObjectTable: fffff8a0128bbe40  HandleCount: 441.

    Image: w3wp.exe

2: kd> .process /p /r fffffa8005083060

Implicit process is now fffffa80`05083060

Loading User Symbols

.....

 

Next we need to analyze the global atom table.  The pointer to the table is stored in the UserAtomTableHandle global.

 

2: kd> dq win32k!UserAtomTableHandle l1

fffff960`003bf7a8  fffff8a0`05e5bc70

 

The UserAtomTableHandle has a pointer to a handle table at offset 0x10 in 64-bit, and offset 0x8 in 32-bit.  Note that although the atom table is defined as a _RTL_ATOM_TABLE, the format shown by dt is for user mode and does not apply to the UserAtomTableHandle in kernel mode.

 

2: kd> dq fffff8a0`05e5bc70+10 l1

fffff8a0`05e5bc80  fffff8a0`05db7740

2: kd> dt nt!_HANDLE_TABLE fffff8a0`05db7740

   +0x000 TableCode        : 0xfffff8a0`109c8001

   +0x008 QuotaProcess     : (null)

   +0x010 UniqueProcessId  : 0x00000000`00000184 Void

   +0x018 HandleLock       : _EX_PUSH_LOCK

   +0x020 HandleTableList  : _LIST_ENTRY [ 0xfffff8a0`05db7760 - 0xfffff8a0`05db7760 ]

   +0x030 HandleContentionEvent : _EX_PUSH_LOCK

   +0x038 DebugInfo        : (null)

   +0x040 ExtraInfoPages   : 0n0

   +0x044 Flags            : 0

   +0x044 StrictFIFO       : 0y0

   +0x048 FirstFreeHandle  : 0x10004

   +0x050 LastFreeHandleEntry : 0xfffff8a0`10ca4ff0 _HANDLE_TABLE_ENTRY

   +0x058 HandleCount      : 0x3fc0

   +0x05c NextHandleNeedingPool : 0x10400

   +0x060 HandleCountHighWatermark : 0x3fc1

 

The FirstFreeHandle contains the handle number that will be given to the next handle allocated from this table.  This value is encoded, to get the next handle number we need to right shift the FirstFreeHandle by 2 bits.

 

2: kd> ?00010004>>2

Evaluate expression: 16385 = 00000000`00004001

 

The result from above, 0x4001, is greater than the number of possible string atoms.  As I mentioned earlier, there is a limit of 0x4000 string atoms.  Now we know that the session is out of string atoms.

 

The next step is to dump the string atoms to identify whether there is an observable pattern in the leaked strings.  The !atom command only works in user mode, so we need to dump the kernel mode strings manually.  An atom table is comprised of multiple buckets.   Each bucket is the head of a list of atoms.  The buckets start at offset 0x20 in the atom table in 64-bit, and offset 0x10 in 32-bit.

 

2: kd> dq fffff8a0`05e5bc70+20

fffff8a0`05e5bc90  fffff8a0`05e5ba60 fffff8a0`05db7be0

fffff8a0`05e5bca0  fffff8a0`08cf1770 fffff8a0`05e5b3d0

fffff8a0`05e5bcb0  fffff8a0`05ea9020 fffff8a0`05e5b8e0

fffff8a0`05e5bcc0  fffff8a0`05ea9b10 fffff8a0`05ea9910

fffff8a0`05e5bcd0  fffff8a0`05ea9f00 fffff8a0`05e5b650

fffff8a0`05e5bce0  fffff8a0`05cda290 fffff8a0`05ea9e80

fffff8a0`05e5bcf0  fffff8a0`05e5b200 fffff8a0`05ea9e30

fffff8a0`05e5bd00  fffff8a0`05e5b7e0 fffff8a0`06c56210

2: kd> dq

fffff8a0`05e5bd10  fffff8a0`06d6b5a0 fffff8a0`05ea9d50

fffff8a0`05e5bd20  fffff8a0`05e5b790 fffff8a0`05e5b9d0

fffff8a0`05e5bd30  fffff8a0`06bd9bc0 fffff8a0`05ea9c90

fffff8a0`05e5bd40  fffff8a0`05e5b0c0 fffff8a0`06ae2020

fffff8a0`05e5bd50  fffff8a0`05e5b930 fffff8a0`04d2af40

fffff8a0`05e5bd60  fffff8a0`05e5b690 fffff8a0`05e5b980

fffff8a0`05e5bd70  fffff8a0`05e5b490 fffff8a0`05e5b410

fffff8a0`05e5bd80  fffff8a0`05e5ba20 fffff8a0`05e5b4f0

2: kd> dq

fffff8a0`05e5bd90  fffff8a0`05e5baa0 fffff8a0`05e5b390

fffff8a0`05e5bda0  fffff8a0`05e5b840 fffff8a0`05ea9c50

fffff8a0`05e5bdb0  fffff8a0`05e5b250 00000000`00000000

fffff8a0`05e5bdc0  00000000`00000000 00000000`00000000

fffff8a0`05e5bdd0  00000000`00000000 00000000`00000000

fffff8a0`05e5bde0  00000000`00000000 00000000`00000000

fffff8a0`05e5bdf0  00000000`00000000 00000000`00000000

fffff8a0`05e5be00  00000000`00000000 00000000`00000000

 

The quick and dirty way to dump the buckets is with !list.  I am sure that some will say it is tedious to dump each bucket list by hand and that there are easier ways to accomplish this.  To prevent this article from becoming a lesson on debugger scripting, I am leaving that as an exercise to the reader.

 

2: kd> !list "-t nt!_RTL_ATOM_TABLE_ENTRY.HashLink -e -x \"du @$extret+10\" fffff8a0`05e5ba60"

du @$extret+10

fffff8a0`05e5ba70  "Native"

 

<snip strings that don't match a pattern>

 

du @$extret+10

fffff8a0`0838a120  "ControlOfs0210000000000700"

 

du @$extret+10

fffff8a0`0f7ff430  "ControlOfs021A000000000C30"

 

du @$extret+10

fffff8a0`162168c0  "ControlOfs020E000000001774"

 

du @$extret+10

fffff8a0`08c33870  "ControlOfs01F70000000007F4"

 

du @$extret+10

fffff8a0`07c46910  "ControlOfs0202000000000BF8"

 

du @$extret+10

fffff8a0`062aab50  "ControlOfs01F5000000001274"

 

du @$extret+10

fffff8a0`0777b150  "ControlOfs0202000000000C80"

 

du @$extret+10

fffff8a0`07dd3410  "ControlOfs0207000000000F00"

 

du @$extret+10

fffff8a0`0f01d190  "ControlOfs0214000000000DAC"

 

Dumping the atoms I found that there is a continuous pattern of the string ControlOfs followed by 16 hexadecimal numbers.  Some time spent with your favorite search engine should find other reports of atom leaks involving the string ControlOfs, and that these leaks have been identified as a problem in some specific software.  In this instance the programmer using that software needs to change their application to avoid the problem.