An In-depth look at memory - SQL Server 2012/2014

I finally had some time on my hands, so thought to try and get around to blogging about the memory architecture of SQL server 2012/2014. The memory architecture for the SQL Server relational database engine was practically overhauled with SQL Server 2012, and most of it has remained the same in SQL 2014. Please note that the In-memory OLTP feature introduced in SQL 2014 has a different engine dedicated to it, which will not be covered as part of this blog post.

Memory Manager

We have a new memory manager in SQL 2012. The new memory manager is responsible for almost all the memory management related activities, especially memory allocation. It includes 2 allocators, the Page Allocator, and the Virtual Address Space (VAS) allocator.

Memory consumers request memory from the memory manager through these allocators. The new memory manager supports allocations of all sizes, we no longer have the concept of single page and multipage allocators. For the sake of simplicity, I will refer to the allocator as "Any Size Page allocator" in this post. Consumers like memory clerks and the Buffer Pool are clients of the memory manager. This is a major change from previous versions, where the Buffer pool was a consumer as well as a provider(allocator) of memory (single pages).

Another major change that you will experience is that with the introduction of the Any Size Page allocator, the SQL Server "Max Server Memory" setting now controls the memory allocated for the Buffer Pool, CLR, as well as larger than 8 kb (single page) allocations. In prior versions, max server memory only controlled the size of the buffer pool, which made it difficult for DBA's/architects to apportion memory between applications/instances on shared servers. SQL Server 2012 gives us the ability to establish much tighter control on memory allocation using tools such as resource governor, because the resource governor is now able to control all page allocations.

The buffer pool in SQL 2012 does not contain any memory management functionality. It just manages the caching of database pages, and is now treated like a regular, external cache. Considering that the Buffer Pool is still likely to be the largest consumer of memory though, it is allowed to use almost all free memory from the SQL memory manager.

There are several visual representations of the changes to memory manager in SQL 2012 floating around on the internet, which I've included for your benefit:

As you might have noticed, memory allocation for CLR forms a bit of a "special case", in that even though it uses VAS Allocator(Virtual Allocator in the diagram), it still is governed by the limits set using the "max server memory" setting in SQL. An auxiliary effect of this is that CLR is now initialized at SQL Server DB Engine startup, as opposed to being initialized when it was called in earlier versions of SQL Server.

The memory manager architecture in SQL Server 2012 can be broken down into the following components:

1. Fragments/Fragment Manager: A fragment is a large region of VAS. The fragment manager deals with fragments, which the memory manager will commit dynamically.
2. Top level block allocator: The Top level block allocator takes these large fragments and splits them up into top level blocks. These top level block sizes are fixed at 16 MB in 64-bit installations and 4 MB in 32-bit installations.
3. Workspace: A Workspace is a set of top level blocks. A workspace contains allocations of similar lifetime. Examples of workspaces include buffer pool, memory objects, etc.
4. Fixed Size Allocator: The most crucial component of the new memory manager hierarchy is the Fixed Size Allocator. This allocator is part of the workspace, and it gets memory from the top level block allocator, and breaks it up into smaller sized blocks. These are called parent blocks, and each parent block has its own descriptor, which has a free list as well as the current state of the block. The parent block can be in one of four states:

• Active - The block from which allocations happen. New allocations requests are serviced from the Active block of the local CPU or the NUMA node.
• Full - The block is full of allocations. There is no more free space in that block. This was previously an Active block.
• Partial - The block is neither Full nor Active. There are multiple lists of Partial blocks. These lists are per NUMA node. A block is made part of a specific list according to the fill factor of allocations inside the block. This supports the mechanism to try to Free blocks which have less allocations sooner or try to allocate from blocks which have more allocation already to make it full. Thus, the overall algorithm pushes against having a large list of Partial blocks to manage.
• Empty - When the block is allocated for the first time, it is in empty state. It is does not have any allocations.

Memory Allocation

Let's talk about how memory is allocated using the new memory manager. The first step is, of course, to determine which allocator to use, based on the amount of memory requested. This involves traversing the workspace hierarchy to find the appropriate allocator. If we're unable to find an allocator of the exact same size, the next highest size allocator is used.

Once we have the allocator, the fastest way to allocate is to look at the local partition, find the Active block and then pop an entry from the Free list. If the Active block does not have enough free space to allocate from, then we need to replace this Active block with one that has enough free space.

Next, we start traversing the partial buckets, looking for a partial block that is almost full. If a partial block is found, we first re-evaluate the fill factor, since many free operations could have happened after this partial block was inserted into this fill factor bucket. If the fill factor has changed, then this partial block is inserted to the appropriate fill factor bucket. Repeat this process to find a partial block with the most allocations in it. Once found this partial block is installed as an Active Block. After this the allocation is performed from this Active block.

During the above process, if another thread installs another active block, then that Active block should be used for allocation instead. If allocation is successful from that active block, then this block should be returned to the partial bucket where it came from.

If no partial block was found, then go up to the parent block allocator and request a new block [empty] to be created. This corresponds to the process of growing against the target memory configured [for conventional and locked page memory].

If during the above process we were not able to find a suitable block, then we switch to the next NUMA node and repeat the same process. Switching to next node happens once the target for current node is reached. If all nodes reach target memory then an OOM (Out Of Memory) error is returned.

In order to free a block, we first find the parent block descriptor for the current block that is being freed. Next, we return the block to the free list and update the state in the descriptor. It can go to two states, Partial or Empty.

When we configure SQL to use large pages, all the top level blocks are committed at startup. For conventional and locked page memory model, the blocks are committed as needed.

The rest of the components work in pretty much the same way as earlier versions of SQL Server. For more details on the memory architecture up to SQL Server 2008 R2, please refer the following posts:

An in-depth look at SQL Server Memory–Part 1

An in-depth look at SQL Server Memory–Part 2

An in-depth look at SQL Server Memory–Part 3

As always, comments and feedback are appreciated.

Disclaimer: The information in this weblog is provided “AS IS” with no warranties, and confers no rights. This weblog does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my opinion. Feel free to challenge me, disagree with me, or tell me I’m completely nuts in the comments section of each blog entry, but I reserve the right to delete any comment for any reason whatsoever (abusive, profane, rude, or anonymous comments) – so keep it polite, please.

Technorati Tags: SQL,SQL Server 2012,SQL Server 2014,SQL 2012,SQL 2014,Memory,Memory architecture,in-depth memory,SQL Server engine,workspaces,any size allocator,fragment manager,Top level block allocator,SQL 2012 memory mangement,SQL 2014 memory management