SQLOS - unleashed

With SQL Server 2005 Beta 1 and Beta 2 you might have noticed that there is no ums.dll in the bin directory. The reason is simple it is no longer exists.  In SQL2000 ums.dll provided SQL Server with user mode non-preemptive scheduling. So what happened in latest version? Have we removed non-preemptive scheduling? The answer is no. Yukon continues relying on non-preemptive scheduling. As adequate research shows for RDBMS engine to meet performance and scalability requirements it needs to leverage non-preemptive scheduling.

 

In Yukon we took idea of non-preemptive scheduling further and have created new component, library, SQLOS, or sometimes referred as SOS. SQLOS is a user mode operating system. It includes OS pieces such as non preemptive scheduling, memory management, resource monitoring, exception handling, I/O, synchronization and hosting subsystems. Keep in mind SQLOS doesn’t provide OS abstraction layer to SQL Server. It doesn’t warp any of Windows APIs for portability purposes. Contrary it continues further binding of SQL Server to Windows by exploiting scalability and performance features of the Operating System.

 

The goals of SQLOS include bringing all system components together, enabling further innovation of SQL Server’s scalability and performance, providing manageability and supportability features. It exposes cohesive API to developers so that they can easily exploit features of hardware and the operating system.  SQLOS models its internals to simplify developing process as for high and as for low end platforms.

 

SOS’s scheduling subsystem consists of scheduling nodes, schedulers, tasks, workers and system threads. A scheduling node provides an abstraction layer over group of CPUs. On NUMA systems scheduling node represents a single NUMA node. On SMP box there is one scheduling node. It is possible to change a number of nodes by editing Windows registry. We call such configuration soft NUMA.  As you would expect a node also is a collection of schedulers. A scheduler is  anabstraction over a CPU. Schedulers deal with tasks that run by workers that get executed by system threads.  Description of SQLOS scheduling could take pages and pages so I will cover it in depth in separate post.

 

SQLOS’s memory management consists of memory nodes, memory clerks, caches, pools, and memory objects. A memory node is an abstraction over the memory attached to a scheduling node. A scheduling node is a proper subset of memory node. Separating scheduling and memory nodes enables us to model different hardware configurations. Every component leverages its own memory clerk to allocate large blocks of memory. Memory objects, heaps, utilize memory clerks to allocate memory. When SQL Server runs it creates a number of different memory clerks and memory objects.  One could monitor them by quering SQL Server's DMVs, Dynamic Management Views. In addition to nodes, clerks and memory objects, SQLOS implements unique caching and pooling frameworks. The whole memory subsystem deserves several  posts and as promised I will start covering it pretty soon.

 

SQLOS contains several threads to provide self monitoring and adequate resource distribution. In addition SOS exposes hosting interfaces. Hosting API significantly simplifies integration of CLR and SQL Server engines. CLR leverages SQLOS’s hosting API for all of its OS needs including thread pooling and memory management.

 

 There is plenty of information to cover about SQLOS. One probably could write a whole book describing it. In my next post I will start going over SQLOS memory manager in deeper details.  If you haven’t read my previous posts about Windows memory management it is time to do so. You need to be ready so that you can fully enjoy the next article.

 

Please feel free to ask any questions. And let me know your comments.

 

Enjoy great weekend!