SharePoint Timer Job Locks and Scope

With this post it's time we take our best shot at discussing a somewhat confusing topic - SharePoint timer job locks and scopes. I'll do my best to describe things; if you have questions or need further clarification post a comment and I'll attempt to clarify further.

All timer jobs have a scope of action - an object type upon which they are intended to work. This scope may be the farm, an individual server, or a content database. That is, for each iteration of a job, it is expected to execute once per farm, once per server, or once per content database. These are reflected in the three values of the SPJobLockType enum: Job, None, and ContentDatabase. Job means the job runs only once per farm per iteration; None means the job runs once per server; and ContentDatabase means the job runs once for each content database. The reason None is an appropriate description for a server-specific lock is that, as described in an earlier post, there is exactly one Timer Service Instance per server. When each timer service instance runs a job without checking or taking any locks, the result is that the job is run once per server. So "no lock" or None is the logical equivalent of a per-server lock.

We'll now discuss some elements of each type of lock.

Job Locks - Farm Scope (SPJobLockType.Job)

Jobs which have a lock of type Job run once per farm (per iteration). Examples of this type of job include the Password Management job, CEIP data collection job, and the Delete Job History job. Jobs with this scope typically take an action effecting farm-wide settings and configuration and run only once (per iteration).

To retrieve all farm-scoped jobs, run the following PowerShell command:

Get-SPTimerJob | ? {$_.LockType -eq "Job"}

To ensure only one server runs each job with a lock of scope Job (i.e. farm), locks are associated with a single specific server via a table in the Configuration Database (TimerLocks). Once taken by a server, the lock is typically held by the same server until the Timer Service on that server is stopped. Every 15 minutes (by default), the internal Lock Refresh job runs to refresh the job locks for this server. Every 60 minutes (by default) the internal Sweep job runs. This job attempts to get a lock for at most one job for which this server did not previously have a lock. By default, job locks time out after 20 minutes (which means that when all is functioning properly the Refresh job will ensure no lock ever times out). See the previous article in this series for more discussion on the internal timer jobs (Refresh and Sweep).

No Locks - Server Scope (SPJobLockType.None)

Jobs which have a lock of type None run once on each server in the farm. Because each server in the farm has one timer service instance, a job intended to run on every server does not need to take any locks - each server simply knows it can run the job without further concern.

Examples of jobs scoped to servers (Lock Type None) are the Config Refresh job, Timer Recycle job, and the Timer Locks job. To retrieve all server-scoped jobs, run:

Get-SPTimerJob | ? {$_.LockType -eq "None"}

Content Database Locks and Scope (SPJobLockType.ContentDatabase)

Last but far from least are jobs intended to run once per content database. Most of these jobs relate to processing of content and other front-end elements. Examples of Content Database jobs are the Expiration and Hold Processing jobs, the Immediate Alerts jobs, Audit Log Trimming job, and Recycle Bin cleanup job. These jobs are associated with a Web Application (or all Web Applications) and consequently run for each content database in the Web Application. To retrieve a sorted list of all the different types of content database jobs configured in a farm, run this command:

Get-SPTimerJob | ? {$_.LockType -eq "ContentDatabase"} | Select-Object -Unique | Sort-Object Name | Format-Table Name

Acquisition and maintenance of content database locks for a server is handled by the Timer Service Lock Management job (job-timer-locks). Locks are tracked by an entry in a table (TimerLock) in each content database associating the database with a single server in the farm. Note that content database locks are handled at the server and database level, not at the individual job level. That is, a server which acquires a lock for a given content database takes responsibility for executing all jobs for that database. No other server in the farm will execute any jobs for that database.

Content database locks expire after 10 minutes, a value which cannot be changed. Other details on content database locks will be presented when the Timer Service Lock Management job and SPContentDatabaseJobDefinition are discussed.

This completes the details on job locks and scopes.