SQL Server 2005 Express setup failure with error - An error occurred during encryption.

Recently I have worked on one of the case where SQL Server Express Edition was failing with Error

The SQL Server service failed to start. For more information, see the SQL Server Books Online topics, "How to: View SQL Server 2005 Setup Log Files" and "Starting SQL Server Manually

This is generic error and can be caused by many reasons. We need to look at SQL Error Logs / Event Logs and find out actual reason. From Event Log error was “The process terminated unexpectedly”. Not very useful in this case.

Next, we looked into SQL Error Log

2009-11-16 15:47:17.26 spid5s      Error: 15209, Severity: 16, State: 1.
2009-11-16 15:47:17.26 spid5s      An error occurred during encryption.

Above was the cause of SQL Startup failure. Basically there was an unhandled exception raise by SQLServr.exe process and terminated. If you search on support.microsoft.com for this error, this is the KB Error message when you try to install SQL Server Express or SQL Server. If you are using NTFS then first check KB before processing further.

In our case KB was not applicable because it was FAT file system and there is no option to set security as mentioned in KB. To troubleshoot further we have taken crash dump of SQL Server process using DebugDiag. You can refer to documentation to see how to take crash dump but I would show you what we got from the dump.

kernel32!RaiseException+0x53
msvcr80!_CxxThrowException+0x46
sqlservr!TurnUnwindAndThrowImpl+0x13c
sqlservr!ex_raise2+0x48f
sqlservr!ex_raise+0x68
sqlservr!RaiseCryptoError+0xda
sqlservr!CreateSrvMasterSecret+0x2d2
sqlservr!LoadSrvMasterSecret+0x139
sqlservr!LoadServiceMasterKey+0x11
sqlservr!StartUp::StartResourceDB+0x3ca
sqlservr!StartUp::OpenDBsAndRecover+0x4d8
sqlservr!StartUp::InitDBMS+0x287

sqlservr!SOS_Task::Param::Execute+0xe2
sqlservr!SOS_Scheduler::RunTask+0xb9
sqlservr!SOS_Scheduler::ProcessTasks+0x141
sqlservr!SchedulerManager::WorkerEntryPoint+0x1ab
sqlservr!SystemThread::RunWorker+0x7f
sqlservr!SystemThreadDispatcher::ProcessWorker+0x246
sqlservr!SchedulerManager::ThreadEntryPoint+0x143
msvcr80!_callthreadstartex+0x1b
msvcr80!_threadstartex+0x66
kernel32!BaseThreadStart+0x37

In layman terms, we were trying to create self generated certificate using Crypto API provided by windows. When we looked further, we found exception as 0xc000003a. This means STATUS_OBJECT_PATH_NOT_FOUND.

We took procmon (available at https://technet.microsoft.com/hi-in/sysinternals/default(en-us).aspx) and found interesting stuff

3:47:17.2609980 PM lsass.exe CreateFile C:\WINDOWS\system32\Microsoft\Protect\S-1-5-18\User\Preferred PATH NOT FOUND
3:47:17.2612661 PM lsass.exe CreateFile C:\WINDOWS\system32\Microsoft    NAME COLLISION
3:47:17.2615210 PM lsass.exe CreateFile C:\WINDOWS\system32\Microsoft\Protect PATH NOT FOUND
3:47:17.2617194 PM lsass.exe CreateFile C:\WINDOWS\system32\Microsoft\Protect\S-1-5-18 PATH NOT FOUND
3:47:17.2619209 PM lsass.exe CreateFile C:\WINDOWS\system32\Microsoft\Protect\S-1-5-18\User PATH NOT FOUND

So I can see PATH NOT FOUND, above this I can see NAME COLLISION. This essentially means that there is a object already existing. When we looked at customer’s machine, we found a file called “Microsoft” (Ideally, this should be a folder) which was causing SQLSetup to not to create a folder and hence failing. Lsass.exe is responsible for many things (refer documentation in KB308356)

To resolve the issue, we have renamed Microsoft file to Microsoft_1 and then as expected, sqlservr.exe was able to start and setup worked like a charm.

If you have any such issue, ProcMon is the first thing to look at along with Event Logs and SQL Error Logs.

Hope this helps someone on the earth (and beyond)!!!

- Balmukund