VSS: SQL Server does not exist or access denied

I recently worked on three customer issues. In all those cases, customers were not able to backup databases using Microsoft Volume Shadow Copy Service (VSS). MSDEWriter does not show up with “vssadmin list writers”. And VSS logs the following error message in event log:

Sqllib error: OLEDB Error encountered calling IDBInitialize::Initialize. hr = 0x80004005. SQLSTATE: 08001, Native Error: 17
Error state: 1, Severity: 16
Source: Microsoft OLE DB Provider for SQL Server
Error message: [DBNETLIB]SQL Server does not exist or access denied.

MSDEWriter uses MDAC SQLOLEDB to make connections to all SQL instances installed on a machine. VSS finds SQL server instances and the instance names by enumrating running SQL server services on the box. It uses the default protocol to connect the servers. In order for MSDEWriter to work correctly, it must be able to connect all SQL server instances of interests. That is, if you have 5 instances and fail to connect to only one of the instances, MSDEWriter will not work. MSDEWriter only works with local instances. We can ignore any possible firewall issue as this is local connection.

In the first case, client shared memory protocol is disabled by the customre unexpectedly. One of the SQL instances installed on the customer machine is WMSDE. As we know, WMSDE can only listen on shared memory protocol. Clients can only connect to a WMSDE instance using shared memory, which is disabled in the customer's box. Enabling the shared memory protocol using SQL Server Client Network Utility fixes this problem.

The second case was due to an outdated alias. The customer had an alias defined previously but forgot to remove it when configuration changes. This alias forces connections to one of the instances to use TCP protocol. However, at some point, the customer disabled TCP protocol for the SQL server instance. This is also an example which tells us that users should use alias with caution, although alias can provide convenience and could be a useful workaround when user faces some specific problems.

The third case is a surprise to us. We eventually found out that our sqloledb.dll is not loaded. A third party OLEDB provider took over the role of Microsoft SQLOLEDB provider. Like SQLOLEDB provider, the third party provider also calls DBNETLIB APIs. However, this provider does not behave the same way as Microsoft SQLOLEDB and passes different connection string to DBNETLIB. This breaks our product integrity and causes the problem. The problem is fixed by replacing the value of the following keys:

 

HKEY_LOCAL_MACHINESOFTWAREClassesCLSID{0C7FF16C-38E3-11d0-97AB-00C04FC2AD98}InprocServer32Default
to
C:Program FilesCommon FilesSystemOle DBsqloledb.dll.

So, if you recently installed any third party database product, you may also want to check this registry when you have similar connectivity issues. We don’t expect any third party product changes our registry value.

Xinwei Hong, SQL Server Protocols
Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights