Windows Azure Worker Role Recycling due to an exception as "System.Data.SqlClient.SqlException: at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds"

  • Article

Recently I was working on an issue and found the worker role was keep recycling. After I logged into Azure VM and looked as Application Event Log, I found the following exception was logged every 30 seconds:

 

 Log Name: Application
 Source: .NET Runtime
 Date: 7/6/2011 9:39:28 PM
 Event ID: 1026
 Task Category: None
 Level: Error
 Keywords: Classic
 User: N/A
 Computer: <Machine_Name>
 Description:
 Application: WaWorkerHost.exe
 
 Framework Version: v4.0.30319
 Description: The process was terminated due to an unhandled exception.
 
 Exception Info: System.Data.SqlClient.SqlException
 Stack:
 at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(System.Data.CommandBehavior, System.Data.SqlClient.RunBehavior, Boolean, Boolean)
 at System.Data.SqlClient.SqlCommand.RunExecuteReader(System.Data.CommandBehavior, System.Data.SqlClient.RunBehavior, Boolean, System.String, System.Data.Common.DbAsyncResult)
 at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(System.Data.Common.DbAsyncResult, System.String, Boolean)
 at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
 at <custom_reference>.SqlErrorLog.Log(Elmah.Error)
 at <custom_reference>.WorkerRole.GenException(System.Exception)
 at <custom_reference>.WorkerRole.DequeueMessages()
 at <custom_reference>.WorkerRole.Run()
 at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.StartRoleInternal()
 at Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge.<StartRole>b__1()
 at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
 at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
 at System.Threading.ThreadHelper.ThreadStart()

 

 Investigation and Solution:

After Investigation, what we found is that SQL Azure database was set to a max size of 1GB, and that database was full. Some of the worker roles were attempting to write trace data to the SQL Azure DB on startup, and since the database wasn’t accepting new data, the roles terminated unexpectedly. After making room in SQL Azure DB the problem was resolved.