Windows Azure: Roles Repetitively Starved of Ports

Article
08/26/2013

You just upgraded the OS family or in case you have auto upgrade on for Windows Azure there is a chance that you might see the following errors in case you are using Storage Client and SQL Azure .Net APIs. The other symptom could be sluggishness/slowness/delay in any storage calls like blobs/queues/tables or even SQL Azure calls in case the application uses both Windows storage and SQL Azure. It could also afflict you in case Windows Azure diagnostics are being run as Windows Azure Diagnostics write to Storage.

Rebooting the VM resolves the issue only for it to come back after a certain time.

Storage

Server Error in ‘/’ Application.

An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified

Stack Trace:

[SocketException (0x2747): An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full]

Systen. Net. Sockets. Socket. DOBi nd(EndPoint endPoi ntSnapshot, SocketAddress socketAddress) t7985914

System. Net. Sockets. Socket. Internal Bi nd(EndPoint localEP) +243

Systen.Net.Sockets.Socket.BeginConnectEx(EndPoint remoteEP, Boolean flowcontext, AsyncCallback callback, Object state) +307

System. Net. Sockets. Socket. Unsafeßeqi nConnect(EndPoint remoteEP, AsyncCal lback callback, Object state) t71

System.Net.ServicePoint.ConnectSocketlnternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, lAsyncResult asyncResult, In

[WebException: Unable to connect to the remote server]

Microsoft.WindowsAzure.StorageClient.Tasks.Task’1.get_Result()

Microsoft.WindowsAzure.StorageClient.Tasks.Task’l.ExecuteAndwait()

Microsoft.WindowsAzure. StorageClient. <LazytnumerateSegmented>d_01.MoveNext()

System.Linq.Enumerable.Count(IEnumerable’1 source)

SQL Server

 Server Error in ‘I’ Application.

 A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where d originated in the code.

 Exception Details: System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection SQL Server)

 Source Error:

 An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

 Stack Trace:

 [SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server

 System.Data.ProviderBase.DbConnectionPool .GetConnection(DbConnection owningObject) +706

 System.Data.ProviderBase.DbConnectionFactory.CetConnection(DbConnection owningConnection) +89

 System.Data.Providerßase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) +6425518

 System.Data.SqlClient.SqlConnection.Open() +300

 System.Data.EntityClient.EntityConnection.OpenStoreConnectionlf(Boolean openCondition, DbConnection storeConnectionToOpen, DbConnection originalConnection, String exceptionCode, String attemptedOperation, Boolean& closeStoreConnectic

 [EntityException: The underlyin9 provider failed on Open.]

 System.Data.EntityClient.EntityConnection.OpenStoreConnectionlf(Boolean opencondition, DbConnection storeConnectionToOpen, DbConnection originalconnection, String exceptioncode, String attemptedoperation, Boolean& closeStoreConnectic
System.Data.EntityClient.EntityConnection.Open() +142

 System.Data.Objects.ObjectContext.EnsureConnection() +97

 System.Data.Ob1ects.Ob1ectQuery’1.GetResults(Nullabl&1 forMergeoption) +66

 System.Data.Objects.ObjectQuery 1.System.Collections .Generic.IEnumerable<T>.GetEnumeratorO ÷47

 System.Linq.Enumerable.SingleOrDefault(IEnumerable 1 source) +271

 System.Linq.Queryable.SingleOrDefault(IQueryabl&1 source) +383

 System.Data.Entity.Internal .Linq.InternalSet’1.FindlnStore(WrappedEntityKey key, String keyValuesParamName) +779

 System.Data.Entity.Internal .Linq.InternalSet’1.Find(Object[] keyvalues) +182

 System.Web.Providers.DefaultSessionStateProvider .SetAndReleaseltemExclusive(HttpContext context, String id, SessionStateStoreData item, Object lockld, Boolean newltem) +259

 System.Web.SessionState.SessionStateModule.OnReleaseState(Object source, EventArgs eventArgs) +929

 System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80

 System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +270

Open a command prompt in elevated mode and type netstat - anob

If you see a pattern as follows where a lot of ports are held by a single PID as below

Active Connections

Proto Local Address Foreign Address State PID

[MonAgentHost.exe]

TCP [::1]:80 [::1]:52108 ESTABLISHED 4 Can not obtain ownership information

TCP [::1]:80 [::1]:52109 ESTABLISHED 4 Can not obtain ownership information

TCP [::1]:80 [::1]:52110 ESTABLISHED 4 Can not obtain ownership information

TCP [::1]:80 [::1]:52111 ESTABLISHED 4 Can not obtain ownership information

TCP [::1]:80 [::1]:52112 ESTABLISHED 4 Can not obtain ownership information