We’ve come across some SQL timeout issues with RM 2013/2015 when it has to process a large amount of users/groups to sync with AD.
If your setup has a large number of users, permissions managed by adding users to AD groups, and in turn they’re added as RM Groups, you may run into this issue.
The users added to AD groups will no longer get permissions in Release Management.
You will see the following under Release Management Client error logs,
“Error - (12048, 6880) - Unable to refresh AD groups at this time. This can happen if AD is temporarily unavailable. Please try again. If the problem persists contact your application administrator.”
You may also see some SQL timeout exception like,
Message: Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.: \r\n\r\n at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
If you have one large AD group, you could try getting the time it takes to fetch all members of the group from AD using the Powershell command below:
“Get-ADGroupMember -identity "<identity>" -server <server> -Recursive| select name”
If you notice that the time taken to return the output takes more than the application timing out, you will have to update some settings on the web.config files.
For Release Management 2013,
- Open ReleaseManagement server web.config from “%ProgramFiles(x86)%\Microsoft Visual Studio 12.0\Release Management\services\web.config”
- Change the value of “SqlCommandsTimeoutInSeconds” to some large number, say, 300 (Default is 30)
Similarly for Release Management 2015,
- Open ReleaseManagement server web.config from “%ProgramFiles(x86)%\Microsoft Visual Studio 14.0\Release Management\services\web.config”
- Change the value of “WebServicesTimeoutInSeconds” to some large number, say, 500 [ number depends on actual time to sync mentioned above, increase it as appropriate]
Note: Once you make this edit, IIS will recycle the AppDomain. This means your existing releases may time out/fail due to connectivity to the server. Be sure to run it post production hours.
If you are using RM only for Releases/Deployments and not connected to TFS, it’s recommended that you move to the latest Release Management 2015 Update 3 version as soon as possible. We have made some optimizations that will increase stability and scalability.
If you are using Team Foundation Server (TFS) with RM, please consider moving to vNext “Release” (available from TFS 2015.2 and above).
This version is a scalable, scriptable, integrated version of Release Management with TFS/Visual Studio Team Services (VSTS).
Hope this helps!
Content: Manigandan Balachandran
(This article is published after getting the content reviewed by the Release Management Engineering Team)