Performance issues with Visual Studio Team Services – 04/24 – Resolved

Final Update: Monday, 24 April 2017 19:41 UTC We’ve confirmed that all systems are back to normal as of 04-24-2017 18:53 UTC. Our logs show the incident started on 04-24-2017 18:40. Customers should now have no issues while accessing their Visual Studio Team Services accounts. Initial investigations suggests that the issue was due to high call rates on…


Connectivity issues with Marketplace – 04/22 – Resolved

Update: Saturday, 22 April 2017 02:44 UTC Users should now be able to access the market place without any issues.  During this impact window (Start Time: 2017/04/22 00:00 UTC  (2017/04/21 17:00 Pacific) and End Time: 2017/04/22 00:10 UTC  (2017/04/21 17:10 Pacific)) users might have faced connectivity issues on Market and might have got 500 errors. This issue was mitigated once…


Connectivity issues with Visual Studio Team Services – 04/22 – Resolved

Update: Saturday, 22 April 2017 03:41 UTC Root cause has been isolated to a missing registry entry for the builds using legacy OAuth validation. To address this issue we deployed a configuration change and the system is now working as expected. Users should now have no issues to connect to Visual Studio Team Services through CI/CD builds….


404 errors on Visual Studio Team Services – 04/21 – Resolved

Update: Friday, 21 April 2017 19:56 UTC The issue has been mitigated after the hotfix has been deployed to our Shared Platform Services (SPS). This was caused due to a forwards-compatibility issue with a feature where a service that hasn’t yet deployed the latest binaries interacts with SPS that is running on latest update, a…


Performance issues with Visual Studio Team Services – 04/21 – Resolved

Update: Friday, 21 April 2017 06:49 UTC Issue has been isolated to one of the web role instances that was unhealthy. All the requests going to this instance were failing. To address this issue we have collected necessary dumps and have pulled the role instance out of load balancer. Visual Studio Team Services is now…


Performance issues with Hosted Build Service – 04/18 – Resolved

Update: Tuesday, 18 April 2017 15:57 UTC Customers should no longer be seeing longer build queue times now. During the course of the incident, we identified that customers in West Europe and Central US regions observed longer than usual build queue times. The symptom for this issue to occur was high processor utilization on the…


Issues with VS Subscriptions in Visual Studio Team Services – 04/14 – Resolved

Final Update: Friday, 14 April 2017 20:25 UTC We’ve confirmed that all systems are back to normal as of 14 April 2017 ~20:00 UTC. Product Activation team has made the necessary changes to their databases to mitigate the issue. Users can now able to access Subscriptions and claim their product keys. Root Cause: Product Activation end points were inaccessible. Chance of Reoccurrence:…


Performance issues with Visual Studio Team Services – 04/14 – Resolved

Update: Friday, 14 April 2017 19:06 UTC The root cause has been isolated to an unexpected state of a specific feature flag for our collection database. To address this issue we worked with the SQL Azure team to get the feature flag fixed. Project creation and work item related functionality are now working as expected….


Invesitgating issues with Visual Studio Team Services – 04/13 – Resolved

Update: Friday, 14 April 2017 22:22 UTC The root cause has been isolated to a recent change in the behaviour of an undelying dependent application which was impacting authentication flow. To address this issue we have deployed a hotfix to our service. Microsoft Teams integration is now working as expected Sincerely,Ian Update: Friday, 14 April…


Issues with VS Subscriptions – 04/13 – Resolved

Update: Thursday, 13 April 2017 22:20 UTC Root cause has been isolated to network issue. To mitigate this issue Product Activation team failed over the DR servers in Singapore DC to primary server. Now everything back to normal and Retail customers of VS Subscriptions can now able to activate the keys. We sincerely apologize for the inconvenience it…