Build definition issues on Visual Studio Team Services – 09/29 – Mitigated

Final Update: Friday, September 29th 2017 22:09 UTC The hotfix rollout is now complete and most users should be able to see all their build definitions as of Friday, September 29 2017 21:38 UTC. In some cases, users may not see build definitions if their view is filtered to show a few, the work around…


Issues with pushing to external nuget sources – 09/25 – Resolved

Update: Monday, 25 September 2017 23:32 UTC The incident has been resolved after the deployment has been completed to all deployment rings. Users should not see any more issues with pushing to external nuget sources.  Sincerely,Sri Harsha Update: Monday, 25 September 2017 21:37 UTC The deployment to final ring is taking longer than expected due…


Issues with Visual Studio Team Services – Build – 09/20 – Mitigated

Final Update: Wednesday, 20 September 2017 19:03 UTC We’ve confirmed that all systems are back to normal as of September 20th 2017 17:20 UTC. The fix for the build queue failures has been deployed globally. During the incident, customers may have seen the following build queue error: “Value cannot be null. Parameter name: value”.  Sincerely,Daniel Update: Wednesday,…


Issues with Visual Studio Team Services – 09/20 – Mitigated

Final Update: Wednesday, September 20th 2017 14:05 UTC The issue has now been fully mitigated.Some users may have observed delays in their build requests during the time. This was due to a specific capacity issue in West Europe Azure region. As a mitigation step additional capacity has been added in North Europe Azure region. Root…


Issues with My.VisualStudio.com in Visual Studio Team Services – 09/18 – Mitigated

Final Update: Monday, 18 September 2017 23:38 UTC We’ve confirmed that all systems are back to normal, users are now able to load the my.visualstudio.com page. During this issue, customers were not able to access the my.visualstudio.com page. Root cause for this issue is still under investigation. Sincerely,Daniel Update: Monday, 18 September 2017 22:50 UTC…


Hosted Build issues with Visual Studio Team Services – 09/18 – Mitigated

Final Update: Monday, September 18th 2017 10:04 UTC We’ve confirmed that all systems are back to normal as of 18 September 2017, 05:45 UTC. Our logs show that the incident started on 18 September 2017, 1:45 UTC. A subset of users in the East Australia region might have experienced significant delays in build processing during…


Issues with build definition page in Visual Studio Team Services – 09/16 – Mitigated

Final Update: Saturday, September 16th 2017 05:20 UTC The hotfix deployment has finished and we’ve confirmed that the functionality to load the build definition is working as expected as of 16 Sept 2017 04:02 UTC.We apologize to the customers who were affected by this issue. Sincerely,Rakesh Reddy Update: Saturday, September 16th 2017 01:52 UTC We…


Issues with Visual Studio Team Services – 09/14 – Mitigated

Final Update: Thursday, September 14th 2017 19:46 UTC VSTS customers in South Central experienced three distinct periods of impact where performance in the service was degraded (see the top graph). At time of peak impact ~1,500 interactive users would have seen intermittent slow commands. The underlying issue was related to one of the five web…


Performance Issues with Visual Studio Team Services – 09/14 – Mitigated

Final Update: Thursday, September 14th 2017 13:10 UTC VSTS in the South Central US region is now back in a healthy state. The incident was mitigated after the bad web tier node was taken out of the load balancer. We’ve confirmed that users are no longer seeing slowness or errors. We are working to analyze…


Issues with Visual Studio Team Services – 09/12 – Mitigated

Final Update: Tuesday, September 12th 2017 12:44 UTC The hotfix deployment has finished and we’ve confirmed that the functionality to increase the Cloud Load Test paid minutes to “No Limit” is working as expected as of 12 Sept 2017 12:30 UTC. We apologize to the customers who were affected by this issue.. Sincerely,Aman Update: Tuesday,…