Work Item linking broken for some users – 08/31 – Resolved

Final Update: Friday, September 1st 2017 00:30 UTC The fix has been deployed to all affected scale units. Users should not be facing issues with work item linking anymore. Sincerely, Sri Harsha   Initial Update: Thursday, August 31st 2017 21:15 UTC Starting yesterday, you may have noticed that you are unable to create a new linked work item or…


Performance Issues with Visual Studio Team Services – 08/31 – Resolved

Update: Friday, September 1st 2017 12:00 UTC Root cause has been identified to be a bug that was introduced in the recent deployment. A fix has already been rolled out so users should not expect to see this issue re-occur again. We apologize for the disruption this has caused you. Sincerely, Sri Harsha Final Update: Thursday, August…


Issues with Visual Studio Team Services – 08/31 – Mitigated

Final Update: Thursday, August 31st 2017 17:15 UTC VSTS in the South Central region is now back in a healthy state. The incident was mitigated after the bad web tier node was taken out of the load balancer. We’ve confirmed that users are no longer seeing slowness or errors. Additionally, several users have confirmed builds…


Notification Delays on Visual Studio Team Services – 08/29 – Mitigated

Final Update: Tuesday, August 29th 2017 15:10 UTC We’ve confirmed that all systems are back to normal as of August 29th 2017 14:40 UTC. Our logs indicate that the incident started on August 29th, 2017 13:10 UTC. During the course of this incident, VSTS customers may have observed a delay in notifications. The load on…


Performance Issues with Visual Studio Team Services – 08/29 – Mitigated

Final Update: Tuesday, August 29th 2017 11:26 UTC We’ve confirmed that all systems are back to normal as of August 29th 2017 10:25 UTC. Our logs show the incident started on August 29th 2017 09:20 UTC. There was an active issue with Azure Active Directory (AAD) Graph API in Europe region that in turn impacted…


Issues with Visual Studio Team Services – 08/24 – Mitigated

Final Update: Thursday, August 24th 2017 07:26 UTC The fix for this issue has been deployed to all the regions. Users using V1 version of the test task using VS 2017 Update 3 in either Hosted 2017 pool or a private agent should now not see any issues. As communicated earlier, there is no impact…


Notification Delays on Visual Studio Team Services – 08/23 – Mitigated

Final Update: Wednesday, August 23rd 2017 09:25 UTC We’ve confirmed that all systems are back to normal as of August 23rd 2017 08:50 UTC. Our logs indicate that the incident started on August 23rd, 2017 02:18 UTC. During the course of this incident, VSTS customers would have observed a delay in notifications. As part of…


Issues with Visual Studio Team Services Hosted Build – 08/16 – Mitigated

Final Update: Wednesday, August 16th 2017 23:27 UTC We see build queue times have returned to normal levels after applying a fix.Root Cause: A bug was found with a cleanup procedure that caused builds to queue longer than expected. A fix has been put in place and we do not expect this issue to reoccur….


Issues with Visual Studio Team Services – 08/16 – Mitigated

Final Update: Thursday, August 17th 2017 15:26 UTC In order to mitigate the issue, the affected image has been rolled back to the previous version. Root Cause: The image which was rolled out for the Hosted 2017 pool was created incorrectly We apologize for the inconvenience this may have caused.Sincerely,Niall Update: Thursday, August 17th 2017…


Issues with Visual Studio Team Services – 08/15 – Mitigated

Final Update: Tuesday, August 15th 2017 21:35 UTC We’ve confirmed that all systems are back to normal as of 21:15 UTC. ~1,213 customers experienced feeds and packaging issues in the South Central US, West Europe, and Australia Southeast regions. We apologize for any inconvenience this may have caused. Incident Timeline: 1 hours and 40 minutes –…