Performance Degradation in Central and South Central US – 05/29 – Mitigated

Final Update: Tuesday, May 29th 2018 17:26 UTC We’ve confirmed that all systems are back to normal. Sorry for any inconvenience this may have caused. A recent change increased the amount of compute resources required to process user profile requests. The Engineering team proactively added compute capacity to deal with the increase. During this process…


Postmortem: Global VSTS availability issues – 22 May 2018

Customer Impact: On 22 May 2018, Visual Studio Team Services (VSTS) experienced a major incident across multiple regions between 15:00 and 16:55 UTC.  An event in a Western European scale unit of the Team Foundation Service (TFS), caused a chain reaction that sporadically took other TFS scale units offline in other regions.  Based on our…


Performance Degradation in West Europe – 05/25 – Mitigated

Final Update: Friday, May 25th 2018 16:39 UTC We’ve confirmed that all systems are back to normal as of 15:00 UTC. Our logs show the incident started on 13:45 UTC and that during the 1 hour and 15 minutes that it took to resolve the issue. Users may have experienced performance degradation in the West…


Performance Degradation in West Europe – 05/25 – Mitigated

Final Update: Friday, May 25th 2018 10:10 UTC We’ve confirmed that all systems are back to normal as of May 25th 2018 9:25 UTC. The issue got self-healed at our partner “Azure Active Directory (AAD) Graph API” end. Our partner team is working on understanding the complete root cause of this issue. We apologize for…


Performance Degradation in West Europe – 05/24 – Mitigated

Final Update: Thursday, May 24th 2018 17:06 UTC We’ve confirmed that all systems are back to normal as of 14:25 UTC. Our logs show the incident started on 14:05 UTC and that during 25 minutes that it took to resolve the issue. customers may have experienced Performance Degradation in the West Europe region. Sorry for…


Performance Degradation in West Europe – 05/23 – Mitigated

Final Update: Wednesday, May 23rd 2018 12:34 UTC We’ve confirmed that all systems are back to normal as of May 23rd, 2018 09:55 UTC. Our logs show the incident started on May 23rd, 2018 07:35 UTC and that during the 2 hours and 30 minutes that it took to resolve the issue customers would have…


Availability issues and Performance Degradation in multiple regions – 05/22 – Mitigated

Final Update: Tuesday, May 22nd 2018 17:08 UTC We’ve confirmed that all systems are back to normal as of 5/22 16:55 UTC. Our logs show the incident started on May 22 2018 14:55 UTC that during the 2 hours that it took to resolve the issue, users have experienced availability impact and severe performance degradation…


Investigating issues with MSA login failure in multiple regions – 05/22 – Mitigated

Final Update: Tuesday, May 22nd 2018 04:59 UTC We have confirmed that all systems are back to normal as of May 22nd 2018 03:40 UTC. Our logs show the incident started on May 22nd 2018 02:28 UTC. Our Partner team has identified preliminary cause to be related to a recent deployment task that impacted instances…