Issues with Visual Studio Team Services - 07/27 - Mitigated

Final Update: Thursday, July 27th 2017 21:42 UTC We’ve confirmed that all systems are back to normal as of 20:40 UTC. We will continue to monitor to ensure all remains stable. Sorry for any inconvenience this may have caused. Root Cause: The Root Cause is currently being determined Chance of Re-occurrence: Medium Incident Timeline: 55…


Performance Issues with Visual Studio Team Services - 07/26 - Mitigated

Final Update: Wednesday, July 26th 2017 10:16 UTC We’ve confirmed that all systems are back to normal as of July 26th 2017 09:26 UTC. Our logs shows that incident started on July 26th 2017 08:40 UTC and lasted for ~46 minutes. We observed that this is a reoccurrence of an issue that happened on July…


Performance Issues with Visual Studio Team Services in West Europe - 07/21 - Mitigated

Final Update: Friday, July 21st 2017 10:52 UTC We can confirm that the issue is mitigated as of July 21st 2017 09:00 UTC. The issue started at July 21st 2017 08:40 UTC and lasted for 20 minutes. Our initial analysis indicates that issue was caused due to a background clean up job with high execution…


Issues accessing profile page in Visual Studio Team Services - 07/21 - Mitigated

Final Update: Friday, July 21st 2017 02:15 UTC We’ve confirmed that the profile service is back to normal as of Friday, July 21st 2017 01:36 UTC . A deployment done earlier today caused the ‘My Profile’ link to be broken. Hotfix has been deployed to resolve this issue and customers should not notice any error…


Issues with Visual Studio Team Services - 07/20 - Mitigated

Final Update: Thursday, July 20th 2017 13:20 UTC We’ve confirmed that all systems are back to normal as of 20 July 2017 11:19 UTC. Our logs show the incident started on 20 July 2017 09:14 UTC.The issue got self-healed. During the course of our trouble shooting, we identified a command that was used to get…


Performance Issues with Visual Studio Team Services - 07/18 - Resolved

Update: Tuesday, 18 July 2017 02:00 UTC We’ve confirmed that all systems are back to normal as of Tuesday, July 18th 2017 01:05 UTC. Our logs show the incident started on Tuesday, July 18th 2017 00:35 UTC. One of our collection database in East Australia region was unhealthy and saw an increase in active requests…


Post Mortem: Networking failures for Visual Studio Team Services on 12 July 2017

On 12 July 2017, we experienced a networking incident that lasted 19 minutes that affected VSTS customers using the service (You can find the incident blog details here). We sincerely regret the inconvenience that this has caused our users. We have conducted an internal post mortem along with our partners in Azure networking to look…


Post Mortem: Authorization related failures for Visual Studio Team Services on 10 July 2017

On 10 July 2017, we experienced a major service incident that lasted just over 4 hours and affected many VSTS customers using the service at that time (incident blog here). We know how important VSTS is to our users and deeply regret the interruption in service. Over the last several days, we’ve worked to understand…


Performance Issues with Visual Studio Team Services - 07/13 - Mitigated

Final Update: Thursday, July 13th 2017 16:25 UTC We’ve confirmed that all systems are back to normal as of Thursday, July 13th 2017 15:06 UTC. Our logs show the incident started on Thursday, July 13th 2017 14:42 UTC. One of our application tier instance in South Central US region was unhealthy and had user requests…


Issues with Visual Studio Team Services - 07/12 - Mitigated

Final Update: Wednesday, July 12th 2017 09:23 UTC We’ve confirmed that all systems are healthy as of Wednesday, July 12th 2017 07:00 UTC. Our logs indicate that the incident started on Wednesday, July 12th 2017 06:34 UTC. During the course of this incident customers were not able to access their Visual Studio Team Services accounts…