Issues with Visual Studio Team Services – 08/04 – Mitigated


Final Update: Friday, August 4th 2017 13:17 UTC

We’ve confirmed that all systems are back to normal as of 4th August 2017 12:34 UTC. Our logs indicate the incident started on 4th August 2017 10:30 UTC and during the 2 hours and 5 minutes that it took to resolve the issue, users in South Central US, North Central US, East US and South Brazil would have observed intermittent "500 internal server error" while accessing their VSTS accounts as shown below

Image

Initial Root Cause:
VSTS traffic was getting routed to recently upgraded DNS servers which were unhealthy. Our DNS service provider has taken the faulty servers out of rotation and have also made sure that any automated upgrades will not add these servers back into rotation accidentally.

We are sorry for the interruption this outage has caused you and your team. We plan to publish a detailed root cause by middle of next week.

Sincerely,
Aman


Update: Friday, August 4th 2017 12:15 UTC

We are still investigating issues with Visual Studio Team Services (VSTS). Customers in several locations have reported receiving ‘500 Internal Server Error’ while accessing their VSTS accounts. We have engaged our DNS service provider to investigate this issue further. We are evaluating a potential mitigation at the moment.

We regret the inconvenience this is causing our affected customers.

  • Next Update: Before Friday, August 4th 2017 14:30 UTC

Sincerely,
Rakesh Reddy


Initial Update: Friday, August 4th 2017 10:59 UTC

We are investigating issues affecting users of the service. Users in North Central US, South Central US, East US may experience degraded performance or slow page load times.

  • Next Update: Before Friday, August 4th 2017 12:30 UTC

Sincerely,
Rakesh Reddy


Skip to main content