Elevated error rates in multiple Azure DevOps regions in the United States – 10/03 – Mitigated


Final Update: Wednesday, October 3rd 2018 19:01 UTC

We’ve confirmed that all systems are back to normal as of 18:15 UTC. Our logs show the incident started on 16:50 UTC. Errors for users in multiple US scale units have subsided within the initial 10 minutes of this incident, but a subset of users with organizations hosted in South Central US have experienced issues for an extended period of time.

We have all necessary teams within Azure DevOps and Azure Networking engaged to understand the root cause and avoid a re-occurrence

Sincerely,
Sri Harsha


Update: Wednesday, October 3rd 2018 17:34 UTC

The impact has been mitigated across all of our global instances, but we are still monitoring closely. We believe the issue is due to some issues in the back-end networking and we are engaged with the necessary teams to understand the root cause. Users should not be facing any more errors or performance degradation while accessing Azure DevOps services.

  • Next Update: Before Wednesday, October 3rd 2018 18:45 UTC

Sincerely,
Sri Harsha


Initial notification: Wednesday, October 3rd 2018 17:24 UTC

We're investigating a possible customer impacting event in multiple locations in North America. We are triaging the issue and will provide more information as it becomes available.

  • Next Update: Before Wednesday, October 3rd 2018 17:55 UTC

Sincerely,
Sri Harsha


Skip to main content