Performance issues with Visual Studio Team Services – 9/29 – Resolved

Final Update: Thursday, 29 September 2016 14:42 UTC We’ve confirmed that all systems are back to normal as of 29 September 2016 13:54 UTC. Our logs show the incident started on 29 September 2016 12:49 UTC. We understand that customers rely on VS Team Services as a critical service and apologize for any impact this…


Issues with Search feature on Visual Studio Team Services – 9/28 – Resolved

Final Update: Wednesday, 28 September 2016 20:17 UTC We have applied the mitigation and confirm that all systems are back to normal as of 28 September 2016 18:48 UTC. Customers should not see any issues rendering VSTS extensions now. We understand that customers rely on VS Team Services as a critical service and apologize for…


Issues with CodeLens – 9/27 – Resolved

Final Update: Wednesday, 28 September 2016 00:23 UTC We’ve confirmed that all systems are back to normal as of 28 September 2016 00:16 UTC. The failure was due to an incorrect CNAME configuration on the service certificate which had to be fixed to resolve the issue. We will be conducting a detailed postmortem and ensure that we have…


Issues with Visual Studio Team Services in East Australia Region – 9/20 – Resolved

Final Update: Wednesday, 21 September 2016 03:37 UTC We’ve confirmed that all systems are back to normal. Our partners in Azure have identified a connectivity issue within Australia East region as the underlying issue. All the faulty connections were isolated to mitigate the issue. Azure engineering will continue to investigate the root cause and will publish a…


Performance issues with Visual Studio Team Services – 9/20 – Resolved

Final Update: Tuesday, 20 September 2016 18:15 UTC We’ve confirmed that all systems are back to normal with no customer impact as of 20 Sep 2016 ~17:30 UTC. Customers should not have any performance issues accessing Market place. We have deployed a hotfix to address the performance issue. We understand that customers rely on VS Team…


Performance issues with Visual Studio Team Services – 9/19 – Resolved

Final Update: Monday, 19 September 2016 20:40 UTC We’ve confirmed that all systems are back to normal  as of 2016/09/19 20:40 UTC . Our logs indicates that  the incident started on 2016/09/19 09:30 UTC. Root Cause: There was a bad role which was continuously causing high CPU usage and we engaged Azure team to mitigate…


Performance issues with Visual Studio Team Services – 9/19 – Resolved

Final Update: Monday, 19 September 2016 07:33 UTC We’ve confirmed that all systems are back to normal  as of 2016/09/19 06:35 UTC . Our logs indicates that  the incident started on 2016/09/19 05:30 UTC. Root Cause: There was an issue with Azure storage in East US region  which has been mitigated. A detailed RCA is…


Performance issues with Visual Studio Team Services – 9/17 – Resolved

Final Update: Saturday, 17 September 2016 09:26 UTC We’ve confirmed that all systems are back to normal as of 17 September 2016 09:14 UTC. Our logs show the incident started on 17 September 2016 02:03 UTC.  We understand that customers rely on VS Team Services as a critical service and apologize for any impact this incident…


Performance issues with Visual Studio Team Services – 9/15 – Resolved

Resolution Statement: Thursday, 15 September 2016 14:39 UTC   Preliminary root cause: A spike in networking traffic was experienced, which caused service-level drops for the DNS service. This resulted in connectivity issues for services reliant on DNS. Once network mitigation was implemented, most services fully recovered. Microsoft SQL Azure had secondary impact due to a…


SignOut issues with Visual Studio Customer Profile page – 9/15 – Resolved

Final Update: Thursday, 15 September 2016 04:11 UTC We’ve confirmed that all systems are back to normal with no customer impact as of 15 September 2016 ~04:08 UTC. We identified that with recent changes to our visualstudio.com homepage the UHF on Profile page were broken causing this issue. Fix was deployed and customers should be able…