Experiencing Data Access Issue in Azure Portal for Many Data Types - 06/23 - Resolved


Final Update: Thursday, 23 June 2016 21:19 UTC

We've confirmed that all systems are back to normal. Access to Application Insights data through Azure portal is back to normal as of 06/23, 19:07 UTC, and the Data Latency is back to normal as of 06/23, 21:00 UTC.  Our logs show the incident started on 06/23 14:45 UTC and that during the 6 hours that it took to resolve the issue some customers experienced Data Access issue, Data Latency, and a small subset of customers who have implemented SDKs without retries may have lost some data.
  • Root Cause: Preliminary root cause analysis indicates a flurry of issues between multiple services caused the impact.  Each issue on their own would normally have been a very minor issue, but this storm of issues combined to cause a much larger impact.
  • Lessons Learned: We are investigating additional improvements to internal telemetry to speed diagnosis and tooling to aid in implementation of mitigation steps.
  • Incident Timeline: 6 Hours & 15 minutes - 06/23 14:45 UTC through 06/23, 21:00 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Elena K



Update: Thursday, 23 June 2016 19:47 UTC

We are still working to understand full root cause of the issue, but we have taken steps to reduce impact to customer data access. Some customers continue to experience data latency, and customers who have implemented SDKs without retries may have lost some data due to the documented behavior of those SDKs.  Customers should see substantially improved performance for Application Insights in the Azure portal.  We currently have no estimate for full resolution.
  • Work Around:
  • Next Update: Before 06/23 22:00 UTC

-Matt


Update: Thursday, 23 June 2016 17:20 UTC

We continue to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers will continue to experience Data Access Issue in Azure portal. A small subset of customers who have implemented SDKs without retries may lose some data. Initial findings indicate that the problem began at 06/23 14:45 UTC. We are actively engaged in mitigation efforts, but currently have no estimate for resolution.
  • Work Around: none
  • Next Update: Before 06/23 19:30 UTC

-Elena K


Initial Update: Thursday, 23 June 2016 15:19 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Access Issue in Azure Portal. The following data types are affected: Availability,Customer Event,Dependency,Exception,Metric,Page Load,Page View,Performance Counter,Request,Trace.
  • Work Around: none
  • Next Update: Before 06/23 17:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Girish K


Skip to main content