Experiencing Data Latency for Many Data Types - 6/9 - Resolved


Final Update: Wednesday, 6/10/2015 05:58 UTC

We’ve confirmed that all systems are back to normal with no customer impact as of 6/10, 05:10 UTC. Our logs show the incident started on 6/9, 02:08 UTC and that during the 27 hours that it took to resolve.

 Incident Timeline:  27 Hours & 2 minutes - 6/9, 02:08 UTC through 6/10, 05:10 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team


Update: Wednesday, 6/10/2015 00:01 UTC

We have restored the system health and data for all data types is processing as expected. There is no latency since 6/9 17:41 UTC however we are still processing data since the time this incident occurred and there is good progress on that. We have processed data from 6/9 08:30 till 6/9 12:20 UTC and expect remaining 5 hours data (6/9 12:20-6/9 17:41 UTC) to be processed soon.

Next Update: Before 6/10 05:00 UTC

-Application Insights Service Delivery Team


Update: Tuesday, 6/9/2015 16:26 UTC

We have recovered the majority of system health.  Customers will see data latency only in the telemetry data stream. All other data streams have been recovered and processing current data. To mitigate telemetry data we are in the process of deploying a hotfix that will enable queue to process current data as well as speed up the recovery. We provide more updates as we move forward.

Next Update: Before 6/10 00:00 UTC

-Application Insights Service Delivery Team


Update: Tuesday, 6/9/2015 07:46 UTC

Our DevOps team continues to investigate issues within Application Insights. Root cause is not fully understood at this time. Some customers continue to experience data latency for all data streams .We are working to restore the system health ; our findings indicate that the problem began at 6/9 01:54 UTC . We currently have no estimate for resolution.

Next Update: Before 17:00 UTC

-Application Insights Service Delivery Team


Initial Update: Tuesday, 6/9/2015 05:15 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Latency. The following data types are affected: Customer Event, Dependency, Exception, Metric, Request , Page Load, Page View, Performance Counter & Trace.

Work Around: None
Next Update: Before 08:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.

-Application Insights Service Delivery Team

 
 
 
 
 

Skip to main content