Experiencing Data Latency for Many Data Types – 5/22/2015 – Resolved


Final Update: Saturday, 5/23/2015 01:03 UTC

We’ve confirmed that all systems are back to normal with no customer impact as of 5/23, 00:22 UTC. Our logs show the incident started on 5/22, 14:12 UTC and that during the impact windows following data types may have been latent

  • Dependency
  • Request
  • Trace

 

Root Cause: There was a master node that was hung and it made the member nodes to stop processing telemetry data.

Chance of Reoccurrence: Low

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team


Update: Friday, 5/22/2015 18:12 UTC

Processing service is continuing to recover telemetry data. All the data types except the following have recovered

  • Dependency
  • Request
  • Trace

 For the above data types customers may experience latency. We are working as fast as we can on the recovery but we don’t have an ETA on the final recovery. We’ll update this post as soon as data types recover.

Work Around: none
Next Update: Before 21:00 UTC

-Application Insights Service Delivery Team


Update: Friday, 5/22/2015 15:50 UTC

The processing services are now working as expected. Some customers may experience data latency for all data types and we estimate 2 hours before all latency is addressed.

Work Around: none
Next Update: Before 18:00 UTC

-Application Insights Service Delivery Team


Initial Update: Friday, 5/22/2015 15:12 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Latency. The following data types are affected: Availability, Customer Event, Dependency, Exception, Metric, Page Load, Page View, Performance Counter, Request, Trace.

Work Around: none
Next Update: Before 17:00 UTC

We are working hard to resolve this issue and apologize for any inconvenience.

-Application Insights Service Delivery Team

 
 
 

Skip to main content