Experiencing Data Latency for Many Data Types – 6/3 – Resolved


Final Update: Wednesday, 6/3/2015 20:42 UTC

We’ve confirmed that all systems are back to normal with no customer impact as of 6/3, 19:50 UTC. Our logs show the incident started on 6/3, 12:50 UTC and that during the 7 hours that it took to resolve the issue some customers experienced latency for Dependency and Request data.

Root Cause: The failure was due to an edge-case bug while writing to our back-end service.
Lessons Learned: We are working on a patch to adjust the way we handle this specific write condition to avoid the bug in the future.
Incident Timeline:  7 Hours & 0 minutes - 6/3, 12:50 UTC through 6/3, 19:50 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team


Update: Wednesday, 6/3/2015 17:41 UTC

Root cause has been isolated to latency writing to our back-end which was impacting our stream processing pipeline  To address this issue we have adjusted the stream processing infrastructure. All processing is now working as expected. Some customers may experience latency in Dependency and Request data and we estimate several hours before all latency is addressed.

Work Around: none
Next Update: Before 20:00 UTC

-Application Insights Service Delivery Team


Initial Update: Wednesday, 6/3/2015 16:25 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience Data Latency. The following data types are affected: Dependency, Request.

Work Around: none
Next Update: Before 18:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.

-Application Insights Service Delivery Team

 
 

Skip to main content