Experiencing Data Latency for Many Data Types – 01/28 (second occurrence) – Resolved

Final Update: Friday, 29 January 2016 04:37 UTC

We've confirmed that all systems are back to normal with no customer impact as of 01/29, 04:20 UTC. Our logs show the incident started on 01/28, 23:08 UTC and that during the 5 Hours 28 minutes that it took to resolve the issue 10% of customers experienced data gaps for multiple data types.
  • Root Cause: The failure was due to an exception in Application Insights pipeline.
  • Lessons Learned: Hot fix has been deployed to handle exceptions of this kind and full RCA regarding these exception is going on.
  • Incident Timeline: 5 Hours 28 minutes - 01/28, 23:30 UTC through 01/29, 04:20 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team

Update: Friday, 29 January 2016 02:47 UTC

Root cause is not fully understood at this moment however as part of mitigation we have applied a hotfix  to catch up with backlog data. Processing of backlog data may take few more hours to complete, till then very small subset of customers may continue to experience gaps in thier historic data. 
  • Work Around: none
  • Next Update: Before 01/29 07:00 UTC

-Application Insights Service Delivery Team

Initial Update: Thursday, 28 January 2016 23:08 UTC

We are aware of issues within Application Insights and are actively investigating. Some customers may experience data gaps while viewing the telemetry data.  The following data types are affected: Availability,Customer Event,Dependency,Exception,Metric,Page Load,Page View,Performance Counter,Request,Trace. Recent data is processed successfully without any issue however data for  some old time period needs to be back filled.

  • Work Around: None
  • Next Update: Before 01/29 03:30 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Application Insights Service Delivery Team

Skip to main content