Experiencing Alerting failure for Availability Data Type – 10/27 – Resolved


Final Update: Friday, 28 October 2016 01:16 UTC

We’ve confirmed that all systems are back to normal with no customer impact as of 10/28 1:00 UTC. Our logs show the incident started on 10/27, 19:00 UTC and that during the 18 hours that it took to resolve the issue a subset of customers who created their apps after 10/26 19:00 UTC would have seen their alerts in disabled state. The workaround now is to delete and recreate the alert.
  • Root Cause: Complete root cause pending from the team.
  • Lessons Learned:  We are working on adding additional telemetry and improvements in the service from the learnings we took from the incident
  • Incident Timeline:  17 Hours & 56 mins 10/27 19:00 UTC through 10/28 00:56 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Vamshi


Update: Thursday, 27 October 2016 20:04 UTC

Root cause has been isolated to a change in one of our service  which was impacting alert feature. To address this issue we are working on rolling out a fix for the affected service. Some customers may still see alerts in disabled state when they create alerts for new apps.  Please note that apps created before 10/17/2016 continue to work and are not affected by this issue.
  • Work Around: None
  • Next Update: Before 10/28 01:30 UTC

-Vamshi


Initial Update: Thursday, 27 October 2016 17:34 UTC

We are aware of the issues within Application Insights and are actively investigating. Only the customers who create
new apps and then create new webtests for that app will see the alert in
disabled state. Initial investigation looks like a change in our dependent service may have caused
this. 

  • Work Around: None
  • Next Update: Before 10/27 20:00 UTC

We are working hard to resolve this issue and apologize for any inconvenience.
-Sapna


Skip to main content