Data Access issue with Query Services – 2/11 – Resolved


Final Update: Wednesday, 2/11/2015 10:17 UTC

We’ve confirmed that all systems are back to normal with no customer impact as of 2/11, 10:06 UTC.  

Root Cause: The failure was due to a bug in a maintenance job which cached specific types of metatdata for presentation to users in the Azure portal.  No data was lost or latent; the issue was purely at the display layer.
Chance of Reoccurrence: The underlying bug in this job has been fixed.  We do not expect to see a recurrence of this issue.
Lessons Learned:  The DevOps team is evaluating updated monitoring and trace logs to detect and diagnose this type of issue more quickly in the future.
Incident Timeline: 110 Hours & 10 minutes – 2/6, 20:09 UTC through 2/11, 10:06 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team


Initial Update: Wednesday, 2/11/2015 09:33 UTC

We are actively investigating issues with Query service. Some customers may experience missing metric data in the Overview blade of the Windows Azure portal (http://portal.azure.com).

We have identified the issue as a bug in a maintenance job causing some types of metric metadata to not be displayed properly for some users when queried through the overview blade. To address this issue we are deploying a hotfix to the maintenance job.

This bug was introduced approximately 2/6/2015 20:00 UTC and only affects some customers who have deployed a new application within Application Insights since then.

Work Around: none
Next Update: Before 12:00 UTC

-Application Insights Service Delivery Team

 



Skip to main content