Experiencing New Application Creation Issue in Azure Portal - 9/29 - Resolved


Final Update: Wednesday, 9/30/2015 18:19 UTC

Our DevOps team has finished the provisioning for the affected applications.  There is no change for customers who implemented the workaround.

Root Cause: The failure was due to a bug in a mitigation deployed for a previous incident which caused provisioning transactions to partially complete.  
Lessons Learned: We are investigating telemetry options for catching provisioning failures for this specific mode, where a customer is provisioned in one half of the system, but missing in the second half.
• Impact Window for Partial App Creation: 9/29, 05:47 UTC through 9/29, 16:51 UTC

• All Affected Customer Applications Restored: 9/30, 18:15 UTC

We understand that customers rely on Application Insights as a critical service and apologize for any impact this incident caused.

-Application Insights Service Delivery Team


Update: Tuesday, 9/29/2015 23:33 UTC

Our DevOps team is still working on the tooling to finish provisioning for the affected applications.  We expect this work will take several more hours.

In the meantime, customers who cannot wait are encouraged to investigate the workaround below.

Work Around: Customers can delete and re-create their application, then re-instrument their application for the new iKey.
Next Update: Before 9/30, 17:00 UTC

-Application Insights Service Delivery Team


Update: Tuesday, 9/29/2015 18:48 UTC

Our DevOps team is testing and validating the process to finish the provisioning pipeline for affected applications.

Work Around: Customers can delete and re-create their application, then re-instrument their application for the new iKey.
Next Update: Before 21:00 UTC

-Application Insights Service Delivery Team


Initial Update: Tuesday, 9/29/2015 17:18 UTC

We have discovered a bug in the mitigation deployed for a previous incident which caused any apps created since 9/29, 05:47 UTC to be partially created.  The application would appear in the portal, but is unable to accept data.  This bug was fixed at 9/29, 16:51 UTC and any applications created since then have been verified to accept data.

We are working on steps to recover applications created during the impact window and finish the provisioning process on the back end.  In the meantime, customers can delete and re-create their application to work around the issue.  This will result in the generation of a new iKey, and customers will then have to reconfigure their application to send data using the new iKey.

• Impact Window for Partial App Creation: 9/29, 05:47 UTC through 9/29, 16:51
Work Around: Customers can delete and re-create their application, then re-instrument their application for the new iKey.
Next Update: Before 19:00 UTC

-Application Insights Service Delivery Team

 
 
 

Skip to main content