Release Pipelines failing intermittently for Azure App Service Deploy task in West Europe – 10/17 – Mitigated


Final Update: Wednesday, October 17th 2018 20:14 UTC

We’ve confirmed that all systems are back to normal as of 2018/10/17 19:35 UTC. Our logs show the incident started on 2018/10/17 13:42 UTC and that during the 5 hours and 53 minutes that it took to resolve the issue. Customers using Azure App Service Deploy task in their Release Pipelines might see intermittent errors like: "##[error]TypeError: Cannot read property 'scmUri' of undefined". Sorry for any inconvenience this may have caused.

  • Root Cause: Initial diagnosis from the Azure Web Apps team points to issues with a few of their back end nodes sending bad api responses.
  • Chance of Re-occurrence: Medium
  • Lessons Learned: We are working both minimizing resource-intensive activities in our post-deployment steps, and are also working targeting monitors specifically to detect post-deployment issues in the future.
  • Incident Timeline: 5 hours & 23 minutes – 2018/10/17 13:42 UTC through 2018/10/17 19:35 UTC

Sincerely,
Randy


Update: Wednesday, October 17th 2018 18:36 UTC

Our DevOps team continues to investigate issues with the Azure Web Apps deployment task. We are actively working with the Azure Web App team. Root cause is not fully understood at this time. The problem began at 05:00 UTC 10/17/2018. We currently have no estimated time for resolution.

  • Work Around: Users can retry the deployment, as the issue seems to be intermittent
  • Next Update: Before Wednesday, October 17th 2018 20:45 UTC

Sincerely,
Randy


Update: Wednesday, October 17th 2018 17:39 UTC

Our DevOps team continues to work with the Azure Web Apps team to investigate issues with deployments to Azure Web Apps in Azure Pipelines. We currently have no estimated time for resolution.

  • Work Around: Users can retry the deployment, as the issue seems to be intermittent
  • Next Update: Before Wednesday, October 17th 2018 18:45 UTC

Sincerely,
Dexter


Update: Wednesday, October 17th 2018 16:17 UTC
Our DevOps team continues to investigate issues with deployments to Azure Web Apps using Azure Pipelines with both Private and Hosted Agents. Root cause seems to be an issue with a particular Get call within Azure Web Apps, and they are working to identify the underlying cause. The problem seems to have begun at approximately 05:00 UTC.

  • Work Around: Users can retry the deployment, as the issue seems to be intermittent
  • Next Update: Before Wednesday, October 17th 2018 17:30 UTC

Sincerely,Dexter


Update: Wednesday, October 17th 2018 15:20 UTCOur DevOps team continues to investigate issues with deployments to Azure Web Apps using Azure Pipelines with both Private and Hosted Agents. Root cause is not fully understood at this time, we are currently working with the Azure Web App team to investigate further. The problem began in West Europe at approximately 05:00 UTC. We currently have no estimated time for resolution.

  • Work Around: Retrying the deployment seems to sometimes work
  • Next Update: Before Wednesday, October 17th 2018 16:30 UTCSincerely,Dexter

Initial notification: Wednesday, October 17th 2018 14:05 UTCWe're investigating Intermittent failures of Release Pipelines in West Europe.

  • Customer using Azure App Service Deploy task in their Release Pipelines might see intermittent errors like: "##[error]TypeError: Cannot read property 'scmUri' of undefined"
  • Retrying the release may succeed.
  • Next Update: Before Wednesday, October 17th 2018 15:50 UTCSincerely,Ariel Jacob

Skip to main content