[Post Incident Report] Dynamics AX sign-in issues and reporting functionality loss

Summary

On Friday, September 9, 2016, at 3:09 PM UTC, internal monitoring detected that customers in Western Europe were unable to sign in to Dynamics AX. Upon investigation, engineers discovered the outage was related to a network connectivity issue. Microsoft engineers applied a network configuration change, restoring service on Friday, September 9, 2016 at 5:48 PM UTC.

 

User Experience

Friday, September 9, 2016, from 3:09 PM UTC to 4:40 PM UTC:

Affected users were unable to sign in to Dynamics AX.

 

Friday, September 9, 2016, from 4:40PM UTC to 5:48 PM UTC:

Service was restored for all users, but reporting functionality was down for a subset of customers.

 

Customer Impact

All customers hosted in Western Europe

 

Incident Start Date and Time

Friday, September 9, 2016, 3:09 PM UTC

 

Date and Time Service was Restored

Friday, September 9, 2016, 5:48 PM UTC

 

Root Cause

A recent update to the network configuration caused the virtual machines (VMs) for Dynamics AX to become inaccessible in Western Europe.

 

Next Steps

Improve monitoring during updates to proactively address performance issues.

Improve redundancy for VM configurations to create a failsafe for region-wide network outages.