Errors accessing Work Items via Client OM event in all regions – 10/03 – Mitigated


Final Update: Thursday, October 4th 2018 05:53 UTC

We’ve confirmed that all systems are back to normal as of 4th October 04:02 UTC. The hotfix deployment is completed on all the rings. Our logs show that the incident started on 3rd October 4:40 UTC. Sorry for any inconvenience this may have caused.

  • Root Cause: We have identified that the issue is related to recent change that was deployed.
  • Chance of Re-occurrence: Low

Sincerely,
Rakesh Reddy


Update: Thursday, October 4th 2018 01:00 UTC

Our hotfix deployment to fix the issue has been progressing steadily across various rings and has got deployed successfully on half of the rings. It is expected to be complete across all regions/rings in another 12 hours.

  • Next Update: Before Thursday, October 4th 2018 13:00 UTC

Sincerely,
Manohar


Update: Wednesday, October 3rd 2018 19:52 UTC

We have narrowed down the issue to a change that recently deployed. We are working on a hotfix deployment to revert this change and we expect the roll out to be completed within the next 6 hours.

  • Next Update: Before Thursday, October 4th 2018 02:00 UTC

Sincerely,
Sri Harsha


Initial notification: Wednesday, October 3rd 2018 18:27 UTC

We're investigating a possible customer impacting event in all regions. We are triaging the issue and will provide more information as it becomes available.

Customers may experience error WorkItemTypeDeniedOrNotExistException when querying work items, via the Client OM. The impact appears contrained to only customers using custom Work Item Types.

  • Next Update: Before Wednesday, October 3rd 2018 19:27 UTC

Sincerely,
Mauro


Skip to main content