Degraded Performance Due to Azure Issue

Incident Report for Document Drafter

Resolved

This incident has been resolved.
Posted Oct 30, 2025 - 06:54 UTC

Update

On October 29, 2025, a global issue affecting Azure Front Door (AFD) caused service disruptions across multiple regions.

Our team implemented manual re-routing measures and successfully restored approximately 98% of our services by 9:00 PM CET on October 29. Microsoft resolved the underlying Azure Front Door problem at 1:00 AM CET on October 30, and all systems have since been fully operational.

We are closely monitoring performance and awaiting the official Post-Incident Report from Microsoft Azure to better understand the root cause and ensure long-term resiliency improvements.

Thank you for your patience and understanding during this incident.
Posted Oct 30, 2025 - 06:46 UTC

Update

As to Rest API and a small sub-set of customers still affected, we have the following update from Microsoft:

At this stage, we anticipate full mitigation within the next four hours as we continue to recover nodes. This means we expect recovery to happen by 23:20 UTC on 29 October 2025. We will provide another update on our progress within two hours, or sooner if warranted.
Posted Oct 29, 2025 - 19:26 UTC

Update

Services are back online except RestAPI and Power Automate,

We have pushed our ‘last known good’ configuration, and customers may begin to see initial signs of recovery. We are currently recovering nodes and routing traffic through healthy nodes, and as we make progress in this workstream, customers will continue to see improvement.

We will keep you updated.
Posted Oct 29, 2025 - 19:12 UTC

Update

Update from Microsoft:

We have initiated the deployment of our 'last known good' configuration. This is expected to be fully deployed in about 30 minutes from which point customers will start to see initial signs of recovery. Once this is completed, the next stage is to start to recover nodes while we route traffic through these healthy nodes.

We do not have an ETA for full mitigation, we will update this communication within 30 minutes, once the deployment is completed.
Posted Oct 29, 2025 - 18:22 UTC

Update

We are continuing to investigate this issue.
Posted Oct 29, 2025 - 18:22 UTC

Update

We have a new update from Microsoft

We are taking several concurrent actions: Firstly, where we are blocking all changes to the AFD services, this includes customer configuration changes as well. At the same time, we are rolling back our AFD configuration to our last known good state. As we rollback we want to ensure that the problematic configuration doesn't re-initiate upon recovery.

We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update.

This message was last updated at 17:40 UTC on 29 October 2025
Posted Oct 29, 2025 - 17:43 UTC

Update

We are continuing to investigate this issue.
Posted Oct 29, 2025 - 17:43 UTC

Update

New update from Microsoft;

We have failed the portal away from Azure Front Door (AFD) to mitigate the portal access issues. Customers should be able to access the Azure management portal directly.

We do not have an ETA for when the rollback will be completed, but we will update this communication within 30 minutes or when we have an update.

This message was last updated at 17:14 UTC on 29 October 2025
Posted Oct 29, 2025 - 17:19 UTC

Update

Update from Microsoft:

Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal and services. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or soon.
Posted Oct 29, 2025 - 16:41 UTC

Investigating

We are currently experiencing degraded response times originating from an issue with our Azure infrastructure. This latency is causing the service to be unavailable or partially unavailable for some users.

Our engineering team is actively investigating the root cause and working toward remediation.

We will provide the next status update in no more than 1 hour.
Posted Oct 29, 2025 - 16:00 UTC
This incident affected: Global (REST API, Document Drafter Portal), Europe 1 (API Services - EU), Europe 2 (API Services - EU 2), US East (API Services - US East), Canada Central (API Services - Canada Central), and Switzerland North (API Services - Switzerland North).