Microsoft has attributed a prolonged Microsoft 365 disruption to a software defect introduced months earlier in network gateway servers, according to details from the company's preliminary post-incident report. The flaw remained dormant after a July deployment before a combination of higher traffic and temporarily reduced capacity triggered a fleet-wide failure in November.
The affected gateway infrastructure supports Exchange Online and other cloud services. Microsoft's account said a Teams failover increased traffic through the primary gateway used by Exchange Online while nearly 14 percent of gateway capacity was unavailable. Neither condition was expected to exhaust the service on its own, but together they exposed lock contention inside a Windows process. CPU and memory resources were then consumed as servers became unresponsive.
The problem was particularly acute on newer systems with more processor cores. Microsoft acknowledged that its load testing had concentrated on weaker hardware with fewer cores, allowing the defect to pass predeployment checks. As individual servers lost connectivity, traffic shifted to other endpoints, which encountered the same contention. The resulting cascade spread across the fleet in roughly two hours; at the lowest point, more than 80 percent of servers were unresponsive.
Standard recovery procedures could not quickly restore service. Servers brought back online were overwhelmed again, while the age of the July change made a conventional rollback impractical. Engineers instead created an update that returned the affected code path to its previously working state, then deployed that mitigation in groups while capacity and traffic controls were adjusted.
The timeline began with the Teams failover at 01:07 UTC on November 12. Monitoring detected rising connectivity problems at 16:03 UTC. Microsoft paused deployments, added capacity and rerouted traffic, but Exchange Online could not leave the affected gateway at the same scale as services including Teams, OneDrive, SharePoint and Office applications. Engineers narrowed the problem to connection negotiation after midnight on November 13 and began applying the corrective update at 07:20 UTC.
Microsoft reported healthy conditions by 16:10 UTC that day and issued its final customer update early on November 14. A fuller post-incident report was scheduled for November 20.
The preliminary findings point to two engineering gaps: tests that did not reflect the behavior of newer high-core-count servers, and automated recovery systems that were not designed for such a broad simultaneous failure. The incident also illustrates how individually manageable conditions—routine failover traffic and a temporary capacity reduction—can interact with latent code defects to cause a much larger service interruption.



