GitHub blames its seven-hour August 17 outage on overloaded infrastructure

On August 17, GitHub was down for 7 hours and 47 minutes. The outage affected github.com, authentication, GitHub Actions, the API, work with pull requests, issues and Copilot — hurting developers and organizations around the world. The company's CTO, Vladimir Fedorov, admitted: “If you were trying to ship your product that day, we let you down.”

This was already the second serious incident in August — on the 6^th, GitHub Actions went down. According to Fedorov, neither outage was caused by changes to code or configuration. In both cases the cause was a capacity shortfall: critical components failed to scale before demand outstripped their capabilities.

What happened

The August 17 outage began when traffic hit a new peak and a key infrastructure component in the Central US data center couldn't scale to keep up with the load. The resulting pressure on capacity spread across systems, triggering authentication failures. Recovery proceeded in stages: teams rerouted traffic, isolated the affected infrastructure and gradually brought services back. Copilot took the longest to recover — errors in it set off a cycle of client-side retries, which drove traffic up during recovery itself, an effect that had to be damped down separately.

Fedorov attributes the rise in load to the sheer scale of the service: since April, the number of monthly commits has grown from 1.4 to 2.9 billion. “That explains the pressure on our systems, but it doesn't excuse the outages,” he stresses.

What has already been done

As part of previously stated reliability commitments, GitHub has added more than 3 million CPU cores, 120 petabytes of high-speed storage and significantly increased network capacity — installing as much hardware as the power limits of existing data centers allowed, while also accelerating the migration to Azure. Today Azure handles about 58% of the platform's load and half of all Git operations, up from 12% in May.

The next milestone is an architecture in which read capacity grows linearly with the number of readers, removing constraints on read operations. It will be rolled out gradually, starting with the largest monorepositories.

Following both August incidents, two immediate changes have been made. First, service-to-service interactions are getting unified limits and retry “budgets,” along with variable timeouts — to prevent retry “storms” and cascading load. Second, the company is reviewing low-priority CPU and memory alerts to find components that could fail under sudden traffic spikes. In parallel, GitHub is isolating critical systems and removing shared dependencies between them.

“We'll earn trust through the platform's scalability and reliability,” Fedorov concludes.

Source: GitHub Blog