GitHub says its 7+ hour August 17 outage was caused by a capacity failure when peak traffic overwhelmed an infrastructure component in a Central US data center
An update on the August 17 outage and the steps we're taking to improve reliability. — On August 17, GitHub experienced an outage that lasted 7 hours and 47 minutes.
The GitHub Blog Vlad Fedorov
Context & Ripple Effects
GitHub had already framed reliability as a priority amid AI-driven growth, putting availability ahead of capacity and new features in its earlier reliability response. The August incident turns that priority into a concrete capacity-planning test: a Central US infrastructure component was overwhelmed at peak traffic.
Initial coverage established that the disruption reached GitHub’s website, API, Actions and pull-request services before mitigation was reported. The later root-cause update matters because it identifies capacity, rather than an external attack, as the operational issue to correct.
First-order effects
- GitHub must focus its reliability work on the Central US component and the capacity controls around peak traffic, after an outage lasting 7 hours and 47 minutes.
- Developers and organizations dependent on GitHub’s API, Actions and pull-request workflows were directly exposed to a single infrastructure-capacity failure.
Second-order effects
- Teams that build deployment and collaboration processes around GitHub services have a stronger incentive to design fallbacks for interruptions across code hosting, APIs and automation rather than treating those functions as independent.
- GitHub’s stated ordering of availability, capacity and features gains operational weight: capacity work competes more directly with feature delivery for engineering attention.
Third-order effects
- If traffic growth continues to expose concentrated components, developer platforms will increasingly treat capacity engineering as a product constraint rather than a back-end scaling task.
- The episode fits a broader shift in which AI-linked demand growth makes reliability depend on how quickly infrastructure capacity and fault isolation keep pace with usage.
The trend: AI-era demand is pushing software platforms to make capacity resilience a first-order product and operations priority.
Related: Compute as a product constraint · AI infrastructure supercycle · GitHub · GitHub’s reliability priorities amid AI growth · GitHub outage mitigation report
Related Coverage
- GitHub traces 7-hour outage to critical infrastructure failure: Here's what we know The Indian Express
- GitHub outage had users weighing options, but finding few TechTarget · Beth Pariseau
- GitHub blames capacity failure and retry storm for nearly eight-hour outage RuntimeWire · Ryan Merket
- GitHub now sees 2.9 billion commits a month — and it can't keep up The New Stack · Frederic Lardinois
- GitHub outage blamed on misconfigured policy as firm pledges resilience improvements ITPro · Nicole Kobie
- The August 17 outage Hacker News
- ‘We let you down’: GitHub pledges to scale up before developers give up The Register · Richard Speed
- GitHub reliability and scalability: a consequence of AI and too much for free ITWriting.com · Tim Anderson
Discussion
-
@abyrd_9
Andrew Byrd
on x
If I read this sentence to a medieval peasant he would report me to the church and I would be burned at the stake as a wizard or sumn
-
@acolombiadev
Andrea
on x
I work at GitHub. yesterday was rough and i'm not pretending otherwise. full root cause report is up if you want the timeline and numbers, and what we are doing to prevent this from happening again. https://www.githubstatus.com/ ...
-
@thehungrybird_
Aryan
on x
@Vlad_GitHub After seeing this graph, it feels like we are not appreciating folks at GitHub for what they deserve.
-
@sethrubenstein
Seth Rubenstein
on x
It's too late. Yes the 17th was all day; but this has been a problem for a while. I pay for a teams account. My devs couldn't work. You bifurcate enterprise from the main product, if free repos are part of our core offering then you need to isolate paid teams accounts. You can
-
@dodyg
@dodyg
on x
This is the reason why Codeberg banned AI projects. Many people were very critical of that decision. If they didn't band AI projects, they will face the same issue as GitHub. > Since April, monthly commits have grown from 1.4 billion to 2.9 billion. https://github.blog/...
-
@nickgeracehacks
Nick Gerace
on x
Ouch! That definitely did not help. > Copilot Token Service traffic increased from a normal 7-9K RPS to 70-100K RPS
-
@jackellis
Jack Ellis
on x
“We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity” 😳
-
@kdaigle
Kyle Daigle
on x
Our CTO shared a blog on GitHub's reliability, what we've already done, and what more we need to do to. Monthly commits and merged PRs alone have nearly doubled since April. As the home for devs, availability is critical. I'm sorry for the pain and thankful for the support. 🙏
-
@daniellockyer
Daniel Lockyer
on x
Some crazy graphs of PR/commit growth on GitHub since 2023
-
@rhyssullivan
Rhys
on x
I really don't think any of us have internalized just how much software development has changed in these few years, truly insane numbers
-
@perrymetzger
Perry E. Metzger
on x
So, this is the wrong set of repairs. I've seen clients of mine make the same mistake time and again. You see a particular failure and you decide “I'll audit that particular configuration mistake better in the future!” The correct response is to make sure that you're doing
-
@hd_nvim
@hd_nvim
on x
Seeing this graph reminds me that storing code for free is a privilege, especially in the AI era. Putting a rate limit/storage limit on free users makes sense for platform usability.
-
@edandersen
Ed Andersen
on x
Had 10 years to port the Ruby codebase to C#
-
@grantjordan
Grant Jordan
on x
@acolombiadev “I'm not pretending otherwise”, said Claude
-
@ikhwanuddin
@ikhwanuddin
on x
Do people still use VS Code with Github Copilot? I find it bloated, which is why I switched to @zeddotdev and @pidotdev
-
@whatranjusaid
Ranju
on x
GitHub's yesterday's outage RCA report Misconfigured policy
-
@0xblacklight
@0xblacklight
on x
fascinating read btw
-
@dbreunig
Drew Breunig
on x
This is insane demand during a time when infra is very expensive. Would-be competitors better have a very good plan and very, very deep pockets.
-
@1rjfinnegan
RJ Finnegan
on x
@acolombiadev Most ppl complaining are on a free account with half baked projects that weren't going anywhere - me included - but appreciate the transparency
-
@thedavidtai
@thedavidtai
on x
So it was self amplification as services responded to saturation induced downtime by retrying calls with aggressively until the retries did a running ddos against the platform.
-
@vlad_github
Vlad F
on x
On August 17, GitHub experienced a significant outage that disrupted developers and organizations around the world. If you were trying to ship software that day, we let you down. I posted in March and April about the steps we're taking to make GitHub more reliable. The work is
-
@yordisprieto
Yordis Prieto
on x
For everyone criticizing GitHub, go ahead, show me the scaling situation you actually deal with. THIS is exactly why, although, yes, please be professional and do your best, it is NOT acceptable to break production. Fine. Nevertheless, these are not trivial problems to face,
-
@kevvoh_
@kevvoh_
on x
A misconfigured policy is what took GitHub down. Engineering is funny sometimes.
-
@ai_for_success
AshutoshShrivastava
on x
if you want to understand how much AI has accelerated software development, look at what's happening on GitHub. Monthly commits have jumped from 1.4 billion to 2.9 billion since April 2026. That's more than 2× growth in just a few months. The amount of code being written is
-
@ahmedehab_01
@ahmedehab_01
on x
What Andrea here didn't mention is why a VS Code retry bug was introduced so easily in their system. They really should try to look into their engineering practices and relax the feature pushing to improve GitHub's health.
-
@kevincodex
Kevin
on x
this graph says it all if agentic coding is driving this much growth among developers today, imagine the demand for agentic infrastructure when it reaches non-technical users that is the massive market we're building for at @gitlawb, and this is only the beginning
-
@thepghcid
@thepghcid
on x
@acolombiadev The first sentence, and in guessing most of this, was written by some variant of Claude.
-
@mehulmpt
Mehul Mohan
on x
Microsoft owns GitHub for almost 8 years now but only 12% workload was on Azure even in May this year I wonder what took so long (and how it got bumped almost to 60% in 3 months)
-
@ethanclinick
Ethan Clinick
on x
Imagine showing this to a senior dev 10 years ago. What a time to be alive. Thank you @Microsoft and @github for eating these costs.
-
@brsanthu
Santhosh
on x
@Vlad_GitHub These numbers are so astronomical that I believe Github cannot support current free offerings as is (I'm benefiting from the free offerings). I think we should expect users to pay beyond some limit (like 10) private repos. Also time to think about pricing by repo act…
-
@acolombiadev
Andrea
on x
our CTO @Vlad_GitHub wrote up the August 17 outage, good to read in full. tldr: outage was not due to code or config changes. both were capacity failures, we didn't scale ahead of demand. that's on us. we'll earn trust back through the platform actually holding up, not through
-
@v_jug
Vjekoslav Jug
on x
@acolombiadev “VS Code bug amplified GH traffic by roughly 10 times”. Wow. Does people still use VS Code that much? It became pretty bloated with not so great AI features (I mean they are not so bad if you haven't seen other tools).
-
@vlad_github
Vlad F
on x
@thehungrybird_ The team is working very hard to make sure we are there for you. Scaling at this rate and rebuilding at the same time for new agentic load is not simple
-
@ibuildthecloud
Darren Shepherd
on x
It's always, “these five random things failed at once in ways we never thought of. So to fix it we are going to ensure these five random things wont ever fail in this exactly way again.”
-
@stanfordnlp
@stanfordnlp
on x
Sounds like a pretty good advertisement for @BuoyantIO's linkerd!
-
@pamelafox
Pamela Fox
on x
Blog post with more details on the GitHub outage: 1) the load increase the servers are seeing 2) specific fixes made to prevent retry storm and cascading load 3) general scaling plans
-
@acolombiadev
Andrea
on x
To prevent recurrence, our follow-up actions include: - Correcting autoscaling policies to account for service-mesh sidecar concurrency and capacity. - Auditing Istio request, concurrency, and scaling limits across affected services. - Reviewing retry limits and backoff
-
@matteocollina
Matteo Collina
on x
Autoscaling is incredibly hard.
-
@acolombiadev
Andrea
on x
our CTO @Vlad_GitHub wrote up the August 17 outage, good to read in full. tldr: outage was not due to code or config changes. both were capacity failures, we didn't scale ahead of demand. that's on us. we'll earn trust back through the platform actually holding up, not through
-
@kylehugh.es
Kyle Hughes
on bluesky
I know it sounds crazy, and I've been saying it for a while, but I don't see how consumer technology will ever regain a foothold in manufacturing preference. Jevon's paradox seems to mean infinite commercial demand for compute. Every graph is straight up, including the marginal…
-
@phillipcarter.dev
Phillip Carter
on bluesky
Write-up on the giga-outage and more from GitHub. Every chart I have seen internally too is literally doing a hockeystick on growth. IMHO this is also a canary in the coalmine for many other systems and products who have an API github.blog/news-insight...
-
r/programming
r
on reddit
The August 17 outage, and the work ahead