Data center operator OVH says a fire destroyed some of its sites in France, affecting cyber threat intelligence company Bad Packets, game maker Rust, and others
Ax Sharma / BleepingComputer :
BleepingComputerAx Sharma
Context & Ripple Effects
The disruption puts OVH alongside a broader set of infrastructure-provider failures in the coverage: Rackspace’s ransomware recovery led it to discontinue Exchange, while Vodafone Portugal’s cyberattack impaired services beyond the company itself. The common issue is customer exposure when a shared operator loses service capacity.
For France, the incident is a physical-infrastructure counterpart to the country’s earlier wave of denial-of-service attacks on French websites: different causes, but both show how concentrated service dependencies can propagate an outage across unrelated users.
First-order effects
OVH customers including Bad Packets and Rust lose services tied to the French sites destroyed by the fire, making recovery and restoration OVH’s immediate operational priority.
Bad Packets and Rust must manage the interruption as a dependency on OVH’s affected facilities rather than as a failure of their own applications.
Second-order effects
The outage gives OVH customers a concrete reason to scrutinize geographic redundancy and recovery arrangements with hosting providers.
Competing infrastructure providers can use separation across facilities as a differentiator for customers whose services were concentrated in the affected OVH sites.
Third-order effects
Repeated service disruptions—from fires to cyberattacks—raise the strategic value of architectures that avoid a single provider or site becoming a common point of failure.
As France expands domestic digital infrastructure, resilience of the underlying facilities becomes as consequential as sovereignty or available computing capacity.
The trend: Digital-service resilience is becoming a data-center design and procurement issue, as physical and cyber incidents at shared providers cascade to many customers.
We have a major incident on SBG2. The fire declared in the building. Firefighters were immediately on the scene but could not control the fire in SBG2. The whole site has been isolated which impacts all services in SGB1-4. We recommend to activate your Disaster Recovery Plan.
Update: We've confirmed a total loss of the affected EU servers during the OVH data centre fire. We're now exploring replacing the affected servers. Data will be unable to be restored.
Out of the 140 known C2 servers we are tracking at OVH that are used by APT and sophisticated crime groups, approximately 64% are still online. The affected 36% include several APTs: Charming Kitten, APT39, Bahamut and OceanLotus.
Update 7:20am Fire is over. Firefighters continue to cool the buildings with the water. We don't have the access to the site. That is why SBG1, SBG3, SBG4 won't be restarted today.
Update 4pm We plan to restart SBG1+SBG4+the network by Monday March,15 and SBG3 by Friday March,19. In RBX+GRA we have the stock of new servers, pcc, pci ready to be delivered for all the impacted customers. Of course for free. We will add 10K servers in the next 3-4 weeks.
Update 1pm Plan for the next 1-2 weeks: 1) rebuilding 20KV for SBG3 2) rebuilding 240V in SBG1/SBG4 3) verifying DWDM/routers/switchs in the network room A (SBG1). checking the fibers Paris/Frankfurt 4) rebuilding the network room B (in SBG5). checking fibers Paris/Frankfurt
🔥🖥️🔥 Several of the Lichess servers burnt down in a datacenter fire tonight. No-one was harmed. Thanks to our zealous sysadmin and his backups, all we lost was 24h of puzzle history. Read more about the disaster: https://www.datacenterknowledge.com/ ...
Update 11:20am All servers in SBG3 are okey. They are off, but not impacted. We create a plan how to restart them and connect to the network. no ETA. Now, we will verify SBG1.
It just hit me that the OVH datacenter fire likely permanently killed a lot of infosec resources (blogs, tutorials, etc.) that will only live on in https://archive.org/ (if anywhere). Sad day :(
Let's be brutally honest, it has been a tough 10 days for defenders so some good news is welcome. Wonder how disaster recovery is going for APT groups https://twitter.com/...
According to Netcraft, more than 3.6 million sites went down today after the OVH data center fire. More than 18% of the IP addresses attributed to OVH also stopped responding. https://news.netcraft.com/... https://twitter.com/...
This is one of the saddest threads I have seen in a while. So many system admins who didn't bother to back up any data off-site. It's all gone now. https://twitter.com/...
OVH has finally decided to fight bad actors in their infrastructure and they are NOT fucking around! 🔥 More seriously, good luck to all the teams* affected by this 😘 *(not including attackers) https://twitter.com/...
An entire OVH data centre was destroyed in a fire - just a reminder to have a DRP, to have off-site backups / sync. You can't rely on someone else's infrastructure to be the sole copy of your data. https://twitter.com/...
Sucks for Rust and others affected, but this is a great example of why people should demand a proper offline mode for @Hitman. Sometimes shit happens, but there's no good reason for a single player game to stop functioning because the servers burned down. https://twitter.com/...
a lot of questions asking why OVH's fire suppression system did not solve this. usually in a catastrophic datacenter fire, the origin is the power distribution system, like a trunk PDU arc fault (which is usually explosive), so there's not much a fire suppression system can do.
Update 10am. (I'm there). We finished to shutdown the UPS in SBG3. Now they are off. We are looking to enter into SBG3 and check the servers. The goal is to create a plan to restart , at least SBG3/SBG4, maybe SBG1. To do so, we need to check the network rooms too.