BSODs hit thousands of Windows PCs due to “a defect” in an update from CrowdStrike, taking banks, airlines, and more businesses offline; Microsoft is aware
or CrowdStrike Microsoft : HELPFUL LINKS — Get notified of outages that impact you — Building reliable applications on Azure The Record : IT teams scramble to recover from CrowdStrike incident as officials warn of ‘risks of consolidation’ Jason Kent / Security Boulevard : Blue Screened: Microsoft Windows Computers Crashed by Automated CrowdStrike Update Mauro Huculak / Pureinfotech : What is CrowdStrike Blue Screen of Death (BSoD), and how do I fix it on Windows 11, 10? The outage recovery. Manish Singh / TechCrunch : Faulty CrowdStrike update causes major global IT outage, taking out banks, airlines and businesses globally Reuters : Global cyber outage grounds flights and disrupts businesses NBC News : Mass IT outage hits airports, businesses and broadcasters around the world Kevin Purdy / Ars Technica : Major outages at CrowdStrike, Microsoft leave the world with BSODs and confusion Simon Sharwood / The Register : CrowdStrike code update bricking Windows machines around the world BBC : CrowdStrike and Microsoft: What we know about global IT outage Rob Pegoraro / PCMag : Prior to Microsoft Meltdown, CrowdStrike Exec Warned of ‘Single Point of Failure’ The Guardian : Slow recovery from IT outage begins as experts warn of future risks Li Zhou / Vox : The “largest IT outage in history,” briefly explained Reuters : Outage for Microsoft Users Knocks Out Systems for Airlines and Hospitals in Chaotic Day Frances Vinall / Washington Post : Massive Windows outage brings ‘blue screen of death’ errors Abdullah / Gizchina : Elon Musk Mocks Microsoft Windows Servers Crashing After Crowdstrike Update Financial Times : Large security software vendors could face greater political and regulatory scrutiny after the CrowdStrike outage showed how they can damage the global economy Jak Connor / TweakTown : Microsoft CEO responds to millions of Windows PCs blue screening globally Peter Sayer / CSO : Blue screen of death strikes crowd of CrowdStrike servers Brinley Hineman / FreightWaves : Business as usual at many of nation's ports despite global tech outage Sky News : Global IT outages latest: Security firm CrowdStrike rules out cyber attack as world copes with tech ‘disaster’ Alyssa Goard / NBC Bay Area : Global IT outage highlights societal reliance on big tech companies Rory Tingle / Daily Mail : A Great Northern train at Hunt's Cross station in Liverpool today. The operator is among those experiencing problems Jacob Lyon / Protos : Crypto appears unaffected by global CrowdStrike outages WRIC-TV : ‘We still have a ways to go:’ Global IT outage disrupts millions worldwide, affecting essential services James Pearson / Reuters : Without backup plans, global IT outages will happen again Jordan Hart / Business Insider : Why Apple doesn't suffer outages like this Associated Press : Microsoft users worldwide report widespread outages affecting banks, airlines and broadcasters Connor Jones / The Register : CrowdStrike shares sink as global IT outage savages systems worldwide The Guardian : Queues and blank screens at airports as Microsoft IT outage disrupts travel - video NBC News : What we know about the global Microsoft outage Bloomberg : Hospitals Around the World Hobbled by Massive Tech Outage Wall Street Journal : Global IT Outage Hits Banks, Airlines and Emergency Services CNBC : Microsoft users hit with ‘blue screen of death’ after massive outage ABC News : United, Delta and American Airlines issue global ground stop on all flights Reuters : Cyber outage hits government, media and telcos across Australia and New Zealand WTRF-TV : Faulty software update causes havoc worldwide for airlines, hospitals and governments Reuters : Microsoft cloud outage causes airlines to ground flights Andrew Griffin / The Independent : World brought to a halt by drastic Microsoft outage with computers showing ‘blue screen of death’ Govind Choudhary / Livemint : Microsoft global outage: IT Minister Ashwini Vaishnaw says reason identified; CERT issues technical advisory Cate Lawrence / Tech.eu : Widespread Windows outage cripples businesses, airlines, the London Stock Exchange and broadcasters Les Pounder / Tom's Hardware : Global IT issue strikes Windows machines, cause now linked to CrowdStrike software update Mike Viray / Phandroid : Massive Outage Hits Microsoft Services Worldwide John Furrier / SiliconANGLE : World reboots as CrowdStrike update cripples global systems running Microsoft Windows Alexei Oreskovic / Fortune : The ghost of Y2K just struck back Samantha Dorisca / AfroTech : CrowdStrike Sparks Global Tech Outage That Has Impacted Businesses, Airlines, And Emergency Services Across The Globe Pranav Dixit / Business Today : American Airlines grounds flights nationwide following Microsoft Cloud outage chaos CNBC : CrowdStrike issue causes major outage affecting businesses around the world Wall Street Journal : Microsoft Reports Major Service Outage Affecting Users Worldwide Reuters : Global outage hits cybersecurity firm Crowdstrike: report William Gallagher / AppleInsider : Global chaos erupts as Windows security update goes bad Barry Collins / Forbes : Huge Windows Blackout Hits Banks, Airports And More Cailley LaPara / Bloomberg : FedEx, UPS Warn of Delivery Delays From Global Tech Outage Politico : Global tech outage takes down banks, media, airports Shaurya Malwa / CoinDesk : CrowdStrike, Windows BSoD Meme Tokens Populate Solana and Ethereum Amid Global Internet Outage Alba Mora / Xataka On : What Does CrowdStrike Do? Here's What You Need to Know About Microsoft's Cybersecurity Provider, Suspected of Causing a Global IT Outage Larry Dignan / Constellation Research : CrowdStrike outage likely to hit cybersecurity's platformization pitch Shubham Sharma / VentureBeat : Crowdstrike's faulty update triggers global Windows blackout, disrupting critical operations Savannah Fortis / Cointelegraph : Microsoft Windows data blackout downs major financial firms, causes travel chaos Julien Pretot / Reuters : Paris 2024 says its IT systems hit by global cyber outage Old Mermaid Town : Deborah Pickett (@futzle@old.mermaid.town) Rebecca Falconer / Axios : U.S. flights grounded as outages disrupt businesses around the world Bloomberg : From ATMs to Flights, Epic IT Crash Leaves Trail of Chaos Kabc / ABC7 : Network issues linked to Microsoft systems causing outages at businesses worldwide Financial Times : Companies around the world hit by Microsoft outage New York Post : Flights grounded and travelers stranded — including at JFK and LaGuardia — after tech outage cripples banks and other companies around the globe Charlotte Karp / Daily Mail : Australia is hit by massive Microsoft outage as major banks, businesses, Qantas, ABC and Foxtel are all struck by huge outages - and computers crash with the blue screen of death Emma Roth / The Verge : The outage appears to have been caused by CrowdStrike pushing a faulty update to its Falcon software that causes Windows machines to get stuck in a boot loop Tom Warren / The Verge : A workaround to fix the BSOD caused by CrowdStrike's faulty update involves booting Windows in Safe Mode and deleting the security firm's “C-00000291*.sys” file David Goldman / CNN : Frontier and other airlines were at a standstill for hours after a massive Microsoft outage Financial Times : Global IT outage latest: Over 4,000 flights cancelled Threads: Ian Betteridge / @ianbetteridge : Summary of how social media is performing over the CrowdStrike BSOD: Bluesky: Crickets, because not enough people are on Mastodon: Deep tech analysis from people currently doing front line support Twitter: Fake screenshots from Cryptobros Threads: Three day old content Facebook: Look at my cat! Christopher Mims / @technicallymims : Unbelievable the extent of the global tech outages right now. It's hospitals, it's banks, it's airports full of people being given little information, lining up for hours until they “just give up” We are dependent on systems with single points of failure in ways few understand. … Casey Newton / @crumbler : Very difficult morning as it appears that I will have to learn what CrowdStrike is @panzer : Craig Federighi sweating and shaking rn thinking this is finally the Mac's chance to run global infrastructure after all. @kashhill : How I found out about the great tech outage of 2024: A Signal from my husband who was at JFK saying it was bonkers there. I just taught a two-week class to high school journalists and we had a session on tech infrastructure, so this was a timely example for them of why it matters. … Anil Dash / @anildash : For folks who aren't familiar: Windows computers around the world are crashing because companies have installed a tool (made by a company called Crowdstrike) that helps them manage those machines for issues like security, and that software is causing an issue. … Tom Warren / @tomwarrenuk : this Windows BSOD / CrowdStrike issue has knocked Sky News offline in the UK and Ryanair is already warning of flight delays. It's going to be a long day for IT admins worldwide https://www.theverge.com/... Mastodon: Kelly Shortridge / @shortridge@hachyderm.io : and this is why we need to stop absolving *commercial* cybersecurity vendors of software quality concerns. — there should be multiple checks preventing this type of broken content in an update. — how did they allow it to ship to so many machines all at once? — #crowdstrike Nicolas Ward / @ultranurd@tacobelllabs.net : #CrowdStrike posted a deeper explanation of their bad update https://www.crowdstrike.com/ ... #infosec They'll definitely need to explain what they're doing to make sure this can't happen again; a lot of this post is “this was a normal thing for us to be doing” which while true in security space I don't think will help them much. Chandler Carruth / @chandlerc@hachyderm.io : Actual info about today's excitement: https://www.crowdstrike.com/ ... Really looking forward to more details on the logic error though... And gotta say, bold move with this sentence, emphasis mine: “Systems that are not currently impacted will continue to operate as expected, continue to provide protection, and have *no risk of experiencing this event in the future*.” … @ruhrscholz@toot.kif.rocks : Seems like Crowdstrike really fucked up https://www.theverge.com/... Kevin Beaumont / @GossiTheDog@cyberplace.social : If anybody is wondering the impact of the Crowdstrike thing - it's really bad. Machines don't boot. — The recovery is boot in safe mode, log in as local admin and delete things - which isn't automateable. Basically Crowdstrike will be in very hot water. Rich Stein / @RunRichRun@mastodon.social : Sitting in a chair at the dentist's office this AM in Philadelphia. They can't bring up a treatment plan on the computer nor take x-rays (which are now fully digital). Also took longer than usual to check in; half the academic institution they're attached to is completely offline … Gerard Braad / @gbraad@mastodon.social : CrowdStrike on Windows (BSOD) workaround steps: — Boot Windows into Safe Mode or the Windows Recovery Environment — Navigate to the C:\Windows\System32\drivers\CrowdStrike directory — Locate the file matching “C-00000291*.sys”, and delete it. — Boot the host normally. … @jwildeboer@social.wildeboer.net : If you are traveling today with an airplane — may you have th best of luck. — “At least three major U.S. airlines, American, United and Delta, grounded all flights, according to the Federal Aviation Administration.” — (possibly paywalled) https://www.nytimes.com/... #Crowdstrike #Outage #Windows @munin@infosec.exchange : And honestly there's nothing actionable to be learned about this. — They had a QA failure and something shitty got out; could happen to anyone; the threat surface isn't really mitigable by other means and the root cause is going to be something very specific to crowdstrike's internal organization and thus not applicable to anyone else. Matthew Garrett / @mjg59@nondeterministic.computer : Ferrari: How can we fuck up impressively this weekend — Mercedes (sponsored by Crowdstrike): Hold my beer Joe Uchill / @JoeUchill@mastodon.social : The worst part of Crowdstrike has to be that I wanted to come up with another “struck” song as a follow up to this toot and accidentally googled “struck dongs” — Truly, I am the real victim. — https://mastodon.social/... @camwilson@mastodon.social : Now confirmed as a Crowdstrike error, two complicating factors for organisations fixing it: — it's a hard problem to fix remotely with some devices are stuck in boot loops — reports of companies uninstalling Crowdstrike to fix — leaving them exposed to the threats @mttaggart@infosec.town : So Crowdstrike is apparently bootlooping, but a Redditor hero has a fix. — www.reddit.com/r/crowdstrike/comments/ 1e6vmkf/comment/ldvydnz/ — www.reddit.com/r/crowdstrike/comments/ 1e6vmkf/bsod_error_in_latest_crowdstrik e_ update/ @HulloItsIcy@mstdn.games : Poor Microsoft. All these “global Microsoft outage” phrasing in headlines/threads when “Crowdstrike security software causing global outage on Windows systems” would be more accurate. Jeffrey / @jtig@infosec.exchange : Thoughts and prayers for every organisation using Crowdstrike right now. Boot loops for every server and endpoint since a few hours. — A lot of people are getting an unexpected long weekend, and some soldiers can work a long weekend to fix this. — https://www.reddit.com/... @merill@infosec.exchange : Looks like there is a global CrowdStrike BSOD outage. In Australia where it's the middle of the day this feels like armageddon with outages everywhere including banks, fuel, airports... Some links to keep an eye on include: — 🔴 Reddit — https://www.reddit.com/... … @camwilson@mastodon.social : (still unconfirmed but, if true, it's a bit funny that CrowdStrike crashed computers worldwide minutes after the official nomination acceptance by Trump, who was obsessed with the company of a baseless conspiracy theory that it made up proof of Russian election interference) Jeremy Kirk / @jkirk@infosec.exchange : CrowdStrike's Falcon sensor is having a technical issue. BSOD reports across Australian organisations. Apparently it is related to a Falcon sensor. Crashes Windows hosts or BSOD. CrowdStrike says its “engineering teams are actively working to resolve this issues and there is no need to open a support ticket.” … @drunkenmadman@aus.social : Pretty happy I'm no longer administering any windows boxes in a corporate gig. There was a time I was running several hundred cloud instances all on windows server core. — Of course, 99% of those were cattle rather than pets, so I'd be having an easier time than many... https://www.reddit.com/... X: Satya Nadella / @satyanadella : Yesterday, CrowdStrike released an update that began impacting IT systems globally. We are aware of this issue and are working closely with CrowdStrike and across the industry to provide customers technical guidance and support to safely bring their systems back online. David Rhodes / @davidgrayrhodes : .@SkyNews have not been able to broadcast live TV this morning, currently telling viewers that we apologise for the interruption. Much of our news report is still available online, and we are working hard to restore all services. @faanews : The FAA is closely monitoring a technical issue impacting IT systems at U.S. airlines. Several airlines have requested FAA assistance with ground stops until the issue is resolved. Monitor https://www.fly.faa.gov/ for updates. Troy Hunt / @troyhunt : Something super weird happening right now: just been called by several totally different media outlets in the last few minutes, all with Windows machines suddenly BSoD'ing (Blue Screen of Death). Anyone else seen this? Seems to be entering recovery mode: [image] George Kurtz / @george_kurtz : All of CrowdStrike continues to work closely with impacted customers and partners to ensure that all systems are restored. I'm sharing the letter I sent to CrowdStrike's customers and partners. As this incident is resolved, you have my commitment to provide full transparency on how this occurred and the steps we're taking to prevent anything like this from happening again. We are working on a technical update and root cause analysis that we will share with everyone as well. Akshay Kothari / @akothari : The Microsoft / CrowdStrike outage has taken down most airports in India. I got my first hand-written boarding pass today 😅 [image] Brody / @brody_n77 : @_JohnHammond There is a faulty channel file, so not quite an update. There is a workaround... 1. Boot Windows into Safe Mode or WRE. 2. Go to C:\Windows\System32\drivers\CrowdStrike 3. Locate and delete file matching “C-00000291*.sys” 4. Boot normally. 1/2 Elon Musk / @elonmusk : @Jason We just deleted Crowdstrike from all our systems, so no rollouts at all Colin McCarthy / @us_stormwatch : 12-hour timelapse of American Airlines, Delta, and United plane traffic after what was likely the biggest IT outage in history forced a nationwide ground stop of the three airlines. [video] @fcc : We're aware of reports of a systems outage causing disruptions in service, including 911. We're working closely with other federal agencies to provide assistance and determine the extent of these service disruptions. Secretary Pete Buttigieg / @secretarypete : If you're affected by today's airline tech issues, bookmark https://flightrights.gov/. We created an easy-to-use dashboard in 2022 to show which airlines cover meals and hotels, and provide free rebooking when an airline causes a major delay or cancellation. [image] @azuresupport : 🛠️ We experienced a Storage incident in Central US which had downstream impact to a number of Azure services. This is currently mitigated, however we are still in the process of validating recovery to a small percentage of those downstream services. This was communicated to Anshel Sag / @anshelsag : For those who don't remember, in 2010, McAfee had a colossal glitch with Windows XP that took down a good part of the internet. The man who was McAfee's CTO at that time is now the CEO of Crowdstrike. The McAfee incident cost the company so much they ended up selling to Intel. [image] @united : While most of our systems have recovered from the worldwide third-party software outage, we may continue to experience some disruption to our operation, including flight delays and cancellations. Our travel waiver allows you to change your flight or connection city. To see your Matt Mullenweg / @photomatt : It's like Y2K happened 24 years late. [image] Kyla Scanlon / @kylascan : On the value of people during a technological crisis “You'll have men in white vans going around to try manually fix this problem even when they put out a fix,” Woodward said in an earlier interview with Bloomberg News. “That is a big job.” [image] Robert Graham / @erratarob : 🧵Today's Crowdstrike fuck up is just proof for what I've been saying for years. A thread (1/207)... Steven Sinofsky / @stevesi : Kernel mode is *the* problem. In 2024 changing software from third parties via a private update channel is about the highest risk setup and should not be a generally available capability. And if it is it should not be used in critical systems. @wired : NEW: By afternoon, over 4,000 flights had been canceled and 35,500 delayed globally, according to FlightAware. Here's how a bad software update from Crowdstrike took down computers running Microsoft Windows—and a cascade of airports with it. https://www.wired.com/... Dan Elton / @moreisdifferent : The @CrowdStrike disaster left 45,000 devices within the Mass General Brigham healthcare system inoperable with a “blue screen of death”. The situation is so bad they are asking me, an AI researcher, to help over the weekend. Similar story at Providence Health (15k devices down) Dan Guido / @dguido : Apple deserves a lot of credit today for having the foresight and the willpower to restrict MacOS agents to simpler interfaces. https://developer.apple.com/ ... @mike_d_ok : Crowdstrike fix. May the force be with you. Always. [image] Nicole Perlroth / @nicoleperlroth : This time it was a software update. It happens. Thank God it was harmless and happened to a responsible company who could ID and issue a fix so quickly. But please, let this be a wake up call to what the impact of a cascading cyberattack will look like. We need to start a cross Jake Williams / @malwarejake : Okay, I'm just going to throw this out there, but maybe - just maybe - a vendor having the ability to change every one of their kernel drivers in the field at the same time without any approval from IT/end users is a model we need to reconsider... @CrowdStrike. [image] Elon Musk / @elonmusk : ... Tim Sweeney / @timsweeneyepic : The amount of eye strain people are experiencing today highlights a simple fact: Windows needs to support dark mode Blue Screen of Death! @secgov : The SEC is aware of the situation related to a widespread IT disruption and is monitoring for market-related impacts. Andrej Karpathy / @karpathy : What a case study of systemic risk with CrowdStrike outage... that a few bits in the wrong place can brick ~1 billion computers and all the 2nd, 3rd order effects of it. What other single points of instantaneous failure exist in the technosphere and how do we design against it. @buccocapital : The plan? Crash every computer owned by our customers so the world realizes how important we are to the economy [image] Michael Higham / @michaelphigham : aint CrowdStrike that final fantasy guy Esther Crawford / @esthercrawford : Everyone is talking about Crowdstrike and the global outage across critical infrastructure but what really concerns me is that Starbucks order ahead is down. [image] @weldpond : I am personally troubled by the recent Microsoft Windows IT outage and its widespread impact on travel, banking, and healthcare systems globally. Such incidents highlight the vulnerabilities in our interconnected systems and the importance of cybersecurity diligence. My thoughts @h4ckmanac : #CrowdStrike Impacts by Region update at 1:20 pm UTC+04:00 Australia - Media: ABC, SBS, Seven Network, Nine Network - Airlines: Qantas, Virgin Australia, Jetstar - Airports: Sydney, Melbourne - Supermarkets: Woolworths, Coles - Banks: NAB, ANZ, Commonwealth Bank, Aaron / @aaronoleary : They got the vegas ball. It's all over. We lost. [image] Boyan Slat / @boyanslat : Congratulating #CrowdStrike for reaching its carbon neutrality targets six years early through its disruption of global air traffic today! @sf_emergency : Overnight, a system update from the cybersecurity company CrowdStrike caused global impacts to online systems. While the majority of City and County IT systems do not utilize this software and were not impacted, some issues were reported and are currently being assessed. Ray / @raymo_g : I don't think you guys fully grasp how big this is. Around a billion computers are bricked worldwide, mostly corporate ones. This isn't just an online service going down for a few hours. Every affected computer needs to be rebooted in fail mode and have a driver manually [image] Aidan Ryan / @aidanfitzryan : Spokesperson for Audacy, owner of WEEI and Magic 106.7, says the company was affected by the global software outage Fri: “Like many companies around the world, we did experience issues as a result of today's outages. All of our stations were back up and running by 8:00 a.m. ET.” Daniel Sugarman / @daniel_sugarman : “Just push the ‘Update All Windows Devices’ button, Akimov! How hard can it be?” [image] @0xtib3rius : CrowdStrike caused the biggest IT outage in history. Here's what it taught me about B2B sales: 🧵 @jeremy_peel : This IT outage is so huge I'm hearing ambient dialogue about it as I pass strangers at the train station. Like the first half an hour of a AAA game. Gaut / @0xgaut : If this is your work laptop, you have to go to work today [image] Charlie Warzel / @cwarzel : so basically Y2K was just 24 years late Frank X. Shaw / @fxshaw : Earlier today, a Crowdstrike update was responsible for bringing down a number of Windows systems globally. We are actively supporting customers to assist in their recovery. Pat McFadden / @patmcfaddenmp : Many people are being affected by today's IT outages impacting services across the country and globally. Ministers are working with their sectors and respective industries on the issue. I am in close contact with teams coordinating our response through the COBR response system Carl Quintanilla / @carlquintanilla : GOLDMAN, on $CRWD: “While it's still early, .. we expect to see minimal share shifts in endpoint as a result of the incident, although we expect to see noise in competitor go to market processes. .. We maintain our 12-month price target at $400 based on 55x Q5-Q8 FCF.” @swiftonsecurity : Just to be clear, fixing this CrowdStrike issue will require basically a human visit to every machine. Some of the machines will not be able to get into the recovery environment, and require a USB stick boot. Centrally fixing this is not possible it happens before anything loads. Thaddeus E. Grugq / @thegrugq : Today's CrowdStrike outage must be the most comprehensive blocking of APT activity in the company's history. @vxunderground : Threat Actors today wondering where the hell all their compromised hosts went [image] Elon Musk / @elonmusk : @FT Biggest IT fail ever Tom Warren / @tomwarren : UPDATE: Microsoft tells me it's “aware of an issue affecting Windows devices due to an update from a third-party software platform. We anticipate a resolution is forthcoming.” #Crowdstrike #BSOD #windows https://www.theverge.com/... Nicole Perlroth / @nicoleperlroth : Oh boy. Not a cyberattack but one of those days when a software update had the same affect. https://www.nytimes.com/... Tom Warren / @tomwarren : it looks like some IT admins are having luck with simply rebooting systems over and over. It seems the network stack comes up long enough to grab CrowdStrike's update https://x.com/... Dan Ives / @divestech : This is clearly a major black eye for CrowdStrike and the stock will be under pressure after this global outage related to Microsoft has caused massive disruption globally. This is a technical update and importantly not a hack/cyber security threat. @piratesoftware : Technically CrowdStrike is doing it's job. Your data is still secure. Even from you. Alex Cohen / @anothercohen : I was fired from Crowdstrike today. I was hired a few weeks ago to run our infrastructure and reliability team. I had to get to the airport to start my 3 month summer vacation in France so I rushed our review process and deployed an update directly to production. Apparently @flightradar24 : About 1000 fewer passenger flights in the air over the US this morning compared to yesterday at the same time. Nearly 1900 flights on the move now and that number will grow as airline's process the backlog of flights created by today's #ITOutage. [image] Brad Sams / @bdsams : Not sure what's more alarming...Crowdstrike nuking have the Internet Or pushing to prod on a Friday. Dr Emma Salisbury / @salisbot : BREAKING: Militaries unaffected by CrowdStrike outage as their IT hasn't been updated since 2004 Whitney Merrill / @wbm312 : We Y2K-ed ourselves anyways it seems. Sophie / @netcapgirl : idk what's worse, that crowdstrike has kernel access to basically every system in the world or that airlines run on windows server Max Pollard / @maxpollard92 : Microsoft outage but Teams and Outlook are both fine is the adult version of snow that doesn't settle enough for a school closure. @nixcraft : Guys, I started working at the cybersecurity firm Crowdstrike. Today is my first day. Eight hours ago, I pushed major code to production. I am so proud of myself. I am going now home. I feel something really good is coming my way tomorrow morning at work 🥰🧑🏻💻 @vxunderground : CrowdStrike has performed the largest ransomware attack in history. Accidentally. Matt Stoller / @matthewstoller : Microsoft has had dominant market power in PC operating systems since the 1980s. For some reason we assume that because there are Macs and smartphones we ‘solved’ that problem. But no. @benlovejoy : Any time you're mulling on your own screw-ups, take comfort in the fact that you didn't break the world ... Sean Gallagher / @thepacketrat : Wow internet seems faster this morning for some reason . -posted from a Mac. Tom Warren / @tomwarren : god bless all the IT admins out there today 🐐In another life I used to look after trade floors and have experienced major outages. It's an incredibly stressful job 🫡 Olivia Solon / @oliviasolon : “Please check on Google for your gate number” - announcement at Heathrow Airport just now. Blue screens of death everywhere in terminal due to CrowdStrike update affecting Windows machines. @sxchopea : happy international bluescreen day😍 [image] @firstsquawk : CROWDSTRIKE CEO ON OUTAGE: ACTIVELY WORKING WITH CUSTOMERS IMPACTED BY DEFECT FOUND IN A SINGLE CONTENT UPDATE FOR WINDOWS HOSTS. MAC AND LINUX HOSTS ARE NOT IMPACTED. Manish Singh / @refsrc : Microsoft spokesperson on the ongoing outage: “We're aware of an issue affecting Windows devices due to an update from a third-party software platform. We anticipate a resolution is forthcoming.” @nixcraft : #linux users right now 🐧 [image] Mehedi / @mehedih_ : excited to announce i am joining Crowdstrike as a Senior Deploy Straight to Production Engineer Tom Warren / @tomwarren : Sky News is calling this “the most serious IT outage the world has ever seen.” I guess they don't remember Blaster then? 🙃 Thomas Baekdal / @baekdal : Again, they are right... if you use Cloudstrike, your business will go down [image] Tom Warren / @tomwarren : this isn't the first time that CrowdStrike's csagent.sys kernel driver has caused Windows BSODs. I'd imagine many executives are waking up this morning and immediately looking at moving away from CrowdStrike. It's very hard to win back trust after an event like this @monkchips : Honestly surprised it seems like neither Microsoft or crowdstrike have put out an official statement yet. Considering how feverishly social media is calling out their names.... Sam Whyte / @samwhyte : Whoever's responsible for the Microsoft outage is getting fired anyway, so the smart thing to do would be knock Teams out for the day too and leave a hero. Jacob Aron / @jjaron : Does this mean that 2024 is the year of Linux on the desktop? @monkchips : Happy International Endpoint Security day to all those who celebrate! Gergely Orosz / @gergelyorosz : @sinnet3000 A reminder that Microsoft / Windows is at fault here as well, not just Crowdstrike. Giving antivirus enough privileges to crash the OS always carries this risk. A more resilient OS does not let this happen. This would not happen at a Linux (or would be much harder to do) Gergely Orosz / @gergelyorosz : Oh wow - sounds like there are global outages across airlines globally (from LAX to BER), TV & radio stations from the UK to Australia and supermarkets in Australia thanks to the “Windows blue screen of death” for companies running Crowdstrike? This kind of impact is wild. [image] Charles Arthur / @charlesarthur : One Crowdstrike and you're out. Mike Dudas / @mdudas : be kind to your it department today they're gonna need a hug @sherieffyi : The greatest culprit for loss of life, value, and property due to computer outages might just end up being security software - eclipsing ransomware in the process by an order or two of magnitude. Congratulations, CrowdStrike. Elon Musk / @elonmuskaoc : Engineers at Microsoft and CrowdStrike right now [image] @h4ckmanac : 🚨CrowdStrike - Massive Outage Globally 🚨 The latest CrowdStrike update is causing a widespread issue resulting in a Blue Screen of Death (BSOD) boot loop globally. Many users are experiencing major outages due to this problem https://www.reddit.com/... #CrowdStrike [image] Ciaran Martin / @ciaranmartinoxf : @ruskin147 @BBCr4today This screenshot is doing the rounds on this platform Rory - apparently from a Crowdstrike user platform which is paywalled. Lots of credible experts seeing this as the source of the problems I cannot verify this but it seems credible [image] @letheforgot : @SwiftOnSecurity What we did was use the advanced restart options to launch the command prompt, skip the bitlocker key ask which then brought us to drive X and ran “bcdedit /set {default} safeboot minimal"which let us boot into safemode and delete the sys file causing the bsod. Runa Sandvik / @runasand : I was joking when I posted this photo from MAD, but appears a wonky update from @CrowdStrike is affecting airports around the world. Curious to know if the update was tested prior to deployment by the vendor and/or the clients. [image] Aaron Levie / @levie : This is like the start of a dystopian sci-fi movie [video] @alx : CrowdStrike? More like CrowdStruck Jim Waterson / @jimwaterson : Someone, somewhere, is the person who pressed go on that software update. And right now they know exactly what they did and what it has done. And as someone who knows that sinking feeling when you realise you've screwed up at work, I cannot imagine the state of them right now. Costin Raiu / @craiu : The potentially faulty Crowdstrike CSagent.sys hit VT last night. Compiled on July 9th. https://www.virustotal.com/... [image] @deitaone : KLM SAYS FORCED TO SUSPEND MOST OF OPERATION DUE TO OUTAGE LUFTHANSA SAYS PROFILE, BOOKING RETRIEVAL FUNCTIONALITY LIMITED LOS ANGELES INTERNATIONAL AIRPORT CURRENTLY EXPERIENCING SOME MINOR DELAYS - SPOKESPERSON JETSTAR JAPAN SPOKESPERSON: MICROSOFT WINDOWS GLITCHES AFFECTING Ryan Petersen / @typesfast : The Microsoft / Crowdstrike outage has taken down the immigration systems at US airports. Only citizens with passports and green card holders can enter right now. Visitors queuing endlessly. Todd / @daviest_ : Another day, another huge outage across multiple industries caused by a systemic dependence on a single firm. If resilience is what the Commission is looking for (not to mention innovation), then it would do well to closely scrutinise bottlenecks and mergers. [image] Dave Troy / @davetroy : 1/The global IT outage caused by a failed Crowdstrike software update can apparently be fixed by booting Windows into Safe Mode and deleting a file. This means millions of Windows machines need to be booted and fixed by hand. [image] Amjad Masad / @amasad : “CrowdStrike” sounds like a name you'd give to a bug like “HeartBleed” Simo Kohonen / @simokohonen : Latest #Crowdstrike update seems to be pushing machines into a BSOD loop. Major outages around the globe. [image] Nathan McNulty / @nathanmcnulty : Heads up for those running Crowdstrike :( For those in charge of any AV/EDR infrastructure, it's worth spending time thinking about how to best control and validate updates Be sure to consider how quickly you can respond and prevent something this this from rolling out broadly Christopher Stanley / @cstanley : This is not the first time this year CrowdStrike has caused issues on servers. I've had great success with SentinelOne for years. As with any EDR, they are a means to an end. @skynewsaust : A major worldwide tech outage, believed to have been caused by a flawed anti-viral update from US cyber security company CrowdStrike, has plunged many of the world's largest companies into crisis and prevented Australian newsrooms from publishing the news. Stephen Punwasi / @stephenpunwasi : CrowdStrike just knocked a bunch of newsrooms, airports, and gov facilities offline. Easy fix: reboot into safe mode & delete “C-00000291*.sys” in the crowdstrike driver folder, reboot, & you're good. But good luck finding someone to figure out that's what they need to do. David Chau / @chaudave : The dreaded Blue Screen of Death is affecting my #Windows laptop as well. (Thanks #Crowdstrike!!!) So that means no #ABC Finance Report from me tonight, because of the outage. Sorry everyone! (It would've been the the greatest finance EVER)! 😅 [image] Joe Tidy / @joetidy : If the windows outage is indeed a bug in Crowdstrike cyber security products it would be something we've not seen yet. Insurers would have a headache as the organisations affected did the *right* thing by having cyber protection which led to today's disaster. Ouch Troy Hunt / @troyhunt : Just an important point on this as I'm seeing some misunderstandings: this is not a “Microsoft outage” (disclosure: I don't work there or speak for them, Regional Directors are totally independent), it's a CrowdStrike issue impacting Microsoft PCs. Andrew Chiles / @andrewchiles : @troyhunt Delta Airlines in ATL hit [image] John Hammond / @_johnhammond : I'd love to be able to see their messaging, but it is behind a login. [image] John Hammond / @_johnhammond : This is CrowdStrike's Director of Overwatch, so I hope to help spread the word. I believe CS stopped these changes from being pushed out so machines late to the party wont get the faulty driver. Command in Safe Mode: del “C:\Windows\System32\drivers\CrowdStrik e\C- 00000291*.sys” @rawsalerts : What an interesting way to start my Friday morning with this CrowdStrike outage causing significant issues nationwide Christopher Stanley / @cstanley : I am happy to report that X is not and will not be affected by the current global outage due to running CrowdStrike on your system. Rory Cellan-Jones / @ruskin147 : Computer chaos takes down Sky News and many other services, flights, etc - @ciaranmartinoxf tells @BBCr4today it appears to be due to a problem with Crowdstrike and Windows [image] Troy Hunt / @troyhunt : If you're watching this unfold and aren't unfamiliar with the name “CrowdStrike”, they're a *massive* player in the security space and have billions of dollars of annual revenue. Their products include “EDR”, which is endpoint detection and response. Think of it as antivirus. Ashish Jha / @the_dream_saver : Seems like this Crowdstrike issue is beyond my Jetstar flight! It is impacting retail payment systems, supermarkets, fuel stations, restaurants, travel systems and banks all around! F M L ! [image] Ryan Mac / @rmac18 : So this was what y2k was supposed to be Jon Michalski / @jon9198 : Stuck on the Tarmac at LAX cause we can't get a minor service card filled dude to IT system outages GLOBALLY with ALL airlines and apparently a bunch of other things. @united #outage #serviceinterruption [video] Sam Butler / @samdcbu : my parents are stranded in Juneau, and are telling me @Delta and @united are both grounded because of a @Microsoft patch gone wrong. Apparently also grounding flights in Australia. it's unfolding as I type, so just relaying what I've heard and am looking for confirmation @m363208 : @iamakshayshar It is the otherway around.. #microsoft users chilling while Mac users still having to work 😂🤯 #Crowdstrike @ghazzzer : @GergelyOrosz @sinnet3000 they had to deploy it on Friday [image] Steve / @jarofsteve : Wow, Crowdstrike issue. Thoughts and prayers fellow IT guys and girls around the world. Lukasz Olejnik / @lukolejnik : Global Windows outage hits computers around the world. This is linked to Crowdstrike update that cripples boot process. There are some workarounds. Do you think it may be fixed automatically, somehow? Oh well ... https://supportportal.crowdstrike.com/ ... [image] @aeyakovenko : TIL all Microsoft windows machines are just thin clients into one giant windows computer. Sun Microsystems was 30 years too early! Senad Aruc / @senadaruc : Looks like Crowdstrike Dooms Day it is real, many airline companies having issues with their systems. Schiphol airport is frozen. [image] John Hammond / @_johnhammond : CrowdStrike Falcon agents are imploding right now and causing a Blue Screen of Death boot loop on every endpoint. Reports of massive outages globally. https://www.reddit.com/... Scott Bryan / @scottygb : Sky News not having a good morning. [image] Desus MF Nice / @desusnice : yo @CrowdStrike you're a miserable bitch Armin Ronacher / @mitsuhiko : Looks like crowdstrike took out loads of windows machines worldwide. If you see some BSODs today you know why. https://www.reddit.com/... Jim Waterson / @jimwaterson : Sky News currently playing whalesong, seemingly because of a globally bodged external software update affecting a lot of companies. Greg Egan / @gregegansf : Crowdstrike have advised that the world will be reverted to its last valid backup set, dated 7 Jan 2014, within the next 30 minutes. Please make paper notes of anything important to you from the intervening period, and tape them to your refrigerator door in a prominent position. @fletcherlad : Crowdstrike, if you are reading this: it's okay sweetie, mistakes happen. Take your own time as we don't need an update on this for at least a few business days. John B / @johnb78 : Crowdstrike appear to have - I think the technical term is - fucked it Tommy / @tferris : Microsoft Windows users right now. #crowdstrike #bsod [image] Cam Smith / @sexenheimer : Bit of personal news - it's my first day at Crowdstrike as the guy who flicks the On switch every morning. Completely blanked on it today but reckon I will nail it Monday. Ben Platnick / @s3pirion : crowdstrike outage has absolutely fucked SEATAC lmao. windows recovery screen the long way [image] Gabriel / @_thevivi : CrowdStrike declaring an early weekend by taking out half the world's systems. Even ransomware isn't this effective 😂 @nswpolice : Police are aware of the current system outage. For emergency situations, please dial 000. Cameron Wilson / @cameronwilson : Reports of Windows computers showing BSOD errors across a lot of different sectors: supermarkets, banks, broadcasters, etc. Downdetector, which shows crowdsourced reports of outages, going crazy rn [image] @itcrowdtalk : @troyhunt Summary CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor. Details Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon Sensor. Current Action Our Engineering teams are actively working to resolve this. Tim Doering / @timdoering97 : @eddiemajor @troyhunt Yeah (Greetings from germany) [image] Troy Hunt / @troyhunt : The global scope of this is *MASSIVE*. Germany: Chris Stokel-Walker / @stokel : Good morning. Stay in bed today. Early reports suggest Crowdstrike Falcon - a computer threat checker used by lots (and lots and lots) of businesses pushed out an update that might have broken a lot of computers. Airlines, businesses etc affected LinkedIn: Paddy Mahadeva : This is exactly why I hate Friday releases! — Also, all the testing groups are now going gaga about importance of testing. … Rahul Arora : Responsibility, responsiveness, support and timely communication is the key! — CrowdStrike is doing everything possible to extend the support required. … Benedict Ojefua : This is an interesting one. Should you allow automatic updates on your endpoint security? Well this is one risk that this strategy brings … Luke Jones : First, an IT update unleashes global chaos. Now, Microsoft faces another grave threat: Outlook collapsing under the sheer weight of PR emails. … Chris Beck : One of the best adverts I've seen for having canary builds as part of your release process... https://lnkd.in/... Igor Volovich : In April 2010, McAfee VirusScan pushed a DAT update 5958 that caused a critical Windows XP file svchost.exe to be misdetected as malware. … Graeme Pietersz : Its not an infrastructure issue. It is a a bad update to Crowdstrike's Windows security software. — It is the result of not being willing to spend on reliability. … Rick Porter : I'm sure CrowdStrike has a highly developed and fine-tuned change control process, but something still went wrong. … Andy Beverley : Whilst we can draw various conclusions from today's major IT outage (which is being reported as the largest in history) for me it highlights how concentrated the tech landscape is. … Mike Rencheck : The events of yesterday begin to demonstrate just how fragile the digital world can be and what happens when it does not work. … Dr. Louis F. D. : Houston, we have a problem. Crowdstrike update allegedly responsible for a major Microsoft IT outage. … Thaddeus Dziekanowski : Hate to say I told you so but I told you so. Companies rushed to the cloud for the wrong reasons and this is the price. … Emil Protalinski : Testing updates isn't sexy, but it can prevent global outages. — CrowdStrike today reported widespread BSODs on Windows hosts after issuing a software update … Qusai Alhaddad : Again I am repeating it — “ Unix for Servers, Linux for Desktop , Windows for Solitaire” — https://lnkd.in/... Forums: Hacker News : CrowdStrike Update: Windows Bluescreen and Boot Loops Hacker News : Crowdstrike Outage Causing Widespread Issues r/crowdstrike : BSOD error in latest crowdstrike update r/crowdstrike : Technical Details on Today's Outage r/PersonalFinanceCanada : Big Global Outage Impacting Banks r/cars : CrowdStrike Global Technology Outage Known to Impact Automotive Sectors r/technews : Huge Microsoft Outage Linked to CrowdStrike Takes Down Computers Around the World r/StallmanWasRight : Global outage? My (Linux) systems are working great! r/sysadmin : Many Windows 10 machines blue screening, stuck at recovery r/de_EDV : CrowdStrike Windows Outage r/neoliberal : Crowdstrike update bricks every single Windows machine it touches. Largest IT outage in history. r/fednews : Widespread technology outage disrupts flights, banks, media outlets and companies around the world r/Shittyaskflying : This must be fake news. Pylotes are very posh and rich. They wouldn't even look at a Windows PC. Most pylotes would not even know how to use anything that isn't Apple. r/tsa : IT outages reported across globe as airlines, airports, banks and media companies experience disruptions r/sysadmin : Crowdstrike BSOD? — Anyone else experience BSOD due to Crowdstrike? I've got two separate organisations in Australia experiencing this. r/technology : Huge Microsoft Outage Linked to CrowdStrike Takes Down Computers Around the World r/programming : CrowdStrike update takes down most Windows machines worldwide r/wallstreetbets : Crowdstrike just took the internet offline. r/WayOfTheBern : IT outages reported across globe as airlines, airports, banks and media companies experience disruptions r/worldnews : Crowdstrike suffers major outage affecting businesses around the world r/ShittySysadmin : See? This is why we don't update. r/newzealand : PSA: Crowdstrike outage r/windows : Major Windows BSOD issue takes banks, airlines, and broadcasters offline r/news : global IT outage due to Crowdstrike r/wallstreetbets : CrowdStrike Major Outage Grounding All Planes In US r/worldnews : United, Delta and American Airlines issue global ground stop on all flights r/wallstreetbets : Cybersecurity giant Crowdstrike suffers major outage affecting businesses around the world r/technology : Major Internet Outage in US r/news : United, Delta and American Airlines issue global ground stop on all flights r/collapse : Global IT outage live updates: Australian banks, airlines, media outlets taken offline r/brisbane : Global IT outage live updates: Australian banks, airlines, media outlets taken offline See also Mediagazer