Discussion

• @eastdakota Matthew Prince on x

  Everyone has a bad day. This one really sucked for @CrowdStrike. Continue to have faith in them as a partner and the best end point security solution on the market. #HugOps

• @dinodaizovi Dino A. Dai Zovi on x

  Good time to re-read “CyberInsecurity: The Cost of Monopoly” by Dan Geer et al: https://www.schneier.com/...

• @eastdakota Matthew Prince on x

  Here's the scary thing that's likely to happen based on the facts of the day if we don't pay attention. Microsoft, who competes with @CrowdStrike, will argue that they should lock all third-party security vendors out of their OS. “It's the only way we can be safe,” they'll

• @LukaszOlejnik@mastodon.social Lukasz Olejnik on mastodon

  My comment for the The New York Times about the global IT outage.  It affects Windows systems with CrowdStrike cybersecurity software.  Our civilisation depends on software.  It depends on other software and systems, these in turn on others.  Something goes wrong and the effect i…

• @danprimack Dan Primack on x

  CrowdStrike stock is down 10%. 1st thought: How could it not be down even more? 2nd thought: Because this shows that it's integral to the entire internet.

• @kylascan Kyla Scanlon on x

  I think this is really interesting. Like on their worst day, Nike was down 20% on reports that sales would decline a little bit. Crowdstrike causes a global catastrophe and is down 12%. [image]

• r/technology r on reddit

  CrowdStrike Stock Tanks 15%—Set For Worst Day Since 2022

• @JMarkOckerbloom@mastodon … John Mark Ockerbloom on mastodon

  Crowdstrike's statement on their Falcon content update includes the line: “We further recommend organizations ensure they're communicating with CrowdStrike representatives through official channels.”  It's a useful reminder that disruptions, besides causing problems directly …

• @perpetualmaniac @perpetualmaniac on x

  Crowdstrike Analysis: It was a NULL pointer from the memory unsafe C++ language. Since I am a professional C++ programmer, let me decode this stack trace dump for you. [image]

• @loxyflo @loxyflo on x

  Anyone know how Liz Truss's first day at Microsoft is going?

• @shanselman Scott Hanselman on x

  Here's the thing folks. I've been coding 32 years. When something like this happens it's an organizational failure. Yes, some human wrote a bad line. Someone can “git blame” and point to a human and it's awful. But it's the testing, the Cl/CD, the A/B testing, the metered

• @eastdakota Matthew Prince on x

  We should be careful creating incentives for systems' designers where when something goes wrong the right answer to satisfy the lawyers is to fail open. #thatsnotsecurity

• @k8em0 @k8em0 on x

  On the CrowdStrike outage: Most organizations of a certain size test software updates before deployment. They do not test “content updates” from OS or security software, but set them to automatically update because they are viewed as safe. IT departments just got a new daily task

• @patrickwardle Patrick Wardle on x

  I don't do Windows but here are some (initial) details about why the CrowdStrike's CSAgent.sys crashed Faulting inst: mov r9d, [r8] R8: unmapped address ...taken from an array of pointers (held in RAX), index RDX (0x14 * 0x8) holds the invalid memory address @_JohnHammond [image]

• @stevesi Steven Sinofsky on x

  Kernel mode is *the* problem. In 2024 changing software from third parties via a private update channel is about the highest risk setup and should not be a generally available capability. And if it is it should not be used in critical systems.

• @hackerfantastic @hackerfantastic on x

  Are we *sure* the @CrowdStrike crash wasn't deliberate? They pushed a file full of NULL bytes to their agents which caused the BSoD...

• @jason @jason on x

  I guess crowdstrike doesn't do staged rollouts?

• r/technology r on reddit

  What is CrowdStrike, and what happened?

• Vox Li Zhou on x

  The “largest IT outage in history,” briefly explained

• @ianbetteridge Ian Betteridge on threads

  Summary of how social media is performing over the CrowdStrike BSOD: Bluesky: Crickets, because not enough people are on Mastodon: Deep tech analysis from people currently doing front line support Twitter: Fake screenshots from Cryptobros Threads: Three day old content Facebook: …

• @technicallymims Christopher Mims on threads

  Unbelievable the extent of the global tech outages right now.  It's hospitals, it's banks, it's airports full of people being given little information, lining up for hours until they “just give up” We are dependent on systems with single points of failure in ways few understand. …

• @crumbler Casey Newton on threads

  Very difficult morning as it appears that I will have to learn what CrowdStrike is

• @panzer @panzer on threads

  Craig Federighi sweating and shaking rn thinking this is finally the Mac's chance to run global infrastructure after all.

• @kashhill @kashhill on threads

  How I found out about the great tech outage of 2024: A Signal from my husband who was at JFK saying it was bonkers there.  I just taught a two-week class to high school journalists and we had a session on tech infrastructure, so this was a timely example for them of why it matter…

• @anildash Anil Dash on threads

  For folks who aren't familiar: Windows computers around the world are crashing because companies have installed a tool (made by a company called Crowdstrike) that helps them manage those machines for issues like security, and that software is causing an issue. …

• @tomwarrenuk Tom Warren on threads

  this Windows BSOD / CrowdStrike issue has knocked Sky News offline in the UK and Ryanair is already warning of flight delays.  It's going to be a long day for IT admins worldwide https://www.theverge.com/...

• @RunRichRun@mastodon.social Rich Stein on mastodon

  Sitting in a chair at the dentist's office this AM in Philadelphia.  They can't bring up a treatment plan on the computer nor take x-rays (which are now fully digital).  Also took longer than usual to check in; half the academic institution they're attached to is completely offli…

• @gbraad@mastodon.social Gerard Braad on mastodon

  CrowdStrike on Windows (BSOD) workaround steps:  — Boot Windows into Safe Mode or the Windows Recovery Environment  — Navigate to the C:\Windows\System32\drivers\CrowdStrike directory  — Locate the file matching “C-00000291*.sys”, and delete it.  — Boot the host normally. …

• @JoeUchill@mastodon.social Joe Uchill on mastodon

  The worst part of Crowdstrike has to be that I wanted to come up with another “struck” song as a follow up to this toot and accidentally googled “struck dongs”  —  Truly, I am the real victim.  —  https://mastodon.social/...

• @camwilson@mastodon.social @camwilson@mastodon.social on mastodon

  Now confirmed as a Crowdstrike error, two complicating factors for organisations fixing it:  — it's a hard problem to fix remotely with some devices are stuck in boot loops  — reports of companies uninstalling Crowdstrike to fix — leaving them exposed to the threats

• @camwilson@mastodon.social @camwilson@mastodon.social on mastodon

  (still unconfirmed but, if true, it's a bit funny that CrowdStrike crashed computers worldwide minutes after the official nomination acceptance by Trump, who was obsessed with the company of a baseless conspiracy theory that it made up proof of Russian election interference)

• @satyanadella Satya Nadella on x

  Yesterday, CrowdStrike released an update that began impacting IT systems globally. We are aware of this issue and are working closely with CrowdStrike and across the industry to provide customers technical guidance and support to safely bring their systems back online.

• @davidgrayrhodes David Rhodes on x

  .@SkyNews have not been able to broadcast live TV this morning, currently telling viewers that we apologise for the interruption. Much of our news report is still available online, and we are working hard to restore all services.

• @faanews @faanews on x

  The FAA is closely monitoring a technical issue impacting IT systems at U.S. airlines. Several airlines have requested FAA assistance with ground stops until the issue is resolved. Monitor https://www.fly.faa.gov/ for updates.

• @troyhunt Troy Hunt on x

  Something super weird happening right now: just been called by several totally different media outlets in the last few minutes, all with Windows machines suddenly BSoD'ing (Blue Screen of Death). Anyone else seen this? Seems to be entering recovery mode: [image]

• @akothari Akshay Kothari on x

  The Microsoft / CrowdStrike outage has taken down most airports in India. I got my first hand-written boarding pass today 😅 [image]

• @george_kurtz George Kurtz on x

  All of CrowdStrike continues to work closely with impacted customers and partners to ensure that all systems are restored.  I'm sharing the letter I sent to CrowdStrike's customers and partners.  As this incident is resolved, you have my commitment to provide full transparency on…

• @brody_n77 Brody on x

  @_JohnHammond There is a faulty channel file, so not quite an update. There is a workaround... 1. Boot Windows into Safe Mode or WRE. 2. Go to C:\Windows\System32\drivers\CrowdStrike 3. Locate and delete file matching “C-00000291*.sys” 4. Boot normally. 1/2

• @elonmusk Elon Musk on x

  @Jason We just deleted Crowdstrike from all our systems, so no rollouts at all

• @us_stormwatch Colin McCarthy on x

  12-hour timelapse of American Airlines, Delta, and United plane traffic after what was likely the biggest IT outage in history forced a nationwide ground stop of the three airlines. [video]

• @fcc @fcc on x

  We're aware of reports of a systems outage causing disruptions in service, including 911. We're working closely with other federal agencies to provide assistance and determine the extent of these service disruptions.

• @secretarypete Secretary Pete Buttigieg on x

  If you're affected by today's airline tech issues, bookmark https://flightrights.gov/. We created an easy-to-use dashboard in 2022 to show which airlines cover meals and hotels, and provide free rebooking when an airline causes a major delay or cancellation. [image]

• @azuresupport @azuresupport on x

  🛠️ We experienced a Storage incident in Central US which had downstream impact to a number of Azure services. This is currently mitigated, however we are still in the process of validating recovery to a small percentage of those downstream services. This was communicated to

• @mike_d_ok @mike_d_ok on x

  Crowdstrike fix. May the force be with you. Always. [image]

• @malwarejake Jake Williams on x

  Okay, I'm just going to throw this out there, but maybe - just maybe - a vendor having the ability to change every one of their kernel drivers in the field at the same time without any approval from IT/end users is a model we need to reconsider... @CrowdStrike. [image]

• @elonmusk Elon Musk on x

  ...

• @anshelsag Anshel Sag on x

  For those who don't remember, in 2010, McAfee had a colossal glitch with Windows XP that took down a good part of the internet. The man who was McAfee's CTO at that time is now the CEO of Crowdstrike. The McAfee incident cost the company so much they ended up selling to Intel. [i…

• @nicoleperlroth Nicole Perlroth on x

  This time it was a software update. It happens. Thank God it was harmless and happened to a responsible company who could ID and issue a fix so quickly. But please, let this be a wake up call to what the impact of a cascading cyberattack will look like. We need to start a cross

• @united @united on x

  While most of our systems have recovered from the worldwide third-party software outage, we may continue to experience some disruption to our operation, including flight delays and cancellations. Our travel waiver allows you to change your flight or connection city. To see your

• @photomatt Matt Mullenweg on x

  It's like Y2K happened 24 years late. [image]

• @kylascan Kyla Scanlon on x

  On the value of people during a technological crisis “You'll have men in white vans going around to try manually fix this problem even when they put out a fix,” Woodward said in an earlier interview with Bloomberg News. “That is a big job.” [image]

• @erratarob Robert Graham on x

  🧵Today's Crowdstrike fuck up is just proof for what I've been saying for years. A thread (1/207)...

• @wired @wired on x

  NEW: By afternoon, over 4,000 flights had been canceled and 35,500 delayed globally, according to FlightAware. Here's how a bad software update from Crowdstrike took down computers running Microsoft Windows—and a cascade of airports with it. https://www.wired.com/...

• @moreisdifferent Dan Elton on x

  The @CrowdStrike disaster left 45,000 devices within the Mass General Brigham healthcare system inoperable with a “blue screen of death”. The situation is so bad they are asking me, an AI researcher, to help over the weekend. Similar story at Providence Health (15k devices down)

• @dguido Dan Guido on x

  Apple deserves a lot of credit today for having the foresight and the willpower to restrict MacOS agents to simpler interfaces. https://developer.apple.com/ ...

• @timsweeneyepic Tim Sweeney on x

  The amount of eye strain people are experiencing today highlights a simple fact: Windows needs to support dark mode Blue Screen of Death!

• @secgov @secgov on x

  The SEC is aware of the situation related to a widespread IT disruption and is monitoring for market-related impacts.

• @karpathy Andrej Karpathy on x

  What a case study of systemic risk with CrowdStrike outage... that a few bits in the wrong place can brick ~1 billion computers and all the 2nd, 3rd order effects of it. What other single points of instantaneous failure exist in the technosphere and how do we design against it.

• @buccocapital @buccocapital on x

  The plan? Crash every computer owned by our customers so the world realizes how important we are to the economy [image]

• @michaelphigham Michael Higham on x

  aint CrowdStrike that final fantasy guy

• @esthercrawford Esther Crawford on x

  Everyone is talking about Crowdstrike and the global outage across critical infrastructure but what really concerns me is that Starbucks order ahead is down. [image]

• @weldpond @weldpond on x

  I am personally troubled by the recent Microsoft Windows IT outage and its widespread impact on travel, banking, and healthcare systems globally. Such incidents highlight the vulnerabilities in our interconnected systems and the importance of cybersecurity diligence. My thoughts

• @h4ckmanac @h4ckmanac on x

  #CrowdStrike Impacts by Region update at 1:20 pm UTC+04:00 Australia - Media: ABC, SBS, Seven Network, Nine Network - Airlines: Qantas, Virgin Australia, Jetstar - Airports: Sydney, Melbourne - Supermarkets: Woolworths, Coles - Banks: NAB, ANZ, Commonwealth Bank,

• @aaronoleary Aaron on x

  They got the vegas ball. It's all over. We lost. [image]

• @boyanslat Boyan Slat on x

  Congratulating #CrowdStrike for reaching its carbon neutrality targets six years early through its disruption of global air traffic today!

• @sf_emergency @sf_emergency on x

  Overnight, a system update from the cybersecurity company CrowdStrike caused global impacts to online systems. While the majority of City and County IT systems do not utilize this software and were not impacted, some issues were reported and are currently being assessed.

• @raymo_g Ray on x

  I don't think you guys fully grasp how big this is. Around a billion computers are bricked worldwide, mostly corporate ones. This isn't just an online service going down for a few hours. Every affected computer needs to be rebooted in fail mode and have a driver manually [image]

• @aidanfitzryan Aidan Ryan on x

  Spokesperson for Audacy, owner of WEEI and Magic 106.7, says the company was affected by the global software outage Fri: “Like many companies around the world, we did experience issues as a result of today's outages. All of our stations were back up and running by 8:00 a.m. ET.”

• @daniel_sugarman Daniel Sugarman on x

  “Just push the ‘Update All Windows Devices’ button, Akimov! How hard can it be?” [image]

• @0xtib3rius @0xtib3rius on x

  CrowdStrike caused the biggest IT outage in history. Here's what it taught me about B2B sales: 🧵

• @jeremy_peel @jeremy_peel on x

  This IT outage is so huge I'm hearing ambient dialogue about it as I pass strangers at the train station. Like the first half an hour of a AAA game.

• @0xgaut Gaut on x

  If this is your work laptop, you have to go to work today [image]

• @cwarzel Charlie Warzel on x

  so basically Y2K was just 24 years late

• @fxshaw Frank X. Shaw on x

  Earlier today, a Crowdstrike update was responsible for bringing down a number of Windows systems globally. We are actively supporting customers to assist in their recovery.

• @swiftonsecurity @swiftonsecurity on x

  Just to be clear, fixing this CrowdStrike issue will require basically a human visit to every machine. Some of the machines will not be able to get into the recovery environment, and require a USB stick boot. Centrally fixing this is not possible it happens before anything loads.

• @patmcfaddenmp Pat McFadden on x

  Many people are being affected by today's IT outages impacting services across the country and globally. Ministers are working with their sectors and respective industries on the issue. I am in close contact with teams coordinating our response through the COBR response system

• @carlquintanilla Carl Quintanilla on x

  GOLDMAN, on $CRWD: “While it's still early, .. we expect to see minimal share shifts in endpoint as a result of the incident, although we expect to see noise in competitor go to market processes. .. We maintain our 12-month price target at $400 based on 55x Q5-Q8 FCF.”

• @thegrugq Thaddeus E. Grugq on x

  Today's CrowdStrike outage must be the most comprehensive blocking of APT activity in the company's history.

• @elonmusk Elon Musk on x

  @FT Biggest IT fail ever

• @vxunderground @vxunderground on x

  Threat Actors today wondering where the hell all their compromised hosts went [image]

• @tomwarren Tom Warren on x

  UPDATE: Microsoft tells me it's “aware of an issue affecting Windows devices due to an update from a third-party software platform. We anticipate a resolution is forthcoming.” #Crowdstrike #BSOD #windows https://www.theverge.com/...

• @nicoleperlroth Nicole Perlroth on x

  Oh boy. Not a cyberattack but one of those days when a software update had the same affect. https://www.nytimes.com/...

• @tomwarren Tom Warren on x

  it looks like some IT admins are having luck with simply rebooting systems over and over. It seems the network stack comes up long enough to grab CrowdStrike's update https://x.com/...

• @divestech Dan Ives on x

  This is clearly a major black eye for CrowdStrike and the stock will be under pressure after this global outage related to Microsoft has caused massive disruption globally. This is a technical update and importantly not a hack/cyber security threat.

• @piratesoftware @piratesoftware on x

  Technically CrowdStrike is doing it's job. Your data is still secure. Even from you.

• @anothercohen Alex Cohen on x

  I was fired from Crowdstrike today. I was hired a few weeks ago to run our infrastructure and reliability team. I had to get to the airport to start my 3 month summer vacation in France so I rushed our review process and deployed an update directly to production. Apparently

• @flightradar24 @flightradar24 on x

  About 1000 fewer passenger flights in the air over the US this morning compared to yesterday at the same time. Nearly 1900 flights on the move now and that number will grow as airline's process the backlog of flights created by today's #ITOutage. [image]

• @bdsams Brad Sams on x

  Not sure what's more alarming...Crowdstrike nuking have the Internet Or pushing to prod on a Friday.

• @salisbot Dr Emma Salisbury on x

  BREAKING: Militaries unaffected by CrowdStrike outage as their IT hasn't been updated since 2004

• @wbm312 Whitney Merrill on x

  We Y2K-ed ourselves anyways it seems.

• @netcapgirl Sophie on x

  idk what's worse, that crowdstrike has kernel access to basically every system in the world or that airlines run on windows server

• @maxpollard92 Max Pollard on x

  Microsoft outage but Teams and Outlook are both fine is the adult version of snow that doesn't settle enough for a school closure.

• @nixcraft @nixcraft on x

  Guys, I started working at the cybersecurity firm Crowdstrike. Today is my first day. Eight hours ago, I pushed major code to production. I am so proud of myself. I am going now home. I feel something really good is coming my way tomorrow morning at work 🥰🧑🏻‍💻

• @vxunderground @vxunderground on x

  CrowdStrike has performed the largest ransomware attack in history. Accidentally.

• @matthewstoller Matt Stoller on x

  Microsoft has had dominant market power in PC operating systems since the 1980s. For some reason we assume that because there are Macs and smartphones we ‘solved’ that problem. But no.

• @benlovejoy @benlovejoy on x

  Any time you're mulling on your own screw-ups, take comfort in the fact that you didn't break the world ...

• @thepacketrat Sean Gallagher on x

  Wow internet seems faster this morning for some reason . -posted from a Mac.

• @tomwarren Tom Warren on x

  god bless all the IT admins out there today 🐐In another life I used to look after trade floors and have experienced major outages. It's an incredibly stressful job 🫡

• @oliviasolon Olivia Solon on x

  “Please check on Google for your gate number” - announcement at Heathrow Airport just now. Blue screens of death everywhere in terminal due to CrowdStrike update affecting Windows machines.

• @sxchopea @sxchopea on x

  happy international bluescreen day😍 [image]

• @firstsquawk @firstsquawk on x

  CROWDSTRIKE CEO ON OUTAGE: ACTIVELY WORKING WITH CUSTOMERS IMPACTED BY DEFECT FOUND IN A SINGLE CONTENT UPDATE FOR WINDOWS HOSTS. MAC AND LINUX HOSTS ARE NOT IMPACTED.

• @refsrc Manish Singh on x

  Microsoft spokesperson on the ongoing outage: “We're aware of an issue affecting Windows devices due to an update from a third-party software platform. We anticipate a resolution is forthcoming.”

• @nixcraft @nixcraft on x

  #linux users right now 🐧 [image]

• @mehedih_ Mehedi on x

  excited to announce i am joining Crowdstrike as a Senior Deploy Straight to Production Engineer

• @tomwarren Tom Warren on x

  Sky News is calling this “the most serious IT outage the world has ever seen.” I guess they don't remember Blaster then? 🙃

• @baekdal Thomas Baekdal on x

  Again, they are right... if you use Cloudstrike, your business will go down [image]

• @tomwarren Tom Warren on x

  this isn't the first time that CrowdStrike's csagent.sys kernel driver has caused Windows BSODs. I'd imagine many executives are waking up this morning and immediately looking at moving away from CrowdStrike. It's very hard to win back trust after an event like this

• @monkchips @monkchips on x

  Honestly surprised it seems like neither Microsoft or crowdstrike have put out an official statement yet. Considering how feverishly social media is calling out their names....

• @samwhyte Sam Whyte on x

  Whoever's responsible for the Microsoft outage is getting fired anyway, so the smart thing to do would be knock Teams out for the day too and leave a hero.

• @jjaron Jacob Aron on x

  Does this mean that 2024 is the year of Linux on the desktop?

• @monkchips @monkchips on x

  Happy International Endpoint Security day to all those who celebrate!

• @gergelyorosz Gergely Orosz on x

  @sinnet3000 A reminder that Microsoft / Windows is at fault here as well, not just Crowdstrike. Giving antivirus enough privileges to crash the OS always carries this risk. A more resilient OS does not let this happen. This would not happen at a Linux (or would be much harder to …

• @gergelyorosz Gergely Orosz on x

  Oh wow - sounds like there are global outages across airlines globally (from LAX to BER), TV & radio stations from the UK to Australia and supermarkets in Australia thanks to the “Windows blue screen of death” for companies running Crowdstrike? This kind of impact is wild. [image…

• @charlesarthur Charles Arthur on x

  One Crowdstrike and you're out.

• @mdudas Mike Dudas on x

  be kind to your it department today they're gonna need a hug

• @sherieffyi @sherieffyi on x

  The greatest culprit for loss of life, value, and property due to computer outages might just end up being security software - eclipsing ransomware in the process by an order or two of magnitude. Congratulations, CrowdStrike.

• @elonmuskaoc Elon Musk on x

  Engineers at Microsoft and CrowdStrike right now [image]

• @h4ckmanac @h4ckmanac on x

  🚨CrowdStrike - Massive Outage Globally 🚨 The latest CrowdStrike update is causing a widespread issue resulting in a Blue Screen of Death (BSOD) boot loop globally. Many users are experiencing major outages due to this problem https://www.reddit.com/... #CrowdStrike [image]

• @ciaranmartinoxf Ciaran Martin on x

  @ruskin147 @BBCr4today This screenshot is doing the rounds on this platform Rory - apparently from a Crowdstrike user platform which is paywalled. Lots of credible experts seeing this as the source of the problems I cannot verify this but it seems credible [image]

• @letheforgot @letheforgot on x

  @SwiftOnSecurity What we did was use the advanced restart options to launch the command prompt, skip the bitlocker key ask which then brought us to drive X and ran “bcdedit /set {default} safeboot minimal"which let us boot into safemode and delete the sys file causing the bsod.

• @runasand Runa Sandvik on x

  I was joking when I posted this photo from MAD, but appears a wonky update from @CrowdStrike is affecting airports around the world. Curious to know if the update was tested prior to deployment by the vendor and/or the clients. [image]

• @levie Aaron Levie on x

  This is like the start of a dystopian sci-fi movie [video]

• @alx @alx on x

  CrowdStrike? More like CrowdStruck

• @jimwaterson Jim Waterson on x

  Someone, somewhere, is the person who pressed go on that software update. And right now they know exactly what they did and what it has done. And as someone who knows that sinking feeling when you realise you've screwed up at work, I cannot imagine the state of them right now.

• @craiu Costin Raiu on x

  The potentially faulty Crowdstrike CSagent.sys hit VT last night. Compiled on July 9th. https://www.virustotal.com/... [image]

• @deitaone @deitaone on x

  KLM SAYS FORCED TO SUSPEND MOST OF OPERATION DUE TO OUTAGE LUFTHANSA SAYS PROFILE, BOOKING RETRIEVAL FUNCTIONALITY LIMITED LOS ANGELES INTERNATIONAL AIRPORT CURRENTLY EXPERIENCING SOME MINOR DELAYS - SPOKESPERSON JETSTAR JAPAN SPOKESPERSON: MICROSOFT WINDOWS GLITCHES AFFECTING

• @typesfast Ryan Petersen on x

  The Microsoft / Crowdstrike outage has taken down the immigration systems at US airports. Only citizens with passports and green card holders can enter right now. Visitors queuing endlessly.

• @daviest_ Todd on x

  Another day, another huge outage across multiple industries caused by a systemic dependence on a single firm. If resilience is what the Commission is looking for (not to mention innovation), then it would do well to closely scrutinise bottlenecks and mergers. [image]

• @davetroy Dave Troy on x

  1/The global IT outage caused by a failed Crowdstrike software update can apparently be fixed by booting Windows into Safe Mode and deleting a file. This means millions of Windows machines need to be booted and fixed by hand. [image]

• @amasad Amjad Masad on x

  “CrowdStrike” sounds like a name you'd give to a bug like “HeartBleed”

• @simokohonen Simo Kohonen on x

  Latest #Crowdstrike update seems to be pushing machines into a BSOD loop. Major outages around the globe. [image]

• @nathanmcnulty Nathan McNulty on x

  Heads up for those running Crowdstrike :( For those in charge of any AV/EDR infrastructure, it's worth spending time thinking about how to best control and validate updates Be sure to consider how quickly you can respond and prevent something this this from rolling out broadly

• @cstanley Christopher Stanley on x

  This is not the first time this year CrowdStrike has caused issues on servers. I've had great success with SentinelOne for years. As with any EDR, they are a means to an end.

• @skynewsaust @skynewsaust on x

  A major worldwide tech outage, believed to have been caused by a flawed anti-viral update from US cyber security company CrowdStrike, has plunged many of the world's largest companies into crisis and prevented Australian newsrooms from publishing the news.

• @stephenpunwasi Stephen Punwasi on x

  CrowdStrike just knocked a bunch of newsrooms, airports, and gov facilities offline. Easy fix: reboot into safe mode & delete “C-00000291*.sys” in the crowdstrike driver folder, reboot, & you're good. But good luck finding someone to figure out that's what they need to do.

• @chaudave David Chau on x

  The dreaded Blue Screen of Death is affecting my #Windows laptop as well. (Thanks #Crowdstrike!!!) So that means no #ABC Finance Report from me tonight, because of the outage. Sorry everyone! (It would've been the the greatest finance EVER)! 😅 [image]

• @joetidy Joe Tidy on x

  If the windows outage is indeed a bug in Crowdstrike cyber security products it would be something we've not seen yet. Insurers would have a headache as the organisations affected did the *right* thing by having cyber protection which led to today's disaster. Ouch

• @troyhunt Troy Hunt on x

  Just an important point on this as I'm seeing some misunderstandings: this is not a “Microsoft outage” (disclosure: I don't work there or speak for them, Regional Directors are totally independent), it's a CrowdStrike issue impacting Microsoft PCs.

• @andrewchiles Andrew Chiles on x

  @troyhunt Delta Airlines in ATL hit [image]

• @_johnhammond John Hammond on x

  I'd love to be able to see their messaging, but it is behind a login. [image]

• @_johnhammond John Hammond on x

  This is CrowdStrike's Director of Overwatch, so I hope to help spread the word. I believe CS stopped these changes from being pushed out so machines late to the party wont get the faulty driver. Command in Safe Mode: del “C:\Windows\System32\drivers\CrowdStrik e\C- 00000291*.sys”

• @rawsalerts @rawsalerts on x

  What an interesting way to start my Friday morning with this CrowdStrike outage causing significant issues nationwide

• @cstanley Christopher Stanley on x

  I am happy to report that X is not and will not be affected by the current global outage due to running CrowdStrike on your system.

• @ruskin147 Rory Cellan-Jones on x

  Computer chaos takes down Sky News and many other services, flights, etc - @ciaranmartinoxf tells @BBCr4today it appears to be due to a problem with Crowdstrike and Windows [image]

• @troyhunt Troy Hunt on x

  If you're watching this unfold and aren't unfamiliar with the name “CrowdStrike”, they're a *massive* player in the security space and have billions of dollars of annual revenue. Their products include “EDR”, which is endpoint detection and response. Think of it as antivirus.

• @the_dream_saver Ashish Jha on x

  Seems like this Crowdstrike issue is beyond my Jetstar flight! It is impacting retail payment systems, supermarkets, fuel stations, restaurants, travel systems and banks all around! F M L ! [image]

• @rmac18 Ryan Mac on x

  So this was what y2k was supposed to be

• @jon9198 Jon Michalski on x

  Stuck on the Tarmac at LAX cause we can't get a minor service card filled dude to IT system outages GLOBALLY with ALL airlines and apparently a bunch of other things. @united #outage #serviceinterruption [video]

• @samdcbu Sam Butler on x

  my parents are stranded in Juneau, and are telling me @Delta and @united are both grounded because of a @Microsoft patch gone wrong. Apparently also grounding flights in Australia. it's unfolding as I type, so just relaying what I've heard and am looking for confirmation

• @m363208 @m363208 on x

  @iamakshayshar It is the otherway around.. #microsoft users chilling while Mac users still having to work 😂🤯 #Crowdstrike

• @ghazzzer @ghazzzer on x

  @GergelyOrosz @sinnet3000 they had to deploy it on Friday [image]

• @jarofsteve Steve on x

  Wow, Crowdstrike issue. Thoughts and prayers fellow IT guys and girls around the world.

• @lukolejnik Lukasz Olejnik on x

  Global Windows outage hits computers around the world. This is linked to Crowdstrike update that cripples boot process. There are some workarounds. Do you think it may be fixed automatically, somehow? Oh well ... https://supportportal.crowdstrike.com/ ... [image]

• @aeyakovenko @aeyakovenko on x

  TIL all Microsoft windows machines are just thin clients into one giant windows computer. Sun Microsystems was 30 years too early!

• @senadaruc Senad Aruc on x

  Looks like Crowdstrike Dooms Day it is real, many airline companies having issues with their systems. Schiphol airport is frozen. [image]

• @_johnhammond John Hammond on x

  CrowdStrike Falcon agents are imploding right now and causing a Blue Screen of Death boot loop on every endpoint. Reports of massive outages globally. https://www.reddit.com/...

• @scottygb Scott Bryan on x

  Sky News not having a good morning. [image]

• @desusnice Desus MF Nice on x

  yo @CrowdStrike you're a miserable bitch

• @mitsuhiko Armin Ronacher on x

  Looks like crowdstrike took out loads of windows machines worldwide. If you see some BSODs today you know why. https://www.reddit.com/...

• @jimwaterson Jim Waterson on x

  Sky News currently playing whalesong, seemingly because of a globally bodged external software update affecting a lot of companies.

• @gregegansf Greg Egan on x

  Crowdstrike have advised that the world will be reverted to its last valid backup set, dated 7 Jan 2014, within the next 30 minutes. Please make paper notes of anything important to you from the intervening period, and tape them to your refrigerator door in a prominent position.

• @fletcherlad @fletcherlad on x

  Crowdstrike, if you are reading this: it's okay sweetie, mistakes happen. Take your own time as we don't need an update on this for at least a few business days.

• @johnb78 John B on x

  Crowdstrike appear to have - I think the technical term is - fucked it

• @tferris Tommy on x

  Microsoft Windows users right now. #crowdstrike #bsod [image]

• @sexenheimer Cam Smith on x

  Bit of personal news - it's my first day at Crowdstrike as the guy who flicks the On switch every morning. Completely blanked on it today but reckon I will nail it Monday.

• @s3pirion Ben Platnick on x

  crowdstrike outage has absolutely fucked SEATAC lmao. windows recovery screen the long way [image]

• @_thevivi Gabriel on x

  CrowdStrike declaring an early weekend by taking out half the world's systems. Even ransomware isn't this effective 😂

• @nswpolice @nswpolice on x

  Police are aware of the current system outage. For emergency situations, please dial 000.

• @cameronwilson Cameron Wilson on x

  Reports of Windows computers showing BSOD errors across a lot of different sectors: supermarkets, banks, broadcasters, etc. Downdetector, which shows crowdsourced reports of outages, going crazy rn [image]

• @itcrowdtalk @itcrowdtalk on x

  @troyhunt Summary CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor. Details Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon Sensor. Current Action Our Engineering teams are actively working to resolve …

• @timdoering97 Tim Doering on x

  @eddiemajor @troyhunt Yeah (Greetings from germany) [image]

• @troyhunt Troy Hunt on x

  The global scope of this is *MASSIVE*. Germany:

• @stokel Chris Stokel-Walker on x

  Good morning. Stay in bed today. Early reports suggest Crowdstrike Falcon - a computer threat checker used by lots (and lots and lots) of businesses pushed out an update that might have broken a lot of computers. Airlines, businesses etc affected

• r/crowdstrike r on reddit

  BSOD error in latest crowdstrike update

• r/crowdstrike r on reddit

  Technical Details on Today's Outage

• r/PersonalFinanceCanada r on reddit

  Big Global Outage Impacting Banks

• r/cars r on reddit

  CrowdStrike Global Technology Outage Known to Impact Automotive Sectors

• r/sysadmin r on reddit

  Many Windows 10 machines blue screening, stuck at recovery

• r/technews r on reddit

  Huge Microsoft Outage Linked to CrowdStrike Takes Down Computers Around the World

• r/StallmanWasRight r on reddit

  Global outage?  My (Linux) systems are working great!

• r/de_EDV r on reddit

  CrowdStrike Windows Outage

• r/neoliberal r on reddit

  Crowdstrike update bricks every single Windows machine it touches.  Largest IT outage in history.

• r/sysadmin r on reddit

  Crowdstrike BSOD?  —  Anyone else experience BSOD due to Crowdstrike?  I've got two separate organisations in Australia experiencing this.

• r/fednews r on reddit

  Widespread technology outage disrupts flights, banks, media outlets and companies around the world

• r/Shittyaskflying r on reddit

  This must be fake news.  Pylotes are very posh and rich.  They wouldn't even look at a Windows PC.  Most pylotes would not even know how to use anything that isn't Apple.

• r/tsa r on reddit

  IT outages reported across globe as airlines, airports, banks and media companies experience disruptions

• r/technology r on reddit

  Huge Microsoft Outage Linked to CrowdStrike Takes Down Computers Around the World

• r/programming r on reddit

  CrowdStrike update takes down most Windows machines worldwide

• r/wallstreetbets r on reddit

  Crowdstrike just took the internet offline.

• r/WayOfTheBern r on reddit

  IT outages reported across globe as airlines, airports, banks and media companies experience disruptions

• r/worldnews r on reddit

  Crowdstrike suffers major outage affecting businesses around the world

• r/ShittySysadmin r on reddit

  See?  This is why we don't update.

• r/newzealand r on reddit

  PSA: Crowdstrike outage

• r/windows r on reddit

  Major Windows BSOD issue takes banks, airlines, and broadcasters offline

• r/news r on reddit

  global IT outage due to Crowdstrike

• r/wallstreetbets r on reddit

  CrowdStrike Major Outage Grounding All Planes In US

• r/worldnews r on reddit

  United, Delta and American Airlines issue global ground stop on all flights

• r/wallstreetbets r on reddit

  Cybersecurity giant Crowdstrike suffers major outage affecting businesses around the world

• r/technology r on reddit

  Major Internet Outage in US

• r/news r on reddit

  United, Delta and American Airlines issue global ground stop on all flights

• r/collapse r on reddit

  Global IT outage live updates: Australian banks, airlines, media outlets taken offline

• r/brisbane r on reddit

  Global IT outage live updates: Australian banks, airlines, media outlets taken offline

• @crumbler Casey Newton on threads

  That's crazy.  I thought it would be C-00000292*.sys

• @vxunderground @vxunderground on x

  How to fix the Crowdstrike thing: 1. Boot Windows into safe mode 2. Go to C:\Windows\System32\drivers\CrowdStrike 3. Delete C-00000291*.sys 4. Repeat for every host in your enterprise network including remote workers 5. If you're using BitLocker jump off a bridge

• @patrickmoorhead Patrick Moorhead on x

  Currently wondering why enterprises globally update a “.sys” file without an airgapped test prior to deployment. Speed? Confidence because “it never happened before”? @CrowdStrike

• @evisdrenova Evis Drenova on x

  null pointers strike again! looks like the crowdstrike issue is a dereferenced null pointer that's trying to access null memory using the ‘mov r9d, dward ptr r[8]’ assembly operation. brutal. [image]

• @russelljkaplan Russell Kaplan on x

  Trying to convince the United gate agent to let me reboot Windows in safe mode and delete “C-00000291*.sys”, to no avail [image]

• @craiu Costin Raiu on x

  Some people report that the files responsible for the CrowdStrike crashes (Eg. C-00000291-00000000-00000032.sys) are full of zeroes. This is not the case for any of the machines I fixed by hand today. One example is

• @craiu Costin Raiu on x

  Deleting this file named C-00000291-00000000-00000032.sys fixed the BSoD and allowed the machine to boot. YMMV. #crashstrike #crashscade

• @craiu Costin Raiu on x

  It looks just like the other C-00* files in the same folder, starting with a 0xAAAAAAAA header. [image]

• @0xtib3rius @0xtib3rius on x

  Lol, Microsoft are suggesting rebooting machines 15 times can solve the problem. This. Is. Chaos. #CrowdStrike #CrowdStroke #CrowdStruck [image]

• @sanjeevsanyal Sanjeev Sanyal on x

  So, the whole thing is about a single rogue file called C-00000291*.sys Tells you how fragile modern civilization is......

• @blenster @blenster on x

  A whole lot of people are learning some hard lessons on process and procedures today. Have empathy for those dealing with this mess; many have asked for changes that would have helped with this and were told no.

• @vadimyuryev Vadim Yuryev on x

  Here's a FIX for the Blue Screen of Death for both Windows and Mac users! #BSOD $CRWD Windows: 1. Boot into safe mode 2. Go to C:\Windows\System32\drivers\Crowdstrike directory 3. Delete C-00000291*.sys 4. Restart (credit @MacPaw) Mac: 1. Don't worry. You're not impacted 😂💯🖥️ [im…

• @marypcbuk.bsky.social Mary Branscombe on bluesky

  Microsoft has been doing a dance with security vendors for a loooooong time trying to get them to use fewer of the things that can screw up systems.  Will be interesting to see if they get more traction in future [embedded post]

• @ashukuhi Azim on x

  Ok this Crowdstrike thing is unprecedented and while the news is being processed I want to bring everyone's attention to the fact that as customers start to recovery, they'll most likely disable or modify their Crowdstrike protections. This is going to leave a whole lore of

• @swagitda_ Kelly Shortridge on x

  listen, if you're worried about “correlated risk” in open source software, you need to take security of commercial security software even more seriously most OSS libs aren't kernel modules and rootkits. much of security software, by design, can harm systems far worse.

• @swagitda_ Kelly Shortridge on x

  and this is why we need to stop absolving *commercial* cybersecurity vendors of software quality concerns. there should be multiple checks preventing this type of broken content in an update. how did they allow it to ship to so many machines all at once?

• @ananayarora @ananayarora on x

  If a regular app crashes, you can just open it up again - because it's in User Mode. Since Falcon sensor is running in Kernel Mode, a simple problem here is what causes a Kernel Panic and that's when you see - you guessed it - a Blue Screen of Death on Windows. (3/n) [image]

• @ananayarora @ananayarora on x

  In the case of Falcon sensor, the faulty driver's file name starts with “C-00000291” ending in .sys. The faulty driver update itself caused a kernel panic. The driver seems to have made a bad read to 0x9c as per the panic's stack trace. (4/n) https://x.com/...

• @ananayarora @ananayarora on x

  Crowdstrike Falcon requires installing a lightweight tool called “Falcon Sensor”. Falcon sensor installs services, but most importantly *drivers* - which run in Kernel mode to monitor system activity at a low level. This is a common practice with all security software. (2/n) [ima…

• @_saagarjha Saagar Jha on x

  Ok so are we finally going to decide that installing kernel-level rootkits for “security” was a bad idea yet or is your CISO going to be wined and dined at the next RSA into renewing their contract again

• @cstanley Christopher Stanley on x

  Millions of executives around the world are being woken up and educated on what CrowdStrike is and what Endpoint Detection and Response tools do. They are also asking why it is taking down their entire company 😅 Happy Friday.

• @george_kurtz George Kurtz on x

  CrowdStrike CEO George Kurtz says “the issue has been identified, isolated and a fix has been deployed”, and CrowdStrike is working with its impacted customers

• r/technews r on reddit

  How One Bad CrowdStrike Update Crashed the World's Computers

• @sung.kim.mw Sung Kim on threads

  How to resolve this CrowdStrike issue.  This assumes you are also using BitLocker (kind of expected if you are using CrowdStrike).  Now do this for every impacted machines.  😀

• @hammancheez.bsky.social @hammancheez.bsky.social on bluesky

  Crowdstrike : its fine u just have to manually visit the PC boot it into safe mode and remove a sys file  —  US Organization with 50,000 pcs and a completely outsourced IT department in Bangalore : what

• @jeremydstanley.com Jeremy Stanley on bluesky

  seems like this is not true.  given enough reboots, machines will eventually download the fix — there's enough time for the machine to get online briefly before it enters the boot loop.  good job experts.  [embedded post]

• @hammancheez @hammancheez on x

  Crowdstrike : its fine u just have to manually visit the PC boot it into safe mode and remove a sys file US Organization with 50,000 pcs and a completely outsourced IT department in Bangalore : what

• @tomwarren Tom Warren on x

  it's amazing that the CrowdStrike fix is literally “ have you tried turning it off and on again?” It's working for some IT admins! [image]

• @swiftonsecurity @swiftonsecurity on x

  You could build a PXE boot WIM file and have it execute a fix script but that will require telling everyone how to boot over the network. Very few have this skillset though and will likely require reconfiguring every network to do DHCP relay and won't work if machine locked down.

• @swiftonsecurity @swiftonsecurity on x

  You will also need the local admin LAPS password to do this... And many machines have a broken WinRE environment at least on the disk. Yeah it's pretty grim recovery situation in theory for any moderately complex organization...

• @dylan522p Dylan Patel on x

  Y2K24 - fuckload Windows Machines are absolutely fucked. Crowdstrike $crdw down 19% premarket They pushed out a buggy update (.sys files are kernel drivers, Crowdstrike's agent lives in the kernel) People's computers crashed It also fucks up booting into loading All affected

• @swiftonsecurity @swiftonsecurity on x

  Note this will not work if your machine is bitlocker encrypted without getting the recovery key for each machine...

• @t3dotgg Theo on x

  Pouring one out for all the IT people who have to explain “safe mode” to Carl on the sales team