Source: Treasury's hackers used a flaw in a SolarWinds product; SolarWinds, which touts 300K+ customers, says the flaw was the result of a “supply chain attack”

Russia's foreign intelligence service is suspected of being behind effort to breach government networks

Wall Street Journal 2020-12-14 Dustin Volz

Discussion

@jgamblin Jerry Gamblin on x
Microsoft says that their earliest IOC for the Solarwind breach is March 2020. https://twitter.com/...
@razhael Raphael Satter on x
Just got this from @solarwinds: https://www.reuters.com/... https://twitter.com/...
@kimzetter Kim Zetter on x
I have report from Microsoft about SolarWinds hack, including IoCs. Excerpts in this thread: “Microsoft security researchers recently discovered a sophisticated attack where an adversary inserted malicious code into a supply chain development process.... 1/
@matthew_d_green Matthew Green on x
If the US government handed you $50m/yr and Presidential authority to address supply chain attacks on US systems, what would you do?
@iblametom Thomas Brewster on x
New - A review of contract records shows DOD, FBI, DHS, Veterans Affairs and many other U.S. agencies have purchased SolarWinds Orion, the tool used as a launchpad for the huge government and private industry espionage campaign disclosed this weekend. https://www.forbes.com/...
@malwarejake Jake Williams on x
Okay folks, let's talk about SolarWinds. For those not familiar with it, SolarWinds is a network management system (NMS). It's probably the most ubiquitous NMS out there, so we shouldn't jump to conclusions that FireEye and Treasury were both breached by an SolarWinds vuln. 1/
@kennwhite Kenn White on x
Considerable engineering went in to evading detection, including steganography: “HTTP response bodies attempt to appear like benign XML related to .NET assemblies, but command data...is spread across multiple strings that are disguised as GUID and HEX strings” https://twitter.com…
@hackingdave Dave Kennedy on x
Also for those giving @FireEye hell last week and poking fun and jest at their data breach because they were a security company, do you think your threat model would have protected against this? Some folks owe some apologies.
@dalperovitch Dmitri Alperovitch on x
BUT here is the good news - no adversary has enough human resources to effectively exploit every potential victim. They pretty much HAVE to focus on those they care most about. But no doubt this is squarely in the #goodproblemtohave department for them. Very impressive op 2/2
@c_c_krebs Chris Krebs on x
There it is - @CISAgov issues Emergency Directive 21-10, directing Fed civilian agencies to take action on SolarWinds compromise. Still digesting, but this is a strong move. Proud of the team. Everyone else should refer to this as they chart next steps. https://cyber.dhs.gov/...
@kennwhite Kenn White on x
Just released by FireEye, confirming signed updates from SolarWinds enterprise monitoring software were backdoored. This is the real deal folks. https://www.fireeye.com/... https://twitter.com/...
@dnvolz Dustin Volz on x
The SolarWinds hack is global and widespread. A source said FireEye has seen customers compromised in North America, Europe, Asia and the Middle East and across a range of sectors including telecom, tech, health care, automotive, energy and government. https://www.wsj.com/...
@caseyjohnellis Cje on x
“We have discovered a global intrusion campaign. We are tracking the actors behind this campaign as UNC2452.” https://www.fireeye.com/... Countermeasures here: https://github.com/... https://twitter.com/...
@malwarejake Jake Williams on x
NMS are usually used to monitor network devices and critical servers. If you've ever seen a network map with devices shown as green/yellow/red, that was probably built by an NMS. How does the NMS generate the map? Sometimes it's as simple as a ping command. 4/
@malwarejake Jake Williams on x
More often, the NMS uses SNMP (network management protocol) or an installed agent to learn the status of remote devices. Now they're called network MANAGEMENT systems for a reason: in addition to status, they can modify configuration, restart services, etc. 5/
@dnvolz Dustin Volz on x
SolarWinds is currently hiring for a vice president of security, according to LinkedIn. Spotted by @bobmcmillan https://www.linkedin.com/...
@malwarejake Jake Williams on x
NMS are excellent targets. They have access to most (often all) systems on the network, so outbound IP ACLs are not a useful control. Netflow usually doesn't help either since the NMS not only has access to everything, but it's also talking A LOT. 3/
@dnvolz Dustin Volz on x
This emergency directive reflects how seriously the government considers the threat posed by the suspected Russian hack of SolarWinds. CISA has only issued five emergency directives in its history. https://twitter.com/...
@srunnels @srunnels on x
Our team worked around the clock on this investigation - regularly we hit 18 hour days. I'm proud to have been part of the smartest and most dedicated team I've ever worked with. We had to get creative and fail a lot but when we succeeded we hit big. https://www.fireeye.com/...
@alt_uscis ALT-immigration on x
This is explains the wide spread hack operation. Why hack one by one, while you can just tweak the code of server tools made by solarwinds while in development, and widely used by government agencies and the private sector. That is fucking smart https://twitter.com/...
@dalperovitch Dmitri Alperovitch on x
Some other thoughts on this #SolarWinds supply chain hack. We don't yet know if every customer of SolarWinds who is autoupdating is compromised (likely not given the overall stealthiness of the intrusions to date - adversary most likely down selected to targets of interest) 1/
@kimzetter Kim Zetter on x
“This attack was discovered as part of an ongoing investigation” 3/ https://twitter.com/...
@kimzetter Kim Zetter on x
.@CISAgov has issued an emergency directive on actions that gov agencies need to take immediately to mitigate against the SolarWinds threat: https://cyber.dhs.gov/... https://twitter.com/...
@nicoleperlroth Nicole Perlroth on x
“Power Down.” If you are just joining, USG federal agencies and untold number of SolarWind clients have been compromised via a malicious SolarWinds software update for as long as six months and CISA, the decapitated cyber agency, is telling SolarWinds clients to power it down. ht…
@dnvolz Dustin Volz on x
Russia has hacked several govt agencies including Treasury and Commerce as part of a widespread attack that also hit FireEye. They got in through a flaw in IT firm SolarWinds, which has 100s of thousands of customers, including military and Fortune 500. https://www.wsj.com/...
@kimzetter Kim Zetter on x
SolarWinds: “We are recommending you upgrade to Orion Platform version 2020.2.1 HF 1 as soon as possible..The latest version is available in the...Customer Portal..An additional hotfix release, 2020.2.1 HF 2 is anticipated to be made available Tues Dec 15” https://www.solarwinds.…
@dalperovitch Dmitri Alperovitch on x
Great writeup from @FireEye on the #SolarWinds supply chain hack. Really good tradecraft by the adversary https://www.fireeye.com/... https://twitter.com/...
@campuscodi Catalin Cimpanu on x
SUNBURST malware samples are now available for download, courtesy of @vxunderground https://twitter.com/...
@campuscodi Catalin Cimpanu on x
SolarWinds has also published a security advisory: -SolarWinds Orion versions 2019.4 through 2020.2.1, released between March 2020 and June 2020, are believed to have been compromised -Company plans to release an update this week to remove the malware https://www.solarwinds.com/.…
@malwarejake Jake Williams on x
Not all NMS are able to change anything, and those that can don't always allow it for all systems on the network. The bad news is that the most critical systems are also those most likely to have change access through NMS configured. This isn't a failure in security modeling. 6/
@weswilson4 Wes Wilson on x
Hackers reportedly slipped malware into prior SolarWinds software updates, which gave them access to a “God-mode” for infected networks, including the Treasury and Commerce departments. The Pentagon is also a SolarWinds customer. https://www.kxan.com/... https://twitter.com/...
@kimzetter Kim Zetter on x
“we do not know how the backdoor code made it into the library..research indicates...the attackers might have compromised internal build or distribution systems of SolarWinds, embedding backdoor..into a legitimate SolarWinds library” - SolarWinds.Orion.Core.BusinessLayer.dll 4/
@kimzetter Kim Zetter on x
Ok, this gives us a little more info. It was a DLL, as I suggested in a previous tweet, and it occurred in March. https://twitter.com/...
@hackingdave Dave Kennedy on x
Full write-up from @FireEye just published, contains TTPs and a lot more detail from the Solarwinds breach: https://www.fireeye.com/...
@campuscodi Catalin Cimpanu on x
Microsoft has also added detection rules to Defender. Company calls the malware Solarigate. https://www.microsoft.com/... https://twitter.com/...
@malwarejake Jake Williams on x
That would be an illusory correlation. If you're jumping to that conclusion (or that FireEye and Treasury use a common MSP), at least be clear that you're guessing. It's a lot like the DC Sniper case where we focused on white vans. SolarWinds (like white vans) is everywhere 2/
@zackwhittaker Zack Whittaker on x
Excellent thread on the USG hacks: https://twitter.com/...
@campuscodi Catalin Cimpanu on x
FireEye has also released a technical breakdown of the malware used in the attacks, which they named SUNBURST: https://www.fireeye.com/... Countermeasures/detection rules are here: https://github.com/... https://twitter.com/...
@hackingdave Dave Kennedy on x
Supply chain attacks are absolutely some of the most devasting type of compromises we see. Any organization running that software could be impacted and selected as a target. This group chose well, Solarwinds is used heavily around the globe.
@swiftonsecurity @swiftonsecurity on x
More disclosure on FireEye/SolarWinds attack chain. Fascinating attack vector - stealing the certificate that signs SAML login tokens lets you just impersonate and include any claims you want https://twitter.com/...
@gossithedog Kevin Beaumont on x
Since earlier in year there has been a serious supply chain attack impacting a very popular enterprise management tool, and multivendor SAML (authentication) attacks allowing for maintained access and data exfiltration. IOCs, Azure Sentinel queries etc: https://msrc-blog.microsof…
@kimzetter Kim Zetter on x
@BrianHaugli “The malicious DLL calls out to a remote network infrastructure using the domains https://avsvmcloud.com/. to prepare possible second-stage payloads, move laterally in the organization, and compromise or exfiltrate data”
@eanmeyer @eanmeyer on x
As @MalwareJake was wise to say anything after this story is speculation. However, I used to run a Managed Service NOC for 40 Colleges using all Solarwinds products. We would talk about how bad it would be if Solarwinds was popped for one school, let alone a supply chain attack. …
@brianhonan @brianhonan on x
It will be interesting to get details on this. If its a security flaw in Solar Winds then many others are at risk. Also, a reminder that all remote management tools while providing benefits are also a useful vector for attackers to use https://twitter.com/...
@thegrugq Thaddeus E. Grugq on x
If this started in March they've had almost 200 years on those networks!! 😱 https://twitter.com/...
@dnvolz Dustin Volz on x
FireEye has published a blog and a more technical alert on the SolarWinds hack. Links below: https://www.fireeye.com/... https://www.fireeye.com/...
@fireeye @fireeye on x
For more information about the global software supply chain threat we identified, please read our blog post. https://www.fireeye.com/...
@pwnallthethings @pwnallthethings on x
Wow, this SolarWinds hack was high-end. Lots of moving parts; long reconnaissance stage; very targeted; silent deplot; delay-before-execute; context-specific exfil to keep quiet; very professionally put together. Serious kudos to the folks who caught it. https://www.fireeye.com/.…
@malwarejake Jake Williams on x
This all happens before an admin even notices the service had an issue. Even when the NMS is in monitor-only mode, it can still be used to read configurations, which often include enough information for attackers to laterally move to those systems. 8/
@hackingdave Dave Kennedy on x
Based on what's coming out it looks like Solarwinds build process was comprised (supply chain attack) and distributed which would mean that potentially other folks that have Solarwinds Orion could have the malicious code as well.
@kimzetter Kim Zetter on x
“A malicious software class was included among many other legitimate classes and then signed with a legitimate certificate. The resulting binary included a backdoor and was then discreetly distributed into targeted organizations.... 2/
@jbizzle703 @jbizzle703 on x
This walks through everything pretty clearly as far as mode and method. Good time to start looking for outbound traffic to avsvmcloud(.)com for beaconing. https://twitter.com/...
@texasvc Aziz Gilani on x
3/ @solarwinds has issued a statement confirming their products have contained the backdoor since March: https://www.reuters.com/...
@dnvolz Dustin Volz on x
New: CISA just issued a rare emergency directive instructing federal civilian agencies to review their networks and immediately power down SolarWinds products, saying the hack “poses unacceptable risks to the security of federal networks.” https://www.wsj.com/... https://twitter.…
@evacide Eva on x
I'm already pissed off that I'm going to hear “But what about SolarWind?” from some wiseass every time I tell people to take their security updates for the next several years.
@malwarejake Jake Williams on x
Recall that security also includes availability (CIA triad). The more critical the system, the more likely it is that you want to ensure the availability of it. NMS ensures availability by monitoring for things like services becoming unresponsive and restarting them. 7/
@nakashimae Ellen Nakashima on x
UPDATE: Sources tell me that the victims—Treasury, Commerce, FireEye—were breached through an IT Management System called Solar Winds https://www.washingtonpost.com/ ...
@bing_chris Chris Bing on x
Common refrain from sources: today's news about USG hacks (Commerce + Treasury) and the larger supply chain compromise at Solar Winds, an IT provider for the USG, is “just the tip of the iceberg” This breach is much worse than it appears atm. And it appears very bad already
@c_c_krebs Chris Krebs on x
If you're a SolarWinds customer & use the below product, assume compromise and immediately activate your incident response team. Odds are you're not affected, as this may be a resource intensive hack. Focus on your Crown Jewels. You can manage this. https://twitter.com/... https:…
@dnvolz Dustin Volz on x
SOLARWINDS in statement said it is aware of a potential vulnerability related to updates of its Orion technology management software that were released between March and June of this year.
@neusummits Elizabeth Neumann on x
Relieved that next month we can finally have the head of our government do what cybersecurity professionals have been calling for for years. Attribution and Consequences. https://twitter.com/...
@k8em0 Katie Moussouris on x
Anybody else old enough to remember “remote IT administration or monitoring tools” were handy hacking tools & were called “Back Orifice”? So when a product has this description, it has access by def. If there's a security hole in it, or it's taken over by criminals, you're sunk. …
@bing_chris Chris Bing on x
Confirming this bit of reporting, Solar Winds is patient zero, per two sources familiar: https://twitter.com/...
@dnvolz Dustin Volz on x
This is potentially a hugely consequential hack. SolarWinds' customers include over 425 of US Fortune 500, all branches of military, the NSA, State, Office of the President, top US accounting firms, defense titans like Lockheed, hundreds of universities. https://www.solarwinds.co…
@natashabertrand Natasha Bertrand on x
“If the Russia connection is confirmed, it will be the most sophisticated known theft of American government data by Moscow since a two-year spree in 2014 and 2015” via ⁦@SangerNYT⁩ https://www.nytimes.com/...
@ericgeller Eric Geller on x
Solar Winds' other government customers, per its website: Census Bureau, DOJ, Oak Ridge and Sandia National Labs, VA, Army, Air Force, Navy, and Marine Corps. https://www.solarwinds.com/... Plus state, local, educational, and foreign customers, e.g. Texas, NHS, and European Parli…
@yashar Yashar Ali on x
NEWS The Trump administration acknowledged on Sunday that hackers acting on behalf of a foreign government — almost certainly a Russian intelligence agency — broke into a range of key government networks and had free access to their email systems. https://www.nytimes.com/...
@tedlieu Ted Lieu on x
Dear @realDonaldTrump: A reminder below that Russia is not our ally and Vladimir Putin is not your friend. Why are you so scared to condemn Russia or Putin? https://twitter.com/...
@ncweaver Nicholas Weaver on x
Sweet jeebus, software “supply chain attack”. This is red-alert time. https://twitter.com/...
@razhael Raphael Satter on x
Anyone who uses Orion Monitoring products and want to walk me through what they are and what they do? https://twitter.com/...
@jsrailton John Scott-Railton on x
Key question, if Solar Winds is vulnerable... where else in the massive customer list has APT 29 been? https://twitter.com/... https://twitter.com/...
@kimzetter Kim Zetter on x
Why would FireEye have been using third-party remote-access software on its own network? I'm wondering if this may turn out to be two different, but related, attacks. One against gov networks using SolarWind, and a separate attack against FireEye using a different vector. https:/…
@selectedwisdom Clint Watts on x
“At Commerce, the Russians targeted the National Telecommunications and Information Administration, an agency that handles internet and telecommunications policy” -Russian government spies are behind a broad hacking campaign that has breached U.S. agencies https://www.washingtonp…
@bing_chris Chris Bing on x
.@nakashimae pushing reporting further. Seems that APT29/Russian intel SVR is behind it: https://www.washingtonpost.com/ ...
@malwarejake Jake Williams on x
Outstanding reporting from @nakashimae (following up on @Bing_Chris who broke the story) of the breach at Treasury. WAPO is saying that attackers used SolarWinds in the breach. It's a good reminder to check the access your network management software has. https://www.washingtonpo…
@hackingdave Dave Kennedy on x
This continues to be an interesting development. Looks like Solarwinds Orion was the original entry point from the FireEye breach including treasury and commerce. Microsoft updated 21 hours ago with defender update with artifact detection . https://www.microsoft.com/... https://t…
@ericgeller Eric Geller on x
.@nakashimae is reporting that Russia's foreign intelligence service, the SVR, is behind these federal agency intrusions: https://www.washingtonpost.com/ ... The SVR was also reportedly behind the FireEye hack.
@atrupar Aaron Rupar on x
“The problem is there's not even been condemnation from the top. President Trump hasn't wanted to say anything bad to Russia, which only encourages them to act irresponsibly across a wide range of activities.” https://www.washingtonpost.com/ ...
@frankfigliuzzi1 Frank Figliuzzi on x
And Trump fired Chris Krebs, the head of @CISAgov; great timing: https://www.washingtonpost.com/ ...
@dalperovitch Dmitri Alperovitch on x
SolarWinds is used by hundreds of thousands of organizations... And it has admin access to the network. Monday may be a bad day for lots of security teams. #2020awesomenesscontinues https://twitter.com/...
@biannagolodryga Bianna Golodryga on x
It never gets old or any less absurd to recall that Trump wanted to have a joint cyber unit with Putin. https://www.nytimes.com/...
@dnvolz Dustin Volz on x
FireEye is likely breathing a bit of a sigh of relief given the news tonight. They can say they went public with this widespread supply-chain attack first, which likely helped alert other victims inside and outside government of a potential Russian intrusion into their systems.
@kimzetter Kim Zetter on x
Good find - the vuln discovered in the SolarWinds system that resulted in a June patch. Was this the patch SolarWinds was referring to when it said an update it made to its software between March-June may have introduced a new vuln the Russians exploited? https://hansesecure.de/.…
@wajahatali @wajahatali on x
Will Trump say anything against Putin? How will Republicans respond? Russian Hackers Broke Into Federal Agencies, U.S. Officials Suspect https://www.nytimes.com/...
@milestaylorusa Miles Taylor on x
I have a one-word suggestion that Trump will ignore but Biden will hopefully heed: CONSEQUENCES. The days of Russia “getting away with it” are over. https://twitter.com/...
@kylieatwood Kylie Atwood on x
If the Russia connection is confirmed to the US Treasury & Commerce hack it'll be the most sophisticated known theft of US gov data by Moscow since 2014/15 when Russian intel agencies gained access to WH, State Dept & Joint Chiefs of Staff emails, per NYT https://www.nytimes.com/…
@kimzetter Kim Zetter on x
According to SolarWinds the vuln the hackers used to breach FireEye and gov agencies appears to be related to an update they made to their software between March and June this year. This suggests the hackers could have been in systems for up to nine months before detection. https…
@thegrugq Thaddeus E. Grugq on x
TASS is authorised to declare: Russia totally innocent of amazing brilliant hack by very skilled officers who will receive medals. “Cyberattacks contradict Russia's foreign policy principles,” Russian diplomats lied. https://tass.com/...
@sangernyt David Sanger on x
Struck by fact that for 6 weeks now @realDonaldTrump and 100+ Republican members of Congress have been talking about a hack that never happened - of the vote. Total silence on the one that did happen: Russian hackers inside the Fed. govt.'s own agencies. https://www.nytimes.com/.…
@nakashimae Ellen Nakashima on x
BREAKING: Russian government spies are behind a broad hacking campaign that has breached U.S. agencies and a top cyber firm https://www.washingtonpost.com/ ...
@bing_chris Chris Bing on x
Incident comes shortly after Trump administration fired heads of DHS' Cybersecurity Infrastructure Protection Agency. There is also no current top cyber official. Multiple agencies wrapped up in the investigation. Keep an eye on story (updating): https://www.reuters.com/...
@mmasnick Mike Masnick on x
Seems like a bad time to have fired the CISA Director. https://twitter.com/...
@michaelpfreeman Michael Freeman on x
Didn't Trump fire the government's director of cybersecurity because he didn't like his comments about the election? https://twitter.com/...
@natsecwatson Ben Watson on x
“This is a much bigger story than one single agency,” said one of the people familiar with the matter. “This is a huge cyber espionage campaign targeting the U.S. government and its interests.” https://twitter.com/...
@zackwhittaker Zack Whittaker on x
The Russian government hackers who breached top cybersecurity firm FireEye are behind a much broader espionage campaign that also compromised the Treasury and Commerce departments and other government agencies. https://twitter.com/...
@stevekopack Steve Kopack on x
“The hack is so serious it led to a National Security Council meeting at the White House on Saturday, said one of the people familiar with the matter.” https://twitter.com/...
@maggiejordanacn Maggie Jordan on x
Reminder when in July 2017 trump said “Putin and I discussed forming an impenetrable cybersecurity unit so that election hacking, and many other negative things, will be guarded and safe.” Fast forward to 2020👇 https://twitter.com/...
@razhael Raphael Satter on x
Huge scoop from ⁦@Bing_Chris⁩: the US Treasury and the US NTIA have been breached by hackers. A foreign government is suspected and the National Security Council met Saturday to discuss the fallout. https://www.reuters.com/...
@robertjdenault Robert J. DeNault on x
Major story here. Foreign government has hacked Treasury; Trump removed the entire leadership at the Cyber and Infrastructure Agency weeks ago and there is a bevy of inexperienced folks having emergency meeting to respond. https://twitter.com/...
@w7voa Steve Herman on x
This #cyberattack of @USTreasury and other US government agencies, from which data was stolen, by a foreign group linked to a hostile government reportedly prompted an emergency @WHNSC meeting. https://twitter.com/...
@mattk Matt on x
Is this part of his declared declassification strategy? https://twitter.com/...
@grantstern Grant Stern on x
Remember when Trump announced a joint US-Russian cyber security service? https://www.washingtonpost.com/ ...
@underthebreach Alon Gal on x
Leaked image of what the hackers managed to do to the US Treasury https://twitter.com/... https://twitter.com/...
@joeuchill Joe Uchill on x
This is obviously bad. But there's also often a rush to assume things like this are attempts at sabotage rather than strategic information gathering. There are tiers of bad that aren't crashing the telecommunications grid or overtly harming Americans. https://www.reuters.com/...
@bing_chris Chris Bing on x
It's kind of wild this hacking campaign went undetected for months and then it hit FireEye and then everyone could see it. Big flex and poor targeting choice?
@daveweigel Dave Weigel on x
All I know is that I woke up with $500b in my bank account and I'm not gonna ask any questions about it. https://twitter.com/...
@emptywheel Dr. emptywheel on x
And I mean, literally. Trump went into FBI and DOJ and made sure that everyone with specific expertise in fighting Russia was chased out of there. Welp, Mr. President. They just pwned Treasury. And you not just own that, you invited that.

Chronicles

Source: Treasury's hackers used a flaw in a SolarWinds product; SolarWinds, which touts 300K+ customers, says the flaw was the result of a “supply chain attack”

Related Coverage

Discussion