JFrog finds ~100 malicious PyTorch and Tensorflow Keras models on Hugging Face, some of which can execute code on users' machines to give attackers a backdoor
BleepingComputerBill Toulas
Context & Ripple Effects
This extends an emerging AI-software supply-chain problem: the earlier discovery of remotely exploitable flaws in the TorchServe model-serving tool showed that risk does not stop at model training or download.
The reported models turn a widely used repository into a potential code-execution entry point, making the provenance and loading behavior of ML artifacts a security boundary rather than a simple distribution concern.
First-order effects
Users who download and load the identified PyTorch or TensorFlow Keras artifacts risk executing attacker-controlled code and receiving a backdoor on their machines.
Hugging Face users and teams must treat affected model files as untrusted software, while JFrog’s finding gives defenders concrete artifacts to identify and remove.
Second-order effects
ML teams will face added review and isolation work before adopting community models, slowing direct reuse of third-party artifacts where provenance cannot be established.
Model repositories and framework ecosystems will be pressured to improve malware scanning, artifact verification, and safer loading defaults; the TorchServe flaws illustrate that exposure can span both model files and serving infrastructure.
Third-order effects
If malicious models remain a recurring distribution channel, model hubs may evolve toward software-package-style trust controls, with signed provenance and execution isolation becoming more important selection criteria.
The broader attack surface will increasingly include model artifacts, dependencies, and infrastructure together; later reports of an AI-driven breach of Hugging Face’s internal systems underscore why repository security is consequential beyond any one malicious upload.
The trend: Open ML distribution is becoming a software supply-chain security problem in which model files themselves must be governed as potentially executable code.
Who could possibly have expected that a highly trafficked site that makes available code for you to run might be compromised in this way? I'm still wondering about all the supply chain vulnerabilities for libraries python, java, and javascript [embedded post]
Yes we must build a marketplace where people can easily purchase and integrate models on the back of a hype wave that has sucked up all the capital and delivered fuck all. We will protect against malware by allowing users to rate the models out of 5.
ML models = software code trained against data. Any software code can have vulnerabilities. Despite Hugging Face's efforts to scan them for malicious intent, bad actors will find a way to get through. No customers should be blindly trusting 3P code. Do your own testing.
ML models are not inspectable and not interpretable. To prevent malware, you should only use models with supply chain provenance and signature from identities you trust https://jfrog.com/...
🚨 Is Hugging Face the target of model-based attacks? Our #security research team conducted a #HuggingFace deep dive and created a detailed explanation of the attack mechanism. Read the full analysis & what's required to identify real threats: https://jfrog.com/... [image]
The research by the @jfrog team is worthy of a read: https://jfrog.com/... What's old is new again and I'm drawn to @marcoslaviero's epic @sensepost Pickle research from 13 years ago https://sensepost.com/...