Cerebras open sources seven GPT-based LLMs, ranging from 111M to 13B parameters and trained using its Andromeda supercomputer for AI, on GitHub and Hugging Face
Artificial intelligence chipmaker Cerebras Systems Inc. today announced it has trained and now released seven GPT-based large language models …
SiliconANGLEMike Wheatley
Context & Ripple Effects
Cerebras had already positioned its wafer-scale hardware around unusually large NLP training runs, including a single-device 20B-parameter training record. Releasing smaller, usable GPT-family checkpoints turns that hardware claim into code and models developers can inspect and run.
The move also lands in a language-model market shaped by the scale leap from GPT-3’s 175B-parameter release. Cerebras is differentiating less through a claim to the largest model than through public evidence of its training stack across several model sizes.
First-order effects
Developers can download and evaluate seven Cerebras-trained GPT models from GitHub and Hugging Face, lowering the friction to test the company’s work without commissioning a training run.
Cerebras gains public reference models for Andromeda: the releases make its compute platform more tangible to researchers and prospective customers.
Second-order effects
Competing AI hardware vendors face added pressure to pair performance claims with accessible model artifacts, benchmarks, or software workflows that demonstrate real training outcomes.
The availability of several sizes gives users more options to match model capacity to experimentation and deployment constraints, rather than treating a single large checkpoint as the only entry point.
Third-order effects
If hardware companies increasingly publish models alongside systems, AI infrastructure competition could shift toward integrated developer ecosystems—compute, training software, and reusable model assets—not chip specifications alone.
Open model releases can make model choice more contestable for buyers, while keeping the underlying availability and economics of large-scale training infrastructure strategically important.
The trend: This is one data point in AI infrastructure vendors using open model artifacts to convert specialized compute into a developer platform and strengthen buyer adoption.
New fully open source GPT-3 architecture models were just released. In 7 sizes, 13B being the largest. The training seems to follow compute optimal chinchilla formula of 20 tokens per model parameter https://huggingface.co/... https://twitter.com/...
🎉 Exciting news! Today we are releasing Cerebras-GPT, a family of 7 GPT models from 111M to 13B parameters trained using the Chinchilla formula. These are the highest accuracy models for a compute budget and are available today open-source! (1/5) Press: https://www.businesswire.c…
One notable output of Cerebras-GPT is a new scaling law that predicts model performance for a given compute budget. This is the first scaling law derived using a public dataset. (3/5) https://twitter.com/...
@CerebrasSystems You can also download the models yourself from HuggingFace, and they should be compatible with @AiEleuther's LM Eval Harness right out of the box: https://huggingface.co/...
they make a big deal about being “compute-optimal” but this doesn't actually make sense > less accuracy gains per FLOP that's only considering *training* FLOPs. Inference is cheaper on smaller models so it makes sense to train smaller models longer than is “compute-optimal” https…
7/ Cerebras CS-2s on the cloud are available today. You can train a model from scratch or fine tune your model of choice. It's faster than procuring a cluster from AWS! https://www.cerebras.net/...
1/ Very excited to be sharing our latest work today! Cerebras-GPT is our way of supporting open language models. We had two main goals: - Create a family of Chinchilla-optimal models using open data & methods - Showcase what Cerebras CS-2s can do today https://www.cerebras.net/..…
Nice work @CerebrasSystems! I'm happy to see that more models are getting out into the world and technology is making these capabilities accessible to many. Our customers @MosaicML have lots of options from which to build! Stay tuned for some updates from us on this... https://tw…