Meta's AI lab creates Open Pretrained Transformer, a language model trained with 175B parameters to match GPT-3's size, and gives it to researchers for free
MIT Technology ReviewWill Douglas Heaven
Context & Ripple Effects
OpenAI’s 175B-parameter GPT-3 release established the scale benchmark that Meta is now matching while changing the access model: researchers receive the weights without charge. The move matters because it gives academic researchers a comparably sized model to study rather than leaving that capability concentrated with its original developer.
Meta’s subsequent open-source translation work across 200 languages shows that OPT sits within a broader research-distribution approach, not an isolated release.
First-order effects
Researchers gain free access to a 175B-parameter language model, enabling experimentation at the same parameter scale as GPT-3.
Meta positions its AI lab as an alternative source of frontier-scale language-model research access, while OpenAI’s GPT-3 becomes the direct comparison point.
Second-order effects
OpenAI faces stronger pressure to differentiate GPT-3 through capabilities and access terms when Meta makes a same-scale research model available for free.
The release expands the pool of researchers able to evaluate and build on large language models, creating demand for the computing and tooling needed to use them.
Third-order effects
Meta’s later open-sourced multilingual translation model suggests a durable competitive pattern in which major AI labs use open model releases to attract research ecosystems around their work.
If large-model access continues to spread through free research releases, differentiation shifts from parameter count alone toward the surrounding tools, deployment channels, and specialized models.
The trend: Frontier AI labs are turning model access into a competitive lever, pairing large-scale research releases with broader open-model ecosystems.
Actually I said: I applaud the transparency here, not just in releasing the model but also the information about training compute cost and the like. I would hope that the transparency extends to very thorough documentation of the source datasets, as well. https://www.technologyre…
And while we're here, I'd like to add: “democratizing” is used in such a misleading way in “AI”. “Democracy” entails shared *governance* not just “anyone can come play with it”. (From the title of the Meta blog post.) https://ai.facebook.com/...
Today Meta AI is sharing OPT-175B, the first 175-billion-parameter language model to be made available to the broader AI research community. OPT-175B can generate creative text on a vast range of topics. Learn more & request access: https://ai.facebook.com/... https://twitter.com…
Meta's OPT 175B is a nice “behind the scenes” take on training LLMs. Instability in both hardware and training is a big challenge https://arxiv.org/...
OPT-175b: Open Pre-Trained language model with 175 billion paramaters is now available to the research community. Blog post: https://ai.facebook.com/... Paper: https://arxiv.org/... Code + small pre-trained models: https://github.com/... (using OPT-175b requires a registration) h…
I've tried using GitHub copilot for LaTeX, but its prose recommendations tend to not be nearly as useful as its code suggestions (which are eerily accurate and well-conditioned). I am really curious how this would change if someone trained this OPT model on arXiv exclusively... h…
Excited about this release! Great example of the kind of transparency that can advance insights into Responsible AI. Hope to try it myself sometime soon.... Congrats @ylecun, @schrep and team! https://twitter.com/...
tangibly useful. refreshing in the era of closed irreproducible LLMs (the day-to-day log book is pretty entertaining to read) congrats @suchenzang @stephenroller @NamanGoyal21 and team https://twitter.com/...
Massive language models have been a tremendous AI advance of the past few years. But for many use cases, they are not ready for prime time. @MetaAI made OPT-175B available to all researchers to help the community study their flaws and remedy them https://www.technologyreview.com/…
Heard of advances in AI called Large Language models - e.g. GPT-2/3? Up till now only researchers at a handful of large companies could actually see what was behind the curtain. Now Meta AI has released a fully trained model for researchers... https://www.technologyreview.com/ ..…
I'm quite proud of the level of transparency that @MetaAI is bringing to this large language model release—sharing an enormous amount of detail about how it was developed, what it was trained on, the compute's carbon footprint, and more. Hopefully this will become the standard. h…
Awesome news coming out of @MetaAI Joelle Pineau, @ylecun @schrep! With openness, transparency & collaboration, we can we foster responsible & inclusive progress, understanding & accountability (vs proprietary, monopolies & power concentration)! 👏👏👏 https://www.technologyreview.c…
Of course, reproducibility would also necessitate hardware accessibility to the greater public, and also deterministic frameworks which Pytorch is currently not
we now have an open-source LM of similar size to that of GPT3 🎉🎉🎉 using it tho will require many many GPUs, i think the required VRAM is > 300GB https://twitter.com/...
Meta has made its own version of GPT-3 - and it's going to give it to researchers to play with, including details of how it was made, reports @strwbilly https://www.technologyreview.com/ ...