The Allen Institute for AI releases Tulu 3 405B, an open source model that it claims outperforms DeepSeek V3 and OpenAI's GPT-4o on certain benchmarks

Move over, DeepSeek. There's a new AI champion in town — and they're American. — On Thursday, Ai2, a nonprofit AI research institute based …

TechCrunch 2025-01-31 Kyle Wiggers

Context & Ripple Effects

DeepSeek had just raised the stakes for open models with its DeepSeek-V3 release, which also positioned itself against leading proprietary systems. AI2's new result turns that comparison into a broader contest among open-model developers rather than a single challenger narrative.

The release also lands alongside reports that OpenAI was preparing a faster o3-mini reasoning model, underscoring how benchmark performance and product cadence were becoming simultaneous fronts in model competition.

First-order effects

AI2 gains an open-source flagship model it says exceeds DeepSeek V3 and GPT-4o on selected benchmarks, giving researchers and developers another large-model option to evaluate.
The claims put immediate comparative pressure on DeepSeek V3 and OpenAI's GPT-4o positioning, while leaving the practical significance dependent on the benchmarks and users' own testing.

Second-order effects

Model buyers and evaluators have more reason to compare open alternatives against proprietary offerings on task-specific performance, not only on brand or access model.
Competing labs are incentivized to answer with new releases, clearer evaluations, or product improvements; OpenAI's reported o3-mini plans show the parallel pace of that response cycle.

Third-order effects

If repeated across releases, strong open models could make benchmark leadership more contestable and shorten the period in which any one model's reported advantage shapes developer choice.
The larger divide may shift from simply having a capable model to distribution, deployment support, and cost per useful task, though benchmark claims alone do not establish those advantages.

The trend: This is one data point in the industrialization of AI models, where open releases increasingly compete with proprietary systems on measured capability while differentiation moves toward deployment and distribution.

Discussion

@cuthrell.com Jay Cuthrell on bluesky
As the newsworthy claims and benchmarks seasons get shorter it will be fascinating to see how @mlcommons.org grows
@natolambert Nathan Lambert on x
Very happy to show that we can do RL finetuning on 405B models with open-source code, beat Llama 405B instruct with their base model, and beat DeepSeek V3 too. Enjoy building off this teams hard work. Here's Tulu 3 405B. A holiday present from @hamishivi, @vwxyzjn and team.
@vwxyzjn Costa Huang on x
🎁 Happy New Year!!! We are bringing new RL curves as presents. This time, we went beeeeeg (405B). RLVR + MATH just worked: training and testing performance are still going up 😍 [image]
@hannahajishirzi Hanna Hajishirzi on x
Excited to release our newest, largest, and best Tulu yet. Our RLVR recipe works at scale, outperforming Deepseek V3. So proud of the team! And @hamishivi @vwxyzjn for scaling up the Tulu recipe. [image]
@hamishivi Hamish Ivison on x
li'l holiday project from the tulu team :) Scaling up the Tulu recipe to 405B works pretty well! We mainly see this as confirmation that open-instruct scales to large-scale training — more exciting and ambitious things to come! [image]
@tim_dettmers Tim Dettmers on x
Beating DeepSeek-V3 with a 405B Llama base is not easy — solid post-training goes a long way. The nice thing is that it is fully open-source, so anyone can use this recipe for their base models.
@allen_ai @allen_ai on x
Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on [i…

Chronicles