ARC Prize Foundation unveils ARC-AGI-3, an AI benchmark with simple video-game-like scenarios designed to measure on-the-fly reasoning rather than memory recall
ARC-AGI-3 tests whether models can reason through novel problems, not just recall patterns, a task even top systems still struggle to do.
Model developers gain a new public yardstick for testing whether systems adapt their reasoning during unfamiliar, sequential tasks rather than exploiting training-set familiarity.
Benchmark comparisons will become less transferable across ARC generations: a strong score on earlier ARC tests is not, by itself, evidence of performance in ARC-AGI-3’s interactive setting.
Second-order effects
Labs are likely to put greater weight on agent-style evaluation and test-time adaptation, because static benchmark gains will not address failures exposed by novel interactive tasks.
Buyers assessing AI for variable workflows get another reason to distinguish polished demonstrations and broad knowledge performance from evidence of reliable reasoning under changing conditions.
Third-order effects
If interactive reasoning benchmarks become influential, frontier-model competition will increasingly center on generalization and evaluation integrity, not just aggregate scores on fixed test sets.
The broader evaluation market may fragment into complementary tests for knowledge, mathematics, coding, research and interactive reasoning; no single headline score is likely to remain sufficient for capability claims.
The trend: AI evaluation is moving from benchmarks vulnerable to memorization toward harder, interactive tests intended to measure adaptable reasoning.
ARC-AGI-3 is an important benchmark. However, I have a major issue with the “Human score 100%” statement. How many humans have tested all 1000 puzzles? How were people selected? This was not published for previous ARCs either. In one case, the human score was based on I think 2
Keep in mind: ARC-AGI is *not* a final exam that you pass to claim AGI. Including ARC-AGI-3. The benchmarks target the residual gap between what's hard for AI and what's easy for humans. It's meant to be a tool to measure AGI progress and to drive researchers towards the most
ARC-AGI-3 benchmark: - 100% solvable by humans - 1% solvable by AI Everybody keep building benchmarks that agents utterly fail at! Proud this was a Laude Slingshot; will fund other benchmarks that reset SotA to 1%: https://www.laude.org/...
The 25 public ARC-AGI-3 games On average, they are easier for humans and AI However, the difficulty ranges, there are very easy games and games which are more difficult Easy for AI: https://arcprize.org/... Hard for AI: https://arcprize.org/...
ARC-AGI-3 scores agents on how close they are to human action efficiency. All ARC-AGI-3 environments were solved by at least 2 human testers out of 10 (most of the time it was 5+). We use the action count of the 2nd best tester (to avoid outlier performance) as our human
ARC-AGI-3 took me a few tries, but it is definitely human winnable. I am curious how much of the very initially very low performance of frontier models is harness, vision, and tools, versus how much are limitations of LLMs. I guess we will find out! https://arcprize.org/...
if we saturate arc 3 this year and there's no meaningful shift in the economy, it's clear benchmarks have become a gimmick between labs and providers to hill climb, market, make bread. whilst it's exciting to see a benchmark the models perform so poorly at. and i love this
At the moment, ARC-AGI-3 is the only unsaturated agentic AI benchmark. Sub-1% scores from frontier models on the private test set. If you want to be among the first to know when an AGI breakthrough happens, monitor the ARC-AGI-3 leaderboard. Any sudden score jump will mean
ARC-AGI-3 and ARC Prize 2026 are now live with $2,000,000 in prizes! As of today, version 3 is the world's only unsaturated agentic intelligence benchmark. Humans score 100% and frontier AI scores ~0%. Play here: https://arcprize.org/... While no single version of ARC is
ARC-AGI-3 is out now! We've designed the benchmark to evaluate agentic intelligence via interactive reasoning environments. Beating ARC-AGI-3 will be achieved when an AI system matches or exceeds human-level action efficiency on all environments, upon seeing them for the first [v…
It's alive! This 3rd version of ARC-AGI represents an incredible amount of work from the ARC Prize team. Hundreds of games. Thousands of levels. Go build agents!
Today we're launching ARC-AGI-3 135 Novel Environments (nearly 1K levels) we build by hand It is the only unsaturated agent benchmark in the world Each game is 100% human solvable, AI scores <1% This gap between human and AI performance proves we do not have AGI Agents today [vid…
We created an in-house game studio and built 135 novel environments from scratch No instructions, Core Knowledge Priors-only In order to beat these games, AI must: • Explore the environment • Form hypotheses • Execute a plan • Learn and adapt [image]
Announcing ARC-AGI-3 The only unsaturated agentic intelligence benchmark in the world Humans score 100%, AI <1% This human-AI gap demonstrates we do not yet have AGI Most benchmarks test what models already know, ARC-AGI-3 tests how they learn [image]