Anthropic says Opus 4.7 uses “an updated tokenizer that improves how the model processes text”, but “the tradeoff is that the same input can map to more tokens”
Guide for migrating to Claude Opus 4.7 and Claude 4.6 models from previous Claude versionsLatent.Space:[AINews] Anthropic Claude Opus 4.7 - literally one step better than 4.6 in every dimension
9to5Mac Zac Hall
Related Coverage
Discussion
-
@realsigridjin
Sigrid Jin
on x
opus 4.7 has a new tokenizer which means a new base model underneath, not just a post-training refresh [image]
-
@maximelabonne
Maxime Labonne
on x
My bet is that Mythos uses a new tokenizer, and they switched Opus over to it (through midtraining) for distillation
-
@myainotez
@myainotez
on x
New Opus is out, they mention a new tokenizer too. Maybe we will have breadcrumbs of mythos in this one
-
@bspk_
@bspk_
on x
New base model!
-
@natolambert
Nathan Lambert
on x
There's good discussion around this one ways that it could just be adaptation at midtraining, but base model is the simplest explanation so that's my bet.
-
@schiste
Christophe Henner
on x
Oh gosh, they removed 4.6 altogether from selectors. So I have to say, this does looks a lot like a downsell disguised in an upsell. A few upgrades, but a new tokenizer eating tokens much faster. Well, we knew the time to stop brut forcing things with Opus had to end. [image]
-
@andrew_n_carr
Andrew Carr
on x
4.7 has a new tokenizer (in-part) because of the 3x vision scaling improvements
-
@topmass
Matthew
on x
opus 4.7 tokenizer is new and uses more tokens for the same inputs... AND the new default reasoning effort inside of claude code will be high - get ready to tear through your limits! [image]
-
@bogdanionutcir2
Bogdan Ionut Cirstea
on x
seems probably good for safety, especially if most capabilities gains came from pretraining
-
@bcherny
Boris Cherny
on x
Opus 4.7 uses more thinking tokens, so we've increased rate limits for all subscribers to make up for it. Enjoy!
-
@eliebakouch
Elie
on x
my take: opus 4.7 is a distilled version of mythos
-
@kunchamsathwik
@kunchamsathwik
on x
Claude Opus 4.7 launched Thing I noticed: 1. They changed the tokenizer which may map to 35% more tokens. 2. Model by default thinks more. Overall, higher token use and faster rate limit hits. [image]
-
@realsigridjin
Sigrid Jin
on x
tldr; @ClaudeDevs opus 4.7 just shipped as expected > the tokenizer changed. same input maps to 1.0 to 1.35x more tokens depending on content type > output tokens also go up at higher effort, the model thinks longer on later turns in agentic loops > new effort level called