2025-11-15
Very rare glimpse inside LLM architecture at closed AI labs. Grok-3 and Grok-4 were 3 trillion total parameters, Grok-5 will be 6 trillion. This still doesn't tell us active parameters for the models (given Mixture of Experts architectures) but it sounds like Grok-5 will be
The Information
Elon Musk says Grok 5 would be released “in Q1 sometime”, later than a deadline he previously set of releasing the model by the end of 2025
Says Musk Is ‘Like Da Vinci’ X: @scaling01 : Huge Leaks on Grok-5 and its predecessors from recent Elon Musk interview: - “Grok-5 is a 6 trillion parameter model, whereas Grok 3 an...
2024-07-19
To continue this trend and evidence of rapid AI/ LLM deflation; OpenAI's new GPT-4o-mini out today is an average ~140x cheaper than GPT-4 was at release in March 2023 (also mostly better!), while ~230x cheaper and vastly better than Da-Vinci 002 in Aug-22 (the best model at the [image]
Simon Willison's Weblog
GPT-4o mini costs $0.15 per 1M input tokens and $0.60 per 1M output tokens, prices lower than those of Claude 3 Haiku and Gemini 1.5 Flash
GPT-4o mini. I've been complaining about how under-powered GPT 3.5 is for the price for a while now (I made fun of it in a keynote a few weeks ago).