Ilya Sutskever says “we're back in the age of wonder and discovery” as AI companies focus on pre-training, inference improvements, and finding the “next thing”
There's been a lot of chatter about the “end of LLMs” or “LLMs starting to fail.” Jon Keegan / Sherwood News : OpenAI models may be starting to plateau Siddharth Jindal / Analytics India Magazine : OpenAI is So Doomed if Inference Time Scaling for o1 Fails Bluesky: Jan Jęcz / @jeczjan.bsky.social : Investors who poured 1 bln $ into his startup in the seed round must be thrilled to hear that they financed “wonder and discovery” [embedded post] X: @modestproposal1 : “This shift will move us from a world of massive pre-training clusters toward inference clouds, which are distributed, cloud-based servers for inference,” Sonya Huang, a partner at Sequoia Capital, told Reuters. https://www.reuters.com/... @modestproposal1 : “Ilya Sutskever, co-founder of AI labs SSI and OpenAI, told Reuters results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued” https://www.reuters.com/... Krystal Hu / @readkrystalhu : As we heard more about the failed training runs in major AI labs, @annatonger & I wrote about the question on everyone's mind—It's been two years since GPT4. What's after the pre-training scaling law? Spoiler: Ilya has something to say about it Link: https://www.reuters.com/... [image] Forums: r/aiwars : OpenAI and others seek new path to smarter AI as current methods hit limitations r/mlscaling : OpenAI and others seek new path to smarter AI as current methods hit limitations Msmash / Slashdot : OpenAI and Others Seek New Path To Smarter AI as Current Methods Hit Limitations