Discussion

• @johnschulman2 John Schulman on x

  Enjoyed chatting with Dwarkesh, Beren, and Charlie. Thanks for having us on, Dwarkesh!

• @miramurati Mira Murati on x

  An important conversation about where AI is going and the work that is cut out for us.

• @andrewdai Andrew M. Dai on x

  Next token prediction doesn't work because it accurately models how human brains learn. It works because it's one of the only objectives that scale, unlike some other ones.

• @sriramk Sriram Krishnan on x

  @dwarkesh_sp This was super interesting

• @berenmillidge Beren Millidge on x

  To address concentration of power risks, I think a healthy US-based open model ecosystem is essential and hope my work at Zyphra as well as John's work at TML and Charlie's work at BaseLabs contributes to this

• @berenmillidge Beren Millidge on x

  I am thus very encouraged by the outpouring of support recently for a pause or pacing the frontier. I think this is a necessary step to avoid a future where humans are, at best, pets for a tiny oligarchic AI elite, and, at worst, are driven extinct entirely by misaligned AGI

• @berenmillidge Beren Millidge on x

  Recent events such as the Huggingface hack have demonstrated without doubt that we are deeply unprepared for RSI and are not close to solving alignment. We are reaching the point where existential risk is becoming real if the pace of capabilities improvement continues.

• @berenmillidge Beren Millidge on x

  Despite its intellectual interest, RSI is also a somewhat terrifying topic with deeply serious alignment and concentration of power concerns. If strong RSI begins to take off in the next few years then humanity is not prepared at all for this.

• @berenmillidge Beren Millidge on x

  It was a great conversation with @johnschulman2 and @oneill_c and I definitely learned a lot. Thanks @dwarkesh_sp for pulling this together! Understanding where we stand with RSI and how well current RL methods scale is an extremely important and interesting question

• @baseten @baseten on x

  Our own @oneill_c joins @dwarkesh_sp, @johnschulman2, and @BerenMillidge for a deep discussion on long-horizon RL, automated AI research, and what comes next at the frontier.

• @thinkymachines @thinkymachines on x

  Our own @johnschulman2 talks with Dwarkesh about where human judgment still matters as models improve and self-improve: teaching them to handle messy real-world tasks, applying taste to what works in the long run, and, above all, specifying what we actually want.

• @oneill_c Charlie O'Neill on x

  I think one of the more interesting things we debated here is whether RSI is a cumulative task. Attention plus MoE plus GRPO etc seems to me like a line in the sand that you can just add to the stack once you discover it. You don't need to take five steps back to take 10 steps fo…

• @dwarkesh_sp Dwarkesh Patel on x

  New episode with @johnschulman2, @oneill_c and @BerenMillidge. I got together with some of the most insightful AI researchers I know who are at the openish companies, because I wanted to hear the details of what's actually happening at the frontier and what comes next. 0:00:00 - …

• @dwarkesh_sp Dwarkesh Patel on x

  In the early days of OpenAI, @johnschulman2 didn't think next-token prediction was going to lead to intelligence, because it would get swamped by noise. He explains it's always been hard to apriori predict what techniques will elicit out-of-distribution generalization.

• @dwarkesh_sp Dwarkesh Patel on x

  Was really interesting to hear John, Beren, and Charlie speculate about why Sonnet 5 and Opus 5 feel like worse models than GLM 5.3 (despite the fact that Anthropic can do raw logit distillation from Fable, and can also train Sonnet/Opus on the environments from which Fable was t…