Q&A with OpenAI VP of Hardware Richard Ho on its Jalapeño inference chip co-designed with Broadcom, using internal OpenAI models to design the chip, and more
Context & Ripple Effects
OpenAI and Broadcom unveiled Jalapeño in June as an LLM-focused inference chip whose design-to-tape-out process used OpenAI models. August coverage then reported that the ASIC outperformed Nvidia, AMD, and Google chips on several open-weight models, making the interview a useful look at the operating choices behind a hardware program already positioned around inference efficiency.
First-order effects
- OpenAI gains a tighter feedback loop between the models it serves and the silicon that runs them, while Broadcom becomes the co-design partner translating those workload requirements into a manufacturable inference chip.
- The account of using internal models in chip development turns Jalapeño from a procurement alternative into evidence that OpenAI is applying its AI stack to the design process itself.
Second-order effects
- Nvidia, AMD, and Google face a more demanding comparison point for inference: chip performance is increasingly judged alongside model-specific latency and work-per-watt, not general-purpose accelerator capability alone.
- Broadcom's role makes system-level co-design a more important route for AI customers seeking tailored inference capacity, raising the value of hardware teams that can work directly with model developers.
Third-order effects
- If model-assisted hardware design and workload-specific ASICs continue to deliver the reported advantages, AI infrastructure competition will shift toward integrated model, software, and silicon stacks rather than standalone chips.
- That shift favors AI developers with both sustained inference demand and specialized hardware talent, potentially widening the gap between firms that can co-design silicon and those that buy broadly available accelerators.
The trend: Jalapeño is part of a shift toward vertically integrated AI inference stacks, in which model developers co-design specialized silicon with semiconductor partners.