Microsoft details HoloLens' Holographic Processing Unit: TSMC-made, 24-core, performs 1T calculations/second, handles all environmental sensing and other inputs
How to make your own virtual reality brain — Hot Chips Microsoft today revealed a first look at the inside …
Context & Ripple Effects
The HPU has been a black box since Microsoft first sketched the HoloLens sensor stack — accelerometer, gyroscope, magnetometer, spatial sound — in its May 2015 deep dive (the inertial measurement unit detail), and the April teardown of the $3K Development Edition listed its components without explaining what its custom coprocessor actually did ($3K Development Edition teardown). At Hot Chips, Microsoft finally answers that question: the HPU is a 24-core, TSMC-manufactured processor running at one trillion calculations per second, absorbing every environmental sensing input so the holograms stay stable.
The timing matters because two months earlier Microsoft opened Windows Holographic to outside hardware partners — Intel, AMD, HTC, Acer among them — turning HoloLens from a single device into a reference platform (the Windows Holographic partner opening). Detailing the HPU publicly is effectively publishing the silicon blueprint those partners are expected to build against.
First-order effects
- Buyers and developers of the $3K Development Edition now know why the headset carries custom silicon: the HPU handles all environmental sensing and inputs on-device, freeing the main processor from perception work.
- TSMC picks up Microsoft as a custom-chip customer in a new device category, and the Hot Chips disclosure hands Windows Holographic partners a concrete perception-silicon architecture to target.
Second-order effects
- Partners like Intel, AMD, HTC, and Acer building Windows Holographic devices must decide whether to license or replicate dedicated perception coprocessing, or differentiate by keeping that workload on their own chips.
- A dedicated sensing coprocessor sets a power and latency baseline competitors' headsets have to match — general-purpose processing of sensor streams starts to look like a design penalty.
Third-order effects
- The pattern held within a year: Microsoft went on to design another custom AI chip for the next HoloLens so mixed reality goggles could recognize speech and images on-device instead of in the cloud (the next-generation HoloLens AI chip) — evidence that AR headsets were heading toward stacks of workload-specific silicon rather than one general-purpose brain.
The trend: Headset makers are shifting perception and AI workloads off general-purpose processors and the cloud onto dedicated, TSMC-fabricated coprocessors built into the glasses themselves.