Facebook announces the Passthrough API, which allows Oculus Quest 2 developers to use real-world images collected via sensors to create mixed reality apps
Developers will get the experimental feature with Oculus' next SDK — Facebook announced a new API that will let developers incorporate video …
The VergeMitchell Clark
Context & Ripple Effects
Oculus Quest’s inside-out tracking supplied the sensor foundation before Facebook opened it to application makers. The Passthrough API turns that tracking and spatial-sensing stack into an experimental developer input for mixed-reality experiences.
The move also follows App Lab’s alternative distribution path for Quest developers and precedes the broader Presence Platform SDK suite. Later Quest 3/3S coverage shows Meta continued treating camera access as experimental and separately constrained app-store distribution.
First-order effects
Quest 2 developers gain experimental access to real-world imagery from headset sensors, allowing mixed-reality apps to combine a user’s surroundings with virtual content.
Facebook expands the Quest SDK from headset interaction and tracking capabilities toward applications that depend on live environmental perception.
Second-order effects
Developers using App Lab have a route to test and distribute mixed-reality software outside the main Oculus Store, increasing the practical value of the new API for smaller Quest publishers.
Facebook’s subsequent Presence Platform SDKs package related mixed-reality capabilities more broadly, raising the incentive for developers to build around its platform rather than isolated headset features.
Third-order effects
The later experimental Quest 3/3S camera API indicates that access to a headset’s view of the physical world is becoming a continuing platform layer, while store-policy controls remain a gate on commercialization.
Mixed-reality software ecosystems are likely to be shaped not only by headset sensors but by the platform owner’s API permissions and distribution rules.
The trend: VR headset makers are evolving their tracking systems into controlled perception APIs for mixed-reality developers.
Developers will be able to start building and testing mixed reality experiences on Quest 2 with Passthrough API Experimental—an important step in unlocking the next generation of immersive experiences: https://ocul.us/... (going to miss getting this FAQ at my AMAs =)) https://twi…
Pretty wild to see Quest becoming a dev kit for the future of mixed reality applications. Still feels like we're stuck in the CRT days, but the fundamental pieces are all here! https://twitter.com/...
Ok this is a huge win towards #MixedReality 🥽 🔥 ! Having a device that does VR and allowing mixed reality is sooo huge, this is a $299 device and there is nothing available today to learn with such a low cost of entry. https://twitter.com/...
Wow! Can't wait to see what devs come up with. Hopefully this also means we get to record it too as passthrough is currently blocked from being captured. https://twitter.com/...
Excited for 10 years in the future when everyone is questioning the SEC and DOJ on how they allowed Facebook become a monopoly in the obviously insanely valuable virtual reality space (when in reality no one really cared)
A bit disappointed that the Oculus Passthrough API doesn't provide images for fun computer vision experiments, but definitely the right decision from a privacy point of view https://developer.oculus.com/ ...
@mattmiesnieks I've seen higher resolution and low-latency VR headsets with passthrough, but the inevitable offset between my actual eyes & the virtual eyes/camera threw off my whole proprioceptive orientation & body schema. I'm sure I'd adapt, but VR passthrough has a weird disc…
It'd be really cool if Oculus could also provide a “person segmentation” feature similar to that of #ARKit. Imagine being able to see real people around you transposed into your VR scene, multiplayer VR with people in the same room would look really cool! https://twitter.com/...
Pretty frustrated that they are neglecting Unreal here. Medieval Designer could have been the first to implement the mixed reality capabilities of the quest because it is 100% perfectly built for it, but because of nonsense decisions like “unity first” it won't be. https://twitte…
Good stepping stone for them to get developers into the full on AR pipeline and show users the potential But Quest 2 Passthrough looks way too garbage for it to be used as a dedicated AR/MR device https://twitter.com/...
Facebook shared some details about its experimental Passthrough API to enable new kinds of mixed reality apps for Oculus Quest 2. https://uploadvr.com/... https://twitter.com/...
@kentbye I think that's the right way to think about passthrough video on a VR HMD form factor. A “feature” to make VR a bit more convenient, and a great way to prototype and test various ideas for future outside-wearable see-through AR glasses
There's going to be a lot of really interesting AR, Mixed Reality, & IoT apps that are prototyped using a VR headset with this new Passthrough API. Rather than thinking about VR vs AR markets, I prefer to think of VR helping bootstrap the HCI, UX, & UI for AR apps. Each has +/- h…
This is fantastic. By making true AR API available on cheap dev devices today, it will jumpstart innovation and the development of apps that will make future AR glasses awesome from day 1 https://twitter.com/...
The Oculus Quest was never just a VR headset - its been an XR headset all along. Now more obvious than ever with the new Passthrough API https://twitter.com/...
@kentbye The passthrough tech will get awesome & close enough UX to see-through.... But the real issue why passthrough won't ever hit scale for AR use-cases is it is a solution to get around the huge black box on your face, that no one would ever wear in public :)
🤓 @Oculus just enabled the ability to create and test #MixedReality experiences on Quest 2 with their Passthrough API. This is now possible on a $299 #VR headset. Pretty remarkable. https://twitter.com/...
Oculus announcing that v31 update for Quest will have passthrough API 🔥 It'll be an exciting new chapter when headsets get good color passthrough. Still a solid range of practical and interesting applications possible even with current black and white.