Google teases Project Astra, a camera-based AI app to process visual data in real time, and showed a person wearing a pair of smart glasses using the feature

It'll even remember things it saw that are no longer in the frame. — Deputy Editor, Reviews

Engadget 2024-05-15 Cherlynn Low

Context & Ripple Effects

Project Astra extends Google’s earlier lightweight AR-glasses concept for live transcription and translation from a single-purpose display toward an assistant that interprets a changing physical environment.

The demo matters because Google positioned real-time visual understanding and recall as a glasses-capable interface; later coverage of Astra recording, summarizing, and answering questions about video shows that the effort continued toward richer multimodal capture rather than a one-off showcase.

First-order effects

Google gives Project Astra a clearer product direction: a camera-led AI interface that can answer questions about what it sees and retain context after an object leaves view.
Smart glasses become a visible potential delivery mechanism for Gemini-era visual assistance, while the report remains a teaser rather than a launch commitment.

Second-order effects

The demonstration raises the bar for AI and wearable rivals: real-time vision needs to be paired with useful memory and conversational responses, not just camera hardware or display features.
It also makes eyewear partnerships more strategically relevant, consistent with reports that Google later explored bringing Gemini to future smart glasses with EssilorLuxottica.

Third-order effects

If these capabilities reach consumer hardware, AI interaction could shift from app-by-app prompts toward persistent, camera-mediated assistance embedded in everyday devices.
That shift would make controls over what is captured, retained, and recalled a core product constraint for ambient AI, especially when the camera observes other people and private spaces.

The trend: Project Astra is one data point in the move toward ambient, sensor-native AI that treats visual context as a continuous input rather than a file a user must explicitly upload.

Discussion

@rihardjarc @rihardjarc on threads
The market about to realize soon that smart glasses are the next computing platform with AI assistants. Guess who already has a great market position with a $300 affordable & stylish product that is selling out in many places - META
@googledeepmind @googledeepmind on x
We're sharing Project Astra: our new project focused on building a future AI assistant that can be truly helpful in everyday life. 🤝 Watch it in action, with two parts - each was captured in a single take, in real time. ↓ #GoogleIO [video]
@kayjebelli Kay Jebelli on x
Some influential policy people have called AI a “vanity project”. Don't elect politicians that listen to those people.
@mishaalrahman Mishaal Rahman on x
Here's a demo of the new Project Astra assistant! Pretty cool to see it on smart glasses too. Some of these agent experiences will come to the Gemini app later this year. [video]
@backlon Dieter Bohn on x
Project Astra: fully mutimodal, understands context, low latency, fast. And don't miss the demo of those glasses 😎 [image]
@jetscott Scott Stein on x
Sergey sees the parallels between Project Astra and Google Glass. So do I https://www.cnet.com/...
@anshelsag Anshel Sag on x
No #XR announcements at #GoogleIO and barely a teaser of what seems to be Gemini running on #AR glasses. That's about it. No @Samsung headset teasers. Honestly the most exciting thing was Gemini Powered Photos which felt like it should've launched last year.
@shiringhaffary Shirin Ghaffary on x
I asked Sergey Brin if Google Glass is coming back, especially given the hardware tease we saw today. He told me: “It's funny because it's the perfect hardware. It's the killer app now, 10 years later. Unfortunately, we sort of messed up on the timing.” https://www.bloomberg.com/…
@caro_milanesi Carolina Milanesi on x
it is so obvious now the value of using your glasses as your camera viewer and the power of coupling that with AI - @Qualcomm @Meta and @ray_ban should be given credit to show the way
@demishassabis Demis Hassabis on x
For a long time, we've been working towards a universal AI agent that can be truly helpful in everyday life. Today at #GoogleIO we showed off our latest progress towards this: Project Astra. Here's a video of our prototype, captured in real time. [video]
@briantong Brian Tong on x
Yoooooo! Project Astra! Was that a new Glasses product teaser? 👀🤯 #GoogleIO [video]
@jeffdean Jeff Dean on x
Our Gemini model enjoyed watching yesterday's OpenAI event. Nice work, @miramurati, @barret_zoph and @markchen90 (and everyone else behind the scenes)!
@anshelsag Anshel Sag on x
Right before she put on the glasses I was thinking this whole demo would be much better hands free with AR glasses lol
@elicollins Eli Collins on x
Project Astra brings us closer to more helpful, intuitive AI assistants. Building on Gemini, we made prototype agents that can process info faster, making interactions feel more natural. Some of these capabilities will appear in Google products this year.
@zsk Zoe Kleinman on x
So long Alexa, Siri and co... What we've seen in the last 24 hours, first from OpenAI and now from Google, blows current digital assistants out of the water. GPT-4o and Project Astra can interpret the world around them, recognising audio, video and images. They do so much more...
@willknight Will Knight on x
I got an early look at ASTRA, Google's “next-gen” AI assistant. Many experts believe AI models will need a more sophisticated understanding of the physical world to overcomecurrent shortcomings. Story incoming... #GoogleIO2024 @WIRED [image]
@waitin4agi_ Varun Mayya on x
Agents are finally here. Google's calling it Astra, and it's a personal agent for everyone. Teachable, personal, no latency. [image]
@alphasignalai Lior on x
Google just announced Astra. It's the future of AI assistants. It can interact with the world around it by taking in information, remembering what it sees, processing that information and understanding contextual details. [video]
@krishnanrohit Rohit on x
Bullish on Google !!!!
@officiallogank Logan Kilpatrick on x
Project Astra is here [video]
@scobleizer Robert Scoble on x
The press is fast. I saw it too. Spatial Computing is XR+AI. The key word in this demo is “prototype.” Betting windows are now open for when we'll have Project Astra on our phones. It's like going to the Consumer Electronics Show. You know that most of the cool things...
@bindureddy Bindu Reddy on x
After watching Google I/O, it's safe to say what OAI showed yesterday was mind-blowing!!🤯🤯 Astra is a prototype voice assistant and seemed like a 2-year-old baby to OAI's Scarlett Johansson!!
@joannastern Joanna Stern on x
Google just quietly showed off the next... Google Glass? A pair of glasses—similar to the Ray-Ban Meta smart glasses—that can see the world via cameras and answer questions. [image]
@julianchokkattu Julian Chokkattu on x
Google's Project Astra in action. Including how it works on prototype AR glasses. #GoogleIO2024 via @WIRED [video]
@markhachman Mark Hachman on x
It has to be “proactive. teachable and personal” and multimodal. Google has developed agents that can understand video, create a timeline of events, and use intonations. Prototype video: “Tell me when you see something that makes sound.” AI identifies a speaker. #GoogleIO2024
r/singularity r on reddit
Project Astra: Our vision for the future of AI assistants
@thesullivan @thesullivan on x
The #Android fraud/scam warning feature is quite cool. Almost a good enough reason to switch my mom to Google phone. Apple, your move. #GoogleIO
@leakerapple @leakerapple on x
Google is one of the few companies which uses AI to make a real difference to people's lives. Other companies are busy with image generation and chat bots for no apparent reason.
@google @google on x
Circle to Search now makes for a great study buddy 📝 You can circle complex physics problems on your phone or tablet to get step-by-step instructions to learn how to solve. #GoogleIO [image]
@google @google on x
Introducing LearnLM: our new family of models based on Gemini and fine-tuned for learning. LearnLM applies educational research to make our products — like Search, Gemini and YouTube — more personal, active and engaging for learners. #GoogleIO [image]
@google @google on x
The new Learning coach Gem uses LearnLM to provide step-by-step study guidance, helping you build understanding instead of just giving you an answer. It will launch in Gemini in the coming months. #GoogleIO [image]
@mishaalrahman Mishaal Rahman on x
LearnLM is a new language model based on Gemini that's optimized for education. Homework help in Circle to Search is powered by it. A new premade learning coach Gem in the Gemini Android is also powered by it. YouTube is also getting a LearnLM-powered lecture question feature. [i…

Chronicles