Google's Gemini Live Can Now Read Phone Screens, Camera Feeds And Answer Queries

Google's conversational AI chatbot application, Gemini Live, now has the ability to read phone screens and camera feeds and answers questions on them in real time, according to a report by the Verge.

This feature is currently only available to a select number of subscribers of Google's Gemini Advanced, as a part of its Google One AI Premium plan.

According to a report by 9to5Google, the feature was made available to a user who saw a 'Share screen with Live' prompt appear above their 'Ask about you screen' option, presented by Gemini Live. The user also reported that they had access to the 'Live with video' feature that enabled them to start a video stream by opening the full application. The AI can then scan the environment and answers questions on what it's seeing in the camera feed.

These features were developed by Google's 'Project Astra' an intiative headed by Demis Hassabis, the chief executive officer and co-founder of Google Deepmind. Project Astra's goal is to develop a multi modal AI that can act as an assistant in real time by analysing the physical world, according to the Verge.

It was expected to be able to process audio, video, images and text, while simultaneously answering questions and providing suggestions to users on the same.

Googe first unveiled these capabilities at the Mobile World Congress 2025 in Barcelona, having interactive demonstrations for attendees.

The tech company also released videos on YouTube ,featuring these capabilities.

The videos displayed the conversational aspect of Gemini Live, where the users asked it questions on aesthetics and design choices for clothing and pottery in each of the videos. The AI showcased the ability to have organic conversations, building on previous answers and adapting to new questions.