Google’s I/O 2024 keynote recently concluded, focusing primarily on artificial intelligence, as anticipated. The presentation explained the future role of AI in Android, introducing Gemini as the new virtual assistant. Gemini is set to replace Google Assistant, offering more integrated and contextual features.
Since launching the Gemini app in February, Google has started to expand Gemini for Android by adding a collection of new features. These enhancements aim to integrate AI into various daily activities.
Google presented these updates as a complete redesign of Android. However, for the time being, they mainly serve as additional enhancements to the overall Android experience.
Gemini is designed to hover over your current screen activity instead of occupying the full screen, similar to how Google Assistant appears. Google is modifying Gemini’s layout to include a large text field for typing prompts, reducing the emphasis on voice commands.
The updated overlay aims to integrate more closely with the app you’re using and provide controls that are relevant to the context.
For instance, Google demonstrated that if you’re watching a YouTube video and open Gemini, a button labeled “Ask this video” appears. This feature lets you ask questions about the video or summarize its content using its knowledge base. The same functionality is available for PDFs with a subscription to Gemini Advanced, which offers a longer context window.
Gemini will operate more smoothly within apps, especially with features like drag-and-drop. At the keynote, Google showed how you can request an image from the chatbot, drag the resulting image into a messaging app, drop it, and then send it to a friend.
Google also mentioned that over time, Gemini will better understand the apps on your phone and enhance navigation through Dynamic Suggestions.
Google is enhancing the Circle to Search feature, which is already in use on more than 100 million Android devices, to assist with homework.
This upgrade will specifically aid students in tackling difficult physics and math word problems. It will provide a detailed explanation on how to solve these problems, eliminating the need for students to refer to their digital information sheets or syllabi.
The new feature is driven by Google’s LearnLM model, which is designed to simplify learning through AI. Google states that in the future, Circle to Search will be capable of solving more intricate problems that include symbolic formulas, diagrams, and graphs.
Google has also revealed an upgrade for Gemini Nano, a model integrated into Android on a limited number of devices. The enhancement, named “Gemini Nano with Multimodality,” will allow the LLM to process various media inputs like text, photos, videos, and voice. This will enable users to interact with Gemini in multiple ways to get answers, information, and more.
This model will also support features such as TalkBack, which provides text descriptions of images, and real-time spam alerts during phone calls. This is useful, for instance, if an unknown caller claims to be a foreign dignitary needing a large sum of money.
These are a few of the AI features that Google plans to introduce to Android 15 and later versions over time. Some features will debut on Pixel devices, while others will be accessible to those who download the Gemini app. It’s uncertain how this will all unfold, but it’s evident that Android phones are set to become significantly more powerful.
What we think?
I think the new Android features from Google’s I/O 2024 are exciting. The Gemini app seems really useful. It can float on my screen and help with things like watching videos or dealing with documents.
The Circle to Search upgrade sounds great for students, and I’m curious about the Gemini Nano with Multimodality. These features should make Android phones much smarter and more helpful in everyday tasks.
