Google will upgrade more Android apps that will use Gemini Nano
On-device GenAI empowers Android apps with content consumption (summarization), creation (text suggestions), and analysis (sentiment detection). Benefits include privacy, offline use, and speed. Lower model size brings limitations but fine-tuning aids accuracy.
This article details the use cases and benefits of on-device generative AI (GenAI) for Android applications. It also highlights upcoming advancements in this field.
Focus on On-Device Processing
On-device AI is a key priority for the future of Android. Google presented developer resources at I/O 2024, showcasing the potential of on-device GenAI.
Use Cases for On-Device GenAI
The "Android on-device AI under the hood" session at I/O 2024 identified several valuable use cases for on-device GenAI:
- Content Consumption: Summarizing or providing overviews of text.
- Content Creation: Suggesting replies in messaging apps or generating/rephrasing text.
- Content Analysis: Detecting sentiment or mood in conversations or text.
Benefits of On-Device GenAI
On-device GenAI offers several advantages:
- Secure Local Processing: Data remains on the device, enhancing privacy.
- Offline Availability: Functionality persists even without an internet connection.
- Reduced Latency: Faster response times due to on-device processing.
- Cost Efficiency: No additional cloud processing costs.
Limitations and Considerations
On-device GenAI models have limitations compared to their cloud-based counterparts:
- Lower Parameter Size: Typically 2-3 billion parameters, an order of magnitude smaller than cloud models.
- Limited Context Window: Smaller context window can affect accuracy.
- Generalization: Models may require fine-tuning for specific tasks.
Gemini Nano: The Foundational Model
Gemini Nano is the preferred foundational model for building on-device GenAI applications on Android. However, other open models like Gemma are also supported.
Current and Upcoming Applications
Currently, only Google apps leverage Gemini Nano: Summarize in Pixel Recorder, Magic Compose in Google Messages, and Gboard Smart Reply. However, Google is collaborating with developers through an early access program to launch new applications in 2024. Additionally, Google plans to integrate Gemini Nano into TalkBack captions, dynamic suggestions, and spam alerts. A multimodality update is also planned for later this year, starting with Pixel devices.
Advancements in On-Device GenAI
Google acknowledged the progress made in on-device GenAI over the past year, including hardware acceleration for improved performance.
