Most AI features on phones today still make a round trip to a data center. Google AI Edge is Google’s attempt to change that: a set of tools for running machine learning and generative AI models directly on phones, tablets, browsers and small embedded devices, with no internet connection required. Its device-testing service, the AI Edge Portal, has been available in a limited private preview, with access restricted to selected developers.
Why on-device matters
Running a model locally has three practical advantages. Responses are faster because nothing travels over a network. Personal data such as photos, voice recordings and messages stays on the device, which simplifies privacy compliance and gives companies a selling point. And apps keep working where connectivity is poor or expensive, which opens markets that cloud-dependent products struggle to serve.
The obstacle has always been consistency. Phones vary enormously in processors, memory and battery capacity, so a model that runs smoothly on one device can be slow, drain the battery or crash on another. Much of Google AI Edge is aimed at that problem.
What is in the stack
Google AI Edge is not a single product but several pieces that fit together:
- LiteRT, formerly TensorFlow Lite, is the runtime that executes models efficiently on a device’s CPU and GPU, with support for dedicated neural processing units. It accepts models built in TensorFlow, PyTorch and JAX, so developers are not locked into one framework.
- MediaPipe Tasks provides low-code APIs for common jobs in vision, audio, text and generative AI. A developer can add object detection, image segmentation or an on-device language model with a few lines of code rather than building a pipeline from scratch.
- Model Explorer is a visualization tool that lets developers inspect a model’s structure, find performance bottlenecks and see how quantization (shrinking a model’s numerical precision to save memory) affects it.
- AI Edge Portal is a Google Cloud service for benchmarking LiteRT models on real physical devices in Google’s labs rather than in simulators. According to Google, it covers more than 100 Android device models across chipsets, memory configurations and Android versions, and returns performance reports, heat maps and comparison charts before an app ships.
Small models for the pocket
The platform is closely tied to Google’s Gemma family of open models, which are small enough to run on consumer hardware. Google has been adding vision and audio capabilities to these models, with the aim of supporting apps that can answer questions about an image or respond to speech entirely on the device.
Google is not alone here. Apple, Qualcomm, Meta and others are all pushing small models and on-device inference, and the direction of travel across the industry is toward hybrid designs that handle routine tasks locally and reserve the cloud for heavier work. Google AI Edge is Google’s bid to make its tooling the default for the local half of that split.
For developers, the practical takeaway is that building an offline AI feature no longer requires deep machine learning expertise, though it still requires careful testing across hardware, which is exactly what the Portal is designed to provide to those who can get access.
