# LiteRT
<p align="center">
<img src="https://raw.githubusercontent.com/google-ai-edge/LiteRT/main/docs/sources/litert_logo.png" alt="LiteRT Logo"/>
</p>
Google's on-device runtime for high-performance ML & GenAI deployment on edge platforms.
๐ [Get Started](#-installation) | ๐ค [Contributing](#-contributing--getting-help) | ๐ [License](#-license) | ๐ก [Security Policy](SECURITY.md) | ๐ [Documentation](https://ai.google.dev/edge/litert)
---
## ๐ Build Status
| Nightly Builds | Continuous Builds | Other Builds |
| :--- | :--- | :--- |
| [](https://github.com/google-ai-edge/LiteRT/actions/workflows/linux_nightly_wheel.yml)<br>[](https://github.com/google-ai-edge/LiteRT/actions/workflows/macos_nightly_wheel.yml)<br>[](https://github.com/google-ai-edge/LiteRT/actions/workflows/windows_nightly_wheel.yml) | [](https://github.com/google-ai-edge/LiteRT/actions/workflows/macos-arm64.yml)<br>[](https://github.com/google-ai-edge/LiteRT/actions/workflows/linux_x86_64.yml)<br>[](https://github.com/google-ai-edge/LiteRT/actions/workflows/windows_x86_64.yml) | [](https://github.com/google-ai-edge/LiteRT/actions/workflows/cmake_android_linux_x86_64.yml) |
---
### Release Cadence
LiteRT provides nightly builds and targets stable releases on a 6-8 week cadence.
## ๐ LiteRT
LiteRT continues the legacy of TensorFlow Lite as the trusted, high-performance runtime for on-device AI. Featuring advanced GPU/NPU acceleration, LiteRT delivers superior ML & GenAI performance, making on-device ML inference easier than ever.
### ๐ What's New
* **๐ง Superior GenAI Inference:** Deploy LLMs directly on-device using [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM).
* **๐ High-Performance Web Inference:** Run secure client-side ML in the browser via WebGPU and WASM with [LiteRT.js](https://ai.google.dev/edge/litert/web).
* **๐งฎ C++ Graph Authoring:** Manipulate high-performance tensors using a lightweight, tensor-centric C++ library via the [Tensor API](https://github.com/google-ai-edge/LiteRT/tree/main/tensor).
* **๐ค Accelerated Agentic Coding:** Streamline AI coding agent workflows using the [LiteRT CLI](https://github.com/google-ai-edge/LiteRT-CLI#-use-in-coding-agent) command-line toolkit.
Quick setup for LiteRT-CLI below
```bash
# 1. Create a virtual environment with Python 3.13.
# TIP: If you hit dependency resolution errors, try:
# export UV_INDEX_URL=https://pypi.org/simple
uv venv --clear --python=3.13 --seed
source .venv/bin/activate
# 2. Install the package into the active virtual environment
uv pip install litert-cli-nightly
# 3. Run help command
litert --help
```
---
### ๐ Key Features of LiteRT V2
* **โ๏ธ Compiled Model API:** **Streamlined Development.** Features automated accelerator selection (no explicit delegates needed), true asynchronous execution, easy NPU distribution, and highly efficient I/O buffer handling
* **๐ Unified NPU Acceleration:** **Broad Silicon Support.** Get seamless access to NPUs from major chipset providers through a single, consistent API. [See LiteRT NPU](https://ai.google.dev/edge/litert/next/npu).
* **๐๏ธ Faster GPU Acceleration via ML Drift:** **Supporting Gen-AI Inference.** Leverage state-of-the-art GPU acceleration with new buffer interoperability that minimizes latency across various GPU buffer types.
---
## โ๏ธ LiteRT Runtime and Tools
From model to on-device deployment for PyTorch, TensorFlow, and JAX models:
```mermaid
graph LR
A[PyTorch Model] --> B[LiteRT Torch
LiteRT Torch Generative/HF export]
a[HF transformer
safe tensors] --> B
B -->|.tflite| F(AI-Edge Quantizer) --> |Optimized .tflite| I
B -->|.litertlm|F --> |Optimized .litertlm| H{Litert-LM
Python, C++, Kotlin, swift, JS} --> I{LiteRT Runtime
C++, Kotlin, JS}
I --> J[CPU - XNNPack <br> GPU - ML Drift <br> Supported TPU/NPU]
```
---
## ๐บ Choose Your Adventure
Every developer's path is different. Here are a few common journeys to help you get started based on your goals:
| If you want to... | Use this path... |
| :--- | :--- |
| **๐Upgrade from TensorFlow Lite/ LiteRT V1.x** | Use [LiteRT Migration Guide](https://ai.google.dev/edge/litert/migration) to upgrade to LiteRT V2.x |
| **๐ฑ Run a pretrained model (like image segmentation) on mobile** | Follow step-by-step instructions via Android Studio to create a [Real-time segmentation](https://developers.google.com/codelabs/litert-image-segmentation-android#0) App for CPU/GPU/NPU inference. [Source code](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation). |
| **๐ Convert PyTorch Models** | Use [LiteRT Torch Converter](https://github.com/google-ai-edge/litert-torch) for `.tflite` (Classic) or [Generative Torch API](https://github.com/google-ai-edge/litert-torch/tree/main/litert_torch/generative) for `.litertlm` (LLMs). |
| **๐ง Deploy Generative AI** | Optimize and run quantized LLMs or diffusion models on-device using [LiteRT LM](https://github.com/google-ai-edge/LiteRT-LM). |
| **โกMaximize Performance** | Explore the [LiteRT API](https://ai.google.dev/edge/api/litert/c) & [LiteRT NPU Acceleration](https://ai.google.dev/edge/litert/next/npu) to leverage underlying hardware acceleration. |
| **๐Run in the Browser** | Deploy secure, client-side web apps leveraging WebGPU and WASM via [LiteRT.js](https://ai.google.dev/edge/litert/web). |
| **๐งฎControl Memory & Graph Execution** | Tensor-centric C++ library for high-performance tensor manipulation on mobile devices.[LiteRT Tensor API](https://github.com/google-ai-edge/LiteRT/tree/main/tensor). |
---
## ๐ป Platforms Supported
LiteRT is designed for cross-platform deployment on a wide range of hardware.
| Platform | CPU | GPU APIs | NPU / Hardware Accelerators |
| :--- | :---: | :--- | :--- |
| **๐ค Android** | โ
| โ
OpenCL <br>โ
OpenGL |โ
Broadcom, โ
Google Tensor,<br> โ
Intel โ
MediaTek, โ
[Qualcomm](https://github.com/google-ai-edge/LiteRT/blob/main/litert/vendors/qualcomm/README.md),<br> โ
S.LSI |
| **๐ iOS** | โ
| โ
Metal | ANE\* |
| **๐ง Linux** | โ
| โ
WebGPU | โ
Broadcom,<br> โ
Intel|
| **๐ macOS** | โ
| โ
WebGPU <br> โ
Metal | ANE\* |
| **๐ป Windows** | โ
| โ
WebGPU | โ
Intel |
| **๐ Web** | โ
| โ
WebGPU | WebNN\* |
| **๐งฉ IoT** | โ
| โ
WebGPU | Raspberry Pi\* |
*Indicates coming soon
---
## ๐ New Models
Recently added supported models to Hugging Face LiteRT Community.
| Model Family | Size / Variant | Modality | Hugging Face Hub |
| :--- | :--- | :--- | :--- |
| **Gemma 4** | Various | Multi-modal | [Explore Models](https://huggingface.co/collections/litert-community/gemma-family) |
| **ASR Models** | Various| Audio | [Explore Models](https://huggingface.co/collections/litert-community/asr) |
| **Image Classification Models** | Various| Vision | [Explore Models](https://huggingface.co/collections/litert-community/image-classification-models) |
Find more models at the [Hugging Face LiteRT Community Page](https://huggingface.co/litert-community)
---
## ๐ Sample Apps & Colabs
Find official sample applications and code examples for LiteRT's Compiled Model API here:
* **[LiteRT Samples](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert):** A collection of sample applications.
* **[ASR Sample App](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/speech_recognition):** Automatic Speech Recognition LiteRT Sample App
* **[Image Segmentation](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation):** C++ and Kotlin Image Segmentation app demonstrating AOT and on-device compilation examples
* **[Agent Skills](https://github.com/google-ai-edge/litert-samples/tree/main/skills):** `SKILL.md` playbooks for AI coding agents covering conversion, quantization, on-device verification, benchmarking, app scaffolding, and TFLite migration.
---
## ๐ Installation
For a comprehensive guide on integrating LiteRT into your specific platform, see the [LiteRT Integration Overview](https://ai.google.dev/edge/litert/overview).
### ๐จ Building from Source
You can build LiteRT artifacts for Linux and Android (via cross-compilation) using Docker:
1. Start a Docker daemon.
2. Run `build_with_docker.sh` inside the `docker_build/` directory.
> **Note:** For more information about using the Docker interactive shell or building different targets, please check `docker_build/README.md`.
For detailed instructions on building runtime libraries with the Docker container, refer to the [CMake Build Instructions](https://github.com/google-ai-edge/LiteRT/blob/main/docs/instructions/CMAKE_BUILD_INSTRUCTIONS.md) and [Bazel Build Instructions](https://github.com/google-ai-edge/LiteRT/blob/main/docs/instructions/BUILD_INSTRUCTIONS.md).
## ๐ Roadmap
Our commitment is to make LiteRT the best runtime for *any* on-device ML deployment. Our core product strategies include:
| โก Hardware Acceleration | ๐ง Generative AI Optimizations |
| :--- | :--- |
| Broadening NPU support and improving performance across all major hardware accelerators. | Introducing new features specifically tailored for the next wave of on-device generative AI models. |
| **๐ Developer Tools** | **๐ Platform Support** |
| Building better utilities for debugging, profiling, and optimizing models. | Enhancing core platform support and exploring emerging ecosystems. |
---
## ๐ฐ Latest from the LiteRT Team & Partners
| Date | Blog Title |
| :--- | :--- |
| Aug 2026 | [Mastering Edge AI on Raspberry Pi](https://developers.googleblog.com/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma/) |
| Jul 2026 | [LiteRT.js, Google's high performance Web AI Inference](https://developers.googleblog.com/litertjs-googles-high-performance-web-ai-inference/) |
| May 2026 | [Google Tensor SDK Beta with LiteRT](https://developers.googleblog.com/google-tensor-sdk-beta-with-litert/) |
| May 2026 | [LiteRT Support for Intel NPUs via OpenVINOโข](https://www.intel.com/content/www/us/en/developer/articles/community/litert-unlocks-core-ultra-npu-performance-for-aipc.html) |
| May 2026 | [Arm and Google AI Edge optimization](https://developers.googleblog.com/accelerating-on-device-ai-a-look-at-arm-and-google-ai-edge-optimization/) |
| April 2026 | [Building real-world on-device AI with LiteRT and NPU](https://developers.googleblog.com/building-real-world-on-device-ai-with-litert-and-npu/) |
[๐ Read more LiteRT posts on the Google Developers Blog](https://developers.googleblog.com/search/?query=LiteRT)
---
## ๐ Related Products
LiteRT is part of a larger [Google AI-Edge](https://ai.google.dev/edge) ecosystem of tools for on-device machine learning:
* **[LiteRT Torch Converter](https://github.com/google-ai-edge/litert-torch):** A tool to convert PyTorch models into the `.tflite` format.
* **[LiteRT Torch Generative API](https://github.com/google-ai-edge/litert-torch/tree/main/litert_torch/generative):** A library to reauthor LLMs for efficient conversion and inference.
* **[AI-Edge Quantizer](https://github.com/google-ai-edge/ai-edge-quantizer):** A quantizer for advanced developers to quantize converted LiteRT models.
* **[LiteRT-LM](https://github.com/google-ai-edge/litert-lm):** A library to efficiently run LLMs across edge platforms.
* **[LiteRT.js](https://ai.google.dev/edge/litert/web):** WebAI runtime, targeting production Web applications.
* **[MediaPipe](https://github.com/google-ai-edge/mediapipe):** A framework for building cross-platform, customizable ML solutions for live streaming media.
* **[XNNPACK](https://github.com/google/XNNPACK):** A highly optimized library of neural network inference operators providing high-performance CPU acceleration.
---
### ๐ AI Coding Directives (Strict Constraints)
When generating application code that uses LiteRT (this does not apply to LiteRT's own runtime internals):
* **MUST USE:** The Compiled Model API for all new inference code in Kotlin, C++, Python, and JavaScript.
* **DO NOT USE:** The TensorFlow Lite `Interpreter` API (for example `tflite::Interpreter`, `org.tensorflow.lite.Interpreter`, `tf.lite.Interpreter`) or manual delegate creation. TensorFlow Lite packages and `tensorflow/lite/` are in maintenance mode and only receive critical security and stability updates.
* **CODE EXAMPLES:** Take code from the [LiteRT documentation](https://ai.google.dev/edge/litert/overview), not from older TensorFlow Lite examples. To move existing code over, follow the [LiteRT Migration Guide](https://ai.google.dev/edge/litert/migration).
* **AGENT SKILLS:** Use the [LiteRT agent skills](https://github.com/google-ai-edge/litert-samples/tree/main/skills) for step-by-step workflows (convert, quantize, verify, benchmark, build apps, and migrate from TensorFlow Lite).
---
## ๐ Contributing & Getting Help
* **Contributing:** We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
* **Contributing Models** Contribute your .tflite or .litertlm models via the LiteRT Hugging Face page [HF LiteRT Community](https://huggingface.co/litert-community)
* **Bug Reports & Features:** File an issue on our [GitHub Issues](https://github.com/google-ai-edge/LiteRT/issues) page.
* **Community Support:** Join the conversation on [GitHub Discussions](https://github.com/google-ai-edge/LiteRT/discussions).
## โค๏ธ Code of Conduct
This project is dedicated to fostering an open and welcoming environment. Please read our [Code of Conduct](CODE_OF_CONDUCT.md) to understand the standards of behavior we expect from all participants.
## ๐ License
LiteRT is licensed under the [Apache-2.0 License](LICENSE).