litert-compiled-model-migration

# Skill: LiteRT Compiled Model Migration SKILL ## Description This skill guides an AI agent to rapidly migrate an Android application from legacy TensorFlow Lite (TFLite) to the modern LiteRT CompiledModel API (2.x) in **Open Source GitHub repositories**. It prioritizes a high-speed, 1st-pass **"Like for Like" baseline migration** with automated self-testing, and encourages advanced performance upgrades including **off-main-thread execution with buffer reuse** and **NPU JIT compilation**. ## Read first: what the Kotlin CompiledModel API (`com.google.ai.edge.litert`, 2.3.0; 2.2.0 has the same API) has - `CompiledModel.create(context.assets, assetName, CompiledModel.Options(Accelerator.CPU), env)` and `create(filePath, options, env)`; `Options` takes one or more of `Accelerator.CPU` / `GPU` / `NPU` (e.g. `Options(Accelerator.GPU, Accelerator.CPU)`); `env` is `Environment.create(context)` (optional; enables the compiler cache) or `Environment.create(context, BuiltinNpuAcceleratorProvider(context))` for the NPU. Nothing takes a ByteBuffer or a mapped file. - Buffers come from the model (`createInputBuffers()` / `createOutputBuffers()`, `List<TensorBuffer>`; `writeFloat(FloatArray)` / `writeInt8(ByteArray)` / `writeInt` / `writeLong`, `readFloat()` and friends; create them once). `run(inputs, outputs)` is synchronous on the CPU; on the GPU it can return before the GPU finishes and `read*()` waits; there is no callback API. `CompiledModel`, `TensorBuffer` and `Environment` are `AutoCloseable`: close buffers, then the model, then an `Environment` the app created. - `CompiledModel.Options()` with no accelerator compiles, but `create` then throws `LiteRtException`. `Interpreter`, `GpuDelegate`, `NnApiDelegate` and the Support library's `Model` (`com.google.ai.edge.litert.support.model.Model`, which runs on `InterpreterApi`) do not appear in migrated code; the rest of LiteRT Support does, under `com.google.ai.edge.litert.support` (Step 1), with `org.tensorflow.lite.DataType` as the type its classes take. The `litert` AAR also contains `org.tensorflow.lite.Interpreter`, so leftover Interpreter code compiles (all but `setUseNNAPI`, which 2.x does not have) and the Stage 1 import check is what catches it. `templates/LiteRtModel.kt` compiles against the 2.3.0 and the 2.2.0 artifacts; start from it. --- ## 0. Automatic Discovery & Upfront Planning Before writing code, the agent MUST inspect the project workspace and present the upfront planning interview to align migration parameters: ### A. Automatic Workspace & Toolchain Discovery The agent must automatically inspect the repository structure: 1. **Ecosystem & Build Engine**: * **Gradle Build System**: Detected by `build.gradle`, `build.gradle.kts`, or `settings.gradle`. -> Enable **Gradle & GitHub PR Workflow**. 2. **Language & JNI Toolchain**: * **Native C++ / NDK**: Detected if `CMakeLists.txt`, `Android.mk`, or `*.cpp` files exist. -> Enable **C++ / JNI Migration Rules**. * **Pure Kotlin / Java**: Default to **JVM / Android SDK Migration Rules**. ### B. Upfront User Interview (Questions Asked Prior to Migration) The agent must present the following review options to the user: ``` Before initiating the LiteRT Compiled Model Migration, please confirm your project preferences: 1. Model Workload & Domain: What type of data does this application process? - [A] Vision (Images / Video / Camera Feeds) -> Enables the camera-frame recipe: preprocess into a reused input buffer via writeFloat / writeInt8 (AHardwareBuffer interop is C++-only). - [B] Audio (Speech / Sound Classification) -> Enables the streaming FloatArray recipe (writeFloat per window). - [C] Text / NLP / GenAI -> Enables the token-id recipe (writeInt / writeLong into the model's input buffers). 2. Hardware Acceleration & Conditional INT8 Quantization: Do you want to enable NPU hardware acceleration via JIT on-device compilation? - [A] Yes (Recommended - replaces deprecated NNAPI; needs the vendor libraries of Upgrade 2.C on a supported SoC) [Default] * If the app uses a Float32 model: Would you like to generate an INT8 integer-quantized model via AI Edge Quantizer for peak NPU speed, or run the original Float32 model? -> Option A.1: Convert to INT8 (Generates model_int8.tflite with int8 weights and activations; needs a few representative inputs for calibration) [Default] -> Option A.2: Keep Float32 (Runs baseline float model directly on NPU) - [B] No (GPU and CPU acceleration only) 3. Encouraged Performance Upgrades: Should the agent upgrade the calling code to use LiteRT's advanced features? - [A] Yes (Move inference off the main thread and reuse buffers) [Default] - [B] No (Keep strict 1-to-1 synchronous baseline execution) 4. Automated Pull Request Provisioning: Should the agent automatically stage, commit, and create a GitHub PR when self-testing passes? - [A] Yes [Default] (Attaches verification test logs and before/after summary diff) - [B] No (Keep changes local in current working branch) ``` With no user to answer (CI, batch runs): take Q1 from the code, Q2 [A] only when the vendor libraries of Upgrade 2.C are already in the repo (otherwise [B]), Float32 unless calibration inputs exist, and the defaults of Q3 and Q4 with the PR opened as a draft; record the choices in the PR description. Today the runtime is the standalone `com.google.ai.edge.litert:litert` AAR: no LiteRT artifact for Google Play services is on Google Maven as of October 2026 (only the `play-services-tflite-*` Interpreter artifacts exist). > [!IMPORTANT] > **LiteRT Support (`litert-support` 2.3.0)**: an app on the TensorFlow Lite Support Library keeps it. The library now ships as LiteRT Support, with the same classes, methods and behavior; the migration is two steps of Step 1, swap the Gradle coordinates and rename the package `org.tensorflow.lite.support` to `com.google.ai.edge.litert.support` (its guide: https://github.com/google-ai-edge/LiteRT/blob/main/litert/support_lib/MIGRATION_GUIDE.md). `ImageProcessor`, `ResizeOp`, `NormalizeOp`, `TensorImage`, `TensorLabel`, `FileUtil` and `MetadataExtractor` stay as they are. The one class replaced is its `Model`, which runs on the Interpreter API: it becomes `CompiledModel` like `Interpreter` (Step 4), with `templates/SupportBridge.kt` between the Support tensors and the model's buffers. An app that never used the Support library does not add it; `Bitmap.toModelInput` in `templates/LiteRtModel.kt` is its preprocessing. --- ## 1. Phase 1: "Like for Like" Baseline Migration (1st Pass Success) Phase 1 prioritizes functional equivalence, fast compilation, and immediate 1st pass self-test success. ### Step 0: Capture the Golden Output Before Step 1 changes any dependency, with the legacy runtime and preprocessing still in the project (no device attached: skip Step 0, stop after Stage 1, and open the PR as a draft that says so): * Define one fixed input: for a vision app a deterministic bitmap (`fixedBitmap` in `templates/GoldenCaptureTest.kt`, or a small image file in `app/src/androidTest/assets/`), otherwise a generated array; never a camera frame. A bitmap of the model's input size with rotation 0 leaves the resize and the rotation out of the gate: use another size (and aspect ratio), and the rotation the camera path passes, in the capture and in Stage 2 alike. * Declare the `androidx.test` dependencies and `testInstrumentationRunner` (Stage 2 needs them too), copy `templates/GoldenCaptureTest.kt` into `app/src/androidTest/java/<package>/`, make it call the app's existing inference entry point (the class the UI calls, with its own preprocessing) on the fixed input on the CPU (no GPU or NNAPI delegate, XNNPACK at its default); when that entry point returns only a thresholded or sorted list, first give the class a function returning the raw output and one returning the preprocessed input (`preprocess(bitmap)`: Stage 2 calls it too), still on the legacy runtime. Then run the four commands in its header: they install both APKs, run the test on the device, and pull `golden.txt` into `app/src/androidTest/assets/golden/output.txt`. A raw tensor fed straight to the model is not a golden output: it leaves the preprocessing out of the gate. * Delete the capture test. Stage 2 compares against the file, so the test APK never needs `org.tensorflow:tensorflow-lite`. ### Step 1: Clean & Modernize Dependencies Inspect `libs.versions.toml` and `build.gradle.kts`: * **Remove Legacy & Deprecated**: * `org.tensorflow:tensorflow-lite` / `tensorflow-lite-gpu` / `tensorflow-lite-select-tf-ops` *(Legacy Flex Delegate — see Deprecated API Remediation below)*, and every other `org.tensorflow:tensorflow-lite*` line, including ones other libraries pull in (`./gradlew :app:dependencies` lists them): the LiteRT artifacts carry the `org.tensorflow.lite` runtime classes and the model schema under their original names, and mixing old and new artifacts fails the build with duplicate classes. `com.google.ai.edge.litert:litert-gpu` **1.x** goes too *(the renamed TFLite GPU delegate, last 1.4.2; `litert-gpu:2.3.0` is the GPU accelerator of the CompiledModel runtime and is added below)*, and `com.google.ai.edge.litert:litert` **1.x** (the renamed Interpreter runtime, e.g. `samples/litert_interpreter/*` on 1.0.0) becomes `litert:2.3.0` below. * **Swap the Support library coordinates** (apps on `org.tensorflow.lite.support.*`): `org.tensorflow:tensorflow-lite-support` or `litert-support` 1.x becomes `implementation("com.google.ai.edge.litert:litert-support:2.3.0")` (the Support API plus the LiteRT runtime: it depends on `litert:2.3.0`); `tensorflow-lite-support-api` becomes `litert-support-api:2.3.0` (the API alone, the runtime being the `litert` line below); `tensorflow-lite-metadata` or `litert-metadata` 1.x becomes `litert-metadata:2.3.0` (`MetadataExtractor`). Then rename the package in every source file and in R8/ProGuard rules (`-keep class org.tensorflow.lite.support.**`): `grep -rl 'org\.tensorflow\.lite\.support' --include='*.java' --include='*.kt' app/src/ | xargs sed -i 's/org\.tensorflow\.lite\.support/com.google.ai.edge.litert.support/g'` (GNU sed; on macOS, `sed -i ''`). `org.tensorflow.lite.DataType`, the type the Support classes take, keeps its name. * **Add Modern LiteRT**: * *Standalone*: `implementation("com.google.ai.edge.litert:litert:2.3.0")` and `implementation("com.google.ai.edge.litert:litert-gpu:2.3.0")` in Kotlin DSL (`implementation 'com.google.ai.edge.litert:litert:2.3.0'` and `'…:litert-gpu:2.3.0'` in Groovy). `litert` pulls in `litert-api` and sets minSdk 24. `litert-support:2.3.0` depends on `litert:2.3.0` and raises a `litert:2.2.0` pin to 2.3.0 without a warning (Gradle resolves `litert:2.2.0 -> 2.3.0`), so pin 2.3.0 here. Since 2.3.0 the GPU accelerator (`libLiteRtClGlAccelerator.so`) ships in `litert-gpu`; in 2.2.0 it was inside `litert`. Without `litert-gpu`, `CompiledModel.Options(Accelerator.GPU, Accelerator.CPU)` creates and runs on the CPU with the log line `GPU accelerator could not be loaded and registered`, and `Options(Accelerator.GPU)` alone throws. * **Toolchain the 2.3.0 artifacts need**: `litert-api:2.3.0` depends on `androidx.lifecycle:lifecycle-runtime:2.10.0` (its POM), and its classes carry Kotlin 2.4 metadata (2.2.0: Kotlin 2.3). Set these before the first build on a project on AGP 8.3 / Kotlin 1.9 (e.g. `samples/litert_interpreter/*`), or each becomes a build error in turn: `compileSdk` 35 and AGP 8.6.0 (`checkDebugAarMetadata`), Gradle 8.7 (AGP 8.6.0), the Kotlin plugin 2.3 or newer (from 1.9 this is a plugin upgrade; `freeCompilerArgs += "-Xskip-metadata-version-check"` only lets a project already on 2.2 stay there, as `samples/litert/image_segmentation/kotlin_cpu_gpu` does; 2.2 otherwise stops at `The binary version of its metadata is 2.4.0, expected version is 2.2.0`, and 2.2 is enough for the 2.2.0 artifacts), and with Kotlin 2.x a Compose app needs the `org.jetbrains.kotlin.plugin.compose` plugin in place of `composeOptions`. On AGP 9 (built-in Kotlin) the Kotlin version is pinned with `id("org.jetbrains.kotlin.android") version "2.3.21" apply false` in the root plugins block. * **Labels and metadata**: `FileUtil.loadLabels` and `MetadataExtractor` work as before on the 2.3.0 artifacts. An app without the Support library reads the label file from assets: a `.tflite` with metadata is a zip, so extract it once (`unzip -o model.tflite labels.txt -d app/src/main/assets/`; `unzip -l` lists the name). * **IDE Portability**: Remove hardcoded `org.gradle.java.home` from `gradle.properties` and exclude `local.properties`. * **Kotlin Compiler DSL**: Use top-level `kotlin { compilerOptions { ... } }` outside `android { ... }`: `kotlinOptions { jvmTarget = "1.8" }` becomes `kotlin { compilerOptions { jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_1_8) } }`. ### Step 2: Deprecated API & Delegate Remediation The agent must audit and replace all deprecated delegate APIs: 1. **NNAPI Delegate (`NnApiDelegate`, `NnApiDelegate.Options`, `setUseNNAPI(true)`)**: * **Status**: NNAPI is deprecated since Android 15, and the LiteRT 2.x Java/Kotlin API has no NNAPI entry point (`Interpreter.Options` has no `setUseNNAPI`). * **Remediation**: Remove `org.tensorflow.lite.nnapi.NnApiDelegate` imports. Replace with `CompiledModel.Options(Accelerator.NPU)` and `Environment.create(context, BuiltinNpuAcceleratorProvider(context))` when `BuiltinNpuAcceleratorProvider(context).isDeviceSupported()` is true (NPU-only options compile as NPU + CPU, so unsupported ops fall back to the CPU). In 2.3.0 `isDeviceSupported()` is true on every Qualcomm, MediaTek or Samsung SoC from Android 12 and on Google Tensor from Android 16 (2.2.0 used a SoC list), and without the vendor libraries `create` still returns: the runtime logs `Failed to apply compiler plugins: No compiler plugin found` (2.2.0 adds `Failed to initialize Dispatch API`) and compiles for the CPU, so a cascade's reported accelerator reads NPU while the CPU runs. Implement an explicit `NPU -> GPU -> CPU` fallback cascade (try each `CompiledModel.create`, catch `LiteRtException`) to handle non-NPU hardware smoothly. Under Q2 [B], an NNAPI choice in the app's UI becomes the GPU option (`Options(Accelerator.GPU, Accelerator.CPU)`). 2. **Flex Delegate (`FlexDelegate`, `org.tensorflow.lite.flex`, `select-tf-ops`)**: * **Status**: Interpreter-only: the Flex delegate ships in the legacy `tensorflow-lite-select-tf-ops` artifact (68 MB for arm64 in 2.16.1) and in no LiteRT 2.x artifact, so a model with Select TF ops needs the re-export below. * **Remediation**: * Remove `org.tensorflow:tensorflow-lite-select-tf-ops` from `build.gradle.kts`. * Find Flex ops by FlatBuffer inspection (custom ops whose `custom_code` starts with `Flex`); `check_gpu_compatibility` in `utilities/litert_gpu_toolkit` (this repository) reports such a model as a compile failure, not op by op. * Replace Flex ops by re-exporting the model without them (LiteRT-Torch for PyTorch sources; the TensorFlow converter without `SELECT_TF_OPS`), or, if custom C++ math is required, implement a `litert::CustomOpKernel` (`litert/cc/litert_custom_op_kernel.h`, shipped in `litert_cc_sdk.zip`; guide: https://developers.google.com/edge/litert/next/custom_op_dispatcher). ### Step 3: Native Build Toolchain (`CMakeLists.txt` / NDK) For native C++ modules, use the C++ SDK the LiteRT docs describe (https://developers.google.com/edge/litert/next/android_cpp_sdk): download `litert_cc_sdk.zip` from the LiteRT GitHub release that matches the AAR version (https://github.com/google-ai-edge/LiteRT/releases/download/v2.2.0/litert_cc_sdk.zip; 2.3.0 has no GitHub release as of 2026-10-07, so a native module stays on the 2.2.0 AAR and zip until the matching zip exists, and an app with both a native module and the Support library waits for that zip, since `litert-support:2.3.0` raises the runtime to 2.3.0), copy `libLiteRt.so` from the AAR's `jni/arm64-v8a/` into its `litert_cc_sdk/` directory (`LITERT_CC_SDK` below), and link the `litert_cc_api` target. The AAR ships no headers or prefab package, and `liblitert_jni.so` is the Kotlin binding, not a C++ target: ```cmake # Replace legacy tensorflowlite_jni with the LiteRT C++ SDK add_subdirectory("${LITERT_CC_SDK}" "${LITERT_CC_SDK}/build") include_directories("${LITERT_CC_SDK}") target_link_libraries(your_native_lib litert_cc_api android log ) ``` ### Step 4: API & Lifecycle Refactoring (Rewrite Initialization & Dynamic Signatures) > [!IMPORTANT]