LLM-Runner

Add or revise a model configuration JSON under model_configuration_files/ and ensure it is exercised by CTest (test/CMakeLists.txt) for the relevant backend; optionally onboard a new model end-to-end (downloads/sha256 entries, backend prompt/template quirks, tests, README updates). Use when onboarding a new model, updating supported models, or fixing model-related load/inference/test failures.

Build and run the benchmarking tools in this repository (including llm-bench-cli) across supported backends, and triage benchmark build/runtime issues (shared libs placement, model paths, threads/tokens, JNI off). Use when changing benchmark code, adding metrics, comparing performance, or verifying benchmark binaries for a backend.

Configure, build, and run CTest for this repository using CMake presets (native, x-android-aarch64, x-linux-aarch64), including common cache flags (LLM_FRAMEWORK, BUILD_JNI_LIB, USE_KLEIDIAI, BUILD_BENCHMARK, CPU_ARCH) and failure triage steps. Use when asked to run tests, reproduce build/CI failures, or diagnose CMake/CTest/JNI issues for this project.

Debug failing LLM integration tests caused by model output drift, incorrect context/runtime parameters (contextSize, batchSize, threads), prompt/template mismatches, or backend/framework regressions. Use when tests fail and you need to see the model response, reproduce a single failing CTest, or trace issues into src/cpp/frameworks (llama.cpp, onnxruntime-genai, mnn, executorch).

Run fast “session start / doctor” checks for this repository (toolchain + wiring sanity, framework version report, optional upstream update check), optionally generate a debug bundle, and when needed bump pinned backend framework versions with build+ctest verification. Use at session start or when upgrading llama.cpp/onnxruntime-genai/mnn/executorch pins.

Cross-compile for Android (x-android-aarch64 preset), optionally build tests/benchmarks, and outline adb push/run steps. Use when building for Android targets or diagnosing Android toolchain issues.

Scaffold and integrate a new LLM backend/framework into this repository (new subdir under src/cpp/frameworks, wire LLM_FRAMEWORK routing, configuration options, downloads/models, tests, and docs). Use when adding a new framework, porting a backend, or refactoring backend glue code.

Make safe changes to the public C++ API (src/cpp/interface/) and its implementations (C++ + optional JNI), including updating tests and preserving compatibility where possible. Use when adding new features/methods, changing behavior, or refactoring interface/bridge code in this repository.

Review a change in this repo and produce actionable review comments (correctness, tests, docs, portability, performance) formatted for copy/paste into a code review tool. Use when the user says “review this change”, “code review”, “CR”, “patchset review”, or wants AI reviewer feedback.

Safely add or change model config schema keys (JSON) and update parsing, tests, and docs. Use when editing model_configuration_files schema or LlmConfig parsing without doing broader model onboarding.

Run a fast JNI-focused build/test smoke check (JNI on, minimal test run), and isolate JNI toolchain issues. Use when changing JNI/Java code or validating JNI setup.

Update scripts/py/requirements.json entries (URLs + sha256sum) for models/tools, validate hash changes, and keep downloads deterministic without committing artifacts. Use when adding or refreshing model/tool downloads.