Systematically explore quantization configurations for a TFLite model using the AI Edge Quantizer API, finding the optimal recipe that balances file size and accuracy. Use this skill whenever the user wants to quantize a model, optimize a recipe, explore quantization tradeoffs, minimize size bounds, or perform selective quantization using AI Edge Quantizer (or AEQ) framework. Applies to any model scale and modality: CNNs, segmentation nets, classifiers, embedding models, audio models, and LLMs.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
La commande reste sur une seule ligne. Faites défiler horizontalement pour la vérifier avant de la copier.
Vous préférez une copie locale ? Téléchargez les fichiers actuellement disponibles dans SkillsMP.
Explorateur de fichiers
18 fichiers
Affichage de SKILL.md
SKILL.md
Instructions source · Aperçu en lecture seule
name
auto_quant
description
Systematically explore quantization configurations for a TFLite model using the AI Edge Quantizer API, finding the optimal recipe that balances file size and accuracy. Use this skill whenever the user wants to quantize a model, optimize a recipe, explore quantization tradeoffs, minimize size bounds, or perform selective quantization using AI Edge Quantizer (or AEQ) framework. Applies to any model scale and modality: CNNs, segmentation nets, classifiers, embedding models, audio models, and LLMs.
Agent Contribution Guide: AI Edge Quantizer Recipe Exploration
Welcome, AI Agent! This skill governs how you autonomously explore and generate
the optimal selective quantization recipe for any given LiteRT FlatBuffer
(.tflite) model using the AI Edge Quantizer (AEQ) framework. Strict adherence
to these protocols ensures code is high-quality, memory safe, and meets
verification protocols.
Your goal is to build an iterative search loop in Python that actively
interrogates the baseline model for numerical degradation, updates the recipe on
the fly to protect highly sensitive tensors, and produces a bounding-box of
Pareto-optimal models for the user.
Skill Index
Leverage these project-specific skill guides to navigate and execute tasks
effectively.
Core guides (read ALL of these before proceeding):
The user states a latency requirement or asks for on-device numbers
Phase 0: Understand the Framework and Classify the Model
Before beginning, explicitly use specific file reading tools to read all core
guides in the Skill Index, plus the ONE task evaluation guide matching the
model's modality, plus any applicable conditional guides.
Then, before writing any plan:
Classify the modality (segmentation, LLM, classification, embedding,
audio, regression) from the model's signatures, I/O tensor shapes, and the
user's description. This selects your metrics
(error_metric_selection.md) and evaluation family.
Classify the scale tier (S/M/L per model_scale_tiers.md) from the
float model file size and signature count. This selects your search
budget, batching strategy, and protection-mechanism priority.
1. Implementation Plan Guidelines
Because complex selective quantization tasks can cause context loss during
execution, your first action MUST be to draft a highly explicit, self-contained
Implementation Plan. Since you will follow this plan closely, you must bake all
necessary context directly into it.
You MUST output this plan as an artifact file (e.g.
implementation_plan.md, using your file-writing tool). In interactive
sessions, HALT after writing the plan and explicitly ask the user for
permission to proceed before you make any code edits or run any loops. If
the user has already provided explicit bounds AND explicitly requested
unattended/autonomous execution (e.g. a CI pipeline), you may proceed without
halting, but the plan artifact is still mandatory. You must strictly follow
this plan once approved.
Your printed plan must contain:
Step 1: Context & Role Declaration
Role Judgment: Declare your role (Contributor, Maintainer, or Reviewer).
Model Classification: State the detected modality, the scale tier, and
the resulting metric selections and search budgets.
Context Summary: Explicitly write out a 2-3 sentence summary of the
specific constraints you read from the skill docs. You MUST explicitly state
that your artifact names will obey the strict rules from file_naming.md
and explicitly confirm you will produce a Pareto visualization per
pareto_curve_plotting.md.
Step 2: Technical Design & File List
Target Files: Explicitly list all files you plan to create or modify.
Please adhere strictly to the output_report.md requirements.
Optimization Constraints: Briefly outline how you will handle target
file size bounds, acceptable error thresholds across MULTIPLE metrics, and
generic regex targeting to prevent data leakage.
Step 3: Experiment Harness Architecture
Metric Selection: Infer from the float model what metrics you should use
for validation following the guidelines of error_metric_selection.md,
including the modality-appropriate stopping metric.
Experiment Blueprint: Explicitly outline your script structure focusing
on proper API sequential updates, memory-safe Quantizer instantiation per
permutation, strictly sequential qt.validate() calls (one validation at a
time), explicit garbage collection (import gc; gc.collect()), and
selecting layer candidates. Most importantly, explicitly confirm you are
implementing the stringent Two-Phase loop mandated by experiment_runner.md
(Phase 1: Freezing protection set, Phase 2: Iterative Greedy INT4 squashing
sorted by robustness, batched per your scale tier). Do not hallucinate
trial-and-error hacks.
Budget Declaration: State the per-tier validation cap and sample count
you will operate under (from model_scale_tiers.md).
Step 4: Verification Plan
Review Hand-off: Document your explicit commitment to run the Two-Pass
Review Loop (see "Agent Verification Protocol" below) on your execution
script before you present the final report. If your harness supports
launching subagents, commit to delegating the review to a reviewer
subagent; otherwise commit to performing the structured self-review pass
against the Reviewer checklist. ONLY start the review after you think
you're done with the quantization exploration process.
Step 5: Post-Execution Walkthrough
Acknowledge that the final response will adhere to the Post-Execution
Walkthrough Guidelines (below).
Phase 1: Setup & Bounding Box
Ask the User for Bounds: If the user did not specify, explicitly ask for
either their Minimum Model Compression Ratio / Model Size OR their Max
Tolerable Error Bound. Our objective is to find either the best quality
model given the minimum compression ratio or the smallest model given the
max tolerable error bound.
Acquire Verification Preprocessing: In order to use qt.validate(), you
need numpy tensor inputs matching the model's signature(s). You may copy
the input processing logic from existing manual human scripts in the
directory (for example processing.py), or construct tokenized prompts
for LLMs per llm_eval.md.
CRITICAL CONSTRAINT (NO LEAKAGE): You MUST NOT copy the human's
manual selective recipe adjustments from existing sources or tutorials
to blindly inject layer names. You must discover which layers to
protect dynamically based on your own validation metric parsing!
This equally applies to the empirical anecdotes and example values
inside these skill guides: treat them as unverified priors that
motivate WHERE to look, never as pre-validated protection lists or
expected metric values. Every protection you commit must be
justified by a measurement from YOUR current run.
Single-source-input warning: If the user provides only ONE test
input, deterministic augmentations of it are acceptable for the
search loop, but the resulting recipe may be mildly overfit to that
input. You MUST state this limitation explicitly in the final report
and recommend re-validating the chosen profiles on a small held-out
set before production use.
Effort & Sizing Logic: The user's size bounds represent a limit, NOT a
target to match exactly. You are authorized to return models that are
smaller than requested! The primary goal is to maximize accuracy (or the
target metric) while guaranteeing the model is small enough. If you can make
it even smaller while retaining acceptable accuracy, do so. Never
artificially inflate the model just to hit a bound. Try multiple iterations
until you have found the best possible quantization recipe, within the
validation budget of your scale tier.
Phase 2: Building the Iteration Harness
Do not try to guess the recipe blindly or write concurrent sweeps. You must
construct a dedicated, repeatable Python evaluation script (e.g.
explore_aeq_model.py) that operates on ONE recipe at a time. Do not run
parallel configurations or concurrent validations.
Load the float baseline .tflite model and run data through preprocessing.
Instantiate quantizer.Quantizer() and load ONE default baseline
starting recipe via the API.
Inject targeted qt.update_quantization_recipe(...) rules sequentially on
the single qt object.
Call qt.quantize() and qt.export_model(). Output every model variation
with a detailed, unique file name following the guidelines in
file_naming.md (respecting the Tier L .tflite retention exception in
model_scale_tiers.md). Save the .json recipe for EVERY variation, no
exceptions.
Measure the stopping metric deployment-faithfully: run each exported
.tflite under the default LiteRT Interpreter and compare outputs
against cached FP32 baseline outputs on a fixed sample set (see the
"Deployment-faithful stopping metric" rule in experiment_runner.md).
Do NOT gate commits on qt.validate() — its backend differs by recipe
(XNNPACK rejects INT4), which makes INT8 and INT4 candidates
incomparable. Reserve qt.validate(...) for per-tensor sensitivity
sweeps, strictly sequentially (one validation at a time, never
concurrent), with use_xnnpack=False on INT4-containing recipes; check
whether your installed release supports multi-metric validation or a
single metric string per call (see the validation API caveats in
experiment_runner.md). CRITICAL: Look at metrics in your script to
decide which tensors to target.
Phase 3: The Empirical Search Loop (Execution)
Two-Phase Iterative Greedy Loop: Read experiment_runner.md for the
exact step-by-step logic.
Phase 0 (Free Pre-Screening): Before any validation, compute the
data-free weight kurtosis ranking (kurtosis_screening.md) to seed
the fragile-candidate list and pre-pin obvious offenders without
spending validation budget.
Phase 1 (Finalize Protection Set): Identify fragile layers using
sensitivity analysis (primary metric, e.g., SNR for vision, KL for
LLMs) and incrementally protect them until overall model quality is
guaranteed via the modality-appropriate stopping metric on the
output tensor. Protection mechanism priority depends on scale tier:
no_quantize skips for Tier S; 8-bit pins and algorithm upgrades
before skips for Tier M/L. Freeze the protection set once the stopping
metric is good.
Phase 2 (Iterative Greedy INT4 Squashing): Rank remaining
unprotected layers by robustness and iteratively squash them
block-by-block (or batch-by-batch with bisection, per your tier) to
4-bit, validating the stopping metric after each step to prevent
compounding noise. Revert failing blocks and continue with the next
candidate; halt only after 3 consecutive reverts or when the
validation budget is exhausted.
CRITICAL: When designing the Phase 2 condition, determine
threshold bounds dynamically relative to the optimal configuration
found in Phase 1. Do not use hardcoded scalar assumptions because
End-to-End metrics can output naturally large arbitrary scalars!
This applies to EVERY script in the search, including post-hoc
audit and refinement scripts.
Phase 3 (Post-Squash Audit & Ablation — MANDATORY, budget
permitting): one cost-aware round of protection additions on the
final mixed model, followed by an ablation of any expensive
protection committed before a cheaper superseding one. Empirically
this phase recovers most of the final quality — see
experiment_runner.md.
Algorithm Axis: If a configuration fails its quality bound, try an
algorithm upgrade (GPTQ, Hadamard rotation, MSE calibration — see
algorithms.md) on the failing layers before retreating to a higher
bit-width. For Tier L transformers, explore the algorithm axis FIRST.
Tier L Direction Inversion (INT4-First): For Tier L models, INT4
weights are the default target, not the aggressive endpoint. Instead
of squashing down from INT8, start from an all-INT4 baseline (best
algorithm, embedding/LM head pinned to INT8) and greedily PROMOTE the
most fragile blocks up to INT8 until the stopping metric passes. See
the "Inverted Search Direction for Tier L" section of
model_scale_tiers.md.
Loop Execution: Build out the script to execute both Phase 1
(protection set finalization via the stopping metric) and Phase 2
(iterative greedy INT4 squashing on unprotected layers) automatically.
Phase 4: Exploring the Pareto Frontier
Provide options. You must output, record, and export to disk EVERY SINGLE
recipe permutation (.json) and validation metrics file generated during your
exploration (including failed baseline attempts, intermediate steps, and the
final optimized models). Export the quantized .tflite for every permutation
on Tier S/M; on Tier L, export .tflite for the baseline and Top 3 profiles
(see model_scale_tiers.md). Do not just keep results in memory. This output
directory is necessary so the user can plot a rich Pareto curve.
Among all explored recipes, you must explicitly highlight and recommend the
Top 3 recipes in your final report, mapped to these key profiles. The
profiles are defined by the USER-FACING trade-off they optimize, not by their
construction (post-audit refinements often blur the constructive
definitions):
Quality Profile (maximum fidelity within the size bound): Typically
remaining unprotected layers strictly at 8-bit, inheriting the finalized
protection list.
Compact Profile (smallest model within the error tolerance):
Typically pushes remaining unprotected layers to 4-bit (with the
best-performing algorithm variant), inheriting the finalized protection
list.
Balanced Profile (best error-per-MB knee of the frontier): Result of
differential sensitivity analysis on remaining unprotected layers
(pushing robust layers to 4-bit and moderately sensitive layers to
8-bit), inheriting the finalized protection list. Select the knee
programmatically — the admissible point preceding the largest marginal
error jump — rather than assuming the mixed recipe is automatically the
knee.
Tag the three chosen artifacts with the profile suffix from
file_naming.md §G so the recommendation is readable from the filename.
For Tier L models under the inverted INT4-first search, the same three
profiles map to: Compact = pure INT4 baseline (+ pinned embedding/LM
head), Balanced = minimal INT8 promotions needed to pass the quality
bound, and Quality = generous promotion of all flagged-fragile blocks to
INT8 (see model_scale_tiers.md).
Agent Verification Protocol (Two-Pass Loop)
To ensure code quality, verifications must follow a strict Two-Pass Recursive
Review:
Pre-flight Self-Check: Ensure the python code parses and runs
successfully before reviewing.
Pass 1 (Comprehensive Audit): If your harness supports subagents, you
MUST literally invoke a reviewer subagent — do not just hallucinate a
review yourself. Pass your python script implementation to the subagent
via the prompt, and instruct it to mathematically evaluate bit depths,
proper API usage (not raw JSON hacking), isolation of quantizers,
multi-metric evaluations, and adherence to the scale-tier budget. Wait for
its message back. If your harness does NOT support subagents, perform a
structured self-review: re-read your script top-to-bottom against every
item in the Reviewer checklist below, and record each pass/fail verdict
explicitly in the report.
Pass 2 (Delta Verification): Provide the updated code for a second
review pass and verify all fixes are applied without regressions.
If an impasse is reached, yield to human.
Role-Based Execution Guidelines
Contributor: Adhere to experiment_runner.md strictly. Prevent Data
Leakage by never hardcoding explicitly named tutorial layers in your
implementations; find and parse the actual broken scopes generically.
Maintainer: Ensure backwards compatibility when updating scripts.
Reviewer: Organize feedback into clear categories: [Quantization Quality], [AEQ API Health], [Experiment Rigor], [Reporting]. Reject
code that lacks multi-metric tracking or multiple Pareto options.
Rule: You MUST reject any script that uses manual trial-and-error
regex arrays to inflate size instead of the proper Two-Phase greedy
loop.
Rule: You MUST reject any script that doesn't explicitly program
the Greedy Phase 2 squashing loop sorting layers by robustness
(single-step or batched-with-bisection per the declared scale tier).
Rule: You MUST reject any script that halts Phase 2 permanently on
the FIRST failed block instead of reverting-and-continuing.
Rule: You MUST reject any script whose stopping metric ignores the
modality (e.g. output MSE used for an LLM instead of output KL).
Rule: You MUST reject any script whose commit/revert gate mixes
validation backends across phases (e.g. XNNPACK-backed numbers for
INT8 steps vs reference-kernel numbers for INT4 steps). The gate
must be the deployment-faithful stopping metric measured on the
exported model under the default runtime.
Rule: You MUST reject any script that ignores the scale-tier
budget (e.g. unbatched per-block validation on a Tier L model).
Rule: You MUST reject the code if the artifact paths don't strictly
generate _validation_metrics.json and follow the
[execution]_[precision]...[ver] schema exactly as defined in
file_naming.md and output_report.md.
Post-Execution Walkthrough Guidelines
After execution, format your final response strictly:
Phase 1: Executive Summary & Bound Achievements: Technical TL;DR.
Provide detailed mathematical reasoning (using the selected validation
metrics) justifying your layer assignments (protected vs int8 vs int4) for
the models on the frontier. State the scale tier and the number of
validations consumed vs the budget.
Phase 2: Architectural Footprint: Paths of generated scripts, quantized
models .tflite files, quantization recipes .json files, Pareto graph,
and quantization_report.md.
Phase 3: Precision & Sanity Guarantee: Confirm validations evaluated
distinct error paths jointly to decide configurations and verify data
leakage checks (generic regex abstraction used over hallucinated names;
calibration/evaluation sample disjointness if static quantization was
used).
Phase 4: Verification Audit Trail: Explicit review pass/fail status
scorecard and delta fixes (subagent-based or structured self-review,
whichever was performed).
