quark-torch-llm-ptq-eval

name	quark-torch-llm-ptq-eval
description	L3 recipe that runs a Torch LLM PTQ end-to-end for AMD Quark — for PyTorch / HuggingFace transformers models (safetensors input): quantize → validate → evaluate. Phase 1 delegates the full PTQ path (model intake → quantization planning → manifest generation → confirmed execution) to the `quark-torch-ptq` workflow; Phase 2 runs mandatory structural validation via `quark-torch-result-validator`; Phase 3 runs opt-in accuracy evaluation via `quark-torch-llm-eval`. Use when the user wants to "quantize and validate", "quantize and evaluate", "run PTQ end to end with accuracy check", "full PTQ pipeline including validation and eval", or "quantize Llama/Qwen/Mistral with FP8/INT4 and measure accuracy". For PTQ only (stop at the quantized output, no validation/eval), use `quark-torch-ptq`. Not for .onnx input models — use `quark-onnx-ptq` / `quark-onnx-autosearch-pro`.
layer	l3-recipes
backend	torch
primary_artifact	eval_report.md
source_knowledge	["examples/torch/language_modeling/llm_ptq/quantize_quark.py","examples/torch/language_modeling/llm_ptq/example_quark_torch_llm_ptq.rst"]

quark-torch-llm-ptq-eval

Purpose

Run the complete PTQ lifecycle for a Torch LLM in one recipe: quantize → validate → evaluate. This recipe composes existing skills rather than re-implementing them — Phase 1 hands off to the quark-torch-ptq workflow (L2) for the PTQ steps, then Phases 2 and 3 chain the atomic validation and evaluation skills (L1). The end result is identical to running the PTQ workflow and then the validator and eval skills by hand; this recipe just drives the whole chain so the user does not have to invoke them separately.

Use this recipe when the user wants quantization plus a correctness and/or accuracy check in a single flow. If they only want the quantized model (no validation, no eval), route to quark-torch-ptq instead — do not run this recipe and then skip its later phases.

Inputs

• Model path (HuggingFace ID or local directory)
• User goal (target precision, hardware, accuracy)
• Output directory for the quantized model
• session_context.json for user goal and constraints
• env_context.json for hardware facts
• workspace_context.json for validated paths
• pytorch_install_result.json and quark_install_result.json to confirm runtime is ready

Outputs

• run_manifest.yaml — executed PTQ command and config (from Phase 1)
• quantized model files in the user's output directory (from Phase 1)
• validation_report.md — structural validation result (from Phase 2, mandatory)
• eval_report.md — accuracy evaluation result (from Phase 3, optional)

CRITICAL RULES

1. Compose, do not re-implement. Phase 1 is the quark-torch-ptq workflow run verbatim, with all its checkpoints. Do not inline or re-derive the intake/plan/manifest/execute steps here.
2. STOP at every checkpoint surfaced by the delegated workflow and at this recipe's eval checkpoint.
3. Validation (Phase 2) is mandatory and automatic. Evaluation (Phase 3) requires explicit user opt-in.
4. NEVER modify Quark's own source code or examples. The Quark repo (quark/, examples/, tools/, docs/, tests/) is read-only. See the same rule in the workflow.