// Use this skill when benchmarking Skippy exact-prefix cache across model families, comparing Skippy against llama-server, producing README benchmark tables, updating crates/skippy-cache/README.md evidence, or diagnosing cache benchmark gaps by family or Hugging Face use case.
Use this skill when running benchmark orchestration, local single-stage or split benchmarks, benchmark report flow, or performance-oriented skippy runtime checks.
Use this skill when validating skippy staged execution against full-model execution, adding model families, changing split boundaries, testing activation wire dtypes, or diagnosing mismatch behavior.
Use this skill when inspecting GGUF models, planning layer ranges, generating or validating skippy package artifacts, fake packages for direct GGUFs, materialized stage cache behavior, or GGUF writer integration.
Use this skill when running, configuring, debugging, or embedding skippy-server, binary stage transport, OpenAI frontend integration, activation wire dtype settings, stage configs, lifecycle status, or nonblocking telemetry.
Use this skill when certifying mesh-llm KV/cache stability under repeated OpenAI tool-call loops, same-prefix cache reuse, suffix-prefill limits, or native Skippy slot/decode/eviction failures.
Use when changing mesh-llm automation or CLI flows that discover Hugging Face GGUF models, plan CPU Hugging Face Jobs for layer-package splitting, estimate max cost, or publish skippy layer packages/catalog entries.
| name | skippy-cache-family-bench |
| description | Use this skill when benchmarking Skippy exact-prefix cache across model families, comparing Skippy against llama-server, producing README benchmark tables, updating crates/skippy-cache/README.md evidence, or diagnosing cache benchmark gaps by family or Hugging Face use case. |
| metadata | {"short-description":"Benchmark Skippy cache by family"} |
Use this skill for reproducible Skippy cache benchmark evidence. The goal is to
compare production cache payloads only: ResidentKv for dense families and
KvRecurrent for recurrent/hybrid families. Do not report FullState as a
production cache mode.
Run the reproducible wrapper. It builds by default, then runs full-GGUF baselines, Hugging Face use-case prompts, and the README report renderer:
evals/skippy-cache-family-bench.sh /tmp/skippy-cache-family-bench
For fast iteration after a build, skip the build step:
SKIPPY_CACHE_SKIP_BUILD=1 evals/skippy-cache-family-bench.sh /tmp/skippy-cache-family-bench
If running the pieces manually, keep these matched settings for full-GGUF family baselines:
LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-cpu \
python3 evals/skippy-cache-production-bench.py \
--output-dir /tmp/skippy-cache-family-bench/full-gguf \
--runtime-lane-count 1 \
--llama-parallel 1 \
--prefix-tokens 128
Run the use-case benchmark matrix against the same family set:
LLAMA_STAGE_BUILD_DIR=.deps/llama-build/build-stage-abi-cpu \
python3 evals/skippy-cache-production-bench.py \
--output-dir /tmp/skippy-cache-family-bench/use-cases \
--runtime-lane-count 1 \
--llama-parallel 1 \
--prefix-tokens 128 \
--use-case all
Render README-ready tables from the combined JSON outputs:
python3 evals/skippy-cache-family-report.py \
--input /tmp/skippy-cache-family-bench/full-gguf/production-cache-bench.json \
--input /tmp/skippy-cache-family-bench/use-cases/production-cache-bench.json \
--output /tmp/skippy-cache-family-bench/readme-tables.md
crates/skippy-cache/TODO.md./tmp/... or another explicit run directory; do
not paste ad hoc numbers without the backing production-cache-bench.json.