ワンクリックでManusで任意のスキルを実行

parity-check

スター6

フォーク0

更新日2026年7月24日 11:12

Cross-check SharpInference numerics against llama.cpp and run accuracy gates. Use when output quality or numerics are in question — new architecture bring-up, kernel or quantization changes, KV-compression work, "model produces garbage" reports.

インストール

Codex または Claude でインストールこの Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。

Manusで実行

ソース

pekkah

pekkah/SharpInference

GitHub リポジトリを開く Creator のリポジトリを見る

ダウンロード

Manusで実行

Numeric parity & accuracy gating

Escalate through these levels; stop at the first one that answers the question.

1. Unit-level parity (no model files needed)

tests/SharpInference.Tests.ForwardPass holds quantization and kernel parity suites (CPU vs CUDA vs Vulkan, Q4_K/Q6_K/Q8 dequant round-trips). Run the narrowest filter first — this catches most kernel regressions without any GGUF.

2. Greedy token cross-check vs llama.cpp

pwsh scripts/xcheck-llamacpp.ps1 -Model <gguf> -Prompt "..." -NTokens 30 -Tag <tag>

Requires tools/llama.cpp (fetch via pwsh scripts/setup-llamacpp.ps1) and a Release build of the CLI (dotnet build src/SharpInference.Cli -c Release).
Runs both engines at --temp 0 and prints SharpInference's greedy token IDs; outputs land in tools/xcheck_<tag>_*.txt for diffing.
First divergent token position is the lead — trace that position, not the last.

3. Reference logits comparison

pwsh scripts/generate-reference-logits.ps1 plus scripts/extract_reference.py / scripts/compare_tokens.py produce and compare per-position logits when token-level divergence needs a numeric root cause.

4. Perplexity gate (statistical accuracy)

dotnet run --project src/SharpInference.Cli -c Release -- \
  perplexity -m <model.gguf> -f <corpus.txt> -c 2048 [--tq] [--tq-mode kvarn|lloydmax]

The accuracy gate for KV compression and lossy changes (issue #180). Compare against a baseline run without the change; corpus helpers live in scripts/kvarn-gate/ (get-wikitext2.ps1, math-eval.ps1).

Notes

Model files go in models/; fetch known presets with pwsh scripts/download-model.ps1 -Model <preset> (see script header for the list).
Levels 2–4 need real model files and are slow — prefer delegating to a background agent and continuing other work.

このリポジトリの他の Skills

同じリポジトリ

implement

pekkah/SharpInference

Drive the architect→workhorse flow — turn an approved plan into delegated Sonnet implementation with test and review gates, keeping the main (Opus/Fable) thread as coordinator.

2026-07-246

issue

pekkah/SharpInference

Work a GitHub issue end-to-end — fetch it, verify the claim, turn it into an architect spec, and drive the delegated implementation flow with correct issue references in branches and commits.

2026-07-246

new-arch

pekkah/SharpInference

Checklist for adding support for a new GGUF model architecture (or a variant of an existing one) to SharpInference.

2026-07-246

run-models

pekkah/SharpInference

CLI run recipes for specific models and modes — VibeThinker, Ornith-1.0, Gemma 4 vision, image generation + upscaling, perplexity gating, whole-turn JSON-schema structured output, DSpark speculative decoding, download-model presets. Use when actually running inference for one of these.

2026-07-246

vulkan-shaders

pekkah/SharpInference

Required workflow when adding, editing, or removing Vulkan GLSL shader consts — the committed precompiled SPIR-V table must be regenerated or tests fail on drift.

2026-07-246

Numeric parity & accuracy gating

Escalate through these levels; stop at the first one that answers the question.

1. Unit-level parity (no model files needed)

2. Greedy token cross-check vs llama.cpp

pwsh scripts/xcheck-llamacpp.ps1 -Model <gguf> -Prompt "..." -NTokens 30 -Tag <tag>

Requires tools/llama.cpp (fetch via pwsh scripts/setup-llamacpp.ps1) and a Release build of the CLI (dotnet build src/SharpInference.Cli -c Release).

Runs both engines at --temp 0 and prints SharpInference's greedy token IDs; outputs land in tools/xcheck_<tag>_*.txt for diffing.

First divergent token position is the lead — trace that position, not the last.

3. Reference logits comparison

4. Perplexity gate (statistical accuracy)

dotnet run --project src/SharpInference.Cli -c Release -- \ perplexity -m <model.gguf> -f <corpus.txt> -c 2048 [--tq] [--tq-mode kvarn|lloydmax]

Notes

Model files go in models/; fetch known presets with pwsh scripts/download-model.ps1 -Model <preset> (see script header for the list).

Levels 2–4 need real model files and are slow — prefer delegating to a background agent and continuing other work.

name	parity-check
description	Cross-check SharpInference numerics against llama.cpp and run accuracy gates. Use when output quality or numerics are in question — new architecture bring-up, kernel or quantization changes, KV-compression work, "model produces garbage" reports.