菜单
Benchmark inference throughput and latency for Sparkstation models. Measures TTFT, tokens/sec, p50/p95/p99 latency, and embedding throughput. Use when asked to "benchmark", "measure latency", "test throughput", or "profile inference performance".
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/kshetrajna12/sparkstation --skill sparkstation-bench命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
正在显示 SKILL.md
基于 SOC 职业分类
| name | sparkstation-bench |
| description | Benchmark inference throughput and latency for Sparkstation models. Measures TTFT, tokens/sec, p50/p95/p99 latency, and embedding throughput. Use when asked to "benchmark", "measure latency", "test throughput", or "profile inference performance". |
Automated inference benchmarking for all Sparkstation model types: chat, embeddings (text + image), and vision.
Run the benchmark script with the desired mode:
# Benchmark chat model (default: qwen3-vl-30b)
python3 .pi/skills/sparkstation-bench/scripts/bench.py chat --model qwen3-vl-30b --concurrency 1 --requests 10
# Benchmark with concurrency sweep
python3 .pi/skills/sparkstation-bench/scripts/bench.py chat --model qwen3-vl-30b --concurrency 1,2,4,8 --requests 20
# Benchmark text embeddings
python3 .pi/skills/sparkstation-bench/scripts/bench.py embedding --model bge-m3 --requests 50
# Benchmark image embeddings (CLIP)
python3 .pi/skills/sparkstation-bench/scripts/bench.py clip --model clip-vit --requests 20
# Benchmark vision (image + chat)
python3 .pi/skills/sparkstation-bench/scripts/bench.py vision --model qwen3-vl-30b --requests 5
# Full suite — benchmarks all running models
python3 .pi/skills/sparkstation-bench/scripts/bench.py all
Outputs structured METRIC name=value lines for autoresearch compatibility, plus a human-readable summary table:
METRIC chat_ttft_p50_ms=142.3
METRIC chat_ttft_p95_ms=210.5
METRIC chat_tokens_per_sec=45.2
METRIC chat_latency_p50_ms=890.1
METRIC chat_latency_p95_ms=1230.4
METRIC embedding_requests_per_sec=120.5
METRIC embedding_latency_p50_ms=8.3
--model: Model alias (default: auto-detect from running models)--concurrency: Concurrent requests, comma-separated for sweep (default: 1)--requests: Total requests per concurrency level (default: 10)--max-tokens: Max tokens for chat responses (default: 256)--prompt: Custom prompt for chat benchmarks--warmup: Number of warmup requests (default: 2)--base-url: Gateway URL (default: http://localhost:8000/v1)--output: Output format: text, json, metrics (default: text)This skill outputs METRIC lines compatible with autoresearch. To optimize inference:
autoresearch.sh that calls bench.py with your target workloadmodels.yamlopenai, httpx (already in sparkstation venv)