메뉴
Download and analyze InferenceX ML inference benchmark data — GPU performance metrics across hardware, frameworks, and models. Use when asked to analyze inference benchmarks, compare GPUs, plot pareto frontiers, or work with InferenceX data.
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.
npx skills add https://github.com/SemiAnalysisAI/InferenceX-app --skill inferencex-data명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.
로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.
SKILL.md 표시 중
Use when reviewing PRs or diffs that add or modify user-visible Simplified Chinese in InferenceX, including refactors and files whose names do not contain zh.
Author an InferenceX benchmark blog post in MDX. Codifies the structure, numeric-verification workflow, frontmatter, MDX components, dashboard links, and FAQ JSON-LD pattern used by published InferenceX posts. Use when asked to draft, write, or scaffold a new blog post comparing GPUs/frameworks/precisions/models, or to write up a specific PR-driven performance change.
Repository-specific guidance for working with InferenceX's Neon PostgreSQL databases, connection variables, migrations, and query code. Use when Neon, Postgres, DATABASE_URL, database, schema, migration, or backend data access is mentioned.
SOC 직업 분류 기준
| name | inferencex-data |
| description | Download and analyze InferenceX ML inference benchmark data — GPU performance metrics across hardware, frameworks, and models. Use when asked to analyze inference benchmarks, compare GPUs, plot pareto frontiers, or work with InferenceX data. |
Download the latest database dump from GitHub releases. It is xz-compressed and split into
one or more .tar.xz.part* files; reassemble them by piping cat through xz (requires xz):
gh release download --repo SemiAnalysisAI/InferenceX-app --pattern 'inferencex-dump-*.tar.xz.part*' --dir .
cat inferencex-dump-*.tar.xz.part* | xz -d -T0 | tar -x
Each .json file corresponds to one database table.
| File | Description |
|---|---|
configs.json | Serving deployment configs: hardware + framework + model + precision + parallelism |
workflow_runs.json | GitHub Actions workflow run metadata |
server_logs.json | Raw benchmark server logs (~140KB per row, very large — avoid loading entirely) |
benchmark_results.json | Performance metrics per config/concurrency/sequence-length/date |
run_stats.json | Per-hardware reliability stats (n_success / total) |
eval_results.json | LM evaluation accuracy results (e.g. gsm8k) |
availability.json| Denormalized date x config availability |
changelog_entries.json | PR/change descriptions per workflow run |
benchmark_results[].config_id -> configs[].idbenchmark_results[].workflow_run_id -> workflow_runs[].idbenchmark_results[].server_log_id -> server_logs[].id (nullable)eval_results[].config_id -> configs[].ideval_results[].workflow_run_id -> workflow_runs[].idrun_stats[].workflow_run_id -> workflow_runs[].idchangelog_entries[].workflow_run_id -> workflow_runs[].idEach config is a unique serving deployment:
id, hardware, framework, model, precision, spec_method, disagg, is_multinode,
prefill_tp, prefill_ep, prefill_dp_attention, prefill_num_workers,
decode_tp, decode_ep, decode_dp_attention, decode_num_workers,
num_prefill_gpu, num_decode_gpu
tp: tensor parallelism, ep: expert parallelism, dp_attention: data parallel attention, num_workers: pipeline parallel workersspec_method: speculative decoding method. mtp = multi-token prediction, none = standard autoregressivedisagg=true means prefill and decode run on separate GPU pools (disaggregated serving). When disagg=false, prefill and decode fields are identicalnum_prefill_gpu + num_decode_gpu for disagg, or just num_prefill_gpu for non-disagg. When not explicitly set, defaults to tp * epid, workflow_run_id, config_id, benchmark_type, date, isl, osl, conc,
image, metrics, error, server_log_id
benchmark_type: currently always single_turndate: ISO 8601 timestamp string (e.g. 2025-10-12T00:00:00.000Z)isl / osl: input/output sequence length in tokensconc: concurrency levelimage: Docker image used for the serving framework (e.g. lmsysorg/sglang:v0.5.8.post1-cu130), null for runs before 2025-12-08error: null means successmetrics: nested JSON object — access as row["metrics"]["tput_per_gpu"], not as top-level fieldsAll latency values in seconds. tput_per_gpu is total throughput (input+output tokens) per second per GPU.
Throughput: tput_per_gpu, output_tput_per_gpu (optional), input_tput_per_gpu (optional)
TTFT (time to first token): median_ttft, mean_ttft, p99_ttft, std_ttft
TPOT (time per output token): median_tpot, mean_tpot, p99_tpot, std_tpot
ITL (inter-token latency): median_itl, mean_itl, p99_itl, std_itl
E2EL (end-to-end latency): median_e2el, mean_e2el, p99_e2el, std_e2el
Interactivity: median_intvty, mean_intvty, p99_intvty, std_intvty
New numeric metrics may appear in future dumps without schema changes — the ETL auto-captures any numeric field not reserved for config dimensions.
id, workflow_run_id, config_id, task, date, isl, osl, conc, lm_eval_version, metrics
task: evaluation task name (e.g. gsm8k)metrics: nested object with n_eff, em_strict, em_flexible, em_strict_se, em_flexible_seid, github_run_id, run_attempt, name, status, conclusion, head_sha,
head_branch, html_url, created_at, run_started_at, date
run_attempt: re-runs of the same github_run_id get incrementing attempts. When computing latest results, first filter to the highest run_attempt per github_run_id.conclusion: success, failure, or cancelledhtml_url: link to the GitHub Actions runid, workflow_run_id, date, hardware, n_success, total
n_success / total per hardware per runDenormalized table for fast date-picker lookups — one row per (model, isl, osl, precision, hardware, framework, spec_method, disagg, date) combination. Not joined by config_id; uses the raw text keys directly.
model, isl, osl, precision, hardware, framework, spec_method, disagg, date
id, workflow_run_id, date, base_ref, head_ref, config_keys, description, pr_link
config_keys: array of strings like ["dsr1-fp8-mi355x-mori-sglang"] identifying which configs changedpr_link: GitHub PR URL (nullable)Framework names are normalized during ingest: dynamo-trtllm -> dynamo-trt, sglang-disagg -> mori-sglang. The dump contains already-normalized values.
hardware: h100, h200, b200, b300, gb200, gb300, mi300x, mi325x, mi355x model: dsr1=DeepSeek-R1-0528, gptoss120b=gpt-oss-120b, llama70b=Llama-3.3-70B-Instruct-FP8, qwen3.5=Qwen-3.5-397B-A17B, kimik2.5=Kimi-K2.5, minimaxm2.5=MiniMax-M2.5, glm5=GLM-5 framework: atom, dynamo-sglang, dynamo-trt, mori-sglang, sglang, trt, vllm precision: bf16, fp4, fp8, int4 spec_method: mtp, none
configs.json and benchmark_results.json, join on config_id, filter out rows where error is not null.workflow_runs to the highest run_attempt per github_run_id, then join to benchmark_results. For each unique (config_id, conc, isl, osl), keep only the row with the latest date.