メニュー
Rank models per pred_len/seed for a dataset, producing MSE and MAE leaderboard CSVs. Use when the user wants leaderboards, rankings, or to compare model performance across prediction horizons.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Add a new model to ModernTSF by scaffolding it with `tool/tsf.py new-model`, filling the architecture, and verifying with `tsf smoke`. Use when the user wants to integrate a new PyTorch model, register a custom architecture, or wire a new forecaster into the benchmark pipeline.
Verify model(s) end-to-end by running their smoke configs concurrently and reporting PASS/FAIL via `tool/tsf.py smoke`. Use when the user wants to check that a model (new or existing) trains and produces the right output shape, validate a port, or run a quick CI-style gate over many models.
Understand a model in ModernTSF by reading its README card as progressive disclosure — paper venue/date/arXiv/abstract first, then source only if needed. Use when the user asks "what is model X", "tell me about X", "了解 X 模型", "how does X work", which paper is X from, or wants to compare/learn what a forecaster does before running it.
Add or run a *probabilistic* forecasting model in ModernTSF — one that outputs quantiles or a distribution (prediction intervals / uncertainty) instead of a single point, and is scored with CRPS / WQL / coverage. Use this whenever the task involves prediction intervals, quantiles, uncertainty, a probabilistic or distributional forecast, pinball / quantile loss, Gaussian NLL, or the metrics `crps` / `wql` / `coverage_80` / `width_80` — even if the user only says "predict a range" or "give me uncertainty bands". Do NOT use the plain `add-model` flow for these: it builds a point model. Probabilistic output is a separate `output_type` axis, orthogonal to `task.mode`.
Package a finished run into a TSEval Submission Report (result + agent trajectory + report) and contribute it via a GitHub PR to the TSEval leaderboard repo, via `tsf trace` / `tsf submit`. Use when the user wants to submit results to the TSEval leaderboard, capture an experiment trajectory, or build a leaderboard from submissions.
Diagnose and report a ModernTSF framework defect upstream, with user approval, as a GitHub issue or a small verified PR against Diaugeia/ModernTSF. Use after reproducing a crash in src/ or tool/, wrong output or shapes, a broken config or registry, a doc/CLI mismatch, or another repository defect rather than a problem in the user's data or code.
| name | rank |
| description | Rank models per pred_len/seed for a dataset, producing MSE and MAE leaderboard CSVs. Use when the user wants leaderboards, rankings, or to compare model performance across prediction horizons. |
After experiments have produced performance.csv files under work_dirs/. Ask the user for the dataset name if not provided.
uv run python tool/rank_models.py --dataset <DATASET>
Writes model_rankings_mse.csv, model_rankings_mae.csv, and model_rankings_long.csv to work_dirs/<DATASET>/. Rankings are computed per (pred_len, seed) group; rank 1 = lowest metric value.
For a fair leaderboard (TFB-style), add --null-threshold 0.3 (exclude models missing on too many cells), --aggregate mean|median|max (collapse duplicates), and/or --fill-nan-with-mean.
Full flags, input requirements, and output formats: docs/en/rank-models.md (or tool/rank_models.py --help).