메뉴
Download GIFT-EVAL datasets from HuggingFace and run the full 53-dataset GIFT-EVAL benchmark sweep. Use when the user wants to run, reproduce, or evaluate against the GIFT-EVAL benchmark.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Add a new model to ModernTSF by scaffolding it with `tool/tsf.py new-model`, filling the architecture, and verifying with `tsf smoke`. Use when the user wants to integrate a new PyTorch model, register a custom architecture, or wire a new forecaster into the benchmark pipeline.
Verify model(s) end-to-end by running their smoke configs concurrently and reporting PASS/FAIL via `tool/tsf.py smoke`. Use when the user wants to check that a model (new or existing) trains and produces the right output shape, validate a port, or run a quick CI-style gate over many models.
Understand a model in ModernTSF by reading its README card as progressive disclosure — paper venue/date/arXiv/abstract first, then source only if needed. Use when the user asks "what is model X", "tell me about X", "了解 X 模型", "how does X work", which paper is X from, or wants to compare/learn what a forecaster does before running it.
Add or run a *probabilistic* forecasting model in ModernTSF — one that outputs quantiles or a distribution (prediction intervals / uncertainty) instead of a single point, and is scored with CRPS / WQL / coverage. Use this whenever the task involves prediction intervals, quantiles, uncertainty, a probabilistic or distributional forecast, pinball / quantile loss, Gaussian NLL, or the metrics `crps` / `wql` / `coverage_80` / `width_80` — even if the user only says "predict a range" or "give me uncertainty bands". Do NOT use the plain `add-model` flow for these: it builds a point model. Probabilistic output is a separate `output_type` axis, orthogonal to `task.mode`.
Package a finished run into a TSEval Submission Report (result + agent trajectory + report) and contribute it via a GitHub PR to the TSEval leaderboard repo, via `tsf trace` / `tsf submit`. Use when the user wants to submit results to the TSEval leaderboard, capture an experiment trajectory, or build a leaderboard from submissions.
Diagnose and report a ModernTSF framework defect upstream, with user approval, as a GitHub issue or a small verified PR against Diaugeia/ModernTSF. Use after reproducing a crash in src/ or tool/, wrong output or shapes, a broken config or registry, a doc/CLI mismatch, or another repository defect rather than a problem in the user's data or code.
| name | gift-eval |
| description | Download GIFT-EVAL datasets from HuggingFace and run the full 53-dataset GIFT-EVAL benchmark sweep. Use when the user wants to run, reproduce, or evaluate against the GIFT-EVAL benchmark. |
Ask the user:
~/.cache/gift_eval.--datasets for all; --list shows the base/freq names.--link-only to skip the download and just create the symlink.# Step 1 — download + symlink (dataset/gift_eval -> <output-dir>)
uv run python tool/gift_eval_download.py [--output-dir DIR] [--datasets electricity/15T m4_monthly ...] [--link-only] [--list]
# Step 2 — run the benchmark sweep (preview first with the `inspect` skill)
uv run modern-tsf --config configs/runs/gift_eval_sweep.toml
# Step 3 — aggregate + plot per dataset
uv run python tool/tsf.py aggregate-plot --dataset <name> --pred-len <len>
dataset/gift_eval symlink is what TOML configs reference (root_path = "./dataset/gift_eval"); re-running with a different --output-dir updates it.--datasets warn but do not abort.work_dirs/<dataset>/<model>/performance.csv.Dataset list, config layout, and troubleshooting: docs/en/gift-eval.md.