メニュー
Use when working on bootstrap-lab, warehouse management, seed export or restore, legacy raw/parquet cleanup, or canonical data layout in polymarket-tmax-lab.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | pmtmax-data-ops |
| description | Use when working on bootstrap-lab, warehouse management, seed export or restore, legacy raw/parquet cleanup, or canonical data layout in polymarket-tmax-lab. |
Use this skill for storage, bootstrap, and long-history data operations.
AGENTS.md.docs/codebase/storage-and-configs.md.docs/agent-skills/data-ops.mddocs/agent-skills/safety-and-rules.mdweather_train weather-real station/date pretrain databootstrap-lab, export-seed, restore-seedinventory-legacy-runs, archive-legacy-runsconfigs/market_inventory/historical_temperature_snapshots.jsonconfigs/market_inventory/full_training_set_snapshots.jsonhistorical_training_set* / historical_backtest_panel overwrite is opt-in only.synthetic_ market ids, fixture forecasts, and fabricated books are trust violations.weather_train uses PMTMAX_DATASET_PROFILE=weather_real and must not contain Polymarket market ids, rule JSON, CLOB books, price history, or publish evidence.historical_real / benchmark / publish commands remain PMTMAX_DATASET_PROFILE=real_market.trust-check before canonical overwrite, long collection, benchmark, or release validation.--http-timeout-seconds, --http-retries,
--http-retry-wait-min-seconds, --http-retry-wait-max-seconds, plus
stderr progress by default. Keep --max-consecutive-429 2: two consecutive
Open-Meteo 429 responses mean the free-path daily limit is effectively hit.backfill-forecasts also supports --max-consecutive-429 2.
If two consecutive Open-Meteo 429 responses appear, the run flushes already
collected rows, cancels, and should be recorded in checker state as
rate-limit-cancelled; do not immediately retry the same free-path job.scripts/pmtmax-workspace weather_train uv run python scripts/run_weather_train_queue_agent.py
It reads checker/weather_train_status.md, advances the next 7-day chunk,
updates checker markdown after every chunk, and auto-refreshes
gaussian_emos pretrain when the configured row-gap threshold is met. When
it sees two consecutive 429 responses, it cancels the remaining chunk,
records rate-limit-cancelled, and waits for cooldown/API-key collection.scripts/pmtmax-workspace historical_real uv run python scripts/run_historical_price_recovery_agent.py
It reads checker/historical_price_status.md, advances the next shard,
rebuilds the canonical backtest panel, refreshes the latest coverage JSON,
and updates the historical-price checker markdown files.weather_train gold has 11,499 rows. Full successful ranges are
2024-01-03..2024-05-24, 2024-06-01..2024-12-30, and
2026-01-01..2026-01-14; partial coverage includes
2024-12-31..2025-01-06 and 2026-01-15..2026-01-21.
2025-01-07..2025-01-27 and 2026-01-22..2026-01-28 are retry-only/free-path
daily-limit ranges. Read the live row count from checker/weather_train_status.md.historical_real collection state on April 27, 2026:
discovery120 appended 1 Tokyo event URL after the completed discovery80
batch, merged the local curated backlog to 2,602 snapshots, and then
promoted that verified backlog into full_training_set_snapshots.json as a
sharded manifest plus full_training_set_snapshots.d/part-000..003.json.
Canonical v2 historical_training_set now has 7,794 rows / 2,598
markets, and canonical v2 historical_backtest_panel has 68,550 token rows
with 30,615 ok. No two-consecutive-429 cancellation occurred, and
follow-up discovery160 appended 0 URLs; a later broader discovery appended
22 URL-manifest entries that remain next-round snapshot backlog.
Read live details from checker/historical_real_status.md.checker/weather_train_status.md and
checker/weather_train_collection_log.md synchronized after every
collection/pretrain turn.checker/historical_price_status.md and
checker/historical_price_collection_log.md synchronized after every daily
price recovery turn.weather_train queue collection can run at the same time
as historical_real price-history collection because they write to separate
workspace roots and hit different upstream APIs. Do not run two mutating
historical_real jobs at once; backfill-price-history,
materialize-backtest-panel, build-dataset, and backfill-forecasts
should stay serialized within that workspace.--output-name values by default; only pass --allow-canonical-overwrite for intentional canonical promotion.artifacts/recovery/ first.--truth-no-cache is the real pmtmax flag; use it when cached truth payloads look stale or malformed. --truth-per-source-limit 1 is NOT a pmtmax CLI flag — it is exposed only by the historical refresh/inventory helper scripts (scripts/run_historical_refresh_pipeline.sh, scripts/build_historical_market_inventory.py, scripts/validate_historical_market_inventory.py, scripts/refresh_historical_event_urls.py).historical_temperature_snapshots.json is the canonical curated backlog for data collection. Use this when the task is “collect the latest historical markets”.full_training_set_snapshots.json is a checked-in training inventory, not an auto-refreshed mirror of the canonical historical backlog.full_training_set_snapshots.json is now a sharded manifest for the verified 2,602-snapshot curated backlog. Canonical v2 materializes 7,794 training rows / 2,598 markets; the legacy gold path may still show the old 5,478 row / 1,826 market dataset.ops_daily collection records forward evidence only; it does not auto-append to full_training_set_snapshots.json or retrain the champion.backfill-forecasts --missing-only so the run only fetches forecast request keys that are absent from bronze_forecast_requests.--model explicitly on
backfill-forecasts, build-dataset, or materialize-training-set.
The base config can default to gfs_seamless; a curated rebuild without
explicit models can keep the same row count but silently become GFS-only.0C / 32F forecasts. After changing
this logic, rebuild a non-canonical variant and compare sentinel/source
availability counts before training.scripts/pmtmax-workspace weather_train uv run pmtmax collect-weather-training --date-from 2026-01-22 --date-to 2026-01-22 --model gfs_seamless --missing-only --rate-limit-profile free --http-timeout-seconds 15 --http-retries 1 --http-retry-wait-min-seconds 1 --http-retry-wait-max-seconds 8scripts/pmtmax-workspace historical_real uv run pmtmax backfill-forecasts --markets-path configs/market_inventory/historical_temperature_snapshots.json --strict-archive --missing-onlyscripts/pmtmax-workspace historical_real uv run pmtmax backfill-forecasts --markets-path configs/market_inventory/historical_temperature_snapshots.json --model ecmwf_ifs025 --strict-archive --missing-only --single-run-horizon market_open --single-run-horizon previous_evening --single-run-horizon morning_of --max-consecutive-429 2scripts/pmtmax-workspace historical_real uv run pmtmax materialize-training-set --markets-path configs/market_inventory/historical_temperature_snapshots.json --model ecmwf_ifs025 --model ecmwf_aifs025_single --model kma_gdps --model gfs_seamless --output-name <variant>scripts/pmtmax-workspace historical_real uv run pmtmax build-dataset --markets-path configs/market_inventory/full_training_set_snapshots.json --forecast-missing-only --allow-canonical-overwritescripts/pmtmax-workspace historical_real uv run pmtmax backfill-price-history --markets-path configs/market_inventory/full_training_set_snapshots.json --only-missing --price-no-cache --limit-markets 25
scripts/pmtmax-workspace historical_real uv run pmtmax materialize-backtest-panel --markets-path configs/market_inventory/full_training_set_snapshots.json --allow-canonical-overwritescripts/pmtmax-workspace historical_real uv run pmtmax summarize-price-history-coverage --markets-path configs/market_inventory/full_training_set_snapshots.json
scripts/pmtmax-workspace historical_real uv run pmtmax backfill-price-history --markets-path configs/market_inventory/full_training_set_snapshots.json --only-missing --price-no-cache --limit-markets 25 --offset-markets 0
scripts/pmtmax-workspace historical_real uv run pmtmax materialize-backtest-panel --markets-path configs/market_inventory/full_training_set_snapshots.json --allow-canonical-overwrite
scripts/pmtmax-workspace historical_real uv run python scripts/run_historical_price_recovery_agent.pyuv run pmtmax bootstrap-lab --forecast-missing-onlyfull_training_set_snapshots.json does not regenerate itself from historical_temperature_snapshots.json.historical_temperature_snapshots.json, then explicitly curate and replace full_training_set_snapshots.json in the same change.restore-seed rebuilds the local warehouse from canonical parquet mirrors directly in DuckDB; it should not need pandas full-table reads.Use when running the LGBM autoresearch loop, editing candidate YAML specs, gating new variants, or promoting winners in polymarket-tmax-lab.
Use when working on dataset materialization, model training, backtesting, scan-edge signals, paper-trading workflows, or experiment artifacts in polymarket-tmax-lab.
Use when working on bootstrap-lab, warehouse management, seed export or restore, legacy raw/parquet cleanup, or canonical data layout in polymarket-tmax-lab.
Use when working on dataset materialization, model training, backtesting, scan-edge signals, paper-trading workflows, or experiment artifacts in polymarket-tmax-lab.
Use when running the LGBM autoresearch loop, editing candidate YAML specs, gating new variants, or promoting winners in polymarket-tmax-lab.
Use when changing Polymarket discovery, temperature market filtering, rule parsing, settlement source handling, station mapping, or official truth adapters.