Menu
Use when working on dataset materialization, model training, backtesting, scan-edge signals, paper-trading workflows, or experiment artifacts in polymarket-tmax-lab.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Use when running the LGBM autoresearch loop, editing candidate YAML specs, gating new variants, or promoting winners in polymarket-tmax-lab.
Use when working on bootstrap-lab, warehouse management, seed export or restore, legacy raw/parquet cleanup, or canonical data layout in polymarket-tmax-lab.
Use when working on dataset materialization, model training, backtesting, scan-edge signals, paper-trading workflows, or experiment artifacts in polymarket-tmax-lab.
Use when working on bootstrap-lab, warehouse management, seed export or restore, legacy raw/parquet cleanup, or canonical data layout in polymarket-tmax-lab.
Use when running the LGBM autoresearch loop, editing candidate YAML specs, gating new variants, or promoting winners in polymarket-tmax-lab.
Use when changing Polymarket discovery, temperature market filtering, rule parsing, settlement source handling, station mapping, or official truth adapters.
| name | pmtmax-research-loop |
| description | Use when working on dataset materialization, model training, backtesting, scan-edge signals, paper-trading workflows, or experiment artifacts in polymarket-tmax-lab. |
Use this skill for the research and trading simulation loop.
AGENTS.md.docs/agent-skills/research-loop.mddocs/codebase/modeling.mddocs/codebase/backtest-execution.mdNEVER record paper-trader output when book_source is missing or fixture-only — those rows are diagnostics and must not pollute forward_paper_trades.json.
Before running paper-trader, ALWAYS verify real temperature markets exist:
import httpx
r = httpx.get('https://gamma-api.polymarket.com/markets?active=true&limit=50', timeout=10)
data = r.json()
temp = [m for m in data if any(k in m.get('question','').lower() for k in ['temperature','highest','celsius','°c','°f'])]
print(f'Active temperature markets: {len(temp)}') # Must be > 0
After paper-trader, check book_source in artifacts/workspaces/ops_daily/signals/v2/paper_signals_latest.json:
book_source contains "clob"book_source is missing or fixture-onlyTemperature markets open daily — real prices available ~KST 09:00. Run paper-trader after markets open.
forward_paper_trades.json was reset 2026-04-22 after non-CLOB priced trades were found.
weather_train with weather_real; it must not include Polymarket ids, rules, price history, CLOB books, or publish evidence.collect-weather-training should now be treated as a monitored batch step:
keep stderr progress enabled unless a wrapper needs quiet output, and prefer
short HTTP bounds (--http-timeout-seconds 15 --http-retries 1) for free-tier
Open-Meteo historical-forecast collection. Keep --max-consecutive-429 2;
two consecutive Open-Meteo 429 responses are a daily-limit hit, so cancel
the remaining chunk and record rate-limit-cancelled in checker state.scripts/pmtmax-workspace weather_train uv run python scripts/run_weather_train_queue_agent.py
It advances the next 7-day checker queue and auto-refreshes
gaussian_emos pretrain once the row-gap threshold is reached. It defaults
to the two-consecutive-429 cancel rule.weather_train gold has
11,499 rows. Full successful ranges are 2024-01-03..2024-05-24,
2024-06-01..2024-12-30, and 2026-01-01..2026-01-14; retry-only/free-path
daily-limit ranges include 2025-01-07..2025-01-27 and
2026-01-22..2026-01-28. Read live details from
checker/weather_train_status.md.checker/; update
checker/weather_train_status.md and checker/weather_train_collection_log.md
after every run.checker/; update checker/historical_price_status.md and
checker/historical_price_collection_log.md after every recovery run.weather_train queue
collection in parallel with historical_real backfill-price-history because
those jobs use separate workspace roots. Do not overlap multiple mutating
historical_real jobs; finish backfill-price-history before
materialize-backtest-panel, and finish panel rebuild before benchmark or
publish gates.scripts/pmtmax-workspace historical_real uv run python scripts/run_historical_price_recovery_agent.py
It advances one checker shard by default and records the latest panel-ready
decision rows plus the latest backtest priced_decision_rows anchor.scripts/pmtmax-workspace historical_real uv run python scripts/run_model_research_agent.py
It keeps one active autoresearch run aligned to the current dataset/panel
signatures, auto-creates the next small candidate YAML when needed, and
tracks train/step/gate/paper/promote state under checker/model_research_*.historical_real / real_market only.--model values when
materializing from the warehouse. The base config can default to GFS-only;
compare source feature counts before training so a row-count increase is not
mistaken for a multi-source dataset.book_source=fixture; missing CLOB books must be missing_book, not tradable signals.synthetic_ market ids, fixture forecasts, and fabricated books are trust violations in canonical research.build-dataset should still be run with an explicit --markets-path configs/market_inventory/full_training_set_snapshots.json.
Without it the CLI now auto-defaults to that same inventory (printing a yellow warning) when the file exists, and a
shrinkage guard (refuses to overwrite when the new frame is <50% of the existing row count) blocks accidental shrink.
Only if the default inventory is missing does it fall back to the ~12 bundled example rows.full_training_set_snapshots.json is a checked-in training inventory. It is not the auto-refreshed canonical historical backlog.configs/market_inventory/historical_temperature_snapshots.json first and then intentionally regenerate full_training_set_snapshots.json from that curated inventory.historical_training_set* / historical_backtest_panel overwrite requires --allow-canonical-overwrite.--output-name values for experiments and rely on the automatic artifacts/recovery/ backup when promoting canonical output.scan-edge MUST include --min-model-prob 0.05 --max-model-prob 0.95.0 9 * * * for scripts/daily_experiment.sh; 0 0 * * * is only the equivalent on a UTC host.scripts/run_price_check.sh and redirect to an absolute logs/price_check.log path. Do not put raw uv run python scripts/log_gamma_prices.py commands directly in crontab.train-advanced --model-name lgbm_emos --variant <variant>.
Passing --pretrained-weather-model injects weather_pretrain_* features at
fit/predict time, not just sidecar lineage. The first high_neighbor_oof
quick eval with active injection worsened CRPS_C from 0.8004 to 0.8241,
and the delta_only follow-up still worsened to 0.8111; do not promote
weather-pretrain feature candidates without a stronger candidate-specific gate.historical_training_set_curated_multisource_20260425
restored all four NWP source feature groups and improved canonical baseline
holdout CRPS_C (0.8004 -> 0.6749), but on the curated holdout it worsened
CRPS_C versus GFS-only (2.0559 -> 2.8771) despite better Brier/DirAcc due
to under-dispersed, cold-biased tails. Do not promote before variance/tail
calibration.scripts/run_tail_calibration_experiment.py
produced diagnostic wrappers under
artifacts/workspaces/historical_real/models/curated_multisource_tailcal_20260426/.
Balanced tailcal reduced curated holdout CRPS_C to 1.7195 and kept baseline
holdout near canonical baseline at 0.8043, but the root signal was
zero-Celsius NWP daily-max sentinels (32F / 0C) marked available. Treat
this as diagnostic; fix feature validity upstream before promotion.historical_training_set_curated_multisource_sentinelfix_20260426 removed
all-source zero-C sentinel rows (1083 -> 0) and retrained
lgbm_emos/high_neighbor_oof, but curated holdout CRPS_C remained worse than
same-holdout GFS-only (2.7690 vs 2.1103).scripts/run_source_gating_experiment.py trained binary/regret-weighted
train-side gates between sentinel-fix multi-source and GFS-only. Best curated
holdout CRPS_C was only 2.7607 vs raw primary 2.7690 and GFS-only
2.1103; baseline holdout stayed safe (0.4932 best vs raw 0.4965).
Gate fallback use stayed near 2% on curated holdout, so this is diagnostic
only. Next experiment should target source-disagreement/region-driven variance
calibration or proper OOF stacking, not simple binary fallback gating.scripts/run_disagreement_calibration_experiment.py showed variance-only
source-disagreement calibration improves curated holdout CRPS_C to 2.2342
but still loses to GFS-only 2.1103. Positive primary-vs-GFS disagreement
mean shrink plus variance calibration improves curated holdout CRPS_C to
1.3153 with Brier 0.0748, but worsens baseline holdout CRPS_C to
0.5627 from raw 0.4965. Treat as the strongest diagnostic wrapper so far,
not promotion. Next serious candidate should be a non-holdout-tuned LGBM
disagreement feature variant or proper OOF stacker.2,602
snapshots. That verified backlog was promoted into
full_training_set_snapshots.json as a sharded manifest plus
full_training_set_snapshots.d/part-000..003.json; canonical v2
historical_training_set now has 7,794 rows / 2,598 markets and zero
all-source daily-max sentinels. Canonical v2 historical_backtest_panel has
68,550 token rows with 30,615 ok, 37,501 missing, and 434 stale. No
two-consecutive-429 cancellation occurred in discovery120, and follow-up
discovery160 appended 0 URLs. A later broader discovery appended 22
URL-manifest entries that still need snapshot promotion. This is dataset
promotion, not model promotion.uv run python scripts/quick_eval.py (champion baseline + OOF variants).observation-report, observation-shadow, observation-daemon, approve-live-candidate.paper-multimodel-report, execution-sensitivity-report, market-bottleneck-report.execution-watchlist-playbook.exact_public intraday -> documented research intraday -> METAR fallback, target-day only.station-dashboard, station-dashboard-daemon.artifacts/workspaces/ops_daily/signals/v2/execution_watchlist_playbook.json when present and raises Tier A ask-threshold alerts without changing live guardrails.station-cycle, station-daemon.artifacts/workspaces/ops_daily/benchmarks/v2/benchmark_summary.jsonartifacts/public_models/champion.json.lgbm_emos / high_neighbor_oof.pmtmax-autoresearch when you are exploring new lgbm_emos candidates around the current champion or the run manifest baseline.publish_gate.decision=GO are invalid; republish with publish-champion after a real recent-core GO instead of using or repairing them manually.scripts/pmtmax-workspace weather_train uv run pmtmax collect-weather-training --city Seoul --date-from 2024-01-01 --date-to 2024-01-07 --http-timeout-seconds 15 --http-retries 1 --missing-only
scripts/pmtmax-workspace weather_train uv run pmtmax train-weather-pretrain --model-name gaussian_emosscripts/pmtmax-workspace historical_real uv run pmtmax build-dataset --markets-path configs/market_inventory/full_training_set_snapshots.json --forecast-missing-only --allow-canonical-overwritescripts/pmtmax-workspace historical_real uv run pmtmax materialize-backtest-panel --markets-path configs/market_inventory/full_training_set_snapshots.json --allow-canonical-overwritescripts/pmtmax-workspace historical_real uv run pmtmax summarize-price-history-coverage --markets-path configs/market_inventory/full_training_set_snapshots.json
scripts/pmtmax-workspace historical_real uv run pmtmax backfill-price-history --markets-path configs/market_inventory/full_training_set_snapshots.json --only-missing --price-no-cache --limit-markets 25
scripts/pmtmax-workspace historical_real uv run pmtmax materialize-backtest-panel --markets-path configs/market_inventory/full_training_set_snapshots.json --allow-canonical-overwriteuv run pmtmax bootstrap-lab --forecast-missing-onlyscripts/pmtmax-workspace historical_real uv run pmtmax backfill-forecasts --markets-path configs/market_inventory/historical_temperature_snapshots.json --strict-archive --missing-onlyscripts/pmtmax-workspace historical_real uv run pmtmax backfill-forecasts --markets-path configs/market_inventory/historical_temperature_snapshots.json --strict-archive --missing-only --single-run-horizon market_open --single-run-horizon previous_evening --single-run-horizon morning_ofscripts/pmtmax-workspace historical_real uv run pmtmax materialize-training-set --markets-path configs/market_inventory/historical_temperature_snapshots.json --model ecmwf_ifs025 --model ecmwf_aifs025_single --model kma_gdps --model gfs_seamless --output-name <variant>scripts/pmtmax-workspace historical_real uv run python scripts/run_tail_calibration_experiment.pyscripts/pmtmax-workspace historical_real uv run python scripts/run_source_gating_experiment.pyscripts/pmtmax-workspace historical_real uv run python scripts/run_disagreement_calibration_experiment.pyscripts/run_recent_core_benchmark_local.sh --model-name lgbm_emos --variant <variant> --retrain-stride 1scripts/pmtmax-workspace recent_core_eval uv run python scripts/run_recent_core_benchmark.py --model-name lgbm_emos --variant <variant> --retrain-stride 10 --backtest-last-n 60 --prebuilt-dataset-path data/workspaces/historical_real/parquet/gold/v2/historical_training_set.parquet --prebuilt-panel-path data/workspaces/historical_real/parquet/gold/v2/historical_backtest_panel.parquet --prebuilt-last-n-market-days 90data/workspaces/historical_real/parquet/gold/v2/historical_training_set.parquetartifacts/workspaces/<workspace>/models/v2/artifacts/public_models/champion.jsonartifacts/workspaces/ops_daily/signals/v2/scan_edge_latest.jsonartifacts/workspaces/ops_daily/signals/v2/live_pilot_queue.jsonartifacts/workspaces/ops_daily/signals/v2/forward_paper_trades.jsonlogs/daily_experiment.logscripts/run_price_check.shlogs/price_check.logconfigs/paper-exploration.yamlconfigs/paper-all-supported-horizon-policy.yamlartifacts/workspaces/ops_daily/signals/v2/execution_watchlist_playbook.json