القائمة
Backtest observation features that receive hardcoded defaults vs training dynamic values — and why fixing them DEGRADES results. Includes walk-forward batch mode fix.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Normalize long-form CODEX cycle folders to short form before notebooks run. Trigger: cyc001_reg001_*, hard-coded cyc paths breaking, staged CODEX raw data failing in Notebooks 1/2.
v5.6.0 joint multi-TF model: single model per symbol with broadcast 1Hour context replaces dual 15Min/1Hour models. Trigger: (1) replacing weighted-voting model aggregation, (2) adding broadcast features to vectorized env, (3) limited training data + worried about overfitting from doubling obs_dim, (4) backtest builder mismatch with newer feature counts.
DEPRECATED in v5.6.0 — see joint-multi-tf-v560 skill. Documents the v5.2.0 dual-model approach (train separate 15Min/1Hour models, combine via weighted voting). Still relevant for: (1) loading legacy v5.5.0 dual models, (2) understanding the historical aggregation layer, (3) resampling pattern via origin='start'.
Surface a shipped-but-undocumented CLI feature in user-facing docs. Trigger: user reports a known feature missing from README/readthedocs even though the CLI command exists.
KINTSUGI Snakefile + CLI changes that route SLURM jobs around accounts saturated by OTHER users on the same QOS pool. Trigger: QOSGrpMemLimit, jobs stuck pending despite available GPU slots in config, noisy neighbor on shared QOS, multi-user investment pool exhaustion, _build_cycle_assignment static-vs-live.
KINTSUGI SLURM batch processing: Maximize throughput using multi-account resource calculation with GPU+CPU pools per account. Trigger: SLURM job submission, batch processing, resource maximization, GPU+CPU concurrent, headless processing, resource pool.
| name | backtest-observation-mismatch |
| description | Backtest observation features that receive hardcoded defaults vs training dynamic values — and why fixing them DEGRADES results. Includes walk-forward batch mode fix. |
| version | 1.0.0 |
| tags | ["backtest","observation","mismatch","walk-forward","win-rate"] |
| triggers | ["backtest win rate","backtest accuracy","observation mismatch","walk-forward batch","BacktestObservationBuilder","sub-50% win rate"] |
16/65 features in BacktestObservationBuilder.get_obs_at_bar() receive hardcoded default values instead of the dynamic values they receive during training. Sub-50% win rates in backtest led to investigation.
Models use high R-multiple strategies — win rate < 50% but avg win >> avg loss. Example: CNM_1Hour has 46.7% win rate but PF=1.80 because avg win ($138.86) is 2.05x avg loss ($67.61). This is normal in quantitative trading.
| Feature Group | Count | Default | Training Value | Why Default Is Better |
|---|---|---|---|---|
| Intraday | 4 | 0.5 | step % 390 / 390 | Training pattern uses per-episode step counter that can't be replicated in backtest |
| Account state | 3 | 0.0/0.5/0.0 | Dynamic P&L/win_rate/drawdown | Episode dynamics (4096 envs, ~400 steps) don't match single-pass backtest |
| Position sizing | 3 | 0.0/1.0/0.0 | Dynamic position/capital state | Same episode dynamics issue |
| Regime bars | 3 | 0.0 | Dynamic 0-1.0 duration | Infrastructure exists but requires GPU Markov system |
| Calendar | 7 | 0.5 fallback | Real calendar data | Usually works; 0.5 is safety fallback |
scripts/run_backtest.py: --all --walk-forward N was silently running single-pass backtests because run_all_models() ignored the walk_forward parameter. Fixed by passing walk_forward through to run_walk_forward() per model.
alpaca_trading/gpu/inference_obs_builder.py — BacktestObservationBuilderalpaca_trading/gpu/vectorized_env.py — Training observation constructionalpaca_trading/backtest/engine.py — BacktestEngine.run()scripts/run_backtest.py — CLI with batch walk-forward fixWould require either: (1) retraining models with backtest-compatible observation patterns, or (2) implementing full episode simulation in backtest that exactly matches training env dynamics (4096 parallel envs with random resets).