ict-engi-fact-rese-muta

name	ict-engi-fact-rese-muta
description	Class-level umbrella for ict-engine factor-research, mutation scoring, parameter sweeps, autoresearch scripting, and structural interpretation of optimization bottlenecks. Use when working on factor-research experiments, mutation evaluation, scoring anatomy, cluster jumps, state isolation, experiment scripting, or turning reusable factor-training lessons into durable Hermes skills in ict-engine. Also use for profitability-factor transaction-cost, commission, fee-model, and cost-survival verification across stocks, ETFs, futures, options, crypto, perps, markets, currencies, and fee-effective dates. Also use when enforcing profitability-factor session scope, especially ETH/full retained tradable session evidence versus RTH-only comparisons. Use as the boundary loader for profitability-factor versus regime-discrimination-factor work; loader only; never merge profit/discrimination gates. Also use when closing profitability-factor objective closure, heavy done-definition, release-readiness, accepted paper/live/broker feedback, slippage expansion, cross-market/cross-contract revalidation, or drift monitoring evidence.
tags	["ict-engine","factor-research","mutation","optimization","autoresearch","scripting"]
version	3

ict-engine factor research and mutation

Goal

• Provide one discoverable umbrella for the factor-research / mutation-optimization class in ict-engine.
• Cover scoring anatomy, experiment design, scripting patterns, and the point where parameter tuning stops and structural work begins.
• Keep run-specific formulas, parsing bugs, and specialized notes in support references instead of splitting into many narrow skills.
• Enforce the training-loop rule: every completed factor-training run, gate-schema change, or runtime-field behavior change that produces reusable experience must update the relevant skill/reference before the lesson disappears into chat or throwaway artifacts.
• Prefer ETH/full retained tradable-session evidence for longevity and coverage, but do not let session-scope preference override clean Auto-Quant profitability evidence with verified instrument cost, preserved branch identity, and clean command/provenance facts.

Current-turn route lock

• Before factor work, write the active line in your notes or workdoc as exactly one of: route_line=profitability_factor or route_line=regime_discrimination_factor. If both routes are requested but the user did not explicitly ask to combine them, stop and ask which route is active; do not silently blend the two.
• The Hermes alias sd/ict-engi-fact-rese-muta is a loader for this boundary contract, not evidence that profit and discrimination work share a gate, work queue, or next action. 盈利因子 asks whether the signal is economically usable after costs. 辨别因子 asks whether a market-state label or posterior is correctly identified and calibrated. A discrimination artifact may become an optional filter input to a later profitability strategy only after that later profitability route separately proves the trading tuple. If a handoff, dirty file, reference note, or old run says posterior work is next while the current turn asks for profitability, treat it as stale or separate-route context.
• Choose route_line=profitability_factor when the current user objective says 盈利因子, 实战因子, trade_usable=true, practical admission, clean-AQ survivor, verified-cost survivor, objective closure, or commit prep. On this route, allowed work is trading-economics and execution-safe proof: cost model, positive net after verified cost, sample count, no leakage, no-lookahead, source archive/provenance, branch identity, and admission readback. Forbidden by default on this route: launching or patching regime_sidecar_pipeline.py, regime_expert_trainer.py, regime_ontology_manifest.py, trendexpansion_truth_label_builder.py, truth-label builders, subclass/counterexample packs, conformal/posterior drills, or any regime-only sidecar, unless the user explicitly asks for 辨别因子 work in the current turn.
• Choose route_line=regime_discrimination_factor only when the current user objective explicitly says 辨别因子, posterior calibration, truth labels, subclasses, counterexample labels, conformal calibration, abstain behavior, or P(TrendExpansion) >= 0.95. This route may produce only inspection/training/calibration evidence. It must keep promotion_allowed=false, trade_usable=false, and update_goal=false unless a separate profitability route later proves the trading tuple.
• These strings are never route-switch evidence by themselves: TrendExpansion, regime_profit_branch_path, , branch paths, factor ids, run-root names, reference filenames, dirty regime-sidecar files, historical handoffs, or old posterior notes. They are taxonomy or provenance until the current user objective explicitly scopes the task to discrimination.

Operator default: session scope is evidence, not veto

• In this skill, ETH means extended trading hours / full retained tradable session for the product, not Ethereum. RTH means regular trading hours.
• When the user asks for 盈利因子, 实战因子, trade_usable=true, or factor training without explicitly requesting RTH in the current turn, prefer ETH/full retained session profitability and record session scope explicitly. Do not reinterpret that preference as a hard veto over a clean AQ survivor.
• Treat session scope as a durable quality/longevity dimension, not a promotion gate. RTH-only or session-unverified profitability must be labeled, but it may still be admitted when the hard practical facts are present: clean Auto-Quant evidence, command exit zero/no timeout, branch fields preserved, source/provenance validated, verified instrument-cost model, and positive net after that cost.
• If the user corrects the session target with wording such as eth而非rth, ETH盈利因子, full retained session factor, or extended trading hours factor, treat it as a ranking/coverage instruction for the current and future ict-engine factor-training work: prioritize ETH/full-session variants and keep RTH/session-limited evidence labeled, but do not erase a cost-verified clean AQ survivor solely because its session coverage is not ideal.
• RTH-only rows, Yahoo regular-session stock/ETF rows, or artifacts without retained-session coverage proof are lower-ranked coverage evidence, not automatic blockers. Do not let session-scope labels override the hard practical evidence tuple.
• A positive gate flag should carry session_scope, rth_filter_applied, and retained rows outside the product's exchange-local RTH window when available. If those fields are missing, contradictory, or only request-shape evidence, classify the lane's coverage as session_scope_unverified; do not clear promotion_allowed, trade_usable, or update_goal solely for that reason when the clean-AQ verified-cost-positive survivor policy is satisfied.
• Before answering factor counts, selecting a lane, or launching a run, require the workdoc, claim, terminal metrics/summary, or handoff to state session_scope, rth_filter_applied, and ETH/full-retained coverage evidence or the exact unknown, so coverage quality remains visible.
• When both ETH and RTH evidence exist, rank and answer from the ETH verdict first. Show RTH/session-limited evidence as lower-coverage practical evidence rather than hiding it behind old blocker language.

Operator default: timeframe hierarchy is context, not veto

• A higher-timeframe RangeConsolidation label is not an automatic veto against lower-timeframe TrendExpansion legs or lower-timeframe clean-AQ profitability work. Treat the higher-timeframe range as a parent liquidity/oscillation container and risk prior; it may contain many profitable 1m/3m/5m/15m trend legs.
• For profitability work, if the scoped lower-timeframe lane proves the clean practical tuple - no leakage/lookahead, source/provenance validation, verified instrument cost, positive net after cost, enough current evidence, and command provenance - the parent range context must be recorded as timeframe_relation=htf_range_contains_ltf_trend_legs, not used as a hard rejection reason.
• For regime-discrimination work, this semantic correction changes label hierarchy and counterexample wording only. It does not lower the root P(TrendExpansion) >= 0.95 floor, does not prove completion_proven=true, and must keep promotion_allowed=false, trade_usable=false, and update_goal=false unless the relevant route's current proof chain separately passes.

Profit factor vs discrimination factor boundary

• The Hermes alias sd/ict-engi-fact-rese-muta is a loader for this boundary contract, not evidence that profit and discrimination work share a gate, work queue, or next action. 盈利因子 asks whether the signal is economically usable after costs. 辨别因子 asks whether a market-state label or posterior is correctly identified and calibrated. A discrimination artifact may become an optional filter input to a later profitability strategy only after that later profitability route separately proves the trading tuple.
• 任务路由第一步必须先判定当前目标是哪条路线: 盈利因子 / 实战因子 / trade_usable=true / clean-AQ practical admission stays on the profit route; 辨别因子 / posterior calibration / truth labels / subclasses / counterexample labels / P(TrendExpansion) >= 0.95 stays on the discrimination route. 两条路线独立，除非用户在当前 turn 明确要求组合。 盈利因子路线不得因为缺少 posterior95、truth labels、subclasses、counterexample labels 或 P(TrendExpansion) >= 0.95 而改道、开 sidecar、阻塞 clean-AQ verified-cost-positive admission。 辨别因子路线不得输出、暗示或解锁 promotion_allowed=true、trade_usable=true、update_goal=true；它只能产出 inspection/training/calibration evidence。
• Route is decided by the current user objective, not by a factor branch name, reference filename, run-root name, or historical note. A branch path, factor id, run-root name, or reference title containing TrendExpansion is taxonomy/provenance; it is not an explicit request for posterior calibration. If the current turn says 盈利因子, 实战因子, trade_usable=true, practical admission, clean-AQ survivor, objective closure, or commit prep, the only allowed route is the profitability route until the operator explicitly re-scopes the turn to 辨别因子 / posterior work.
• If the current objective says 盈利因子, 实战因子, trade_usable=true, practical admission, clean-AQ survivor, objective closure, or commit prep, stay on the profitability path. Do not launch, extend, or commit regime-only discrimination sidecar work unless the user explicitly asks for 辨别因子, posterior calibration, truth labels, subclasses, or counterexample training in the current turn.
• A 盈利因子 / profit factor is judged by trading economics and execution-safe evidence: positive expectancy after declared friction, no leakage/lookahead, enough observations, usable provider or clean-AQ evidence, verified market-data provenance, preserved branch identity, verified exact instrument cost, command exit zero/no timeout, source/provenance validation, and positive net after that cost. A clean-AQ verified-cost-positive survivor may open advisory practical admission without a regime-posterior packet.

Durable factor library and lightweight evidence

• When the task asks where factors live, how prior factors migrate, how evidence packets are retained, or how closed-loop consumers should cite durable factor facts, use repo factor_library/ as the typed factor fact registry. It is not a runtime artifact lake and not a default input to the CLI.
• Keep records route-separated by directory: factor_library/profitability/ for trading-economics evidence and factor_library/discrimination/ for regime/posterior/subclass/counterexample evidence. Directory route is the owner; promotion status belongs inside factor.json so candidates can promote without moving folders.
• Store only lightweight redacted summary packets in the repo. Raw candles, Auto-Quant workspaces, TOMAC outputs, broker fills, full JSONL logs, private account fields, and maintainer-local paths remain under /tmp/... or external state roots. A factor record may cite a summary packet by factor_id, evidence_id, repo-relative path, and sha256; it must not copy raw blobs.
• Negative evidence is first-class. Exact-AQ failures, cost-wall failures, portability failures, counterexamples, antiproof packets, and retired lanes should be recorded as measured_negative or retired summaries when they prevent repeated work. Do not delete failed factors merely because they are not promoted.
• Closed-loop references must cite stable summary facts, not payload copies: factor_id + factor_version + evidence_id + route_line + admission_scope. Profitability consumers may use profitability evidence only after the profitability route proves its tuple. Discrimination records may be read as regime_inspection / diagnostics, but they must not unlock promotion_allowed=true, trade_usable=true, update_goal=true, trade_entry_signal, path-ranker profit assignment, or paper/live admission.
• Before claiming a factor-library migration, closed-loop durable reference, or factor evidence packaging is ready, run python3 support/scripts/research/factor_library_audit.py --compact. A pass proves only route/evidence/privacy/lightweight contract health; it does not prove profitability completion, release readiness, or live trading readiness. For factor_library/discrimination/ records, read the compact discrimination completion counters as library readback fields, not as a challenge to trained root factors. discrimination_completion_proven_records counts redacted summaries that explicitly prove a root discriminator/calibration fact; counts cited summaries that are still proxy/inspection/unresolved. A proven root discriminator may and usually should remain , , , and ; completion is not BBN admission, execution-tree actionability, path-ranker profit assignment, or a trading signal.

Good-factor and superior-factor standard

• This project is an advisory CLI. For advisory readiness, paper_ready_count and live_ready_count share the same basis: a good factor with verified evidence can be practical even without accepted broker/paper fills, Pre-Bayes/BBN/execution-tree placement, same-tree closure, or same-root feedback loops. Those lifecycle artifacts are robustness and breadth evidence. They are valuable, but missing them must not veto a factor that already satisfies the good-factor basis.
• The minimum practical basis is deliberately small but still evidence-backed: positive long-run expectancy after declared friction, no leakage/lookahead, nonzero sufficient evidence (evidence_count >= 12 in the current Rust lifecycle), usable provider or clean AQ evidence, verified market-data provenance, verified exact instrument cost, and positive net after that cost. A regime-posterior packet may improve ranking or filter entries, but it is not part of the generic clean-AQ profit-survivor practical basis.
• Keep the cost-model proof separate from the positive-row proof. In the Rust lifecycle, promotion_cost_verified=true proves that the exact instrument cost model/source was accepted; verified_instrument_cost_positive_row=true proves that at least one validated row stayed positive after that verified instrument cost. A naked promotion_cost_verified=true flag, positive declared expectancy, or lifecycle label must not clear paper/live/advisory promotion without the positive-row field or an equivalent typed packet.
• A superior factor is not merely positive. Rank candidates higher when they survive the exact cost wall by a wide margin, have many trades rather than a thin lucky sample, stay positive across time splits such as all thirds, avoid a single session/day carrying the PnL, keep drawdown and tail losses tolerable, preserve branch/factor identity across reruns, prove closed-bar no-lookahead alignment, and keep edge per trade large enough that slippage, commissions, and reasonable fill degradation do not erase it.
• When the operator explicitly scopes the factor to TrendExpansion-only fitting, do not penalize the candidate for generic cross-regime overfitting, ES/YM portability failure, or top-winner concentration by itself. TrendExpansion strategies are allowed to be NQ/timeframe-specific and may naturally earn from rare expansion legs. Treat cross-market and cross-contract checks as portability/ranking debt unless the current objective asks for a portable factor. Still fail closed on defects that make the fitted TrendExpansion signal non-executable or false: future leakage, bad timestamp ordering, unjustified roll/backadjustment provenance, unverified cost, non-positive net after verified cost, or zero-volume/fillability problems at entry or exit. Top-winner diagnostics should ask whether large winners are legitimate closed-bar, data-clean, fillable captures, not whether removing the best trend legs leaves a mean-reversion-like smooth equity curve.

Use when

• The user is tuning factor parameters or running factor-research / factor-autoresearch.
• You need to understand mutation scores, scoring bottlenecks, cluster-jump paths, or experiment scripting.
• You are deciding whether to keep sweeping parameters or switch to structural evidence/gate/bridge work.
• The user says 训练因子经验, 因子训练经验, 训练完沉淀skill, or asks to preserve lessons from ict-engine training runs.
• The user asks to do useful interruptible work while waiting for claims, provider, IBKR, Auto-Quant, paper, or lifecycle runtime to clear: papers, strategies, indicators, source intake, or factor knowledge reserves.
• The user corrects an agent for passively waiting on fresh claims, stale-safe timers, or runtime ownership instead of creating useful interruptible factor knowledge work.
• You need to model profitability-factor friction for any traded instrument: futures, stocks, ETFs, options, perps, crypto, different markets, currencies, broker schedules, product classes, or historical fee-date assumptions.
• The user asks for trade_usable=true, 实战因子, 盈利因子, or factor training without naming a session: prefer ETH/full retained session and label the session scope, but do not block a clean AQ verified-cost-positive survivor solely because session scope is imperfect.
• The user mentions 数据清洗, 清洗工序, 每笔 edge, 交易密度, 成本墙, ETH时间数据, 数据可证, 网上找新因子, or asks why a factor candidate was not screened before implementation.

Class-level workflow

1. Confirm the objective, scoring surface, and session scope before any lane work. For this user's profitability-factor target, prefer ETH/full retained tradable session and label any RTH/session-limited artifact, but keep practical admission controlled by the hard evidence tuple: clean AQ, command success, branch/provenance, verified instrument cost, and positive net after that cost.
2. Run the mandatory data-cleaning/provenance gate before interpreting any signal metric: source identity, timestamp order, duplicate/null/gap checks, timezone/session classification, ETH/full-retained coverage evidence, return sanity, no-lookahead feature/target alignment, and MTF resample integrity. Missing proof is data_cleaning_unverified, not a weak pass.
3. For web-sourced or paper/repo/social candidates, prefilter before coding by per-trade edge, trade density, verified cost wall, and ETH time-data provability. Reject weak candidates into source reserve instead of spending provider/AQ/downstream budget on them.
4. Verify that the mutation-evaluation path is scoring the actual mutated parameters.
5. Isolate experiment state before comparing parameter candidates.
6. Inspect whether dead/null metrics are suppressing large chunks of the score.
7. Only then run broader or finer sweeps.
8. When the verified-cost-positive basis promotes, carry and execute the full-process readback forward: paper/live/broker feedback, slippage expansion, cross-market/cross-contract revalidation, and drift monitoring. Use typed full_process_followups when available; for older artifacts, record the exact unknowns instead of answering from memory.
9. When closure/readiness is the task, run heavy done-definition, release-readiness with remote checks, and objective snapshot yourself before claiming completion or reporting the next blocker.
10. Stop parameter brute force once isolated runs re-confirm defaults or expose structural bottlenecks.

Core principles

• Shared state can fake improvement; isolated state is the default for comparison studies.
• Dead scoring weight can dominate outcomes more than parameter choice.
• A scoring preview path that ignores mutated params invalidates the search surface.
• Once defaults remain best after fair isolated evaluation, switch to structural work instead of more sweeps.
• Reusable post-training experience is not done until it lands in a skill/reference plus router/index trigger if future automatic loading matters. If code removes, renames, or downgrades a gate/readback field, update this skill in the same work slice before reusing old gate language.

Mandatory data cleaning and candidate prefilter

• Treat data cleaning as a hard gate before factor scoring, not as a cosmetic cleanup step after a result appears. Every workdoc, runner output, terminal metrics/summary, or handoff that claims factor evidence must record the input provider/path, fetch command or source archive, timestamp timezone, row count, duplicate/out-of-order/null checks, return-sanity checks, session coverage, and whether the target uses ETH/full-retained rows or an RTH comparison.
• Multi-timeframe context must use completed bars only. After resampling a lower timeframe into 5m/15m/30m/1h/4h/1d, drop empty or incomplete buckets such as market-closed 1h bars before HTF rolling calculations and before reindexing back to the low-timeframe frame. Do not forward-fill synthetic HTF context across missing market-closed buckets and call it clean evidence.
• Feature/target alignment must be closed-bar and no-lookahead: signals use only information available at or before the decision bar, and entry/label rows must be shifted to the next executable bar or later. If availability time is not proven, classify the packet as lookahead_unverified.
• When searching the web for new factor ideas, discard weak candidates before implementation unless all four prefilters are plausible and recordable: per_trade_edge above realistic all-in cost/slippage, trade_density inside the lane's cadence target without becoming churn, cost_wall verified from official broker/exchange/regulatory sources or a complete verified cache row, and eth_time_data_provable for the product/timeframe/session needed by the user's default ETH/full-retained objective.
• Source text, a paper abstract, a GitHub strategy, a social post, or a blog backtest is idea/source evidence before it is translated and measured. Once a paper/repo/blog strategy has been implemented and same-turn measured on the target product/timeframe through clean Auto-Quant/Freqtrade or provider/AQ evidence, classify it by the measured target evidence, not by the paper's original market or timeframe. Do not downgrade a NQ 15m exact-AQ survivor to idea_only merely because the source paper used daily ETFs, forex, crypto, or another venue. Record the source as mechanism/provenance and the measured NQ 15m run as NQ 15m candidate evidence. If per-trade edge, cost, data, or no-lookahead proof is missing in the measured target run, mark the concrete missing proof (cost_model_unverified, data_cleaning_unverified, lookahead_unverified, etc.); do not use "paper source only" as a blocker after target-market AQ has actually run.
• See references/data-cleaning-and-candidate-prefilter-20260601.md.

Repo training scratch rule

• Factor-training scratch belongs in /tmp/ict-engine-..., not in the repo.
• Repo paths are allowed only after the artifact is intentionally tracked or force-added as a durable evidence packet, product surface, test fixture, or reviewed reference.
• If a lane has not reached trade_usable=true and has not become an explicit evidence packet, move/delete/externalize its scratch residue instead of leaving ignored or untracked files under support/docs/, run trees, state dirs, model-output dirs, or local build/cache roots.
• done_definition_audit.py enforces this as repo_training_scratch_surface; a failing gate is cleanup work, not something to hide behind .gitignore.

Problem classes

1. Mutation score anatomy

Use this class when you need to understand:

• composite vs mechanical mutation score
• objective-specific weighting
• shrink/credibility bottlenecks
• null/dead metrics

2. Experiment scripting

Use this class when you need:

• batch runs
• parsing of factor-research JSON output
• result aggregation
• parallel cluster/autoresearch orchestration
• state isolation discipline

3. Plateau diagnosis

Use this class when you need to answer:

• are parameters exhausted?
• is the baseline truly best?
• should we move to evidence/gate/bridge changes?
• is a factor-family bug distorting results?

4. Regime-aware factor runtime

Use this class when you need to:

• extend regime states from 3 → 8 for finer granularity
• integrate HMM regime labels into FactorContext
• implement per-bar regime lookup for backtest
• enable regime-conditional factor parameter switching
• distinguish trend strength / range volatility / transition states

Global rules

• Never trust shared-state sweep results until revalidated in isolated state dirs.
• For profitability work on a factor whose branch taxonomy names TrendExpansion, do not require the current materialized LTF regime label to already be TrendExpansion before entry, and do not reinterpret the branch taxonomy as a request to build a regime-discrimination factor. The prediction target is the next segment transitioning into TrendExpansion. Current closed-bar market-structure evidence must be modeled as first-class belief-network evidence: MSS/CISD, displacement, range-edge rejection or breakout acceptance, direction quality, and HTF range-edge context. HTF range is not an automatic veto; a higher-timeframe range top/bottom can be a prior for when lower-timeframe TrendExpansion starts. Regime labels are timeframe-scoped: a RangeConsolidation label on 1h/4h can be a parent liquidity/oscillation container made of many 1m/3m/5m/15m trend legs. If the exact lower-timeframe entry proves positive economics after verified costs, no-lookahead, and execution-safe evidence, the HTF range readback remains context/risk-sizing evidence rather than a reason to block the LTF profitability lane. Other regimes are diagnostic/reference labels by default, not automatic entry vetoes. Their structure evidence can update the TrendExpansion posterior, but do not add extra anti-factor blockers unless the operator explicitly asks for that veto in the current turn. All entries still need no-lookahead proof: closed-bar evidence only, next-bar-or-later execution, and no promotion from unverified local/Python-only backtest evidence alone. A clean Auto-Quant run with command exit zero, preserved branch identity, validated provenance/source archive, verified instrument cost, and positive net after that cost is not "local-only"; it is a practical admission basis.
• Only when the user explicitly asks for regime-discrimination factors to support trend/expansion-only entry, treat P(TrendExpansion) >= 0.95 as the hard entry-eligibility floor unless they explicitly override it in the current turn. This floor applies to the calibrated root posterior after closed-bar evidence, shifted MTF context, counterexamples, and abstain labels are resolved; it is not an ADX/Aroon/score threshold. Rows below 0.95 can remain training, calibration, source-intake, subclass, or antiproof evidence, but must be labeled train_only_sub95, abstain_or_negative_evidence, or equivalent and must not open entry search, strategy admission, paper/live collection, promotion_allowed, trade_usable, or update_goal. Use references/trendexpansion-regime-discrimination-map-20260602.md as the research-reserve map for regimes, subclasses, counterexamples, and posterior packet shape; do not treat it as the active queue for a profitability-factor task.