Use when committing ict-engine work, repairing provider/runtime readiness, changing shell/env setup, or turning repeated repo maintenance lessons into durable skills and routing entries.
Instrucciones de origen · Vista previa de solo lectura
name
ict-engine-maintenance-loop
description
Use when committing ict-engine work, repairing provider/runtime readiness, changing shell/env setup, or turning repeated repo maintenance lessons into durable skills and routing entries.
ICT Engine Maintenance Loop
Core rule
Every ict-engine commit, completed factor-training run, gate-schema change, or runtime-field behavior change must ask: did this run teach a durable reusable lesson or invalidate existing skill wording? If yes, update the right skill/reference before or in the same commit. If no, record why in the final evidence.
For training work specifically: every time factor training, factor-research, factor-autoresearch, mutation scoring, provider-backed Auto-Quant, or CatBoost/path-ranking work leaves reusable experience, that experience must be landed in a skill/reference. Do not leave it only in chat, terminal scrollback, or throwaway run artifacts.
For gate/readback work specifically: if code removes, renames, demotes, or marks a field non-blocking, patch every active skill that treats that field as a hard gate before using the old language again. Historical references may keep old artifact values, but active SKILL.md rules must describe the current source/schema contract.
Use this for provider readiness, shell/Python path repairs, TradingViewRemix/TVR, IBKR, Hubble, Kraken/yfinance provider matrix, repo hygiene, routing, and commit discipline.
Commit gate
For public release mirror README-only syncs, use references/release-mirror-readme-only-sync.md: identify the mirror repo, apply the upstream README patch only, prove byte-for-byte parity, handle non-fast-forward by shallow fetch + rebase, push main, and do not touch tags or the dirty development worktree.
Before git commit in ~/projects-ict-engine/ict-engine:
Run git status --short; do not stage unrelated agent work.
If inheriting abandoned multi-agent Board A/B work, follow references/board-ab-handoff-audit.md before staging.
For large evidence trees and multi-agent residue, follow references/commit-slice-hygiene.md: stage by explicit path, prove excluded run trees are not staged, and preserve unrelated untracked work.
Identify the durable lesson:
provider/runtime repair -> this skill or ict-engine-runtime reference
Patch the skill first when the lesson will recur or when a code/schema change makes old skill wording stale.
Add Chinese trigger keywords and index entries when future routing should auto-load it.
Commit only the coherent slice; include evidence path and verification command.
If no skill update is needed, final/summary must say skill_update=not_needed and give the reason. If a code/gate/readback change happened, skill_update=not_needed is only valid after checking current active skills for stale field names.
Provider-ready is only a prerequisite. For data claims, run concrete fetch and record row count.
After repeated IBKR historical ladders return zero rows/timeouts, do not launch another same-shape IBKR stock Gate 1 lane just because provider-status is green. Classify with support/scripts/auto_quant_external/ibkr_provider_guard.py; when cooldown_recommended=true, repair/probe the IBKR historical path or switch to a non-IBKR real-provider cell before new factor work.
Auto-Quant/backtest provider and realtime advice provider must name the same exchange family when the user asks for practical trading use. For crypto public lanes, pair binance_public with binance_public_runtime, or bybit_public with bybit_public_runtime; do not silently train on one exchange and advise from another.
Binance and Bybit public crypto OHLCV are no-key, zero-config lanes in ict-engine; do not classify them as missing just because the Python fetch_external.py optional dependency set is incomplete.
IBKR on this host needs zsh to prefer ~/.local/bin/python3 with redis, ib_async, pandas importable.
For TOMAC clean-AQ reruns, do not assume removing --max-rows on the same root will widen the history window. If a prior bounded clean bundle/AQ workspace already exists on that root, the rerun may reuse it and preserve the bounded window. Use a fresh root when the user asks for a true max-window exact rerun.
For TOMAC prep wrappers that claim to stage a full-window exact rerun, default command generation must omit --max-rows unless the operator explicitly asks for a bounded diagnostic replay. A hardcoded --max-rows 300000 in the wrapper contract silently converts multi-year retained-history work into a 2021-only slice and can create false no_survivor or zero_trade terminalizations. Preserve bounded replay as an explicit --max-rows <N> option, and pair the true full-window relaunch with a fresh root.
For TOMAC wrappers that launch broad source scanners such as tomac_fast_daily_scan.py without a native --families selector, do not trust the raw top rows as branch truth. The wrapper must post-filter scan_results.json / back to the exact rooted branch (for example by prefix or target regime) before claiming branch-specific readback or terminal decisions.
Evidence packet pattern
Create transient lane evidence under /tmp/ict-engine-.../<timestamp>-<owner>-<topic>/.
Repo admission is allowed only for typed product surfaces, tests, reviewed
support examples, or explicit structured evidence packages outside
support/docs.
Minimum files:
checks/provider_status_compact.txt or command output
focused provider JSON outputs when relevant
summaries/terminal_decision_summary.md
See references/auto-quant-closure-evidence-packages.md for the recurring auto-quant import / apply / parity closure shape.
See references/pa-agent-intake-consumer-docs-map.md as historical intake
context only; current external material should land as embedded defaults,
opt-in source/profile fixtures, compact artifact_index.json, privacy scan,
and observation-only trade_usable=false boundaries outside support/docs.
For coordinated objective snapshots, prefer reusing a current heavy done-definition packet with objective_closure_snapshot.py --done-definition-proof <done_definition_audit.json> instead of rerunning heavy gates every lightweight refresh. This proof is valid only when the referenced packet has completion_ready=true, no skipped gates, the same selected head as the live child done-definition audit, and the same tracked_worktree_fingerprint when the live child reports one; with --output-dir, the snapshot stages it as done_definition_proof.compact.json so the evidence packet is self-contained. Partial/skipped/no-head/stale-head/fingerprintless-or-mismatched proof must remain fail-closed and cannot suppress done-definition blockers.
When a valid heavy done-definition proof is applied, the parent done-definition surface must take pass_count, fail_count, skip_count, and total_gates from the proof while preserving the current light child source-debt/quarantine surfaces. Mixed proof status with light-child gate counts is misleading evidence and needs a regression before repair.
For coordinated objective snapshots that are used as release/completion evidence, run with --check-remotes. A snapshot without remote checks must fail closed as release_remote_checks_not_run when release_readiness_audit.py skips remote_readback or release_version_tag_available; otherwise the parent packet can hide source-origin drift or reused release tags behind a surface-green local release audit.
For coordinated objective snapshots from a dirty shared tree, a clean selected-export release audit may be reused with objective_closure_snapshot.py --release-readiness-proof <release_readiness_audit.json> only when the parent snapshot itself uses , the proof was remote-checked, has no skipped gates, and proves . If the proof comes from the release mirror clone, it may have a different commit hash than the development checkout; accept it only when the proof , proof , and current remote are identical and the proof summary is . Do not require or recommend pushing the source origin for mirror-only release closure. Partial/no-remote/stale-mirror proof remains fail-closed.
Do not commit home dotfiles in the repo. Record their paths and exact env keys in the evidence summary.
Red flags
Committing ict-engine after a repair but leaving the maintenance lesson only in chat.
Saying 8/8 ready without naming which provider was missing.
Treating provider-status ready as a successful data fetch.
Updating router files but not skill-index.json / skill-index.md.
Staging broad dirty Board A/B files from other agents.
Using .gitignore as proof that a run tree is excluded. Already tracked files still appear as modified; prove git diff --cached --name-only has no run-tree paths.
Running git add . in ict-engine when evidence runs, local docs, or agent skill drafts are present.
Committing generated evidence run trees before checking untracked file count and byte size.
Claiming Board A achieved universal 95% regime confidence when the artifacts only prove a runnable evidence scaffold.
In Board B rooted-branch handoffs, treating a synthetic regime-bundle branch score as equivalent to a real history/runtime-scored branch. If runtime reuse mode is prefer_history, verify the exact regime_profit_branch_path remains the selected workflow/execution-tree path and that path_ranker_runtime_source is a real source such as history_path, not only a synthetic score.
Treating visible/observation/cache/history ranker evidence as execution-permitted. Execution consumers must distinguish visible=true from used=true, require production validation plus execution_gate_status=pass for execution-gate use, and keep observation/cache/handoff evidence as observe or fail_closed until cost/slippage/provider/sample gates pass. For the profitability lifecycle, legacy ranker values such as execution_gate_status=pass|ready|execution_ready|admissible are not live-trade usability; promotion_allowed, trade_usable, and update_goal require an explicit live plane (live_trade_ready/live_trade_usable) or same-root closed-loop admission with status=admitted, live_trade_status=ready, and all three practical flags true after the current consumer revalidates branch shape. wait_for_reversion is observe-only in the current live plane; only fill_viable may materialize same-root live execution. Policy-training lifecycle ready strings such as , , or must also have mature training evidence (, calibrated label, positive ) before counting readiness. Diagnostic-only tools must not reuse those practical field names for interesting-signal telemetry; use diagnostic names such as and keep downstream live-gate requirements explicit.
For local AQ wrappers derived from run_tomac_nq_local_autoquant_loop_v1.py, do not point non-NQ Databento-style futures CSVs such as YM straight at stage_exact_source_data() without checking the schema. That staging owner expects ts_event,open_adj,high_adj,low_adj,close_adj,volume; raw Databento exports with only open,high,low,close must be normalized first or routed through a clean-bundle staging path, otherwise the launch fails before round 0 with pandas Usecols do not match columns.
TradingViewRemix remote HTTP can be 429; prefer local stdio for OHLCV:
ICT_ENGINE_TRADINGVIEW_MCP_CMD=uv
ICT_ENGINE_TRADINGVIEW_MCP_ARGS='--directory $HOME/tradingview-mcp/tradingview-mcp run tradingview-mcp'
Hubble provider-status needs ICT_ENGINE_HUBBLE_BASE_URL; still validate actual Hubble fetch before using rows as evidence.
--check-remotes
worktree_clean_for_release=pass
head
release_mirror_main
release_mirror_main
pass
givenup-ict-engine
For heavy done-definition smoke evidence in a shared worktree, distinguish a real zero-config CLI hang from cargo lock/build contention. If support/scripts/smoke_acceptance.sh times out with empty command stdout/stderr while many cargo processes are active, reproduce the same smoke/update path with an already-built binary by setting ICT_ENGINE_BIN=/path/to/ict-engine. The smoke harness supports this binary-reuse mode while preserving the default consumer-facing cargo run path.
For coordinated objective snapshots, do not treat a compact done-definition child as clean just because a source-surface gate reports status=pass. practical_admission_source_surface and await_launch_source_surface can pass for tracked release hygiene while still reporting untracked wrapper debt. Objective closure must preserve/read untracked_violation_count; practical-admission debt blocks on practical_admission_source_debt until unsafe scratch wrappers are retired, quarantined, or explicitly tracked, and await-launch debt blocks on await_launch_source_debt until wrappers that can launch with active/fresh claims present are retired, quarantined, or explicitly guarded.
For repo-local factor-training hygiene, done_definition_audit.py includes repo_training_scratch_surface. Non-promoted factor-training scratch docs/scripts, repo-local state/model output, and untracked evidence-run residue must live under /tmp; repo paths are allowed only when tracked or force-added as a durable evidence packet or product surface. A clean worktree with no live-usable factor is not a reason to preserve ignored training residue in the repo.
When an objective-level blocker is driven by many source-scan violations, samples are not enough. The child audit should write a full manifest and objective_closure_snapshot.py --output-dir should stage it into the packet, for example practical_admission_source_debt_manifest.json, so the next agent can retire or quarantine every violating wrapper instead of chasing the first 10 examples.
For reusable factor candidate packs, keep provenance clone-safe and self-describing. A pack directory should carry pack_manifest.json, factor_expression.json, factor_eval_grid_summary.json, and transfer_score.json; generated manifest fields such as source_artifact must not preserve caller-local absolute paths. Product-visible candidate selection remains explicit through the registry/preset layer, not implicit directory scanning.
Do not commit loose, one-off factor wrappers as product-ready work. If stale agent residue contains useful factor logic, convert or quarantine it into the repo's reusable structured format first, such as a candidate pack with pack_manifest.json plus expression/evaluation/transfer artifacts or another established manifest-backed surface. Otherwise leave the loose script unstaged and record the packaging blocker in the handoff.
paper_ready
live_trade_ready
live_trade_usable
maturity_mask
training_weight
diagnostic_candidate_passed_gate
Treating same_tree_practical_closure (or downstream / execution-tree / path-ranker / accepted-feedback / retained-session-scope completion) as a hard promotion gate. It is an OPTIONAL enhanced-lifecycle path, not a promotion prerequisite. Per src/application/factor_lifecycle/profitability_admission.rs (promotion basis good_factor_verified_cost_positive, policy good_factor_verified_cost_positive_downstream_feedback_optional_20260602), a clean-AQ verified-cost-positive good factor — learning-admitted, evidence_count >= 12, no lookahead, validated provenance, verified instrument cost, positive net — is promotion_allowed/trade_usable WITHOUT same-tree closure, accepted paper/live/broker feedback, Pre-Bayes/BBN/execution-tree placement, path-ranker use, or ranker validation. The prior blockers accepted_execution_feedback_missing, execution_readiness_below_live_floor, execution_tree_gate_status_not_ready, execution_tree_branch_not_live_ready, path_ranker_score_not_used_by_execution_tree, ranker_validation_not_ready, and retained_session_scope_unverified are now robustness/longevity debt, not practical vetoes; this matches the ict-engi-fact-rese-muta good-factor/superior-factor standard. Still never advertise a factor as funded-live-trade ready on this advisory CLI.
Accepting a claimedsame_tree_practical_closure packet — produced only as the optional stricter-lifecycle path above, never required for promotion — as valid without the full lifecycle tuple. A valid same_tree_practical_closure must prove the referenced evidence packet is same-root, every command result has exit zero with explicit timed_out=false, and command names cover the provider/data, Pre-Bayes, BBN/workflow, path-ranker, execution-tree, feedback/update, and policy-training stages. It must also prove exact branch survived, execution candidate is actionable, validation counters cover raw/production/observation floors, policy training summary exists, and the lifecycle tuple is exactly learning_admission_status=admitted, paper_admission_status=ready, and live_trade_status=ready. It must also carry deploy_ready=true, funded_live_fill_required=false, and readiness contract deploy_ready_from_backtest_autoquant_provider_or_paper_sim_execution_chain_not_funded_fill at packet and policy-lifecycle level, with positive deploy_ready_count. Raw promotion_allowed=true, trade_usable=true, or a single aggregate command row such as name=all is not enough, and practical closure must not require a funded live fill.
Letting compact audit formatting hide pass-gate debt. If a pass gate carries warning/debt fields that matter for objective closure, compact output must preserve the minimal fields needed by the parent snapshot; otherwise closure packets can look surface-green while the current worktree still contains unsafe practical-admission code.
Letting parent objective snapshots name factor_closure_blocked without reusable blocker detail. The parent packet must preserve compact factor-closure counts, coordination-only count, invalid active count, live process count, attention groups, action queue roots/claims, blocking reasons, and next action so the packet stays actionable without opening the child factor audit.
Letting parent objective snapshots name release_readiness_blocked without reusable blocker detail. The parent packet must preserve release head, unresolved gates, pass/fail/skip counts, release next actions, skipped remote gates, proof rejection fields when present, and compact remote readback status such as origin vs release-mirror failure so operators can act without opening the child release audit.
Letting parent objective snapshots name done_definition_not_completion_ready without reusable blocker detail. The parent packet must preserve done-definition head, tracked worktree fingerprint, completion/evidence status, pass/fail/skip/total gate counts, unresolved and skipped gates, next action, quickstart status, and proof rejection fields when present so operators can see whether the blocker is heavy-gate coverage, failing gates, stale proof, or source-surface debt without opening the child done-definition audit.
Letting await-launch wrappers use live_factor_processes=0 as the only launch guard. Await-launch wrappers that call child prep/launch code must also fail closed on active/fresh claim counters from factor_claim_terminalization_audit.py; otherwise a fresh claim without a live process can be bypassed and a partial collision launch can start. done_definition_audit.py surfaces this as await_launch_source_surface; objective snapshots must preserve pass-state debt and block on await_launch_source_debt when it is not retired/quarantined/tracked.
Reporting a debt blocker without a full manifest. Counts plus sample rows can prove non-completion, but they cannot drive systematic closure of all violating wrappers. Stage a complete manifest into the objective packet when the blocker names a many-file source debt surface.
Keeping retired or non-blocking gate fields as active hard blockers in skills. Before repeating an older blocker name, inspect current source/readback semantics and update the skill when the contract drifted.
When a user asks whether an ict-engine profitability factor is "实战"/live-usable and a readiness floor changes, audit both repo consumers and active runtime skills for stale threshold language before committing. For the current profitability loop, execution_readiness >= 0.45 is the closed-loop return-to-duty/live-plane admission floor for cost-positive same-root candidates; >= 0.65 is only the stronger execution_ready class. Preserve 0.65 in historical references only when it is clearly framed as strong-ready, not as a hard exile rule.
Same-root execution-tree materialization in src/analyze_shared.rs must reuse ict_engine::application::factor_lifecycle::LIVE_EXECUTION_READINESS_FLOOR; do not hardcode 0.65 there. Regression should prove execution_readiness=0.45 with branch=fill_viable, used+validated path ranker, admitted closed-loop flags, and even high hybrid_transition_hazard still materializes an actionable same-root candidate. transition_hazard may remain telemetry/source-scan debt, but must not re-enter branch_local_admitted, promotion_allowed, trade_usable, or update_goal as a hard live gate.
Readiness-floor audits must include non-lifecycle consumers that can silently re-exile same-root candidates after the core lifecycle passes. In particular, inspect src/analyze_shared.rs same-root execution-tree admission (actionable_admission) and strict trace fallback paths: 0.65 there is only valid for strong-ready classification, not as the live-plane floor. Add or run a regression proving execution_readiness=0.45 still permits a cost-positive same-root closed-loop candidate to remain actionable when all other live-plane gates pass.
Treating release mirror readback as a generic network failure when an HTTPS GitHub URL reports SSH-style port 22 errors. Preserve no-rewrite HTTPS fallback diagnostics for both source origin and release mirror probes so operators can distinguish GitHub reachability from local git URL rewrite / SSH transport drift. A fallback named https_public_no_rewrite must actually neutralize local git rewrite config, for example by running with GIT_CONFIG_GLOBAL=/dev/null and GIT_CONFIG_NOSYSTEM=1; otherwise url.*.insteadof can silently turn the HTTPS fallback back into SSH and create false remote_readback blockers.
Letting release signoff and release notes drift across tags. release_readiness_audit.py should parse the signoff Selected candidate, release-notes Version, and current Cargo package version; signoff/notes tag mismatch or Cargo-version drift must fail release_docs_fresh_for_selected_tag before publishing. Do not use current HEAD as an implicit signoff source-commit equality check because committing refreshed signoff docs changes HEAD; source alignment stays under source_origin_matches_selected_source and the clean selected export flow.
Counting command-introspection probes or zero-config smoke children as live factor owners. run_tomac.py --help, python -m unittest ...run_tomac..., grep/sed/rg/find readbacks, ps/awk process pollers, exit-file cat/Python-heredoc readbacks, TOMAC diagnostic probes, and direct ict-engine demo/DEMO smoke commands under ict-engine-done-definition-audit-smoke, ict-engine-smoke-acceptance, or ict-engine-first-run are not provider/AQ writers and must be filtered before declaring live_factor_processes > 0. Add regression tests to support/scripts/tests/test_factor_claim_terminalization_audit.py when a new probe shape appears.
Counting zero-config consumer smoke as live factor ownership. ict-engine analyze|workflow-status|pre-bayes-status|policy-training-status --symbol DEMO --demo/--state-dir /tmp/ict-engine-*-smoke* is first-run or done-definition evidence, not Board B profitability runtime. Filter these smoke/readback commands before declaring factor closure blocked, while keeping real TOMAC/factor lifecycle commands under Board B run roots classified as live. Compact factor action queues should preserve a short command_excerpt for live runtime rows so parent objective packets remain actionable without opening the child audit.
Missing direct ict-engine factor lifecycle commands in live-process occupancy. cargo run --quiet -- factor-research ... --state-dir /tmp/ict-engine-* and direct ict-engine factor-research / factor-autoresearch / auto-quant-agent-material* commands writing under a Board B /tmp root are live factor/runtime work, even when they are not named run_tomac or fetch_external.py. Keep ps|rg, unittest, and help probes ignored, but add a focused classifier regression when a real factor CLI shape is missed.
Missing generic Python factor scripts in live-process occupancy. A Python command that runs a script from a Board B /tmp/ict-engine-* lane root, for example /tmp/ict-engine-.../scripts/run_*_pybacktest.py, is live factor/runtime work even when it is not named run_tomac, run_ibkr_*, or fetch_external.py. Keep help, unittest, grep/sed/rg, and TOMAC diagnostic probes ignored, but classify generic .py scripts with an extracted Board B run root as live and cover new shapes with support/scripts/tests/test_factor_claim_terminalization_audit.py.
Letting shell wrapper variables pollute live-process run roots. Parent commands such as root=/tmp/ict-engine-... followed by --root "$root/run" must resolve the simple shell assignment, and unresolved $root/... tokens must be ignored rather than surfaced as literal run roots. macOS ps may render wrapper newlines as \\012; normalize that before extracting assignments so action queues do not report roots like /tmp/ict-engine-...\\012python3.
Collapsing all active claims into the same cleanup action. Fresh active claims without a live process are wait/inspect targets until stale-safe timeout or terminal evidence exists; fresh wait-only prep claims are also wait targets; stale-safe wait-only claims are cleanup/externalization targets; live-runtime-owned active claims belong under live-runtime wait actions, not generic terminalize or externalize active claims wording.
Treating every workdoc Decision: field as terminal evidence. Workdocs can contain workflow sections such as ## TDD Route with Decision: skipped, strict, light, auto, or off; those are process-route decisions, not factor terminalization. factor_claim_terminalization_audit.py should terminalize workdocs from terminal/final sections, explicit terminal_* fields, terminal-looking status, or terminal-looking decisions only, and any new nonterminal workflow section needs a focused regression test.
Treating every terminalized missing run_root as live evidence debt. A terminalized duplicate/no-launch/collision-deferred claim with explicit false promotion_allowed and trade_usable may intentionally have no AQ root because no launch produced factor economics; it should not block closure as missing_run_roots. Active missing roots and terminalized claims with positive practical flags still require attention, restoration, or explicit evidence review.
Treating /tmp and /private/tmp run roots as different Board B owners on macOS. Claim/live-runtime matching must normalize tmp aliases and common lane subdirs before deciding a claim is fresh_active_without_live_process; otherwise the same live AQ lane can appear twice in the parent action queue.
Applying a heavy done-definition proof by replacing the current light child surface. Proof reuse may clear full-gate coverage only; it must preserve current practical-source debt/quarantine details from the same packet, or parent snapshots can hide untracked wrapper residue.
Letting objective-level audit wrapper timeouts kill only the direct child. objective_closure_snapshot.py and similar parent wrappers must kill the full child process group on timeout, or broad child scanners such as downstream_practical_admission_source_check.py can survive, consume shared resources, and distort later claim/process readbacks.
Flattening practical-source scanner failures into generic source-debt counts. If downstream_practical_admission_source_check.py times out, returns non-zero without JSON violations, or otherwise fails as tooling, done_definition_audit.py and objective_closure_snapshot.py must preserve scanner command, error/timeout, return code, stdout, and stderr so parent packets distinguish tooling failure from a clean source surface.
Reporting remote_readback as a release-mirror repair when only source origin failed and the release mirror is readable. Release audits should preserve the failed remote side and point the operator at origin vs release mirror specifically.
Letting audit-only, coordination-only, or inventory-only claims block factor closure. A claim with explicit active_audit_only / active_coordination_only / active_inventory status, false practical flags, and no-launch/read-only or artifact-scan language is bookkeeping, not an active profitability factor owner. It should remain visible as coordination_only_active_claims but must not pollute compact factor action queues or practical closure blockers.
Fingerprinting untracked practical-admission source debt with incidental line/column positions. Quarantine matching should use the stable violation signature (file, key, value, violation) so scratch-wrapper line churn does not create false unquarantined debt. If shared untracked wrapper residue oscillates between reviewed stable signatures for the same count/file set, record the alternate digest under reviewed_alternative_untracked_violations_sha256; do not replace count/file/schema/decision checks, and do not treat the quarantine as release or trade evidence.
