deepscientist-optimize

> Codex adapter note: this stage skill is bundled for the DeepScientist Codex native plugin. Use `scripts/dsctl.py call <ds_tool_name> --json ... --format json` for DeepScientist state instead of unavailable Hermes/MCP tool calls. Do not use MCP transport or the external `ds` command. Runtime state lives under `<project>/DeepScientist/`. # Optimize Use this skill for algorithm-first quests where the goal is the strongest justified optimization result rather than paper packaging. This skill is the lightweight optimization control layer for DeepScientist. It does not replace the normal quest runtime. It tells you how to use the existing DeepScientist artifact, memory, ds_bash_exec, Git, and worktree mechanisms as an optimization system. ## Interaction discipline - Follow the shared interaction contract injected by the system prompt. - For ordinary active work, prefer a concise progress update once work has crossed roughly 6 tool calls with a human-meaningful delta, and do not drift beyond roughly 12 tool calls or about 8 minutes without a user-visible update. - Ordinary candidate creation, smoke checks, and route updates should stay concise. - Use richer milestone updates only when a candidate is promoted, a strong run finishes, the frontier shifts materially, or a fusion/debug route becomes the new main path. - When the user asks for the current optimization state, answer from the frontier and durable artifacts rather than from chat memory. - Codex-native execution boundary: use Codex-native tools for routine file inspection, Git mechanics, dependency checks, smoke checks, quick validations, tests, and process management. Use `ds_bash_exec` for optimization runs whose logs must become DeepScientist quest evidence, especially formal evaluations, long runs, or frontier-shifting candidate evidence. ## Stage purpose The optimize stage should do four things: 1. turn loose ideas into candidate briefs 2. rank and promote only the strongest briefs into durable lines 3. manage candidate attempts within a durable line 4. choose when to explore, exploit, fuse, debug, or stop This skill is especially appropriate when `startup_contract.need_research_paper = false`. Treat `optimize` as one stable stage skill with six internal submodes: - `brief` - `rank` - `seed` - `loop` - `fusion` - `debug` Do not treat these as separate public skills. Treat them as internal execution modes inside one optimize workflow. InternAgent maps most naturally onto the `brief` and `rank` side of this stage. MLEvolve maps most naturally onto the `seed`, `loop`, `fusion`, and `debug` side of this stage. Do not collapse those two layers into one vague "optimize more" loop. ## Required working files Before broad optimization search or candidate management becomes substantial, maintain these quest-visible control files: - `OPTIMIZE_CHECKLIST.md` - `CANDIDATE_BOARD.md` Use: - the integrated `optimize checklist template` appendix section - the integrated `candidate board template` appendix section `OPTIMIZE_CHECKLIST.md` is the execution control surface. It should track: - current frontier mode - current optimize submode - candidate brief count - promoted line count - current smoke queue - current full-eval queue - stagnation / fusion checks - next concrete action `CANDIDATE_BOARD.md` is the compact candidate ledger. It should track: - candidate id - candidate type: brief or implementation attempt - parent line or parent candidate - strategy: explore / exploit / fusion / debug - status - expected gain - observed result - promote / archive recommendation ## Required native dsctl-driven workflow Treat this as the concrete optimize workflow. Do not skip these steps just because the quest is algorithm-first. ### 1. Recover the optimization state first At the start of each meaningful optimize pass, use this order unless a stronger local reason exists: 1. `ds_artifact_record / analysis artifact lookup(...)` 2. `ds_memory_search(scope='quest', limit=5)` 3. `ds_memory_search(...)` 4. `artifact.get_quest_state(detail='summary')` 5. `artifact.read_quest_documents(...)` when exact durable wording matters Do not create new candidates before the frontier, recent optimization lessons, and current runtime refs are checked. If the frontier is missing or obviously stale, recover that state before proposing more work. ### 2. Shape candidate briefs before branch promotion When the next direction is still fuzzy, do not jump straight into code or branch creation. First turn the direction into a compact candidate brief. The brief-shaping sequence is: 1. clarify the bottleneck, constraints, and comparability boundary 2. identify the incumbent or baseline that this brief must beat or complement 3. generate a small differentiated slate, usually `2-3` serious approaches 4. compare them on one shared surface 5. recommend exactly one lead brief 6. self-check the recommended brief before submission Every serious brief should answer: - bottleneck - why_current_line_is_limited - mechanism - why_now - keep_unchanged - expected_gain - implementation_surface - main_risks The durable call for this step is usually: - `artifact.submit_idea(mode='create', submission_mode='candidate', ...)` Use `idea` when the mechanism family itself is still unresolved. Use `optimize` when the family is already chosen and the work is now branchless brief shaping, ranking, or within-line search. ### 3. Rank candidate briefs on one explicit surface Before promoting a line, compare the serious briefs on one shared ranking surface. At minimum evaluate: - expected information gain - feasibility in current repo - comparability against baseline - implementation surface - novelty or distinctiveness - family diversity - change-layer diversity - incumbent-improvement potential - failure risk Then state: - winner justification - non-winner defer / reject reasons - promotion cap: how many lines should actually be promoted now Do not promote every plausible brief. Default rule: promote only `1-3` candidate briefs, and usually fewer. The durable call for this step is one of: - `artifact.submit_idea(mode='create', submission_mode='line', source_candidate_id=..., ...)` - `ds_artifact_record payload={'kind': 'decision', 'action': 'branch'|'continue'|'stop', ...})` ### 4. Hand off promoted lines into experiment cleanly Once a brief is promoted, the next main work belongs to `experiment`, not to vague optimize chatter. Before substantial implementation or compute: - activate or confirm the intended durable line - update `OPTIMIZE_CHECKLIST.md` - update `CANDIDATE_BOARD.md` - create or revise `PLAN.md` - create or revise `CHECKLIST.md` - define the smoke queue and full-eval queue explicitly Then hand off into `experiment` for: - one clean implementation pass - one bounded smoke or pilot run - one real measured main run Do not keep reshaping the method after the run contract is already concrete. ### 5. Record every meaningful result durably Use these artifact forms consistently: - candidate brief: - `artifact.submit_idea(..., submission_mode='candidate')` - durable optimization line: - `artifact.submit_idea(..., submission_mode='line')` - implementation-level candidate attempt inside one line: - `ds_artifact_record payload={'kind': 'report', 'report_type': 'optimization_candidate', ...})` - real measured main result: - `ds_artifact_record_main_experiment(...)` - route change after the result: - `ds_artifact_record payload={'kind': 'decision', 'action': 'iterate'|'branch'|'continue'|'stop', ...})` Do not treat chat summaries as substitutes for these durable records. ### 6. Manage process lifecycle explicitly Optimize uses the same long-run process discipline as `experiment`. - Use Codex-native execution for routine smoke checks and quick validations; use `ds_bash_exec` for long or formal optimization runs that need durable quest-local provenance. - Before launching a new run, inspect current managed sessions first. - Do not start a duplicate process for the same purpose if a valid live session already exists. - Use bounded smoke before long runs unless direct quick validation is already cheap and equally informative. - Use `ds_bash_exec mode='detach', ...)` for long runs and monitor with `list/read/await`. - Read logs before retrying a failed or suspicious run; do not relaunch blindly. - Kill only on explicit invalidity, supersession, or checked no-progress conditions. - After pause, resume, or runtime recovery, recover session state before spawning new runs. ### 7. Route from evidence, not from momentum After every real measured result: 1. refresh the frontier 2. compare the result against the incumbent and backlog 3. choose exactly one dominant next action: - explore - exploit - fusion - debug - stop 4. record that route durably Do not treat one candidate creation, one smoke pass, or one detached launch as stage completion. ## Integrated templates and playbooks Use the following integrated structures directly inside this skill. They replace the old optimize reference files conceptually, even if those files still exist on disk. ### Candidate brief template Every serious candidate brief should include: - title - bottleneck - why_current_line_is_limited - mechanism - mechanism_family - change_layer: `Tier1` / `Tier2` / `Tier3` - source_lens - keep_unchanged - expected_gain - implementation_surface - risks - foundation - promote_now - next_target ### Brief-shaping playbook Use this when a candidate direction is still fuzzy and needs to become a ranking-ready brief. - clarify the concrete bottleneck before widening - resolve the evaluation or comparability boundary - identify the main hard constraint - identify the current incumbent - generate only a small differentiated slate - compare on one shared surface - recommend exactly one lead brief - self-check for ambiguity, overlap, and weak justification ### Candidate ranking template When several briefs compete, produce: - candidate set - ranking scope - comparison surface - ranked candidates with score summary, why each ranks there, and promote / hold / reject - winner justification - non-winner notes - promotion cap ### Candidate board template `CANDIDATE_BOARD.md` should expose at least these columns: - candidate id - level: `brief` or `implementation` - parent - strategy - status - expected gain - observed result - promote / archive recommendation ### Optimize checklist template `OPTIMIZE_CHECKLIST.md` should track at least: - frontier has been refreshed - primary optimize submode chosen - current route mode chosen - recent optimization memory reviewed - brief slate checked for family diversity - candidate briefs updated or confirmed - candidate ranking updated - promotion decision made - current implementation pool recorded - smoke queue defined - full-eval queue defined - failures classified - stagnation check performed - fusion eligibility checked - next concrete action written ### Frontier review template Whenever route choice is unclear, write down: - current frontier - evidence summary - route choice - active optimize submode - immediate next action ### Code-generation route playbook Choose one route deliberately: - brief-only when the direction is still unclear - stepwise generation for first substantial implementation of a new line - diff / patch generation for improve / exploit / debug / most fusion work - full rewrite only when the current implementation is structurally broken or mismatched Do not jump to a rewrite merely because one local patch failed. ### Debug response template When a candidate fails but still looks strategically valuable, record: - error - retrieved memory - root cause - minimal fix - keep unchanged - next check - archive threshold ### Fusion playbook Before opening a fusion candidate, answer: - what exactly is being fused? - why are the source strengths complementary rather than redundant? - what remains unchanged for comparability? - what bounded evidence would prove the fusion worthwhile? - what bounded first validation step should run before any broad rollout? Do not fuse two weak lines or two same-mechanism lines under different names. ### Optimization memory template When writing reusable optimization lessons, capture: - type - context - observation - why it matters - retrieval hint - reuse hint ### Plateau response playbook If one line keeps producing non-improving results: 1. state that the line is plateauing 2. identify the most likely root cause 3. choose one larger route change: - widen search - promote a stronger alternative - fuse - debug - stop 4. record one explicit non-repeat rule Do not hide plateau under a sequence of tiny "one more tweak" loops. ### Prompt patterns worth preserving For candidate-brief, improve, fusion, and debug prompts, preserve: - introduction - task description - memory - previous solution or previous line - instructions - explicit response format Preserve these reasoning contracts whenever possible: - WHAT is changing? - WHY is the current line limited? - HOW should the change address the limitation? - KEEP UNCHANGED - NEXT ACTION ## Non-negotiable rules - Do not treat every patch or micro-attempt as a new durable idea line. - Do not create a new Git branch/worktree for every implementation-level candidate. - Use `artifact.submit_idea(..., submission_mode='candidate')` for candidate briefs that should be ranked before promotion. - Use `artifact.submit_idea(..., submission_mode='line')` only for directions that deserve a durable optimization line and branch/worktree. - Use `ds_artifact_record payload={'kind': 'report', 'report_type': 'optimization_candidate', ...})` for implementation-level candidate attempts inside one durable line. - Before deciding the next route, call `ds_artifact_record / analysis artifact lookup(...)` when available and use it as the primary optimization-state summary. - Keep all major optimization successes and failures durable through artifacts and memory. - Do not drift into paper-outline, bundle, or finalize work by default while this stage is active. - Do not convert ranking uncertainty into premature branch creation. - Do not treat an implementation-level candidate report as a new durable optimization line. - Do not keep widening the frontier once a small serious slate already exists. - Do not let one optimize pass mix multiple major route changes. One pass may inspect several possibilities, but it should finish with one dominant next action. ## When to use - the quest is algorithm-first - the baseline gate is already confirmed or waived - the task has at least one plausible optimization direction - multiple candidate directions exist and the system should rank them before promotion - a durable line exists and the next step is to manage explore / exploit / fuse / debug ## Do not use when - the baseline gate is unresolved - the main need is a paper draft, rebuttal, or review task - the quest is still in broad literature scouting with no concrete optimization handle ## Core object model Use these three object levels consistently: 1. candidate brief `artifact.submit_idea(mode='create', submission_mode='candidate', ...)` This records a possible direction or method brief without opening a branch yet. 2. durable optimization line `artifact.submit_idea(mode='create', submission_mode='line', ...)` This opens a real branch/worktree and becomes a formal optimization path. 3. implementation-level candidate attempt `ds_artifact_record payload={'kind': 'report', 'report_type': 'optimization_candidate', ...})` This is a within-line attempt such as one patch, one smoke candidate, one debug candidate, or one fusion candidate. ## Recommended workflow 1. Read the current frontier and recent durable state. 2. If only loose candidate directions exist, create or refine candidate briefs first. 3. Rank the candidate briefs and promote only the best `1-3` into durable lines. 4. Inside a durable line, generate a small candidate pool, then run bounded smoke checks before full evaluations. 5. Record each implementation-level attempt durably with status, change plan, and result. 6. After each real result, decide whether to explore, exploit, fuse, debug, or stop. 7. Write optimization lessons to memory before leaving the stage. At the start of each meaningful optimize pass, update `OPTIMIZE_CHECKLIST.md` before spending significant code or compute. ## Mandatory first-call sequence At the start of a meaningful optimize pass, use this order unless a stronger local reason exists: 1. `ds_artifact_record / analysis artifact lookup(...)` 2. `ds_memory_search(...)` 3. `artifact.get_quest_state(detail='summary')` 4. `artifact.read_quest_documents(...)` when exact durable wording matters Do not start generating new candidates before the frontier and recent optimization lessons are checked. ## Stage-start requirement Stage-start requirement: - run `ds_memory_search(scope='quest', limit=5)` - run at least one `ds_memory_search(...)` - read `ds_artifact_record / analysis artifact lookup(...)` - update `OPTIMIZE_CHECKLIST.md` If the frontier is missing or obviously stale, recover that state before proposing more work. ## Internal submode selection Choose exactly one primary optimize submode for the current meaningful pass. Default selection order: 1. `fusion` - when the frontier explicitly says `fusion` 2. `debug` - when a strategically valuable candidate failed for a concrete and likely fixable reason 3. `rank` - when several candidate briefs already exist and promotion is the main unresolved question 4. `brief` - when the candidate-brief slate is too thin or too weak 5. `seed` - when a durable line exists but there is no live implementation-candidate pool 6. `loop` - when a live candidate pool or leading durable line already exists and the main need is bounded execution progress Do not bounce among submodes repeatedly in one pass. If the best submode changes after new evidence appears, record that route shift explicitly. ## Candidate brief protocol When a direction is interesting but not yet worthy of a new branch: - create a candidate brief with `submission_mode='candidate'` - keep it branchless - record enough structure that later ranking or promotion is possible Good candidate-brief fields include: - title - problem - hypothesis - mechanism - mechanism_family - change_layer - source_lens - expected_gain - risks - decision_reason - foundation_ref - lineage_intent Do not promote every candidate automatically. Use the integrated `method brief template` section for the minimum acceptable candidate-brief structure. Use the integrated `brief shaping playbook` section when the brief is still too vague, too implementation-first, or too collapsed onto one familiar mechanism. Candidate briefs should explicitly answer: - WHAT bottleneck is being targeted? - WHY is the current line limited? - HOW does this mechanism address the limitation? - WHAT must remain unchanged for comparability? If the brief cannot answer those four questions clearly, it is not ready for promotion or implementation. Treat a candidate brief as the DeepScientist form of a method brief. It should sit between "idea intuition" and "code implementation". Preserve this brief-shaping discipline: 1. clarify the bottleneck, constraints, and comparability boundary first 2. generate a small differentiated slate, usually `2-3` serious approaches 3. recommend one approach with explicit tradeoffs against the alternatives 4. self-check the winning brief for ambiguity, overlap, and weak justification before submission Do not jump from "interesting intuition" to branch creation. Do not jump from "I know how to code this" to "this deserves promotion." When running the `brief` submode: - produce only `2-4` serious candidate briefs by default - ask or answer the minimum clarifying questions needed to remove ambiguity around bottleneck, constraint fit, and comparability - explicitly keep one incumbent-compatible refinement when possible - explicitly keep one orthogonal alternative when possible - explicitly keep one broader lens or paradigm shift candidate when possible - avoid generating several renamed variants of the same mechanism - prefer mechanism-level distinctness over volume - present the differentiated slate on one shared comparison surface before choosing a recommended brief - keep the questioning bounded and execution-oriented rather than open-ended brainstorming Use a coverage contract for every serious brief slate: - one `incumbent-deepening` direction when justified - one `orthogonal-mechanism` direction when justified - one `paradigm/objective/data-view shift` direction when justified If all serious briefs belong to the same mechanism family, do one widening pass before ranking. Do not treat a same-family slate as sufficient merely because the local scores look good. For each serious brief, record at least: - bottleneck - why_current_line_is_limited - mechanism - why_now - mechanism_family - change_layer: `Tier1` / `Tier2` / `Tier3` - source_lens - keep_unchanged - expected_gain - implementation_surface - main_risks