praxist-task-initialization

name

praxist-task-initialization

description

Convert an existing runnable computer-based research project into a formal Praxist task project, or repair a task harness that fails task-init validation. Use when a user wants an agent to transform AI algorithm, robotics, control, simulation, SLAM, LLM, optimization, or other executable research code into a Praxist task directory with task.yaml, baseline harness, evaluator, baseline performance records, robust metric/ranking policy, protocol-integrity checks, reachable task-justified durable/Pareto retention lanes, role prompts, audit rules, dataset/simulator metadata, high-value research directions, initial-generation DIG plus independently controlled QD, continuous-evolution/Gems research-loop settings, run-report tooling, and hardware-aware or user-selected fixed Praxist run parameters. The skill requires a project that already runs on the current machine or in an available environment/container. Abort when required code, data/simulator assets, or declared runtime dependencies are missing.

Praxist Task Initialization

Use this skill to turn a user's existing runnable research project into a Praxist task project. The output is a new task directory selected later with praxist start --task-path ...; do not edit Praxist core or generic plugins.

Mandatory Opening Banner

Before scanning or editing, display this prominently:

**IMPORTANT PRECONDITION**
Praxist task initialization assumes:
1. You already have all source code for the research project.
2. The project already runs smoothly on this machine in some environment/container.
3. Required datasets, simulators, benchmark fixtures, or control environments are already present or reachable from this machine.

If any item is false, stop and ask the user to provide the missing path, environment, dataset, or simulator before building the task.

When praxist-takeover, praxist-takeover-codex, or the user invokes this skill to repair an existing task that fails a required task-init check, preserve its established scientific objective, evaluator protocol, metric directions, baseline, and runtime ownership. Limit edits to the task harness components responsible for the failed contract, rerun the same validations, and avoid rebuilding unrelated task files. Ask the user before changing scientific selection semantics that cannot be derived unambiguously from the existing task and project evidence.

Inputs

Resolve paths in this order:

1. User-provided research project path.
2. Current agent working directory.

Default output task path is <research_project>/praxist_task unless the user provides another output path. Do not mutate the original project except to read it and, if explicitly requested, run its documented smoke/evaluation commands.

Locate Task Templates

Resolve the template root before reading or copying a scaffold. Do not assume a source checkout exists after pip installation:

1. In a source checkout, use <praxist-repo>/templates/tasks/.

2. Otherwise locate the installed package with importlib.resources and use <praxist-package>/resources/templates/tasks/:

   python - <<'PY'
   from importlib.resources import files
   
   print(files("praxist").joinpath("resources", "templates", "tasks"))
   PY
   

Call the resulting directory TASK_TEMPLATE_ROOT. References below such as templates/tasks/machine_learning_template name the source-checkout form; an installed-only session must read the equivalent directory under TASK_TEMPLATE_ROOT.

Do not use examples/ as the generic scaffold. Complete examples preserve task-specific code, metrics, evidence, and resource assumptions. If the user selects a bundled example, operate only on the writable copy produced by praxist examples install; never create or modify a harness inside Praxist source or package resources.

User-Owned Protocol Intent

Protocol strictness is not universal. Before designing the evaluator, maturity policy, lanes, or close gate, derive one task-owned protocol-intent table from:

1. the user's explicit current instruction;
2. an existing project protocol that does not conflict with that instruction;
3. a scientifically defensible agent proposal only where neither source decides.

The table must list each allowed evaluator mode/stage and whether it may launch, produce a comparable score, rank candidates, count as mature evidence, supply a durable parent, or satisfy generation close. Keep the table in an existing task-facing description or protocol document and encode the same decisions in the evaluator summary, evaluation.maturity_policy, frontier lanes, Gems, and close settings. Do not create a second runtime fact registry for it.

There is no Praxist-wide full-protocol-only rule. If the user explicitly requests a partial, reduced-coverage, scout, diagnostic, or otherwise incomplete protocol for ranking, promotion, or the whole run, preserve that choice, expose its actual effort/coverage and stage in every result, and align all downstream policies with it. Do not silently upgrade it to a complete protocol, refuse it merely for being incomplete, or repeatedly ask the user to reverse an already clear decision. If the user has not decided, recommend a complete mature protocol plus clearly labeled cheaper signals when justified, but present that as a proposal rather than a system requirement.

Keep existing schema names for compatibility: complete_stage_labels means "task-declared mature labels," not "globally full-protocol labels." Likewise, scored_complete means the result satisfies the task's declared comparable evidence contract; it does not erase a reduced mode's actual metadata.

Make each canonical evaluator summary internally coherent under the task-owned policy. Do not infer incompleteness from a token such as capped alone: a fixed-budget task may explicitly define reaching its configured cap as mature completion. Instead, verify that completion booleans, status fields, achieved effort/coverage, and the declared policy produce one unambiguous decision. A premature stop must remain incomplete; a successfully reached terminal budget may retain a truthful cap descriptor when the task contract clearly authorizes it. Exercise both cases through the evaluator's real summary writer in a task-local regression; never repair a genuine conflict by weakening the maturity policy.

Only undeclared drift is a protocol-integrity failure. A run whose actual mode, coverage, or effort differs from the task-owned table must remain visible as a validation/diagnostic signal and must not impersonate a more mature class. Validate structured evaluator mode and output metadata, not command keywords: never reject a launch merely because an argv, path, or source file contains words such as smoke, scout, or partial.

Abort Conditions

Abort instead of producing a weak task when clear evidence shows:

• necessary source code is missing, e.g. evaluator exists but model/training/simulator interface is absent;
• required dataset, checkpoint, simulator, robot environment, or benchmark fixture is missing;
• the user specified a runtime environment/container and it lacks required imports, binaries, or simulator bindings;
• no executable baseline or no credible evaluation path can be identified.

Report: what is missing, why it blocks Praxist, and whether the user should supply a path, install dependencies, mount data, or name the correct environment.

Scan Workflow

1. Announce the precondition banner.

2. Identify the project root and output task path.

3. Run the bundled inventory helper:

   python skills/praxist-task-initialization/scripts/project_inventory.py \
     --root /path/to/research-project \
     --out /path/to/output-task/assets/project_scan
   

4. Read the generated inventory.json and inventory_summary.md.

5. Deep-read the project material needed to understand the task:

   • README, docs, design notes, papers, markdown, notebooks, configs.
   • PDFs and reports; extract text when available and inspect figures/tables when relevant.
   • Presentation files or report documents when they explain methods/results.
   • Code for training, inference, model definition, simulator/control loop, evaluation, metrics, and data loading.
   • Dataset manifests, schemas, resolvers, loader code, and non-secret path variables.
   • Simulator adapters and documented simulator launch commands.
   • Existing result/log directories; summarize text logs and structured summaries. For large history, split by directory/time range and use subagents, then merge into one evidence summary.

6. Detect runtime environment:

   • conda envs: environment.yml, conda.yaml, shell docs.
   • venv/uv/poetry/pip: .venv, pyproject.toml, requirements.txt, uv.lock, poetry.lock.
   • containers: Dockerfile, compose.yaml, launch scripts.
   • external runtimes: ROS, MuJoCo, Isaac, Gymnasium, MATLAB, database services, or task-local services.

7. Verify the declared or inferred environment with the lightest safe command: import check, --help, unit smoke, dry-run, tiny fixture, or simulator startup check. Do not run a long training or evaluation sweep unless the user asks.

Hardware And Run-Parameter Planning

Do this before writing task.yaml. Praxist task initialization must produce an execution plan, not just static task files.

Default path: estimate parameters from hardware and task cost. If the user explicitly requests a fixed default profile, a no-load-measurement profile, or wording equivalent to "skip sizing and use default parameters", do not derive cohort size and duration from current utilization. Use this fixed research profile instead and document that it is user-selected rather than measured:

generation_policy:
  cohort_size: 8
  max_generations: 20
  per_generation_hours: 2.0   # 120 minutes

The fixed profile is an operator convenience, not the normal recommendation. Still verify that the project can run and that required data/simulator/runtime assets exist; only skip load-derived sizing.

1. Inspect current hardware with bounded, read-only commands:

   • CPU: use a platform-appropriate summary such as lscpu, /proc/cpuinfo, or sysctl.
   • Memory: use free -h, vm_stat, sysctl, or an available portable library. Treat macOS unified-memory execution and other unified-memory platforms as normal paths and account for their shared capacity.
   • Accelerators: first infer the backend actually used by the unchanged baseline, then use a matching read-only telemetry source when available. Record backend, device count, capacity, and current load without assuming a vendor, discrete GPU, CUDA, or any concrete device model. Absence of a vendor utility is not an error and does not imply absence of acceleration.
   • Disk and I/O risk: df -h for relevant project/data/output mounts; avoid full-disk scans.

2. Observe the unchanged public baseline program under its normal runtime. Prefer an existing smoke, one normal evaluation unit, or the shortest representative complete-protocol unit. Do not write synthetic CPU-only and GPU-only competitors, do not move baseline computation between devices, and do not change data loading, worker counts, model code, precision, or accelerator visibility merely to classify the bottleneck. Sample external CPU utilization/load, RAM, I/O pressure, observable accelerator memory/utilization, elapsed time, and completion progress while the original command runs. Stop after a representative observation if continuing would be expensive. If a tiny preliminary check does not create representative pressure, use one larger unchanged protocol unit and terminate it after the resource shape is clear; never publish a terminated probe as performance evidence. Resource observation must cover the process lifetime, not use one snapshot. Start the sampler before launching the unchanged baseline child, record timestamped accelerator utilization and memory every 100-200 ms when the active backend exposes trustworthy telemetry, and stop only after the child exits. Sample CPU/RAM/I/O at a similarly bounded cadence. For workloads shorter than roughly five seconds, use at least three safe repetitions of the exact same workload or select a longer unchanged representative unit so startup/teardown cannot dominate the samples. Do not concatenate repetitions into a different scientific protocol. If neither is safe, record accelerator demand as unknown. Record the baseline process tree and backend-visible compute PIDs when available so unrelated host jobs are not attributed to the task. When attribution is ambiguous, postpone calibration or keep the profile unknown/exclusive.

3. Classify the observed pressure, allowing more than one domain and an explicit unknown result:

Optional Managed-Accelerator Handoff Contract

Apply this section only when the unchanged baseline uses a discrete accelerator backend that Praxist can bind through the following public environment contract. Do not generate these variables for CPU-only, task-managed, unified-memory, or other backends merely because a device exists. For a compatible evaluator, wrapper, trainer launcher, worker launcher, shell bridge, or container bridge:

• PRAXIST_ASSIGNED_GPU_UUIDS: authoritative ordered physical GPU assignment;
• CUDA_VISIBLE_DEVICES: CUDA visibility mask delivered by Praxist;
• NVIDIA_VISIBLE_DEVICES: container/runtime visibility mask delivered by Praxist.

When PRAXIST_ASSIGNED_GPU_UUIDS is non-empty, the harness must preserve that exact GPU UUID list across every child-process boundary. If either visibility variable is missing, restore it from PRAXIST_ASSIGNED_GPU_UUIDS. If a present visibility variable disagrees, fail the evaluator with a clear accelerator-binding integrity error instead of silently choosing a device. Framework-local devices such as cuda:0 are correct inside the inherited mask, but must never be written back to a descendant's visibility environment as host ordinal 0. Legacy lease markers may remain compatibility signals but must never be the only way the harness recognizes a Praxist assignment.

Only standalone mode, where no Praxist assignment is present, may construct a visibility mask from an operator-selected integer device. Explicit CPU mode must clear Praxist/CUDA/NVIDIA accelerator assignment for that child. Do not add automatic GPU-to-CPU fallback unless the task protocol explicitly declares both executions scientifically equivalent.

Implement this handoff once in the smallest existing evaluator/launcher helper and reuse it throughout the process chain. Do not copy resource-binding logic from an older task. Inspect Python launchers with AST-aware analysis where practical and inspect shell/container wrappers too. Treat assignments such as CUDA_VISIBLE_DEVICES = "0" or str(gpu_id) after a Praxist assignment has been observed as launch-blocking harness defects.

Add focused task-owned contract tests, preferably to an existing harness test module, covering:

1. one UUID is preserved exactly;
2. an ordered multi-UUID mask is preserved exactly;
3. missing CUDA/NVIDIA masks are restored from PRAXIST_ASSIGNED_GPU_UUIDS;
4. a conflicting existing mask fails loudly;
5. standalone integer selection remains compatible;
6. explicit CPU mode clears accelerator visibility;
7. evaluator -> trainer -> worker propagation keeps the same physical mask.

For a task that explicitly selects the Praxist-managed NVIDIA/CUDA backend on a host with at least two usable devices, run one bounded non-zero-device launch-readiness preflight. Select a non-zero device by UUID, pass it through the real evaluator-to-compute-child path, create only a tiny CUDA allocation, and compare the compute PID's physical UUID from the available driver telemetry with the scheduler assignment. This is a binding test, not CPU-vs-accelerator benchmarking, bottleneck detection, or training. Stop it promptly and verify the process and allocation are gone. If the selected backend cannot perform this physical check, run all applicable environment-contract tests and record physical verification as unavailable; never claim it passed. A task requiring this managed multi-device placement is not launch-ready when its binding preflight fails. CPU-only tasks, unified-memory platforms, task-managed accelerators, and other backends follow their own observed launch contract and must not be forced through CUDA/UUID checks.

For the first real run of a newly generated compatible Praxist-managed NVIDIA/CUDA harness, perform one bounded, read-only consistency audit after several jobs start: compare scheduler UUIDs, evaluator/trainer/worker visibility masks, and driver-observed process UUIDs. Record the result in the initialization report and stop auditing after a clean sample. A mismatch is a task-harness compatibility failure, not evidence about the research variant. 7. Set Praxist parameters to use the scheduler without obvious oversubscription:

• compute_budget.per_experiment_gpu_hours: estimated full-eval GPU-hours only when that accounting unit actually applies; otherwise omit it or use 0.
• compute_budget.max_parallel_runs_per_peer: keep as a legacy compatibility cap; central mode's host-wide limit is authoritative.
• generation_policy.cohort_size: number of peers that should keep the measured bottleneck highly utilized without crossing pressure, safety, license, service, simulator, or memory limits.
• generation_policy.per_generation_hours: safety upper bound long enough for one serious peer loop: implement, smoke, evaluate, publish, and drain.
• generation_policy.max_generations: long enough for several research cycles unless the user requests a smoke-only task.
• task-local evaluator timeouts and tier expected durations: 2-3x the measured or estimated runtime of each tier, with a hard timeout for stuck training/simulation.

8. Reserve real work capacity for both evidence maturity and exploration. Set mature_supply_fraction: 0.25, mature_supply_redundancy: 3.0, and mature_assessment_min_completion_probability: 0.25 unless measured task physics justify an explicit override. For P peers the default evidence target is Q=max(1, ceil(P*0.25)); the controller maintains at most min(P, ceil(3*(Q-M))) queued/running mature plans and mature-priority leases, where M comes from canonical effort/coverage results. This is bounded tail-risk redundancy, not three GPU slots per result and not permission to create filler work. Require peer prompts to submit complete-protocol work with --work-class=mature early in the generation rather than waiting until the assessment boundary. Allocate the first wave so up to Q peers begin a justified direct mature evaluation while at least one peer explores when the cohort has multiple peers; use ordinary and scout for the other classes. The scheduler retains at least one exploration slot when such work is queued. Do not start a no-checkpoint complete evaluation when its measured/estimated duration cannot fit the remaining generation time. Independently set synthesis_trigger.mature_quorum_fraction: 0.25 whenever the task distinguishes close-grade evidence through a ratio gate, complete stage, protocol-integrity contract, or mature parent lane. This positive quorum makes task-defined mature evidence the normal-completion condition. mature_supply_fraction only prioritizes work and cannot replace this close gate. Never set the quorum to 0.0 merely to avoid a deadlock: 0.0 allows fixed/adaptive information density, including progress and diagnostic findings, to become the normal close condition. Safety-cap and cohort-drained outcomes already provide bounded liveness when mature evidence remains insufficient.
9. Prefer two operator launch profiles when useful:
   • smoke profile: small cohort and short tiers for validating wiring.
   • research profile: hardware-efficient cohort and complete stages for real Praxist runs.

Write the unchanged observation command, sampling interval, observed pressure, uncertainty, scheduler profiles, initial/max concurrency, and duration evidence to assets/resource_plan.md; mirror only supported compact values in task.yaml. If current Praxist docs/templates do not expose an exact field name for a setting, do not invent a silent no-op key.

Praxist Model And Agent Runtime Defaults

When generating operator instructions, README launch examples, or assets/resource_plan.md, recommend this Praxist launch profile when the host has DEEPSEEK_API_KEY available, unless the user explicitly supplied another provider, model, runtime, or agent-system preference:

model provider: model_provider:deepseek_alias
model: deepseek-v4-pro
agent runtime: agent_runtime:claude_sdk

This corresponds to DeepSeek V4 Pro through the DeepSeek alias provider plus the Claude SDK runtime. It is the recommended initial profile for large Praxist research runs because the long-context DeepSeek route is cost-effective when prompt layout stability is preserved. If DEEPSEEK_API_KEY is not configured, do not block initialization solely for that reason; document the missing recommended key and preserve any user-configured provider fallback or explicit launch override.

User-provided configuration always wins. If the user specifies a provider, model, runtime, PRAXIST_MODEL_PROVIDER_REF, PRAXIST_MODEL, PRAXIST_AGENT_RUNTIME_REF, PRAXIST_AGENT_SYSTEM, or equivalent CLI flags, preserve that choice and document it in assets/resource_plan.md and the generated README instead of overwriting it.

Generate a run-wide agent.reasoning_effort policy for peers, DIG, PIs, and Chair. Use max unless the user explicitly requests auto, off, low, or high; preserve an explicit choice without inferring one from the research domain. auto explicitly leaves provider/runtime behavior unchanged. Keep legacy premium_mode only when repairing a task that already uses it. Praxist owns the provider-specific mapping, so do not add DeepSeek request fields, relay arguments, or runtime wrappers to the task harness.

Do not store raw API keys in the task directory. Provider keys belong to user-level Praxist config or the shell environment, usually prepared by the runtime-install skill. In task initialization, only document required key names such as DEEPSEEK_API_KEY.

When writing launch examples, verify the current CLI help when possible. The intended profile is:

praxist start \
  --task-path /path/to/task \
  --model-provider model_provider:deepseek_alias \
  --model deepseek-v4-pro \
  --runtime agent_runtime:claude_sdk

Do not change the default runtime solely because a particular agent operator interface is present. When the user explicitly chooses agent_runtime:codex_sdk, carry that ref into generated launch instructions and record the choice in assets/resource_plan.md. Verify the Praxist environment provides openai-codex==0.147.0, claude-agent-sdk==0.2.136, codex-relay==0.5.5, and MCP support. For the default agent_runtime:claude_sdk, verify claude-agent-sdk==0.2.136. Do not generate task-owned relay launchers, per-peer relay ports, or runtime transport helpers: Praxist owns long-lived local app-server clients, direct MCP attachment, and the private run-scoped relay needed by DeepSeek/OpenRouter. OpenAI uses the direct SDK path. For native OpenAI, generated instructions may use either OPENAI_API_KEY or a saved ChatGPT login, but authentication remains operator-owned and must never be copied into the task. State that environment API credentials take normal priority outside explicit --codex-native mode; that mode suppresses API and inherited provider/model overrides and is valid only for model_provider:openai_compatible. Invocation through praxist-takeover-codex is already an explicit user choice of agent_runtime:codex_sdk, model_provider:openai_compatible, and saved ChatGPT authentication. Preserve that choice without asking for a provider or API key, do not replace it with the DeepSeek/Claude recommendation, and use gpt-5.6-luna unless the user explicitly selected another account-supported model. Require praxist doctor --codex-native --task-path <task> to report the selected model catalog entry as ready before resolve or launch. For a newly created task on this invocation path, use dig_lite.contract.min_rejected_alternatives: 2 unless the user chose a stricter value; keep every other task-justified DIG diversity and integrity check. Do not create a task-local runtime shadow or model-specific schema adapter to support this selection.

Praxist owns provider-specific context efficiency. Do not copy context batching, cache routing, memory-store, prompt compression, or session-interval settings into task.yaml, task prompts, roles, or evaluator code. Codex-native mode and OpenRouter runs automatically use Praxist's lossless event-coalescing path; direct DeepSeek runs deliberately retain their established event timing and cache behavior. This policy preserves canonical artifacts and only changes when a continuation session opens. Record an operator override such as PRAXIST_CONTEXT_EFFICIENCY_MODE=off only in launch documentation when the user explicitly requests it. Never summarize away or delete task evidence to reduce token use.

Required Research-Loop Defaults

Enable these by default for real research tasks unless the user explicitly asks for a minimal smoke fixture:

• Initial DIG: default ON only for absolute generation 0 through dig_lite.enabled: true and dig_lite.generation_scope: initial_only. Do not enable DIG for later generations unless the user explicitly requests the legacy all-generation behavior.

• Quality diversity: default ON independently through quality_diversity.enabled: true, with both initial_generation_enabled: true and later_generations_enabled: true. Generation 0 applies QD to DIG candidate pools. Later generations keep DIG off and apply soft QD guidance through the existing PI synthesis path. In a Multi-PI topology, the PI memo proposal union is the candidate pool and Chair assigns contracts. In a single-PI topology, the PI forms and selects proposals in its existing synthesis call. Preserve task-owned diversity labels so both paths have useful axes. Each generation switch may be disabled without disabling the other. Initial QD still depends on gen0 DIG for its candidate pool; later QD is independent of DIG execution. Define evaluation.diversity_dimensions as task-owned axes before enabling QD. PI/Chair contracts use those axes under planned_dimensions to state the intended design before work starts. Peers must publish the implemented, evaluated values under design_dimensions; this is realized evidence, not a copy of the plan. Never backfill missing realized values from the PI plan. Diagnostics must compare planned and realized distributions separately and report per-axis HHI, sample size, missingness, and plan-to-execution drift. Missing dimension reports are advisory evidence-quality gaps, not a reason to block an otherwise valid experiment.

• Constructive peer mix: default advisory feedback is explicitly controlled by evaluation.constructive_peer_mix_enabled: true; the target remains constructive_target_ratio: 0.75. Set the boolean false when the task should not bias the next generation toward a constructive-work floor. This switch is independent of dig_lite.innovation.enforce_forward_slots.

• Gems reset: default OFF for newly initialized user tasks. Use continuous evolution first through gems.enabled: false or an omitted Gems reset block when current templates/docs support omission. Do not guess a reset cadence during task initialization. If the user explicitly asks for periodic Gems, or a later diagnostic run detects a significant performance ceiling, enable Gems reset then and set gems.reset_interval_generations from the observed plateau-onset generation id or the task owner's explicit choice.

When writing exact config for a machine learning project, inspect TASK_TEMPLATE_ROOT/machine_learning_template/task.yaml first (the source form is templates/tasks/machine_learning_template/task.yaml). Use TASK_TEMPLATE_ROOT/template/task.yaml only as the minimal scaffold/smoke shape, and use richer domain reference templates such as TASK_TEMPLATE_ROOT/sam_optimizer/task.yaml only for task-specific inspiration. Also inspect current Praxist docs/CLI help. Use supported field names. The current standard task-local fields are:

praxist_plugins:
  tools:
    - tool_server:evaluation_tools
    - tool_server:frontier_tools
    - tool_server:finding_graph_query
    - tool_server:memory_tools
    - tool_server:prior_work_tools
    - tool_server:run_report
    - tool_server:literature_lookup

evaluation:
  maturity_policy:
    min_effort_ratio: 0.75
    min_coverage_ratio: 0.80
    require_ratio_gate: true
    # Include labels only when this task actually uses staged evaluation.
    complete_stage_labels: [complete]
    # Omit this field when the task has no early stages.
    preliminary_stage_labels: [preliminary, aligned]
  constructive_peer_mix_enabled: true
  constructive_target_ratio: 0.75
  launch_guard:
    enabled: true
    # Calibrate from observed p90 runtimes. Close-grade is the task-authorized
    # evaluator that can satisfy normal close.
    estimated_heavy_eval_minutes: <estimate_from_resource_plan>
    estimated_close_grade_eval_minutes: <close_grade_p90_from_resource_plan>
    safety_factor: 1.25
  frontier_lanes:
      
       
       
       
      
         { ,  }
      
      
      
       [, ]
       
       []
       []
       [, , ]
      
       
       
       
       
      
         { ,  }
        
        
        
        
        
      
        
        
        
         { ,  }
         { ,  }
       [, ]
       []
       []
      
      
       [, , , , , , , , ]
       
       
       
       
      
       
       
       
      
         { ,  }
       [, ]
       
       
       
       
      
       
       
       
      
         { ,  }
       [, , , , ]
       
       
       
       


   
   
   
   
   
   
   
   
  
     [, , ]
     
     
     
  
     
     
     
     


   
   
   
   


   
   
   
   
    
     


     
   
   
  
     

If the unchanged baseline observation and generation bound do not make Q mature results physically plausible, do not silently weaken effort/coverage or set mature_quorum_fraction: 0.0 as a liveness workaround. Recalibrate the generation bound, peer/concurrency plan, or the task-owned accepted evidence protocol; changing scientific acceptance semantics requires user confirmation. If mature evidence still cannot be produced, retain the positive gate so safety-cap or cohort-drained close records the insufficiency explicitly. Use 0.0 only when the task intentionally has no separate maturity distinction and the user explicitly confirms that ordinary information-density findings are sufficient to close a generation; record that decision in the resource plan and initialization report.

If the user explicitly enables periodic Gems reset, add the task-owned reset policy then:

gems:
  enabled: true
  selection_policy: mature_evidence_top_k
  min_mature_eval_units: <parent_authorized_protocol_required_evaluation_units>
  evidence_stage_min_units:
    <preliminary_stage_label>: <preliminary_protocol_required_evaluation_units>
    <parent_authorized_stage_label>: <parent_authorized_protocol_required_evaluation_units>
  reset_interval_generations: <plateau_generation_id_or_user_choice>
  max_resets: 3
  max_gems_per_reset: 4
  max_gems_total: 4
  max_gems_per_family: 2
  prompt_max_gems: 4
  archive_ordinary_findings: true

Set gems.min_mature_eval_units from the number of task-owned evaluation units required by the protocol the user authorizes for Gems and durable parent use. That is normally the complete protocol, but it may be an explicitly selected reduced protocol. When the task has staged evidence, map its own labels to cumulative unit requirements with evidence_stage_min_units. Always use units in Praxist configuration, even when a task-local evaluator uses a different term. Do not emit compatibility-only historical maturity keys in a newly generated task, and never copy another task's labels or unit counts.

If a task omits evaluation.frontier_lanes, Praxist still has a legacy primary metric frontier, but promising validation candidates may not appear as a separate incubator-style lane in frontier/frontier_manifest.json. For any expensive task with preliminary, aligned, partial, or diagnostic validation, or multiple meaningful metric axes, configure both a durable incubator lane and a lower-confidence candidate lane so future PI/Chair synthesis can keep strong parents and weaker signals in view. Size the incubator to the metric space: use at least k = max(8, 2 * number_of_distinct_metric_families) for real multi-axis tasks and a cumulative_cap about 4-6x k; use larger values such as k: 16 and cumulative_cap: 96 for broad multi-axis research.

The incubator is intentionally a lower-admission long-term variant library, not a stricter confirmed-winner lane. It should preserve protocol-authorized, protocol-passed, non-suspect Pareto/new-high candidates across every genuinely distinct metric family. Configure admit_new_high: true so a candidate must improve the incubator's current Pareto surface rather than merely repeat an older point. Do not make incubator stricter than confirmed promotion; if it stays empty over many generations, later peers and PI/Chair synthesis can lose promising parents and the run can stagnate.

When the user-approved maturity contract uses effort and coverage ratios, newly generated task evaluators must emit effort_ratio and coverage_ratio in each canonical machine-readable result summary. effort_ratio compares actual effort to the task-defined reference effort; coverage_ratio compares completed required evaluation units to total required units. Praxist uses the same extractor to project these facts into the auto-materialized finding; do not make task code duplicate source-owned facts. A standalone manually authored result finding without a canonical summary reference must carry the ratios itself. These fields let Praxist start generation assessment on mature evidence without making stage labels into hard-coded task logic. Set require_ratio_gate: true only for a contract that uses those ratios. If the user explicitly chooses label/flag-based or information-density semantics, preserve that choice and document the limitation instead of manufacturing ratio requirements. With a required ratio gate, missing ratios remain unknown even when a complete-looking label is present.

When ratio gating is enabled, before declaring the task launch-ready execute the shortest valid scored path that uses the evaluator's real canonical summary writer. Then validate that actual file with praxist resolve <task_path> --result-summary <summary_path>. For an expensive evaluator, a deterministic contract probe may stop before a performance claim, but it must call the same serialization path as real scored evaluation. A hand-written lookalike JSON file is not proof. The preflight must pass with finite ratios when require_ratio_gate: true; stage labels are optional task audit vocabulary and cannot substitute for missing ratios. The same preflight must reject completion metadata only when the task-owned policy actually resolves it to contradictory decisions. A configured terminal cap may be mature; an early cap is incomplete. Validate the achieved protocol and ratios instead of rejecting status vocabulary by substring.

Evaluator Fan-Out Preflight

Before any aligned, complete, or otherwise expensive multi-unit fan-out, prove the actual evaluator path in increasing-cost order:

1. Run the task-appropriate build, load, or startup check in the exact runtime that fan-out will use. For interpreted code this may be compile/import; for a binary, library call, simulator, container, notebook, or service, exercise its corresponding real loading or startup boundary. Source-text inspection alone is not execution proof.
2. Validate the evaluator's actual public invocation contract before reserving expensive resources. Exercise its real parser/help path when it has a CLI; otherwise validate the function, RPC, simulator, container, notebook, or service interface that the task actually calls.
3. Run a one-unit canary through the same public evaluator, scheduler path when Praxist owns launch, runtime environment, and canonical summary writer that fan-out will use. One unit means the smallest scientifically valid task-defined case; it does not imply a universal seed, epoch, iteration, split, or device model. For a protocol without discrete units, use its shortest valid invocation.
4. Validate the produced summary and its summary-to-finding projection. Only then fan out. A repaired command or implementation must pass a new canary; an earlier canary does not authorize changed code.

The canary verifies wiring and publication, not performance or maturity. Keep valid low scores and negative scientific outcomes; stop fan-out only for an execution, schema, publication, or declared protocol-integrity failure.

If the task explicitly claims that evidence is produced by an external or otherwise independently trusted evaluator, also prove the task's declared trust boundary: a peer must not be able to replace the authoritative result, and the verifier must reject a modified or unattested result. The attestation mechanism remains task-owned. Do not impose this mode on ordinary peer-authored evaluators, and do not hard-code a particular signature scheme, filesystem owner, dataset split, or service into Praxist task initialization.

Task result summaries may be nested under results/**/ and should use one of the compact names summary.json, evaluation_summary.json, eval_summary.json, tiered_eval_summary.json, or custom_*_tiered_eval_summary.json. result_summary.json remains a supported compatibility name. Every summary must carry a stable top-level variant_id, or an explicit child-result ID when one evaluator emits multiple candidates; repeated stages for the same candidate must reuse that identity. Put lane, maturity, effort/coverage ratio, protocol, and diagnostic metadata in structured top-level, metrics, current_aggregate, or extra fields so the Praxist materializer can transfer fields such as frontier_lane, promotion_lane, evidence_stage, evidence_valence, diagnostic_role, failure_mode, parent_candidate, and next_step_intent into canonical findings. Do not hide these facts only in prose, filenames, or side files.

When central scheduling owns evaluator launch, every submission must pass a result-specific directory through a recognized explicit output option such as --out-dir, --output-dir, or --result-dir; do not rely on the process cwd or claim the run-wide results/ root as one peer's output. The canary must confirm that summary-to-finding materialization preserves the scheduler's canonical generation and peer attribution. Where available, also write the scheduler-provided generation_id and peer_id into the summary as structured cross-check fields rather than deriving them from free-form names.

Use evaluation.launch_guard.estimated_heavy_eval_minutes for the most expensive ordinary evaluator and estimated_close_grade_eval_minutes for the task-authorized protocol that can satisfy normal close. Record both observed p90 values in assets/resource_plan.md and task prompts. With enabled: true, CLOSING_SIGNAL freezes every new evaluation or training launch in the standard peer runtime, while preserving already-started work for natural drain. Generated prompts must say that after close a peer may inspect outputs, publish results, and update notebook/memory, but must not run another evaluator, script, shell launcher, or background process. Background evaluations still need the standard protected_pids launch facade so Praxist can queue and track their process group during final synchronization. Its semantic tag must not change for retries, output paths, timestamps, or harmless flags.

For a task that requires mature/complete evidence before normal close, measure the unchanged complete evaluator enough times to estimate its p90 wall time and enforce this launch-readiness inequality:

estimated_close_grade_eval_minutes * safety_factor
  < effective_generation_close_horizon_minutes - drain_margin_minutes

Use a drain margin of at least 30 minutes unless measured publication and shutdown latency justify more. The close-grade estimate may describe an explicitly user-authorized reduced protocol while the heavy estimate preserves a longer optional protocol. The effective horizon is the earliest enabled generation/peer hard bound, including an enabled adaptive synthesis ceiling. If the inequality fails, raise the relevant generation and synthesis bounds or ask the user to authorize a different protocol; never relabel an incomplete run as complete. An explicitly user-authorized late-signal or reduced protocol remains valid when the task's launch, maturity, lane, and close policies consistently encode that choice.

Declare the single public evaluator in task_entrypoints.evaluation.command. Praxist normalizes that structured field into the legacy-compatible toolchain.eval_entrypoint when needed, so do not store the same path twice unless a legacy task consumer explicitly requires it. The default Claude SDK runtime uses the resulting entrypoint to register a direct evaluator invocation with the protected-PID manifest, keeping active_evals and generation drain truthful. Prompts must still require the explicit wrapper for compound or task-owned background shell commands. Keep task-owned evaluator, trainer, config, and harness paths relative to the task root. Absolute paths are appropriate only for external assets such as an existing dataset, simulator, or environment that cannot live inside the task; verify every such path immediately before launch and record why it is external.

Before marking the task launch-ready, run the public evaluator from both the task root and a temporary run-like subdirectory through the declared task interpreter. Verify that the child imports task dependencies without runner site-packages, resolves the same evaluator/config/data paths in both cases, and writes its canonical summary under the requested output directory. Do not inject the Praxist checkout through task PYTHONPATH; if a task genuinely owns a custom PYTHONPATH, declare it explicitly in runtime_environment.env and test that exact environment.

Staged Evaluation Alignment

When the target task evaluation is too expensive for every early attempt, normally build a three-level task-owned evaluation contract. The user may instead explicitly choose a simpler or intentionally incomplete protocol. For expensive repeated-case, seeded, episodic, simulation, benchmark-suite, or multi-dataset tasks, use the following default unless the protocol-intent table says otherwise:

• Preliminary check: the cheapest executable sanity check. By default it may use a tiny fixture, one split, one episode, very low effort, or failure-only checks. Its purpose is wiring validation, impossible-idea filtering, failure diagnosis, and qualitative validation candidates. It must not be used for ranking variants, selecting clean parents, Gems reset, mature frontier promotion, performance-ceiling detection, or "best so far" claims unless the user-owned protocol intent explicitly authorizes those uses and reports the reduced protocol transparently.
• Aligned evaluation: the only early protocol that can rank or prioritize variants before complete evaluation. It must use the same evaluator code path, primary metric direction, aggregation semantics, invalid-result rules, leakage checks, and protocol-integrity checks as the complete protocol. Preserve the complete protocol's evaluation-unit coverage by default: use the same folds, datasets, task cases, scenes, seeds, episodes, or operating regimes whenever possible. Save compute primarily by reducing optimization budget, such as epochs, gradient steps, rollout horizon, simulator steps, inner-loop iterations, checkpoint sweeps, or repeated restarts. This keeps the measured loss/metric landscape close to complete evaluation while accepting a less-converged parameter location. Reducing evaluation units is a last resort, not the default. If full coverage is truly impossible, the aligned evaluation must still use a near-complete, fixed, stratified coverage set that preserves all major regimes and hard cases, and must record the coverage ratio and omitted-unit rationale in task-owned assets. If the coverage is materially small or convenience-sampled, label the result preliminary or partial instead of aligned. For example, if the complete protocol spans many evaluation units, a small convenience subset is normally preliminary/partial, not aligned; a better aligned evaluation would keep near-complete unit coverage and reduce epochs, iterations, rollouts, or training steps. Do not mix failed-unit smoke checks, ad-hoc partial runs, or varying epoch/step budgets into this ranking stage.
• Complete evaluation: the default highest-evidence task protocol. This stage, or any reduced protocol explicitly declared equivalent or sufficient by the user-owned intent, can create clean implementation parents, confirmed frontier entries, or Gems parents.

Tasks with genuinely staged protocols choose their own literal labels. preliminary, aligned, and complete are recommended neutral defaults; an existing staged task may retain its established names by listing them under preliminary_stage_labels and complete_stage_labels. Unstaged protocols may omit stage labels. Labels never carry global effort, coverage, ordering, or parent-eligibility semantics by themselves.

Generated evaluator summaries and findings must report enough fields for later agents to tell which protocol produced a result:

• for staged protocols, extra.evidence_stage: the task-local preliminary, aligned, or complete label;
• extra.protocol_name or extra.eval_mode;
• metrics.evaluation_units or extra.completed_required_eval_units, plus task-local counts such as n_seeds or n_episodes when relevant;
• fixed budget fields such as epochs, environment steps, wall-clock cap, or simulation horizon;
• coverage fields such as extra.coverage_ratio, extra.complete_protocol_evaluation_units, and extra.aligned_evaluation_units when the task has discrete evaluation units;
• exact maturity fields: effort_ratio and coverage_ratio in a supported scalar fact container for every scored canonical evaluator summary; Praxist projects them into auto-materialized findings, while standalone findings without a canonical summary reference must carry them directly;
• metrics.scored_complete or a task-owned authority marker only for evidence that satisfies the declared mature protocol. If that protocol is explicitly reduced, the result must still expose its reduced stage, effort, and coverage;
• protocol-integrity flags or invalid-result reasons when a run is not clean.
• structured negative-evidence metadata on every finding: extra.is_negative must be explicit, extra.evidence_valence should be one of positive, neutral, negative, or mixed, and negative or mixed results should include compact extra.failure_mode plus extra.disconfirming_claim_ids when a hypothesis/claim is weakened.
• when non-code launch settings can change the treatment, one secret-free top-level effective_config object containing all result-affecting CLI arguments, environment overrides, protocol choices, and task-local config values, plus explicit effective_config_complete. This object must contain values after the evaluator has applied defaults, parsing, aliases, and type conversion. Do not hash a raw environment snapshot or distinguish an omitted setting from an explicit setting equal to the same resolved default. Exclude unrelated runtime environment. Keep the full object only in the canonical evaluator summary; downstream Praxist artifacts use its digest and source_result_path. For derived work, make the task evaluator or its existing launch helper read the selected parent summary, resolve the child through the same schema, and report a secret-free key-level mismatch before expensive execution. Do not make each peer manually replay a parent process environment and do not blindly export every parent environment entry.

Unless the user-owned protocol-intent table assigns different permissions, task prompts, role prompts, and resource plans should use these defaults:

• Preliminary results are useful validation signals, not ranking evidence.
• Only aligned results may be used to prioritize which variants deserve complete evaluation.
• The aligned protocol should be close to complete in data/evaluation coverage and cheaper mainly because training or optimization budget is reduced.
• Negative evidence is still useful research memory. Failed, dominated, constraint-violating, non-generalizing, null-ablation, or falsifying results should be published with the structured fields above instead of being omitted or relabeled as neutral.
• The protocol explicitly authorized for durable parents remains the final parent-promotion gate; by default this is complete mature evidence.
• Reports that compare early to complete performance must compute calibration separately for preliminary and aligned evidence. Weak preliminary correlation is not a failure; weak aligned-to-complete correlation means the task protocol should be redesigned before trusting early rankings.

Artifact Ownership Instructions For Generated Tasks

When writing the generated task README, task prompt, role skills, and resource plan, include this rule in task-specific language:

• Peers and task harnesses publish task evidence through evaluator result summaries and share_finding.
• Praxist owns frontier/incubator state, Gems state, prompt-layout artifacts, PI evidence packs, research memory, diagnostic tables, and generation boundary markers.
• Current facts come from result/finding evidence, frontier/frontier_manifest.json, committed gems/gems_state.json, and gen_N/generation_boundary.json.
• When configured, the durable incubator is part of frontier state. It should contain task-authorized, protocol-passed, non-suspect Pareto/new-high candidates that are not yet clean enough for confirmed promotion. If it is empty despite such evidence, treat the task lane policy or peer metadata as a high-priority task-harness problem.
• Validation candidates are non-frontier signals. They may include partial, lower-stage, failed-but-informative, preliminary, aligned, repair, ablation, or diagnostic evidence, plus late-after-boundary result summaries that Praxist retained after a generation closed. Generated tasks should route them through structured findings/result summaries with task-owned lanes, not through new side files. Later agents may use them for validation, repair, falsification, comparison, ablation, or full scoring, but not as clean implementation parents unless canonical evidence later promotes or revalidates them.
• Leaderboards, PI evidence packs, PI agendas, rendered prompts, prompt-layout manifests, diagnostics, and behavior reports are derived views or audit snapshots. They are useful context and replay evidence, but generated peer or PI instructions must not tell agents to hand-write them or use them to override evaluator evidence.
• Partial .tmp, .candidate, .rejected, or incomplete final-generation files should be handled by control/resume tooling, not by task peers.

This is not a new task entity. It is guidance that keeps generated tasks aligned with Praxist's current artifact semantics and reduces cross-generation information drift.

Metric Discovery

Derive metrics from the project first: evaluator code, papers/docs, result tables, benchmark scripts, and logs. If metrics are absent or ambiguous, use no-key public literature/database/open-access lookup when available, otherwise agent-host web search, for the domain's standard benchmark metrics. Cite sources in assets/literature/research_directions.md and label them as literature/context signals rather than measured task facts. If metric standards depend on external datasets, services, packages, or licensed assets that are not present locally, do not make task execution depend on acquiring them; document them as external requirements and design the Praxist evaluator around the currently available local resources.

Also research the domain's ranking convention, not just metric names. For each task domain, determine whether the field normally ranks by a single mean score, a mean-plus-risk tradeoff, confidence intervals, lower confidence bounds, statistical significance, Pareto efficiency, safety/regret constraints, or multi-objective dominance. Use project docs first; if absent, search relevant papers, benchmark leaderboards, challenge rules, and methodology guides. If the domain commonly cares about variance, seed sensitivity, tail risk, simulator stochasticity, confidence intervals, or lower-bound robustness, encode that in evaluation.frontier_lanes through task-owned axes or explicit robust metrics such as primary_metric_lower_confidence_bound, primary_metric_std, seed_robustness_std, risk_metric, or constraint_violation_rate. Do not leave the mature lane as raw-primary-only when the domain standard says robustness matters.

Record:

• primary metric and direction;
• an explicit direction for every metric used as a comparative anchor, declared through anchor_metrics or lane axes/optional_axes; never assume the primary metric's direction applies to auxiliary metrics;
• primary ranking rule: raw mean, confidence/lower-bound, Pareto lane, or task-specific robust composite;
• incubator axis families: the distinct metric dimensions that should define durable Pareto/new-high retention before clean confirmation;
• auxiliary metrics;
• constraints and invalid-result conditions;
• aggregation over seeds, folds, maps, scenes, episodes, datasets, or other task-owned evaluation units;
• whether variance, confidence intervals, or lower confidence bounds should affect frontier ordering;
• preliminary sanity protocol, if any, and its user-approved permissions;
• aligned protocol, if complete evaluation is expensive, and its permissions;
• for aligned evaluation, the expected evaluation-unit coverage ratio and reduced training/optimization budget that makes it cheaper than complete;
• the protocol or protocols the user authorizes as mature;
• how future diagnostics should compare each early stage to the mature protocol without mixing cheap preliminary checks into ranking statistics.
• how future diagnostics should detect an empty or stale incubator despite parent-authorized, protocol-passed candidates.

Build The Task Directory

For machine learning projects, create a task directory shaped like TASK_TEMPLATE_ROOT/machine_learning_template, adapted to the project. For non-ML projects, use only the file/directory layout of TASK_TEMPLATE_ROOT/template as the minimal shape. Its checked-in policy demonstrates optional Praxist features; do not copy its stage labels, lane set, metrics, or resource values unless the target task independently justifies them:

task.yaml
README.md
description.md
prompt_base.jinja2
prompt_generation.jinja2
prompt_task.jinja2
.gitignore
.praxist/plugins/panel_topologies/<task_topology>/plugin.yaml
roles/
audit_rules/
evaluations/<primary_eval>/run.py
assets/harness/