| name | task-design |
| description | Use when writing or reviewing a task spec that will be given to an implementation agent. Guides task design through interactive Q&A so the agent can succeed on the first attempt without relying on general harness principles. |
Task Design
Write a spec an implementation agent can execute correctly on the first attempt.
This is an interactive conversation: research what you can, ask about what you
cannot, draft, and review with the user as you go.
What belongs in a spec
Task-specific facts — intent, scope, metric definitions, mechanism
semantics. These are things the implementation agent cannot know without being
told.
Generic implementation guardrails — verifying configs, avoiding statistical
proxies, modeling topology, reusing modules — belong in sim-dev and run at
implementation time. A spec that restates them absorbs the harness's role and
grows without adding information.
The repository is a discrete-event simulator. A task may model existing
real-system behavior, a novel mechanism with no real-world counterpart, or a
mix. In every case the spec must stand alone: when the user cites an
external system, extract the semantics the simulator needs and write them out as
definitions. Do not link to external source and expect the agent to derive
intent from it — that code can change, and the agent will over-emulate it.
Information boundaries
Get from the user — never infer
- Feature intent — what to build and why it matters
- Target hardware and serving config — stated explicitly, or a specific
config file the user names
- Workload — which one to use
- Expected simulator output — the metric, its precise definition, its
granularity, and any internal-vs-exported semantic split
- Scope boundary — expected realism versus what is out of scope
Ask what the simulator should produce, not how it will be evaluated. The
spec describes required behavior, not a validation pipeline.
Research yourself — safe to look up and propose
- Codebase structure — what paths exist, what state is tracked, which
decisions are sensitive to the feature, which traits and registries apply
- Artifact schemas — open the real files the user named and document what
they actually contain
Do not pre-verify model architecture against upstream sources. Record the model
name; authoritative config verification is sim-dev's job at implementation
time. Doing it here hardcodes values that may drift and removes the
implementation agent's reason to read the real config.
Never touch
Validation data, evaluation scripts, answer keys, and hidden evaluator
implementations. Encoding their content into a spec makes the evaluation
meaningless.
Config values are the dangerous middle ground
Existing scenario configs may target different hardware, be outdated, or encode
another experiment's assumptions. Code structure is safe to reference; config
values require user authority. If the user names a config, use it but confirm
the critical values. If the user says nothing about hardware, ask — do not fill
the gap from whatever config happens to exist.
Deciding versus asking
Stop and ask when a wrong choice changes what the simulator does or
produces: metric definitions, output semantics, config values, scope
boundaries. Recording a "conservative assumption" for these is not acceptable —
resolving them is the entire point of this skill.
Decide yourself for implementation details the user will not care about:
edge-case boundary behavior, format minutiae, internal bookkeeping granularity.
Note the choice in the spec as an explicit assumption so the agent knows it was
deliberate.
How to ask
Present 2–4 concrete options with a recommendation rather than an open-ended
question, so the user can reply with a letter instead of a paragraph.
I need to decide the export format.
⭐ (A) Long format — recommended
One row per (time, instance) sample. Handles multiple instances without a
schema change.
(B) Wide format
One row per timestamp, one column per instance. Compact, harder to extend.
Pick one, or describe something different.
Batch related decisions into one message. Always offer an escape hatch. Block
all other work while waiting — do not keep researching or drafting in
parallel, or your options scroll off the user's screen before they answer.
Spec sections
1. Purpose and scope
One sentence stating the goal, explicit non-goals, and the realism boundary.
Bad: "Implement caching support."
Good: "Implement cache-aware memory modeling. The goal is
decision-relevant memory state, not allocator-level byte accuracy. Kernel
throughput effects are out of scope."
2. Mechanism background
Give the agent a mental model, not just rules — an agent that understands the
mechanism handles edge cases the spec did not anticipate.
- Summarize the algorithm or system behavior self-contained, so the agent never
needs the original paper
- State invariants as properties, not code
- Describe each phase with its inputs, outputs, and relation to its neighbors
- Define terminology here so later sections can use it unambiguously
⚠️ A worked computation trace is mandatory. Trace one iteration end to end
with real numbers: tokens in, what computes, tokens out, and the resulting
values for every axis the compute model consumes. Use a concrete scenario and
substitute actual values — leave nothing as K or N. Abstract descriptions
hide off-by-one and off-by-factor errors that silently produce wrong results.
⚠️ Then cross-check the trace. For each pair of quantities in a step, decide
whether they are independent or derived from the same operation. Quantities from
the same operation must agree. If two dependent quantities differ, name the
concrete mechanism that makes them differ — if you cannot, the trace is wrong.
Do not rationalize a discrepancy you cannot explain physically; that is the
failure mode this check exists to catch.
Skip this section only for pure configuration changes.
3. Target system details
Hardware, serving config, model name, and workload location — all from the
user, not adopted from existing configs. If the user says "same as scenario
X", confirm those values before adopting them.
4. Expected simulator output
The metric, its standalone definition, its granularity, its output channel, and
the mapping if internal state and exported metric differ.
Bad: "Export stats that support later validation."
Good: "Export per-iteration cache memory usage in bytes, counting only
blocks with at least one live reference, as a time series rather than an
end-of-run aggregate."
5. Required mechanism behavior
Frame requirements as observable, falsifiable properties — what must be true,
not how to code it. Cover state evolution, integration points with existing
decisions, and cross-subsystem consistency.
Three levels of detail, handled differently:
| Level | In spec? | Example |
|---|
| Target mechanism | ✅ In detail | How the modeled system actually behaves, step by step |
| Navigation hints | ✅ Briefly, labeled | "The relevant cache lives near the scheduler" |
| Code-level prescriptions | ❌ Never | Specific enums, API signatures, TOML fields, file-by-file change lists |
Labeling a code-level prescription "suggested" does not help — the agent treats
any specific path or signature as a requirement. If you are writing API shapes
or edit lists, you have crossed from spec into implementation plan. The
implementation agent is an expert coder and understands the codebase more
deeply than you do; give it the mechanism and correct inputs, and it will design
the structure itself.
When a feature depends on external statistics, say what conditioning variables
the data contains so the agent uses that structure rather than collapsing it to
one global average. Do not invent structure the data does not have.
Include a "good enough" boundary: the minimum behavioral properties that
constitute completion, and a short "not required" list. If a suitable
abstraction already exists, say "use it".
6. Forbidden implementations — task-specific only
Shortcuts the agent might plausibly try that are wrong for this feature's
data, scope, or semantics, each with the reason it is wrong here. Generic
anti-patterns belong to sim-dev. Zero items is fine; most tasks have none
to two.
Good: "Do not export only an end-of-run aggregate — this task's metric is
defined as a per-iteration time series."
7. Artifacts and schema
For each input and output: path, format, key fields, and their real semantics,
verified against the actual files. Note legacy quirks. Sanity-check numeric
summary fields — if a field's plain-English name implies values its data
contradicts, the name is probably a legacy label.
Include schema facts only. Do not restate what another section already says.
8. Self-validation
A checklist of falsifiable properties with expected outcomes, covering
correctness, sensitivity, cross-subsystem consistency, regression, and
observability. State what must be verified, not in what order or how many
iterations.
9. Deliverables
Working implementation; a reproducible scenario showing the feature has real
behavioral effect; a self-validation report; brief design notes.
Discipline
Every feature must be general. The task's specific parameter values are
inputs to validate against, not the feature's scope. If a config parameter
changes to another valid value, the simulator must handle it with no code
change. State this explicitly in the spec, and do not write validation criteria
that only hold for the task's particular values.
Qualitative expectations, not derived numbers. If the inputs are correctly
specified, the agent can compute the numbers. Pre-computed values invite
hardcoding.
| Instead of | Write |
|---|
| A step-by-step byte formula with a final figure | "Halving the element width should roughly halve per-token bytes" |
| "Capacity = 56853 blocks" | "The narrower format should admit more requests at the same memory budget" |
| A prescribed metadata layout | "Per-block overhead must be documented, parameterized, and nonzero" |
No redundancy. Each fact appears in exactly one section; others reference it.
Before handing off
