agent-wiki-tasks

name	agent-wiki-tasks
description	Discover task families across summaries and write per-family comparison pages with findings narrative. Updates wiki-twobatch/_config.yaml task definitions and writes tasks/<slug>__task.md.

Agent Wiki — Task Comparisons

Overview

Two cognitive moves in one pass:

1. Discover — read across all summaries and identify task families (groups of sessions that attempted the same thing across trials and conditions).
2. Compare — for each family, write a tasks/<slug>__task.md page with a per-trial table and a findings narrative that calls out the experimental signal.

This is the cross-trajectory analysis pass of the agent-wiki family.

When to run

• After enough summaries exist that a comparative pattern is visible (typically ≥3 sessions per family).
• When the experiment design (e.g. trial × condition matrices) explicitly cries out for a comparison page.

Workflow

Step 1: Read the corpus

uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py dump-summaries > /tmp/summaries.json

Output is a JSON array of one row per summary: {session_id, goal, family, trial, condition, tool_calls, errors, recall_used, summary_filename}. family, trial, condition come from existing classification rules — they may be null if no rule has matched yet.

Read the file:

Read /tmp/summaries.json

Step 2: Decide task families

For each candidate task family:

• Slug: kebab-case identifier (e.g. extract-focal-length).
• Family: short label used to group sessions (often equals slug, but can be looser e.g. focal-length for a slug extract-focal-length).
• Family-match rules: how a future session gets classified. Currently supported: goal_substring: [list of substrings]. A session matches the family if its goal contains any substring (case-insensitive).
• Tags: a few short tags.
• Intro: 1–2 sentences setting up the question.
• Findings: 2–5 bullets summarizing what the data shows. This is the actual product — a comparison page without findings is just a table.

Rules:

1. A family needs ≥3 sessions. Smaller groups should not get their own page.
2. Findings must be evidence-grounded. Cite tool-call counts, error counts, recall-used Y/N from the dump.
3. Don't repeat what's in the table. Findings should explain why the metrics differ, not restate them.
4. Use overrides for sessions whose goal doesn't auto-match. The override key in _config.yaml/session_family_overrides is the session id.

Step 3: For each family, output JSON

{
  "slug": "extract-focal-length",
  "title": "Extract focal length from JPEG EXIF",
  "family": "focal-length",
  "family_match": {
    "goal_substring": ["focal length"]
  },
  "intro": "Question template: *what focal length was used to take @sample.jpg?* FocalLength (tag 0x920A) and FocalLengthIn35mmFilm (tag 0xA405) live in the Exif sub-IFD.",
  "findings": "**Net signal:** the gap between IFD0/GPS-only scripts and the Exif sub-IFD is the dominant cost. Sessions whose recall pointed at a script that already covered the sub-IFD finished in 2-3 tool calls; sessions that had to write an inline parser took 5+.",
  "tags": ["exif", "focal-length", "comparison"]
}

Pipe to:

echo '<json>' | uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py render-task

The helper:

• Updates _config.yaml/tasks.<slug> entry.
• Reads classified sessions; selects those matching family.
• Writes tasks/<slug>__task.md with the per-trial table + findings.

Step 4: Add overrides if needed

If a session that should be in a family didn't classify automatically, patch _config.yaml:

echo '{"session_family_overrides": {"<session-id>": {"family": "image-dims", "trial": 0, "condition": "claude_md_strong"}}}' \
  | uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py update-config

Step 5: Subtask pass — mandatory before refresh

Before refreshing indexes, scan the corpus for subtask candidates. The default reflex of "the dataset is uniform, no subtasks needed" is wrong for almost every dataset; even a 30-session benchmark of short workflows typically has 4-6 subtask-worthy sessions. See "## Subtasks" below for the heuristics + JSON contract + a worked example.

The minimum viable subtask layer for a condition × trial dataset: one subtask per condition, anchored in the session that best demonstrates that condition's distinctive behavior. Don't write 5 redundant subtasks when 1 representative captures the pattern.

Step 6: Refresh indexes

uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py catalog

This re-reads _config.yaml, re-classifies every summary, regenerates each tasks/<slug>__task.md, scans tasks/<slug>__subtask.md files, and regenerates tasks/index.md and the root index.md.

Subtasks: per-session workstream pages

The tasks/ directory holds two kinds of pages distinguished by filename suffix:

• <slug>__task.md — cross-session task-comparisons (the workflow above).
• <slug>__subtask.md — narrative slices of a single session.

After Step 5 above, run a second pass to scan for subtask candidates. Don't skip this just because the dataset is uniform — a 30-session benchmark of short workflows still has 4-6 subtask-worthy sessions. The default "there are no subtasks worth writing" reflex is wrong for almost every dataset.

When to propose a subtask

Treat each session in the corpus as a potential subtask candidate. Promote to a subtask page when at least one of these is true:

1. Exemplar of a condition or arc. When the corpus has experimental conditions (no_recall / guidelines / skill, or arc-1 / arc-2), pick the session that best demonstrates that condition's distinctive behavior — its representative-best, representative-worst, or representative-failure trace — and write a subtask. Aim for one subtask per condition × dataset, not one per session.
2. Multi-iteration debug arc. A session where the agent retried 3+ times against the same goal, with each iteration teaching something non-obvious (offset bugs, syntax gotchas, missing prerequisites). The subtask captures the debug walkthrough as a how-to.
3. Recall miss / hit pattern. A session where the recall layer surfaced material that turned out to be wrong, stale, or scope-mismatched — and the agent's recovery path is itself instructive.
4. Workstream within a long arc-split session. When a session has been split into multiple arc-summaries (<sid>__arcN.md), each arc usually has 1-3 internal workstreams worth their own subtask page (e.g. "split runner from results", "rebuild sandbox images", "walker fix for late bot batches"). Document each.

When not to write a subtask

• The session is short and atomic — its key_turns already captures everything worth capturing.
• The lesson is already an atomic guideline. (A subtask is a walkthrough; a guideline is a rule. Same insight, different artifacts.)
• The session is one of N redundant repetitions of the same pattern. Pick the most illustrative; don't document all 5.

Output JSON

{
  "slug":              "<kebab-case-id, ideally including the source session prefix, e.g. multi-tool-dead-end-stack-66f11622>",
  "title":             "<short title; mention the session prefix and condition for context>",
  "parent_session_id": "<session_id>",
  "parent_summary":    "<filename inside summaries/, e.g. abc123.md or abc123__arc1.md>",
  "tags":              ["...", "<condition-name>", "<arc-slug>"],
  "narrative":         "<1-2 paragraphs framing the pattern; reference numerical cost (tool calls, errors, retries) when relevant>",
  "key_steps":         ["concrete step 1", "concrete step 2", "..."]
}

Pipe to:

echo '<json>' | uv run python explorations/agent-wiki/skills/scripts/build_agent_wiki.py render-subtask

Subtask pages are authored (not regenerated from _config.yaml). The catalog pass picks them up, lists them in tasks/index.md under their parent session, and adds rows to _index.jsonl with kind: "subtask".

Worked example: 4 conditions → 4 subtasks

When the dataset has 5 trials × 4 conditions, the simplest non-trivial subtask layer is one subtask per condition, anchored in the session that best demonstrates that condition's distinctive behavior. Concrete pattern from wiki-twobatch/:

Subtask	Condition	What it captures
Stdlib EXIF parser walkthrough	seed	Canonical stdlib path that produces the artifact later sessions recall
Multi-tool dead-end stack	no_recall	Worst-case 4-tool exhaustion before stdlib fallback
Recalled script path is stale	guidelines	Recall hit but stored paths missing → multi-retry recovery
Skill scope mismatch fallback	skill	Synthesized skill wrong for the question; inline anyway

Pick one representative session per row; don't document every session.

Best practices

1. Findings is the product. No findings → no task page.
2. Three sessions minimum before committing a task family.
3. Tag families consistently. comparison tag belongs on every task page.
4. Leverage condition in your findings narrative — it's the experimental variable.
5. Subtasks need a parent_summary. A subtask without a parent is just a short note — keep it inline in its parent summary's narrative instead.
6. Always tail-call catalog after any task or subtask loop.