experiment

name	experiment
description	Run experiments proposed by Research-Intel reports. Parses the json:intel-data block from V0 reports, presents ranked experiments, and implements the chosen one in a git worktree.
version	1.0.0
tags	["research","experiments","research-intel","worktree"]

Run Research-Intel Experiments

Read a Research-Intel V0 report, present proposed experiments ranked by impact/effort, and implement the user's choice in an isolated git worktree.

Trigger

• /experiment — find the latest report automatically
• /experiment <direction> — filter to a specific direction (e.g., procedural-memory)
• /experiment <file-path> — use a specific report file

Workflow

Step 1: Find the Latest Report

Check these paths in order to find V0 reports:

1. ~/.research-intel/reports/ — V0 default output (subdirectories are directions)
2. .research-intel/reports/ — V0 local output (relative to cwd)

Within each location, subdirectories represent research directions (e.g., procedural-memory/). Each contains dated markdown files (e.g., 2026-03-12.md).

If the user passed an argument:

• If it looks like a file path (contains / or ends in .md), use it directly.
• Otherwise, treat it as a direction name and filter to that subdirectory.

Finding the most recent report:

• Glob for *.md files across all direction subdirectories in the report locations above.
• Sort by filename (filenames are dates like YYYY-MM-DD.md), pick the most recent.
• If no reports found, tell the user: "No Research-Intel reports found. Run research-intel first, or pass a report path: /experiment path/to/report.md"

Step 2: Parse the json:intel-data Block

Read the report file and extract the machine-readable JSON block:

1. Find the content between ```json:intel-data and the closing ```
2. Parse it as JSON
3. Extract recommended_experiments (array of experiment objects) and papers (array of paper objects)

If no json:intel-data block is found, tell the user: "This report doesn't have a machine-readable data block. It may be from an older version of Research-Intel. V0 reports include a json:intel-data block at the bottom."

Expected JSON structure:

{
  "direction": "string",
  "date": "YYYY-MM-DD",
  "total_papers_scanned": number,
  "papers": [
    {
      "paper_id": "string",
      "title": "string",
      "relevance_score": number,
      "suggested_experiment": "string",
      ...
    }
  ],
  "recommended_experiments": [
    {
      "description": "string",
      "inspired_by": "paper_id (Paper Title)",
      "effort": "low|medium|high",
      "impact": "low|medium|high|medium to high",
      "rationale": "string"
    }
  ],
  "themes": [...],
  "field_trends": [...]
}

Step 3: Present Experiments

Sort experiments for display:

• Impact descending: HIGH > MEDIUM TO HIGH > MEDIUM > LOW
• Within same impact, effort ascending: LOW < MEDIUM < HIGH

Present as a numbered list. For each experiment:

## Experiments from: <direction> (<date>)

1. [HIGH impact / MEDIUM effort] <first sentence of description>
   Inspired by: <paper title from inspired_by field>
   Rationale: <rationale>

2. [HIGH impact / HIGH effort] ...
   ...

After the list, also mention if any papers have a suggested_experiment field that adds useful context beyond what's in recommended_experiments.

Then ask:

"Which experiment would you like to try? (number, or 'skip')"

Step 4: Human Review Before Execution

When the user picks an experiment, present a clear summary:

## Experiment: <short title derived from description>

**Hypothesis:** <extract the testable hypothesis from description + rationale>

**What changes:**
- <list specific files/modules that will be modified, extracted from the description>

**Metrics to measure:**
- <extract metrics from description, e.g., "MAP on LOO benchmark", "runtime reduction">

**Baseline:**
- <extract baseline numbers from description if mentioned>

**Inspired by:** <paper title> (<paper URL from papers array>)

Then ask:

"Create a worktree and implement this? [yes / edit / skip]"

• yes — proceed to Step 5
• edit — let user modify the plan, then re-confirm
• skip — return to experiment list or end

This is a mandatory gate. Never skip human approval.

Step 5: Worktree + Implementation

Create the worktree:

# Generate a short slug from the experiment description (e.g., "task-aware-retrieval-cues")
git worktree add .worktrees/exp-<slug> -b exp/<slug>

Then work inside the worktree to implement the minimum viable experiment (MVE):

1. Read the relevant source files mentioned in the experiment description
2. Implement the minimum changes needed to test the hypothesis — no over-engineering
3. Add a brief comment at the top of modified files noting the experiment branch
4. Run the experiment — execute tests, benchmarks, or evaluation scripts as described
5. Capture results — metrics, outputs, any errors

Throughout implementation, keep the user informed at milestones:

• "Created worktree at .worktrees/exp-<slug>"
• "Implementing changes to <file>..."
• "Running evaluation..."
• "Results ready."

Step 6: Decision

Present results clearly:

## Results: <experiment name>

**Branch:** exp/<slug>
**Worktree:** .worktrees/exp-<slug>

**Metrics:**
- <metric>: <value> (baseline: <baseline value>)
- ...

**Observations:**
- <key findings>

**Verdict:** <brief assessment — did the hypothesis hold?>

Then ask:

"What next? [merge / iterate / discard]"

• merge — merge the experiment branch into the current branch, clean up worktree:

  git merge exp/<slug>
  git worktree remove .worktrees/exp-<slug>
  git branch -d exp/<slug>
  

• iterate — keep the worktree, continue making changes
• discard — remove worktree and delete branch:

  git worktree remove .worktrees/exp-<slug>
  git branch -D exp/<slug>
  

Guidelines

1. Always parse V0 format only — look for json:intel-data blocks. Do not attempt to parse the markdown prose of older report formats.
2. Never execute without approval — Step 4 is mandatory. Always show what will change and get explicit confirmation.
3. Minimum viable experiment — implement the smallest change that tests the hypothesis. Don't refactor surrounding code or add unrelated improvements.
4. Direction-aware — if reports span multiple directions, show which direction each experiment came from.
5. Worktree isolation — all experiment code goes in a git worktree. Never modify the working branch directly.
6. Link back to papers — when presenting experiments, include paper titles and URLs so the user can read the source material.
7. Preserve existing benchmarks — when running evaluations, always compare against the baseline numbers mentioned in the experiment description.
8. Log results — if a research_logs.md exists in the project root, append experiment results there using the research-experiment-logger pattern.