# massgen-refinery
A Claude Code plugin that brings [MassGen](https://github.com/massgen/massgen)'s
quality enforcement patterns to any project. Iterative quality loops where
specialized agents evaluate, improve, and verify work products until they
genuinely meet a high bar.
## What It Does
The `/refine` skill runs an autonomous quality loop: produce a deliverable,
evaluate it with a brutally honest critic agent, iterate until it's genuinely
good. Each round gets a fresh context to prevent approach anchoring.
```
/refine "Build a landing page for an AI startup"
```
That's it. The loop handles criteria generation, builder agents, regression
guards, evaluation, and iteration automatically.
---
## Getting Started
### Option A: Docker Sandbox (recommended)
Sandboxes run Claude Code in an isolated microVM. With the recommended tmux
launch helper, the refinement loop can auto-clear context between rounds and
run unattended. Requires [Docker Desktop 4.58+](https://www.docker.com/products/docker-desktop/).
```bash
# 1. Clone the plugin
git clone https://github.com/massgen/massgen-refinery.git
# 2. Create the sandbox with your project + plugin mounted
docker sandbox run claude \
~/my-project \
/path/to/massgen-refinery \
-- --plugin-dir /path/to/massgen-refinery
# 3. Authenticate (one time — persists across sessions)
# When Claude Code starts, type:
/login
# Then Ctrl+C to exit.
# 4. Install uv + massgen + AI coding CLIs (one time — persists)
# Run as separate calls to avoid connection timeout on long installs
docker sandbox exec claude-my-project bash -c \
"curl -LsSf https://astral.sh/uv/install.sh | sh && \
~/.local/bin/uv tool install massgen"
docker sandbox exec claude-my-project bash -c \
"npm install -g @openai/codex @google/gemini-cli"
docker sandbox exec claude-my-project bash -c \
"curl -fsSL https://gh.io/copilot-install | bash"
# 5. Reconnect with the sandbox tmux launch helper
docker sandbox exec -it claude-my-project bash -lc \
"bash /path/to/massgen-refinery/scripts/launch-claude-tmux.sh"
# 6. Use it
/massgen-refinery:refine "Build a landing page for an AI model router"
```
After setup, reconnect anytime with just step 5. Launching Claude without the
tmux helper still works, but context clearing falls back to manual selection.
```bash
docker sandbox ls # List all sandboxes
docker sandbox exec -it claude-my-project bash # Shell access
docker sandbox rm claude-my-project # Delete sandbox
```
### Option B: Local on macOS (no Docker)
```bash
# 1. Clone the plugin
git clone https://github.com/massgen/massgen-refinery.git
# 2. Run the macOS setup script (checks/installs tmux, uv, massgen via Homebrew)
bash massgen-refinery/scripts/setup-local-mac.sh
# 3. Set your API key (or use /login after launching)
export ANTHROPIC_API_KEY=sk-ant-api03-xxxxx
# 4. Launch with tmux helper (enables auto-clear between refinement rounds)
cd ~/my-project
bash /path/to/massgen-refinery/scripts/launch-claude-tmux.sh
# 5. Use it
/massgen-refinery:refine "Build a landing page for an AI model router"
```
With eval backend CLIs (Codex, Gemini, Copilot):
```bash
bash massgen-refinery/scripts/setup-local-mac.sh --with-eval-backends
```
---
## Authentication
### Docker Sandbox
API keys from your host shell do **not** pass into the sandbox. Instead:
1. Create the sandbox: `docker sandbox run claude ~/my-project`
2. When Claude Code starts, type `/login` and follow the prompts
3. Press Ctrl+C to exit
4. Reconnect — login persists across sessions
> The OAuth shortcut (`CLAUDE_CODE_OAUTH_TOKEN` / `claude setup-token`) does
> not reliably work with Docker sandboxes. Use `/login` instead.
### Authenticating other CLIs (one time — persists in named sandbox)
**Codex**
```bash
docker sandbox exec -it claude-my-project bash -c "codex login --device-auth"
# Prints a URL + code → open in browser, approve, done
docker sandbox exec -it claude-my-project bash -c "codex"
```
**Gemini** — prints an OAuth URL; open it in your browser, paste the code back
```bash
docker sandbox exec -it claude-my-project bash -c "gemini"
```
**Copilot**
```bash
COPILOT_TOKEN=$(security find-generic-password -s "copilot-cli" -w 2>/dev/null)
docker sandbox exec claude-my-project bash -c \
"echo 'export COPILOT_GITHUB_TOKEN=$COPILOT_TOKEN' >> ~/.bashrc"
docker sandbox exec -it claude-my-project bash -c "copilot"
```
### Without Docker
Set your API key in your shell:
```bash
export ANTHROPIC_API_KEY=sk-ant-api03-xxxxx
```
### Media tools (optional)
For image/video/audio generation and vision, set these via `export` or in a
`.env` file in your project root:
```
OPENAI_API_KEY=sk-... # Image gen, vision, audio
GOOGLE_API_KEY=AI... # Gemini vision, video, image
XAI_API_KEY=xai-... # Grok video gen
```
The plugin hooks detect keys from both environment variables and `.env` files.
---
## Git Hooks
This repo includes a `pre-commit` config with:
- basic hygiene checks (`trailing-whitespace`, `end-of-file-fixer`, YAML validation, merge-conflict detection)
- `detect-private-key`
- a local staged-diff scanner that blocks obvious API keys and tokens before commit
- a matching `pre-push` scanner that checks outgoing commits, so local snapshot branches or accidental later commits do not slip through push
Install it with:
```bash
uv tool install pre-commit
pre-commit install
pre-commit install --hook-type pre-push
```
Run it manually with:
```bash
pre-commit run --all-files
pre-commit run block-pushed-secrets --hook-stage pre-push
```
The secret scanner only inspects added git diff lines and never prints secret values back to the terminal.
---
## Skills
```bash
# Full quality refinement loop (main feature)
/massgen-refinery:refine "Build a landing page for an AI model router"
# With custom evaluation criteria
/massgen-refinery:refine "Build an API" --criteria "E1: REST conventions, E2: Error handling"
# Improve an existing deliverable
/massgen-refinery:refine "Rewrite the docs" --prior-answer ./docs/api.md
# Quick one-shot evaluation (no iteration)
/massgen-refinery:evaluate ./output.html
# Media skills
/massgen-refinery:read-media screenshot.png # Vision analysis
/massgen-refinery:image-generation "A sunset..." # Image generation
/massgen-refinery:video-generation "A timelapse..." # Video generation
/massgen-refinery:audio-generation "Narrate..." # Audio/TTS
```
---
## How `/refine` Works
```
┌─────────────────────────────────────────────────────┐
│ /refine │
│ │
│ 1. Setup │
│ ├─ git init (if needed) │
│ ├─ init_session → timestamped session dir │
│ └─ generate_eval_criteria → 3-7 criteria │
│ │
│ 2. Answer Production (inner loop) │
│ ├─ Build/improve deliverable │
│ ├─ Spawn builder(s) in parallel for large work │
│ ├─ Output-first verification (read_media) │
│ └─ Build → verify → fix → re-verify → repeat │
│ │
│ 3. Pre-Submission Gate │
│ └─ regression-guard → pass/fail/mixed verdict │
│ │
│ 4. Submit │
│ └─ new_answer → snapshot deliverables │
│ │
│ 5. Evaluation (parallel) │
│ ├─ round-evaluator → verdict + critique + tasks │
│ └─ trace-analyzer → process report + scores │
│ │
│ 6. Synthesize │
│ ├─ Read evaluation results │
│ ├─ Save process learnings to memory │
│ └─ converged → done, iterate → continue │
│ │
│ 7. Context Boundary (if iterating) │
│ └─ Plan mode → next-round handoff │
│ └─ Next iteration starts fresh │
└─────────────────────────────────────────────────────┘
```
## Agents
| Agent | Purpose | When Used |
|-------|---------|-----------|
| **round-evaluator** | Brutally honest critique + structured improvement spec | After each submission |
| **execution-trace-analyzer** | Process optimization (errors, effort, tools) | Parallel with evaluator |
| **builder** | Large artifact generation from prescriptive spec | During answer production |
| **regression-guard** | Verify improvements don't lose prior strengths | Before submission |
| **media-worker** | Background media generation/analysis | When media ops would block |
## MCP Tools
Provided by the `massgen` package:
- **quality-tools**: `init_session`, `generate_eval_criteria`, `submit_checklist`, `propose_improvements`, `reset_evaluation`
- **workflow-tools**: `new_answer`, `vote`
- **media-tools**: `generate_media`, `read_media`
## Key Principles
- **Output-first verification**: Run, interact, verify — don't just write
- **Default to iterate**: First drafts are 6s, not 8s
- **Rebuild, don't patch**: Use existing work as reference, not starting point
- **Breakthrough amplification**: Spread techniques that work to weaker components
- **No feature accumulation**: Features on mediocre foundation = mediocre result
## Context Rotation
Between refinement rounds, the plugin automatically clears context to prevent
approach anchoring and token bloat. When the evaluator's verdict is "iterate,"
Claude finishes a handoff plan and shows the plan approval dialog. When Claude
is launched through the recommended sandbox tmux helper, a lightweight watcher
polls the Claude pane and presses `Enter` when that dialog appears with the
default "Yes, clear context" option selected.
**Requires tmux inside the sandbox and the tmux helper launch command** for
automatic clearing. Without that watcher, the plan approval UI appears and you
select option 1 manually.
This technique is adapted from the
[VNX Orchestration System](https://github.com/Vinix24/vnx-orchestration)'s
context rotation pattern, which uses tmux keystroke injection to manage context
boundaries in multi-agent workflows.
---
## Multi-Agent Evaluation (Experimental)
> **Status**: Coming soon. The single-agent `/refine` loop above is the main
> supported workflow. Multi-agent evaluation via MassGen is experimental.
The `/refine` skill supports opt-in multi-model evaluation where multiple LLM
backends (GPT, Gemini, Claude, Grok) evaluate your deliverable in parallel and
reach consensus:
```bash
/massgen-refinery:refine "Build a landing page" \
--eval-backends openai/gpt-5.4 gemini/gemini-3-flash-preview
```
Or run a standalone multi-agent session:
```bash
/massgen-refinery:massgen-run "Explain quantum computing" \
--models openai/gpt-5.4 gemini/gemini-3-flash-preview
```
**`/team-massgen-run`**: Multi-model orchestration via MassGen step mode. The
lead session launches each backend as a background process, tracks answers/votes
via a shared session directory, and synthesizes the consensus winner.
> Agent Teams integration (experimental) is in progress — currently blocked by a
> known limitation where teammate background tasks get killed on idle
> ([#17764](https://github.com/anthropics/claude-code/issues/17764),
> [#28875](https://github.com/anthropics/claude-code/issues/28875),
> [#29271](https://github.com/anthropics/claude-code/issues/29271)).
> The current approach uses lead-managed background tasks instead.
Requires API keys for each backend. Falls back to the normal single-model
round-evaluator if massgen or API keys are unavailable.
See `skills/massgen-run/` for details.
---
## Development Setup (massgen from local source)
For experimenting with a local massgen checkout instead of the published package.
### 1. Update plugin.json for local source
Edit `.claude-plugin/plugin.json` to point MCP servers at your local massgen
using `uv run --directory` instead of `uvx`:
```json
{
"quality-tools": {
"type": "stdio",
"command": "uv",
"args": ["run", "--directory", "/path/to/massgen", "python", "-m", "massgen.mcp_tools.standalone.quality_server"]
},
"workflow-tools": {
"type": "stdio",
"command": "uv",
"args": ["run", "--directory", "/path/to/massgen", "python", "-m", "massgen.mcp_tools.standalone.workflow_server"]
},
"media-tools": {
"type": "stdio",
"command": "uv",
"args": ["run", "--directory", "/path/to/massgen", "python", "-m", "massgen.mcp_tools.standalone.media_server"]
}
}
```
### 2. Create the sandbox (with local source mounted)
Mount your project, massgen source, and the plugin — all read-write since
`uv sync` needs to create a venv in the massgen directory:
```bash
docker sandbox run claude \
~/my-project \
/path/to/massgen \
/path/to/massgen-refinery
```
### 3. Install uv and sync massgen (one time — persists)
```bash
docker sandbox exec -it claude-my-project bash
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.bashrc
# Sync massgen dependencies
cd /path/to/massgen && uv sync
exit
```
### 4. Launch with the plugin
```bash
# With tmux helper (recommended — enables auto-clear between rounds)
docker sandbox exec -it claude-my-project bash -lc \
"bash /path/to/massgen-refinery/scripts/launch-claude-tmux.sh"
# Without tmux (no auto-clear)
docker sandbox run claude-my-project -- \
--plugin-dir /path/to/massgen-refinery
```
### Local dev without Docker
```bash
# With tmux helper (recommended)
bash /path/to/massgen-refinery/scripts/launch-claude-tmux.sh
# Without tmux
claude --plugin-dir /path/to/massgen-refinery
```
Reload after plugin changes without restarting:
```
/reload-plugins
```
## License
MIT