ade-bench-plugin

Install or verify the ade-bench harness, the Python project that actually runs the benchmark tasks this plugin generates. Use when ade-bench isn't yet on the user's machine, when a generated task fails because the harness is missing, or when the user explicitly asks to set up ade-bench. Also invoked automatically by plan-tasks and create-task when they detect ade-bench is missing.

Scan a dbt project and generate ADE-Bench benchmark tasks. Analyzes models, proposes bug-injection scenarios by difficulty, and outputs complete task scaffolding (task.yaml, patches, setup/solution scripts, test SQL). Use when you want to benchmark AI agents against your own dbt project and data.

Interactively plan and generate ADE-Bench benchmark tasks from a dbt project. Explores the codebase with the user in a pair-planning loop — understanding the domain, reasoning about what makes good benchmarks, and building up a task plan incrementally. Use when you want high-quality, tailored benchmark tasks for your own dbt project.