| name | wandb-project-analyst |
| description | Analyze live W&B Models and Weave project data using the W&B MCP server, including runs, histories, configs, traces, evaluations, artifacts, registries, comparisons, diagnoses, and reports. Use when the user asks what happened in a W&B project, which run or model is best, why a run or trace failed, how evaluations performed, what changed between versions, or to save an evidence-backed analysis as a W&B Report. |
W&B Project Analyst
Use W&B MCP as the primary interface for reading and analyzing data already stored in W&B. Ground conclusions in retrieved run, trace, evaluation, and artifact evidence.
Operating Rules
- Identify the entity, project, time range, metric, and comparison target from the request or discover missing scope with MCP.
- Probe unfamiliar Models projects and infer unfamiliar Weave schemas before querying them.
- Use summary-level trace data for broad analysis and full trace payloads only for a small set of relevant traces.
- Prefer purpose-built history, comparison, diagnosis, and evaluation tools over manual broad queries.
- Distinguish observed facts from inferences and recommendations. Include run ids, trace ids, artifact versions, timestamps, or W&B links when available.
- Confirm full retrieval for broad rankings instead of assuming the newest page is complete.
- Do not persist an analysis run or create a W&B Report unless the user explicitly asks to save or publish the findings.
- Never request, print, or store the value of
WANDB_API_KEY.
- Use the official
wandb-primary skill when available for W&B SDK patterns or as a fallback when an MCP capability is unavailable.
Workflow
- Normalize the analytical question and target W&B project.
- Verify MCP access. If the entity or project is unknown, discover it before proceeding.
- Select the product surface:
- Models runs: probe keys, then query runs, histories, comparisons, or diagnoses.
- Weave traces: infer the schema, then count or query traces and resolve roots when needed.
- Evaluations: use evaluation summaries before raw trace analysis.
- Artifacts and Registry: list, inspect, and compare explicit versions.
- Retrieve only the fields and time range needed to answer the question.
- Compute or synthesize the answer, checking denominators, missing data, and baseline comparability.
- Return evidence, caveats, and the next useful investigation.
- If explicitly requested, log computed analysis values and create a W&B Report.
Product Routing
For Models questions, prefer this sequence:
probe project -> query runs or history -> compare or diagnose -> summarize
For Weave questions, prefer this sequence:
infer schema -> count or query summary traces -> resolve roots -> inspect selected full traces
For evaluation questions, start with the evaluation summary and inspect raw traces only when the aggregate result is insufficient.
References
- Read
references/mcp-analysis-workflows.md for concrete W&B MCP tool chains, evidence rules, and setup fallback.
Demo Prompts
Use a real accessible entity and project:
Analyze the top five runs by eval/primary_score in ENTITY/PROJECT and compare their configs.
Find failed Weave traces in ENTITY/PROJECT from the last 24 hours and group the root workflows by failure mode.
Summarize the latest evaluation in ENTITY/PROJECT and identify the weakest scorer.
Compare artifact versions model:v2 and model:v3 and explain what changed.
Response Shape
Lead with the answer, then list supporting evidence, caveats or missing data, and recommended next action. Include a W&B Report link only when a report was explicitly requested and successfully created.
