dogfood

name	dogfood
description	Run a full dogfooding session against a codegraph release (stable from npm, or dev from GitHub release tarballs) — install, test all commands, compare engines, find bugs, and write a report
argument-hint	<version>
allowed-tools	Bash, Read, Write, Glob, Grep, Task, Edit

Dogfooding Session for @optave/codegraph@$ARGUMENTS

You are running a comprehensive dogfooding session for codegraph v$ARGUMENTS. Your goal is to install the published package, exercise every feature, compare engines, find bugs, and produce a structured report.

Reference: Read generated/dogfood/DOGFOOD-REPORT-2.1.0.md and generated/dogfood/DOGFOOD_REPORT_v2.2.0.md (if present) for the format and depth expected. Match or exceed that quality.

---

Phase 0 — Setup

1. Create a temporary working directory (e.g., /tmp/dogfood-$ARGUMENTS or a system temp).

2. Install the package — the method depends on whether this is a stable or dev version:

   Stable release (version does NOT contain -dev., e.g. 2.5.1):

   npm init -y && npm install @optave/codegraph@$ARGUMENTS
   

   Dev build (version contains -dev., e.g. 3.1.6-dev.12): Dev builds are not published to npm. They are attached as tarballs to GitHub pre-releases. Install from the GitHub release:

   npm init -y
   
   # Main package
   npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-$ARGUMENTS.tgz
   
   # Platform-specific native binary (pick one for the current platform):
   # Windows x64:
   npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-win32-x64-msvc-$ARGUMENTS.tgz
   # macOS ARM:
   npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-darwin-arm64-$ARGUMENTS.tgz
   # macOS x64:
   npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-darwin-x64-$ARGUMENTS.tgz
   # Linux x64:
   npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-linux-x64-gnu-$ARGUMENTS.tgz
   

   Tip: To find the latest dev version, run:

   gh release list --repo optave/codegraph --json tagName --jq '.[] | select(.tagName | startswith("dev-v")) | .tagName' | head -1 | sed 's/^dev-v//'
   

3. Verify the install: npx codegraph --version should print $ARGUMENTS.

4. Verify the native binary installed. The native Rust addon is delivered as a platform-specific optional dependency. Check that the correct one exists:

   ls node_modules/@optave/codegraph-*/
   

   Expected packages by platform:

   • Windows x64: @optave/codegraph-win32-x64-msvc
   • macOS ARM: @optave/codegraph-darwin-arm64
   • macOS x64: @optave/codegraph-darwin-x64
   • Linux x64: @optave/codegraph-linux-x64-gnu

   If the native package is missing or a different version than $ARGUMENTS, file it as a bug — the optionalDependencies in package.json may have a pinned version mismatch. Verify by checking:

   node -e "const p = require('@optave/codegraph/package.json'); console.log(p.optionalDependencies)"
   

   Then confirm the native engine actually loads:

   npx codegraph info
   

   This should report engine: native. If it falls back to wasm, record why.

5. Record: platform, OS version, Node version, native binary package name + version, engine reported by info.

6. Update the native binary in the source repo. The source repo's node_modules may have a stale native binary from a previous release. Since Phases 1–4 run commands against the source repo (using --db or directly), a version mismatch produces wrong results for all phases, not just benchmarks. Check and fix immediately:

   # In the codegraph source repo:
   node -e "const p = require('@optave/codegraph-win32-x64-msvc/package.json'); console.log('Source repo native binary:', p.version)"
   

   If the version does not match $ARGUMENTS, install the correct binary now:

   • Stable release: npm install @optave/codegraph-win32-x64-msvc@$ARGUMENTS (adjust platform suffix)
   • Dev build: npm install https://github.com/optave/ops-codegraph-tool/releases/download/dev-v$ARGUMENTS/optave-codegraph-win32-x64-msvc-$ARGUMENTS.tgz

   Verify with npx codegraph info in the source repo. Revert package.json / package-lock.json changes after the session (do not commit them on the fix branch).

7. Do NOT rebuild the graph yet. The first phase tests commands against the codegraph source repo without a pre-existing graph.

---

Phase 1 — Cold Start (No Graph)

Using the installed binary (npx codegraph from the temp dir, pointed at the codegraph source repo):

1. Self-discover all commands: Run npx codegraph --help and extract every command and subcommand (including registry list|add|remove|prune).
2. Run each command before running build. Record which ones:
   • Fail gracefully with a helpful message (PASS)
   • Crash with a stack trace (BUG)
   • Silently return empty/wrong results without warning (BUG)
3. Then run npx codegraph build <path-to-codegraph-repo> and record: file count, node count, edge count, time taken, engine used.

---

Phase 2 — Full Command Sweep

After the graph is built, exercise every command and subcommand. Discover the list from --help but use this reference to ensure thorough flag coverage:

Query commands

Test each with -j/--json and -T/--no-tests where supported:

Command	Key flags to exercise
query <name>	--depth <n>, --db <path>
impact <file>
map	-n/--limit <number>
stats
deps <file>
fn-impact <name>	--depth <n>, -f/--file, -k/--kind
context <name>	--depth <n>, -f/--file, -k/--kind, --no-source, --include-tests
audit <target>	test with both a file path and a function name; replaces the old explain command
where <name>	also test where -f <file> for file-overview mode
diff-impact [ref]	--staged, test vs main, vs HEAD, and with no arg (unstaged)
cycles	--functions for function-level cycles
structure [dir]	--depth <n>, --sort cohesion|fan-in|fan-out|density|files
triage	--level file|function|directory, -n/--limit, --json

Export commands

Command	Flags
export	-f dot, -f mermaid, -f json, --functions, -o <file>

Embedding & search

Command	Flags
models	(no flags)
embed [dir]	-m minilm (use this — jina-code default requires HF auth)
search <query>	-n/--limit, --min-score, -k/--kind, --file <pattern>, multi-query with ; separator, --rrf-k

Infrastructure commands

Command	Flags
info
--version
watch [dir]	start, verify it detects a file change, then Ctrl+C
registry list	-j/--json
registry add <dir>	-n/--name <custom>
registry remove <name>
registry prune	--ttl <days>
mcp	initialize via JSON-RPC stdin, verify tool list response

Edge cases to test

Scenario	Expected
Non-existent symbol: query nonexistent	Graceful "No results" message
Non-existent file: deps nonexistent.js	Graceful "No file matching" message
Non-existent function: fn-impact nonexistent	Graceful message
structure .	Should work (was a bug in v2.2.0 — verify fix)
--json on every command that supports it	Valid JSON output
--no-tests effect: compare counts with/without	Test file count should drop
--kind with invalid kind	Graceful error or ignored
--verbose on build	Should show per-file parsing details
build --no-incremental	Force full rebuild
search with no embeddings	Should warn, not crash
embed then search with dimension mismatch model	Should warn about mismatch
Pipe output: codegraph map --json | head -1	Clean JSON, no status messages in stdout
Embed → rebuild → search pipeline: embed -m minilm, modify a file, build, search "build graph" without re-embedding	Results should still return; stale embeddings should be detected or warned about
Watch mode lifecycle: start watch, modify a file, run a query to verify update, then Ctrl+C	Watcher detects change, graph reflects it, graceful shutdown (journal flush, no crash)

---

Phase 3 — Rebuild & Staleness Testing

Test that incremental rebuilds, full rebuilds, and cross-feature state remain consistent. Codegraph uses three-tier change detection: journal → mtime+size → content hash.

1. Incremental no-op: Run build again with no file changes. It should report "Graph is up to date" and touch nothing. Verify node/edge counts are identical.
2. Incremental with change: Touch or slightly modify one source file, run build again. Verify with --verbose:
   • Only the changed file is re-parsed
   • Node IDs for unchanged symbols remain stable
   • Edge counts are consistent
   • The journal (.codegraph/ directory) tracks the change
3. Force full rebuild: Run build --no-incremental. Compare node/edge counts with the incremental result — they should match exactly.
4. Embed then rebuild: Run embed --model minilm, then run build again (even with no changes). After the rebuild:
   • Run search "build graph" — do results still return? If embeddings reference stale node IDs, search will return 0 results
   • Compare embedding node_ids against actual node IDs in the graph (use --json outputs)
5. Embed, modify, rebuild, search: Modify a file, build again (incremental), then search without re-running embed. This is the most likely path to stale embeddings. Record whether results are correct, empty, or wrong.
6. Full rebuild after incremental: Delete .codegraph/graph.db, rebuild from scratch, then verify search still works (it shouldn't — embeddings should be gone too, or the tool should warn).
7. Watch mode integration: Start watch, modify a file in another terminal, verify the watcher detects the change and incrementally updates (check output). Then run a query to verify the graph reflects the change. Stop the watcher with Ctrl+C and verify graceful shutdown (journal flush).
8. Revert any file modifications before continuing.

---

Phase 4 — Engine Comparison

1. Build with --engine wasm: record nodes, edges, time.
2. Build with --engine native: record nodes, edges, time.
3. Compare in a table: node count, edge count, function count, call edges, call confidence (from stats), graph quality score.
4. Note any significant parity gaps (>5% difference in any metric).
5. Run the same set of queries with both engines and flag any result differences:
   • fn buildGraph — compare callers/callees
   • context parseFileAuto — compare source extraction
   • cycles --functions — compare cycle detection
   • stats --json — full metric comparison
   • triage --json — compare risk rankings

---

Phase 4b — Performance Benchmarks

Run all four benchmark scripts from the codegraph source repo and record results. These detect performance regressions between releases.

Benchmark	Script	What it measures	When it matters
Build	node scripts/benchmark.js	Build speed (native vs WASM), query latency	Always
Incremental	node scripts/incremental-benchmark.js	Incremental build tiers, import resolution throughput	Always
Query	node scripts/query-benchmark.js	Query depth scaling, diff-impact latency	Always
Embedding	node scripts/embedding-benchmark.js	Search recall (Hit@1/3/5/10) across models	Always

Pre-flight: verify native binary version

If you followed Phase 0 step 6, the source repo's native binary should already match $ARGUMENTS. Double-check before benchmarking:

node -e "const p = require('@optave/codegraph-win32-x64-msvc/package.json'); console.log('Native binary version:', p.version)"

If it doesn't match, go back to Phase 0 step 6 and fix it — all benchmark results with a stale binary are invalid.

Why this matters: The native engine computes complexity metrics during the Rust parse phase. If the binary is from an older release that lacks this, the complexity phase silently falls back to WASM — inflating native complexity time by 50-100x and making native appear slower than WASM.

Running benchmarks

1. Run all four from the codegraph source repo directory.
2. Record the JSON output from each.
3. Compare with the previous release's numbers in generated/benchmarks/BUILD-BENCHMARKS.md (build benchmark) and previous dogfood reports.
4. Flag any regressions:
   • Build time per file >10% slower → investigate
   • Query latency >2x slower → investigate
   • Embedding recall (Hit@5) drops by >2% → investigate
   • Incremental no-op >10ms → investigate
5. Sanity-check the complexity phase: If native complexityMs is higher than WASM complexityMs, the native binary is likely stale — go back to the pre-flight step.
6. Include a Performance Benchmarks section in the report with tables for each benchmark.
7. Build phase breakdowns must show both engines side by side. The benchmark script outputs phases (full build) and oneFilePhases (1-file rebuild) for each engine. Present them in a single table with columns for both engines so readers can compare per-phase performance at a glance — don't show only one engine. Flag phases where the ratio is anomalous (e.g., incremental slower than full).

Note: The native engine may not be available in the dev repo (no prebuilt binary in node_modules). Record WASM results at minimum. If native is available, record both.

IMPORTANT: If your bug-fix PR touches code covered by a benchmark (builder.js, parser.js, queries.js, resolve.js, db.js, embedder.js, journal.js), you must run the relevant benchmarks before and after your changes and include the comparison in the PR description.

---

Phase 5 — Changes Since Last Release

1. Read CHANGELOG.md to identify what changed in v$ARGUMENTS vs the previous version.
2. Read package.json for the previous version tag.
3. For every feature added and bug fixed in this release, write a targeted test:
   • Verify the feature works as described
   • Verify the bug is actually fixed
   • Try to break it with edge cases
4. Present results in a "Release-Specific Tests" table.

---

Phase 6 — Thinking Space

Before writing the report, stop and think about:

• What testing approaches am I missing?
• Cross-command pipelines: Have I tested build → embed → search → modify → build → search? Have I tested watch detecting changes then diff-impact?
• MCP server: Have I tested the mcp command? Initialize via JSON-RPC on stdin, send tools/list, verify all 35 tools are present (34 in single-repo mode; 35 with list_repos in multi-repo). Test single-repo mode (default — list_repos should be absent, no repo parameter on tools) vs --multi-repo mode.
• Programmatic API: Have I tested require('@optave/codegraph') or import from index.js? Key exports to verify: buildGraph, loadConfig, contextData, explainData, whereData, fnDepsData, fnImpactData, diffImpactData, statsData, queryNameData, rolesData, auditData, triageData, complexityData, EXTENSIONS, IGNORE_DIRS, EVERY_SYMBOL_KIND.
• Config options: Have I tested .codegraphrc.json? Create one with include/exclude patterns, custom aliases, build.incremental: false, query.defaultDepth, search.defaultMinScore. Verify overrides work.
• Env var overrides: CODEGRAPH_LLM_PROVIDER, CODEGRAPH_LLM_API_KEY, CODEGRAPH_LLM_MODEL, CODEGRAPH_REGISTRY_PATH.
• Credential resolution: apiKeyCommand in config — does it shell out via execFileSync correctly? Test with a simple echo command.
• Multi-repo registry flow: registry add ., registry list, mcp --repos <name>, registry remove <name>, registry prune --ttl 0.
• Concurrent usage: Two builds at once, build while watching.
• Different repo: Have I tested on a repo besides codegraph itself? Try a small open-source project.
• False positive filtering: Does stats report false positives? Are FALSE_POSITIVE_NAMES (run, get, set, init, main, etc.) filtered from high-caller warnings?
• Symbol kinds: Test --kind with all valid kinds: function, method, class, interface, type, struct, enum, trait, record, module.
• Database migrations: If testing an upgrade path (older graph.db → new version), do schema migrations (v1→v4) run correctly?
• What would a real user hit that I haven't simulated?

Add any additional tests you identify here and run them before writing the final report.

---

Phase 7 — File Bugs & Fix

For each bug found during testing:

7a. Check for duplicates

gh issue list --repo optave/codegraph --state open --search "<bug title keywords>"

If a matching issue already exists, skip creating a new one. Add a comment with your findings if you have new information.

7b. Open an issue

For each new bug, create a GitHub issue:

gh issue create --repo optave/codegraph \
  --title "bug: <concise title>" \
  --label "bug,dogfood" \
  --body "$(cat <<'ISSUE_EOF'
## Found during dogfooding v$ARGUMENTS

**Severity:** <Critical | High | Medium | Low>
**Command:** `codegraph <command that failed>`

## Reproduction

\`\`\`bash
<exact commands to reproduce>
\`\`\`

## Expected behavior

<what should happen>

## Actual behavior

<what actually happens — include output/stack traces>

## Root cause

<analysis if known>

## Suggested fix

<approach if known>
ISSUE_EOF
)"

Record the issue number for each bug.

7c. Fix and submit PRs

For each bug you can fix in this session:

1. Create a branch: git checkout -b fix/dogfood-<short-description> main
2. Implement the fix.
3. Run npm test to verify no regressions.
4. Run npm run lint to verify code style.
5. Run benchmarks before and after if your fix touches code covered by a benchmark (see Phase 4b table). Include the comparison in the PR body.
6. Commit with a message referencing the issue:

   fix(<scope>): <description>
   
   Closes #<issue-number>
   

   The Closes #N footer tells GitHub to auto-close the issue when the PR merges.
7. Push and open a PR. If benchmarks were run, include them in the body:

   gh pr create --base main \
     --title "fix(<scope>): <description>" \
     --body "$(cat <<'PR_EOF'
   ## Summary
   <what was wrong and how this fixes it>
   
   ## Found during
   Dogfooding v$ARGUMENTS — see #<issue-number>
   
   ## Benchmark results
   <before/after table if applicable — see Phase 4b>
   
   ## Test plan
   - [ ] <how to verify the fix>
   PR_EOF
   )"
   

8. Return to the main working branch before continuing to the next bug.

If a bug is too complex to fix in this session, leave the issue open and note it in the report.

7d. Green path — no bugs found

If the entire dogfooding session finds zero bugs, the release is validated. Update the native binary version pins in the main repository to match:

1. In the codegraph repo (not the temp dir), edit package.json optionalDependencies to pin all @optave/codegraph-* packages to $ARGUMENTS.
2. Run npm install to update the lockfile.
3. Create a PR to update the native binary pins:

   git checkout -b chore/pin-native-binaries-v$ARGUMENTS main
   git add package.json package-lock.json
   git commit -m "chore: pin native binaries to v$ARGUMENTS after clean dogfood"
   gh pr create --base main \
     --title "chore: pin native binaries to v$ARGUMENTS" \
     --body "Validated in dogfooding session — zero bugs found."
   

This signals that v$ARGUMENTS has been manually verified end-to-end.

---

Phase 8 — Cleanup

1. Delete the temporary directory.
2. Confirm no artifacts were left behind.

---

Phase 9 — Report

Write the report to generated/dogfood/DOGFOOD_REPORT_v$ARGUMENTS.md with this structure:

# Dogfooding Report: @optave/codegraph@$ARGUMENTS

**Date:** <today>
**Platform:** <OS, arch, Node version>
**Native binary:** <package name and version, or "not available">
**Active engine:** <auto-detected engine>
**Target repo:** codegraph itself (<file count> files)

---

## 1. Setup & Installation
<Install results, native binary verification, any issues>

## 2. Cold Start (Pre-Build)
<Table of commands tested without a graph>

## 3. Full Command Sweep
<Table: Command | Status | Notes>

### Edge Cases Tested
<Table: Scenario | Result>

## 4. Rebuild & Staleness
<Incremental rebuild results>
<Embed-rebuild-search pipeline results>
<Watch mode results>

## 5. Engine Comparison
<Table: Metric | Native | WASM | Delta>
<Analysis of parity gaps>
<Per-query comparison results>

## 6. Release-Specific Tests
<What changed in this version>
<Table: Feature/Fix | Test | Result>

## 7. Additional Testing
<MCP server testing results>
<Programmatic API testing results>
<Config/env var testing results>
<Multi-repo registry testing results>
<Any other tests from Phase 6 thinking space>

## 8. Performance Benchmarks

### Build Benchmark
| Metric | WASM | Native | Speedup |
|--------|------|--------|---------|
| Full build | ... | ... | ... |
| No-op rebuild | ... | ... | ... |
| 1-file rebuild | ... | ... | ... |
| Query time | ... | ... | ... |

### Build Phase Breakdown
| Phase | WASM Full | Native Full | WASM 1-File | Native 1-File |
|-------|-----------|-------------|-------------|---------------|
| Setup | ... | ... | ... | ... |
| Parse | ... | ... | ... | ... |
| Insert | ... | ... | ... | ... |
| Resolve | ... | ... | ... | ... |
| Edges | ... | ... | ... | ... |
| Structure | ... | ... | ... | ... |
| Roles | ... | ... | ... | ... |
| AST | ... | ... | ... | ... |
| Complexity | ... | ... | ... | ... |
| CFG | ... | ... | ... | ... |
| Dataflow | ... | ... | ... | ... |
| Finalize | ... | ... | ... | ... |

<Flag any phases where WASM/Native ratio is anomalous or incremental is slower than full>

### Query Benchmark
<Table: query | WASM | Native>

### Incremental Benchmark
<Table: metric | WASM | Native>

### Benchmark Assessment
<Bullet-point analysis of results, regressions, and anomalies>

## 9. Bugs Found
### BUG 1: <title> (<severity>)
- **Issue:** #<number> (link)
- **PR:** #<number> (link) or "open — too complex for this session"
- **Symptoms:**
- **Root cause:**
- **Fix applied:**

## 10. Suggestions for Improvement
### 10.1 <suggestion>
### 10.2 <suggestion>

## 11. Testing Plan

### General Testing Plan (Any Release)
<Checklist of standard tests every release should pass>

### Release-Specific Testing Plan (v$ARGUMENTS)
<Focused checklist based on CHANGELOG changes>

### Proposed Additional Tests
<Tests you thought of in Phase 6 that should be added to future dogfooding>

## 12. Overall Assessment
<Summary paragraph>
<Rating: X/10 with justification>

## 13. Issues & PRs Created
| Type | Number | Title | Status |
|------|--------|-------|--------|
| Issue | #N | ... | open / closed via PR |
| PR | #N | ... | open / merged |

---

Phase 10 — Commit the Report

The dogfood report must be committed to the repository — do not leave it as an untracked file.

1. If bug-fix PRs were created during Phase 7: Add the report to the first PR's branch:

   git checkout <first-pr-branch>
   git add generated/dogfood/DOGFOOD_REPORT_v$ARGUMENTS.md
   git commit -m "docs: add dogfood report for v$ARGUMENTS"
   git push
   

2. If no PRs were created (zero bugs / green path): Create a dedicated PR for the report:

   git checkout -b docs/dogfood-report-v$ARGUMENTS main
   git add generated/dogfood/DOGFOOD_REPORT_v$ARGUMENTS.md
   git commit -m "docs: add dogfood report for v$ARGUMENTS"
   git push -u origin docs/dogfood-report-v$ARGUMENTS
   gh pr create --base main \
     --title "docs: add dogfood report for v$ARGUMENTS" \
     --body "Dogfooding report for v$ARGUMENTS. See generated/dogfood/DOGFOOD_REPORT_v$ARGUMENTS.md for full details."
   

3. Verify the report file appears in the PR diff before moving on.

---

Rules

• Be thorough but honest. Don't inflate the rating.
• If codegraph crashes or produces wrong results when analyzing itself, file it as a bug — don't work around it.
• Report the raw truth. A dogfood report that finds 0 bugs is suspicious.
• Include exact command invocations and outputs for any bugs found.
• The report should be useful to a developer who wasn't in the session.

Self-Improvement

This skill lives at .claude/skills/dogfood/SKILL.md in the codegraph repo. If during the dogfooding session you discover that this skill is missing steps, has outdated command references, or could be improved — you are encouraged to edit it. Fix inaccuracies, add missing test cases, update flag lists, and improve the phase instructions so the next dogfood run benefits from what you learned. Commit any skill improvements alongside the dogfood report.