セッションtranscriptを分析してドキュメントの効果を評価する。 doc参照がAgentの行動を改善したか、Context浪費か、腐敗して有害かを判定。 last-validated がないドキュメントにもgit履歴ベースで対応。 HTML reportでper-doc健全性、Context budget、freshness、推奨アクションを表示。 Use when user says "audit docs", "docs audit", "run docs-auditor", "ドキュメント監査", "doc ROI", "ドキュメントの効果分析".
disable-model-invocation
true
Docs Auditor
Transcript-based documentation effectiveness analysis. Evaluates whether
reading docs actually improves agent behavior or wastes context tokens.
Generates an interactive HTML report with per-doc health, freshness dashboard,
context budget, and actionable recommendations.
Prerequisites
pip install tiktoken (optional — falls back to character-based estimation)
No external API keys required. Analysis uses Claude sub-agents.
Workflow
Run all steps sequentially. The coordinator (you) manages data flow between
scripts and sub-agents.
Step 1: Initial Questions
Ask the user for the report language using AskUserQuestion:
"レポートの言語は? (e.g. 日本語, English)"
Default to the user's conversation language if not specified.
Pass the language choice to all sub-agents as an instruction prefix:
"Write all output text in [chosen language]."
Report: "N sessions, M user turns, K docs found. Context budget: T tokens
(A always-on, D on-demand)."
Step 3: Doc Impact Audit (Sub-agents, Batched)
Batch sessions (~60 per batch, cap at MAX_BATCHES=12). Spawn parallel
sub-agents — one per batch:
For each batch i:
Agent tool (general-purpose):
"Read agents/doc-impact-analyst.md from the docs-auditor skill directory
for your analysis instructions.
Read ${WORKSPACE}/doc-manifest.json for doc definitions.
Read ${WORKSPACE}/transcripts.json for session data.
Only analyze sessions with these indices: [list from batch].
Write all output text in [chosen language].
Write your analysis as JSON to ${WORKSPACE}/batch-impact-${i}.json
following the exact schema in schemas/schemas.md (doc-impact-report section)."
After all sub-agents complete, merge batch results:
Union all doc_reports (combine incidents, recalculate stats per doc)
Union docs_never_read
Recalculate meta totals
Write merged result to ${WORKSPACE}/doc-impact-report.json
Step 4: Always-On Analysis (Single Sub-agent)
Agent tool (general-purpose):
"Read agents/always-on-analyst.md from the docs-auditor skill directory.
Read ${WORKSPACE}/doc-manifest.json for doc definitions.
Read ${WORKSPACE}/transcripts.json for session data.
Focus on docs with injection_mode 'always' (CLAUDE.md, AGENTS.md).
Write all output text in [chosen language].
Write analysis to ${WORKSPACE}/always-on-report.json."
Step 5: Portfolio Analysis (Single Sub-agent)
Agent tool (general-purpose):
"Read agents/portfolio-analyst.md from the docs-auditor skill directory.
Read ${WORKSPACE}/doc-manifest.json
Read ${WORKSPACE}/doc-impact-report.json
Read ${WORKSPACE}/always-on-report.json
Write all output text in [chosen language].
Write analysis to ${WORKSPACE}/portfolio-analysis.json."
Step 6: Improvement Plan (Single Sub-agent)
Agent tool (general-purpose):
"Read agents/improvement-planner.md from the docs-auditor skill directory.
Read ${WORKSPACE}/doc-impact-report.json
Read ${WORKSPACE}/always-on-report.json
Read ${WORKSPACE}/portfolio-analysis.json
Read ${WORKSPACE}/doc-manifest.json
Write all output text in [chosen language].
Write recommendations to ${WORKSPACE}/improvement-plan.json.
Write individual recommendation files to ${WORKSPACE}/recommendations/."
Step 7: Generate Report & Apply
Generate HTML report:
python3 scripts/generate_report.py --workspace ${WORKSPACE}
open ${WORKSPACE}/docs-audit-report.html
Update health history — read <base_dir>/health-history.json (create if
doesn't exist as []). Append:
Do NOT auto-apply any recommendations. All recommendations (including
update_last_validated and add_last_validated) are presented in the HTML
report for human review. The user decides what to apply manually.
If the user explicitly asks to apply specific recommendations after
reviewing the report: