wikipedia-en-article-audit

name	wikipedia-en-article-audit
description	Audit an English Wikipedia article for structural issues, factual errors, and NPOV violations, then produce a machine-readable task graph (DAG) that another agent can execute to fix all identified problems
license	MIT
compatibility	opencode
depends_on	["wikimedia-api-access","wikipedia-page-anatomy","wikimedia-wikitext","wikimedia-page-assessment","wikimedia-database"]
skill_discovery_hints	[{"keywords":["article audit","audit","NPOV check","factual verification","taskgraph"]},{"keywords":["article diagnosis","structural audit","sentence verification","DAG"]}]
last_verified	"2026-06-10T00:00:00.000Z"

⚠️ Read-only policy: This skill audits only. It never edits live Wikipedia articles.

All three phases fetch data from Wikipedia but never write back. The output is a taskgraph.json specification — a work order for a separate agentic system to execute. The audit agent does not and should not have edit permissions.

If a user wants to act on the findings, they should start a new, clean session with edit-capable tools and feed it the taskgraph.json as input. Mixing audit and edit in the same session risks accidental writes, edit conflicts, and BLP violations.

Separate the concerns: one agent audits, a different agent edits (or the same agent in a different context with explicit user confirmation).

⚠️ Audit is a three-phase process. Phase 1 (diagnosis) and Phase 3 (taskgraph generation) are fully automated. Phase 2 (sentence-by-sentence verification) requires a human reviewer to read sources and classify each claim. Do not skip Phase 2 — the automated pipeline cannot reliably detect factual contradictions.

Typical Invocation

A user invokes this skill with a simple request. The agent follows the workflow below, stopping at Phase 2 for human review and resuming after the reviewer returns their findings.

User:  "Audit the English Wikipedia article on Pyrrhus Concer."

Agent: ▶ Runs Phase 1 (diagnosis)
       ▶ Presents diagnosis + sentences.jsonl to human reviewer
       ▶ Waits for human to return verification.json
       ▶ Runs Phase 3 (taskgraph generation)
       ▶ Outputs taskgraph.json + analysis.md
       ▶ "Audit complete. 10 tasks in the DAG:
            p0: 2 (factual correction, NPOV rewrite)
            p1: 4 (infobox, sections, missing details, assessment)
            p2: 4 (grammar, epitaph, reflist, sources)
          The taskgraph is at pyrrhus_concer/taskgraph.json"

The human reviewer's role is focused and bounded: review each sentence, read the cited sources, and fill in verdicts. The rest is handled by the scripts.

---

Enables the agent to produce a structured work order — a taskgraph.json validated against a formal JSON Schema — for improving any English Wikipedia article. The output is designed for handoff to a separate agentic execution system.

Related skills (prerequisites for understanding page structure and tools):

• wikipedia-page-anatomy — Understanding infoboxes, categories, templates, sections
• wikimedia-wikitext — AST-based wikitext parsing
• wikimedia-page-assessment — WikiProject quality ratings
• wikimedia-api-access — User-Agent headers, rate limiting, error handling
• wikipedia-en-biography-writing — NPOV / NOR / BLP standards for content evaluation

---

Overview

The audit pipeline converts a Wikipedia article URL into a repair task graph:

Article URL
    │
    ▼
┌─────────────────────────────────────┐
│  Phase 1: Diagnosis                 │  ← Automated
│  ─────────────────────               │
│  • Fetch article, talk page, WD     │
│  • Structural audit (infobox,       │
│    sections, categories, assess)    │
│  • Identify NPOV trigger keywords   │
│  • Extract sentences + citations    │
│  │                                   │
│  Output: diagnosis.json              │
│          sentences.jsonl             │
│          NPOV flags                  │
└────────┬────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│  Phase 2: Verification              │  ← Human-in-the-loop
│  ─────────────────────               │
│  • Review each sentence             │
│  • Read cited sources               │
│  • Cross-reference sister articles  │
│  • Classify as CONFIRMED /          │
│    CONTRADICTED / NPOV_OR /         │
│    UNVERIFIABLE                      │
│  • Write corrections                │
│  │                                   │
│  Output: verification.json           │
└────────┬────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│  Phase 3: Taskgraph Generation      │  ← Automated
│  ────────────────────────            │
│  • One task per identified issue    │
│  • Dependency edges from DAG rules  │
│  • Verify steps from diagnosis      │
│  • Schema validation at output      │
│  │                                   │
│  Output: taskgraph.json              │
│          analysis.md                 │
└─────────────────────────────────────┘

---

Prerequisites

Tools and accounts

Requirement	For	Notes
Python 3.10+	Phase 1 scripts, Phase 3 generator	Standard library + requests
pip install requests mwparserfromhell	Wikitext parsing, API calls	See wikimedia-wikitext skill
Wikipedia account	Phase 3 execution (optional — not needed for audit)	Needed if you run the resulting taskgraph
SSH tunnel to Toolforge (optional)	Database assessment queries	See wikimedia-database skill
Text editor or JSON-aware IDE	Phase 2 human review	Needed to edit verification_stub.json

User-Agent

All API calls must include a descriptive User-Agent header or they will be blocked with HTTP 403 or 429. See the wikimedia-api-access skill for the correct format.

Environment variables

Variable	Required for	Default
WIKIPEDIA_USER_AGENT	All API calls	None (must be set)
TOOLFORGE_USER	Database queries (optional)	—
TOOLFORGE_SQL_USER	Database queries (optional)	—
TOOLFORGE_SQL_PASSWORD	Database queries (optional)	—

---

SOP: Phase 1 — Diagnosis

Purpose

Produce a structured inventory of everything wrong with the article. Phase 1 is entirely mechanical — no creativity or judgment required.

Steps

1a. Resolve the article title

curl -s -H "User-Agent: MyBot/1.0 ([e-mail]) AuditTool" \
  "https://en.wikipedia.org/w/api.php?action=query&titles=$TITLE&redirects=1&format=json"

• If the title is a redirect, follow it to the canonical page.
• If it's a disambiguation page, abort and list the options.
• Record the canonical title and page ID.

1b. Fetch article components

Fetch and store:

Component	API call	Stored as
Wikitext	action=raw	sources/article.wikitext
Talk page wikitext	action=raw&page=Talk:TITLE	sources/talk.wikitext
Page info (protection, length, watchers)	prop=info	diagnosis.json
Wikidata entity	https://www.wikidata.org/wiki/Special:EntityData/{Q}.json	sources/wikidata.json
Categories	prop=categories	diagnosis.json
Templates used	prop=templates	diagnosis.json
Page assessments (if tunnel available)	SQL on enwiki_p.page_assessments	diagnosis.json

1c. Structural audit

Check for:

• Infobox: Is an {{Infobox ...}} template present? If not, note which infobox type would be appropriate based on the article category.
• Sections: How many == Level 2 == headings exist? Are standard sections present (Early life, Career, Legacy, etc.)?
• Categories: How many? Any obvious gaps (e.g., birth/death year, occupation)?
• Protection: Is the page protected? At what level? Does that affect what the taskgraph can do?
• Assessment: What class is the article assessed at (Stub/Start/C/B/GA/FA)? Is it under-assessed?
• Maintenance templates: Any {{citation needed}}, {{POV}}, {{expand section}}, {{dead link}}?
• Short description: Does the short description use correct demonym adjectives? Check for the common error pattern China + noun (e.g. "China businesswoman") which should be Chinese + noun ("Chinese businesswoman"). Other patterns: America + noun → American + noun, Britain + noun → British + noun.
• Reference count: How many <ref> tags? Any citation dumping (one ref covering many paragraphs)?
• Image count: Does the article have a lead image? An infobox image? If no image is found, check whether a Commons category exists for the subject (https://commons.wikimedia.org/wiki/Category:{PAGENAME}) and whether a free-licensed portrait is available. Note the result so Phase 2 can recommend adding an image to the infobox if one exists.

1d. Article-type classification

Before the NPOV scan, determine the article type from its infobox, categories, and BLP status. This shapes the expected section structure and the severity of certain findings.

def classify_article(categories, infobox_type, blp_status):
    if "Living people" in categories or blp_status:
        return "BLP"
    if any("births" in c for c in categories) and any("deaths" in c for c in categories):
        if "business" in str(categories) or "executive" in str(categories):
            return "biography_executive"
        if "scientist" in str(categories) or "engineer" in str(categories):
            return "biography_academic"
        return "biography_general"
    return "topic_article"

Each type has standard expected sections. The diagnosis flags missing ones:

Article type	Expected sections	Examples
BLP	Early life and education, Career, Personal life (minimal), Awards	He Tingbo
biography_executive	Early life, Career, Board positions, Awards, Personal life	—
biography_academic	Early life and education, Career, Research, Selected works, Awards	—
biography_general	Early life, Career, Later life, Legacy	Pyrrhus Concer
topic_article	Varies by topic — use subject-specific conventions	—

For living subjects, skip checks like "missing Later life section" or "missing Legacy section" — these only apply to deceased subjects. Similarly, a "Personal life" section is appropriate for BLPs but optional for historical biographies.

1e. NPOV keyword scan

Scan the article text (after stripping templates and references) for these trigger words and phrases:

# Editorial praise / promotional
undoubtedly, unquestionably, clearly, obviously,
notably, noteworthy, interestingly, importantly,
remarkably, impressively, extraordinarily,
incomparably, unbeatably, famously, notoriously,
renowned, legendary, pioneering, world-class,
visionary, iconic, groundbreaking, transformative

# Original research / speculation
it is possible that, it is likely that, it is plausible that,
it seems that, one can imagine that,
may have been, might have been, could have been,
was likely, was probably, appears to have been,
suggests that, is suggestive of, is indicative of

# Essay-like conclusions
a testament to, serves as a testament,
speaks to his, speaks to her, speaks to their,
is a testament, stands as a testament,
demonstrates his, demonstrates her,
reflects his, reflects her, indicative of

# Weasel words
some say, many believe, it is said that,
critics claim, it is widely regarded,
is considered to be, is thought to be,
is often described as, has been called

Each flagged occurrence is recorded with its sentence number, the trigger word, and its surrounding context (50 chars before/after).

1f. Extract sentences

Split the article body text (lead + all sections) into individual sentences:

import re
# Split on sentence boundaries: . ! ? followed by space + capital letter
sentences = re.split(r'(?<=[.!?])\s+(?=[A-Z"])', plain_text)

Each sentence is recorded as one JSON line:

{
  "index": 1,
  "text": "Pyrrhus Concer (March 17, 1814 – August 23, 1897) was a formerly enslaved American sailor from Southampton, New York who was aboard the whaling ship Manhattan, the first American ship to visit Tokyo in 1845.",
  "char_offset": 0,
  "section": "lead",
  "has_inline_ref": true,
  "ref_indices": [0],
  "npov_flags": [],
  "verdict": null
}

1g. Extract citations

Extract all <ref> tags and citation templates. For each:

{
  "ref_index": 0,
  "template": "cite book",
  "params": {
    "last": "Van Zandt",
    "first": "Howard",
    "title": "Pioneer American Merchants in Japan",
    "year": "1984",
    "publisher": "Tuttle Publishing",
    "isbn": "9994648144",
    "pages": "68–72"
  },
  "url": null,
  "accessible": false,
  "sentence_indices": [1, 2, 3, 4, 5]
}

Also resolve which sentence(s) each reference supports. If one reference is attached to the end of a multi-sentence paragraph, mark all sentences in that paragraph as potentially supported by that ref.

Additionally, produce a sources summary table — a quick-reference view of all citations with their accessibility status and whether sister articles or Wikidata also corroborate the same claims. This table lets the Phase 2 human reviewer skip sources that are confirmed redundant.

{
  "sources_summary": [
    {
      "ref_index": 0,
      "short_ref": "Reuters 2026",
      "url": "https://www.reuters.com/...",
      "accessible": true,
      "accessible_via": ["direct_http"],
      "language": "en",
      "corroborated_by": ["wikidata", "nikkei_asia"],
      "sentences_supported": [1, 2, 3, 4]
    },
    {
      "ref_index": 3,
      "short_ref": "WSJ (paywalled)",
      "url": "https://www.wsj.com/...",
      "accessible": false,
      "accessible_via": ["web_archive"],
      "language": "en",
      "corroborated_by": ["nikkei_asia"],
      "sentences_supported": [28, 29]
    }
  ]
}

This summary enables a quick triage: sources marked accessible: true can be read directly; inaccessible sources with corroboration from sister articles can be deprioritized; inaccessible sources with no corroboration need alternative retrieval attempts.

Output files

File	Format	Content
diagnosis.json	JSON	All structural findings, assessment, keyword flags, metadata
sentences.jsonl	JSONL	One line per sentence with metadata and null verdict
sources/article.wikitext	Text	Raw wikitext
sources/talk.wikitext	Text	Raw talk page wikitext
sources/wikidata.json	JSON	Wikidata entity dump
diagnosis.json (sources_summary)	JSON	Sources accessibility table with corroboration status

---

SOP: Phase 2 — Verification

Purpose

For every sentence in the article, determine whether it is factually correct, neutrally framed, and properly cited. This is the human review phase — the automated system cannot reliably detect factual contradictions.

Input

• sentences.jsonl from Phase 1
• diagnosis.json from Phase 1
• Article talk page
• Cited sources (fetched in Phase 1; some may be inaccessible)

Procedure

2a. Set up the working file

Copy sentences.jsonl to verification_stub.json and convert to a single JSON array (easier for humans to edit):

python3 -c "
import json, sys
sentences = [json.loads(line) for line in open('sentences.jsonl')]
with open('verification_stub.json', 'w') as f:
    json.dump({'article_title': '$TITLE', 'sentences': sentences}, f, indent=2)
"

The stub has every sentence's "verdict": null. The human will fill these in.

2b. BLP spotlight check

⚠️ Stop here if the article is about a living person (BLP).

If diagnosis.json shows blp: yes, apply extra scrutiny before proceeding with sentence-by-sentence verification:

Check	What to look for	Action if found
Contentious claims	Allegations of wrongdoing, legal disputes, national security implications, negative personal details	Require multiple high-quality, independent sources. If only one source supports a contentious claim, flag it.
Private information	Home address, phone number, email, exact birth date without source, family details not in public record	Remove immediately per WP:BLPPRIVACY. Do not leave in the article even temporarily.
Geopolitical framing	Sanctions, technology bans, national security designations — these are often politically charged	Ensure every geopolitical claim is attributed to a specific source (e.g., "According to Reuters..." not "The US added Huawei to the Entity List for...")
Editorial monikers	"Chip queen," "pioneer," "trailblazer" — any nickname or superlative	Must be attributed to the source that coined it. If the article asserts it in Wikipedia's voice, flag as NPOV.
Harm principle	Could publishing this fact cause real-world harm to the subject?	If the answer is yes and the fact is not essential to the biography, omit it per WP:BLPBALANCE.

If any BLP violations are found at this stage, pause the audit and flag them immediately. Do not proceed with sentence-by-sentence verification until the violations are resolved — there is no point verifying 28 clean sentences if the 29th is a BLP violation that will get the article deleted.

2c. For each sentence, determine its verdict

Verdict	When to use	Example
confirmed	A reliable source supports the claim, and the claim is framed neutrally	Sentence 1 (dates, ship, event — supported by Newsday + Wikidata + sister articles)
contradicted	A reliable source contradicts the claim, or the claim is demonstrably false	Sentence 2 ("Pyrrhus family" — Newsday says "Pelletreau family")
npov_or	The claim is plausible but framed in editorial/promotional language, or is speculative	Sentences 10–16 (essay-like financial analysis with "undoubtedly impressive", "testament to his business acumen")
unverifiable	No source could be found for the claim, or all sources are inaccessible	A claim supported only by a paywalled article that couldn't be accessed
mixed	The sentence contains multiple sub-claims with different verdicts	Sentence 2 (true: farmhand, 1832, Sag Harbor / false: Pyrrhus family)

Protocol for reading sources:

1. First check the article's own cited sources. Open each URL or citation and read the relevant passages. Do not trust the article's interpretation — read the source yourself.
2. If the cited source is inaccessible (paywall, Cloudflare, 404), try the Wayback Machine at web.archive.org.
3. Cross-reference against Wikipedia sister articles (e.g., if the article says "first American ship to visit Tokyo", check the Mercator Cooper and Manhattan ship articles for confirmation).
4. For biographical articles, check Wikidata for birth/death dates, occupation, and known-for statements.
5. If no source can be found after reasonable effort, mark the sentence as unverifiable.

Protocol for contradicted verdict:

{
  "index": 2,
  "verdict": "contradicted",
  "contradiction": {
    "source": "Newsday 2005 via Wayback Machine",
    "source_url": "https://web.archive.org/web/20050918211029/http://www.newsday.com/community/guide/lihistory/ny-history-hs509b,0,7157701.story",
    "source_quote": "He was owned by the Pelletreau family",
    "explanation": "The article says 'enslaved by the Pyrrhus family' but the Newsday article, based on Southampton Historical Museum records, says 'owned by the Pelletreau family'. Pyrrhus was Concer's given name (after the Greek king), not his enslaver's surname."
  },
  "correction": "Concer, named after the Greek king Pyrrhus, was enslaved by the Pelletreau family",
  "correction_citation": {
    "template": "cite news",
    "params": {
      "last": "Bleyer",
      "first": "Bill",
      "title": "Legacy: Pyrrhus Concer",
      "work": "Newsday",
      "year": "2005",
      "url": "https://web.archive.org/web/20050918211029/http://www.newsday.com/community/guide/lihistory/ny-history-hs509b,0,7157701.story"
    }
  }
}

Protocol for npov_or verdict:

{
  "index": 10,
  "verdict": "npov_or",
  "npov_explanation": "The phrase 'indicate a level of financial stability and engagement with the financial institutions of his community' is the editor's interpretation of raw data, not a factual claim. The underlying data (savings accounts, shares) may be in the cited thesis, but the framing is editorial.",
  "suggested_action": "replace with neutral factual statement: 'Probate records showed he held four savings accounts, shares in the Southampton Water Works, and had lent money to others.'",
  "source_for_facts": {
    "citation_index": 2,
    "note": "Button thesis pp. 181, 183 — verify data access"
  }
}

Protocol for unverifiable verdict:

{
  "index": 20,
  "verdict": "unverifiable",
  "reason": "The claim that Elihu Root wrote the epitaph is cited to a paywalled NYT article and previously to findagrave.com. Neither could be independently verified. The epitaph text itself is confirmed by Newsday, but the Root attribution is not.",
  "suggested_action": "add {{citation needed}} after 'engraved with an epitaph written by Elihu Root'"
}

Protocol for mixed verdict:

{
  "index": 2,
  "verdict": "mixed",
  "sub_claims": [
    {
      "text": "enslaved by the Pyrrhus family",
      "verdict": "contradicted",
      "correction": "Pelletreau family"
    },
    {
      "text": "worked as a farmhand",
      "verdict": "confirmed",
      "source": "Newsday",
      "source_quote": "worked as a farmhand"
    },
    {
      "text": "until 1832",
      "verdict": "confirmed",
      "source": "Newsday",
      "source_quote": "Freed when he turned 18 in 1832"
    },
    {
      "text": "slavery in New York formally ended in 1827",
      "verdict": "confirmed",
      "source": "Historical fact (NY Gradual Emancipation Act)"
    },
    {
      "text": "worked on whale ships out of Sag Harbor, New York",
      "verdict": "confirmed",
      "source": "Newsday",
      "source_quote": "began shipping out on whaleships"
    }
  ]
}

2d. Handle inaccessible sources

If a source cannot be accessed despite reasonable effort:

1. Check the Wayback Machine at web.archive.org
2. Check Wikipedia sister articles that cite the same source
3. Check Google Scholar or other databases for an accessible version
4. If still inaccessible, note it in the sentence's verdict

The accessible field in the citations array records the outcome:

{
  "ref_index": 2,
  "accessible": false,
  "access_attempted": ["direct_http", "wayback_machine", "google_scholar"],
  "access_note": "Cloudflare protection on repository.library.brown.edu; PDF not publicly accessible"
}

2e. Finalize verification.json

After reviewing all sentences, save the file as verification.json. The structure is identical to verification_stub.json but with all verdict fields filled in.

Validation checklist before saving:

• Every sentence has a non-null verdict
• Every contradicted verdict has a contradiction object with source, source_quote, and correction
• Every npov_or verdict has a suggested_action
• Every citation has an accessible boolean
• The file is valid JSON (run python3 -m json.tool verification.json)

---

SOP: Phase 3 — Taskgraph Generation

Purpose

Given the diagnosis from Phase 1 and the verification from Phase 2, produce a taskgraph.json (validated against the schema) and a human-readable analysis.md.

Algorithm

For each sentence with a verdict other than confirmed, generate a task:

Verdict	Task category	Criticality
contradicted	factual_correction	p0
npov_or	npov_rewrite	p0
unverifiable	citation	p1 (add {{cn}})

For each structural issue found in the diagnosis, generate a task:

Diagnosis finding	Task category	Criticality
Missing infobox	infobox	p1
No section headings	structural	p1
Missing categories	maintenance	p2
Under-assessed (Stub → Start)	maintenance	p1
Grammar / typo	grammar	p2
Citation formatting ({{Reflist}} → `{{Reflist	30em}}`)	maintenance

Dependency rules

The generator applies these dependency edges:

Task	Depends on	Rationale
All factual corrections	fetch_and_verify_sources	Sources must be available before editing
All NPOV rewrites	fetch_and_verify_sources	Same
Structural changes (section headings, infobox)	NPOV rewrites (if same area)	Avoid merge conflicts — rewrite first, then restructure
Citation additions	The affected sentence's rewrite task	Don't add {{cn}} to text that's about to be rewritten
Assessment update	All p0 and p1 tasks	Only update the rating after substantive fixes are done
Grammar fix	(none)	Independent — can run anytime
Reflist formatting	(none)	Independent

Output template

Each task follows this structure (from taskgraph.schema.json):

{
  "id": "fix_enslaver_identity",
  "summary": "Correct 'Pyrrhus family' to 'Pelletreau family'",
  "depends_on": ["fetch_and_verify_sources"],
  "category": "factual_correction",
  "criticality": "p0",
  "tools": ["wikipedia_api"],
  "actions": [
    {
      "type": "edit_wikitext",
      "page": "Pyrrhus Concer",
      "oldText": "Concer was enslaved by the Pyrrhus family",
      "newText": "Concer, named after the Greek king Pyrrhus, was enslaved by the Pelletreau family",
      "summary": "fix enslaver identity per Newsday 2005",
      "citation": { ... }
    }
  ],
  "verify": [
    { "type": "page_contains", "page": "Pyrrhus Concer", "text": "Pelletreau" },
    { "type": "page_not_contains", "page": "Pyrrhus Concer", "text": "enslaved by the Pyrrhus family" }
  ]
}

One-directional generation rule

analysis.md is a derived artifact, not a source document. It is generated from verification.json + taskgraph.json and must never be edited by hand.

verification.json ──► taskgraph.json ◄── schema validates
       │                    │
       │                    │
       ▼                    ▼
   analysis.md ◄── generated from both, never edited directly

To enforce this:

1. Auto-generated header. Every generated analysis.md begins with:

   <!-- AUTO-GENERATED from verification.json + taskgraph.json
        Do not edit by hand. To update, modify the source data or the
        generator script (04-generate-taskgraph.sh) and regenerate.
        Generated: 2026-06-02T12:34:56Z
        Article: Pyrrhus Concer -->
   

2. Staleness check. The generator checks file mtimes. If verification.json is newer than analysis.md, it prints:

   ⚠️  analysis.md is stale (verification.json changed since last generation).
      Regenerate with: 04-generate-taskgraph.sh --force
   

3. No reverse path. There is no script that reads analysis.md to produce verification.json or taskgraph.json. The .md file is a report, not an input to any downstream process. This prevents sync drift.

Schema validation

Before writing the output file, validate it against taskgraph.schema.json:

import json, jsonschema

with open('taskgraph.schema.json') as f: schema = json.load(f)
with open('taskgraph.json') as f: graph = json.load(f)
jsonschema.validate(graph, schema)

If validation fails, fix the generator. Do not output an invalid taskgraph.

Output files

File	Content
taskgraph.json	Validated JSON task DAG
analysis.md	Human-readable report from the diagnosis + verification data

The analysis.md is generated from a template that includes:

• Quick stats (page length, assessment, infobox status, reference count)
• Per-sentence verdict table
• Key issues summary (grouped by verdict type)
• Recommended action procedures (condensed from the taskgraph)
• Execution order (topological sort)

---

Workflow: Running a Full Audit

Step-by-step checklist

# ── Phase 1: Diagnosis ──
./scripts/01-diagnose.sh "Pyrrhus Concer"
# Output: diagnosis.json, sentences.jsonl, sources/

# ── Phase 2: Verification (human review) ──
# 1. Read analysis.md for context
# 2. Read each cited source from sources/ or via web
# 3. Edit verification_stub.json → verification.json
#    Fill in verdict, correction, source_quote for every sentence
# 4. Validate: python3 -m json.tool verification.json > /dev/null

# ── Phase 3: Taskgraph Generation ──
./scripts/04-generate-taskgraph.sh "Pyrrhus Concer"
# Output: taskgraph.json, analysis.md

# ── Optional: Validate ──
python3 validate.py taskgraph.json

Per-article resource estimates

Article size	Phase 1	Phase 2 (human)	Phase 3
Stub (<2 KB, 3–5 sentences)	<30s	5–10 min	<10s
Start (2–15 KB, 10–30 sentences)	<1 min	20–60 min	<10s
C-class (15–30 KB, 30–80 sentences)	1–2 min	1–3 hours	<10s
B-class+ (30+ KB, 80+ sentences)	2–5 min	3–8 hours	<10s

Phase 2 dominates the runtime. Budget accordingly.

---

Tooling

🔧 Diagnosis Script (scripts/01-diagnose.sh)

Runs Phase 1 end-to-end for a given article title.

./scripts/01-diagnose.sh "Pyrrhus Concer"
./scripts/01-diagnose.sh "Albert Einstein" --include-db  # with database assessment lookup
./scripts/01-diagnose.sh "Python (programming language)" --format compact

Produces: diagnosis.json, sentences.jsonl, sources/

🔧 Taskgraph Generator (scripts/04-generate-taskgraph.sh)

Runs Phase 3 end-to-end.

# Must have verification.json + diagnosis.json in working directory
./scripts/04-generate-taskgraph.sh
./scripts/04-generate-taskgraph.sh --project-dir /path/to/project

Produces: taskgraph.json, analysis.md

Includes a staleness check: if verification.json is newer than analysis.md, the script warns and requires --force to overwrite.

🔧 Taskgraph Validator (validate.py)

Validates a taskgraph against the schema and checks the DAG.

python3 validate.py taskgraph.json
python3 validate.py path/to/taskgraph.json --verbose

Reports:

• JSON Schema conformance
• DAG acyclicity (Kahn's algorithm)
• Dependency reference integrity
• Verify step well-formedness
• Topological execution order

📚 Intermediate Format Reference (references/intermediate-formats.md)

Full spec for:

• diagnosis.json — all structural fields and their types
• sentences.jsonl — sentence-at-a-time schema
• verification_stub.json / verification.json — verdict data model with all verdict types
• Citation accessibility tracking

📚 Verification Protocol (references/verification-protocol.md)

Extended guidance for the human Phase 2 reviewer:

• Source hierarchy (when to trust one source over another)
• How to handle conflicting sources
• BLP-specific considerations
• What to do when a source contradicts itself
• Working with the Wayback Machine
• Assessing source reliability (academic journals → news → primary sources → blogs)

🧩 Taskgraph Schema (assets/taskgraph.schema.json)

The JSON Schema defining the taskgraph contract. Shared across all articles. Supports:

• 7 action types: edit_wikitext, insert_after, edit_talk_page, replace_text, fetch_source, run_shell, write_file
• 8 verify types: page_contains, page_not_contains, infobox_exists, section_exists, assessment_equals, revision_has_comment, file_exists, shell_check
• 3 failure policies: revert_and_report, skip_and_report, retry_and_report
• 6 task categories: factual_correction, npov_rewrite, structural, infobox, citation, grammar, verification, maintenance

🧩 Analysis Template (assets/analysis-template.md)

Markdown template for the Phase 3 human-readable report. Fill-in-the-blanks style with slots. The generator fills these from verification.json + taskgraph.json.

<!-- AUTO-GENERATED from verification.json + taskgraph.json
     Do not edit by hand. Generated: {{TIMESTAMP}} -->

# Article Audit: {{ARTICLE_TITLE}}

{{QUICK_STATS_TABLE}}

## Per-Sentence Verdicts

{{SENTENCE_VERDICT_TABLE}}

## Key Issues

{{KEY_ISSUES}}

## Execution Order

{{EXECUTION_ORDER}}

⚠️ Never edit the generated analysis.md directly. If the report needs changes, modify the template above or the source data (verification.json) and regenerate. The auto-generated header at the top of every output file warns against hand-editing.

🧩 Example Output — Clean Audit (assets/example-output.md)

A complete Phase 1–2 audit report for the article [[He Tingbo]], demonstrating the expected format and depth of the analysis.md output. Includes:

• Structural diagnosis with findings table
• NPOV keyword scan results
• Per-sentence verification (29 sentences, all classified)
• Recommended taskgraph

28 of 29 sentences confirmed. No factual errors, no NPOV violations. Useful as a baseline for articles that are already in good shape.

🧩 Example Output — Problematic Audit (assets/example-output-problematic.md)

A complete audit report for the article [[Pyrrhus Concer]], showing the same format applied to an article with multiple issues:

• Factual error: enslaver identity (Pyrrhus → Pelletreau)
• NPOV/OR block: 9 sentences of editorial essay (45% of article body)
• Under-assessment: Stub when it should be Start
• Missing infobox, poor section structure, citation dumping

12 of 20 sentences confirmed. 5 NPOV/OR violations. 1 factual error. Useful for pattern-matching against articles that need substantive fixes.

🧩 NPOV Keywords List (assets/npov-keywords.txt)

Machine-readable list of NPOV trigger words and phrases, one per line. Used by the Phase 1 diagnosis script. Categorized:

# Editorial praise
undoubtedly
unquestionably
noteworthy
...

# Speculation
it is possible that
was likely
...

---

Relationship to Other Skills

• wikipedia-page-anatomy — Prerequisite for understanding what the diagnosis script finds (infoboxes, sections, categories, templates)
• wikimedia-wikitext — Powers the sentence extraction and template parsing in Phase 1
• wikimedia-page-assessment — Provides the assessment lookup that Phase 1 uses to determine if an article is under-assessed
• wikipedia-en-biography-writing — Defines the NPOV/NOR/BLP standards that Phase 2's human reviewer must apply
• wikimedia-api-access — Required for all API calls throughout the pipeline
• wikimedia-database — Optional, for enhanced assessment queries in Phase 1