| name | source-evaluation |
| description | Use when admitting any source into research and assessing its provenance, authority, independence, timeliness, bias, media integrity, and fit for a claim; use source-verification afterward to test URLs, quotes, statistics, claim links, and release readiness. |
| metadata | {"portable":true,"compatible_with":["claude-code","codex"],"priority":"critical"} |
Source Evaluation
Use When
- Assess every candidate source before its claims enter the evidence corpus.
Do Not Use When
- Do not use as the final URL, quotation, statistic, or claim-link check; use source-verification after evaluation.
Required Inputs
| Input | Source/provider | If absent |
|---|
| Source artifact, provenance, date, author/publisher, and intended claim | Provider and research brief | Exclude or mark the source unresolved; never invent metadata. |
| Corroborating sources where the claim is material or contested | Independent providers | Record the corroboration gap and narrow the claim. |
Workflow
- Load evidence-discipline and classify the source type.
- Apply the credibility ladder and matching source rubric.
- Separate source content from analyst inference and record contradictions.
- Admit, qualify, or reject the source for each intended claim.
- Route admitted claims to source-verification; stop on fabricated material and recover from untraceable claims by removing them or recording a gap.
Outputs
| Artifact | Consumer | Acceptance condition |
|---|
| Source evaluation record | Researcher and verifier | Provenance, rubric, claim fit, limitations, and admission decision are explicit. |
Evidence Produced
| Category | Artifact | Acceptance condition |
|---|
| Correctness | Claim-level source assessment | Every admitted claim identifies a real source and qualification. |
Capability Contract
Evaluation defaults to read-only. Network lookup may verify provenance when authorised; downloading restricted material, contacting sources, altering evidence, or publishing findings requires explicit authority.
Degraded Mode
If provenance, network, media-forensics tools, or corroboration are unavailable, mark the relevant checks unassessed and return the narrowest qualified assessment. Never upgrade an unassessed source.
Decision Rules
| Evidence | Action | Failure/risk avoided |
|---|
| Primary authoritative source fits claim | Prefer and verify it | Citation laundering |
| Material conflict between credible sources | Preserve conflict and investigate | False certainty |
| Provenance cannot be established | Exclude or label unresolved | Fabricated attribution |
Quality Standards
Every admission decision is claim-specific, traceable, explicit about limitations, and independent of whether the source supports the desired conclusion.
Anti-Patterns
- Scoring a source without its intended claim. Fix: evaluate claim fit.
- Equating official with infallible. Fix: test incentives and method.
- Hiding credible contradiction. Fix: preserve it.
- Treating popularity as authority. Fix: assess provenance.
- Filling missing metadata from memory. Fix: mark a gap.
Worked Example
For a statistic quoted by a news article, locate the originating dataset or publication, evaluate that source for the precise statistic, record any definition mismatch, and send the claim-source pair to verification.
References
Freshness and uncertainty gate
Before admitting a material source, classify the intended claim as stable, context-bound, time-sensitive, partial/early-release, or unusable using references/freshness-and-uncertainty.md. Record the source status, completeness, access date, and next verification trigger in the source register. A historical source may support a historical claim but must not silently support a current command, law, product capability, benchmark, price, safety instruction, or platform fact. When evidence is incomplete, narrow the wording and mark the claim unassessed, partial, or historical-only; never fill a gap from memory.
Research improvement handoff
After a research wave, send source defects, duplication, verification failures, access gaps, and analyst toil to the Kaizen research loop in ../research-orchestration/references/kaizen-research-loop.md. The experiment must preserve source immutability and use a guardrail such as verification coverage, source independence, or unresolved-claim count.
Single entry skill for vetting any source the engine consults. Plagiarism prevention is in academic-writing; this skill governs whether a source is trustworthy in the first place and whether claims drawn from it have hallucinated detail.
The non-negotiable guardrail (load first)
Always load references/evidence-discipline.md. The engine's first rule: no claim, statistic, quote, name, case, statute, URL, date, or place appears in output unless traceable to a real source. This applies to every output of every skill.
Source-type router — load the matching reference
| Source type | Load |
|---|
| Any source — the universal floor | references/evidence-discipline.md (always) + references/credibility-ladder.md |
| Primary document (archival record, legal filing, government publication, leaked file, manuscript) | references/burke-five-term-doubt.md |
| Media, journalism, analyst report, think-tank publication | references/tudor-twelve-points.md |
| Misinformation-prone news / viral information stream | references/tudor-twelve-points.md + references/misinformation-and-bias-checks.md |
| Image, video, audio, screenshot, suspect document | references/silverman-media-forensics.md |
| Encyclopaedic / reference work | references/credibility-ladder.md (tier 4: tertiary) |
| Social media post / forum / comment | references/tudor-twelve-points.md + references/silverman-media-forensics.md (provenance) |
| Statistic / dataset | references/credibility-ladder.md + Walker four-axis (data-quality-assessment) |
| Automated source / claim reliability triage | references/automated-reliability-triage.md + references/credibility-ladder.md |
The 5-tier credibility ladder (universal)
Detail in references/credibility-ladder.md. Quick reference:
| Tier | Type | Trust default |
|---|
| 1 — Primary | Original document, dataset, transcript, recording, photo with provenance | High; still apply Burke pentad |
| 2 — Authoritative secondary | Peer-reviewed paper, government statistical release, audited corporate filing, court judgment | High |
| 3 — Vetted journalism / analyst | Established outlet, named analyst report, named expert quote in mainstream press | Medium-high; apply Tudor twelve points |
| 4 — Tertiary / reference | Encyclopaedia, textbook, well-cited Wikipedia article, dictionary | Medium for orientation; never sole proof |
| 5 — Unvetted | Social media post, blog, forum, AI output, anonymous source | Low; treat as lead, not evidence |
Rule: any tier-5 claim must be triangulated against ≥2 tier-1 to tier-3 sources before it appears as fact. Tier-4 claims need ≥1 tier-1 to tier-3 source for any load-bearing assertion.
Burke's five-term source-doubt pentad (primary documents)
Detail in references/burke-five-term-doubt.md. For every primary document, ask:
- Author — who wrote it; what was their position, motive, knowledge?
- Provenance — chain of custody from creation to your hands; gaps; tampering opportunities.
- Production — when, how, under what conditions; first or revised; original or copy.
- Mechanics — language, format, conventions of the era; do they match?
- Aims — what was the document trying to do; who was the intended audience?
A document that survives the pentad is evidence with known limits. A document that hasn't faced it is evidence with unknown limits, which is dangerously close to hallucination input.
Tudor's twelve-point evaluation (media / journalism / analyst)
Detail in references/tudor-twelve-points.md. Twelve criteria:
- Recency · 2. Relevancy · 3. Authority · 4. Completeness · 5. Accuracy · 6. Clarity · 7. Verifiability · 8. Statistical validity · 9. Internal consistency · 10. External consistency · 11. Context · 12. Comparative quality
Apply when a media or analyst source carries a load-bearing claim. The audit produces a one-line verdict that the engine logs in the manifest.
Misinformation and bias checks (viral / politicized / contested claims)
Detail in references/misinformation-and-bias-checks.md. Before a contested or viral claim enters the corpus:
- Separate fact, opinion, and interpretation.
- Check the claim against the original source, not just reposts or reactions.
- Identify the likely biases in the source and in the analyst.
- Treat virality as a warning sign, not corroboration.
- Downgrade any claim whose chain leads only to commentary, screenshots, or clipped excerpts.
Silverman / Bellingcat media-forensics (images, video, documents)
Detail in references/silverman-media-forensics.md. Workflow:
- EXIF extraction — camera, GPS, timestamp.
- Reverse image search — Google Lens, Yandex, TinEye, Bing.
- Geolocation by landmark / shadow-time computation.
- Archive recovery — Wayback, archive.today, original posting site.
- Provenance tracing — who first published; cross-platform trail.
- Tampering detection — error-level analysis, metadata anomalies.
Output: provenance chain logged with timestamps, URLs, and confidence.
Universal output rule
Every cited claim in engine output carries:
- Source reference (URL, DOI, archive link, document ID).
- Source tier (1–5 from the credibility ladder).
- Verification trail (Burke / Tudor / Silverman audit if applicable, otherwise "credibility-ladder" only).
- Confidence (high / medium / low).
- Date accessed (UTC).
If any of those is missing, the claim does not ship.
If automated reliability scoring was used, the output must also expose the
axis-level breakdown and human-review flags from
references/automated-reliability-triage.md. A composite score alone is not
evidence.
Universal anti-patterns
- Trusting a tier-5 claim without triangulation.
- Treating an automated reliability score as a source verdict instead of a
triage signal.
- "Studies show" without footnote.
- Citing a stat older than 3 years without flagging.
- Single-source paragraph on contested terrain.
- Image used as evidence with no reverse-image-search trail.
- Wikipedia footnote chasing skipped — citing the article instead of its source.
- Author of a document never named, motive never considered.
- AI-generated source list passed through unchecked — must be 100 % verified before any output.
- Quote attribution to a person without locating the original utterance.
- URL cited without archival snapshot — link rot kills evidentiary trails.
- Mixing source-tier verdicts in one paragraph without flagging the weakest.
Universal ship gate
For book-informed work, apply book-driven source admission and currentness before admitting any time-sensitive claim.
Companion skills
data-quality-assessment — for source-vetting at the dataset level (Walker four axes).web-scraping-foundations — when you collect the source yourself.osint-investigation — when sources sit inside a broader investigation.academic-writing — when sources feed an academic artifact (citation styles in references).analytic-tradecraft — when source disputes become estimative judgments.
