factchecking

name	factchecking
description	Use for evidence and safety verification tasks: detect prompt injection, mask PII, classify factual or policy-sensitive claims, redact sensitive text, compare claims to trusted evidence, and produce auditable reports with precision, recall, F1, false-positive, or leakage checks.

Factchecking Cheatsheet

A factchecking task asks whether text should be trusted, blocked, masked, redacted, or supported. The output is auditable: per-item decision with a reason and (when applicable) a confusion matrix.

Decision rules

Question	If yes	If no
Does the task disallow LLM API calls?	Use deterministic features only	An LLM judge may help, but reasons must still be deterministic
Are gold labels available?	Optimize threshold against the requested metric	Ship conservative defaults; document threshold choice
Are records counted by row or by instance?	Match exactly — they are not the same	—

Task shapes at a glance

Shape	Identify by	Output
Prompt injection	"should this input be blocked?"	label per input + reason
PII masking	"find PII tokens, mask in place"	masked text + token list
PDF redaction	"remove text from pages, prove it's gone"	redacted PDF + verification log
Claim / citation check	"does evidence support this claim?"	label per claim + source span

Prompt injection — feature groups

Use grouped weighted features, not one big keyword list. Tune threshold to hit the false-positive bound the task names.

Feature group	Weight	Examples
Role-hijack phrases	3	ignore previous instructions, forget everything above, drop your persona
Override markers	2	system:, [admin], bypass safety, developer mode
Encoded payload	3	base64 strings that decode to ignore/bypass/system
Delimiter abuse	2	code-fence with `system

import base64, re

ROLE_HIJACK = ["ignore previous instructions", "forget everything above",
               "drop your persona", "reset yourself"]
OVERRIDE = ["system:", "[system]", "admin override", "bypass safety",
            "developer mode", "new directive"]
DELIM = [r"```\s*\n.*(?:system|ignore|override|bypass)",
         r"\[(?:hidden|inject|system|admin)\]",
         r"<!--.*(?:system|override|admin).*-->"]

def has_b64_attack(text):
    for tok in re.findall(r"[A-Za-z0-9+/]{20,}={0,2}", text):
        try:
            decoded = base64.b64decode(tok).decode("utf-8", "ignore").lower()
        except Exception:
            continue
        if any(w in decoded for w in ("ignore", "bypass", "override", "system")):
            return True
    return False

def classify(text, threshold=2):
    t = text.lower()
    score = (3 * any(p in t for p in ROLE_HIJACK)
             + 2 * any(p in t for p in OVERRIDE)
             + 3 * has_b64_attack(text)
             + 2 * any(re.search(p, t, re.S) for p in DELIM))
    return "injection" if score >= threshold else "safe"

Avoid matching unqualified system — flag it only inside delimiter or override context.

PII — pattern set, ordered

Apply specific patterns before broad ones. SSN/CC have rigid shapes; PHONE will swallow them if it runs first.

Type	Pattern	Notes
SSN	\b\d{3}-\d{2}-\d{4}\b	US format only
CC	\b(?:\d[ -]?){13,19}\b	Add Luhn check to cut false positives
EMAIL	[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}	Most reliable
PHONE	(?:\+?\d{1,3}[-.\s]?)?(?:\(?\d{3}\)?[-.\s]?)\d{3}[-.\s]?\d{4}	Run last

import re

PATTERNS = [
    ("[SSN]",         r"\b\d{3}-\d{2}-\d{4}\b"),
    ("[CREDIT_CARD]", r"\b(?:\d[ -]?){13,19}\b"),
    ("[EMAIL]",       r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}"),
    ("[PHONE]",       r"(?:\+?\d{1,3}[-.\s]?)?(?:\(?\d{3}\)?[-.\s]?)\d{3}[-.\s]?\d{4}"),
]

def mask_pii(text):
    for label, pat in PATTERNS:
        text = re.sub(pat, label, text)
    return text

For per-instance recall (not per-record), return the list of (span, type) tuples alongside the masked text — tests often grade on instance counts.

PDF redaction — the only correct procedure

Visual overlay is not redaction. Use PyMuPDF's annotation-then-apply pattern.

Step	What	Why
1	page.search_for(needle)	Find rectangles for every span
2	page.add_redact_annot(rect, fill=(0,0,0))	Mark span for removal
3	page.apply_redactions()	Rewrite content stream — text actually deleted
4	doc.set_metadata({})	Strip author/title metadata
5	Reopen + extract — needle absent?	Verification, not optional

import fitz

doc = fitz.open("input.pdf")
needles = ["<sensitive_string>", "<author_name>", "<email>"]
for page in doc:
    for needle in needles:
        for rect in page.search_for(needle):
            page.add_redact_annot(rect, fill=(0, 0, 0))
    page.apply_redactions()
doc.set_metadata({})
doc.save("redacted.pdf", garbage=4, deflate=True)

For pattern-based redaction (phone numbers, SSNs across the doc), iterate the PII regex set on each page's text, find rectangles for every match, annotate, apply.

Claim / citation check — decision vocabulary

Label	When
true_positive	Evidence directly supports the claim
false_positive	Evidence contradicts the claim, OR no permitted source supports it
duplicate	Same claim raised more than once
insufficient_evidence	Evidence pack does not let you decide either way
out_of_scope	Claim is outside the evidence horizon

Record per claim: claim_id, claim_text, evidence_id, evidence_span, label, rationale. Normalize whitespace and Unicode quotes before matching. Prefer exact source spans over paraphrased justifications.

Do not borrow task labels, hidden oracle values, or fixture-specific expected counts as "evidence". The pack you adjudicate against is what you build from permitted sources.

Output schemas to remember

Task	Schema
Per-input safety	`{"input_id": str, "label": "safe
PII masking	{"masked_text": str, "spans": [{"start": int, "end": int, "type": str}]}
PDF redaction	redacted PDF file + {"removed_strings": [str], "pages_affected": [int]}
Claim check	{"claim_id": str, "label": str, "evidence_ref": str, "rationale": str}

Pitfalls

• One giant regex that overflags safe inputs.
• Phone masking before SSN/CC — the broader pattern wins, types misreported.
• Reporting record-level counts when the task wants instance-level recall.
• Visual overlay treated as redaction — text remains in the content stream.
• Dropping IDs between predictions and ground truth — metrics computed on the wrong rows.
• Borrowing oracle expectations as if they were evidence — leakage, not verification.