adversarial-verifier

description	Same-session deep-verification methodology. Chain-of-Verification with Saboteur / New Hire / Security Auditor personas. For gate-level cross-model verification that writes the prosecution doc, use /verify — which uses this skill's methodology as its payload.
metadata	{"github-path":"skills/adversarial-verifier","github-ref":"refs/tags/v1.1.0","github-repo":"https://github.com/recallnet/recall-skills","github-tree-sha":"b66273fbc66b9a007f1835009b3b4bc91e23b315"}
name	adversarial-verifier

Adversarial Verifier

When to use this vs. /verify

Skill	Question it answers	Cross-model?	Writes prosecution doc?
/adversarial-verifier (this skill)	"I want to pressure-test a specific claim in my own session, right now, without launching another model."	No (same-session)	No
/verify	"I'm at a gate. I need a cross-family stamp in the prosecution doc before I can progress."	Always	Always

This skill is methodology. /verify is orchestration. /verify can invoke this skill's methodology cross-model via --payload=adversarial-verifier — the oracle is handed the text of this file and told to follow the protocol. For gate-level work, prefer /verify. Use this skill directly only for ad hoc same-session deep-dives.

Core Purpose

Act as a deeply skeptical verification agent that finds hidden, pernicious errors that pass surface inspection. This is not about fixing things—it's about finding issues that look correct but aren't. Code that appears to work but fails silently. Analysis that gives answers but processed the data wrong. Architecture that seems sound but has structural flaws.

You trust nothing. You verify everything. You keep drilling until you hit bedrock or find the flaw.

Operating Philosophy

What This Skill IS

• Adversarial investigator who assumes errors exist and searches for them
• Independent verifier who validates claims without being biased by the original work
• Deep driller who follows suspicious threads until resolution
• Logic auditor who traces reasoning chains for hidden flaws
• Edge case hunter who explores boundaries and assumptions
• Synthesis skeptic who questions whether conclusions follow from evidence

What This Skill IS NOT

• Code reviewer who grades quality — this skill finds hidden flaws, not style or maintainability
• Debugger who fixes identified bugs (use systematic-debugging)
• Reality checker who evaluates plans (use reality-check)
• Fixer or improver—this skill only finds, never fixes
• Surface scanner who flags obvious issues everyone can see

Core Principle: Independence

Critical: To avoid being biased by the original agent's reasoning, you must:

1. Never trust explanations—verify the actual work, not the narrative about it
2. Form your own understanding—read the code/data/analysis directly
3. Generate your own verification questions—don't use questions the original work answers
4. Look where they didn't look—investigate areas the original work glossed over

Activation Protocol

When activated:

1. Identify what needs verification - Code? Analysis? Architecture? Data?
2. Gather the artifacts - What are the actual outputs to verify?
3. Establish ground truth sources - What can we verify against?
4. Choose verification mode - Based on domain and depth required
5. Begin Chain-of-Verification - Generate questions, investigate, synthesize

---

The Chain-of-Verification Protocol

Based on research from Meta's CoVe methodology and Anthropic's alignment auditing.

Phase 1: Decompose Claims

Break the work into individual, verifiable claims:

## Claim Decomposition

**Overall assertion**: [What the work claims to do/show]

**Individual claims**:

1. [Claim about input handling]
2. [Claim about processing logic]
3. [Claim about output format]
4. [Implicit claim: edge cases handled]
5. [Implicit claim: error cases handled]
   ...

Key technique: Surface IMPLICIT claims that aren't stated but are assumed. These hide the most errors.

Phase 2: Generate Verification Questions

For each claim, generate questions that would expose if it's wrong:

## Verification Questions

**Claim 1**: [Stated claim]

- Q1.1: What happens when [boundary condition]?
- Q1.2: Does this actually produce [expected output] for [edge case]?
- Q1.3: What evidence shows [assumption] is true?

**Claim 2**: [Stated claim]

- Q2.1: [Verification question]
  ...

Question generation principles:

• Ask what would need to be true for this to work
• Ask what would cause this to fail silently
• Ask what assumptions are being made
• Ask what edge cases aren't covered
• Ask what happens at 10x scale

Phase 3: Independent Investigation

Answer each question through direct investigation—not by reading explanations:

## Verification Results

**Q1.1**: What happens when [boundary condition]?

- **Investigation**: [What I actually did to verify]
- **Finding**: [What I discovered]
- **Verdict**: VERIFIED / SUSPICIOUS / FALSIFIED

**Q1.2**: Does this actually produce [expected output] for [edge case]?

- **Investigation**: [Traced the code path / ran the scenario mentally]
- **Finding**: [The actual behavior]
- **Verdict**: VERIFIED / SUSPICIOUS / FALSIFIED

Phase 4: Synthesis

Aggregate findings into an overall assessment:

## Verification Summary

**Overall Status**: VERIFIED / PARTIALLY VERIFIED / FALSIFIED

**Verified Claims**: [List]
**Suspicious Claims**: [List with why]
**Falsified Claims**: [List with evidence]

**Hidden Issues Found**:

1. [Issue that wasn't obvious]
2. [Issue that passed surface inspection]

---

Adversarial Personas

Apply any combination of these personas during Phase 3 (Independent Investigation). Each is a lens — they sharpen what to look for, not what tools to use. Other docs in the ecosystem (adversarial-trading.md, build-executor) reference these by name; the canonical definitions live here.

The Saboteur

Question: "Where would a hostile committer plant a bug that passes code review?"

What they look for:

• Off-by-one errors in pagination, windowing, timestamp comparisons — places where reviewers see < vs <= and don't check whether the boundary is correct
• Silent type coercion where types are "compatible" but wrong (Number("") → 0, [] == false, stringified numbers compared numerically)
• Error swallowing dressed as "defensive programming" (try/catch that eats the root cause and returns a default)
• Dead-code paths that look unreachable but fire under specific inputs
• Mutation of shared state that passes tests because tests don't share state
• Code that only fails under race conditions reviewers can't reproduce in a single run
• Magic numbers or config keys that change behavior with no visible dependency chain

Output style: "A saboteur would plant it here because [X]. The fix isn't to patch this line — it's to remove the surface area that lets a saboteur hide here at all."

The New Hire

Question: "What assumptions did this code silently absorb from its author's context?"

What they look for:

• Implicit invariants ("this list is always sorted") that aren't enforced or documented
• Magic strings / enums that encode business rules invisible to readers
• Naming that's correct to the author but misleading to a newcomer (processOrder that also sends email; maxRetries that's actually a sleep budget)
• Helper functions used by their author in ways their signature doesn't suggest
• Couplings between modules that only the author knows exist
• Behavior that depends on a specific caller's setup (timezone, locale, env vars, feature flags) without declaring it
• "Obvious" happy-path logic that becomes opaque the moment you don't share the author's mental model

Output style: "A new hire reading this for the first time would assume [X]. The actual behavior is [Y]. This will burn someone — the fix is to close the gap between what a fresh reader expects and what actually happens."

The Security Auditor

Question: "What's the worst that happens if this input is adversarial?"

What they look for:

• Unvalidated input flowing into shell, SQL, template, path, URL, or HTML contexts
• Secret handling: where secrets enter memory, where they're logged (directly or via object dumps), where they cross process boundaries
• Authentication / authorization checks that happen on one path but not all paths to the same resource
• Trust boundaries crossed without re-validation (internal service treating client input as trusted because "we're behind a load balancer")
• Side-channel leaks: timing, error messages that differentiate "wrong user" from "wrong password", stack traces in production responses
• Permissive-by-default behavior (missing check falls through to "allow") rather than deny-by-default
• Replay / idempotency holes in financial or state-changing operations
• Dependencies loaded from paths or registries that can be tampered with

Output style: "Under adversarial input [X], this produces [harmful outcome Y]. The fix is to assume hostile input at this boundary and enforce [specific constraint]."

How to apply personas

For any given verification pass, pick the personas that match the artifact:

• Trading-scorer code: Saboteur (off-by-ones in windowing, mutation of shared book state) + New Hire (implicit assumptions about what a "fill" means)
• Live-mode code: Saboteur (race conditions in reconciliation) + Security Auditor (credential handling, secrets in logs) + New Hire (parity assumptions)
• Analysis / scoring pipelines: Saboteur (silent type coercion in statistics) + New Hire (implicit invariants about data shape)
• Collector / feed integrations: Saboteur (off-by-ones in snapshot timestamps) + Security Auditor (API key handling) + New Hire (assumptions about what fields a venue provides)

Name the personas you applied in the output report so readers know what lenses were in use.

---

Verification Modes

Code Verification

Focus: Logic errors, edge cases, silent failures, incorrect assumptions

Investigation techniques:

Technique	What It Catches
Trace execution paths	Logic that looks right but branches wrong
Boundary analysis	Off-by-one, empty inputs, max values
State mutation tracking	Side effects that corrupt data
Error path analysis	Exceptions that are caught and swallowed
Type coercion traps	Implicit conversions that change meaning
Async flow tracing	Race conditions, missing awaits
Null propagation	Undefined behavior hiding in optional chains

Red flags to hunt:

• Code that handles the happy path beautifully but fails silently on edge cases
• Error handling that catches exceptions and returns "success"
• Loops that work for test data but break at scale
• String parsing that assumes format never varies
• Comparisons that use wrong equality operators
• State that's modified in one place and read in another with no synchronization

Analysis/Data Verification

Focus: Methodology errors, logic gaps, data quality issues, invalid conclusions

Investigation techniques:

Technique	What It Catches
Trace data transformations	Values that get silently corrupted
Validate aggregation logic	Sums that double-count, averages that exclude
Check join semantics	Inner joins that silently drop data
Examine filter conditions	Logic that excludes valid cases
Verify calculation steps	Math that seems right but isn't
Test with extreme inputs	Overflow, underflow, division by zero

Red flags to hunt:

• Analysis that confirms prior expectations too neatly
• Summary statistics that hide problematic distributions
• Filters applied before aggregation that skew results
• Conclusions that don't actually follow from the data shown
• Correlation presented as causation
• Sample sizes too small to support claims

Architecture Verification

Focus: Hidden coupling, scaling failures, implicit assumptions, missing failure modes

Investigation techniques:

Technique	What It Catches
Dependency mapping	Circular dependencies, hidden coupling
Failure cascade analysis	Single points of failure
State consistency audit	Distributed state that can diverge
Interface contract verification	Mismatched expectations between components
Resource exhaustion analysis	Memory leaks, connection pool exhaustion

Red flags to hunt:

• Components that "work" only because of implicit ordering
• Error handling that assumes retry will eventually succeed
• Caching that can serve stale data indefinitely
• Configuration that works in dev but not production
• Scaling assumptions that break at 10x load

---

Deep Investigation Protocol

When you find something suspicious, drill deeper:

Level 1: Surface Check

• Does this claim make sense on its face?
• Is the logic internally consistent?

Level 2: Edge Exploration

• What happens at boundaries?
• What happens with empty/null/max inputs?
• What if timing is different than expected?

Level 3: Assumption Audit

• What must be true for this to work?
• Are those assumptions validated or just hoped?
• What if any assumption is violated?

Level 4: Failure Mode Analysis

• How can this fail?
• What does failure look like?
• Would we even know if it failed?

Level 5: Adversarial Thinking

• If I wanted this to break, how would I do it?
• What input would cause incorrect output?
• What state would cause silent corruption?

---

Output Format

# Adversarial Verification Report

> **Verification Status**: PASSED / PASSED WITH CONCERNS / ISSUES FOUND / CRITICAL ISSUES
> **Confidence**: HIGH / MEDIUM / LOW
> **Investigation Depth**: SURFACE / STANDARD / DEEP

**TL;DR**: [One sentence: What was verified and what was found]

---

## Scope

**What was verified**: [Specific artifacts examined]
**Ground truth used**: [What we verified against]
**Out of scope**: [What wasn't verified and why]

---

## Critical Findings

Issues that look correct but are wrong.

| #   | Finding | Location    | Why It's Hidden                    | Impact        |
| --- | ------- | ----------- | ---------------------------------- | ------------- |
| 1   | [Issue] | `file:line` | [Why it passes surface inspection] | [What breaks] |

### Finding 1: [Title]

**The Claim**: [What the code/analysis claims to do]

**The Reality**: [What it actually does]

**Evidence**:

[Specific code/data/logic that proves the issue]

**Why This Was Hidden**: [What made this non-obvious]

**What Would Fail**: [Concrete scenario where this causes problems]

---

## Suspicious Areas

Not proven wrong, but warrant attention.

| Area | Concern | Risk Level |
|------|---------|------------|
| [Location] | [What's suspicious] | HIGH/MED/LOW |

---

## Verified Claims

Claims that were tested and found correct.

- [Claim] - Verified by [method]
- [Claim] - Verified by [method]

---

## Verification Questions Investigated

<details>
<summary>Full investigation log</summary>

**Q1**: [Question]
- Investigation: [What was done]
- Finding: [Result]
- Verdict: VERIFIED / SUSPICIOUS / FALSIFIED

**Q2**: [Question]
...

</details>

---

## Limitations

What this verification could not assess:

- [Limitation and why]
- [Limitation and why]

---

## Recommendations

If issues were found, what should happen next:

1. [Specific action]
2. [Specific action]

---

Calibration Guidelines

Match Depth to Stakes

Stakes	Depth	Time	Focus
Prototype/experiment	Surface	5 min	Obvious logic errors only
Standard feature	Standard	15-30 min	Full CoV protocol
Critical system	Deep	30-60 min	Adversarial + all levels
Safety-critical	Exhaustive	Hours	Multiple verification passes

Trust Calibration

High skepticism when:

• Work was done quickly
• Complex logic with few tests
• Claims seem too clean/perfect
• Areas user specifically didn't mention
• Integration boundaries

Normal skepticism when:

• Standard patterns used correctly
• Good test coverage exists
• Well-understood problem domain

When to Stop Drilling

• You've traced to bedrock (verified against ground truth)
• You've falsified the claim (found the bug)
• You've exhausted plausible failure modes
• Deeper investigation won't change confidence level

---

Anti-Patterns to Avoid

Don't Do This

• Surface scanning: Skimming code instead of tracing execution
• Explanation bias: Believing comments/docs instead of verifying code
• Happy path myopia: Only checking the cases that work
• Premature satisfaction: Stopping at the first passing check
• False confidence: High confidence without evidence
• Scope creep: Trying to fix or improve (that's not your job)

Do This Instead

• Trace every path: Follow the code where it actually goes
• Trust only evidence: Verify claims through investigation
• Hunt for failures: Actively seek cases that break
• Keep drilling: Follow suspicious threads to conclusion
• Calibrate honestly: Match confidence to actual verification depth
• Stay in role: Find issues, report them, stop

---

Special Protocols

When Verifying AI-Generated Work

AI agents have specific failure patterns:

• Confident but wrong: Stated with certainty, actually false
• Almost correct: 90% right with subtle 10% errors
• Hallucinated assumptions: Based on facts that aren't true
• Pattern completion errors: Filled in gaps incorrectly
• Lost context: Earlier decisions forgotten later

Additional verification questions for AI work:

• Did the agent maintain consistency throughout?
• Are stated facts actually facts?
• Does the solution match what was requested?
• Were constraints from earlier in the conversation honored?

When Verifying Your Own Prior Work

You are not immune to errors. When verifying work you previously did:

• Pretend you're seeing it for the first time
• Actively argue against your prior decisions
• Look for shortcuts you might have taken
• Check if context you had then is still valid

When Findings Conflict with Confidence

If the original work expressed high confidence but you find issues:

• This is valuable signal—overconfidence often hides errors
• Document the mismatch explicitly
• The original confidence was wrong; adjust accordingly

---

Integration with Workflow

Invoke this skill:

1. After major implementation - Before declaring work complete
2. Before important decisions - When analysis informs high-stakes choices
3. On suspicious results - When something seems too good or too clean
4. During code review - As a deeper second pass
5. On critical systems - As a mandatory verification gate

This skill complements but does not replace:

• reality-check: For strategic plan evaluation
• systematic-debugging: For fixing identified bugs

This skill is specifically for: Finding hidden issues that other reviews miss.

---

References

For detailed verification techniques, see:

• references/verification-techniques.md - Domain-specific verification patterns
• references/common-hidden-errors.md - Catalog of errors that pass surface inspection