external-research

name

external-research

description

Large-scale external research across four trigger modes: (1) Pipeline plateau — breaking optimization bottlenecks when self-improve stalls; (2) Security/CVE — researching known vulnerabilities, CVE advisories, and security mitigations; (3) Compliance — researching regulatory requirements (GDPR, PCI-DSS, SOC2, etc.) and their technical implications; (4) Competitor/benchmark — researching industry-standard patterns, reference implementations, and design alternatives. TRIGGER for any of these four modes. DO NOT TRIGGER when the pipeline is making steady progress or the user just wants a quick web search.

External Research — Outside Knowledge for Stuck or Specialized Problems

Searches the broader world for new ideas, threat intelligence, regulatory guidance, or reference implementations. Four distinct trigger modes with different research strategies.

When to Use

Mode 1: Pipeline Plateau (optimization)

• The AI pipeline has plateaued (score not improving for 5+ cycles)
• User explicitly requests ("research externally", "break the bottleneck", "find new approaches")
• Self-improve's internal researcher has run out of novel ideas within the codebase

Mode 2: Security / CVE Research

• input-classifier emits Input-Type: Security (A6 frame)
• User reports a potential security vulnerability or references a CVE number
• Need to understand attack vectors, mitigations, or patch patterns for a known class of vulnerability

Mode 3: Compliance Research

• input-classifier emits Input-Type: Compliance (A8 frame)
• User references a regulation (GDPR, PCI-DSS, SOC2, HIPAA, etc.)
• Need to map regulatory requirements to technical implementation constraints

Mode 4: Competitor / Benchmark Research

• input-classifier emits Input-Type: Feedback/Idea with competitor comparison signal
• User asks "how does X solve this?" or "what's the industry standard for Y?"
• Need reference implementations or design patterns from analogous systems

Do NOT use when:

• The pipeline is making steady progress — internal iteration is sufficient
• The user just wants a quick web search (use WebSearch directly)
• The task is simple enough that external research would be overkill

Mode Dispatch

Detect the mode from context before selecting a research strategy:

Signal	Mode
[Intake] Input-Type: Security or CVE ID mentioned	Mode 2: Security/CVE
[Intake] Input-Type: Compliance or regulation name mentioned	Mode 3: Compliance
competitor comparison or "industry standard" question	Mode 4: Competitor
pipeline plateau (no score gain 5+ cycles)	Mode 1: Pipeline Plateau

Mode 2 research targets: NVD/CVE database, OWASP advisories, GitHub Security Advisories, language-specific security mailing lists, vendor patches.

Mode 3 research targets: Official regulation text, compliance framework documentation (SOC2 Trust Services Criteria, PCI-DSS requirements), NIST publications, industry implementation guides.

Mode 4 research targets: GitHub reference implementations, official documentation of analogous systems, technical blog posts from companies with similar architecture, benchmark results.

For Modes 2–4, skip the pipeline-specific Input fields and output a structured findings report to .claude/runs/task-briefs/<YYYY-MM-DD>_<slug>_research.md instead of idea_output_path.

---

Input (Mode 1: Pipeline Plateau)

Field	Required	Description
objective	Yes	The original improvement goal
current_score	Yes	Current benchmark score
target_score	Yes	Target score (acceptance criteria)
score_gap	Yes	Difference between current and target
approaches_tried	Yes	List of approach families and specific hypotheses already attempted, with outcomes
adrs	Yes	List of ADR files recording prior architectural decisions
codebase_summary	Yes	Brief description of the codebase, tech stack, and architecture
idea_output_path	Yes	Path to write the output ideas file (typically <self-improve-root>/config/idea.md)
iteration_history_path	No	Path to full iteration history for deeper analysis

Output

Write to idea_output_path a structured list of actionable ideas, each containing:

## External Research Ideas

### Research Context
- Objective: {objective}
- Current score: {current_score} / Target: {target_score} (gap: {score_gap})
- Approaches exhausted: {list of approach families tried}
- Research date: {date}

---

### Idea 1: {Short Title}

**Source**: {URL or paper citation}
**Approach family**: {architecture|optimization|data|infrastructure|training_config|testing|other}
**Specific technique**: {concrete description of what to do}
**Why it might work**: {reasoning based on the source evidence}
**Estimated impact**: {quantified estimate, e.g., "5-10% improvement"}
**How it differs from prior attempts**: {explicitly state what's new vs what was already tried}
**Trade-offs**: {what we gain vs what we lose}
**ADR conflicts**: {any prior ADR decisions this conflicts with, or "none"}

### Idea 2: {Short Title}
...

Instructions

Step 1: Analyze the Bottleneck

Before searching externally, deeply understand WHY the pipeline is stuck:

1. Read the iteration history: what approach families have been tried? What were the outcomes?
2. Read the ADRs: what decisions were made and why? Are any of them now suspect?
3. Identify the specific bottleneck: is it algorithmic? architectural? data-related? infrastructure?
4. Formulate the core question: "What technique could {specific improvement} given {specific constraint}?"

This analysis directly informs search queries — vague queries yield vague results.

Step 2: Search GitHub

Search for reference implementations and alternative architectures:

1. Repository search: Find repos that solve the same or similar problems
   • Search queries: combine the problem domain with keywords like "optimization", "high-performance", "alternative", "benchmark"
   • For each promising repo: read README, check star count and recent activity, examine architecture
2. Code search: Find specific implementation patterns
   • Search for the specific algorithm, data structure, or pattern that could break the bottleneck
3. Issue/Discussion search: Find known solutions to the specific problem
   • Search issues for keywords related to the bottleneck

For each finding, extract:

• What approach they use
• Why it works (from their docs/benchmarks)
• Key code patterns or architectural decisions
• Applicability to our codebase

Step 3: Search arXiv

Search for recent academic research:

1. Paper search: Find papers on the specific optimization problem
   • Focus on: optimization algorithms, architecture patterns, evaluation methodologies
   • Prioritize recent papers (last 2 years) for state-of-the-art techniques
2. Extract practical insights: Academic papers often contain theoretical insights that can be translated into practical improvements
   • Look for: algorithmic improvements, provably better approaches, new evaluation metrics
3. Assess practicality: Filter out purely theoretical work that cannot be implemented within the project's constraints

For each finding, extract:

• Core technique and why it's theoretically better
• Practical implementation considerations
• Expected magnitude of improvement
• Required changes to current architecture

Step 4: Search Technical Blogs & Documentation

Search for engineering experience and best practices:

1. Engineering blogs: Company engineering blogs often share real-world optimization stories
   • Search for: performance optimization case studies, architecture migration stories
2. Official documentation: Framework/tool docs may reveal features not yet utilized
   • Check for: configuration options, performance tuning guides, lesser-known features
3. Performance tuning guides: Platform-specific optimization advice
   • Look for: language-specific optimizations, runtime tuning, compiler flags

Step 5: Search Forums & Communities

Search for practitioner solutions:

1. Stack Overflow: Specific technical Q&A
2. Reddit/Discord communities: Broader discussion and experience sharing
3. GitHub Discussions: Project-specific optimization advice

Step 6: Deep Analysis & Filtering

Synthesize all findings into actionable ideas:

1. Cross-reference with prior attempts: Every idea must explicitly state how it differs from what was already tried. Ideas that merely repeat failed approaches must explain why the outcome might be different this time.

2. Cross-reference with ADRs: Check each idea against existing architectural decisions:

   • If an idea conflicts with an ADR: note the conflict, explain why revisiting the decision might be warranted
   • If an idea aligns with an ADR: note the reinforcement

3. Feasibility assessment: For each idea, assess:

   • Can it be implemented within the current codebase structure?
   • Does it require new dependencies? (prefer ideas that don't)
   • What is the estimated implementation complexity?
   • What is the risk of regression?

4. Impact estimation: Rank ideas by estimated impact (high/medium/low confidence)

5. Diversity check: Ensure ideas span at least 3 different approach families. If all ideas are in the same family, search more broadly.

Step 7: Write Output

Write the structured ideas to idea_output_path following the output format above.

Quality requirements:

• Minimum 3 ideas, maximum 10
• Every idea must have a specific, citable source
• Every idea must explain how it differs from prior attempts
• Ideas sorted by: confidence × estimated impact (highest first)
• At least 2 different approach families represented

Step 8: Report

Present a summary to the user:

=== External Research Complete ===

Bottleneck analysis: {why we're stuck}
Sources searched: GitHub ({N} repos), arXiv ({N} papers), Blogs ({N} posts), Forums ({N} threads)

Ideas generated: {count}
- High confidence: {count}
- Medium confidence: {count}
- Low confidence: {count}

Approach families covered: {list}

Top 3 ideas:
1. {title} ({approach_family}, {estimated_impact})
2. {title} ({approach_family}, {estimated_impact})
3. {title} ({approach_family}, {estimated_impact})

Ideas written to: {idea_output_path}

Hard Constraints

These constraints are absolute and must NOT be relaxed regardless of what external research finds:

1. Original goal objective — do not lower the bar or redefine "done"
2. Acceptance criteria (target score) — do not adjust the target downward
3. Evaluation benchmark — do not weaken the test suite or change grading criteria
4. Regression thresholds — do not allow more regression

New ideas must achieve the SAME target with DIFFERENT means, not a different (easier) target. If external research suggests the target is unrealistic, report this to the user but do NOT unilaterally lower it.

Failure Strategy

Situation	Action
No relevant findings on GitHub	Expand search terms, try adjacent domains, continue to arXiv
No relevant papers on arXiv	Try different keyword combinations, search citation graphs
All findings are impractical	Report honestly — sometimes the bottleneck is fundamental. Suggest whether the user should reconsider the target.
Findings conflict with hard constraints	Discard the finding. Do NOT suggest relaxing constraints.
Search returns too many results	Narrow queries with more specific terms, filter by recency and relevance
Cannot access a source	Note the source, try alternative access (cache, mirrors), do not skip without trying

Pipeline Integration

This skill is a conditional phase in the AI pipeline loop. It sits between cycles as Phase 6 (conditionally triggered):

Phase 5 (ai-slop-cleaner)
     │
     ▼
Acceptance Check ─── Goal met? ──→ Deliver & Exit
     │
     │ NO
     ▼
Bottleneck Check ─── Score plateaued 5+ cycles? ──→ Phase 6: external-research
     │                                                    │
     │ NO                                                 ▼
     │                                              Re-enter loop
     ▼                                             (Phase 4 or Phase 1)
Re-enter loop (Phase 4)

After external-research completes, the pipeline re-enters at:

• Phase 4 (self-improve) if the new ideas are incremental improvements within the current architecture
• Phase 1 (blueprint) if the findings suggest a fundamentally different approach requiring re-planning

This skill can also be used standalone outside the pipeline when the user wants targeted external research on a specific problem.