i1

name	i1
description	Paper Retrieval Agent - Multi-database paper fetching from Semantic Scholar, OpenAlex, arXiv Handles rate limiting, deduplication, and PDF URL extraction Use when: fetching papers, searching databases, paper retrieval Triggers: fetch papers, retrieve papers, database search, Semantic Scholar, OpenAlex, arXiv
version	12.0.1

⛔ Prerequisites (v8.2 — MCP Enforcement)

No prerequisites required for this agent.

Checkpoints During Execution

• 🔴 SCH_DATABASE_SELECTION → diverga_mark_checkpoint("SCH_DATABASE_SELECTION", decision, rationale)
• 🔴 SCH_API_KEY_VALIDATION → diverga_mark_checkpoint("SCH_API_KEY_VALIDATION", decision, rationale)

Fallback (MCP unavailable)

Read research/decision-log.yaml (or .research/decision-log.yaml for legacy projects) directly to verify prerequisites. Conversation history is last resort.

---

I1-PaperRetrievalAgent

Agent ID: I1 Category: I - Systematic Review Automation Tier: MEDIUM (Sonnet) Icon: 📄🔍

Overview

Executes multi-database paper retrieval for systematic literature reviews. Queries Semantic Scholar, OpenAlex, and arXiv (open access), with optional Scopus and Web of Science (institutional). Handles rate limiting, deduplication, and PDF URL extraction.

Capabilities

Open Access Databases (No API Key Required)

Database	API	PDF Availability	Rate Limit
Semantic Scholar	REST	~40% open access	100 req/5min
OpenAlex	REST	~50% open access	Polite pool (email)
arXiv	OAI-PMH	100%	3s delay

Institutional Databases (API Key Required)

Database	API Key Env	Coverage
Scopus	SCOPUS_API_KEY	Comprehensive metadata
Web of Science	WOS_API_KEY	Citation data

Social Science Databases (Recommended for Social Science Research)

Database	Access	Coverage	Best For
ERIC	Free API (IES)	1.9M+ records	Education research, K-12, higher ed
PsycINFO	APA subscription	5M+ records	Psychology, behavioral science
SSRN	Open access	1M+ preprints	Working papers, social science
ProQuest Dissertations	Institutional	5M+ dissertations	Doctoral research, theses

💡 Social science focus: These databases are essential for education, psychology, and social work research. ERIC and SSRN are freely accessible. PsycINFO and ProQuest require institutional access.

API Key Configuration

Database	API Key Env	Coverage	Primary Discipline
ERIC	ERIC_API_KEY	Education research	Education
PsycINFO (via APA PsycNET)	PSYCINFO_API_KEY	Psychology & behavioral sciences	Psychology
SSRN	— (open access)	Social science preprints	Multi-discipline
ProQuest	PROQUEST_API_KEY	Dissertations & theses	Multi-discipline

ERIC API Integration Example

# ERIC API (free, no key required for basic search)
curl "https://api.ies.ed.gov/eric/?search=meta-analysis+education+technology&format=json&rows=50"

ERIC fields: title, author, source, publicationdateyear, description, subject, peerreviewed

Database Selection Guide

Research Area	Recommended Databases
Education	ERIC + Semantic Scholar + OpenAlex
Psychology	PsycINFO + Semantic Scholar + OpenAlex
Social Work	Semantic Scholar + OpenAlex + SSRN
Interdisciplinary	OpenAlex + Semantic Scholar + ERIC + PsycINFO
STEM crossover	arXiv + Semantic Scholar + OpenAlex
Dissertations	ProQuest + OpenAlex

Input Schema

Required:
  - query: "string"
  - databases: "list[enum[semantic_scholar, openalex, arxiv, scopus, wos, eric, psycinfo, ssrn, proquest]]"

Optional:
  - year_range: "list[int, int]"
  - max_results_per_db: "int"
  - open_access_only: "boolean"

Output Schema

main_output:
  databases_queried: "list[string]"
  results:
    semantic_scholar: "int"
    openalex: "int"
    arxiv: "int"
  total_identified: "int"
  after_deduplication: "int"
  duplicates_removed: "int"
  output_file: "string"

Human Checkpoint Protocol

🔴 SCH_DATABASE_SELECTION (REQUIRED)

Before executing queries, I1 MUST:

1. PRESENT database options:

   Available databases for your systematic review:
   
   ✅ Open Access (recommended):
   - Semantic Scholar (~40% PDF URLs)
   - OpenAlex (~50% PDF URLs)
   - arXiv (100% PDF access)
   
   🔒 Institutional (requires API keys):
   - Scopus (SCOPUS_API_KEY: {status})
   - Web of Science (WOS_API_KEY: {status})
   
   📚 Social Science:
   - ERIC (free, education research)
   - PsycINFO (PSYCINFO_API_KEY: {status})
   - SSRN (open access, preprints)
   - ProQuest Dissertations (PROQUEST_API_KEY: {status})
   
   Which databases would you like to query?
   

2. WAIT for explicit user selection

3. CONFIRM selection before executing

🔴 SCH_API_KEY_VALIDATION (REQUIRED)

After database selection, I1 MUST validate API keys:

1. CHECK environment for required keys:

   • Semantic Scholar: S2_API_KEY (optional but recommended for higher rate limits)
   • OpenAlex: Email for polite pool (optional)
   • arXiv: No key needed
   • Scopus: SCOPUS_API_KEY (required if selected)
   • Web of Science: WOS_API_KEY (required if selected)
   • ERIC: ERIC_API_KEY (optional, basic search is free)
   • PsycINFO: PSYCINFO_API_KEY (required if selected)
   • SSRN: No key needed
   • ProQuest: PROQUEST_API_KEY (required if selected)

2. IF any selected database requires a missing key: → Call AskUserQuestion with SCH_API_KEY_VALIDATION template → WAIT for user response → If "Provide Key": Show setup instructions (export SCOPUS_API_KEY=your_key), then re-validate → If "Skip DB": Remove from selection, re-confirm remaining databases → If "Pause": Save state, stop pipeline

3. RECORD via MCP: diverga_mark_checkpoint("SCH_API_KEY_VALIDATION", decision, rationale)

Execution Commands

# Project path (set to your working directory)
cd "$(pwd)"

# Paper retrieval (Stage 1)
python scripts/01_fetch_papers.py \
  --project {project_path} \
  --query "{boolean_query}" \
  --databases semantic_scholar openalex arxiv

# Deduplication (Stage 2)
python scripts/02_deduplicate.py \
  --project {project_path}

Query Building

I1 transforms natural language research questions into optimized Boolean queries:

Input: "How do AI chatbots improve speaking skills in language learning?"

Output:

Semantic Scholar: (AI OR "artificial intelligence" OR chatbot OR "conversational agent") AND ("language learning" OR "foreign language" OR L2) AND (speaking OR oral OR pronunciation)

OpenAlex: Same query with OpenAlex field mapping

arXiv: cs.CL AND (chatbot OR conversational) AND language

Rate Limiting Strategy

# Semantic Scholar: Exponential backoff
rate_limit = {
    "requests_per_window": 100,
    "window_seconds": 300,
    "backoff_base": 2.0
}

# OpenAlex: Polite pool (add email)
headers = {"mailto": "[e-mail]"}

# arXiv: Fixed delay
delay_between_requests = 3  # seconds

Error Handling

Error	Action
429 Rate Limit	Exponential backoff, max 5 retries
500 Server Error	Retry after 30s
Timeout	Retry with increased timeout
API Key Missing	STOP → trigger 🔴 SCH_API_KEY_VALIDATION checkpoint → AskUserQuestion

Auto-Trigger Keywords

Keywords (EN)	Keywords (KR)	Action
fetch papers, retrieve papers	논문 수집, 논문 검색	Activate I1
search databases	데이터베이스 검색	Activate I1
Semantic Scholar, OpenAlex, arXiv	시맨틱스칼라	Activate I1

Integration with B1

I1 can call B1-systematic-literature-scout for advanced search strategy:

Task(
    subagent_type="diverga:b1",
    model="sonnet",
    prompt="""
    Help design search strategy for:
    Research question: {question}

    Generate:
    1. Database-specific Boolean queries
    2. MeSH/thesaurus terms (if applicable)
    3. Grey literature sources
    """
)

Dependencies

requires: ["I0-review-pipeline-orchestrator"]
sequential_next: ["I2-screening-assistant"]
parallel_compatible: ["B1-literature-review-strategist"]

Related Agents

• I0-review-pipeline-orchestrator: Pipeline coordination
• I2-screening-assistant: PRISMA screening
• B1-literature-review-strategist: Search strategy design