literature-scout

name	literature-scout
description	Literature Scout Module. Orchestrates 5 agents to complete the entire literature retrieval process: keyword search / seed recommendation → relevance filtering → batch download. Triggered proactively by the user. Trigger keywords: search papers, find literature, 搜索文献, 找论文, seed recommendation, expand from seeds, 种子推荐, 扩展论文.
metadata	{"version":"2.0.0","last_updated":"2026-05-29","status":"active","task_type":"open-ended"}

⚠️ Mandatory Prerequisite Loading

Before starting execution, you MUST read the following files:

1. Read agents/literature_scout_agent.md — Master control agent definition
2. Read agents/query_builder_agent.md — Keyword search agent
3. Read agents/seed_expander_agent.md — Seed recommendation agent
4. Read agents/relevance_judge_agent.md — Relevance judging agent
5. Read agents/batch_fetch_agent.md — Batch fetch agent

If any file is not loaded, execution must NOT commence.

---

⚠️ Prerequisites

This module relies on the Semantic Scholar MCP server. Before using, ensure that:

1. The MCP server is registered in Claude Code (check if .mcp.json contains semantic-scholar-cars).
2. Python dependencies are installed: pip install mcp httpx.
3. The API Key is configured: cars/config/api_keys.json contains semantic_scholar.
4. user_profile.json exists (the Socratic dialogue must be completed first).

If the MCP tools are unavailable, this module cannot be executed. DO NOT substitute with WebSearch.

---

Literature Scout Module

Orchestrates multiple agents to complete the entire literature retrieval process from "Search → Filter → Download", saving relevant papers to the project literature library.

Core Positioning: This module does not deep read papers, does not generate research matrices, and does not conduct Socratic dialogues with the user. It only does one thing — find relevant papers and download them locally.

---

Trigger Conditions

Trigger Method

Proactively triggered by the user. This module does not run automatically.

Trigger Keywords

• "search papers" / "find literature" / "help me find related papers"
• "seed recommendation" / "expand papers" / "find more based on these papers" / "expand from seeds"
• "搜索文献" / "找论文" / "帮我找相关论文"
• "种子推荐" / "扩展论文" / "基于这些论文找更多"

Scenarios NOT Triggering This Module

Scenario	Alternative Module
User wants to deep read downloaded papers	Literature Reader Module
User wants to generate a research matrix	Matrix Engine Module
User wants to explore research directions	Socratic Method Module

---

Available MCP Tools

Tool	Purpose	Caller
search_papers(query, limit)	Natural language keyword search	query_builder_agent
get_recommendations(positive_paper_ids, limit)	Recommendation expansion based on seed papers	seed_expander_agent
batch_get_papers(paper_ids)	Batch fetch complete metadata (including openAccessPdf)	batch_fetch_agent
save_paper_json(paper_data, save_dir, title)	Save a single paper as JSON	batch_fetch_agent
download_pdf(pdf_url, save_dir, title)	Download Open Access PDF	batch_fetch_agent

---

Agent Team (5 Agents)

#	Agent	Role	Invocation Condition
1	literature_scout_agent	Master Control: User interaction, routing & dispatch, midway presentation, result reporting	Throughout
2	query_builder_agent	Extracts keywords from profile → Calls search_papers → Outputs candidate set	User selects "keyword search"
3	seed_expander_agent	Reads search_seeds from profile → Calls get_recommendations → Outputs candidate set	User selects "seed recommendation"
4	relevance_judge_agent	Reads abstract → Confidence scoring → Grading (high/medium/low)	After search completes
5	batch_fetch_agent	Batch fetch metadata + Save JSON + Download PDF	After filtering completes

---

Three Search Methods

Method	Input Source	MCP Tool	Applicable Scenario
Keyword Search	insights/open_threads/reading_refs from profile	search_papers	Early project phase, empty literature library
Seed Recommendation	search_seeds[] from profile (derived from supervised dialogue)	get_recommendations	User targets specific matrix cells, wants to dig deeper
Use Both	Simultaneously uses both methods above	Both in parallel	Comprehensive coverage

---

Orchestration Flow

User triggers "search papers"
     |
=== Pre-checks ===
     |
     +-> Does user_profile.json exist?
     |   - No → Instruct user to complete Socratic dialogue first, terminate
     |
     +-> Are MCP tools available?
     |   - No → Instruct user to check configuration, terminate
     |
=== Prompt for Search Method ===
     |
     +-> User selects:
     |   1. Keyword Search
     |   2. Seed Recommendation (Checks if search_seeds has active entries)
     |   3. Use Both
     |
=== Execute Search ===
     |
     +-> [Keyword Search] → query_builder_agent
     |   - Extracts keywords from profile
     |   - Generates 2-5 queries
     |   - Calls search_papers × N times
     |   - Deduplicates and outputs candidates_*.json
     |
     +-> [Seed Recommendation] → seed_expander_agent
     |   - Reads search_seeds[] (status=active)
     |   - Calls get_recommendations × N times
     |   - Deduplicates and outputs candidates_seed_*.json
     |
     +-> [Use Both] → Execute in parallel → Merge and deduplicate
     |
=== Relevance Filtering ===
     |
     +-> relevance_judge_agent
     |   - Reads candidates + user_profile
     |   - Evaluates confidence level item by item (high/medium/low)
     |   - Outputs filtered_*.json
     |   - Branches into: auto_fetch_paper_ids + user_review_paper_ids
     |
=== User Review ===
     |
     +-> literature_scout_agent presents medium/low papers to the user
     |   - User confirms whether to keep or discard
     |
=== Batch Download ===
     |
     +-> batch_fetch_agent
     |   - Deduplication check (Do not re-download existing papers)
     |   - Calls batch_get_papers to fetch metadata
     |   - Calls save_paper_json to save individual JSONs
     |   - Calls download_pdf to download Open Access PDFs
     |   - Updates raw_metadata.json
     |
=== Status Update (During Seed Recommendation) ===
     |
     +-> Triggers Profile Module to update search_seeds.status = "consumed"
     |
=== Result Reporting ===
     |
     +-> Reports to user: Download count, PDF count, suggestions for next steps

---

Output Directory Structure

workspaces/ProjectX/literature/
├── candidates_YYYYMMDD_HHmm.json       ← Keyword search candidate set
├── candidates_seed_YYYYMMDD_HHmm.json  ← Seed recommendation candidate set
├── filtered_YYYYMMDD_HHmm.json         ← Post-filtering results
├── raw_metadata.json                    ← Aggregated metadata (incrementally appended)
├── jsons/                               ← Individual JSON for each paper
│   ├── Concept_Bottleneck_Models.json
│   └── ...
└── pdfs/                                ← Open Access PDFs
    ├── Concept_Bottleneck_Models.pdf
    ├── processed/                       ← Archived deep-read PDFs
    └── ...

---

Quantity Control

Phase	Quantity Limit
query_builder per query	10-15 papers
query_builder total candidates after deduplication	40-60 papers
seed_expander per seed	10 papers
relevance_judge → auto_fetch (high)	Max 10 papers
relevance_judge → user_review (medium)	Max 10 papers
Final amount entering batch_fetch	10-20 papers

---

Ironclad Rules

1. NO Fictional Papers — Only output results returned by MCP tools.
2. NO WebSearch Substitution — Terminate if MCP is unavailable; do not degrade.
3. User MUST Have Review Rights — Papers with medium/low confidence MUST be presented to the user for confirmation.
4. Auto-Routing for High Confidence — High confidence flows automatically without bothering the user.
5. Errors are NEVER Silent — Any API failure MUST be reported to the user.
6. NO Duplicate Downloads — batch_fetch MUST check raw_metadata.json for deduplication.
7. Seed Status MUST Update — Using seed recommendation MUST trigger a status update.

---

Failure Paths

Failure Scenario	Trigger Condition	Recovery Strategy
Profile does not exist	User has not completed Socratic dialogue	Inform user, terminate
MCP tools unavailable	Server not registered	Inform user to check configuration, terminate
No active search_seeds	User selects seed recommendation but has no seeds	Suggest switching to keyword search
API Call Failed	Network / Quota / Key issues	Log error, skip this call, continue
All API Failed	All searches fail	Inform user, terminate
Candidate Set Empty	Search yields no results	Suggest user adjust keywords or direction
User does not review	Long period without response	Remind user, or suggest "keep all"

---

Relationship with Other Modules

[Socratic Method] → session → [Profile Module] → user_profile.json (contains search_seeds)
    ↓
[Literature Scout] → Reads profile → Searches → Filters → Downloads
    ↓
literature/jsons/ + literature/pdfs/
    ↓
[Literature Reader] → Deep reads PDF → hypotheses/translated/*.json
    ↓
[Matrix Engine] → Generates matrix.json
    ↓
[Socratic Method (Supervised)] → Reads matrix → Continues dialogue

---

Agent File References

Agent	Definition File
literature_scout_agent	agents/literature_scout_agent.md
query_builder_agent	agents/query_builder_agent.md
seed_expander_agent	agents/seed_expander_agent.md
relevance_judge_agent	agents/relevance_judge_agent.md
batch_fetch_agent	agents/batch_fetch_agent.md

---

MCP Tool Dependencies

Tool	File	Purpose
search_papers	tools/s2_mcp_server.py	Keyword search
get_recommendations	tools/s2_mcp_server.py	Seed recommendation
batch_get_papers	tools/s2_mcp_server.py	Batch fetch metadata
save_paper_json	tools/s2_mcp_server.py	Save individual JSON
download_pdf	tools/s2_mcp_server.py	Download PDF
batch_download_arxiv_pdfs	tools/s2_mcp_server.py	Batch download arXiv PDFs