deep-science-writer

name	deep-science-writer
description	End-to-end scientific research pipeline combining Exa Search, Playwright, deep-research, text-humanization, and iterative Remi peer review.

Deep Science Writer (End-to-End Pipeline)

This skill orchestrates a multi-stage pipeline for scientific research, combining neural search, browser automation, academic synthesis, and humanized writing. It merges the capabilities of Exa Search, Playwright, Google Science Skills, Deep Research, and Text Humanization into one cohesive workflow.

🎯 Trigger Conditions

• User asks to write a scientific article, literature review, or deep-dive research report.
• User requests comprehensive research using web searching and scraping (Exa + Playwright).
• The output requires rigorous academic citations (APA 7th) but a highly natural, human-like reading flow.

📋 Workflow Steps

Phase 0: Mandatory Deep Context Gathering & Research Plan

1. Incorporate academic rigor, structured literature parsing, and cross-validation techniques into your approach and keep those principles active in context. Reference material for these lives in the local repos C:/Users/User/workspace/nature-skills/, C:/Users/User/workspace/academic-research-skills/, and C:/Users/User/workspace/superpowers/ — these are not installed skills, so read from them with Read/Grep rather than trying to invoke them.
2. Mandatory Interrogation: You MUST deeply understand the user's current research topic before doing anything else. If the user's initial prompt is too brief, you MUST actively interrogate them to extract their exact pain points, research hypotheses, and core objectives. Do NOT proceed until you fully grasp the context.
3. Mandatory Pre-Flight Discussion: Once the context is understood, formulate a preliminary research plan (proposed keywords, target databases, expected article structure, and goals).
4. Present this blueprint to the user to discuss the research direction. Focus on delegating the heavy lifting to parallel subagents via Agent to protect the main context window.
5. Halt & Wait: You MUST wait for the user's "Explicit Approval" of the research plan before proceeding to Phase 0.5.

Phase 0.5: Background Sourcing, Full-Text Deep Reading, & Gap Analysis

1. Large-Scale Abstract Screening: Delegate subagents or write a background script to fetch a large pool of abstracts (e.g., 100+ papers) via APIs. Screen these abstracts for strict relevance and Q1/Q2 quality first.
2. Targeted Full-Text Downloading: Once the abstracts are verified to match the research need, the script MUST download or scrape the entire full text of the filtered subset (e.g., top 20-30 papers).
3. Deep Reading & Filtering: Read the downloaded full texts (specifically Methodology and Results) to extract the actual mechanisms. Discard any papers that fail to substantiate the claims in their abstracts or overstate findings.
4. Long-Running Execution: Execute the script with the Bash tool using run_in_background: true. You are re-invoked automatically when it exits — do not poll. Wait for that notification before proceeding.
5. Mandatory Justification & Gap Report: Before moving to synthesis or drafting, you MUST present a structured evaluation report to the user containing exactly these three sections:
   • Selection Rationale (為什麼讀這幾篇): Explicitly explain how these specific papers map precisely to the user's stated topic and pain points.
   • Research Gap (研究缺口在哪): Identify what these papers leave unsolved or unaddressed, defining the exact gap in the current literature.
   • Topic Enhancement (對課題有什麼提升): Clearly articulate how these findings can be fed back into the user's project to improve their experimental design, strengthen their argument, or correct their research trajectory.
6. Synthesis & Synthesis Mastery: Only after the user approves the Gap Report, spawn Subagent C (Master Synthesizer) to merge and critically analyze the findings into a formal academic synthesis report. (Note: Override the user's general chat preference for concise lists; use standard formal academic prose here.)

Phase 1: Strict Multi-Agent Discovery (scopus-mcp + exa-search + openalex + semantic-scholar + anysearch)

1. Journal Quality Filter: ONLY include Q1 and Q2 papers. If a Q3 paper provides crucial evidence, you MUST explicitly mark it in the text/table with [Q3]. STRICTLY EXCLUDE any Q4 papers and ANY papers published by MDPI.

2. Mandatory 5-Subagent Deployment: You MUST spawn exactly five concurrent subagents using Agent. Assign each subagent to specialize in one core database: Subagent 1 (Scopus), Subagent 2 (Exa Search), Subagent 3 (OpenAlex), Subagent 4 (Semantic Scholar), and Subagent 5 (AnySearch vertical academic search and citation graph).

3. Mandatory Subagent Rules: The subagents MUST strictly adhere to the MDPI/Q4 exclusion rules.

4. Mandatory Toolset Utilization: Each subagent (or the collective effort) MUST explicitly utilize ALL of the following databases to ensure exhaustive coverage:

   • scopus-mcp (for Elsevier/authoritative DB)
   • exa-search (for neural web search and Open Access discovery)
   • OpenAlex API (via Python/Node.js scripts, strictly filtering out MDPI)
   • Semantic Scholar API (via Python/Node.js scripts)
   • AnySearch vertical academic search (via the anysearch skill CLI -- see Subagent 5 below) Failure to use all five sources is a violation of this skill.

5. Subagent 5 -- AnySearch (vertical academic + citation graph). Installed at C:/Users/User/.claude/skills/anysearch/. Runs a bundled CLI against public HTTP endpoints; no MCP server, and no API key required (anonymous access works; a key only raises rate limits).

   Its distinct value is the academic vertical domain, which the other four cannot match on cost or reliability:

   sub_domain	Use for
   academic.citation	Citation graph by DOI. op=citations (cited-by), op=references (full reference list with DOIs), op=citation-count, op=author. The highest-value call in this phase.
   academic.search	Cross-discipline paper search by keyword, title, author, institution; returns metadata and open-access links
   academic.preprint

Phase 2: Deep Extraction (cloakbrowser + deep-research)

1. Dynamic Scraping: For high-value sources that require JavaScript rendering, interaction, or are heavily dynamically loaded, use the CloakBrowser tool/CLI to navigate, bypass anti-bot protections, wait for specific DOM selectors, and extract the full text. Do NOT use standard Playwright for this, as CloakBrowser is the primary scraper.
2. Synthesis: Cross-reference extracted findings across multiple sources to validate claims and identify consensus vs. controversy.

Phase 3: Structural Drafting (article-writing)

1. Load the article-writing skill via the Skill tool with skill: "hermes-research:article-writing", then outline the article on its principles: strong hook, logical progression, evidence-backed claims, and clear logical headings.
2. Integrate data explicitly. Use block equations or Mermaid diagrams if explaining complex systems or workflows.
3. Format all inline citations and the final reference list strictly in APA 7th format. When generating via python-docx, programmatically implement APA hanging indents (p.paragraph_format.first_line_indent = Inches(-0.5) and p.paragraph_format.left_indent = Inches(0.5)) and ensure journal/book titles and volume numbers are properly italicized.

Phase 4: Humanization & Polish (text-humanizer)

1. Load the text-humanizer skill via the Skill tool with skill: "hermes-research:text-humanizer", then review the generated draft and ruthlessly strip AI-isms, applying the strict "No Fluff" style profile.
2. Banned AI Vocabulary: "delve", "tapestry", "in conclusion", "crucial", "testament", "realm", "fosters", "underscores", "moreover".
3. Style Adjustments:
   • Use varied sentence lengths (short, punchy sentences mixed with complex ones).
   • Prefer active voice over passive voice.
   • Remove introductory meta-commentary ("Here is the article...").
   • Ensure the prose flows like a subject-matter expert speaking directly to a peer.

Phase 4.5: Anti-Hallucination & Evidence Verification

Strict Requirement: This phase MUST be completed before Remi (Phase 5) is allowed to review the manuscript.

1. Extraction: Scan the draft and extract every single in-text citation (e.g., Smith, 2024) and its corresponding Reference List entry.
2. DOI/URL Liveness Test: Use the Bash tool (via curl -I), Exa Search MCP, or a Python script (e.g., requests.get / urllib) to ping every DOI or URL in the reference list. For bulk or programmatic URL resolution when links are missing, execute scripts/verify_urls.py (uses ddgs and requests).
   • STRICT REPLACEMENT RULE: If a DOI/URL is dead (404) or fake, you MUST NOT simply delete the sentence or claim. You MUST trigger a targeted secondary search (Exa/Scopus) to find a real Q1/Q2 replacement paper that supports the exact claim. Download the new full-text, verify it, and insert the new citation.
3. Claim Grounding Check: Cross-reference the specific claims made in the draft against the raw data/abstracts collected in Phases 1 & 2.
   • Strict Literalism (就事論事): Never over-extend findings. If a claim overstates the actual findings, down-modulate it.
   • No Concept Stitching: Do not stitch a macro/global finding (e.g., "planetary boundaries") with a localized context (e.g., "European summer extremes") in the same citation unless the source explicitly connects them. Stick strictly to the literal facts. If a study analyzes a broad region (e.g., "global temperate drylands"), do not misattribute its specific localized mechanism to a single city (e.g., "Paris") just because that city sits within the region.

• Differentiate Plant Functional Types (PFTs): When interpreting drought resilience, strictly differentiate between shallow-rooted ground vegetation (grass) and deep-rooted urban trees. Do not mistakenly apply a "plant shutdown" mechanism to trees if the literature explicitly states only the ground vegetation was affected due to shallow soil moisture depletion.

4. Continuous Replacement Loop: Repeat the search, replacement, and verification process until EVERY single factual statement or claim in the manuscript is backed by a verified, live reference.
5. Pass Condition: Zero dead links, zero fake DOIs, zero ungrounded claims, and 100% of claims supported by valid references. Only then proceed to Phase 5.

Phase 4.6: Zotero Archiving & PDF Download

Strict Requirement: This phase bridges verified evidence with bibliography management before Remi's review.

1. PDF Retrieval & Anti-Bot Bypass: For every paper successfully verified in Phase 4.5, you MUST download the original Publisher PDF to the local assets/ directory.
   • Step 1 (Open Access): Attempt fast retrieval via scripts/fetch_oa_fulltexts.py (Unpaywall API).
   • Step 2 (Paywall/Cloudflare Bypass): If Paywalled or blocked by 403/Cloudflare (e.g., Wiley, Elsevier), you MUST execute a Python script utilizing cloakbrowser (launch_context_async) to navigate to the DOI landing page or PDF direct link. cloakbrowser will natively bypass bot protection by leveraging the user's University IP environment and stealth fingerprints.
2. Zotero Ingestion (via pyzotero):
   • Load the pyzotero skill via the Skill tool with skill: "hermes-research:pyzotero" to integrate with the Zotero v3 API.
   • For each verified paper, create a Parent Item in Zotero (e.g., journalArticle) populated with title, authors, year, DOI, and URL.
3. PDF Attachment:
   • Use zot.attachment_simple([local_pdf_path], parentid=parent_key) to upload the downloaded PDF as an attachment to its respective Zotero Parent Item.
4. Error Handling: If a PDF cannot be downloaded due to aggressive Paywalls, log the failure but still create the Parent Item in Zotero with the DOI/URL to ensure the bibliography record exists.

Phase 5: Peer Review & Iteration (remi)

1. Load the Remi skill via the Skill tool with skill: "hermes-research:remi" to act as a strict Nature/Science-level peer reviewer.
2. Submit the Phase 4 draft to Remi for a rigorous evaluation against evidence backing, APA 7th formatting, logical flow, and academic tone.
3. Iteration Loop (No-Lazy-Editing Rule): Analyze Remi's (or the user's) critique. If structural gaps, missing evidence, or logical errors are found, you are STRICTLY FORBIDDEN from simply deleting the problematic text or rewording it to sound better.
4. Mandatory Secondary Search: For any substantive critique or missing evidence, you MUST regress to Phase 1/2, conduct a new targeted literature search to find missing evidence, download the new papers, and rewrite the section based on the newly acquired data.
5. Repeat the Remi review process until Remi approves the manuscript with zero critical concerns.

Phase 6: Data Visualization & Final Document Generation (.docx)

CRITICAL RULE: You MUST NOT generate or export the final .docx file until Phase 4 (Text Humanizer), Phase 4.5 (DOI Verification), and Phase 5 (Remi Review) have been explicitly executed and passed. Skipping these steps before exporting is a strict violation of this pipeline.

1. Infographic & Statistical Plotting:
   • For highly engaging, data-driven storytelling and statistical plots, you MUST utilize the AntV Infographic (@antv/infographic) framework.
   • Write a short Node.js/HTML script that uses the AntV declarative infographic syntax to render the key research takeaways or statistics.
   • Use playwright-mcp or a local headless browser script to take a high-quality screenshot (.png) of the rendered AntV infographic. Save it to assets/.
   • You may still use mermaid.ink for simple flowcharts, but all major data visualizations should leverage the AntV infographic capabilities for professional aesthetics.
2. Document Compilation (docx or python-docx): You MUST NOT just output Markdown as the final product. Write a Python script using python-docx to programmatically build the final Word document. Important Windows Environment Note: When executing the Python script via Bash, use absolute paths with forward slashes and enclose them in quotes (e.g., python "C:/path/to/generate_docx.py") to prevent MSYS bash from stripping backslashes and causing [Errno 2] No such file or directory.
3. Embed Assets: Insert the generated AntV Infographic images and any Mermaid charts into the .docx file at the appropriate logical sections. Ensure formatting aligns with APA 7th standards (e.g., proper figure captions).
4. Final Delivery: Save the generated .docx file directly to the D:\ drive (e.g., D:\Research_Report.docx). Deliver this absolute D:\ path to the user.

Phase 7: Knowledge Base & NotebookLM Integration

1. Obsidian Wiki Update: Synthesize the core findings, literature insights, and strategic takeaways. Write or append this synthesis directly into the user's Obsidian Vault (C:\Users\User\Documents\Obsidian Vault\) to maintain an ongoing, centralized knowledge base.
2. NotebookLM Source Ingestion: Utilize the notebooklm MCP tools to create a dedicated notebook for this research project. You MUST explicitly upload every single cited reference as an individual source into NotebookLM (do not just upload one compiled document). Upload the raw abstracts or full-texts for each cited paper so NotebookLM can accurately cross-reference and map individual citations.

📚 Linked References

• references/python-docx-manipulation.md: Patterns for reading, creating, and safely removing XML paragraphs from .docx files.
• references/academic-api-patterns.md: Reliable curl and Python patterns for hitting Crossref and OpenAlex APIs, including critical URL-encoding fixes.
• scripts/verify_urls.py: Python script utilizing ddgs and requests to programmatically search and verify evidence URLs for Phase 4.5.
• scripts/node/fetch_openalex_papers.js: Node.js script for safely fetching from OpenAlex and strictly filtering out MDPI/Q4.
• scripts/node/fetch_unpaywall_oa.js: Resolves DOI to Open Access PDF URLs via Unpaywall.
• scripts/node/scrape_html_fulltext.js: Scrapes HTML full texts to bypass basic PDF blocks.
• scripts/node/extract_pdf_text.js: PDF parsing script template using pdf-parse.
• scripts/node/generate_docx.js: Programmatically generates APA 7th compliant DOCX using the docx library.
• templates/fetch_openalex_background.js: Node.js template for background fetching from the OpenAlex API, including the logic to decode abstract_inverted_index and respect rate limits.
• scripts/fetch_oa_fulltexts.py: Python script utilizing the Unpaywall API and PyMuPDF to automatically locate, download, and extract text from Open Access PDFs for a given list of DOIs. Crucial for Phase 0.5.
• templates/fetch_openalex_background.js: Node.js template for background fetching from the OpenAlex API, including the logic to decode abstract_inverted_index and respect rate limits.

⚠️ Pitfalls & Strict Rules

• Zotero Credentials & Setup: Phase 4.6 relies on the pyzotero client. The pipeline will crash or fail to archive if the required environment variables (ZOTERO_API_KEY, ZOTERO_LIBRARY_ID, ZOTERO_LIBRARY_TYPE) are missing. Always verify these are present in the environment or a .env file before executing the pipeline. CRITICAL: ZOTERO_LIBRARY_ID MUST be the integer ID (e.g., 16500033), NOT the username. If the user provides a string username, resolve it programmatically via GET https://api.zotero.org/keys/<API_KEY> (r.json()['userID']).
• Unpaywall API Email Requirement: When querying the Unpaywall API to find Open Access PDFs, you MUST append a valid email parameter (?email=...) or the request will fail with HTTP 422.
• STRICT COMPLIANCE REQUIRED: You MUST strictly follow EVERY step of this pipeline in order. Skipping phases, ignoring instructions, or taking shortcuts (such as writing drafts without full-text verification) is strictly forbidden. There are no exceptions.
• English Output ONLY: All generated drafts, reports, and final documents MUST be written strictly in English, regardless of the conversational language.
• FULL-TEXT READING IS MANDATORY (NO ABSTRACT-ONLY SHORTCUTS): You MUST NOT rely solely on abstracts. You MUST download, scrape, or extract the entire full text of the papers before synthesizing or extracting mechanisms. If full text is inaccessible, you MUST drop the paper or ask the user for help. DO NOT proceed to drafting based only on OpenAlex/Scopus abstracts.
• No Hallucinated Citations: Every claim MUST map to a real URL/DOI found during Phase 1/2.
• Strict Literalism vs. Concept Stitching (Scale & Region Mismatch): NEVER map physiological/mechanistic findings from one geographical study to an unrelated region, OR from a macro-biome scale to a specific city. If a study covers "Global Temperate Drylands", do NOT explicitly attribute its findings to the urban environment of "Paris" (Geographical Over-Extrapolation). If you lack city-specific data, explicitly run an additional targeted search instead of concept-stitching macro data to fit the prompt. Read and report the literal findings.
• Cherry-Picking and Data Reversal (PFT Conflation): Do not ignore the primary conclusion of a paper to isolate a minor variable that fits a preconceived narrative. Pay extreme attention to contrasting functional types within a paper: if a study finds that shallow-rooted suffer severe drought stress while deep-rooted remain resilient, do not conflate the two and falsely claim the trees are suffering (Plant Functional Type Conflation). Read the nuances of the data.

Lessons from the 2026-08 Fukuoka run (field-tested, not theoretical)

A full four-subagent literature sweep was executed end to end in August 2026. These are the failures that actually occurred and the fixes that actually worked.

Elsevier full text: use the Scopus key, not a browser

The registered SCOPUS_API_KEY also authorises Elsevier's Article Retrieval API. This retrieves full text for every Elsevier journal — Remote Sensing of Environment, Urban Climate, Urban Forestry & Urban Greening, Applied Geography, Cities, Solar Energy — as clean text, with no browser, no proxy, and no PDF extraction step.

import requests, json
KEY = json.load(open(r"C:/Users/User/.claude.json", encoding="utf-8"))["mcpServers"]["scopus"]["env"]["SCOPUS_API_KEY"]
r = requests.get(f"https://api.elsevier.com/content/article/doi/{doi}?httpAccept=text/plain",
                 headers={"X-ELS-APIKey": KEY, "Accept": "text/plain"}, timeout=60)

pii/{PII} also works (take the PII from Crossref's alternative-id); httpAccept=text/xml preserves section structure. Sleep ~1 s between calls. Six paywalled papers were retrieved this way in one pass after every crawler route had failed. Try this before reaching for cloakbrowser on any 10.1016/ DOI.

Publisher routing that works:

Publisher	Route
Elsevier (10.1016/)	Article Retrieval API with the Scopus key, as above
MDPI (10.3390/)	www.mdpi.com/.../pdf returns Access Denied, but the asset CDN https://res.mdpi.com/<journal>/<journal>-<vol>-<page>/article_deploy/<...>.pdf serves it
Copernicus (BG, ACP, ESSD)	Fully open: https://bg.copernicus.org/articles/<vol>/<page>/<year>/bg-<vol>-<page>-<year>.pdf
Nature / Sci Rep	https://www.nature.com/articles/<id>.pdf when OA
PMC	Public PMC sits behind reCAPTCHA; the Europe PMC mirror https://europepmc.org/articles/PMC<id>?pdf=render does not
Wiley / AGU	No reliable automated route found; needs institutional access

Note that OpenAlex oa_status is not a reliable predictor here — papers reading closed in OpenAlex retrieved fine through the Elsevier API.

Verify downloaded content, not just the file format

A guessed NTRS identifier returned a perfectly valid 1.2 MB PDF that was a NASA launch-abort-system paper, not the requested Landsat calibration study. Checking for a %PDF magic number and a plausible size is not sufficient. Read page 1 and match expected title keywords before trusting any download:

d = fitz.open(path)
first = d[0].get_text("text").lower()
assert "expected keyword" in first and "second keyword" in first

Never propagate a subagent's number without checking the primary source

Subagent reports are leads, not evidence. Two concrete errors from this run:

• A Scopus subagent reported that a manuscript value "sits almost exactly on the national mean". The source paper stated plainly "biomass and ANPP are all dry-matter based" and defined ANPP as aboveground only. The comparison needed a dry-matter-to-carbon conversion (x0.45-0.50) and a belowground addition (+20-40%). Both were missing; the conclusion inverted once corrected.
• A subagent could not attribute two numbers to the right study site from the indexed abstract and correctly said so. Once the full text was retrieved, the direction of the finding was the opposite of what the abstract fragment implied.

Rule: before a subagent-reported number enters a synthesis or a manuscript, grep it out of the primary full text and read the sentence around it. Reserve this for load-bearing numbers; you cannot re-verify everything, so say plainly which numbers were re-verified and which were taken on report.

Background scripts must checkpoint per query, never write at the end

Two of four subagents initially reported "nothing usable" and both were wrong:

• One launched its sweep as python ... | tail -70, so all stdout buffered inside tail and nothing appeared until exit; the script also wrote its JSON only after all 53 queries finished, so an interrupted run yielded nothing. It had in fact completed, with 9,088 records.
• One hit anonymous-pool 429s on Semantic Scholar and stopped before its citation-graph track.

Fix in the script, not in the retry: redirect to a log file instead of piping to tail, and checkpoint the results JSON after every query so a killed run still yields partial data. Cache responses so a re-run resumes rather than repeats.

Citation-graph traversal finds what keyword search cannot

Semantic Scholar's /paper/DOI:{doi}/citations revealed the structural reason a literature gap existed: the 53 papers citing the main site-context paper contained zero carbon or NPP work — the urban-greenspace literature and the carbon-cycle literature were disconnected communities. Keyword search on the site name could never have surfaced the numbers, because they live in a citation neighbourhood that never touches it. A single bridging paper connected the two.

Use citation-graph traversal when keyword search returns thematically narrow results. It is a ~20-call job with a key. Note that SEMANTIC_SCHOLAR_API_KEY is currently unset, and the backup file stores only a masked placeholder (s2k-V9...Zw6N, literal ellipsis) — the anonymous pool 429s heavily and a real key turns a 25-minute grind into about a minute.

Reading PDFs

Prefer the markitdown skill over raw PyMuPDF text extraction: it renders tables as Markdown pipe tables, which matters for papers whose load-bearing numbers live in a regional or parameter table. Fall back to PyMuPDF only when markitdown errors on a specific file.