# review-gen
`review-gen` is a workflow toolkit for long-horizon literature reviews in management, strategy, entrepreneurship, innovation, and organization studies.
It is designed for one practical goal: **turn a large paper library into a controlled, auditable review pipeline** instead of ad-hoc prompting.
## What It Does
1. Search high-quality journals through ABS/AJG-aware OpenAlex queries.
2. Build and maintain a reusable corpus workspace.
3. Prioritize and download full text incrementally from a manifest.
4. Convert PDF to Markdown (MinerU) and chunk text for retrieval.
5. Build a frozen review plan before drafting.
6. Draft review prose under plan constraints.
7. Prevent citation hallucination with allowlist + DOI audit.
## System Architecture
The system is split into four skills plus two embedded backends.
### Skills
- `openalex-ajg-insights`
Search, merge corpus, prepare full-text manifest, download PDFs incrementally, convert PDFs to Markdown, and retrieve chunks.
- `management-review-planner`
Build `review_plan.md`, archive plan revisions, support EN/ZH planning, and use dynamic core-paper selection.
- `management-review-writer`
Build `review_packet.md` and `review_guardrails.md`, generate `citation_allowlist.jsonl`, and support draft citation auditing.
- `review-orchestrator`
Enforce workflow gates: plan approval before drafting, citation audit before final delivery.
### Embedded Backends
- `backend/openalex-ajg-mcp`
- `backend/paper-download-mcp`
## Workspace Contract (Single Source of Truth)
Each project uses one workspace (`<review-workspace>`) with stable folders:
```text
01_search/ # raw search outputs
02_corpus/ # merged corpus (master_corpus.jsonl)
03_screening/ # screening and evidence tables
04_fulltext/ # manifest + PDF inbox/archive
05_mineru/ # MinerU raw/extracted outputs
06_chunks/ # chunk index for retrieval
07_plan/ # live plan + history snapshots
08_outputs/ # packets, drafts, citation allowlist, audit report
```
Key control files:
- `02_corpus/master_corpus.jsonl`
- `03_screening/screening_table.csv`
- `04_fulltext/fulltext_manifest.csv`
- `07_plan/review_plan.md`
- `08_outputs/review_packet.md`
- `08_outputs/citation_allowlist.jsonl`
- `08_outputs/citation_audit_report.md`
## End-to-End Workflow
Use placeholders:
- `<workflow-python>`
- `<review-gen-home>`
- `<review-workspace>`
### 1) Initialize workspace
```bash
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
init-workspace \
--topic "Your topic"
```
### 2) Start the ModelScope-style visual workspace
This entry keeps the ModelScope-style SaaS workspace UI, but only exposes projects, search, corpus screening, and export. It does not include registration/login, credits, framework pages, or writing pages.
```bash
cd <review-gen-home>
PORT=7860 uvicorn app:app --host 0.0.0.0 --port 7860
```
Open:
```text
http://127.0.0.1:7860/
```
The browser workflow can:
1. create a project and enter a review question
2. copy the Step 1 keyword-strategy prompt, calibrate English keywords, synonyms, exclusions, and `query_rounds` in GPT/Claude, then paste the returned JSON back into the workspace
3. tune AJG field, journal rank, year start, target range, result limit, and citation expansion
4. run precision search; pasted strategies are executed through `query_rounds` first, while blank strategies fall back to local English keyword hints. The search includes AJG filtering, protected seed queries, citation expansion, and OpenAlex 429 backoff retries
5. screen papers across UTD24, FT50, AJG, and other filters; include/exclude actions update the current rows without moving away from Step 2
6. click the top-right next button after screening to review included papers in Step 3, then export search conditions, included papers, the full corpus, and screening state in Step 4; the Codex/Claude handoff prompt is shown in the frontend and can be copied with one click
The external AI should only return keyword strategy JSON. It should not return DOIs, specific papers, authors, years, citations, or references; those are verified through OpenAlex results.
The handoff bundle includes:
- `search_conditions.json`
- `included_papers.csv`
- `included_papers.jsonl`
- `papers.csv`
- `corpus.jsonl`
- `screening.json`
- `project.json`
To hand off to Codex or Claude Code, copy the prompt shown in Step 4 and point it at the exported package files.
You can also skip the browser and continue with the command-line search flow below.
### 3) Search ABS/AJG journals from the command line
```bash
python <review-gen-home>/skills/openalex-ajg-insights/scripts/openalex_ajg_bridge.py \
search-abs \
--query "Your query" \
--field "INFO MAN" \
--min-rank "4" \
--year-start 2018 \
--limit 50
```
### 4) Merge corpus, export a handoff package, and prepare full-text manifest
```bash
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
--workspace <review-workspace> \
merge-search-results
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
--workspace <review-workspace> \
export-literature-package
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
--workspace <review-workspace> \
prepare-fulltext-manifest --min-priority medium
```
### 5) Download priority papers incrementally
```bash
python <review-gen-home>/skills/openalex-ajg-insights/scripts/download_manifest_papers.py \
--workspace <review-workspace> \
--min-priority high \
--max-papers 10
```
#### Auto-download scope and limits
The auto-downloader only targets papers that already exist in `04_fulltext/fulltext_manifest.csv`.
A paper is eligible only when all of these are true:
1. `full_text_priority` is greater than or equal to `--min-priority`
2. `need_full_text` is marked yes/true
3. DOI is present
Default behavior is incremental:
1. If a local PDF already exists and `pdf_status=ready`, it is skipped
2. `--paper-keys` can limit to specific records
3. `--max-papers` caps one batch size
4. `--all-candidates` overrides skip and retries all candidates
Typical out-of-scope cases for auto-download:
1. records without DOI
2. resolver failures behind publisher paywalls
3. local-language databases outside the current resolver chain
4. books/reports/theses and manually collected classics
#### Manual PDF intake
When auto-download cannot cover a paper, add PDFs manually:
1. put files into `04_fulltext/pdf_inbox/`
2. preferred file name: `YEAR__FirstAuthor__ShortTitle.pdf`
3. run MinerU conversion normally; converted files are archived to `04_fulltext/pdf_archive/`
### 6) Convert PDFs and chunk Markdown
```bash
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
--workspace <review-workspace> \
convert-pdfs-with-mineru \
--env-path <review-workspace>/04_fulltext/mineru.env
python <review-gen-home>/skills/openalex-ajg-insights/scripts/review_workflow.py \
--workspace <review-workspace> \
chunk-markdown
```
### 7) Build plan (dynamic high-quality selection)
```bash
python <review-gen-home>/skills/management-review-planner/scripts/build_review_plan.py \
--workspace <review-workspace> \
--topic "Your topic" \
--word-count 2500 \
--language en \
--top-papers-mode dynamic \
--top-papers 0
```
Notes:
- `dynamic` mode prioritizes screened-in (`included`) papers.
- It does not backfill low-value non-included papers unless `--allow-fallback` is set.
- Chinese planning supports CNKI RIS from `02_corpus/cnki_ris/`.
Chinese literature source and location:
1. export RIS files from CNKI (or other Chinese databases that can export RIS-like metadata)
2. place files in `02_corpus/cnki_ris/`
3. planner loads these records into planning context, and writer can include them in citation allowlist generation
### 8) Build writing packet and citation allowlist
```bash
python <review-gen-home>/skills/management-review-writer/scripts/build_review_packet.py \
--workspace <review-workspace> \
--topic "Your topic" \
--top-papers-mode dynamic \
--top-papers 0 \
--output-path <review-workspace>/08_outputs/review_packet.md
```
### 9) Validate citations before final delivery
```bash
python <review-gen-home>/skills/management-review-writer/scripts/validate_draft_citations.py \
--workspace <review-workspace> \
--draft-path <review-workspace>/08_outputs/review_draft.md
```
Audit checks:
- DOI in draft must exist in `citation_allowlist.jsonl`.
- DOI can be resolver-checked via Crossref/OpenAlex.
- Optional strict mode checks author-year mapping.
## Quality Gates (Non-Negotiable)
1. No prose drafting before plan approval.
2. No final delivery before citation audit passes.
3. No citation outside allowlist unless explicitly verified and added.
## Platform and Setup
Install the base dependencies:
```bash
pip install -r requirements.txt
```
### MinerU API for PDF-to-Markdown
In this project, the MinerU integration is mainly used to convert collected PDF papers into Markdown so the downstream chunking, retrieval, planning, and drafting steps work on clean text instead of raw PDFs.
How to get the API:
1. Register or sign in on the [MinerU official site](https://mineru.net/).
2. Apply for an API token from the MinerU web console after login. The official API docs describe this token as a token "obtained from the official website", but do not publish a stable deep link to the token page, so the safest entry is the main site after login.
3. Read the official API docs before use:
- [MinerU API docs (EN)](https://mineru.net/doc/docs/index_en/)
- [MinerU API docs (ZH)](https://mineru.net/doc/docs/)
- [Rate limits / quota notes](https://mineru.net/doc/docs/limit/)
4. Put the token into `04_fulltext/mineru.env` as:
```env
MINERU_API_KEY=your-token-from-mineru
```
Notes:
- The official docs state that API calls require an `Authorization: Bearer <token>` header.
- The API is currently in beta, and the published limits include up to 200 MB and 600 pages per file, with 2000 pages per day at the highest priority tier per account.
The toolkit is path-agnostic and works in PowerShell, macOS Terminal, and Linux shells.
## Repository Layout
```text
review-gen/
├── backend/
│ ├── openalex-ajg-mcp/
│ └── paper-download-mcp/
├── README.md
├── README_zh.md
├── LICENSE
├── requirements.txt
└── skills/
├── openalex-ajg-insights/
├── management-review-planner/
├── management-review-writer/
└── review-orchestrator/
```
## License
MIT. See [LICENSE](LICENSE).