academic-pipeline

# Academic Pipeline v3.7.0 — Full Academic Research Workflow Orchestrator A lightweight orchestrator that manages the complete academic pipeline from research exploration to final manuscript. It does not perform substantive work — it only detects stages, recommends modes, dispatches skills, manages transitions, and tracks state. **v3.6.3 (opt-in):** Set `ARS_PASSPORT_RESET=1` to promote FULL checkpoints to context-reset boundaries. Use `resume_from_passport=<hash>` in a fresh session to continue from the recorded stage. See [`references/passport_as_reset_boundary.md`](references/passport_as_reset_boundary.md). **v2.0 Core Improvements**: 1. **Mandatory user confirmation checkpoints** — Each stage completion requires user confirmation before proceeding to the next step 2. **Academic integrity verification** — After paper completion and before review submission, 100% reference and data verification must pass 3. **Two-stage review** — First full review + post-revision focused verification review 4. **Final integrity check** — After revision completion, re-verify all citations and data are 100% correct 5. **Reproducible** — Standardized workflow producing consistent quality assurance each time 6. **Process documentation** — After pipeline completion, automatically generates a "Paper Creation Process Record" PDF documenting the human-AI collaboration history ## Quick Start **Full workflow (from scratch):** ``` I want to write a research paper on the impact of AI on higher education quality assurance ``` --> academic-pipeline launches, starting from Stage 1 (RESEARCH) **Mid-entry (existing paper):** ``` I already have a paper, help me review it ``` --> academic-pipeline detects mid-entry, starting from Stage 2.5 (INTEGRITY) **Revision mode (received reviewer feedback):** ``` I received reviewer comments, help me revise ``` --> academic-pipeline detects, starting from Stage 4 (REVISE) **Resume from passport (cross-session context reset, opt-in):** ``` resume_from_passport=<hash> [stage=<n>] [mode=<m>] ``` --> Loads the Material Passport (Schema 9), locates the `kind: boundary` entry matching `<hash>`, and confirms it has no later `kind: resume` entry consuming it. If `pending_decision` is set, the decision prompt fires first to capture the user's branch choice for the audit ledger; the prompt is never skipped, even when the user supplies `stage=`. After the prompt (or immediately if no `pending_decision`), the next stage is determined by: (a) `stage=<n>` CLI override if provided, else (b) the matched option's `next_stage`, else (c) the `next` field recorded in the boundary entry. CLI `stage=`/`mode=` overrides win over option routing. - **Gate (emit)**: `ARS_PASSPORT_RESET=1` must be set in the emitting session. Without the flag, no `kind: boundary` entries are written and there is nothing to resume from. - **Gate (resume)**: No flag required. Any session can invoke `resume_from_passport=<hash>` against a passport that carries a valid boundary entry matching the hash. - **Intent**: Invoke in a *fresh* Claude Code session. Resuming within the same session that emitted the boundary provides no token savings and may drop still-live in-session context. - **Stage**: Any. Resumes at whatever stage the routing rules above determine. - **Reference**: [`references/passport_as_reset_boundary.md`](references/passport_as_reset_boundary.md) — see §"`resume_from_passport` mode contract". **Execution flow:** 1. Detect the user's current stage and available materials 2. Recommend the optimal mode for each stage 3. Dispatch the corresponding skill for each stage 4. **After each stage completion, proactively prompt and wait for user confirmation** 5. Track progress throughout; Pipeline Status Dashboard available at any time --- ## Trigger Conditions ### Trigger Keywords **English**: academic pipeline, research to paper, full paper workflow, paper pipeline, end-to-end paper, research-to-publication, complete paper workflow ### Non-Trigger Scenarios | Scenario | Skill to Use | |----------|-------------| | Only need to search materials or do a literature review | `deep-research` | | Only need to write a paper (no research phase needed) | `academic-paper` | | Only need to review a paper | `academic-paper-reviewer` | | Only need to check citation format | `academic-paper` (citation-check mode) | | Only need to convert paper format | `academic-paper` (format-convert mode) | ### Trigger Exclusions - If the user only needs a single function (just search materials, just check citations), no pipeline is needed — directly trigger the corresponding skill - If the user is already using a specific mode of a skill, respect that entry point; the pipeline is opt-in - The pipeline is optional, not mandatory --- ## Pipeline Stages (10 Stages) | Stage | Name | Skill / Agent Called | Available Modes | Deliverables | |-------|------|---------------------|----------------|-------------| | 1 | RESEARCH | `deep-research` | socratic, full, quick | RQ Brief, Methodology, Bibliography, Synthesis | | 2 | WRITE | `academic-paper` | plan, full | Paper Draft | | **2.5** | **INTEGRITY** | **`integrity_verification_agent`** | **pre-review** | **Integrity verification report + corrected paper** | | 3 | REVIEW | `academic-paper-reviewer` | full (incl. Devil's Advocate) | 5 review reports + Editorial Decision + Revision Roadmap | | 4 | REVISE | `academic-paper` | revision | Revised Draft, Response to Reviewers | | **3'** | **RE-REVIEW** | **`academic-paper-reviewer`** | **re-review** | **Verification review report: revision response checklist + residual issues** | | **4'** | **RE-REVISE** | **`academic-paper`** | **revision** | **Second revised draft (if needed)** | | **4.5** | **FINAL INTEGRITY** | **`integrity_verification_agent`** | **final-check** | **Final verification report (must achieve 100% pass to proceed)** | | 5 | FINALIZE | `academic-paper` | format-convert | Final Paper (default MD; DOCX via Pandoc when available, otherwise conversion instructions; ask about LaTeX; confirm correctness; PDF) | | **6** | **PROCESS SUMMARY** | **orchestrator** | **auto** | **Paper creation process record MD + LaTeX to PDF (bilingual)** | **Parallelization opportunity (v3.3)**: Within Stage 2, the `academic-paper` skill's Phase 1 (literature_strategist_agent) and the `visualization_agent` can operate in parallel after Phase 2 (structure_architect_agent) completes the outline. Specifically: - Once the outline includes a visualization plan, `visualization_agent` can begin figure generation - Simultaneously, `argument_builder_agent` can build CER chains - `draft_writer_agent` waits for both to complete before beginning Phase 4 This mirrors PaperOrchestra's parallel execution of Plot Generation (Step 2) and Literature Review (Step 3) after Outline (Step 1), which reduces overall pipeline latency. The parallelization is optional — sequential execution remains the default for simplicity. --- ## Pipeline State Machine 1. **Stage 1 RESEARCH** -> user confirmation -> Stage 2 2. **Stage 2 WRITE** -> user confirmation -> Stage 2.5 3. **Stage 2.5 INTEGRITY** -> PASS -> Stage 3 (FAIL -> fix and re-verify, max 3 rounds) 4. **Stage 3 REVIEW** -> Accept -> Stage 4.5 / Minor|Major -> Stage 4 / Reject -> Stage 2 or end 5. **Stage 4 REVISE** -> user confirmation -> Stage 3' 6. **Stage 3' RE-REVIEW** -> Accept|Minor -> Stage 4.5 / Major -> Stage 4' 7. **Stage 4' RE-REVISE** -> user confirmation -> Stage 4.5 (no return to review) 8. **Stage 4.5 FINAL INTEGRITY** -> PASS (zero issues) -> Stage 5 (FAIL -> fix and re-verify) 9. **Stage 5 FINALIZE** -> MD -> DOCX via Pandoc when available (otherwise instructions) -> ask about LaTeX -> confirm -> PDF -> Stage 6 10. **Stage 6 PROCESS SUMMARY** -> ask language version -> generate process record MD -> LaTeX -> PDF -> end See `references/pipeline_state_machine.md` for complete state transition definitions. --- ## Adaptive Checkpoint System ⚠️ **IRON RULE — Core rule: After each stage completion, the system must proactively prompt the user and wait for confirmation. The checkpoint presentation adapts based on context and user engagement.** ### Checkpoint Types | Type | When Used | Content | |------|-----------|---------| | FULL | First checkpoint; after integrity boundaries; before finalization | Full deliverables list + decision dashboard + all options | | SLIM | After 2+ consecutive "continue" responses on non-critical stages | One-line status + explicit continue/pause prompt | | MANDATORY | Integrity FAIL; Review decision; Stage 5 | Cannot be skipped; requires explicit user input | ### Decision Dashboard (shown at FULL checkpoints) ``` ━━━ Stage [X] [Name] Complete ━━━ Metrics: - Word count: [N] (target: [T] +/-10%) [OK/OVER/UNDER] - References: [N] (min: [M]) [OK/LOW] - Coverage: [N]/[T] sections drafted [COMPLETE/PARTIAL] - Quality indicators: [score if available] Deliverables: - [Material 1] - [Material 2] Flagged: [any issues detected, or "None"] Ready to proceed to Stage [Y]? You can also: 1. View progress (say "status") 2. Adjust settings 3. Pause pipeline ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` ### Adaptive Rules 1. **First checkpoint**: always FULL 2. **After 2+ consecutive "continue" without review**: prompt user awareness ("You've continued [N] times in a row. Want to review progress?") 3. **Integrity boundaries (Stage 2.5, 4.5)**: always MANDATORY 4. **Review decisions (Stage 3, 3')**: always MANDATORY 5. **Before finalization (Stage 5)**: always MANDATORY 6. **All other stages**: start FULL, downgrade to SLIM if user says "just continue" ### Checkpoint Rules 1. ⚠️ **IRON RULE**: **Cannot auto-skip MANDATORY checkpoints**: Even if the previous stage result is perfect, explicit user input is required at MANDATORY checkpoints 2. **User can adjust**: At FULL and MANDATORY checkpoints, users can modify the mode or settings for the next step 3. **Pause-friendly**: Users can pause at any checkpoint and resume later 4. **SLIM mode**: If the user says "just continue" or "fully automatic," subsequent non-critical checkpoints switch to SLIM format (one-line status + explicit continue/pause prompt) 5. **Awareness guard**: After 4+ consecutive continue responses, the system inserts a FULL checkpoint regardless of stage type to ensure user remains engaged ### Self-Check Questions (at every FULL checkpoint) Before presenting the checkpoint to the user, the orchestrator asks itself: 1. **Citation integrity**: Are there any unverified citations in the latest output? 2. **Sycophantic concession**: Did the latest stage uncritically accept all feedback without pushback? 3. **Quality trajectory**: Is the latest output ≥ the quality of the previous stage? If declining, PAUSE and flag. 4. **Scope discipline**: Did the latest stage add content not requested by the user or the revision roadmap? 5. **Completeness**: Are all required deliverables for this stage present? If ANY answer raises concern, include it in the checkpoint presentation to the user. --- ## Agent Team (4 Agents) | # | Agent | Role | File | |---|-------|------|------| | 1 | `pipeline_orchestrator_agent` | Main orchestrator: detects stage, recommends mode, triggers skill, manages transitions | `agents/pipeline_orchestrator_agent.md` | | 2 | `state_tracker_agent` | State tracker: records completed stages, produced materials, revision loop count | `agents/state_tracker_agent.md` | | 3 | `integrity_verification_agent` | Integrity verifier: 100% reference/citation/data verification (blocking) | `agents/integrity_verification_agent.md` | | 4 | `collaboration_depth_agent` | **Observer (advisory only — never blocks).** Reads dialogue log and scores user-AI collaboration pattern against `shared/collaboration_depth_rubric.md`. Invoked at FULL/SLIM checkpoints and at pipeline completion. Based on Wang & Zhang (2026). | `agents/collaboration_depth_agent.md` | --- ## Orchestrator Workflow ### Step 1: INTAKE & DETECTION ``` pipeline_orchestrator_agent analyzes the user's input: 1. What materials does the user have? - No materials --> Stage 1 (RESEARCH) - Has research data --> Stage 2 (WRITE) - Has paper draft --> Stage 2.5 (INTEGRITY) - Has verified paper --> Stage 3 (REVIEW) - Has review comments --> Stage 4 (REVISE) - Has revised draft --> Stage 3' (RE-REVIEW) - Has final draft for formatting --> Stage 5 (FINALIZE) 2. What is the user's goal? - Full workflow (research to publication) - Partial workflow (only certain stages needed) 3. Determine entry point, confirm with user ``` ### Step 2: MODE RECOMMENDATION ``` Based on entry point and user preferences, recommend modes for each stage: User type determination: - Novice / wants guidance --> socratic (Stage 1) + plan (Stage 2) + guided (Stage 3) - Experienced / wants direct output --> full (Stage 1) + full (Stage 2) + full (Stage 3) - Time-limited --> quick (Stage 1) + full (Stage 2) + quick (Stage 3) Explain the differences between modes when recommending, letting the user choose ``` ### Step 3: STAGE EXECUTION ``` Call the corresponding skill (does not do work itself, purely dispatching): 1. Inform the user which Stage is about to begin 2. Load the corresponding skill's SKILL.md 3. Launch the skill with the recommended mode 4. Monitor stage completion status After completion: 1. Compile deliverables list 2. Update pipeline state (call state_tracker_agent) 3. [MANDATORY] Proactively prompt checkpoint, wait for user confirmation ``` ### Step 4: TRANSITION ``` After user confirmation: 1. Pass the previous stage's deliverables as input to the next stage 2. Trigger handoff protocol (defined in each skill's SKILL.md): - Stage 1 --> 2: deep-research handoff (RQ Brief + Bibliography + Synthesis) - Stage 2 --> 2.5: Pass complete paper to integrity_verification_agent - Stage 2.5 --> 3: Pass verified paper to reviewer - Stage 3 --> 4: Pass Revision Roadmap to academic-paper revision mode - Stage 4 --> 3': Pass revised draft and Response to Reviewers to reviewer - Stage 3' --> 4': Pass new Revision Roadmap + R&R Traceability Matrix (Schema 11) to academic-paper revision mode - Stage 4/4' --> 4.5: Pass revision-completed paper to integrity_verification_agent (final verification) - Stage 4.5 --> 5: Pass verified final draft to format-convert mode 3. Begin next stage ``` ### Mid-Conversation Reinforcement Protocol At every stage transition, the orchestrator MUST inject a brief core principles reminder. This prevents context rot in long conversations. **Template** (adapt to the upcoming stage): ````