paper-workflow-orchestrator

name	paper-workflow-orchestrator
description	Use when you want an end-to-end workflow across preprocess, analysis, and reporting with clear stage gates and rerun strategy. Portable — no hardcoded paths. The user only needs to provide WoS txt files. Keywords: end-to-end, orchestrate, full pipeline, one-click workflow, 全流程, 一键运行, 编排, 分阶段执行.

Paper Workflow Orchestrator Skill

Purpose

Coordinate end-to-end execution from raw WoS data through final report, while keeping the processing and analysis skills decoupled. Fully portable — works on any machine.

User Contract

The user provides bibliometric data in one of the supported formats. Everything else — processing, analysis, visualization, report generation — is orchestrated by this skill.

Pre-flight Interactive Inquiry

Before executing ANY stage, the agent MUST ask the user to confirm:

1. 工作区根目录: “您的项目根目录在哪里？（包含 src/, data/, .github/ 的文件夹）”
   • 默认使用当前工作区
2. 数据位置与格式: “您的数据文件在哪个目录？格式是什么？”
   • WoS .txt → 放入 data/raw/，从 Stage A（解析转换）开始
   • CSV → 检查是否已含 UT/国家/作者列，智能跳过已完成的阶段
   • Excel (.xlsx) → 读取后等同 CSV 处理
   • 具体规则见下方“Data Format Routing”
3. 模型配置（仅当流程包含主题分析时）:
   • "是否已有训练好的 BERTopic 模型？如有，请提供路径和格式 (safetensors/pkl/pytorch)"
     • 有模型 → 设置 BERTOPIC_MODEL_PATH=<路径>，直接加载 + transform，跳过训练
     • 无模型 → 从头训练，结果保存到 models/bertopic_model/ (safetensors 格式)
   • "Embedding 模型用什么？"
     • EMBEDDING_MODEL=auto（默认）→ 自动检测数据语言，选择最佳模型：
       • 英文为主 → all-MiniLM-L6-v2
       • 中文为主 → BAAI/bge-base-zh-v1.5
       • 中英混合 → sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
     • 也可手动指定 HuggingFace 模型名或本地路径
     • 注意：更换 embedding 模型后需删除旧的 doc_embeddings.npy 缓存
   • "您本地的 LLM 后端是什么？Ollama 还是 MLX？模型名称是什么？"（用于主题命名）
     • Ollama: 设置 LLM_BACKEND=ollama, LLM_MODEL=<你的本地模型名>（例如 qwen2.5:14b）
     • MLX: 设置 LLM_BACKEND=openai, LLM_URL=http://localhost:8080/v1/chat/completions, LLM_MODEL=<模型名>
     • 其他 OpenAI 兼容接口: LLM_BACKEND=openai, LLM_URL=<接口地址>, LLM_API_KEY=<key>

确认后，设置环境变量 PROJECT_ROOT 指向用户确认的根目录，所有脚本会自动使用该路径。

4. 研究课题名称: "您的研究课题简称是什么？（用于报告标题、图表标注、输出文件名）"
   • 默认值为空，必须由用户确认
   • 示例："ClimateAI"、"AI4S"、"Quantum"、"LLM" 等
   • 设置环境变量 RESEARCH_TOPIC=<课题简称>
   • 所有图表标题、数据源标注、报告文件名均使用该变量
   • Do not continue to final report export if RESEARCH_TOPIC is missing or ambiguous

Data Format Routing

根据用户提供的数据格式，自动决定从哪个阶段开始：

数据格式	入口阶段	操作
WoS .txt	Stage A	完整流程: 解析 → 合并 → 清洗 → 分析
CSV/Excel 无 UT 列	Stage A	视为非 WoS 数据，跳过 txt 解析，直接尝试字段映射后进入 Stage B
CSV/Excel 含 UT + 基础列 (TI, AB, PY等)	Stage B	复制到 data/processed/wos_merged.csv，跳过 Stage A
CSV/Excel 含 UT + 国家/机构列	Stage C	复制到 data/processed/wos_cleaned.csv，跳过 A+B
CSV/Excel 含 UT + 国家 + 作者列	Stage D	直接进行质量门检查，跳过 A+B+C

格式检测逻辑（agent 在 Pre-flight 阶段执行）：

import pandas as pd
df = pd.read_csv("user_file.csv", nrows=5)  # 或 pd.read_excel
cols = set(df.columns)
has_ut = "UT" in cols
has_base = {"TI", "AB", "PY"}.issubset(cols)
has_country = any(c in cols for c in ["country_c1", "Country", "C1"])
has_author = any(c in cols for c in ["author_id", "AU", "paper_authors"])
print(f"UT={has_ut}, base={has_base}, country={has_country}, author={has_author}")
# 根据结果决定入口阶段

Portable Directory Convention

Same as paper-data-processing. All paths are workspace-relative.

{workspace}/
├── data/raw/              # ← User input: WoS txt files
├── data/processed/        # Processing + analysis outputs
├── reports/               # Final deliverables
│   ├── figures/
│   ├── tables/
│   └── {RESEARCH_TOPIC}_bibliometric_report.md
├── models/
├── src/
└── .github/skills/

Delegation Rule

1. Use paper-data-processing for raw data → processed artifacts.
2. Use paper-processing-analysis-handoff as the mandatory stage gate between processing and analysis.
3. Use paper-analysis-report for analysis → report artifacts, only after handoff returns analysis-ready.

Portability Enforcement

Before running ANY script at ANY stage:

1. The agent reads the script and checks for hardcoded absolute paths.
2. If found, patches them to workspace-relative paths.
3. This is a non-negotiable step at every stage.

Stages

Stage 1: Data Processing

• Delegate to paper-data-processing skill.
• Input routing by format:
  • WoS .txt → src/wos_txt_to_csv.py → data/processed/wos_merged.csv
  • CSV/Excel → agent 检测字段完整度，复制到对应的 processed 位置，跳过已完成的子阶段
• Output: data/processed/wos_merged.csv, data/processed/wos_cleaned.csv, data/processed/paper_authors.csv, data/processed/authors_summary.csv

Stage 2: Handoff Gate

• Delegate to paper-processing-analysis-handoff skill.
• Validates UT contract, column schema, artifact existence.
• Returns: analysis-ready, ready-with-risks, or blocked-*.
• Do NOT proceed to Stage 3 unless status is analysis-ready or ready-with-risks.

Stage 3: Analysis and Reporting

• Delegate to paper-analysis-report skill.
• Sub-stages executed in order:
  1. Topic modeling → data/processed/paper_topics.csv, reports/topic_info_*.csv
  2. Mobility analysis → reports/flow_matrix.csv, reports/talent_flow_sankey.html
  3. Chapter figures/tables → reports/figures/, reports/tables/
  4. Final report assembly → reports/{RESEARCH_TOPIC}_bibliometric_report.md
  5. Export figure-integrated Word report → reports/{RESEARCH_TOPIC}_bibliometric_report.docx
  6. Report quality gate → run skills/paper-analysis-report/scripts/report_quality_gate.py --report reports/{RESEARCH_TOPIC}_bibliometric_report.md
  7. Manual report checklist → skills/paper-analysis-report/checklists/report-quality-checklist.md

The orchestrator must not export a final report to a generic bibliometric_report.* name when RESEARCH_TOPIC is known. Final report paths should bind to the current topic to avoid stale-report reuse in cold-start testing.

Stage 4: Verification

• All expected artifacts exist at workspace-relative paths.
• Final report references valid figure paths.
• Final Word report exists and is suitable as the formal illustrated deep-analysis deliverable.
• Report quality gate passes with no blocking issues.
• Final report is not accepted if it only provides descriptive ranking/chart narration without data-backed structural judgment.
• Summary of outputs and any residual risks emitted.

Stage Gate Contracts

Gate	Check	Blocking?
Processing → Analysis	UT valid, required columns present, artifacts exist	Yes
Analysis → Reporting	Topic + mobility outputs exist, schema correct	Yes
Final	Report artifacts generated, key metrics summarized, report quality gate passed	Yes

Rerun Strategy

• Full rerun: execute all stages 1 → 4.
• Partial rerun: execute only impacted stage + downstream.
• No rerun of unaffected upstream stages unless schema changed.

Failure Handling

• Stop at failed stage.
• Emit: failing script, error summary, missing artifact list.
• Do not proceed downstream until gate checks pass.

New Machine Quick Start

1. Copy workspace folder to new machine.
2. python -m venv .venv
3. .venv\Scripts\activate        # Windows
   source .venv/bin/activate     # Linux/Mac
4. pip install -r requirements.txt
5. Place WoS .txt files in data/raw/
6. Ask: "从 WoS 数据开始走完整分析流程"

Done Criteria

• All requested stages complete.
• Artifacts generated at expected workspace-relative paths.
• Final summary lists outputs and residual risks.
• On a fresh machine with only WoS data, the full pipeline runs without path editing.