reference-pdf-txt-conversion

name	reference-pdf-txt-conversion
description	Packages the workspace PDF-to-TXT and OCR workflow into a reusable thesis skill. Use when converting PDF literature to TXT mirrors, checking text-layer quality, generating OCR candidate boards, improving PDF-TXT coverage, or handling scanned PDFs in references and external libraries.

Reference PDF-TXT Conversion

Purpose

Use this skill when the task is to:

• convert PDF literature into TXT mirrors
• check whether PDFs already contain usable text
• route low-text or scanned files into OCR flow
• batch-generate conversion reports, manifests, and candidate boards
• improve PDF-TXT coverage without directly rewriting thesis text

Core rules

1. Start with audit or dry-run first.
2. Prefer direct text extraction before OCR.
3. Do not overwrite usable TXT unless explicitly requested.
4. Always leave a reviewable report or manifest.
5. If the output quality is poor, mark it as needs_ocr instead of pretending it succeeded.

Recommended routes

Route A — Bulk books or external library conversion

Primary script:

• scripts/books/convert_books_to_txt.py

Use this route when the source is a book folder or an external library.

Typical pattern:

1. Preview first.
2. Apply in small batches.
3. Check the generated manifest and conversion log.

Outputs:

• 03-resource/books-txt/
• 03-resource/books-txt/_manifest.json
• 03-resource/books-txt/conversion_log*.json

Route B — OCR board driven conversion inside the references library

Primary script:

• 05-scripts/batch_ocr_from_board.py

Use this route when you already have or need an OCR candidate board.

The script will:

• read the candidate CSV
• try mixed extraction or OCR fallback
• write TXT mirrors when appropriate
• generate CSV and Markdown reports under the refined management directory

Preferred usage pattern:

1. limit the first run
2. review quality grades
3. only then expand the batch

Outputs:

• mirrored TXT files under the references tree
• 02-references/management/refined/reference-ocr-batch-*.csv
• 02-references/management/refined/reference-ocr-batch-*.md

Route C — Quick text-layer audit and classification

Useful helpers:

• scripts/classify_pdfs.py
• scripts/check_conversion_status.bat
• check_ocr_progress.bat

Use these before large OCR batches when you need to separate text PDFs from scanned PDFs or check batch progress.

Quality gate

Only treat the task as complete when:

• the TXT file exists and is readable
• the extraction method is recorded
• poor-quality files are honestly marked as needs_ocr, placeholder, or failed
• a manifest or report is left behind for review

Preferred deliverables

• txt mirror
• manifest json
• markdown report
• OCR candidate board
• short PMO summary when the batch changes coverage or processing strategy

Handoff note

This skill is for conversion and audit only. If the user wants evidence extraction or paragraph writing after conversion, hand off to the literature or thesis-writing skills instead of mixing the steps.