تشغيل أي مهارة في Manus بنقرة واحدة

source-extraction-coverage

النجوم١

التفرعات٢

آخر تحديث٨ يونيو ٢٠٢٦ في ٠٩:٠١

Runs doc-type-aware deterministic extraction on a source and makes shallow extraction visible by recording an extraction_estimate before and an extraction_yield after, flagging any source whose yield falls short. Use when ingesting a new source file (PDF/DOCX/XLSX/HTML/scanned) into the provisional store.

التثبيت

التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.

تشغيل في Manus

المصدر

vamseeachanta

vamseeachanta/raw-to-knowledge-playbook

فتح مستودع GitHub عرض مستودعات المنشئ

تنزيل

تشغيل في Manus

المهن ذات الصلةSOC

استنادا إلى تصنيف SOC المهني

مطوّرو البرمجياتمهن الحاسوب والرياضيات·SOC 15-1252

مستكشف الملفات

2 ملفات

SKILL.md

readonly

name	source-extraction-coverage
description	Runs doc-type-aware deterministic extraction on a source and makes shallow extraction visible by recording an extraction_estimate before and an extraction_yield after, flagging any source whose yield falls short. Use when ingesting a new source file (PDF/DOCX/XLSX/HTML/scanned) into the provisional store.
license	CC-BY-4.0
compatibility	Requires a deterministic extractor per doc type (pdfplumber/Docling, python-docx, openpyxl, OCR) and a landing-page frontmatter store
metadata	{"version":"1.0","enforcement_level":"L2","status":"template","incident_refs":"coverage-2pct,A-defects,D4","params":"path:str \| type:enum(auto,pdf,docx,xlsx,html,scanned)=auto"}

source-extraction-coverage

Template skill (doc 08, doc 01). The estimate/yield pair is the whole point: it is how the 2%-coverage failure (LLM-extracted a doc and silently produced ~2% of its text) becomes a loud, machine-checkable signal instead of a silent loss.

Trigger

Preconditions

Source lives in the off-repo read-only archive; record its sha256.
Type is detected by content/structure, not filename (incident D4: names lie about content).

Steps

Estimate first. Before extracting, declare extraction_estimate in the landing-page frontmatter: roughly how much extractable content this source holds (page count, table count, "text-heavy" vs "image-only"). This is a commitment made before you see the result, so a shortfall is undeniable.
Pick the recipe by type (use ADOPT-tier permissive tools from doc 12):
- pdf (born-digital): deterministic text+geometry (pdfplumber/Docling); pdf (scanned/image-only) → route to the scanned recipe (OCR as labeled interpretation, doc 11).
- docx: structured access (python-docx) + semantic Markdown (mammoth).
- xlsx: formula-graph read (openpyxl) / fast data read (calamine) — classify by scanned structure, not sheet names (D4).
- html: typed-element extraction; strip nav/boilerplate.
- scanned: OCR with a blur gate (OpenCV Laplacian-variance → too blurry stays provisional); OCR output is a claim, not source text.
Extract deterministically. Tools, not an LLM, do the bulk copy. (LLM bulk extraction yielded ~2% coverage — never the bulk path.)
Record yield. Write extraction_yield after extraction (pages/tables actually produced, % of estimate).
Flag shortfall. If yield << estimate, mark the source extraction: shallow and queue it for a different recipe. Do not let it pass as "done."

Verification

Frontmatter has BOTH extraction_estimate (pre) and extraction_yield (post); a validator script rejects a page missing either.
Sources flagged shallow appear in the re-extraction backlog, not the trusted set.

Cleanup

Raw source never copied into any repo; only derived parts + a sources: sha256 pointer.

Incident appendix

Rule	Why
Estimate before, yield after	The 2%-coverage loss was invisible until the pair made it loud
Classify on structure not name	D4: sheet/file names misclassify content
OCR output is a claim	doc 11: scanned text is interpretation, enters the provisional gate

المزيد من هذا المستودع

نفس المستودع

format-coverage-ledger

vamseeachanta/raw-to-knowledge-playbook

Runs the right deterministic extraction lane per office/email format and records each format's KNOWN losses UNDER A TEXT/CSV LANE in a coverage ledger so a faithful extract never masquerades as complete. Use when extracting docx/xlsx/msg/pdf/pptx sources whose richest content (formulas, charts, diagrams, attachments, images, speaker notes) does not survive a text/CSV dump.

2026-06-271

archive-extraction-integrity

vamseeachanta/raw-to-knowledge-playbook

Extracts a compressed archive (RAR/ZIP/7z) and then proves the extraction is byte-faithful by re-hashing every extracted file against the container's stored per-entry checksums — because an open-source decoder can report success while silently writing wrong bytes at the correct size (presence and size are NOT fidelity). Falls back across decoders (7z, bsdtar, official unrar) for entries one engine mis-decodes, and reports the exact set of files that no available decoder can reproduce. Use when extracting any archive before its contents enter an ingestion pipeline, especially RAR4 archives or when only partial decoders (p7zip, libarchive) are installed.

2026-06-201

independent-oracle-validation

vamseeachanta/raw-to-knowledge-playbook

Scores committed extracts against an independent extraction engine (a different codebase than the pipeline lane) to produce measured per-lane precision/recall, with two-tier numeric scoring and a hard rule that every sub-1.0 number is attributed to a named cause. Use when a conversion lane needs correctness evidence beyond the fidelity verify loop — turning "we believe the extracts are faithful" into per-lane measured numbers.

2026-06-101

adversarial-verify-loop

vamseeachanta/raw-to-knowledge-playbook

Runs an independent adversarial verification loop on a produced change (extraction code, corpus batch, or page set) until a clean PASS — the producer never self-certifies, the reviewer is prompted to refute and to build runnable reproducers for code findings, and every fix is gated on re-running the reviewer's own reproducer. Use when a change must earn justified confidence before merge, especially pipeline-code changes where a plausible fix can be narrower than the defect class.

2026-06-091

source-extract-fidelity

vamseeachanta/raw-to-knowledge-playbook

Independently reviews a batch of source-extract reference pages against the source binary so that no prose claims more than the committed extract supports — checking hash match, extract fidelity, and that every quoted figure traces to a committed extract with a cross-reference treated as not-a-quote and a derived number as not-quoted. Use when reviewing extracted/source-derived pages before merge, especially text-faithful pages whose prose may overreach.

2026-06-091

xlsx-input-code-output-canary

vamseeachanta/raw-to-knowledge-playbook

Classifies XLS/XLSX workbooks before extraction and requires a traceable input data or logic contract, code artifact, and verified output artifact for a ten-file canary. Use when spreadsheet files may contain formulas, named ranges, charts, cached values, protection, or mixed data and calculation logic.

2026-06-091

name	source-extraction-coverage
description	Runs doc-type-aware deterministic extraction on a source and makes shallow extraction visible by recording an extraction_estimate before and an extraction_yield after, flagging any source whose yield falls short. Use when ingesting a new source file (PDF/DOCX/XLSX/HTML/scanned) into the provisional store.
license	CC-BY-4.0
compatibility	Requires a deterministic extractor per doc type (pdfplumber/Docling, python-docx, openpyxl, OCR) and a landing-page frontmatter store
metadata	{"version":"1.0","enforcement_level":"L2","status":"template","incident_refs":"coverage-2pct,A-defects,D4","params":"path:str \| type:enum(auto,pdf,docx,xlsx,html,scanned)=auto"}

source-extraction-coverage

Template skill (doc 08, doc 01). The estimate/yield pair is the whole point: it is how the 2%-coverage failure (LLM-extracted a doc and silently produced ~2% of its text) becomes a loud, machine-checkable signal instead of a silent loss.

Trigger

Preconditions

Source lives in the off-repo read-only archive; record its sha256.
Type is detected by content/structure, not filename (incident D4: names lie about content).

Steps

Estimate first. Before extracting, declare extraction_estimate in the landing-page frontmatter: roughly how much extractable content this source holds (page count, table count, "text-heavy" vs "image-only"). This is a commitment made before you see the result, so a shortfall is undeniable.
Pick the recipe by type (use ADOPT-tier permissive tools from doc 12):
- pdf (born-digital): deterministic text+geometry (pdfplumber/Docling); pdf (scanned/image-only) → route to the scanned recipe (OCR as labeled interpretation, doc 11).
- docx: structured access (python-docx) + semantic Markdown (mammoth).
- xlsx: formula-graph read (openpyxl) / fast data read (calamine) — classify by scanned structure, not sheet names (D4).
- html: typed-element extraction; strip nav/boilerplate.
- scanned: OCR with a blur gate (OpenCV Laplacian-variance → too blurry stays provisional); OCR output is a claim, not source text.
Extract deterministically. Tools, not an LLM, do the bulk copy. (LLM bulk extraction yielded ~2% coverage — never the bulk path.)
Record yield. Write extraction_yield after extraction (pages/tables actually produced, % of estimate).
Flag shortfall. If yield << estimate, mark the source extraction: shallow and queue it for a different recipe. Do not let it pass as "done."

Verification

Frontmatter has BOTH extraction_estimate (pre) and extraction_yield (post); a validator script rejects a page missing either.
Sources flagged shallow appear in the re-extraction backlog, not the trusted set.

Cleanup

Raw source never copied into any repo; only derived parts + a sources: sha256 pointer.

Incident appendix

Rule	Why
Estimate before, yield after	The 2%-coverage loss was invisible until the pair made it loud
Classify on structure not name	D4: sheet/file names misclassify content
OCR output is a claim	doc 11: scanned text is interpretation, enters the provisional gate