relation-extraction-pipeline

# Relation Extraction Pipeline Use this sub-skill when the task is to create, validate, split, or troubleshoot the remote-supervised relation-extraction dataset, or to understand the TensorFlow PCNN training stack in this agriculture knowledge-graph repository. ## Route here for - Turning aligned Wikidata/Wikipedia sentence rows into `rel2id.json`, `entity2id.json`, `dataset.json`, `word2vec.json`, `train_dataset.json`, and `test_dataset.json`. - Validating six-column training TSV rows and the JSON schemas expected by the PCNN data loader. - Diagnosing Fire command names, working-directory-sensitive paths, stale `_processed_data`, bad entity positions, relation-label filters, or NA-sample split failures. - Reviewing the PCNN model configuration, TensorFlow 1.x assumptions, GPU settings, and large word-vector limits before any expensive training run. ## Route elsewhere - General Wikidata crawling, Wikidata JSON-to-CSV conversion, Neo4j import CSV creation, Hudong/weather crawlers, and network collection pipelines belong to the crawler/Wikidata workflow, not this sub-skill. - Django relation-label annotation UI, Mongo-backed tagging pages, and web-service startup belong to the web-app workflow, not this sub-skill. - Neo4j graph querying or Cypher import plans belong to graph query/data management. ## Operating map 1. For the end-to-end data build sequence, required source artifacts, Fire commands, and safe replacement commands, read [dataset preparation](references/dataset-preparation.md). 2. For exact row and JSON schemas, path-sensitive generated files, and validator usage, read [data formats](references/data-formats.md). 3. Before training or editing `config.py`/`train.py`, read [PCNN training](references/pcnn-training.md). 4. For known failure modes and recovery actions, read [troubleshooting](references/troubleshooting.md). ## Bundled scripts - [scripts/relation_dataset_schema_check.py](scripts/relation_dataset_schema_check.py) validates tiny training TSV, `rel2id`, `entity2id`, dataset JSON, and optional word-vector JSON files without importing TensorFlow. - [scripts/deduplicate_training_rows.sh](scripts/deduplicate_training_rows.sh) deduplicates training rows with explicit input and output paths and a safe `--help` path. ## Safe default Do not launch PCNN training, live Neo4j alignment, Mongo annotation, network crawling, or large word-vector conversion as a first check. First run the schema checker or a tiny deduplication fixture, then escalate only after data files, working directory, and TensorFlow 1.x/GPU constraints are explicit.