메뉴
Debug and fix DPLA hub ingestion failures (harvest/mapping/enrichment/jsonl/s3-sync/anomaly). Use when user asks why a hub failed, to debug an ingest failure, check an escalation report, or retry a failed hub/stage.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Workflow for generating and sending DPLA monthly pre-ingest emails to hub partners via AWS SES. Use this skill whenever someone asks to: send the monthly ingest email, prepare the DPLA pre-ingest notice, generate the hub email for a given month, preview the pre-ingest email, or do anything involving the pre-ingest email pipeline. Trigger even if the user just says something like "time to send the ingest email" or "it's email day" — context from the project makes the intent clear.
Run Community Webs ingest. Use when the user says harvest community-webs, run community-webs ingest, export community webs, or process community webs DB.
Remove DPLA items from the live site and search index in response to takedown requests. Use when the user says take down, delete, remove item(s) from dp.la, or action a takedown request. Accepts DPLA IDs directly or criteria (hub, institution, collection) to generate an ID list. Includes mandatory pre-flight verification, search-before-delete on Elasticsearch, and post-deletion confirmation with count delta verification.
Show key i3.conf config for a hub (provider, harvest.type, harvest.endpoint, schedule, email, setlist). Use when user asks for hub config, harvest type/endpoint, who gets emails, schedule months, or OAI setlist details.
Debug and fix DPLA hub ingestion failures (harvest/mapping/enrichment/jsonl/s3-sync/anomaly). Use when user asks why a hub failed, to debug an ingest failure, check an escalation report, or retry a failed hub/stage.
Show all active and completed ingests in one consolidated view. Use when user asks status of the ingests, ingest status, which hubs are running, show active ingests, or ingest progress.
| name | dpla-ingest-debug |
| description | Debug and fix DPLA hub ingestion failures (harvest/mapping/enrichment/jsonl/s3-sync/anomaly). Use when user asks why a hub failed, to debug an ingest failure, check an escalation report, or retry a failed hub/stage. |
Quickly identify what stage failed, find the relevant logs/escalation report, apply a targeted fix, and re-run only the necessary steps.
For any commands that depend on project env (especially the orchestrator), run source .env first so JAVA_HOME, DPLA_DATA, I3_CONF, and SLACK_WEBHOOK are available.
ls -lt data/escalations/ | head
# then open the relevant failures-<run_id>.md
./scripts/status/ingest-status.sh --watch
# or for one hub:
./scripts/status/ingest-status.sh <hub>
ls -lt logs/ | head
Look for one of: harvest, mapping, enrichment, jsonl, sync, anomaly.
If you have only a hub name and “it failed”, the quickest approach is:
logs/status/<hub>.status (JSON), andlogs/.All commands below assume you’re running from repo root.
Harvest failed:
./scripts/harvest.sh <hub>
Mapping/enrichment/jsonl failed (re-run remap):
./scripts/remap.sh <hub>
S3 sync failed (or you want to re-sync after a successful run):
./scripts/s3-sync.sh <hub>
Orchestrator retry (failed hubs from last run):
./venv/bin/python -m scheduler.orchestrator.main --retry-failed
Orchestrator retry (one hub):
./venv/bin/python -m scheduler.orchestrator.main --hub=<hub>
pgrep -fl 'sbt|java.*ingestion3'kill <pid> (avoid broad pkill patterns unless you’re sure).After re-running, verify _SUCCESS markers and counts:
# Examples
ls "$DPLA_DATA/<hub>/harvest"/*/_SUCCESS
ls "$DPLA_DATA/<hub>/mapping"/*/_SUCCESS
ls "$DPLA_DATA/<hub>/jsonl"/*/_SUCCESS
cat "$DPLA_DATA/<hub>/mapping"/*/_SUMMARY | head