菜单
Run S3 sync, find latest S3 data, and perform AWS data operations for DPLA ingestion using the correct profile and scripts. Use when the user says sync to S3, check S3 sync, upload to S3, AWS bucket, check JSONL sync, latest S3 data for a hub, or when was hub last ingested.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | dpla-s3-ops |
| description | Run S3 sync, find latest S3 data, and perform AWS data operations for DPLA ingestion using the correct profile and scripts. Use when the user says sync to S3, check S3 sync, upload to S3, AWS bucket, check JSONL sync, latest S3 data for a hub, or when was hub last ingested. |
Perform S3 and AWS data operations for the ingestion pipeline using the project's profile and scripts so credentials and buckets are correct. Includes finding latest data in S3 and syncing hub data.
Environment: Source .env from repo root before running scripts that need DPLA_DATA or AWS env (e.g. source .env).
All AWS CLI commands must use:
aws ... --profile dpla
Scripts in scripts/ use AWS_PROFILE=dpla by default (see scripts/common.sh / SCRIPTS.md). When invoking aws directly, always add --profile dpla.
To find the latest harvest/mapping/jsonl exports in S3 for a hub:
source .env && bash scripts/status/s3-latest.sh <hub>
This checks s3://dpla-master-dataset/<hub>/ for the most recent timestamped directories under harvest, mapping, and jsonl.
Use the project script (handles anomaly detection and paths):
./scripts/s3-sync.sh <hub>
./scripts/s3-sync.sh <hub> <subdir> # Optional: sync a specific subdir
The script uses the correct bucket and prefix; do not bypass it with raw aws s3 sync unless you have a specific reason and use --profile dpla.
To see how local JSONL exports compare to S3:
./scripts/status/check-jsonl-sync.sh
Uses AWS profile via script env; see scripts/SCRIPTS.md for options.
The orchestrator (and s3-sync.sh when used in that flow) runs anomaly checks before syncing. If counts or failure rates change sharply, sync may be blocked (critical) or proceed with a warning. Escalation and Slack alerts are sent; see AGENTS.md and GOLDEN_PATH.md.
If you must run aws directly (e.g. list bucket, copy one file):
aws s3 ls s3://bucket-name/ --profile dpla
aws s3 cp local s3://bucket/key --profile dpla
Never omit --profile dpla.
| Resource | Path |
|---|---|
| Script reference | scripts/SCRIPTS.md (s3-sync.sh, check-jsonl-sync.sh, s3-latest.sh) |
| Agent / notify policy | AGENTS.md |
Workflow for generating and sending DPLA monthly pre-ingest emails to hub partners via AWS SES. Use this skill whenever someone asks to: send the monthly ingest email, prepare the DPLA pre-ingest notice, generate the hub email for a given month, preview the pre-ingest email, or do anything involving the pre-ingest email pipeline. Trigger even if the user just says something like "time to send the ingest email" or "it's email day" — context from the project makes the intent clear.
Run Community Webs ingest. Use when the user says harvest community-webs, run community-webs ingest, export community webs, or process community webs DB.
Remove DPLA items from the live site and search index in response to takedown requests. Use when the user says take down, delete, remove item(s) from dp.la, or action a takedown request. Accepts DPLA IDs directly or criteria (hub, institution, collection) to generate an ID list. Includes mandatory pre-flight verification, search-before-delete on Elasticsearch, and post-deletion confirmation with count delta verification.
Show key i3.conf config for a hub (provider, harvest.type, harvest.endpoint, schedule, email, setlist). Use when user asks for hub config, harvest type/endpoint, who gets emails, schedule months, or OAI setlist details.
Debug and fix DPLA hub ingestion failures (harvest/mapping/enrichment/jsonl/s3-sync/anomaly). Use when user asks why a hub failed, to debug an ingest failure, check an escalation report, or retry a failed hub/stage.
Show all active and completed ingests in one consolidated view. Use when user asks status of the ingests, ingest status, which hubs are running, show active ingests, or ingest progress.