fetch-data

name	fetch-data
description	Stage 1 of the pipeline. Fetch a small CRSP return panel from WRDS and write it to data/returns.csv. Update state.json. Verify schema. Triggers when state.stage == "init".

Fetch data

Pre

Read state/state.json. Confirm state.stage == "init". If not, refuse and report.

Act -- pick the universe

This skill leaves the universe choice to you, the orchestrator. Pick 8 well-known US large-cap tickers spanning at least 3 sectors (e.g. tech, financials, energy, staples, healthcare). Avoid recent IPOs and recent ticker re-uses -- the names should have a stable CRSP nameendt >= today.

Before running the fetch, write the picks and a one-sentence rationale into state.last_action.note, e.g.:

"picks: AAPL,MSFT,JPM,GS,XOM,CVX,KO,PG -- mega-caps across tech/financials/energy/staples"

This is the claim the verifier subagent will check the CSV against.

Act -- fetch

Run:

python scripts/fetch_wrds.py --tickers <your 8 picks> --months 24 --out data/returns.csv

The script resolves each ticker to its current CRSP permno via crsp.msenames (filtering on shrcd IN (10, 11) and nameendt >= CURRENT_DATE), pulls 24 months of monthly returns from crsp.msf, joins ff.factors_monthly for mkt_rf and rf, and writes the CSV. It prints a one-line summary on success.

The script requires WRDS_USERNAME to be set so libpq can match the password entry in %APPDATA%\postgresql\pgpass.conf. The script auto-loads .env from the demo root (copy .env.example to .env and fill in), or you can export it directly in the shell. If the fetch exits non-zero for any reason (WRDS unreachable, credentials fail, etc.): set state.stage = "halted", put the error in state.halt_reason, write state, run python scripts/verify_state.py, and exit.

Verify -- inline

Load data/returns.csv and confirm:

• File exists, > 100 rows
• Columns include date, permno, ticker, ret, mkt_rf, rf
• No NaN in ret or mkt_rf
• date parses as datetime

If any check fails: set state.stage = "halted", set state.halt_reason, write state, run python scripts/verify_state.py, and exit.

Verify -- subagent

Spawn the verifier subagent (.claude/agents/verifier.md) with this prompt:

Read state/state.json and data/returns.csv. Confirm the tickers present in the CSV match the universe declared in state.last_action.note. Confirm the row count is consistent (~24 monthly observations per ticker), the date range is the last 24 months, and the schema has columns date, permno, ticker, ret, mkt_rf, rf. Reply OK or list discrepancies.

If the verifier returns anything other than OK: set state.stage = "halted", put the discrepancies in state.halt_reason, write state, run python scripts/verify_state.py, and exit.

Post

Update state.json:

state.stage = "fetched"
state.artifacts.data_path = "data/returns.csv"
state.last_action = {tool: "fetch-data", ts: <now>, ok: true, note: "<picks + rationale>"}

Run python scripts/verify_state.py. If it fails: halt.