| name | fetch-data |
| description | Stage 1 of the pipeline. Fetch a small CRSP return panel from WRDS and write it to data/returns.csv. Update state.json. Verify schema. Triggers when state.stage == "init". |
Fetch data
Pre
Read state/state.json. Confirm state.stage == "init". If not, refuse and report.
Act -- pick the universe
This skill leaves the universe choice to you, the orchestrator. Pick 8 well-known
US large-cap tickers spanning at least 3 sectors (e.g. tech, financials, energy,
staples, healthcare). Avoid recent IPOs and recent ticker re-uses -- the names should
have a stable CRSP nameendt >= today.
Before running the fetch, write the picks and a one-sentence rationale into
state.last_action.note, e.g.:
"picks: AAPL,MSFT,JPM,GS,XOM,CVX,KO,PG -- mega-caps across tech/financials/energy/staples"
This is the claim the verifier subagent will check the CSV against.
Act -- fetch
Run:
python scripts/fetch_wrds.py --tickers <your 8 picks> --months 24 --out data/returns.csv
The script resolves each ticker to its current CRSP permno via crsp.msenames
(filtering on shrcd IN (10, 11) and nameendt >= CURRENT_DATE), pulls 24 months
of monthly returns from crsp.msf, joins ff.factors_monthly for mkt_rf and
rf, and writes the CSV. It prints a one-line summary on success.
The script requires WRDS_USERNAME to be set so libpq can match the password
entry in %APPDATA%\postgresql\pgpass.conf. The script auto-loads .env from
the demo root (copy .env.example to .env and fill in), or you can export it
directly in the shell. If the fetch exits non-zero for any reason (WRDS
unreachable, credentials fail, etc.): set state.stage = "halted", put the
error in state.halt_reason, write state, run python scripts/verify_state.py,
and exit.
Verify -- inline
Load data/returns.csv and confirm:
- File exists, > 100 rows
- Columns include
date, permno, ticker, ret, mkt_rf, rf
- No NaN in
ret or mkt_rf
date parses as datetime
If any check fails: set state.stage = "halted", set state.halt_reason,
write state, run python scripts/verify_state.py, and exit.
Verify -- subagent
Spawn the verifier subagent (.claude/agents/verifier.md) with this prompt:
Read state/state.json and data/returns.csv. Confirm the tickers
present in the CSV match the universe declared in state.last_action.note.
Confirm the row count is consistent (~24 monthly observations per ticker),
the date range is the last 24 months, and the schema has columns
date, permno, ticker, ret, mkt_rf, rf. Reply OK or list discrepancies.
If the verifier returns anything other than OK: set state.stage = "halted",
put the discrepancies in state.halt_reason, write state, run
python scripts/verify_state.py, and exit.
Post
Update state.json:
state.stage = "fetched"
state.artifacts.data_path = "data/returns.csv"
state.last_action = {tool: "fetch-data", ts: <now>, ok: true, note: "<picks + rationale>"}
Run python scripts/verify_state.py. If it fails: halt.
