langfuse-extract

# Langfuse Extract Read-side tool: pulls data OUT of Langfuse (traces, sessions, observations/generations, prompts, scores, daily metrics) via the public REST API. For instrumenting an app to SEND traces to Langfuse, use the `langfuse-observability` skill instead. ## When to Activate - "dame las traces de ayer con error" - "extrae las sessions del usuario X" - "necesito el historial de prompts de support_agent_v2" - "cuánto costó el proyecto Y la semana pasada" (daily metrics) - Building an eval/fine-tuning dataset from production traces - Auditing what a specific session/trace actually sent to the model ## Credentials — `.env` convention Same pattern as `~/.intercom_*.env` / `~/.twilio_*.env`: one `.env` file per workspace/project at `~/.langfuse_<workspace>.env`, never committed to git. Template: `.env.example` in this skill's directory. Required vars: ``` WORKSPACE_NAME= # label for logs LANGFUSE_HOST= # https://cloud.langfuse.com (EU) / https://us.cloud.langfuse.com (US) / self-hosted LANGFUSE_PUBLIC_KEY= # pk-lf-... LANGFUSE_SECRET_KEY= # sk-lf-... ``` Get keys from the Langfuse project: Settings > API Keys. Auth is HTTP Basic — username=public key, password=secret key. Region matters: a US cloud project's keys will 401 against the EU host and vice versa. Set `chmod 600` on the env file. ## Tools in this skill - `scripts/langfuse_client.py` — reusable `LangfuseClient` class (auth, pagination, retry on 429/5xx). Import this if writing a custom extraction script. - `scripts/extract.py` — CLI wrapper. Covers the common cases without writing code. ### CLI usage ```bash cd ~/.claude/skills/langfuse-extract/scripts python extract.py <kind> --env-file ~/.langfuse_prod.env [options] ``` `<kind>`: `traces` | `trace` | `sessions` | `session` | `observations` | `prompts` | `prompt` | `scores` | `metrics` | `health` ```bash # Traces in a date range, filtered by session python extract.py traces --env-file ~/.langfuse_prod.env \ --from 2026-07-01 --to 2026-07-16 --session-id abc123 -o traces.jsonl -f jsonl # Full session detail (includes its traces) python extract.py session --env-file ~/.langfuse_prod.env --session-id abc123 # Debugging shortcut: "what changed in this agent's prompt" for a whole session — # every GENERATION across all its traces, with model + linked prompt name/version + input/output python extract.py debug-session --env-file ~/.langfuse_prod.env --session-id abc123 # Generations only (skip spans/events) for a trace — useful to see exact model I/O python extract.py observations --env-file ~/.langfuse_prod.env \ --trace-id abc123 --type GENERATION # Specific prompt version/label python extract.py prompt --env-file ~/.langfuse_prod.env \ --name support_agent_v2 --label production # Scores (evals / human feedback) in a window python extract.py scores --env-file ~/.langfuse_prod.env --from 2026-07-01 # Daily cost/usage metrics for one traced function python extract.py metrics --env-file ~/.langfuse_prod.env \ --trace-name generate_reply --from 2026-07-01 # Sanity check credentials/host before a big pull python extract.py health --env-file ~/.langfuse_prod.env ``` Output: `--output/-o file --format/-f json|jsonl|csv` (default: prints JSON to stdout). `csv` flattens nested fields (input/output/metadata) to JSON strings per cell. `--max-items N` (default 500) caps TOTAL items across all pages for `traces`, `observations`, `scores` — not just page size (`--limit` is page size only). Without this cap an unscoped list call on an active project pages forever. `session`/`trace`/`prompt` (single-resource get) and `debug-session` (bounded by one session's traces) ignore it — there's nothing to cap. ## API reference (what the client wraps) Base path: `{LANGFUSE_HOST}/api/public/...`. All list endpoints are paginated (`page`, `limit` params; response `{ data: [...], meta: { page, totalPages, totalItems } }`) — `LangfuseClient._paginate` walks all pages automatically, so callers just get an iterator of items. | Resource | List | Get one | Key filters | |---|---|---|---| | Traces | `GET /api/public/traces` | `GET /api/public/traces/{id}` | `fromTimestamp`, `toTimestamp`, `sessionId`, `userId`, `name`, `tags`, `environment` | | Sessions | `GET /api/public/sessions` | `GET /api/public/sessions/{id}` (includes its traces) | `fromTimestamp`, `toTimestamp` | | Observations | `GET /api/public/observations` | `GET /api/public/observations/{id}` | `traceId`, `type` (SPAN/GENERATION/EVENT), `userId`, `name`, `fromStartTime`, `toStartTime` | | Prompts | `GET /api/public/v2/prompts` | `GET /api/public/v2/prompts/{name}` | `version` or `label` (e.g. `production`, `latest`) for get-one; `name` for list | | Scores | `GET /api/public/scores` | — | `traceId`, `userId`, `name`, `fromTimestamp`, `toTimestamp` | | Daily metrics | `GET /api/public/metrics/daily` | — | `traceName`, `userId`, `tags`, `fromTimestamp`, `toTimestamp` | | Debug session (custom, not a raw endpoint) | `debug-session --session-id` -> session's traces -> each trace's GENERATION observations, sorted by time | — | `session_id` only | Full reference: https://api.reference.langfuse.com ### Gotchas - **`fromTimestamp`/`toTimestamp` need full ISO-8601 datetime, not a bare date.** The API 400s on `2026-07-01` — `LangfuseClient._ts` auto-expands a plain `YYYY-MM-DD` to `T00:00:00Z`, so passing dates on the CLI is fine; only matters if you're calling the client's methods directly with something else shaped. - **Region-locked keys.** Cloud keys are tied to one region (EU `cloud.langfuse.com` vs US `us.cloud.langfuse.com`). Wrong host -> 401, not a clearer error. Run `health` kind first when setting up a new `.env` — though note `health` doesn't check auth, only connectivity/version, so a 200 there doesn't mean the keys are valid. - **Unscoped list calls can hang or time out server-side**, not just return a lot of data — `/sessions` and `/traces` with no date range measured 30s+ per page on a live project in testing. `LangfuseClient` retries timeouts/connection errors with backoff (default 60s per-request timeout, 5 attempts) and `--max-items` caps total items, but the real fix is scoping `--from`/`--to` tight — don't rely on the cap to make a lazy unscoped query fast, it only stops it from running forever. - **Rate limits.** Public API is rate-limited per plan; `LangfuseClient` retries on 429 honoring `Retry-After`, and backs off on 5xx. Don't lower `max_retries` to 0 for bulk pulls. `/metrics/daily` tends to have tighter limits than `/traces`. https://langfuse.com/faq/all/api-limits - **`GENERATION` observations carry the actual prompt/completion** sent to the model (`input`/`output` fields) — that's what you want for eval datasets or debugging a bad reply, not the trace object itself (which is closer to a request-level summary). `promptName`/`promptVersion` on the observation are only populated if the app fetched the prompt via `langfuse.get_prompt(...)` and passed it through — a hardcoded prompt string in the code won't show a linked version at all. - **Sessions ≠ traces.** A session groups multiple traces (e.g. one per turn in a conversation). Pulling `session` gives you the full multi-turn context in one call; `debug-session` goes one step further and flattens every generation across all of the session's traces — the shortcut for "what did we send this agent, turn by turn." - **Prompt versioning.** `label="production"` gets whatever is currently deployed; a numeric `version` pins to an exact snapshot. Use the label when auditing "what's live right now", the version when reproducing a specific past run. To see if a prompt changed recently: pull `debug-session` for a session before/after the suspected change and diff `prompt_version` across generations of the same `trace_name`. - **Large pulls.** No built-in checkpointing in `extract.py` (unlike the Intercom extractor) — for multi-day/multi-GB pulls, chunk by date range in a loop and write incrementally, or extend `langfuse_client.py`'s `_paginate` with a resume cursor. ## Reference Skills - Instrumenting an app to send traces → skill: `langfuse-observability` - Building eval datasets from extracted traces → skill: `rag-patterns`, `llm-engineering`