screenpipe-api

name	screenpipe-api
description	Query the user's data via the local screenpipe REST API at localhost:3030 — screen recordings, audio, UI elements, usage analytics, meetings, connected services, and persistent memory. Use for questions about screen activity, meetings, apps, productivity, media export, retranscription, connections, or durable memory.

Screenpipe API

Local REST API at $SCREENPIPE_LOCAL_API_URL (fallback http://localhost:3030). Always use ${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030} as the base in shell calls so a fallback-port or development app cannot reach another running Screenpipe instance.

Prefer this over the CLI for reads. A curl against the local API returns in ~0.02s; a screenpipe CLI call costs ~0.15s at best and ~4s when it has to resolve screenpipe@latest from npm. Reach for the CLI only for state changes it uniquely owns (pipe enable, connection set).

Authentication

If screenpipe MCP tools are available in your session, prefer them — same data, no key or network handling. Some agent sandboxes (e.g. Codex) block all shell network access including localhost, so curl can never work there.

Every curl request needs auth (403 without it). Resolve the key in order, stop at the first hit:

1. $SCREENPIPE_LOCAL_API_KEY is already set in your env → use it as-is.
2. Not set → fetch it once: export SCREENPIPE_LOCAL_API_KEY="$(cd "$(mktemp -d)" && bun x screenpipe@latest auth token)"
3. curl fails instantly (Failed to connect ... after 0 ms) even though screenpipe is running → your shell is network-sandboxed; stop retrying curl and use the MCP tools.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  -H "X-Screenpipe-Client: api" \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/..."

The fixed X-Screenpipe-Client: api value attributes a successful, nonempty external retrieval to the API surface. Never put an agent name, customer name, project, prompt, or other dynamic value in this header.

No-auth endpoints: /health, /ws/health, /audio/device/status, /connections/oauth/callback, /frames/*, /notify, /pipes/store/*.

Context Window Protection

Responses can be large. Write curl output to a file (-o /tmp/sp.json), check size (wc -c), and if over ~5KB read only the first 50-100 lines. Never dump full large responses into context.

Only assume curl, wc, head, grep, sed and bun exist. jq is not installed on every machine — stock macOS and the bundled Windows bash both lack it. To pull fields out of JSON, either ask the API for flat rows (format=csv, below) and read them with head, or use bun, which always ships with screenpipe:

bun -e 'const d=await Bun.file("/tmp/sp.json").json(); for (const r of d.data.slice(0,20)) console.log(r.type, r.content.app_name??"", (r.content.text??r.content.transcription??"").slice(0,120))'

Use jq only after confirming it exists (command -v jq).

Cut tokens at the source on list endpoints (/search, /elements). Two independent knobs, both shown in the examples below — copy them:

• &fields=a,b,c — always set it. Dotted paths (content.text, content.app_name). Applies to every content type, including text-heavy ocr/audio, where you should also set max_content_length.
• &format=csv (or tsv) — columnar table, column names written once instead of per-row keys. ~70% cheaper on uniform rows, so use it on /elements and on single-content_type /search calls. Skip it on mixed content_type=all, where rows have different shapes and CSV gains little.

---

1. Activity Summary — GET /activity-summary

Default broad-context call. Bundles apps, windows, key_texts, audio, edited_files, recording health, top memories, deduped screen+audio snippets, and a data_status/query_status/guidance triple.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  -H "X-Screenpipe-Client: api" \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/activity-summary?start_time=30m%20ago&end_time=now"

Required: start_time, end_time. Optional: app_name, q (filters memories+snippets, drives query_status); include_recording|memories|snippets|guidance=false to slim (each defaults true); max_snippets, max_snippet_chars, max_memories. For a lean time-tracking sweep also set include_key_texts=false (biggest win), include_apps=false, include_windows=false — total_active_minutes + per-app/window minutes + the status triple still return.

• data_status ∈ ok|empty_but_recording|no_capture_in_range|not_recording — check before claiming "no activity".
• query_status ∈ not_requested|matched|no_query_matches; guidance.next_best_query is a ready hint when empty.
• Escalate to /search only for verbatim quotes / frame_ids.

---

2. Search — GET /search

Use when /activity-summary says ok but you need verbatim quotes, media paths, frame IDs, or a specific match.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  -H "X-Screenpipe-Client: api" \
  -o /tmp/sp.json \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/search?q=QUERY&content_type=all&limit=10&start_time=1h%20ago&fields=type,content.app_name,content.text,content.transcription,content.timestamp"
wc -c /tmp/sp.json && head -c 2000 /tmp/sp.json

Parameter	Required	Description
q	No	Keywords. Avoid for audio — transcriptions are noisy, q over-filters.
content_type	No	all (default), accessibility, audio, input, ocr, memory, parsed. Use parsed for compact app-specific messages, emails, tasks, documents, and code review. Parsed capture is experimental, may be empty when disabled/unsupported, and is not included in all. Screen text is primarily the accessibility tree; OCR is the fallback for apps without it (videos, games, remote desktops).
limit	No	Default 20. Must be 1-20 — never pass a larger value; page with offset instead.
offset	No	Pagination. Default 0.
start_time	Yes	ISO 8601, relative (16h ago, 2d ago, 30m ago), or local calendar literal (today, yesterday, YYYY-MM-DD).
end_time	No	Same forms as start_time; defaults to now.
app_name	No	Substring, e.g. "Google Chrome", "Slack".
window_name	No	Window title substring.
frame_id	No	With content_type=parsed, return parsed data attached to one frame.
actor_id	No	With content_type=parsed, filter by a resolved actor identity.
speaker_name	No	Filter audio by speaker (case-insensitive partial).
focused

Calendar ranges are local: today, yesterday, and bare YYYY-MM-DD dates mean the user's LOCAL calendar days in their timezone, not UTC days or rolling 24-hour ranges. Pass calendar literals directly to the API (start_time=today&end_time=now, start_time=yesterday&end_time=today). Never calculate midnight with date -u or append T00:00:00Z.

Other critical rules: always include start_time (unbounded queries timeout) · "recent" = 30 min · if /search is empty, fall back to /activity-summary and check data_status before saying "no data" · on timeout, narrow the range. · always pass fields= with only the columns you need · always keep limit between 1 and 20 · always write the response to a file with -o and read it with head, never straight to stdout · "recent" = 30 min, "today" = since midnight, "yesterday" = yesterday's range · if /search is empty, fall back to /activity-summary and check data_status before saying "no data" · on timeout, narrow the range.

Single content_type means uniform rows, so add format=csv too:

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  -H "X-Screenpipe-Client: api" \
  -o /tmp/sp.csv \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/search?content_type=ocr&limit=20&start_time=2h%20ago&format=csv&fields=content.timestamp,content.app_name,content.text"
head -20 /tmp/sp.csv

Tags link people/projects/topics across screen, audio, and memories under one namespace (person:ada, project:atlas, topic:pricing). Add to a frame/audio: POST /tags/vision/{frame_id} or POST /tags/audio/{chunk_id} body {"tags":["person:ada"]}; to a memory: tags in POST /memories. Retrieve: GET /search?tags=person:ada&start_time=30d%20ago (add content_type=memory for memories). Frames are pruned by retention — tag a memory for durable links (memories carry created_at + a frame_id back to the moment). include_related=true returns co-occurring tags grouped by namespace, replacing 2-3 follow-up calls.

Response: {"data": [{"type":"OCR","content":{"frame_id":...,"text":...,"app_name":...}}, {"type":"Audio","content":{"chunk_id":...,"transcription":...,"speaker":{"name":...}}}, {"type":"Parsed","content":{"frame_id":...,"text":...,"items":[...],"actors":[...]}}], "pagination":{"limit":10,"offset":0,"total":42}}.

---

3. Elements — GET /elements

Lightweight FTS over UI elements (~100-500 bytes each vs 5-20KB from /search). Uniform rows, so format=csv pays off most.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/elements?frame_id=12345&format=csv&fields=role,text,bounds.left,bounds.top"

Params: q, frame_id, source (accessibility|ocr), role, start_time, end_time, app_name, limit, offset, format, fields.

Use format=outline for token-efficient reading. Use format=automation only for automation planning: it keeps interactive controls and returns a snapshot revision, short response-local refs, best-effort stable keys, state, bounds, and allowed actions. Refresh before each action and verify key + role + name + bounds. Database element ids and response refs are not durable live UI handles. format=preferred follows the desktop AI context setting; its default is the read/memory outline.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/frames/12345/elements?format=automation"

Frame context (accessibility text, parsed nodes, extracted URLs): GET /frames/{id}/context.

Roles are not normalized across platforms — use the right one for the user's OS:

Concept	macOS	Windows	Linux
Button	AXButton	Button	Button
Static text	AXStaticText	Text	Label
Link	AXLink	Hyperlink	Link
Text field	AXTextField	Edit	Entry
Menu item	AXMenuItem	MenuItem	MenuItem
Checkbox	AXCheckBox	CheckBox	CheckBox
Web area	AXWebArea	Pane	DocumentWeb
Heading	AXHeading	Header	Heading
List item	AXRow	ListItem	ListItem

OCR-only roles (accessibility-unavailable fallback): line, word, block, paragraph, page.

---

4. Frames (Screenshots) — GET /frames/{frame_id}

curl -o /tmp/frame.png "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/frames/12345"

Raw PNG. Never fetch more than 2-3 frames per query (~1000-2000 tokens each).

---

5. Media Export — POST /export

Real-time MP4 (screen frames at true timestamps + synced mic audio). Duration matches the wall-clock span — NOT a timelapse.

curl -X POST "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/export" -H "Content-Type: application/json" \
  -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" -d '{"start": "5m ago", "end": "now"}'

Fields: start+end (ISO 8601 or relative; end defaults to now), OR meeting_id for a whole meeting. Optional output_path (absolute, e.g. ~/Downloads/clip.mp4); else lands in the data dir's exports/. Returns {output_path, frame_count, audio_chunk_count, duration_secs, file_size_bytes} — show output_path as inline code. Long ranges take minutes.

ffmpeg on audio file_path from search results (always -y, save to ~/.screenpipe/exports/):

ffmpeg -y -i audio.mp4 -q:a 2 out.mp3                              # convert
ffmpeg -y -i in.mp4 -ss 00:01:00 -to 00:05:00 -q:a 2 clip.mp3      # trim
ffmpeg -y -i in.mp4 -t 10 -vf "fps=10,scale=640:-1" out.gif        # GIF

---

6. Retranscribe — POST /audio/retranscribe

curl -X POST http://localhost:3030/audio/retranscribe -H "Content-Type: application/json" \
  -d '{"start": "1h ago", "end": "now"}'

Optional: engine (deepgram, screenpipe-cloud, whisper-large, whisper-large-v3-turbo, whisper-large-v3-turbo-quantized, qwen3-asr, parakeet, parakeet-mlx, openai-compatible), vocabulary (array of {"word","replacement"}), prompt (Whisper topic context). Keep ranges ≤1h. Show old vs new.

---

7. Raw SQL — POST /raw_sql

curl -X POST "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/raw_sql" -H "Content-Type: application/json" \
  -d '{"query": "SELECT ... LIMIT 100"}'

Rules: every SELECT needs LIMIT · always filter by time · read-only. Never use frame counts for time estimates — frames are event-driven; use /activity-summary for screen time.

Timestamp caveat: DB timestamps are stored as RFC3339 strings — usually 2026-06-26T18:01:14.214586+00:00 (frames / audio_transcriptions / ui_events), though some tables (e.g. meetings.meeting_start, memories) use a Z suffix with milliseconds: 2026-06-26T18:01:14.214Z. Do not compare either form directly to SQLite datetime() strings like timestamp > datetime('now','-10 seconds'): the T vs space makes it a lexical string comparison and can include stale same-day rows. Use datetime(timestamp) > datetime('now','-10 seconds') (works for both forms), or for indexed string comparisons use an RFC3339-shaped cutoff: timestamp > strftime('%Y-%m-%dT%H:%M:%f+00:00','now','-10 seconds').

Table	Key Columns	Time Column
frames	app_name, window_name, browser_url, focused	timestamp
ocr_text	text, app_name, window_name	join via frame_id
elements	source, role, text, bounds_*	join via frame_id
audio_transcriptions	transcription, device, speaker_id, is_input_device	timestamp
audio_chunks	file_path	timestamp
speakers	name, metadata	—
ui_events	event_type, app_name, window_title, browser_url	timestamp
accessibility	app_name, window_name, text_content, browser_url	timestamp
meetings	meeting_app, title, attendees, detection_source	meeting_start
memories	content, source, tags, importance	created_at

-- Most used apps (last 24h)
SELECT app_name, COUNT(*) AS frames FROM frames
WHERE timestamp > strftime('%Y-%m-%dT%H:%M:%f+00:00','now','-24 hours') AND app_name IS NOT NULL
GROUP BY app_name ORDER BY frames DESC LIMIT 20;

-- Context switches per hour
SELECT strftime('%H:00', timestamp) AS hour, COUNT(*) AS switches
FROM ui_events WHERE event_type='app_switch' AND timestamp > strftime('%Y-%m-%dT%H:%M:%f+00:00','now','-24 hours')
GROUP BY hour ORDER BY hour LIMIT 24;

Patterns: GROUP BY date(timestamp) (daily), GROUP BY strftime('%H:00', timestamp) (hourly), HAVING frames > 5 (filter noise).

---

8. Connections — GET /connections

curl http://localhost:3030/connections            # list all integrations (40+)
curl http://localhost:3030/connections/telegram   # status + non-secret settings

Each entry's description is self-describing — for control surfaces (browsers, gateways, OAuth proxies) it includes the exact endpoint + body shape. Read it before guessing. If not connected, tell the user to set it up from the Connections page in the desktop app.

Connection reads return status and declared non-secret settings only. Stored secrets never appear in API responses. Use local boundaries:

• Telegram: POST /connections/telegram/send with {"text":"..."}
• n8n / Zapier / Make: POST /connections/<id>/proxy with arbitrary JSON
• Discord: POST /connections/discord/proxy with {"content":"..."}
• Teams webhook: POST /connections/teams/proxy with {"text":"..."}

API proxy integrations — credentials stay server-side. Call the local wildcard proxy; it injects auth and forwards upstream. There is no /connections/<id>/token endpoint.

# GitHub create issue (repo from pipe settings). Same shape for comments: .../issues/42/comments {"body":...}
curl -X POST http://localhost:3030/connections/github/proxy/repos/OWNER/REPO/issues \
  -H "Content-Type: application/json" -d '{"title":"Bug","body":"Steps..."}'

# Generic OAuth proxy (Zoom, Vercel, Google Docs, Microsoft 365, ...)
curl -X POST http://localhost:3030/connections/<id>/proxy/<upstream-api-path> \
  -H "Content-Type: application/json" -d '{...}'

Don't call https://api.github.com/... directly from a pipe — use the proxy.

Calendar — use calendar endpoints for appointments/upcoming events. If /connections shows ics-calendar.connected: true, include ICS results too before saying the calendar is empty:

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  "http://localhost:3030/connections/calendar/events?hours_back=0&hours_ahead=72"
# also: /connections/google-calendar/events , /connections/ics-calendar/events

Browser control (owned-default) — an embedded browser, shown in the chat. Cookies persist (isolated profile); password fields are stripped from snapshots. Try snapshot first; reach for eval only when needed.

# Navigate → {"ok":true,"url":"<final>"}
curl -X POST -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://en.wikipedia.org/wiki/Giraffe"}' \
  http://localhost:3030/connections/browsers/owned-default/navigate

# Snapshot (no JS) → {title, url, tree:"[h1] ...\n  [a] ... → /href", truncated}. Best for "what's on the page?".
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  http://localhost:3030/connections/browsers/owned-default/snapshot

# Eval (escape hatch) — arbitrary JS return value, for clicks / values the snapshot tree omits.
curl -X POST -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" -H "Content-Type: application/json" \
  -d '{"code":"return [...document.querySelectorAll(\".title>a\")].slice(0,5).map(a=>a.innerText)"}' \
  http://localhost:3030/connections/browsers/owned-default/eval

---

9. Meetings — GET /meetings, PUT /meetings/:id

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/meetings?start_time=1d%20ago&end_time=now&limit=10"
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/meetings/42"

# Partial update — omitted fields stay as-is. Read first and re-include existing `note` so user notes survive.
curl -X PUT http://localhost:3030/meetings/42 -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  -H "Content-Type: application/json" -d '{"title":"Q3 planning","note":"<existing>\n\n## Summary\n<summary>"}'

Detected from calendar, app detection, window titles, UI elements, multi-speaker audio. q is a case-insensitive substring over title/attendees/notes. Uses PUT, not PATCH. Fields: id, meeting_start, meeting_end (null if ongoing), meeting_app, title?, attendees?, note?, detection_source. Also queryable via raw SQL on the meetings table.

---

10. Speakers — POST /speakers/*

All POST with Content-Type: application/json unless noted:

• GET /speakers/search?name=John — search by name
• GET /speakers/unnamed?limit=20 — unnamed speakers (for labeling)
• GET /speakers/similar?speaker_id=29&limit=5 — similar by voice embedding
• /speakers/update {"id":29,"name":"Jordan"} — rename/metadata
• /speakers/reassign {"audio_chunk_id":456,"new_speaker_name":"Jordan","propagate_similar":true} — returns new_speaker_id, transcriptions_updated, old_assignments (for undo)
• /speakers/undo-reassign {"old_assignments":[{"transcription_id":1,"old_speaker_id":29}]}
• /speakers/merge {"speaker_to_keep_id":5,"speaker_to_merge_id":29}
• /speakers/hallucination {"speaker_id":29} — mark false detection
• /speakers/delete {"id":29} — also removes audio chunk files

"That was actually Jordan, not Karishma": find the audio result's chunk_id → POST /speakers/reassign with audio_chunk_id + new_speaker_name; propagate_similar:true (default) also fixes similar chunks.

---

11. Parsed app data and actors

Semantic parsing is optional and disabled by default. When enabled, parser actor labels are heuristic observations. The API exposes a separate durable identity that a user or Pipe can correct without overwriting source evidence.

• GET /semantic/actors/search?q=Alice&limit=20 — canonical and observed names
• GET /search?content_type=parsed&actor_id=12&limit=20 — parsed app data assigned to an actor
• POST /semantic/actors/create {"name":"Alice Smith"} — create a separate identity
• POST /semantic/actors/update {"id":12,"name":"Alice Smith"} — rename
• POST /semantic/actors/merge {"actor_to_keep_id":12,"actor_to_merge_id":31} — merge current and future aliases
• POST /semantic/actors/reassign {"item_id":902,"actor_id":12} — correct one semantic item
• POST /semantic/actors/aliases/reassign {"alias_id":44,"actor_id":12} — move one alias, its heuristic history, and future observations

Each Parsed search result includes compact corrected text plus typed items and a parallel actors array. items[*].actor is always the original parser label; actors contains item_id, canonical actor_id/name, observed name, and assignment source. Use actor IDs for edits; never merge by display name alone. Prefer moving a specific alias when a full actor merge would be too broad; explicit item corrections are preserved.

---

12. Memories — High-Signal Persistent Knowledge

Memories are the highest-signal source — curated facts, preferences, decisions, project context distilled from hours of data. If you're calling /search, also query /memories: search gives you what happened, memories give you what matters and why. Query memories first when answering about preferences/decisions/past context, building background on a project/person/workflow, or generating any summary/recommendation/plan.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/memories?q=preference&limit=20"          # FTS search
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/memories?min_importance=0.5&limit=20"    # recent, high importance
curl -X POST http://localhost:3030/memories -H "Content-Type: application/json" \
  -d '{"content":"User prefers dark mode","source":"user","tags":["preference","ui"],"importance":0.7}'                   # create
curl -X PUT http://localhost:3030/memories/1 -H "Content-Type: application/json" -d '{"content":"...","importance":0.8}' # update
curl -X DELETE http://localhost:3030/memories/1                                                                          # delete

GET /memories params: q, source, tags, min_importance, start_time, end_time, limit, offset. Memories also come via GET /search?content_type=memory (NOT included in content_type=all — ask explicitly), which adds tags + include_related. When you learn a genuinely useful long-lived fact, store it with importance 0.0-1.0 — not transient observations.

---

13. Notifications — POST http://localhost:11435/notify

Notify the desktop UI. This is the Tauri sidecar (port 11435), not the main API. body supports markdown (**bold**, `code`, [text](url)).

priority is high, normal (default), or low. Every priority appears in the top-right notification panel. Only use high for a time-sensitive failure or a decision needing the human now; it also enters the focused Priority view. Normal stays available in All, while low is toast-only by default.

curl -X POST http://localhost:11435/notify -H "Content-Type: application/json" \
  -d '{"title":"3 new voice memos","body":"found recordings from today"}'

# Markdown body + action buttons. action types: "link" (web), "deeplink" (screenpipe://), "dismiss".
curl -X POST http://localhost:11435/notify -H "Content-Type: application/json" \
  -d '{"title":"Meeting summary","body":"**Q3 Planning** saved\n\nopen [notes](~/Documents/q3.md)","actions":[{"id":"view","label":"view","type":"deeplink","url":"screenpipe://timeline"},{"id":"skip","label":"skip","type":"dismiss"}]}'

# Ask permission, then run a pipe on approval — the opt-in flow. `type:"pipe"`
# runs the TARGET pipe when clicked; `context` is injected into that pipe's
# prompt. Set `pipe` explicitly (omit it and it falls back to the sender = no-op).
# Actions persist to the notification bell, so the user can approve later even
# if the toast already faded. Use `open_in_chat:true` to surface the run live.
curl -X POST http://localhost:11435/notify -H "Content-Type: application/json" \
  -d '{"title":"share meeting notes with the team?","body":"approve to send the adriaan call notes","priority":"high","actions":[{"id":"approve","label":"approve","type":"pipe","primary":true,"pipe":"share-data","context":{"meeting_id":274}},{"id":"no","label":"decline","type":"dismiss"}]}'

# No installed pipe? Use `type:"chat"` to run an inline prompt in a fresh chat
# session — write the whole task in `prompt`, attach data in `context`.
curl -X POST http://localhost:11435/notify -H "Content-Type: application/json" \
  -d '{"title":"summarize this call into a CRM note?","body":"approve to draft it","priority":"high","actions":[{"id":"go","label":"draft it","type":"chat","primary":true,"prompt":"summarize meeting 274 into a short CRM follow-up note and save it to output/","context":{"meeting_id":274}},{"id":"no","label":"no","type":"dismiss"}]}'

Action types: link (web URL), deeplink (screenpipe://), pipe (run an installed pipe — needs pipe, optional context, optional open_in_chat), chat (run an inline prompt in a fresh chat session, no installed pipe needed — optional context, optional auto_send), api (POST a local endpoint — needs url, optional method/body), dismiss. Fields: title* , body* (markdown), type (default "pipe"), priority (high/normal/low, default normal), timeout/autoDismissMs (ms, default 20000), actions (buttons; up to 5, each needs id/label/type). Body links: web URL → browser, file path (~/notes.md, /var/log/app.log) → default app, screenpipe://... → in-app. Returns {"success":true}.

---

14. AI Feedback — GET /feedback

Read local human ratings and comments before regenerating recurring AI output. One target contract covers notifications, chats, memories, blocks, artifacts, and exact-version structured outputs. Pipe-scoped tokens only receive records attributed to that Pipe.

curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/feedback?limit=20"

# Optional filters
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" \
  "${SCREENPIPE_LOCAL_API_URL:-http://localhost:3030}/feedback?kind=notification&producer=pipe:day-recap&rating=down&q=project&limit=20"

Each record includes target: { kind, id, version? }, rating, optional comment, the bounded local snapshot that was rated, producer attribution, context, and timestamps. Preserve patterns that earned up; directly address down comments. Do not treat a rating as permission for an unrelated external action.

---

15. Other Endpoints

curl http://localhost:3030/health        # health check
curl http://localhost:3030/audio/list    # audio devices
curl http://localhost:3030/vision/list   # monitors

Deep Links & Videos

Reference real moments with clickable links (only IDs/timestamps from actual results — never fabricate):

• [10:30 AM — Chrome](screenpipe://frame/12345) — screen results (use frame_id)
• [meeting at 3pm](screenpipe://timeline?timestamp=ISO8601) — audio results (use timestamp)

Show a search result's file_path as inline code to make it a playable video: `/Users/name/.screenpipe/data/monitor_1_..._10-30-00.mp4`.