authoring-scouts

# Authoring Signals scouts A **scout** is a scheduled agent that wakes on its own interval, looks at one PostHog project, decides what's genuinely worth surfacing, and writes it into the Signals inbox as a **report** — or closes out empty, which is a real outcome. PostHog ships a fleet of **canonical scouts** (a cross-product generalist plus per-surface specialists). This skill helps you and your agent **adapt those canonical scouts to a specific project**, or **author new scouts from scratch** for a use case the fleet doesn't cover. A scout's output is the **report channel**: it lists `emit_report` / `edit_report` in its frontmatter `allowed_tools` and authors or edits full inbox reports 1:1 directly. The canonical fleet runs this way, and **every new scout should too** — always include the `allowed_tools` opt-in when authoring one. Where that output _lands_ is a separate, per-scout config decision: the report goes to the Signals inbox, and the same report can be delivered to a Slack channel or DM at the same time (`output_destinations` under Run posture) — so don't rule a scout out of a job because the user wants the result in Slack. (A historical signal-emitting channel — weak `emit-signal` findings a pipeline consolidated — still exists in the harness for scouts that never opted in, but it is deprecated: don't author new scouts on it, and opt an old one in rather than extending it.) A scout is an `LLMSkill` that holds a `SignalScoutConfig`. The harness loads the body **verbatim** as the agent's system prompt, and progressively reads any bundled reference files on demand. **The config row is what makes a skill a scout.** Any valid skill name works, so the `signals-scout-` prefix is optional. The prefix controls one thing: the coordinator globs `signals-scout-*` to auto-register a config for a skill that has none. A skill with any other name needs its config created alongside it, which is what `scout-create` does. ## The job before the writing Don't write a scout in the abstract. Ground it in the target project first — a scout is only as good as its fit to the data it watches. (The scout tools were recently renamed from `signals-scout-*` to `scout-*`; if a `scout-*` name comes back unknown, the server may still expose it under the legacy `signals-scout-*` name — search the tool catalog and call whichever name it returns.) 1. **Read the project.** `posthog:scout-project-profile-get` returns the deterministic snapshot the scout itself cold-starts from: products in use, top events with reach/burst metrics, integrations, existing inbox counts. If the scout watches a specific event, confirm it exists and check its shape with `posthog:read-data-schema`. A scout for an event the project doesn't capture is dead on arrival. 2. **See what already runs.** `posthog:scout-config-list` lists every existing scout on the project with its schedule, `enabled`, and `emit` posture, plus each scout's `description` (pulled from the skill's frontmatter) so you can tell what a scout watches without loading its body. Don't duplicate a surface a canonical scout already covers — adapt that one instead. 3. **Read the closest canonical scout.** It's your template and your reference shape. Pull it with `posthog:skill-get {"skill_name": "signals-scout-<x>"}` (per-team rows) or read it from the repo at `products/signals/skills/signals-scout-*/`. The generalist (`signals-scout-general`) is the broad template; if your scope is domain-tight, pick the specialist closest to your surface — list the live roster with `posthog:skill-list {"search": "signals-scout"}` (specialists exist for most product surfaces: error tracking, logs, AI observability, experiments, feature flags, session replay, web analytics, surveys, and more). 4. **Skim the inbox.** `posthog:inbox-reports-list` shows what reports are actually landing — calibrate so your scout adds signal, not noise. ## Choose the path There are two independent decisions: **what** you're building, and **where** it lives. ### What | Situation | Approach | | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | A canonical scout is close but too broad / too noisy / missing a disqualifier for this project | **Adapt** it — narrow the scope, add disqualifiers, retune thresholds. | | You want a surface no canonical scout covers (a custom event, a product-specific funnel) | **New scout from scratch** — copy the closest canonical scout as scaffolding, replace the domain discriminator + explore patterns. | | You only want to change _when_ / _whether_ a scout runs | **No authoring** — just tune the config (see Run posture). | | You have one-off feedback, a pointer, or short-lived context for a scout | **No authoring** — leave a note (see Steering with notes). | ### Where | Path | Mechanism | Use when | | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | **Per-team** (the common user path) | Create a new runnable scout via `posthog:scout-create`; edit its prompt or files later via `posthog:skill-update` / `-file-create`, and tune its runtime config via `posthog:scout-config-update`. | Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. | | **Canonical** (PostHog contributors) | Edit disk under `products/signals/skills/signals-scout-*/`, lint/build, open a PR. | Improving a scout for _every_ enrolled project. `lazy_seed` mirrors it onto all enrolled teams on the next tick. | **Adapting-in-place tradeoff:** editing a canonical scout's row for your team marks it **diverged** — you stop receiving upstream improvements to that scout. If you only need an _additional_ behavior, prefer authoring a **new, differently-named** scout (`signals-scout-<your-scope>`) and leaving the canonical one intact. See [`references/lifecycle-and-testing.md`](references/lifecycle-and-testing.md) for the exact skills-store calls, the build/lint commands, and how seeding works. ## Write the scout First pick the **shape**. [`references/scout-patterns.md`](references/scout-patterns.md) is a cookbook of the reference architectures scouts fall into — anomaly watcher, liveness/absence watcher, zero-result/unmet demand, watchlist explore/exploit, cross-product correlation, recommendation/gap, warehouse-backed source, custom single-event, open-text theme, adversarial/abuse concentration, external-tool/code, state∩code intersection, custom issue-tracker/work-queue, daily digest/roll-up, triage over a pre-detected stream, first-person dogfooding/probe, recurring measurement/LLM-judge, maintainer/steward, owner-scoped book/queue, trigger-to-brief enrichment, dispatcher/campaign, fleet meta-scout/reviewer — each mapped to a canonical scout or a proven custom shape you can copy as scaffolding. It also makes the key point that **a scout can watch any source PostHog ingests into the data warehouse, not just analytics events** (a Slack channel sync, a billing system, a CRM, a support inbox), plus external systems reachable from the sandbox, other agents' output, and the scout fleet itself. And where a built-in signals source already covers the surface (GitHub and Linear issues), the issue-tracker pattern says where that source stops and a scout starts paying for itself. A scout does not have to stop at describing a fix: the **maintainer / steward** pattern is how a scout holding `write_scopes` keeps a family of dashboards, alerts, warehouse views, or scanner prompts healthy itself, with a write ladder that says which changes it applies, which it verifies and reports, and which it hands to a human. Find the closest pattern, then write the body. Follow [`references/scout-anatomy.md`](references/scout-anatomy.md) — it has the frontmatter schema (including the `allowed_tools` report-channel opt-in every scout needs), the canonical body structure (quick close-out → orient → domain discriminator → explore patterns → save-memory → decide → disqualifiers → close-out), the lean-body rule, and copy-ready skeleton templates for both a specialist and the generalist. **Write the body feature-forward.** The body is a prompt a run reads in full, every run, for as long as the scout lives — so it states what the scout watches and how it decides, and nothing else. Four things do not belong in it: - **No rollout state.** A preview flag, an early-access gate, a "recently renamed" tool, a port that is happening one scout at a time. It is true this month and misleading next month, and a run cannot tell which. Write the behavior the scout should follow now; if a capability may be absent, say how the run tells, not when it is due. - **No backlog.** Work that is planned, a field that does not exist yet, a lane somebody means to add. A run cannot act on it, so it only spends tokens and invites the run to wait for something. - **No project-only facts.** A metric, dashboard, event, id, or reviewer that exists in one project. A canonical scout ships to every project, so a fact from one of them is wrong for almost every run. Text about an optional product says how the run tells whether the project uses it, and never assumes it does. - **No duplicated harness mechanics.** The report-channel contract, the run gates, and the close-out format come from the harness prompt, which every report-channel scout is given. A copy in the body drifts from the real one and the run then holds two versions of the same rule. Keep the body to your scout's own domain framing. The same four apply to a bundled reference: it is read at run time too, so it carries no more rollout state or backlog than the body. Two craft references the whole fleet reasons in terms of — a good scout's **Decide** and **memory** sections are built on them, so read them before writing those sections: - [`references/report-contract.md`](references/report-contract.md) — the report tools (`scout-emit-report` / `scout-edit-report`), the report bar (author 1:1 only for a finding you'd own end-to-end), `suggested_reviewers` routing, the dedup-via-`report_id` discipline (the channel isn't idempotent — reconcile against existing reports via the vanilla `inbox-reports-list` / `inbox-reports-retrieve` before authoring), and the accepted caveat that the pipeline may later rewrite an authored title/summary. This is how your scout decides _what clears the bar_ and _how to file it_. - [`references/dedupe-and-memory.md`](references/dedupe-and-memory.md) — the four-states classifier (net-new / material-update / already-covered / addressed-or-noise), the scratchpad key-prefix vocabulary, and the cross-project noise patterns. This is how your scout avoids re-filing and learns across runs. - [`references/report-checks.md`](references/report-checks.md) — the follow-up checks a scout attaches to a report so "did the fix hold?" is measured later instead of remembered. This is how your scout closes the loop on a finding after somebody acts on it. A report is backward-looking; a **check** is the opposite direction — an expectation plus a time to test it, written onto a report with `scout-report-check-create`. Give your scout a checks section when its findings are the kind whose fix shows up in data later. Three rules belong in the body, and the reference has the rest: - **List the report's existing checks first** (`scout-report-check-list`), every time. An open check for the same claim makes a second one noise, and a report holds at most five open checks. - **`metric_threshold` wherever one number settles the claim and an event or action series can carry it.** The coordinator measures it itself, with no scout run. - **`agent` when no single number settles it, or when the number lives outside events** — a log rate, a fix whose effect shows in which entities fire rather than how many, a claim that needs a stack trace read. The single most important design decision in any scout is its **signal-vs-noise discriminator** — the cheap profile-shape read that separates "worth investigating" from "baseline". For error tracking it's the `count` vs `distinct_users` ratio; for CSP it's reach over raw count. Your new scout needs its own. Name it explicitly near the top of the body so every run anchors on it. (The one exception: a **measurement scout** on the structured-output channel holds no bar — it applies a **rubric** to every sampled item, and the rubric takes the discriminator's slot as the design surface to name, dogfood, and calibrate. See the recurring measurement / LLM-judge pattern in `references/scout-patterns.md`.) A second design consideration applies to a **metric-shaped scout** — one that scores, ranks, or reports a named, reusable measure, whether a business measure (MRR, churn risk, usage revenue, activation) or operational telemetry it computes every run to monitor or report (cost per run, failure or error rates, latency, throughput). If the project has an approved metric for that measure (`metric-list` shows what exists), name it in the body and run it with `data-catalog-metric-run`, so the scout's number matches the one the team already reports. ## Run posture (config) A scout's schedule and emit behavior live on its `SignalScoutConfig`, separate from the skill body. For a **brand-new scout**, pass these settings in the nested `config` object of the `posthog:scout-create` call, including creating it disabled or in dry-run **before it ever runs**. The endpoint creates the skill and config atomically, always opts the scout into the report channel, and safely re-applies config fields when the same definition is retried. Otherwise the coordinator auto-registers an enabled config on the default every-24-hours schedule on its next tick (up to ~30 min). For an **existing scout**, tune with `posthog:scout-config-update` (find the `id` via `-config-list`): - `run_interval_minutes` — 30 to 43200. Default 1440 (every 24 hours). Slow a chatty or expensive scout by raising this. - `run_cron_schedule`: a five-field cron expression (`'30 9 * * *'`, `'0 9 * * 1-5'`) evaluated in the project's timezone; occurrences must be at least 30 minutes apart. When set it takes precedence over `run_interval_minutes`, so a "slow it down" edit on a cron scout has to change or clear the cron (set `null` to return to the rolling interval). A new or edited schedule anchors on the edit time and waits for its next slot rather than catching up on a past one. Reach for it when the run should land at a wall-clock time: a digest before standup, a weekday-only watch. - `enabled` — `false` pauses the scout entirely (coordinator skips it). - `emit` — defaults to **`true`**: the scout writes its reports straight to the inbox. The standard flow is to make a scout and let it write — seeing what actually lands is the fastest way to calibrate it. Set **`emit=false` (dry-run)** only when you want to be extra careful: the scout still runs and logs its reasoning but writes nothing to the inbox. Reach for dry-run on a scout you expect to be chatty, expensive, or high-stakes; for most scouts, just writing and watching the inbox is the better loop. - `network_access` — defaults to **`trusted`**: the scout's sandbox can only reach the platform's trusted-domain allowlist (PostHog, GitHub, common package registries), which covers the MCP loop and `gh` but blocks everything else. Set **`full`** for a scout whose skill needs to read arbitrary external sites, e.g. documentation, papers on arxiv.org, or a vendor status page. Applies from the scout's next run, and changes are activity-logged. - `auto_pause_exempt` — defaults to `false`. A scout whose reports nobody engages with (no open, rating, or action — the cloud web inbox records reads; other clients don't yet) is warned and then paused automatically (`pause_reason=ignored`) — every run costs a sandbox agent, so a scout producing output no human consumes shouldn't keep running forever. A scout that is merely quiet is only flagged (`pause_reason=no_output`, a warning that never advances to a pause), since a watch scout's silence can be its job. `-config-list` shows the warning as `status=pending_pause` and the pause as `status=paused_by_system`; setting `enabled=true` again resumes the scout with a fresh grace window before the sweep may judge it again. Set `auto_pause_exempt=true` up front for a watchdog scout whose whole job is to stay quiet, so it never even picks up the quiet flag. - `write_scopes` — defaults to `[]`: the scout reads the project and writes only what every scout writes (its findings, its memory, and notebooks).