setup

name

setup

description

One-time interactive setup for this experiment-bud repository. Use this skill whenever the user wants to set up, install, configure, deploy, or onboard this CRO agent — e.g. 'run the setup skill', 'set this up for my site', 'configure experiment-bud', 'make this runnable', or right after cloning the repo. Walks through collecting the user's real integration values, replacing every placeholder, writing .env, seeding the owner mapping, verifying the Notion schema, and validating each integration.

experiment-bud Setup

This repo ships as a reference implementation: every infrastructure identifier is a placeholder, so it will not run as-is. Your job is to turn this clone into a working deployment for the user's own marketing site by collecting their real values, applying them everywhere, and verifying each integration actually works before declaring success.

Work from the repository root (the directory containing AGENTS.md and .env.example).

Ground rules

• Never invent, guess, or scaffold placeholder values. If the user doesn't have a value yet, leave the placeholder in place, record it in the final report as "not configured", and move on. A partially-configured repo with honest gaps is far more useful than one silently filled with fake IDs.
• Never commit .env or print secret values (tokens, keys) to the session output. .gitignore already excludes .env — verify that before writing it.
• Integrations are independent. The user may want to start with only Notion + BigQuery and add Slack or Google Ads later. Configure what they have; report what's missing.

Phase 0 — Preflight

1. If the repo is not cloned locally yet, clone it first and work from the clone:

   git clone https://github.com/warpdotdev/cro-agent-oss.git
   cd cro-agent-oss
   

   Ask the user where to put it if the destination is unclear, and never clone over an existing directory. If the user has a fork or private copy of the repo (recommended, so they can commit and push their configured version — a private copy avoids exposing internal identifiers, since forks of public repos are always public), clone that instead.
2. Confirm you are at the repo root and it is clean enough to edit: git status --short (warn only about uncommitted changes to files a search-and-replace will touch — untracked or unrelated files are fine).
3. Check python3 --version (3.10+ required by the stats scripts).
4. Verify the Oz CLI (needed for the Phase 6 Oz wiring — environments, schedules):

   oz whoami
   

   • Command not found → if the Warp app is already installed, the CLI ships with it. Otherwise, prefer the standalone Oz CLI — there is no need to install the full Warp app just for the CLI. See Installing the CLI; on macOS: brew tap warpdotdev/warp && brew install --cask oz.
   • Not authenticated → run oz login (interactive), or for CI/headless environments export WARP_API_KEY.
5. Read README.md ("How to deploy" section) and skim .warp/skills/cro/references/config.md so you know where values land.

Phase 1 — Collect values from the user

Ask for values grouped by integration, one group at a time, and tell the user they can answer "skip" for any integration they don't have yet. Collect:

GCP / BigQuery

• GCP project id (replaces your-gcp-project)
• Actual BigQuery table names for conversion and pageview data, fully qualified as project.dataset.table to avoid ambiguity (the SQL files use illustrative names like analytics.website_conversion, events.pages)
• Actual conversion-event column names (illustrative ones include did_download_same_day, is_excluded_page_type_1)
• Rudderstack source names (replace your-rudderstack-source, your-legacy-rudderstack-source) and property name (your-rudderstack-property)

Notion

• Workspace slug (replaces notion.so/your-workspace/)
• CRO Experiment Agent page id (NOTION_CRO_AGENT_PAGE_ID)
• Database/page ids: NOTION_PAGE_REGISTRY_DB_ID, NOTION_ROADMAP_DB_ID, NOTION_KNOWLEDGE_BASE_DB_ID, NOTION_ANALYSIS_LOG_DB_ID, NOTION_REPORTS_PAGE_ID, and optionally NOTION_TEAM_PRINCIPLES_PAGE_ID, NOTION_TEAM_BACKLOG_PAGE_ID
• If the user hasn't created these databases yet, jump to Phase 5 first, then return here.

Slack

• Channel ids and names: SLACK_NOTIFICATIONS_CHANNEL_ID / #your-notifications-channel, SLACK_RECAP_CHANNEL_ID / #your-recap-channel, SLACK_TEST_CHANNEL_ID / #your-test-channel
• Bot identifiers: SLACK_BOT_USER_ID, SLACK_APP_ID, SLACK_BOT_ID, @your-bot

Google Ads

• Customer id (replaces ADS_CUSTOMER_ID — also used as a suffix on Google Ads BigQuery table names)

Site + repos

• Marketing-site repository (replaces your-org/your-marketing-site)
• Sanity project id (your-sanity-project)
• Production site host (www.example.com)

Hosts + Oz

• Metabase base URL (metabase.example.com)
• Webhook host (your-webhook-host.example.com) and Oz host (your-oz-host.example.com)
• Oz ids, if already created: CRO_SILENT_ENV_ID, CRO_MAIN_ENV_ID, SCHEDULE_ID_* (these usually don't exist yet — Phase 6 creates them; you can circle back)

Team

• Owner label → Slack user id → GitHub username mappings for references/owners.py (or note that seed_owners.py --apply can generate them later from live Notion + Slack)

Phase 2 — Write .env

1. cp .env.example .env (do not overwrite an existing .env without asking).
2. Have the user fill in the secret values themselves, or accept them via the environment — do not echo secrets back. The variables are documented in .env.example.
3. If the user deploys via Oz, remind them these same values must also be configured as Oz team secrets: the Slack token (EXPERIMENT_BUD_SLACK_BOT_TOKEN) belongs only in the main environment, never the silent one (see Phase 6).

Phase 3 — Replace placeholders

For each value collected in Phase 1, find every occurrence and replace it:

grep -rl 'PLACEHOLDER' . --exclude-dir=.git --exclude-dir=setup --exclude='.env*'

(Note: grep's --exclude-dir matches a directory name, not a path — setup is how you exclude .warp/skills/setup.)

Notes:

• Exclude this setup skill's own directory from search-and-replace and from verification greps — it necessarily names every placeholder.
• Leave .env.example pristine (it's the template; real values go in .env) and leave the placeholder inventory in README.md's manual "Replace placeholders" list as-is — it documents the placeholders for future readers and should not be rewritten with the user's values.
• Placeholders appear in AGENTS.md, .warp/skills/cro/README.md, the per-skill SKILL.md files, references/config.md, references/config-slack.md, and the SQL/Python files under */scripts/.
• BigQuery table and column names in the SQL files are illustrative, not exact-match placeholders. Open each SQL file (research/scripts/research_queries.sql, experiment_monitor/scripts/monitoring_queries.sql) and adapt the queries to the user's actual schema — this is editing, not string substitution. Confirm each rewritten query's intent with the user if their schema differs structurally. For illustrative datasets the user did not map (e.g. Google Ads or Search Console tables), replace only the project id, leave the rest, and list them as unmapped in the final report.
• Keep the permanent Rudderstack unified-history filter (telemetry_source IN (...)) intact — just swap in the user's source names.

Phase 4 — Owner mapping

Edit .warp/skills/cro/references/owners.py: replace the placeholder OWNERS and GITHUB_USERS dicts with the user's real mappings (keys must exactly match the Notion Page Registry Owner select labels). If Notion and Slack are already configured, offer to run the self-driving refresh instead:

python3 .warp/skills/cro/references/seed_owners.py --apply

Sanity-check the result:

python3 .warp/skills/cro/references/owners.py "SomeRealOwnerLabel"

Phase 5 — Notion schema

The skills expect this structure (all databases as children of the CRO Experiment Agent page):

• Page Registry — one row per marketing-site page: Page Path (title), Owner (select), Primary Metric, traffic/CVR fields, Research Status (select)
• Roadmap & Backlog — experiment hypotheses: Page (select), Status (select with labels 0-archived, 1-proposed, 2-prioritized, 3-building, 4-ready for review, 5-live:validating, 6-live:running, 7-analyzing, 8-no-ship, 8-inconclusive, 9-shipped), Slack Thread TS (text)
• Knowledge Base — per-page insights
• Analysis Log — research and experiment analysis records
• Reports page — parent page for analysis report subpages

The authoritative field lists are in references/config.md and the per-skill SKILL.md files — check them rather than trusting this summary if anything conflicts. If the user has a Notion MCP server or an integration token, offer to create/verify the databases for them; otherwise give them the checklist and wait. The Notion integration must be shared with the CRO Experiment Agent page (Connections → add the integration), or every API call will 404.

Phase 6 — Warp / Oz wiring

This part happens in the Oz UI, not the repo, so guide the user through it and then write the resulting ids back into the repo (they appear in .warp/skills/cro/README.md):

1. Two environments (Slack credential isolation is a deliberate guardrail):
   • cro-silent (CRO_SILENT_ENV_ID) — all secrets EXCEPT EXPERIMENT_BUD_SLACK_BOT_TOKEN; runs page_registry_sync, research, owner_sync
   • main / experiment-bud (CRO_MAIN_ENV_ID) — all secrets including the Slack token; runs backlog_monitor, build, experiment_monitor, analyze, weekly_recap
2. Three schedules (SCHEDULE_ID_*):
   • cro-page-registry-sync — 0 6 * * *
   • cro-monitor — 0 13,19 * * 1-5 (backlog_monitor + experiment_monitor passes)
   • cro-weekly-recap — 0 14 * * 5
3. Four Notion webhooks pointing at the user's webhook host:
   • Page Registry: Research Status change → notion-cro-research
   • Page Registry: Owner change → notion-cro-owner-sync
   • Roadmap & Backlog: Status → 3-building → notion-cro-build
   • Roadmap & Backlog: Status → 7-analyzing → notion-cro-analyze

Phase 7 — Verify

Run every check that the user's configured integrations allow. Do not skip this phase — it's the difference between "configured" and "works".

1. No stray placeholders:

   grep -rn -e 'your-gcp-project' -e 'your-org/' -e 'your-workspace' \
     -e 'your-rudderstack' -e 'your-sanity-project' -e 'example\.com' \
     -e 'NOTION_[A-Z_]*_ID' -e 'SLACK_[A-Z_]*_ID' -e 'SCHEDULE_ID_' \
     -e 'CRO_SILENT_ENV_ID' -e 'CRO_MAIN_ENV_ID' -e 'ADS_CUSTOMER_ID' \
     -e 'your-bot' -e 'your-.*-channel' \
     . --exclude-dir=.git --exclude-dir=setup --exclude='.env*' --exclude=README.md
   

   Triage the hits rather than expecting zero. Expected hits that are NOT problems: placeholders the user chose to skip (list them, don't hide them); runtime identifiers that merely resemble placeholders, notably NOTION_PAGE_ID (a webhook payload variable in the skill files) and $GOOGLE_ADS_CUSTOMER_ID (an env var reference). Anything else is an unfinished replacement — fix it.
2. Notion: curl https://api.notion.com/v1/users/me with $NOTION_INTEGRATION_TOKEN, then retrieve each configured database id to confirm the integration can see it.
3. BigQuery: run the auth + SELECT 1 snippet from references/config.md → Data Access (mind the token-hygiene notes there), or the Metabase fallback if bq auth isn't available.
4. Slack (only if configured): auth.test with $EXPERIMENT_BUD_SLACK_BOT_TOKEN, and optionally post a test message to the test channel — never to the production channels.
5. Owners: the owners.py CLI check from Phase 4.
6. .env safety: git status --short shows no .env, and git check-ignore .env succeeds.

Final report

Finish with a short summary for the user:

• Integrations configured and verified (with the check that proved each one)
• Integrations skipped, and exactly what's needed to finish them later
• Oz wiring still to be done manually (environments, schedules, webhooks), if any
• A reminder not to commit .env, and that seed_owners.py --apply refreshes the owner mapping as the team changes