auto-webhooks

name	auto-webhooks
description	Configure ADO service hooks and PR Review build validation policy for AI automation agents. Uses ADO REST API via az rest (no native CLI command for service hooks). Reads all config from .ai/automation/infra.json.
when_to_use	Use when configuring ADO service hooks and build validation policies for AI automation, when the user says 'set up webhooks', 'configure service hooks', 'connect ADO to Lambda', or 'wire up the automation triggers'.
argument-hint

You configure ADO service hooks and the PR Review build validation policy. ADO has no native az devops service-hooks CLI command — all REST calls use az rest (ADO PAT auth, audit-wrapped).

0. Prerequisites

Read .ai/automation/infra.json. Extract automationProfile.

source .ai/lib/audit.sh
export AUDIT_LOG_PREFIX=infra

Profile determines scope:

• Full hub: WI hooks (project-scoped) + PR Answer hook (repo-scoped) + PR Review build policy
• Consumer: PR Answer hook (repo-scoped) + PR Review build policy only. Consumers do NOT create WI hooks (hub owns those). Consumers DO need their own PR Answer hook and PR Review policy.

Extract:

• adoOrg — ADO org URL
• adoProject — ADO project name
• pipelines.pr-review.id — PR Review pipeline ID (for build validation policy)
• pipelines.pr-answer.id — PR Answer pipeline ID (for registering with hub)
• For full-hub only: webhooks.wi-userstory.url (BugFix uses the Azure-native webhooks.bugfix comment hook — see 2c — not a WI-Router URL)
• Check if webhooks.*.status is already configured — skip if so

For consumers: The PR Answer hook URL points to the hub's PR Router Lambda. Ask:

Hub's PR Router Lambda URL? The API Gateway URL from the hub project's infra.json (e.g., https://<id>.execute-api.us-east-1.amazonaws.com/prod/pr-answer).

Ask once for shared values needed for hooks:

Webhook username? (same as Lambda BASIC_USER set in hub's /auto-lambda-env)

Webhook password? (same as Lambda BASIC_PASS) — secret, not stored

Webhook secret? (same as Lambda WEBHOOK_SECRET) — secret, not stored

Ask for ADO project ID (required by service hook API):

ADO project ID? Find in ADO > Project Settings > Overview > Project ID (GUID format). You can also run: az devops project show --project "<adoProject>" --query id --output tsv

Get the repository ID for repo-scoped hooks:

REPO_ID=$(az repos show --repository "<repo-name>" --project "<adoProject>" --query id --output tsv)

Read scm.base-branch from .ai/config.yaml for branch filtering.

0b. List Existing Service Hooks

Before creating any hooks, list all existing subscriptions to avoid duplicates:

EXISTING_HOOKS=$(az rest --method GET \
  --uri "<adoOrg>/_apis/hooks/subscriptions?api-version=7.1" \
  --query "value[?consumerInputs.url != null].{id:id, event:eventType, url:consumerInputs.url, repoId:publisherInputs.repository}" \
  --output json)

For each hook to create, check if a subscription already exists with the same eventType AND the same url AND (for repo-scoped hooks) the same repository:

• If found: skip creation, report ⏭ <hook> already exists (ID: <id>) — skipping
• If not found: create the hook

Work-item hooks are project-scoped and should exist only ONCE per project (hub only). If already configured from another repo, skip.

PR Answer hooks are per-repo — each repo needs its own hook filtered to that repo + base branch. Without repo filtering, a single project-scoped hook fires on every PR comment across hundreds of unrelated repos, flooding the Lambda with noise. Migration: If an existing project-scoped PR Answer hook exists (no repository filter — from an older setup), warn the user and offer to delete it after creating the repo-scoped replacement.

PR Review build validation policy is per-repo — always check by repositoryId before creating.

1. User Story WI Hook (Work Item Updated — User Story) — HUB ONLY

Skip for consumer profile.

Routes all User Story workitem.updated events to the WI Router Lambda at /wi. The Lambda uses tag-based routing to determine which agent to invoke (DoD, QA, DevAgent, DOCAgent, Estimation). DoR is no longer here — it has its own Azure-native @kai-dor comment hook (§2d).

az_resource "ado/hooks/wi-userstory" \
  az rest --method POST \
    --uri "$(adoOrg)/_apis/hooks/subscriptions?api-version=7.1" \
    --headers "Content-Type=application/json" \
    --body "{
      \"publisherId\": \"tfs\",
      \"eventType\": \"workitem.updated\",
      \"resourceVersion\": \"1.0\",
      \"consumerId\": \"webHooks\",
      \"consumerActionId\": \"httpRequest\",
      \"publisherInputs\": {
        \"projectId\": \"<PROJECT_ID>\",
        \"workItemType\": \"User Story\",
        \"tag\": \"KAI-TRIGGER\"
      },
      \"consumerInputs\": {
        \"url\": \"<wi-userstory-url>\",
        \"basicAuthUsername\": \"<BASIC_USER>\",
        \"basicAuthPassword\": \"<BASIC_PASS>\",
        \"httpHeaders\": \"x-webhook-secret:<WEBHOOK_SECRET>\"
      }
    }" \
    --query 'id' --output tsv

Update infra.json:

• webhooks.wi-userstory.subscriptionId → returned ID
• webhooks.wi-userstory.status → "configured"

2. BugFix trigger — moved to a comment hook (see 2c), no longer Lambda-routed

There is no Bug workitem.updated → WI-Router hook anymore. The BugFix agent was migrated to the same Azure-native model as SimpleAgent: it is triggered by a Bug comment containing @kai-bugfix (section 2c), delivering to the BugFix pipeline's Incoming WebHook — no Lambda, no tag hook. bugfix is intentionally absent from wi-router.mjs's AGENTS array. Configure the hook in 2c instead.

2b. SimpleAgent Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda)

Skip for consumer profile. This is the only hook that does not route through a Lambda. SimpleAgent (ado-cli-simple.yml, /dx-simple) is triggered entirely inside ADO: a comment containing @kai-simple on a work item fires this Service Hook, which delivers to the SimpleAgent pipeline's Incoming WebHook service connection (declared under resources.webhooks in the pipeline). The same event drives both the first run and recovery — the pipeline's Phase 0 decides fresh-vs-resume.

Prerequisite — Incoming WebHook service connection. Names are fixed to match ado-cli-simple.yml (and ado-cli-simple-router.yml):

• Webhook Name: kai-simple (matches the webhook: alias in the pipeline)
• Service connection name: kai-simple-trigger-sc (matches connection: in the pipeline)
• Secret: optional HMAC secret reused by the Service Hook below (loop-safety / authenticity)

Because SimpleAgent is fully Azure-native (no Lambda / no API Gateway), this connection is the one piece of trigger infra and can be created via the ADO REST API — no manual UI step. Ask once:

Incoming WebHook secret? (optional — HMAC secret shared between the Service Hook and the Incoming WebHook connection. Leave blank for none.) — secret, not stored

First check whether the connection already exists (idempotent):

EXISTING_SC=$(az rest --method GET \
  --uri "<adoOrg>/<adoProject>/_apis/serviceendpoint/endpoints?endpointNames=kai-simple-trigger-sc&api-version=7.1" \
  --query "value[0].id" --output tsv 2>/dev/null)

• If non-empty: skip creation, reuse it. Report ⏭ kai-simple-trigger-sc already exists (ID: <id>) — skipping.
• If empty: create it via az_resource (audit-wrapped). Include "secret" in data only if a secret was provided:

az_resource "ado/service-connection/kai-simple-trigger-sc" \
  az rest --method POST \
    --uri "<adoOrg>/_apis/serviceendpoint/endpoints?api-version=7.1" \
    --headers "Content-Type=application/json" \
    --body "{
      \"name\": \"kai-simple-trigger-sc\",
      \"type\": \"incomingwebhook\",
      \"url\": \"<adoOrg>\",
      \"authorization\": { \"scheme\": \"None\", \"parameters\": {} },
      \"data\": { \"webhookName\": \"kai-simple\"<SECRET_FIELD> },
      \"serviceEndpointProjectReferences\": [{
        \"projectReference\": { \"id\": \"<PROJECT_ID>\", \"name\": \"<adoProject>\" },
        \"name\": \"kai-simple-trigger-sc\"
      }]
    }" \
    --query 'id' --output tsv

Where <SECRET_FIELD> is , \"secret\": \"<webhook-secret>\" when a secret was provided, or empty otherwise. <PROJECT_ID> is the GUID from step 0.

Fallback (UI): if the REST create is rejected (older ADO without the incomingwebhook endpoint type), create it manually: Project Settings → Service connections → New → Incoming WebHook, Webhook Name kai-simple, Service connection name kai-simple-trigger-sc.

The trigger token is config-driven — read it so the hook filter matches the skill (dx-simple.recovery.trigger-token, default @kai-simple):

TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-simple.recovery.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-simple}

Create the Service Hook on the "Work item commented on" event, filtered so the comment contains $TRIGGER_TOKEN, delivering to the kai-simple Incoming WebHook (the pipeline's webhook resource) — not an API Gateway / Lambda URL. In the ADO UI: Project Settings → Service Hooks → + → select the trigger that posts to the pipeline's Incoming WebHook, set event = Work item commented on, and set filter "Comment contains" = <TRIGGER_TOKEN>.

The filter string MUST equal dx-simple.recovery.trigger-token — the skill, the pipeline header comment, and this hook all read from that one source of truth. The bot never emits the literal token in its own comments, so its own replies cannot self-trigger the hook (loop-safety).

Which pipeline does the hook target?

• Single-platform projects (default): point the hook at the simple pipeline's kai-simple Incoming WebHook, exactly as described above. The one dx-simple pipeline applies the change directly.
• Multi-repo projects: point the hook at the hub (KAI-HUB router) Incoming WebHook (ado-cli-hub.yml) instead of simple directly. The hub parses the @kai-<agent> tag, runs dx-discover-repos, and queues each agent's worker once per resolved repo (reading the repos.json/agents.json registries). The per-repo workers keep their resources.webhooks listener (they are dual-mode), but the comment hook fires the hub, not the individual workers. A single hub hook (filter: comment contains @kai) covers all agents.

Update infra.json:

• webhooks.simple.connection → kai-simple-trigger-sc
• webhooks.simple.connectionId → service connection ID ($EXISTING_SC or the created endpoint's id)
• webhooks.simple.subscriptionId → returned Service Hook ID
• webhooks.simple.status → "configured"

2c. BugFix Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda)

Skip for consumer profile. Identical mechanism to 2b (SimpleAgent) — the BugFix agent (ado-cli-bug-fix.yml, /dx-bug-all) is also fully Azure-native: a comment containing @kai-bugfix on a Bug fires a Service Hook delivering to the BugFix pipeline's Incoming WebHook (resources.webhooks). The same event drives the first run and every recovery re-trigger; Phase 0 (resume-check.sh) decides fresh-vs-resume. BugFix is NOT in the WI-Router Lambda's AGENTS array — do not configure a workitem.updated/tag hook for it.

Prerequisite — Incoming WebHook service connection (names match ado-cli-bug-fix.yml):

• Webhook Name: bugfixHook (matches the webhook: alias)
• Service connection name: kai-bugfix-trigger-sc (matches connection:)

Create it exactly as in 2b (idempotent az rest GET → POST incomingwebhook endpoint), substituting kai-bugfix-trigger-sc / bugfixHook for the simple names. UI fallback is the same.

The trigger token is config-driven:

TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-bug-all.recovery.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-bugfix}

Create the Service Hook on "Work item commented on" with two filters — comment contains $TRIGGER_TOKEN AND Work Item Type = Bug (the pipeline's resources.webhooks.filters already enforces the Bug type, but filtering at the hook avoids firing the pipeline for non-Bug comments). Deliver to the bugfixHook Incoming WebHook — never a Lambda URL.

The filter string MUST equal dx-bug-all.recovery.trigger-token — skill, pipeline header, and hook share that one source of truth. The bot never emits the literal token, so its replies can't self-trigger (loop-safety).

Update infra.json:

• webhooks.bugfix.connection → kai-bugfix-trigger-sc
• webhooks.bugfix.connectionId → service connection ID
• webhooks.bugfix.subscriptionId → returned Service Hook ID
• webhooks.bugfix.status → "configured"

2d. DoR Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda)

Skip for consumer profile. Same mechanism as 2b/2c — the DoR agent (ado-cli-dor.yml, /dx-dor) gets a comment trigger: a comment containing @kai-dor on a User Story fires a Service Hook delivering to the DoR pipeline's Incoming WebHook (resources.webhooks.dorHook). The work item id comes from the payload (${{ parameters.dorHook.resource.id }}); manual runs pass workItemId and the pipeline falls back to the payload id.

DoR is fully comment-triggered now (no Lambda). Like SimpleAgent and BugFix, dor has been removed from the WI-Router Lambda's AGENTS array — the §1 "WI User Story" workitem.updated hook no longer routes to DoR (it still serves the remaining tag-routed agents: dod, qa, devagent, docagent, estimation). Configure only this comment hook for DoR; there is no tag/Lambda path to disable.

Prerequisite — Incoming WebHook service connection (names match ado-cli-dor.yml):

• Webhook Name: dorHook (matches the webhook: alias)
• Service connection name: kai-dor-trigger-sc (matches connection:)

Create it exactly as in 2b (idempotent az rest GET → POST incomingwebhook endpoint), substituting kai-dor-trigger-sc / dorHook for the simple names. UI fallback is the same.

The trigger token is config-driven:

TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-dor.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-dor}

Create the Service Hook on "Work item commented on" with two filters — comment contains $TRIGGER_TOKEN AND Work Item Type = User Story (the pipeline's resources.webhooks.filters already enforces the type, but filtering at the hook avoids firing the pipeline for non-Story comments). Deliver to the dorHook Incoming WebHook — never a Lambda URL.

The filter string MUST equal dx-dor.trigger-token — skill, pipeline header, and hook share that one source of truth. /dx-dor's own comment is prefixed [DoRAgent] and never contains the token, so its replies can't self-trigger (loop-safety).

Update infra.json:

• webhooks.dor.connection → kai-dor-trigger-sc
• webhooks.dor.connectionId → service connection ID
• webhooks.dor.subscriptionId → returned Service Hook ID
• webhooks.dor.status → "configured"

3. PR Answer Service Hook (PR Commented On) — ALL PROFILES

Routes to the PR Router Lambda. This hook is repo-scoped — each repo (hub and every consumer) creates its own hook filtered to that repo and base branch. Without repo filtering, a project-scoped hook fires on every PR comment across all repos in the ADO project.

The PR Router Lambda uses ADO_PR_ANSWER_PIPELINE_MAP to route each repo's events to the correct PR Answer pipeline.

az_resource "ado/hooks/pr-answer" \
  az rest --method POST \
    --uri "<adoOrg>/_apis/hooks/subscriptions?api-version=7.1" \
    --headers "Content-Type=application/json" \
    --body "{
      \"publisherId\": \"tfs\",
      \"eventType\": \"ms.vss-code.git-pullrequest-comment-event\",
      \"resourceVersion\": \"1.0\",
      \"consumerId\": \"webHooks\",
      \"consumerActionId\": \"httpRequest\",
      \"publisherInputs\": {
        \"projectId\": \"<PROJECT_ID>\",
        \"repository\": \"<REPO_ID>\",
        \"branch\": \"<BASE_BRANCH>\"
      },
      \"consumerInputs\": {
        \"url\": \"<pr-answer-url>\",
        \"basicAuthUsername\": \"<BASIC_USER>\",
        \"basicAuthPassword\": \"<BASIC_PASS>\",
        \"httpHeaders\": \"x-webhook-secret:<WEBHOOK_SECRET>\"
      }
    }" \
    --query 'id' --output tsv

• <pr-answer-url> — for hub: from webhooks.pr-answer.url in infra.json. For consumer: the hub's PR Router Lambda URL (asked in step 0).
• <REPO_ID> — this repo's ADO repository GUID (from step 0).
• <BASE_BRANCH> — from scm.base-branch in config.yaml (e.g., development).

Update infra.json:

• webhooks.pr-answer.subscriptionId → returned ID
• webhooks.pr-answer.status → "configured"

4. PR Review Build Validation Policy — ALL PROFILES

PR Review is not a service hook — it's a build validation policy on the target branch. This triggers the PR Review pipeline on every PR against that branch.

Use REPO_ID from step 0 and BASE_BRANCH from config.yaml (already read). No need to ask again — default to scm.base-branch. Only ask if not set in config:

Target branch for PR Review? (e.g. main, develop) — pull requests targeting this branch will trigger the PR Review agent.

REPO_ID="<repository-id>"
BRANCH="<base-branch>"
PR_REVIEW_PIPELINE_ID=$(python3 -c "import json; print(json.load(open('.ai/automation/infra.json'))['pipelines']['pr-review']['id'])")

az_resource "ado/policy/pr-review-build-validation" \
  az rest --method POST \
    --uri "<adoOrg>/<adoProject>/_apis/policy/configurations?api-version=7.1" \
    --headers "Content-Type=application/json" \
    --body "{
      \"isEnabled\": true,
      \"isBlocking\": false,
      \"type\": {\"id\": \"0609b952-1397-4640-95ec-e00a01b2f659\"},
      \"settings\": {
        \"buildDefinitionId\": $PR_REVIEW_PIPELINE_ID,
        \"queueOnSourceUpdateOnly\": true,
        \"manualQueueOnly\": false,
        \"displayName\": \"AI PR Review\",
        \"validDuration\": 720,
        \"scope\": [{
          \"repositoryId\": \"$REPO_ID\",
          \"refName\": \"refs/heads/$BRANCH\",
          \"matchKind\": \"Exact\"
        }]
      }
    }" \
    --query 'id' --output tsv

Update infra.json:

• webhooks.pr-review.policyId → returned ID
• webhooks.pr-review.status → "configured"

5. Summary Report

Adapt the report to the profile:

For full-hub:

## ADO Webhooks Configured (Hub)

| Hook | Event | Scope | URL | Status |
|------|-------|-------|-----|--------|
| WI User Story | workitem.updated | Project (tag: KAI-TRIGGER) | <wi-url> | ✓ configured |
| WI Bug | workitem.updated | Project (tag: KAI-TRIGGER) | <wi-url> | ✓ configured |
| SimpleAgent | workitem.commented (comment contains @kai-simple) | Project → pipeline Incoming WebHook (no Lambda) | kai-simple-trigger-sc | ✓ configured |
| BugFix | workitem.commented (comment contains @kai-bugfix, type Bug) | Project → pipeline Incoming WebHook (no Lambda) | kai-bugfix-trigger-sc | ✓ configured |
| DoR | workitem.commented (comment contains @kai-dor, type User Story) | Project → pipeline Incoming WebHook (no Lambda) | kai-dor-trigger-sc | ✓ configured |
| PR Answer | git.pullrequest.comment-event | Repo: <repo>, branch: <branch> | <pr-answer-url> | ✓ configured |
| PR Review policy | Build validation | Repo: <repo>, branch: <branch> | (pipeline trigger) | ✓ configured |

**Tag-based routing:** The WI Router Lambda uses work item tags to determine which agent pipeline to queue. Adding a new agent only requires Lambda env vars (TAG_GATE_* + pipeline ID) — no new hook or route needed.

**infra.json** updated with subscription IDs and status.
**Audit log:** `.ai/logs/infra.<week>.jsonl`

### Verify
Test the WI hook: Go to ADO > Project Settings > Service Hooks > find "WI User Story" > click "Test" to send a sample event.

### Next step
`/auto-alarms` — Set up CloudWatch monitoring

For consumer:

## ADO Webhooks Configured (Consumer)

| Hook | Event | Scope | URL | Status |
|------|-------|-------|-----|--------|
| PR Answer | git.pullrequest.comment-event | Repo: <repo>, branch: <branch> | <hub-pr-answer-url> | ✓ configured |
| PR Review policy | Build validation | Repo: <repo>, branch: <branch> | (pipeline trigger) | ✓ configured |

WI hooks are managed by the hub project — not created here.

**infra.json** updated with subscription IDs and status.
**Audit log:** `.ai/logs/infra.<week>.jsonl`

### Next step
`/auto-doctor` — Verify setup health

Success Criteria

• Service hooks created (or confirmed existing) for each enabled agent type
• No duplicate hooks — existing hooks preserved, not recreated
• PR Review build validation policy attached to the base branch
• Summary report lists each hook with status (created/exists/skipped)

Examples

1. /auto-webhooks (hub project) — Creates 2 WI hooks (User Story + Bug, tag-filtered to KAI-TRIGGER) in the work-item ADO project (from scm.wiki-project config), 1 PR Answer hook scoped to the repo + base branch, and 1 PR Review build validation policy. Updates infra.json with subscription IDs.

2. /auto-webhooks (consumer project) — Skips WI hooks (managed by hub). Creates 1 PR Answer hook scoped to this repo + base branch pointing to the hub's Lambda URL, and 1 PR Review build validation policy. Reminds user to register the PR Answer pipeline ID in the hub's ADO_PR_ANSWER_PIPELINE_MAP.

3. /auto-webhooks (re-run, hooks already exist) — Lists existing service hooks via az rest, detects that the PR Answer hook and PR Review policy already exist for this repo. Skips creation with "already configured" status. Reports all hooks as healthy.

Troubleshooting

• "Hook created but Lambda not receiving events" Cause: The hook was created in the wrong ADO project (hooks are project-scoped) or the API Gateway URL is incorrect. Fix: Verify the hook's ADO project matches where the repo lives (the code ADO project (from scm.project config) for code repos). Check the hook's URL points to the correct API Gateway endpoint from infra.json.

• Duplicate Lambda invocations for the same event Cause: Multiple hooks exist for the same event/repo combination (e.g., from a previous failed cleanup). Fix: List hooks with az rest and delete duplicates. The skill checks for existing hooks before creating, but orphaned hooks from manual creation won't be detected.

• PR Review build policy not triggering Cause: The build validation policy is scoped to a specific branch and repository. PRs targeting a different branch won't trigger it. Fix: Verify scm.base-branch in .ai/config.yaml matches the branch your PRs target. The policy is created for refs/heads/<base-branch> on the specific repository.

Rules

• Always source audit.sh first — wrap az rest calls with az_resource
• Never store passwords or secrets in infra.json — only subscription IDs and status
• Idempotent — check status: configured in infra.json before creating (skip if already done)
• Check remote before create — always list existing hooks and compare by eventType+URL+repository before creating. This catches hooks created from another repo that aren't in the local infra.json
• Work-item hooks are project-scoped (hub only) — 2 hooks (User Story + Bug) should exist only once per project. Consumer profile skips these entirely.
• PR Answer hook is per-repo — each repo (hub and every consumer) creates its own hook filtered to that repository + base branch. A project-scoped hook would fire on every PR comment across all repos in the ADO project, flooding the Lambda with noise from hundreds of unrelated repos.
• PR Review is per-repo — build validation policy scoped to repositoryId+refName. Check by repositoryId before creating
• Consumer profile runs this skill — consumers create their own PR Answer hook (pointing to hub's Lambda URL) and PR Review build policy. They do NOT create WI hooks.
• Never create duplicate hooks — duplicate hooks cause double Lambda invocations (deduplication catches it but wastes resources)
• Tag-based routing — all WI webhooks route to a single /wi endpoint. The WI Router Lambda scans work item tags against configured TAG_GATE_* env vars to determine which agent to invoke. No per-agent routes needed.
• Note about service hooks CLI: ADO has no az devops service-hooks subcommand — az rest is the correct approach
• PR Review is a build validation policy — not a service hook (different ADO API endpoint)