smoke-test

# Smoke Tests — Registry Server Starts the Registry Server stack with `docker compose`, waits for it to be ready, then executes a suite of `curl` smoke tests organised as Gherkin scenarios. Pass `keep-up` as an argument to leave the stack running after the tests complete (useful for interactive exploration). ## Embedded Scenarios ```gherkin Feature: Registry Server smoke tests Background: Given the Registry Server is running at http://localhost:8080 And the "default" registry is seeded from the upstream-registry.json file source # ── System endpoints ────────────────────────────────────────────────────── Scenario: Health check When I GET /health Then the response status is 200 And the body contains "healthy" Scenario: Readiness check When I GET /readiness Then the response status is 200 And the body contains "ready" Scenario: Version endpoint When I GET /version Then the response status is 200 And the body contains "version" Scenario: OpenAPI spec When I GET /openapi.json Then the response status is 200 And the body contains "openapi" # ── MCP Registry v0.1 — read-only ───────────────────────────────────────── Scenario: List servers in default registry When I GET /registry/default/v0.1/servers Then the response status is 200 And the body contains "servers" Scenario: List servers with search filter When I GET /registry/default/v0.1/servers?search=mysql Then the response status is 200 And the body contains "servers" Scenario: List servers with limit When I GET /registry/default/v0.1/servers?limit=2 Then the response status is 200 And the count of servers is at most 2 Scenario: List servers filtered to latest version only When I GET /registry/default/v0.1/servers?version=latest Then the response status is 200 And the body contains "servers" Scenario: List versions for a known server When I GET /registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions Then the response status is 200 And the body contains "servers" Scenario: Get a specific server version (latest) When I GET /registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions/latest Then the response status is 200 And the body contains "io.github.stacklok/adb-mysql-mcp-server" Scenario: Unknown registry returns 404 When I GET /registry/nonexistent-registry/v0.1/servers Then the response status is 404 Scenario: Unknown server version returns 404 When I GET /registry/default/v0.1/servers/com.example%2Fdoes-not-exist/versions/1.0.0 Then the response status is 404 # ── Admin API v1 — registries ───────────────────────────────────────────── Scenario: List registries When I GET /v1/registries Then the response status is 200 And the body contains "registries" Scenario: Get the default registry by name When I GET /v1/registries/default Then the response status is 200 And the body contains "default" Scenario: Get a nonexistent registry returns 404 When I GET /v1/registries/does-not-exist Then the response status is 404 Scenario: Create a new registry via PUT When I PUT /v1/registries/test-registry with an empty source list Then the response status is 201 And the body contains "test-registry" Scenario: Update an existing registry via PUT When I PUT /v1/registries/test-registry again with a description Then the response status is 200 Scenario: Delete an API-created registry When I DELETE /v1/registries/test-registry Then the response status is 204 Scenario: Delete a nonexistent registry returns 404 When I DELETE /v1/registries/does-not-exist Then the response status is 404 # ── Admin API v1 — sources ──────────────────────────────────────────────── Scenario: List sources When I GET /v1/sources Then the response status is 200 And the body contains "sources" Scenario: Get a known source by name When I GET /v1/sources/local-file Then the response status is 200 And the body contains "local-file" Scenario: Get a nonexistent source returns 404 When I GET /v1/sources/does-not-exist Then the response status is 404 Scenario: Create a managed source When I PUT /v1/sources/managed-test with body {"managed":{}} Then the response status is 201 And the body contains "managed-test" Scenario: Delete the managed source When I DELETE /v1/sources/managed-test Then the response status is 204 # ── Entry lifecycle — publish and delete ────────────────────────────────── Scenario: Publish a server version Given a managed source exists When I POST /v1/entries with a server payload Then the response status is 201 And the body contains the server name Scenario: Publish the same server version again returns 409 When I POST /v1/entries with the same server payload Then the response status is 409 Scenario: Publish a skill version When I POST /v1/entries with a skill payload Then the response status is 201 And the body contains the skill name Scenario: Delete the published server version When I DELETE /v1/entries/server/com.example%2Ftest-server/versions/1.0.0 Then the response status is 204 Scenario: Delete a nonexistent entry returns 404 When I DELETE /v1/entries/server/com.example%2Fnope/versions/9.9.9 Then the response status is 404 Scenario: Publish with both server and skill in body returns 400 When I POST /v1/entries with both server and skill fields set Then the response status is 400 Scenario: Publish with neither server nor skill in body returns 400 When I POST /v1/entries with an empty payload Then the response status is 400 # ── Claims update on edit ────────────────────────────────────────────────── Scenario: Registry claims are updated on PUT When I PUT /v1/registries/claims-test with claims {"org":"acme"} Then the response status is 201 And the body contains "acme" When I PUT /v1/registries/claims-test with claims {"org":"contoso"} Then the response status is 200 And the body contains "contoso" When I GET /v1/registries/claims-test Then the body contains "contoso" And the body does not contain "acme" Scenario: Source claims are updated on PUT When I PUT /v1/sources/claims-test with file-data and claims {"org":"acme"} Then the response status is 201 When I GET /v1/sources/claims-test Then the body contains "acme" When I PUT /v1/sources/claims-test with file-data and claims {"org":"contoso"} Then the response status is 200 When I GET /v1/sources/claims-test Then the body contains "contoso" And the body does not contain "acme" Scenario: Entry claims are updated via PUT When I PUT /v1/entries/skill/test-skill/claims with {"claims":{"team":"eng"}} Then the response status is 204 # ── Managed source limit ────────────────────────────────────────────────── Scenario: Second managed source is rejected with 409 Given the managed-test source still exists When I PUT /v1/sources/second-managed with body {"managed":{}} Then the response status is 409 And the body contains "at most one managed source is allowed" # ── List completeness ───────────────────────────────────────────────────── Scenario: All created sources appear in list Given three file-data sources are created: smoke-src-a, smoke-src-b, smoke-src-c When I GET /v1/sources Then the response status is 200 And the body contains "smoke-src-a" And the body contains "smoke-src-b" And the body contains "smoke-src-c" Scenario: All created registries appear in list Given three registries are created referencing the file-data sources When I GET /v1/registries Then the response status is 200 And the body contains "smoke-reg-a" And the body contains "smoke-reg-b" And the body contains "smoke-reg-c" # ── OAuth / auth enforcement ─────────────────────────────────────────────── # The stack is restarted with auth.mode: oauth before these scenarios run. # No real OIDC provider is needed: "missing token" and "malformed token" # cases are rejected before JWKS is consulted. Scenario: Public paths are accessible without a token in OAuth mode Given the server is restarted in OAuth mode When I GET /health Then the response status is 200 When I GET /readiness Then the response status is 200 When I GET /version Then the response status is 200 When I GET /openapi.json Then the response status is 200 Scenario: OAuth protected-resource metadata is publicly accessible When I GET /.well-known/oauth-protected-resource Then the response status is 200 And the body contains "authorization_servers" Scenario: MCP list-servers requires a token in OAuth mode When I GET /registry/default/v0.1/servers without an Authorization header Then the response status is 401 And the response includes a WWW-Authenticate header with Bearer scheme And the WWW-Authenticate header contains resource_metadata Scenario: Admin registries endpoint requires a token When I GET /v1/registries without an Authorization header Then the response status is 401 Scenario: Admin sources endpoint requires a token When I GET /v1/sources without an Authorization header Then the response status is 401 Scenario: Malformed Bearer token returns 401 invalid_token When I GET /registry/default/v0.1/servers with Authorization: Bearer not-a-real-jwt Then the response status is 401 And the WWW-Authenticate header contains error="invalid_token" ``` --- ## Steps ### 1. Start the stack ```bash BASE_URL="http://localhost:8080" PROJECT="thv-smoke-test" COMPOSE_FILE="docker-compose.smoke-test.yaml" PASS=0 FAIL=0 # Build the image only when it does not already exist locally. # Separating build from up avoids a Docker Hub metadata fetch that can hang # when the base image is pinned by digest (the default for this project). if ! docker image inspect thv-smoke-test-registry-api:latest > /dev/null 2>&1; then echo "=== Building Registry Server image ===" # Use the same project name so the image is tagged thv-smoke-test-registry-api:latest, # matching what docker-compose.smoke-test.yaml expects. docker compose --project-name "$PROJECT" build 2>&1 || { echo "ERROR: docker compose build failed" exit 1 } fi echo "=== Starting Registry Server stack ===" docker compose --project-name "$PROJECT" -f "$COMPOSE_FILE" up --detach --wait 2>&1 || { echo "ERROR: docker compose failed to start" docker compose --project-name "$PROJECT" -f "$COMPOSE_FILE" logs exit 1 } echo "Stack is up." ``` ### 2. Wait for readiness ```bash # docker compose --wait already blocks until all healthchecks pass, so this # step is a lightweight confirmation rather than an active poll. BASE_URL="http://localhost:8080" echo "=== Confirming /readiness ===" curl -sf "$BASE_URL/readiness" > /dev/null 2>&1 && echo "Server is ready." || { echo "ERROR: /readiness did not respond" docker compose --project-name thv-smoke-test -f docker-compose.smoke-test.yaml logs registry-api exit 1 } ``` ### 3. Define test helper and run scenarios ```bash BASE_URL="http://localhost:8080" PASS=0 FAIL=0 # ─── helper ─────────────────────────────────────────────────────────────────── # check DESCRIPTION EXPECTED_STATUS ACTUAL_STATUS BODY [GREP_PATTERN] check() { local desc="$1" expected="$2" actual="$3" body="$4" pattern="${5:-}" local ok=true # Accept a range like "4xx" or "5xx" if echo "$expected" | grep -qE '^[45]xx$'; then local prefix="${expected:0:1}" if ! echo "$actual" | grep -qE "^${prefix}[0-9]{2}$"; then ok=false fi elif [ "$actual" != "$expected" ]; then ok=false fi if $ok && [ -n "$pattern" ]; then if ! echo "$body" | grep -q "$pattern"; then ok=false fi fi if $ok; then echo " ✓ $desc" PASS=$((PASS + 1)) else echo " ✗ $desc [expected HTTP $expected, got $actual]" if [ -n "$pattern" ] && ! echo "$body" | grep -q "$pattern"; then echo " (body did not contain: $pattern)" echo " body: $(echo "$body" | head -3)" fi FAIL=$((FAIL + 1)) fi } curl_get() { curl -s -o /tmp/thv_body -w "%{http_code}" "$BASE_URL$1"; } curl_post() { curl -s -o /tmp/thv_body -w "%{http_code}" -X POST -H "Content-Type: application/json" -d "$2" "$BASE_URL$1"; } curl_put() { curl -s -o /tmp/thv_body -w "%{http_code}" -X PUT -H "Content-Type: application/json" -d "$2" "$BASE_URL$1"; } curl_del() { curl -s -o /tmp/thv_body -w "%{http_code}" -X DELETE "$BASE_URL$1"; } body() { cat /tmp/thv_body; } # ─── System endpoints ───────────────────────────────────────────────────────── echo "" echo "── System endpoints ──" SC=$(curl_get /health); check "GET /health returns 200 with healthy" 200 "$SC" "$(body)" "healthy" SC=$(curl_get /readiness); check "GET /readiness returns 200 with ready" 200 "$SC" "$(body)" "ready" SC=$(curl_get /version); check "GET /version returns 200 with version field" 200 "$SC" "$(body)" "version" SC=$(curl_get /openapi.json); check "GET /openapi.json returns 200 with openapi field" 200 "$SC" "$(body)" "openapi" # ─── MCP Registry v0.1 ─────────────────────────────────────────────────────── echo "" echo "── MCP Registry v0.1 ──" SC=$(curl_get /registry/default/v0.1/servers) check "GET /registry/default/v0.1/servers returns 200" 200 "$SC" "$(body)" "servers" SC=$(curl_get "/registry/default/v0.1/servers?search=mysql") check "GET .../servers?search=mysql returns 200" 200 "$SC" "$(body)" "servers" SC=$(curl_get "/registry/default/v0.1/servers?limit=2") check "GET .../servers?limit=2 returns 200" 200 "$SC" "$(body)" "servers" SERVER_COUNT=$(body | grep -o '"count":[0-9]*' | grep -o '[0-9]*' | head -1) SERVER_COUNT="${SERVER_COUNT:-0}" if [ "$SERVER_COUNT" -le 2 ]; then echo " ✓ limit=2 returned at most 2 servers (count=$SERVER_COUNT)" PASS=$((PASS + 1)) else echo " ✗ limit=2 returned more than 2 servers (count=$SERVER_COUNT)" FAIL=$((FAIL + 1)) fi SC=$(curl_get "/registry/default/v0.1/servers?version=latest") check "GET .../servers?version=latest returns 200" 200 "$SC" "$(body)" "servers" SC=$(curl_get "/registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions") check "GET .../servers/{name}/versions returns 200" 200 "$SC" "$(body)" "servers" SC=$(curl_get "/registry/default/v0.1/servers/io.github.stacklok%2Fadb-mysql-mcp-server/versions/latest") check "GET .../servers/{name}/versions/latest returns 200 with name" 200 "$SC" "$(body)" "adb-mysql-mcp-server" SC=$(curl_get /registry/nonexistent-registry/v0.1/servers) check "GET /registry/nonexistent-registry/... returns 404" 404 "$SC" "$(body)" SC=$(curl_get "/registry/default/v0.1/servers/com.example%2Fdoes-not-exist/versions/1.0.0") check "GET unknown server version returns 404" 404 "$SC" "$(body)" # ─── Admin v1 — registries ──────────────────────────────────────────────────── echo "" echo "── Admin v1 — registries ──" SC=$(curl_get /v1/registries) check "GET /v1/registries returns 200" 200 "$SC" "$(body)" "registries" SC=$(curl_get /v1/registries/default) check "GET /v1/registries/default returns 200" 200 "$SC" "$(body)" "default" SC=$(curl_get /v1/registries/does-not-exist) check "GET /v1/registries/does-not-exist returns 404" 404 "$SC" "$(body)" SC=$(curl_put /v1/registries/test-registry '{"sources":["local-file"]}') check "PUT /v1/registries/test-registry (create) returns 201" 201 "$SC" "$(body)" "test-registry"