validating-deployments

name	validating-deployments
description	End-of-session validation for MI and OBO deployment paths. Use after code changes to verify deploy → test → teardown works end-to-end.

Validating Deployments

Run this at the end of any session that changes backend, frontend, infra, or auth code.

Prerequisites

• All unit tests pass (dotnet test + npm test)
• Frontend builds (npm run build)
• Bicep validates (az bicep build --file infra/main.bicep)
• Playwright installed (npx playwright install chromium)

Security Checks

Run before every deployment:

# Backend: check for vulnerable NuGet packages
cd backend && dotnet list package --vulnerable

# Frontend: check for vulnerable npm packages
cd frontend && npm audit

# Docker: lint Dockerfile for security issues (if hadolint installed)
hadolint deployment/docker/frontend.Dockerfile

The Dockerfile runs as non-root (USER app). Verify this hasn't been removed after changes.

MI Path (Default)

1. Deploy

azd env new mi-test --no-prompt
azd env set ENTRA_SERVICE_MANAGEMENT_REFERENCE "<guid-from-admin>"  # Required by some orgs
azd env set AZURE_LOCATION eastus2 --no-prompt
azd up --no-prompt

2. Test Remote

$endpoint = azd env get-value WEB_ENDPOINT
node deployment/scripts/smoke-test.js $endpoint

Verify: health 200, agent name displayed, chat streaming works, token usage visible.

3. Test Local Dev

cd backend/WebApp.Api
$env:ASPNETCORE_ENVIRONMENT = "Development"  # CRITICAL — without this, uses ManagedIdentityCredential which fails locally
$env:ASPNETCORE_URLS = "http://localhost:8080"
dotnet watch run --no-launch-profile &

cd ../../frontend
npm run dev &

# Wait for both servers, then:
node deployment/scripts/smoke-test.js http://localhost:5173

4. Teardown

azd down --force --purge --environment mi-test --no-prompt
# Verify:
# - frontend/.env.local deleted
# - backend/WebApp.Api/.env deleted
# - .azure/mi-test/ deleted
# - Entra app deleted (check azd logs)

OBO Path

1. Deploy

azd env new obo-test --no-prompt
azd env set ENABLE_OBO true --no-prompt
azd env set AZURE_LOCATION eastus2 --no-prompt
# No SMR needed if using a non-MSFT tenant
azd up --no-prompt

2. Verify OBO Wiring

# Check backend logs for OBO mode
az containerapp logs show --name <app-name> --resource-group <rg> --type console --tail 20 | Select-String "OBO"
# Expected: "OBO mode enabled: backendClientId=..."
# Expected: "Created OBO credential for request" (after first chat)

3. Test

Run smoke test. OBO requires interactive MSAL login (user must sign in with test tenant credentials). The smoke test will wait at auth if no cached session exists.

4. Teardown

Same as MI path. Additionally verify:

• Backend API app registration deleted (postdown.ps1 handles this)
• FIC removed with the app

Gotchas

Gotcha	Detail
ASPNETCORE_ENVIRONMENT=Development	Required for local dev — without it, backend tries ManagedIdentityCredential which fails on dev machines
SMR required	Set ENTRA_SERVICE_MANAGEMENT_REFERENCE or Entra app creation fails
Bicep FIC creation may fail	Graph API eventual consistency issue. Workaround: create FIC via az ad app federated-credential create
v2 agent API requires kind: "prompt"	When creating agents via REST, use definition: { kind: "prompt", model: "...", instructions: "..." }
OBO scope is api://{BACKEND}/Chat.ReadWrite	NOT api://{SPA}/Chat.ReadWrite. Token audience mismatch = AADSTS500131
AI scope resolves to Azure ML Services	https://ai.azure.com/.default → appId 18a66f5f-... (Azure Machine Learning Services), NOT 7d312290-... (Cognitive Services)
Conversation history not user-scoped in MI mode	MI uses shared identity — all users see all conversations

Related Skills

• testing-with-playwright — Playwright MCP patterns for interactive testing
• validating-ui-features — Step-by-step UI feature validation
• deploying-to-azure — azd commands and troubleshooting
• troubleshooting-authentication — MSAL/JWT debugging