| name | starter-execution |
| description | Orchestrates complete Azure AI Foundry deployment to Azure DevOps. Coordinates repository-setup, service-connection-setup, environment-setup, pipeline-setup, and deployment-validation skills. Use when deploying the complete Azure AI Foundry starter template end-to-end. |
| license | Apache-2.0 |
Azure AI Foundry Starter Execution Orchestrator
Orchestrates the complete deployment of the Azure AI Foundry starter template to Azure DevOps by coordinating five specialized skills.
Overview
End-to-end orchestration skill that coordinates five specialized skills to deploy the Azure AI Foundry starter template from template-app/ to Azure DevOps.
Prerequisites
Before using this orchestrator:
- ✅ configuration-management - Configuration must be set up FIRST
- ✅ resource-creation - Azure resources and Service Principal must exist
- ✅ environment-validation - Environment prerequisites validated
- ✅ Bearer token valid - 30+ minutes remaining for deployment
- ✅ Azure DevOps permissions - Contributor access to target project
What Gets Created
- 1 Repository:
azure-ai-foundry-app with template application code
- 3 Service Connections: dev/test/prod with Workload Identity Federation (no secrets!)
- 3 Variable Groups:
{projectName}-{env}-vars with environment-specific configuration (projectName from config)
- 3 Environments: dev, test, production with approval gates
- 3+ Pipelines: Agent creation, evaluation, and red team testing
Orchestration Flow
This orchestrator executes five specialized skills in sequence:
- repository-setup - Create Azure DevOps repository and push template code
- service-connection-setup - Configure service connections with federated credentials
- environment-setup - Create variable groups and environments
- pipeline-setup - Create CI/CD pipelines from YAML templates
- deployment-validation - Validate complete deployment
Usage
# Complete deployment (all phases)
cd .github/skills/starter-execution
# Follow Phase 1-6 commands below
# Or execute individual skills as needed
cd .github/skills/repository-setup
# See individual skill documentation
Deployment Workflow
Execute these six phases in sequence for successful deployment:
Phase 1: Authentication & Configuration
# Load configuration
. ./.github/skills/configuration-management/config-functions.ps1
$config = Get-StarterConfig
# Extract values
$org = $config.azureDevOps.organizationUrl
$project = $config.azureDevOps.projectName
# Login to Azure
az login
# Set subscription
az account set --subscription $config.azure.subscriptionId
# Get bearer token (valid for ~1 hour)
$env:ADO_TOKEN = az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query "accessToken" -o tsv
$env:AZURE_DEVOPS_EXT_PAT = $env:ADO_TOKEN
# Configure Azure DevOps CLI
az devops configure --defaults organization=$org project=$project
Write-Host "✅ Authentication complete"
Phase 2: Repository Setup
Skill: repository-setup
Purpose: Create Azure DevOps repository and push template application code
Key Actions:
- Create repository
azure-ai-foundry-app
- Initialize git and push template code from
template-app/
- Create
.env file from sample.env
Direct Usage:
cd .github/skills/repository-setup
# Follow skill documentation for detailed steps
Quick Validation:
az repos show --repository "azure-ai-foundry-app" -o table
Troubleshooting: See repository-setup/SKILL.md
Phase 3: Service Connection Setup
Skill: service-connection-setup
Purpose: Create service connections with Workload Identity Federation (passwordless, no secrets!)
Key Actions:
- Create 3 service connections:
azure-foundry-dev, azure-foundry-test, azure-foundry-prod
- Configure federated credentials on Service Principal
- Authorize connections for all pipelines
- Verify RBAC roles (Contributor + Cognitive Services User)
Critical: Federated credential issuer/subject format must match Azure DevOps exactly!
Direct Usage:
cd .github/skills/service-connection-setup
# Follow skill documentation for detailed steps
Quick Validation:
az devops service-endpoint list -o table
az ad app federated-credential list --id $config.servicePrincipal.appId -o table
Troubleshooting: See service-connection-setup/SKILL.md
Phase 4: Environment Setup
Skill: environment-setup
Purpose: Create variable groups and environments for all deployment stages
Key Actions:
- Create 3 variable groups:
{projectName}-dev-vars, {projectName}-test-vars, {projectName}-prod-vars (using config.naming.projectName)
- Configure environment-specific variables (endpoints, model names, connection strings)
- Create 3 environments:
dev, test, production
- Authorize variable groups for pipeline access
Critical: Variable group names must match pipeline YAML exactly!
Direct Usage:
cd .github/skills/environment-setup
# Follow skill documentation for detailed steps
Quick Validation:
az pipelines variable-group list -o table
az pipelines environment list -o table
Troubleshooting: See environment-setup/SKILL.md
Phase 5: Pipeline Setup
Skill: pipeline-setup
Purpose: Create CI/CD pipelines from template YAML files
Key Actions:
- Update pipeline YAML files - Automatically replaces
REPLACE_WITH_YOUR_PROJECTNAME with config.naming.projectName
- Commit and push changes - Updates YAML files in repository with correct variable group names
- Create pipeline:
Azure AI Foundry - Create Agent
- Create pipeline:
Azure AI Foundry - Agent Evaluation
- Create pipeline:
Azure AI Foundry - Red Team
- Link pipelines to repository branch
- Configure with
--skip-first-run flag
Critical: Automated script updates YAML files to match variable group names created in Phase 4!
Direct Usage:
cd .github/skills/pipeline-setup
./scripts/create-pipelines.ps1 -UseConfig
Follow skill documentation for detailed steps
**Quick Validation**:
```powershell
az pipelines list -o table
Troubleshooting: See pipeline-setup/SKILL.md
Phase 6: Deployment Validation
Skill: deployment-validation
Purpose: Validate complete deployment and readiness for pipeline execution
Key Actions:
- Validate repository exists and has code
- Validate service connections and federated credentials
- Validate variable groups and environments
- Validate pipelines are configured correctly
- Verify RBAC permissions (Contributor + Cognitive Services User)
- (Optional) Execute first pipeline run
Direct Usage:
cd .github/skills/deployment-validation
# Follow skill documentation for detailed steps
Quick Validation:
# Comprehensive validation
az repos list -o table
az devops service-endpoint list -o table
az pipelines variable-group list -o table
az pipelines environment list -o table
az pipelines list -o table
First Pipeline Run:
# Run Create Agent pipeline
$pipelineId = az pipelines list --query "[?name=='Azure AI Foundry - Create Agent'].id" -o tsv
az pipelines run --id $pipelineId
# Monitor at: $config.azureDevOps.organizationUrl/$config.azureDevOps.projectName/_build
Troubleshooting: See deployment-validation/SKILL.md
Specialized Skills
This orchestrator delegates to five specialized skills:
1. Repository Setup Skill
Location: ../repository-setup
Purpose: Create Azure DevOps repository and push template application code
When to Use: Initial deployment or repository recreation
2. Service Connection Setup Skill
Location: ../service-connection-setup
Purpose: Configure service connections with Workload Identity Federation
When to Use: Initial deployment, fix federated credentials, add new environments
3. Environment Setup Skill
Location: ../environment-setup
Purpose: Create variable groups and environments
When to Use: Initial deployment, update variables, add new environments
4. Pipeline Setup Skill
Location: ../pipeline-setup
Purpose: Create CI/CD pipelines from YAML templates
When to Use: Initial deployment, add new pipelines, fix pipeline configuration
5. Deployment Validation Skill
Location: ../deployment-validation
Purpose: Validate complete deployment readiness
When to Use: After each phase, troubleshooting, pre-deployment verification
Troubleshooting
For detailed troubleshooting, refer to the specific skill documentation:
Common Issues
Symptom: AADSTS70021: No matching federated identity record found
Cause: Federated credential issuer/subject mismatch
Solution: See service-connection-setup troubleshooting
Symptom: The pipeline is not valid. Could not find service connection
Cause: Service connection not authorized for pipeline use
Solution: See service-connection-setup troubleshooting
Symptom: Variable group name contains invalid characters
Cause: Invalid characters in variable group name
Solution: See environment-setup troubleshooting
Symptom: 401 Unauthorized errors
Cause: Bearer token expired (valid for ~1 hour)
Solution:
$env:ADO_TOKEN = az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 --query "accessToken" -o tsv
$env:AZURE_DEVOPS_EXT_PAT = $env:ADO_TOKEN
Complete Troubleshooting Guide: docs/troubleshooting.md
Best Practices
Execution Strategy
- Follow the phase order: Always execute phases 1-6 in sequence
- One environment at a time: Test dev thoroughly before test/prod
- Use modular skills: Run individual skills for easier troubleshooting
- Validate early and often: Use deployment-validation after each phase
Security
- Workload Identity Federation: Zero secrets stored - use federated credentials
- RBAC principle: Contributor + Cognitive Services User roles only (least privilege)
- Configuration management: Store starter-config.json securely (add to .gitignore)
- Token management: Refresh bearer token every 30-45 minutes
Configuration
- Use starter-config.json: Centralized configuration for consistency
- Variable group naming: Must match pipeline YAML exactly
- Federated credential format: Must match Azure DevOps issuer/subject precisely
- Document customizations: Track any template modifications
Deployment
- Skip first run: Use
--skip-first-run when creating pipelines for control
- Test incrementally: Validate each phase before proceeding
- Monitor first execution: Watch logs for the initial Create Agent pipeline run
- Use feedback mechanism: Report issues via template-app/FEEDBACK.md
Related Skills
Prerequisite Skills (Run Before This Orchestrator)
Orchestrated Skills (Called By This Orchestrator)
Documentation
