Use this skill to work with Microsoft Foundry (Azure AI Foundry): deploy AI models from catalog, build RAG applications with knowledge indexes, create and evaluate AI agents.
USE FOR: Microsoft Foundry, AI Foundry, deploy model, model catalog, RAG, knowledge index, create agent, evaluate agent, agent monitoring.
DO NOT USE FOR: Azure Functions (use azure-functions), App Service (use azure-create-app).
Use this skill to work with Microsoft Foundry (Azure AI Foundry): deploy AI models from catalog, build RAG applications with knowledge indexes, create and evaluate AI agents.
USE FOR: Microsoft Foundry, AI Foundry, deploy model, model catalog, RAG, knowledge index, create agent, evaluate agent, agent monitoring.
DO NOT USE FOR: Azure Functions (use azure-functions), App Service (use azure-create-app).
Microsoft Foundry Skill
This skill helps developers work with Microsoft Foundry resources, covering model discovery and deployment, RAG (Retrieval-Augmented Generation) applications, AI agent creation, evaluation workflows, and troubleshooting.
When to Use This Skill
Use this skill when the user wants to:
Discover and deploy AI models from the Microsoft Foundry catalog
Build RAG applications using knowledge indexes and vector search
Create AI agents with tools like Azure AI Search, web search, or custom functions
Evaluate agent performance using built-in evaluators
Set up monitoring and continuous evaluation for production agents
Troubleshoot issues with deployments, agents, or evaluations
Prerequisites
Azure Resources
An Azure subscription with an active account
Appropriate permissions to create Microsoft Foundry resources (e.g., Azure AI Owner role)
Resource group for organizing Foundry resources
Tools
Azure CLI installed and authenticated (az login)
Azure Developer CLI (azd) for deployment workflows (optional but recommended)
Language-Specific Requirements
For SDK examples and implementation details in specific programming languages, refer to:
Python: See language/python.md for Python SDK setup, authentication, and examples
Core Workflows
1. Getting Started - Model Discovery and Deployment
Use Case
A developer new to Microsoft Foundry wants to explore available models and deploy their first one.
Step 1: List Available Resources
First, help the user discover their Microsoft Foundry resources.
Using Azure CLI:
Bash
# List all Microsoft Foundry resources in subscription
az resource list \
--resource-type "Microsoft.CognitiveServices/accounts" \
--query "[?kind=='AIServices'].{Name:name, ResourceGroup:resourceGroup, Location:location}" \
--output table
# List resources in a specific resource group
az resource list \
--resource-group <resource-group-name> \
--resource-type "Microsoft.CognitiveServices/accounts" \
--output table
Using MCP Tools:
Use the foundry_resource_get MCP tool to get detailed information about a specific Foundry resource, or to list all resources if no name is provided.
Step 2: Browse Model Catalog
Help users discover available models, including information about free playground support.
Key Points to Explain:
Some models support free playground for prototyping without costs
Models can be filtered by publisher (e.g., OpenAI, Meta, Microsoft)
Models can be filtered by license type
Model availability varies by region
Using MCP Tools:
Use the foundry_models_list MCP tool:
List all models: foundry_models_list()
List free playground models: foundry_models_list(search-for-free-playground=true)
Filter by publisher: foundry_models_list(publisher="OpenAI")
Filter by license: foundry_models_list(license="MIT")
Example Output Explanation:
When listing models, explain to users:
Models with free playground support can be used for prototyping at no cost
Some models support GitHub token authentication for easy access
Check model capabilities and pricing before production deployment
Step 3: Deploy a Model
Guide users through deploying a model to their Foundry resource.
Using Azure CLI:
Bash
# Deploy a model (e.g., gpt-4o)
az cognitiveservices account deployment create \
--name <foundry-resource-name> \
--resource-group <resource-group-name> \
--deployment-name gpt-4o-deployment \
--model-name gpt-4o \
--model-version "2024-05-13" \
--model-format OpenAI \
--sku-capacity 10 \
--sku-name Standard
# Verify deployment status
az cognitiveservices account deployment show \
--name <foundry-resource-name> \
--resource-group <resource-group-name> \
--deployment-name gpt-4o-deployment
Using MCP Tools:
Use the foundry_models_deploy MCP tool with parameters:
resource-group: Resource group name
deployment: Deployment name
model-name: Model to deploy (e.g., "gpt-4o")
model-format: Format (e.g., "OpenAI")
azure-ai-services: Foundry resource name
model-version: Specific version
sku-capacity: Capacity units
scale-type: Scaling type
Deployment Verification:
Explain that when deployment completes, provisioningState should be Succeeded. If it fails, common issues include:
Insufficient quota
Region capacity limitations
Permission issues
Step 4: Get Resource Endpoint
Users need the project endpoint to connect their code to Foundry.
Using MCP Tools:
Use the foundry_resource_get MCP tool to retrieve resource details including the endpoint.
Expected Output:
The endpoint will be in format: https://<resource>.services.ai.azure.com/api/projects/<project-name>
Save this endpoint as it's needed for subsequent API and SDK calls.
2. Building RAG Applications with Knowledge Indexes
Use Case
A developer wants to build a Retrieval-Augmented Generation (RAG) application using their own documents.
Understanding RAG and Knowledge Indexes
Explain the Concept:
RAG enhances AI responses by:
Retrieving relevant documents from a knowledge base
Augmenting the AI prompt with retrieved context
Generating responses grounded in factual information
Knowledge Index Benefits:
Supports keyword, semantic, vector, and hybrid search
Enables efficient retrieval of relevant content
Stores metadata for better citations (document titles, URLs, file names)
Integrates with Azure AI Search for production scenarios
Step 1: List Existing Knowledge Indexes
Using MCP Tools:
Use foundry_knowledge_index_list with your project endpoint to list knowledge indexes.
Step 2: Inspect Index Schema
Understanding the index structure helps optimize queries.
Using MCP Tools:
Use the foundry_knowledge_index_schema MCP tool with your project endpoint and index name to get detailed schema information.
Schema Information Includes:
Field definitions and data types
Searchable attributes
Vectorization configuration
Retrieval mode support (keyword, semantic, vector, hybrid)
Step 3: Create an Agent with Azure AI Search Tool
Implementation:
To create a RAG agent with Azure AI Search tool integration:
Initialize the AI Project Client with your project endpoint and credentials
Get the Azure AI Search connection from your project
A developer has built an agent and wants to evaluate its quality, safety, and performance.
Understanding Agent Evaluators
Built-in Evaluators:
IntentResolutionEvaluator: Measures how well the agent identifies and understands user requests (score 1-5)
TaskAdherenceEvaluator: Evaluates whether responses adhere to assigned tasks and system instructions (score 1-5)
ToolCallAccuracyEvaluator: Assesses whether the agent makes correct function tool calls (score 1-5)
Evaluation Output:
Each evaluator returns:
{metric_name}: Numerical score (1-5, higher is better)
{metric_name}_result: "pass" or "fail" based on threshold
{metric_name}_threshold: Binarization threshold (default or user-set)
{metric_name}_reason: Explanation of the score
Step 1: Single Agent Run Evaluation
Using MCP Tools:
Use the foundry_agents_query_and_evaluate MCP tool to query an agent and evaluate the response in one call. Provide:
Agent ID
Query text
Project endpoint
Azure OpenAI endpoint and deployment for evaluation
Comma-separated list of evaluators to use
Example Output:
{"response":"The weather in Seattle is currently sunny and 22°C.","evaluation":{"intent_resolution":5.0,"intent_resolution_result":"pass","intent_resolution_threshold":3,"intent_resolution_reason":"The agent correctly identified the user's intent to get weather information and provided a relevant response.","task_adherence":4.0,"task_adherence_result":"pass","tool_call_accuracy":5.0,"tool_call_accuracy_result":"pass"}}
Step 2: Evaluate Existing Response
If you already have the agent's response, you can evaluate it directly.
Using MCP Tools:
Use the foundry_agents_evaluate MCP tool to evaluate a specific query/response pair with a single evaluator.
Verify the tool choice is set to "required" or "auto"
Confirm the Azure AI Search connection is configured correctly
Resolution:
Update the agent's instructions to explicitly request citations in the format [message_idx:search_idx†source] and to only use the knowledge base, never the agent's own knowledge.
Use the foundry_knowledge_index_list MCP tool to verify the index exists and get the correct name.
Resolution:
Verify AI_SEARCH_INDEX_NAME environment variable matches actual index name
Check the connection points to correct Azure AI Search resource
Ensure index has been created and populated
Problem: 401/403 Authentication Errors
Common Cause: Missing RBAC permissions
Resolution:
Bash
# Assign Search Index Data Contributor role to managed identity
az role assignment create \
--assignee <managed-identity-principal-id> \
--role "Search Index Data Contributor" \
--scope /subscriptions/<subscription-id>/resourceGroups/<rg>/providers/Microsoft.Search/searchServices/<search-service>
# Verify role assignment
az role assignment list \
--assignee <managed-identity-principal-id> \
--output table
Evaluation Issues
Problem: Evaluation Dashboard Shows No Data
Common Causes:
No recent agent traffic
Time range excludes the data
Ingestion delay
Resolution:
Generate new agent traffic (test queries)
Expand the time range filter in the dashboard
Wait a few minutes for data ingestion
Refresh the dashboard
Problem: Continuous Evaluation Not Running
Diagnostics:
Check evaluation run status to identify issues. For SDK implementation, see language/python.md.
Resolution:
Verify the evaluation rule is enabled
Confirm agent traffic is flowing
Check project managed identity has Azure AI User role
Verify OpenAI endpoint and deployment are accessible
Rate Limiting and Capacity Issues
Problem: Agent Run Fails with Rate Limit Error
Error Message:Rate limit is exceeded or HTTP 429
Resolution:
Bash
# Check current quota usage
az cognitiveservices usage list \
--name <resource-name> \
--resource-group <resource-group>
# Request quota increase (manual process in portal)echo"Request quota increase in Azure Portal under Quotas section"
Request quota increase (manual process in portal)
Write-Output "Request quota increase in Azure Portal under Quotas section"
**Best Practices:**
- Implement exponential backoff retry logic
- Use Dynamic Quota when available
- Monitor quota usage proactively
- Consider multiple deployments across regions
## Quick Reference
### Common Environment Variables
```bash
# Foundry Project
PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com/api/projects/<project>
MODEL_DEPLOYMENT_NAME=gpt-4o
# Azure AI Search (for RAG)
AZURE_AI_SEARCH_CONNECTION_NAME=my-search-connection
AI_SEARCH_INDEX_NAME=my-index
# Evaluation
AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-4o
Useful MCP Tools Quick Reference
Resource Management
foundry_resource_get - Get resource details and endpoint
Models
foundry_models_list - Browse model catalog
foundry_models_deploy - Deploy a model
foundry_models_deployments_list - List deployed models
Knowledge & RAG
foundry_knowledge_index_list - List knowledge indexes
foundry_knowledge_index_schema - Get index schema
Agents
foundry_agents_list - List agents
foundry_agents_connect - Query an agent
foundry_agents_query_and_evaluate - Query and evaluate
