foundry-cross-resource

# Cross-Resource Model Invocation via Foundry AI Gateway > **Status — verified live on 2026-04-23** with Foundry account > `xtest-foundry-mr5kfi` / project `xtest-proj-mr5kfi` (Sweden Central) calling > deployment `gpt-4o-mini v2024-07-18` hosted on a different Azure OpenAI > account (`acme-aoai-shared`) through APIM `acme-ai-apim`. Both **ApiKey** > and **ProjectManagedIdentity** auth paths returned `PONG` on all three > invocation patterns. See "Verified working configuration" at the end. > > **Default in worked examples below: `gpt-5.4-mini`** (current > Foundry-routable chat-mini family, May 2026). The original verification > was on `gpt-4o-mini`; the gateway routes by `deployment-name-in-path` > regardless of model family, so the recipe is mechanically identical for > any deployment your APIM backend actually carries. The exact API > `version` string in the connection metadata below is illustrative — > use whatever your backend deployment is registered with (check > `az cognitiveservices account deployment show`). --- ## 1. What this skill solves You have a Foundry project ("**consumer**") that needs to invoke models deployed on a **different** Azure OpenAI / AI Services account ("**backend**"), fronted by an Azure API Management instance acting as the AI Gateway. You want to address those models via the Foundry-native string `connectionName/deploymentName` so the application code is identical to a local-deployment call. ``` ┌────────────────────────┐ ┌──────────────────────────┐ ┌─────────────────────────────┐ │ Consumer Foundry │ │ APIM AI Gateway │ │ Backend AI/OpenAI account │ │ project │ Foundry MI │ acme-ai-apim │ APIM MI → │ acme-aoai-shared │ │ xtest-proj-mr5kfi ├───or────────► │ /xtest-aoai ├──Bearer───────►│ gpt-5.4-mini deployment │ │ (Sweden Central) │ ApiKey │ /xtest-aoai-pmi │ (msi token) │ (East US 2) │ │ │ │ │ │ │ │ ApiManagement │ │ validate token / │ │ Cognitive Services User RBAC│ │ connection │ │ enforce subscription │ │ granted to APIM MI │ │ category=ApiManagement│ │ set-backend-service │ │ │ │ target=APIM API URL │ │ authentication-managed- │ │ │ │ authType=ApiKey | PMI │ │ identity → backend │ │ │ └────────────────────────┘ └──────────────────────────┘ └─────────────────────────────┘ ``` The consumer project never holds the backend's API key or RBAC. APIM is the trust boundary. --- ## 2. The 100% reliable checklist Before any agent call, ALL of these must be true. Cross them off in order; the official validator script [`test_apim_connection.py`](https://github.com/microsoft-foundry/foundry-samples/blob/main/infrastructure/infrastructure-setup-bicep/01-connections/apim/test_apim_connection.py) plus the field-tested probes in §8 below verify each step end-to-end. | # | Item | How to verify | |---|------|---------------| | 1 | Backend AI Services / Azure OpenAI account exists with the model deployed | `az cognitiveservices account deployment list -g <rg> -n <backend>` | | 2 | APIM exists (any SKU; Developer is fine for testing, Standard v2/Premium for prod) and has system-assigned managed identity enabled | `az apim show -g <rg> -n <apim> --query identity` | | 3 | APIM MI has `Cognitive Services User` (or `Cognitive Services OpenAI User`) on the backend account | `az role assignment list --assignee <apim-mi-objectId> --scope <backend-id>` | | 4 | APIM has an API whose path you'll target (e.g., `/xtest-aoai`) with **inbound policy that calls `set-backend-service` + `authentication-managed-identity` + Bearer header injection** (see §6) | `Invoke-RestMethod -Uri https://<apim>.azure-api.net/<api>/deployments/<dep>/chat/completions?api-version=2024-10-21 -Headers @{api-key='<sub-key>';...}` returns 200 | | 5 | (ApiKey only) An APIM subscription scoped to that API (or to a product the API is in) exists; you have the primary key | `az apim subscription show ...` | | 6 | (PMI only) APIM API has `subscriptionRequired: false` AND inbound policy includes `<validate-azure-ad-token>` with `<client-application-ids>` containing the **consumer project MI's client/app ID** | API GET shows `subscriptionRequired:false`; policy GET shows the GUID | | 7 | Consumer Foundry project exists, has Azure AI User RBAC for the caller, and (PMI) has system-assigned managed identity enabled | `az cognitiveservices account project show ... --query identity` | | 8 | An `ApiManagement` connection exists on the consumer project (NOT `AzureOpenAI`, NOT `AIServices`) | See §5 for the verified PUT body | | 9 | The connection's `metadata.models` is a **JSON-stringified** array (NOT a real array) and `metadata.deploymentInPath` matches the backend style | See §5 — this is the most common silent error | | 10 | Caller code uses `model="<connectionName>/<deploymentName>"` and calls `responses.create(...)` (NOT `chat.completions.create`) | See §7 | If step 4 doesn't return 200 outside Foundry, no Foundry call will work. Always smoke-test the gateway directly first. --- ## 3. Decision tree ``` ┌──────────────────────────────────────┐ │ What backend does APIM forward to? │ └─────────────┬────────────────────────┘ │ ┌─────────────────────────┴────────────────────────┐ │ │ AOAI / AI Services on /openai OpenAI v1 (/v1/...) (most common — Azure OpenAI) or Anthropic etc. │ │ ▼ ▼ metadata.deploymentInPath = "true" metadata.deploymentInPath = "false" metadata.inferenceAPIVersion = "2024-10-21" metadata.inferenceAPIVersion = "" (or omit) models[].properties.model.format = "OpenAI" models[].properties.model.format = "OpenAI"|"Anthropic"|"NonOpenAI" modelDiscovery.listModelsEndpoint = "/models" modelDiscovery.deploymentProvider = "AzureOpenAI"|"OpenAI"|"Anthropic"|"NonOpenAI" ┌──────────────────────────────────────┐ │ How do callers prove identity to APIM?│ └─────────────┬────────────────────────┘ │ ┌─────────────────────────┼────────────────────────┐ │ │ │ APIM subscription key Project Managed Identity Both (dual-auth) │ (no static secrets) │ ▼ ▼ ▼ authType: "ApiKey" authType: "ProjectManagedIdentity" Two connections OR credentials.key: <key> credentials: {} APIM <choose><when> APIM api: subscription req audience: policy that branches APIM policy: pass-through "https://cognitiveservices.azure.com" APIM policy: validate-azure-ad-token with <client-application-ids> containing project MI clientId APIM api: subscriptionRequired=false ``` --- ## 4. APIM-side configuration ### 4.1 Service URL on the API The Foundry connection target is `https://<apim>.azure-api.net/<apiPath>`. Inside the API's inbound policy you `set-backend-service` to the backend account. Foundry then appends `/deployments/{dep}/chat/completions?api-version=...` (or `/v1/responses` etc.) to the gateway URL. Operations on the API (i.e., the routes Foundry will call) must therefore match the AOAI / OpenAI surface. The simplest setup is **catch-all** with one operation per HTTP verb on the wildcard path `/{*path}`. For Azure OpenAI backends, all of the following must reach the inbound policy and 200: ``` POST /deployments/{deployment-id}/chat/completions?api-version=2024-10-21 POST /deployments/{deployment-id}/embeddings?api-version=2024-10-21 POST /v1/responses?api-version=preview (Responses API) GET /models?api-version=2024-10-21 (model catalogue, used for dynamic discovery) ``` ### 4.2 ApiKey-only inbound policy (verified working — `xtest-aoai`) ```xml <policies> <inbound> <base /> <set-backend-service base-url="https://acme-aoai-shared.openai.azure.com/openai" /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="msi-access-token" ignore-error="false" /> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value> </set-header> </inbound> <backend><base /></backend> <outbound><base /></outbound> <on-error><base /></on-error> </policies> ``` Subscription enforcement is at the API/product level (set `subscriptionRequired: true` and configure `subscriptionKeyParameterNames.header` to `api-key` so the caller sends `api-key: <key>`, not `Ocp-Apim-Subscription-Key`). Foundry sends the `api-key` header. ### 4.3 PMI-only inbound policy (verified working — `xtest-aoai-pmi`) ```xml <policies> <inbound> <base /> <validate-azure-ad-token tenant-id="<your-tenant-id>" header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized: token did not match expected audience or application"> <client-application-ids> <application-id><consumer-project-MI-clientId></application-id> </client-application-ids> <audiences> <audience>https://cognitiveservices.azure.com</audience> <audience>https://cognitiveservices.azure.com/</audience> </audiences> </validate-azure-ad-token> <set-backend-service base-url="https://acme-aoai-shared.openai.azure.com/openai" /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="msi-access-token" ignore-error="false" /> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value> </set-header> </inbound> <backend><base /></backend> <outbound><base /></outbound> <on-error><base /></on-error> </policies> ``` PMI API also needs `subscriptionRequired: false` on the API resource (otherwise APIM rejects with 401 before the policy runs). > **Why `<client-application-ids>` and not `<required-claims>` with `xms_mirid`?** > Both mechanisms work in principle, but the `xms_mirid` value Foundry's > project MI puts in its token is **not** the project ARM ID — it varies by > resource provider and current APIs. The MI's `appid` claim is universally > present and stable, so `<client-application-ids>` is the documented and > most reliable check. (Verified 2026-04-23: a policy requiring > `xms_mirid = <project ARM ID>` returned 401; switching to `<client-application-ids>` returned 200.) ### 4.4 Dual-auth inbound policy (ApiKey OR PMI) ```xml <policies> <inbound> <base /> <choose> <when condition="@(context.Subscription == null)"> <validate-azure-ad-token tenant-id="<your-tenant-id>" header-name="Authorization" failed-validation-httpcode="401"> <client-application-ids> <application-id><consumer-project-MI-clientId></application-id> </client-application-ids> <audiences> <audience>https://cognitiveservices.azure.com</audience> <audience>https://cognitiveservices.azure.com/</audience> </audiences> </validate-azure-ad-token> </when> </choose> <set-backend-service base-url="https://<backend>.openai.azure.com/openai" /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="msi-access-token" /> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["msi-access-token"])</value> </set-header> </inbound> </policies> ``` API must have `subscriptionRequired: false` (so PMI calls aren't rejected up front); the `<choose>` branch enforces token validation only when no subscription was used. --- ## 5. Connection schema (verified live) ARM type: `Microsoft.CognitiveServices/accounts/projects/connections@2025-04-01-preview`. Reachable also via the Foundry data-plane endpoint `https://<account>.services.ai.azure.com/api/projects/<project>/connections/<name>?api-version=v1`. > ⚠ **The `metadata.models` and `metadata.modelDiscovery` fields are > JSON-encoded strings, not real JSON objects.** This is because Azure > connection metadata is a flat string→string dictionary. Pass the JSON as a > `string` value or your PUT will be silently ignored / rejected. ### 5.1 ApiKey connection — verified working PUT body ```json { "properties": { "category": "ApiManagement", "target": "https://acme-ai-apim.azure-api.net/xtest-aoai", "authType": "ApiKey", "credentials": { "key": "<APIM subscription primary key>" }, "isSharedToAll": true, "metadata": { "deploymentInPath": "true", "inferenceAPIVersion": "2024-10-21", "models": "[{\"name\":\"gpt-5.4-mini\",\"properties\":{\"model\":{\"name\":\"gpt-5.4-mini\",\"format\":\"OpenAI\",\"version\":\"2026-04-30\",\"publisher\":\"Microsoft\"}}}]" } } } ``` PUT URL: ``` https://management.azure.com/subscriptions/<sub>/resourceGroups/<rg>/providers/ Microsoft.CognitiveServices/accounts/<consumer-acct>/projects/<consumer-proj>/ connections/<connectionName>?api-version=2025-04-01-preview ``` Or via the data plane: ``` PUT https://<consumer-acct>.services.ai.azure.com/api/projects/<consumer-proj>/connections/<name>?api-version=v1 ```