| name | foundry-cost-monitoring |
| description | Continuous Foundry token-cost monitoring, chargeback, budget alerts, and FinOps automation. Joins gen_ai.usage spans emitted by foundry-observability with the Azure Retail Prices API to compute per-agent / per-project / per-tenant cost projection. Wires Cost Management budgets + Action Groups, and uses Foundry's project-tag attribution (preview) for chargeback. USE FOR: foundry cost monitoring, token cost dashboard, FinOps Foundry, project-level chargeback, project tag cost attribution, Azure retail prices API, gen_ai.usage cost KQL, budget alert Foundry, Action Group cost webhook, token cost anomaly detection, OTel vs Cost Management reconciliation, per-agent / per-tenant token spend. DO NOT USE FOR: one-shot PTU vs PAYGO sizing (use paygo-ptu-cost-analyzer); emitting cost telemetry from agent code (use foundry-observability); gateway-only chargeback via x-app-id (use citadel-spoke-onboarding); fine-tuning cost estimates.
|
| metadata | {"version":"1.0.4"} |
Foundry Cost Monitoring
Continuous, FinOps-grade cost monitoring for Microsoft Foundry. Where
foundry-observability emits the
gen_ai.usage.* spans, this skill consumes them — joining them with
the Azure Retail Prices API to project per-agent, per-project, and
per-tenant cost in near-real time, then wiring Cost Management budgets
and Action Groups so the FinOps team sees overruns within minutes
instead of next-month's invoice.
Sibling skills. Pair with foundry-observability for the telemetry
emit path, paygo-ptu-cost-analyzer
for one-shot PTU sizing decisions, and
citadel-spoke-onboarding when
chargeback is gateway-mediated via APIM x-app-id headers.
§1 — Three layers of cost truth
Foundry cost lands in three places, with three different latencies and
three different fidelities:
| Layer | Source | Latency | Fidelity | Use for |
|---|
| 1. Foundry portal estimate | ai.azure.com → Operate → Overview / Build → Agent / Model Monitor | ≈ 2–5 min | Estimate — not invoiceable | Near-real-time engineering insight (per-agent rollup) |
| 2. OTel real-time projection | App Insights customDimensions['gen_ai.usage.*'] × Retail Prices rate card | ≈ 2–5 min ingest + your KQL window | Engineering projection — not invoiceable | Per-tenant / per-agent / per-conversation projection, anomaly detection, dashboards |
| 3. Cost Mgmt actuals | Microsoft.CostManagement/query REST + invoice CSV exports | ≈ 8–24 h | Source of truth for billing | Finance reconciliation, chargeback, budgets |
Reconciliation rule (and this is the rule that prevents the awkward
quarterly call with finance): treat layer 3 as authoritative for
anything that ends up on the bill. Layers 1 and 2 are engineering tools
to see overruns before layer 3 catches up. Per the MS Learn cost
guidance: "treat your invoice and meter records as the source of
truth."
§2 — Pricing source: Azure Retail Prices REST API
The Retail Prices API
is anonymous, public, and rate-card complete:
GET https://prices.azure.com/api/retail/prices
?$filter=<OData filter>
[&$top=N]
[&$skip=N]
No auth header. No subscription. Just curl (or urllib.request). The
filter shape that matters for Foundry models in Sweden Central:
curl -s "https://prices.azure.com/api/retail/prices?\$filter=serviceName eq 'Foundry Models' and armRegionName eq 'swedencentral' and contains(meterName, 'Tokens')" | jq '.Items[0:3]'
curl -s "https://prices.azure.com/api/retail/prices?\$filter=serviceName eq 'Foundry Models' and armRegionName eq 'swedencentral' and contains(productName, 'embedding')"
Each Items[] row carries the fields you need to price a token row:
| Field | Example | What it means |
|---|
serviceName | Foundry Models | Top-level service classification |
productName | Azure OpenAI, Azure Llama Models, Azure OpenAI GPT5 | Model family bucket |
skuName | gpt 4.1 Inp regnl | Deployment-shape SKU |
meterName | gpt 4.1 Inp regnl Tokens | Billing meter |
retailPrice | 0.00242 | Per unitOfMeasure |
unitOfMeasure | 1K, 1M | Token unit (mix of 1K and 1M — normalize before multiplying) |
currencyCode | USD | Pricing currency |
armRegionName | swedencentral | Region |
type | Consumption | Consumption for PAYGO, Reservation for PTU |
Pagination: response includes NextPageLink; iterate until null when
caching the full catalog (~thousands of rows for AI + Machine Learning).
Service-name drift. The Retail Prices API used to surface OpenAI
meters under serviceName eq 'Cognitive Services'. Current GA
classification is Foundry Models. The drift detector for this
skill polls the Retail Prices endpoint weekly — if a new classification
appears, refresh § 11 of this skill and bump PATCH.
Cache TTL recommendation: 24 h. Rates change weekly at most; daily
refresh comfortably beats invoice cadence.
§3 — Foundry project tag chargeback (Preview)
Foundry automatically tags Models-sold-by-Azure usage records with a
project tag whose value is the Foundry project name. The chargeback
flow is then:
- In Cost Management → Cost Analysis, scope to your Foundry resource.
- Add filter → Tag →
project.
- Pick one or more projects → cost split by project, time-series.
CLI equivalent:
az consumption usage list \
--subscription "$SUB" \
--start-date "$(date -u -v-30d +%Y-%m-%d)" \
--end-date "$(date -u +%Y-%m-%d)" \
--query "[?tags.project=='proj-fsi-claims'].{date:usageStart, meter:meterDetails.meterName, qty:quantity, cost:pretaxCost}" \
-o table
Verbatim preview limit (MS Learn, manage-costs#chargeback-with-project-level-cost-attribution-preview):
"Project-level cost attribution is currently supported for Models
sold by Azure (Azure Direct models, including Azure OpenAI). It isn't
yet supported for models served through Azure Marketplace."
Concretely: Azure OpenAI / Foundry-native models → tagged automatically.
Llama / Mistral / DeepSeek / Cohere via Azure Marketplace → no project
tag; fall back to layer-2 OTel projection (§ 4) or per-resource
grouping (one project ↔ one resource group).
The tag is not manually applied — do not try to add it via
az tag create-or-update; Foundry control plane writes it on the
underlying meter records.
§4 — In-flight cost projection: KQL on gen_ai.usage.*
This is the engineering-insight layer. Requires foundry-observability
already wired (account-level App Insights connection + ACA-side
configure_azure_monitor()). The MAF 1.6.0+ schema emits OTel spans
with the following customDimensions:
| Key | Type | Example |
|---|
gen_ai.system | string | azure_openai |
gen_ai.request.model | string | gpt-5.4-mini |
gen_ai.response.model | string | gpt-5.4-mini-2026-01-15 |
gen_ai.usage.input_tokens | int | 1284 |
gen_ai.usage.output_tokens | int | 342 |
gen_ai.operation.name | string | chat, embeddings |
Spans land in traces / dependencies depending on instrumentation. The
canonical filter for MAF-emitted spans is cloud_RoleName == "agent_framework"
(verified at MAF 1.6.0+ — see foundry-observability
§ "OTel cloud_RoleName" for the rationale).
4.1 Per-agent cost rollup (last 24 h)
// Per-agent token cost (last 24h) — projection-only, not invoiceable
let rateCard = externaldata(model:string, in_rate_usd_per_1k:real, out_rate_usd_per_1k:real)
[@"https://raw.githubusercontent.com/<your-org>/<your-repo>/main/rate-card.csv"]
with (format="csv", ignoreFirstRecord=true);
dependencies
| where timestamp > ago(24h)
| where cloud_RoleName == "agent_framework"
| where isnotempty(customDimensions["gen_ai.usage.input_tokens"])
| extend
agent = tostring(customDimensions["gen_ai.agent.name"]),
model = tostring(customDimensions["gen_ai.request.model"]),
in_tokens = toint(customDimensions["gen_ai.usage.input_tokens"]),
out_tokens = toint(customDimensions["gen_ai.usage.output_tokens"])
| join kind=leftouter rateCard on model
| extend
in_cost = (in_tokens / 1000.0) * in_rate_usd_per_1k,
out_cost = (out_tokens / 1000.0) * out_rate_usd_per_1k
| summarize
calls = count(),
input_tokens = sum(in_tokens),
output_tokens = sum(out_tokens),
projected_cost_usd = sum(in_cost + out_cost)
by agent, model
| order by projected_cost_usd desc
4.2 Per-project cost rollup
gen_ai.agent.name carries the agent name. To get per-project rollup
without the preview tag, instrument the project name as a resource
attribute on the OTel exporter (e.g. OTEL_RESOURCE_ATTRIBUTES="foundry.project=proj-fsi-claims")
and group on customDimensions["foundry.project"] instead of agent.
Where the project tag (§ 3) IS available, prefer Cost Management for
authoritative reconciliation and reserve KQL for in-flight projection.
4.3 Per-tenant cost rollup (Citadel APIM gateway path)
If traffic goes through the AI Citadel gateway (see § 5), the APIM
fragment-policy stamps the tenant identifier into the OTel span as
enduser.id (gateway maps x-app-id header → enduser.id). The
per-tenant rollup is identical to § 4.1 but groups on
tostring(customDimensions["enduser.id"]).
§5 — Citadel APIM x-app-id chargeback path
When Foundry traffic is fronted by an APIM AI Gateway (the AI Citadel
Hub topology), tenant identity arrives on the x-app-id request header
and the gateway logs it into App Insights. This is the cleanest
chargeback path for multi-tenant SaaS deployments because it does not
depend on Foundry's project-tag preview.
Wiring path (full detail in
citadel-spoke-onboarding):
Tenant request → APIM (validates JWT, stamps x-app-id) → Foundry
↓
OTel span tagged with
enduser.id = x-app-id
KQL aggregates over enduser.id then rejoin to the customer-ID-to-name
table (kept outside this skill — usually a Cosmos/Kusto reference
table).
§6 — Budget alerts via Azure Cost Management
Set bottom-up: subscription budget + per-RG budgets for spike isolation.
6.1 Bicep (recommended)
// budget.bicep — per-RG budget with notifications at 80% / 100% / 120%
@description('Budget amount in USD per month')
param budgetAmount int = 5000
@description('FinOps distribution list')
param finopsEmails array = [
'finops@<example-bank>.com'
]
resource budget 'Microsoft.Consumption/budgets@2024-08-01' = {
name: 'budget-foundry-${resourceGroup().name}'
properties: {
amount: budgetAmount
timeGrain: 'Monthly'
timePeriod: {
startDate: '2026-06-01'
}
category: 'Cost'
notifications: {
Warning_80: {
enabled: true
operator: 'GreaterThan'
threshold: 80
contactEmails: finopsEmails
thresholdType: 'Actual'
}
Critical_100: {
enabled: true
operator: 'GreaterThan'
threshold: 100
contactEmails: finopsEmails
contactGroups: [
actionGroup.id
]
thresholdType: 'Actual'
}
Forecast_120: {
enabled: true
operator: 'GreaterThan'
threshold: 120
contactEmails: finopsEmails
thresholdType: 'Forecasted'
}
}
filter: {
tags: {
name: 'project'
operator: 'In'
values: [
'proj-fsi-claims'
'proj-fsi-onboarding'
]
}
}
}
}
resource actionGroup 'Microsoft.Insights/actionGroups@2024-10-01-preview' = {
name: 'ag-finops-${resourceGroup().name}'
location: 'global'
properties: {
groupShortName: 'finops'
enabled: true
webhookReceivers: [
{
name: 'finops-teams'
serviceUri: '<teams-incoming-webhook-url>'
useCommonAlertSchema: true
}
]
}
}
6.2 CLI equivalent
az consumption budget create-with-rg \
--resource-group "$RG" \
--budget-name "budget-foundry-$RG" \
--amount 5000 \
--time-grain Monthly \
--start-date "$(date -u +%Y-%m-01)" \
--category Cost \
--notifications-properties \
"[{operator:'GreaterThan',threshold:80,contactEmails:['finops@<example-bank>.com'],thresholdType:'Actual'}]"
Hard-limit caveat (MS Learn manage-costs § Create budgets):
"Azure OpenAI doesn't currently provide [hard-limit] functionality.
You can start automation from action groups as part of your budget
notifications to take more advanced actions, but this functionality
requires additional custom development."
Budgets are alerts, not throttles. For hard ceilings, wire the
Action Group → Logic App / Function that disables the offending
deployment (e.g. az cognitiveservices account deployment update --properties.callRateLimit=0).
§7 — Action Group → Webhook / Logic App / Teams
Standard Azure pattern. The Action Group from § 6.1 wires three
common downstream channels:
| Channel | Use case |
|---|
| Teams incoming webhook | FinOps channel notification (useCommonAlertSchema: true so payload is parseable) |
| Logic App | Disable / scale-down offending deployments automatically |
| Azure Function | Custom remediation (e.g. notify owner, file ticket, page on-call) |
The Action Group is fire-and-forget — Cost Management posts the budget
breach event payload to every receiver in parallel.
§8 — Anomaly detection on token cost
The KQL queries in § 4 are point-in-time. To catch unexpected spend
spikes (a leaked API key, a runaway agent loop, a prompt that exploded
context-window utilisation), wrap the time-series in
series_decompose_anomalies:
// 7-day token-cost anomaly detection per agent
let bin_size = 1h;
dependencies
| where timestamp > ago(7d)
| where cloud_RoleName == "agent_framework"
| where isnotempty(customDimensions["gen_ai.usage.input_tokens"])
| extend
agent = tostring(customDimensions["gen_ai.agent.name"]),
model = tostring(customDimensions["gen_ai.request.model"]),
in_tokens = toint(customDimensions["gen_ai.usage.input_tokens"]),
out_tokens = toint(customDimensions["gen_ai.usage.output_tokens"])
| summarize total_tokens = sum(in_tokens + out_tokens) by agent, bin(timestamp, bin_size)
| make-series series=sum(total_tokens) default=0 on timestamp from ago(7d) to now() step bin_size by agent
| extend (anomalies, score, baseline) = series_decompose_anomalies(series, 2.5)
| mv-expand timestamp, series, anomalies, score, baseline
| where toint(anomalies) != 0
| project agent, timestamp=todatetime(timestamp), tokens=toint(series), anomaly=toint(anomalies), score=todouble(score)
| order by score desc
Wire this query to an Azure Monitor scheduled query alert (15-min
frequency, threshold count() > 0) → same Action Group from § 6.
§9 — Reconciliation drift: OTel projection vs Cost Mgmt actuals
The two layers will never match exactly. Expected drift:
| Source | Latency | Causes of drift vs invoice |
|---|
| OTel projection | 2–5 min ingest | Stale rate card; dropped spans; OTel sampling; cached input pricing not modeled; reservation discounts not applied |
| Cost Mgmt actuals | 8–24 h | Authoritative — the bill |
Acceptable drift: ±5% day-over-day, ±2% month-over-month. If your
projection consistently overshoots, the rate card is stale (refresh from
Retail Prices API). If it consistently undershoots, you're missing a
meter — usually the cached-input meter (some models split cached input
into a separate, cheaper meter) or the fine-tuned-hosting hourly meter.
Reconciliation cadence:
- Daily: run § 4.1 KQL for previous day, compare to
query.usage
result for the same scope + day. Log delta.
- Weekly: refresh the rate-card CSV from Retail Prices.
- Monthly: full reconciliation against invoice CSV export. Adjust
series_decompose_anomalies baseline if drift bias exceeds ±2%.
§10 — Common pitfalls
| Pitfall | Symptom | Fix |
|---|
| Model-name normalization | KQL model join misses (gpt-5.4-mini vs gpt-5.4-mini-2026-01-15) | Strip the date suffix before joining: extend model_base = extract(@"^([a-z0-9\-\.]+?)(?:-\d{4}-\d{2}-\d{2})?$", 1, model) |
| Missing meter for new SKU | Recently-released model returns no rate | Refresh rate card (Retail Prices is updated within hours); fall back to a same-family proxy SKU temporarily |
| Project-tag preview limit | Marketplace models (Llama, Mistral, DeepSeek) return zero rows when filtered by tags.project | § 3 verbatim — fall back to OTel customDimensions["foundry.project"] |
| OTel ingestion lag | Cost dashboard shows zero for the last 5 min | Expected. Use ago(15m) minimum window. Pattern 13 (LAW soft-PASS) for any alert |
| Marketplace-vs-direct boundary | Reconciliation drift > 10% on a Marketplace-heavy tenant | Marketplace meters bill under different scopes (Global resources resource group); scope KQL accordingly |
| Unit-of-measure mix | Cost projection 1000× too high | Rate card has both 1K and 1M units — normalize to 1K in your rate-card cache |
| Cached input pricing | Projection consistently overshoots actuals | Some models emit gen_ai.usage.input_tokens.cached separately; price that with the cheaper cached-input meter |
| Currency mix | Multi-region tenant has both USD and EUR rows | Filter currencyCode eq 'USD' (or normalize FX) in your rate-card cache |
| Cost Mgmt 429 | query.usage throttled under load | Cache to a per-day Kusto table; never query Cost Mgmt from a per-request hot path |
§11 — Permissions matrix
Per MS Learn manage-costs#configure-permissions-to-view-costs:
| Role | Scope | What it grants | Required for |
|---|
| Cost Management Reader (built-in) | Subscription or RG | View costs and usage data via Cost Mgmt | § 1 layer 3, § 3, § 6, § 9 |
| Foundry User (built-in, formerly Azure AI User) | Foundry account / project | View Foundry resource data and usage context | § 1 layer 1 (portal estimates), per-agent Monitor tab |
| Monitoring Reader (built-in) | Subscription or RG containing App Insights | Read App Insights metrics + alerts metadata | § 7 Action Group inspection |
| Log Analytics Reader (built-in) | LAW workspace | Run KQL via query_workspace | § 4, § 8 |
| Application Insights Component Contributor (built-in) | App Insights resource | Manage alerts (create scheduled-query rules) | § 8 alert wiring |
| Foundry Cost Reader (custom — example below) | Foundry resource | Least-privilege read for cost-only dashboards | Optional — when you don't want to grant full Cost Management Reader |
MS Learn role rename notice (current as of 2026-06). "The Foundry
RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry
Account Owner, and Foundry Project Manager were previously named
Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI
Project Manager. … The role IDs and core permissions are unchanged."
Example custom role: Foundry Cost Reader
MS Learn ships an example JSON for a least-privilege custom role
covering cost-only views. Use this when full Cost Management Reader
is too broad:
{
"Name": "Foundry Cost Reader",
"IsCustom": true,
"Description": "Can see cost metrics in Foundry",
"Actions": [
"Microsoft.Consumption/*/read",
"Microsoft.CostManagement/*/read",
"Microsoft.Resources/subscriptions/read",
"Microsoft.CognitiveServices/accounts/AIServices/usage/read"
],
"NotActions": [],
"DataActions": [],
"NotDataActions": [],
"AssignableScopes": [
"/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.CognitiveServices/accounts/<foundryResourceName>"
]
}
Create with:
az role definition create --role-definition foundry-cost-reader.json
Important — the custom role on its own does NOT grant Foundry
resource visibility. Pair it with Foundry User when the role
recipient also needs to inspect Foundry agents / models / projects.
CI-side grants used by awesome-gbb's own fixture
The CI UAMI currently has Contributor + AcrPush + Cognitive Services
OpenAI User + Foundry User on the CI resource group (see
AGENTS.md § 9.7 and .github/ci-shared-preamble.md for the canonical
identifiers). The fixture for this skill exercises the Cost
Management read path; the role needed is Cost Management Reader
at subscription scope:
az role assignment create \
--assignee-object-id "$AZURE_OBJECT_ID" \
--assignee-principal-type ServicePrincipal \
--role "Cost Management Reader" \
--scope "/subscriptions/$AZURE_SUBSCRIPTION_ID"
The fixture soft-passes on 403 from the Cost Management REST call
per AGENTS.md § 9.7 Pattern 25, so the CI run still emits PASS when this
grant is pending — the role grant is tracked separately and does not
block the catalog from shipping.
§12 — Inference cost optimization levers (FinOps)
§ 1–§ 9 tell you what you spend; these levers cut it. Inference cost
turns on three knobs distinct from generic compute right-sizing: token
volume, deployment type (Standard pay-per-token vs Provisioned/PTU),
and deployment SKU/region. Apply in priority order:
| Priority | Lever | Detection | Action | Savings |
|---|
| 🔴 Crit | Idle PTU deployment | sku.name Provisioned + sustained util below breakeven | PTU bills 24/7 — cut count or move to GlobalStandard | 30–70% |
| 🔴 Crit | Flagship on simple tasks | gpt-5.4 for classify/extract/route at scale | move to -mini/-nano | 50–95% |
| 🟠 High | Sync endpoint for batchable work | offline evals/backfills on real-time endpoint | Batch API (24h window) | ~50% |
| 🟠 High | Uncached repeated context | large static system prompt resent every call | enable prompt caching | 10–40% |
| 🟠 High | Regional SKU where Global fits | Standard/DataZoneStandard, no residency need | switch to GlobalStandard | 5–30% |
| 🟡 Med | Unbounded output | no max_completion_tokens cap | cap output (priced > input) | 10–30% |
| 🟡 Med | Oversized embedding | text-embedding-3-large where small fits | text-embedding-3-small | 60–80% |
Deployment-type breakeven: PTU only beats Standard once sustained
throughput clears the per-token breakeven — model it from real volume
(paygo-ptu-cost-analyzer) before reserving. Below breakeven, idle PTU
is the single largest inference waste. Per-app showback needs
token-level attribution Cost Management lacks — emit gen_ai.usage.*
tagged by consumer (§ 4) and reconcile against deployment cost (§ 9).
See also
