devops-agent-cost-insights

## Overview This skill provides the **methodology** for gathering and calculating AWS DevOps Agent cost data. It intentionally does NOT prescribe a fixed output format. The consuming agent (chat, custom agent, etc.) decides its own presentation (plain text, multi-space tables, single-space visual dashboard with charts, etc.) based on its own prompt/output spec. Follow the steps below to gather and compute correct numbers; format them however the calling context requires. ### Execution visibility The consuming agent's output contract controls visibility. When it requires silent, final-only execution: - Perform skill/reference reads, planning, tool calls, pagination, distillation, calculations, retries, and Step 7 validation without assistant-authored progress messages. - Keep checklist state, intermediate values, query IDs, raw records, arithmetic, and rejected records internal. - Do not announce file reads or tool calls, react to results, or narrate what happens next. - Return only the consuming agent's single final response after its artifact operation. A compact validation result table is allowed only when its final template explicitly requires one. > **This skill must add exactly $0.00 to the bill.** Never `StartQuery`, `StartQueryExecution`, or `GetMetricData` — and never any other operation the **zero-cost rule** names below. Each is a paid API, so issuing one bills the account this report is about. If a figure is only reachable through a paid call, it is unavailable; say so. This is the **zero-cost rule**, restated at Step 1 and attested in Step 7. > > **"Read-only" is not the safety test.** `GetMetricData` reads only your own metrics and is billed per metric anyway. Before using any call not already named in the steps below, classify it per *Operation classification* in Step 3; an operation you cannot confirm as free is unclassified, not free. Four things to cover every time: 1. **Discovery**: enumerate Agent Space(s) and DevOps Agent role(s) in scope (may be one space or all spaces in the account, depending on what's requested) 2. **Direct usage**: agent-seconds billed (investigations, evaluations, chat), per space 3. **Downstream**: AWS API costs from agent-initiated calls (Logs Insights, Athena, X-Ray, CloudWatch), attributed via CloudTrail, traceable by query ID / query execution ID 4. **Grouping**: if multiple spaces/roles are in scope, group cost by Agent Space ID and by DevOps Agent Role ARN; if scoped to a single space/role, a single summary suffices --- ## Workflow Checklist Work through this checklist in order and do not skip a step. The steps are dependent, not independent: Step 1 fixes the `<START>`/`<END>` window that Steps 3 and 7 must reuse, and Step 7 validates the figures produced by Steps 1 through 6. Track checklist completion internally. Do not print step-by-step progress or skipped-step narration; surface only trust-impacting omissions in the consuming agent's single final response. - [ ] Step 0: Discover the Agent Space(s) and DevOps Agent role(s) in scope. Skip only if scope is already fixed to one space/role - [ ] Step 1: Resolve the reporting window from the request, then compute direct usage (agent-seconds) via CloudWatch `GetMetricStatistics` - [ ] Step 2: Apply Support plan credits to get net direct cost - [ ] Step 3: Attribute downstream cost using journal for call counts (primary), history APIs for exact bytes, and CloudTrail for cross-check, reusing the exact Step 1 window - [ ] Step 4: Group by Agent Space ID and by Role ARN. Skip if scope is a single space/role - [ ] Step 5: Extract tool usage from journal records and cross-reference against CloudTrail - [ ] Step 6: Produce a per-investigation breakdown, only when requested - [ ] Step 7: Validate every computed figure before reporting it ### Scope precedence The consuming agent defines the scope. When it supplies an explicit `ROLE_ARN` and declares exact-role scope, that scope overrides this skill's multi-role discovery and role-coverage guidance: - Treat `ROLE_ARN` as the complete allowlist of one role. Skip role discovery and never add historical, rotated, inferred, or similarly named roles. - Require exact account and role identity. For CloudTrail, `userIdentity.sessionContext.sessionIssuer.arn` must equal `ROLE_ARN`. For an STS assumed-role identity, normalize only an ARN with the same account and exact configured role name to `ROLE_ARN`. - Discard every other role immediately. Do not count, analyze, name, summarize, or mention excluded roles. - Never widen exact-role scope because direct Agent Space usage exists on days with no matching downstream activity. - Label direct usage as Agent Space-scoped because the `AgentSpaceUUID` metric cannot isolate one role. Label downstream and tool usage as exact-role-scoped. Do not present the combined result as the complete Agent Space total. The multi-role discovery guidance below applies only when the consuming agent requests Agent Space-wide or account-wide scope. ### Definitions These named rules are defined once here and referenced by name throughout. "Per the quantity source rule", "passes the attribution test", "per the Logs source rule", and "match the active role scope" mean exactly what is written below; the step does not restate them. - **Active role scope:** the identity a record must match to count (see *Scope precedence*): the single configured `ROLE_ARN` in exact-role mode, or one of the Step 0 discovered role ARNs in multi-role mode. - **Attribution test:** a record counts only when it matches BOTH the reporting window AND the active role scope; one alone is not enough. In exact-role mode, discard non-matching roles without analysis or mention. Failing this test is a leading source of run-to-run drift. - **Logs source rule:** for CloudWatch Logs Insights, `describe_queries` is the sole source of query count, role attribution, scan bytes, and cost. Never substitute a CloudTrail `StartQuery` lookup; journal data is a cross-check only. If `describe_queries` is unavailable, report Logs Insights unavailable rather than substituting another source. - **Identity-not-Username rule:** attribute on the parsed `userIdentity` / `sessionIssuer.arn`, never on a CloudTrail `Username` filter. DevOps Agent roles in one account share a single CloudTrail session name, so a `Username` count aggregates across roles and overstates the target. See *Cross-check: CloudTrail* for the mechanics. - **CloudTrail-lower-bound rule:** CloudTrail Event History does not reliably return every matching event, so a CloudTrail-derived count is a lower bound, never an exact total. This is why the journal is primary. - **Quantity source rule:** the Pricing API supplies a rate, never a quantity. Which quantity an operation needs follows from the **unit class** of its price dimension, not its service. A **count-class** unit (`Requests`, `Metrics`, `traces`) prices from the attributed call count, which Step 3 produces for every operation, so any per-call price dimension can be priced, including one from a service named nowhere in this skill; where a service tiers its requests, name the tier or do not price. A **data-class** unit (`GB`, `TB`) needs a byte figure, so it prices only from the sources Step 3 names, never from a call count. When the quantity is missing, report the call count with no dollars: these are all **paid** operations, so a row without dollars is a measurement gap, never a free call, and makes the Priced Downstream Total a **lower bound**. A confirmed `0` calls is exactly `$0.00`, not a gap. Each is countable when the audited agent issued it, forbidden when this run would (see *Zero-cost rule*). - **Zero-cost rule:** this run must cost $0.00: no `StartQuery`, no `StartQueryExecution`, no `GetMetricData`, no log scanning. Use only the confirmed-free calls named in these steps. If a step appears to need a billable call, it finishes without that figure and reports the gap. Defined in the Overview, restated at the point of temptation in Step 1, and attested in Step 7. The rule binds by effect, not by name, across three evasion paths: - **Wrappers and aliases count.** The prohibition is on the paid operation, whatever name reaches it. Do not issue the canonical operations (`StartQuery`, `StartQueryExecution`, `GetMetricData`, `BatchGetTraces`) or any wrapper or tool form of them, including `start_query`, `query_cloudwatch_logs`, `get_metric_data`. Do not call `GetQueryResults` either: it is free, but it is the second half of a `StartQuery` pair and `describe_queries` already returns `bytesScanned`. An operation not named here is classified per *Operation classification* in Step 3 before it is called, and is unclassified rather than free if it cannot be confirmed free. - **`shell` is not an escape hatch.** A paid operation invoked through `shell` (for example `aws logs start-query`, `aws cloudwatch get-metric-data`, or an equivalent CLI or script) bills the account exactly as the tool form would. Do not use `shell`, `use_aws`, or any command runner to issue a scan or paid read this rule forbids by name. - **Delegation inherits the rule.** A delegated agent or subagent acts on the audited account's behalf, so a scan it issues is billed the same. Carry this rule into any delegated request explicitly, and never delegate an action you could not issue directly. Journal access does not normally require delegation: `get_investigation_journal_records` and `get_other_agentspace_journal_records` are called directly. When those tools are blocked this run, the Context Gatherer subagent is the fallback (see Step 3), and this rule travels with the delegated request. --- ## Step 0: Discover Spaces and Roles (skip if scope is already fixed to one space/role) What this step resolves depends on the scope the consuming agent asked for (see *Scope precedence* above): the single space this run is scoped to, or every space in the account. Nothing in it is inferred. **Always — resolve the space this run is running in, and its role.** This is the whole purpose of the step when the consuming agent supplied no identifiers. - `get_agent_space` for the current space returns its `name` and `agentSpaceId`. - `list_associations` for that space returns its AWS configuration. Take the role from `configuration.aws.assumableRoleArn`, with `accountId` and `accountType` alongside it. Pair on this field and nothing else. For a single-space request that is the whole step. Everything below applies **only** when the consuming agent requested Agent Space-wide or account-wide scope. **Account-wide or Agent Space-wide scope — enumerate the remaining spaces.** - `list_agent_spaces` returns every Agent Space in the account, each with `name`, `agentSpaceId`, and `createdAt`. This is the authoritative inventory of *live* spaces. - Call `list_associations` for each one and read `configuration.aws.assumableRoleArn` the same way. > **Never pair a role to a space by creation timestamp.** Role and space creation times are not correlated: a space's role may be created long after the space, while an unrelated role is created moments before it. Nearest-timestamp pairing therefore produces confident but wrong pairings, attributing one space's downstream cost to another space's role. `configuration.aws.assumableRoleArn` is the only pairing source. ### Reconcile live spaces against the spaces in the metrics (account-wide scope only) `list_agent_spaces` returns spaces that exist now. CloudWatch retains `AgentSpaceUUID` dimension values for spaces that have since been deleted, so the metric namespace routinely carries many more space IDs than the account has spaces. Collect the `AgentSpaceUUID` values Step 1 will read and split them: - **Live** — present in `list_agent_spaces`. Name and role are both known. - **Retired** — present in the metrics only. The billed usage is real and must be counted, but the space cannot be named and has no association from which to resolve a role. Report retired spaces as one `retired spaces` line with their count. Never fold them into a named space, and never drop them. An account-wide total that omits them is a **lower bound**; one that includes them is complete for direct usage only, because their downstream activity has no resolvable role. State which of the two you produced — Step 2's credit gate depends on it. ### Anomalies to report separately (account-wide or Agent Space-wide scope only) - **Cross-account roles.** A trust policy names the account permitted to use the role in `aws:SourceAccount`. When that value is not the audited account, the role serves another account's space: exclude it from totals and report it as a cross-account anomaly. Read `aws:SourceAccount` for this and nothing else — see the warning below before reading `aws:SourceArn`. - **Orphan roles.** An agent role that no live space's `assumableRoleArn` points at is residue from a deleted space. List these separately, and never pair one to a live space to make the counts reconcile. Apply this only to the agent role type below; the web app role is never an `assumableRoleArn`, so testing it this way reports every one of them as an orphan. > **A trust policy does not pair a role to a space.** On the agent role, `aws:SourceArn` is normally the wildcard `agentspace/*` (no space ID); where a specific `agentspace/<id>` appears it is usually a web app role, which must never be paired. `aws:SourceArn` therefore fails on the role that matters and succeeds on the wrong one. **Role types are not interchangeable, and names are not a discovery mechanism.** One space commonly has several roles with unrelated suffixes: - `DevOpsAgentRole-AgentSpace-<suffix>` — assumed by the agent to reach your resources. Cost attribution uses this one only, and only when `configuration.aws.assumableRoleArn` names it. - `DevOpsAgentRole-WebappAdmin-<suffix>` — enables the off-console web app. Never a cost-attribution role. Paths vary (`role/...` and `role/service-role/...`), other patterns exist, and the suffixes of one space's roles do not match each other. Use names only to find orphans. --- ## Step 1: Direct Usage (CloudWatch GetMetricStatistics) **The reporting window is an input, not a fixed period.** Take `<START>`/`<END>` from the request — an explicit range, a named period ("last 7 days", "August"), or the consuming agent's configured default. This skill prescribes no window of its own; a custom agent that wants a standing trailing window pins it in its own configuration and passes it in. Resolve it to absolute UTC timestamps once, here, then reuse those exact values everywhere downstream: Steps 3 and 7 depend on one window for the whole report, and a report that mixes windows describes no real period. Where scope affects credit eligibility (Step 2) or exactness labels (Step 7), those rules read the window resolved here. Read metrics with `GetMetricStatistics`, never `GetMetricData` (**zero-cost rule**). ### Discover the activity metrics, do not assume them **Enumerate the namespace first; never work from a fixed list of activity types.** Billable activities are whatever `AWS/AIDevOps` currently publishes; a hardcoded list silently drops any activity added later and understates the total with no error. ```python use_aws( service_name="cloudwatch", operation_name="list_metrics", aws_account_id="<ACCOUNT_ID>", aws_region="<REGION>", parameters={"Namespace": "AWS/AIDevOps", "Dimensions": [{"Name": "AgentSpaceUUID", "Value": "<AGENT_SPACE_ID>"}]} ) ``` Paginate until no `NextToken` remains. From the result, take **every metric whose name matches `Consumed*Time`** as a billable agent-time metric, and give each one a term in the sum below. Treat consumed-*time* metrics as billable seconds; a `Consumed*Requests` or `*Count` metric is a volume measure, not agent-seconds, so never multiply it by the agent-second rate — report it separately if useful. Report the metric names you found. If enumeration returns a name this skill has not seen before, include it, price it at the agent-second rate, and say that it was discovered rather than expected. If `list_metrics` is denied, fall back to reading the names known at authoring time — `ConsumedInvestigationTime`, `ConsumedEvaluationTime`, `ConsumedOnDemandTime` — and label direct cost a **lower bound**, because an unenumerated namespace cannot be shown to be complete. ### Primary: CloudWatch GetMetricStatistics The `AWS/AIDevOps` namespace metrics are NOT OTel-enriched, so PromQL/Prometheus queries return empty for these. Read them with `get_metric_statistics`, one call per discovered metric per Agent Space in scope: ```python use_aws( service_name="cloudwatch", operation_name="get_metric_statistics", aws_account_id="<ACCOUNT_ID>", aws_region="<REGION>", parameters={ "Namespace": "AWS/AIDevOps", "MetricName": "<each Consumed*Time metric returned by list_metrics>", "Dimensions": [{"Name": "AgentSpaceUUID", "Value": "<AGENT_SPACE_ID>"}], "StartTime": "<START>", "EndTime": "<END>", "Period": 86400, "Statistics": ["Sum"] } ) ``` > **AgentSpaceUUID dimension is required.** Without it, you receive account-level aggregates that can't distinguish between spaces. Use `Period: 86400` (daily) so `Datapoints` can be broken out per day if the consuming agent needs a time-series/trend view, not just a single aggregate. ### Calculate direct cost (per space): **Resolve the agent-second rate on this run; never carry one in from memory.** Try these in order and stop at the first that yields a rate, then report which one did: 1. **The Pricing API**, per *Resolving rates*, if the price list exposes a DevOps Agent service code. Preferred, because it is authoritative and region-scoped. Such a service code has never been confirmed to exist, so an empty result here is the expected outcome rather than a fault — fall through. 2. **[AWS DevOps Agent pricing](https://aws.amazon.com/devops-agent/pricing/).** The rate is published as a per-agent-second figure inside the worked pricing examples rather than in a rate table, so read it from an example and **verify it against that example's own arithmetic**: each example states its seconds and its total, so the rate must reproduce the total. A rate that fails the example it came from was misread — discard it and try another example. If neither yields a rate, **report direct cost as `unavailable`, naming the reason.** Do not substitute a remembered figure. Direct cost is the largest number in this report, so a stale rate misstates the entire report rather than one line. Report the agent-seconds from Step 1 either way — those are measured, exact, and independent of any rate. State the resolved rate and its source wherever direct cost appears. Then sum over **every** metric discovered above, so no activity is silently dropped: ``` billable_seconds = Σ( Sum(m) for each Consumed*Time metric m discovered ) direct_cost = billable_seconds × agent_second_rate ``` The sum is over the discovered set, not a fixed list of activities: one term per metric `list_metrics` returned, whatever they turn out to be. A metric the deployment does not emit contributes `0`. Report the metrics that made up the sum, and name any that this skill did not anticipate. **If a discovered activity bills on its own charge rather than the agent-second rate**, price that metric's seconds separately by resolving its rate per *Resolving rates*, and keep it as its own line rather than folding it into `billable_seconds`. Release-readiness review is the case to check for: a separate DevOps Agent charge has never been confirmed to exist in the price list. > **If a separate-charge lookup fails, report that activity's hours with cost `unavailable`** — not `$0.00`, and not at the agent-second rate. An empty or denied lookup for a DevOps Agent charge is the expected outcome rather than a fault, since none has been confirmed to exist. Guessing either way misstates the total. A separately-charged activity left unpriced makes direct cost a **lower bound**, so label it. --- ## Step 2: Credits DevOps Agent usage is offset by credits equal to a percentage of the account's monthly AWS Support charge, applied toward usage at the agent-second rate. Credits are issued by the 10th of each month, apply to that month's charges, and expire at month-end without rolling over. These percentages are **published on the [DevOps Agent pricing page](https://aws.amazon.com/devops-agent/pricing/)**, so the table below reflects public pricing, not an operator assumption. They have no Pricing API entry, so never try to resolve them from the price list; and because that page is script-rendered, a live fetch is unreliable, so the values are stated here directly. Only the account's own tier applies: | Support plan | Credit | `CREDIT_RATE_DECIMAL` | |---|---|---| | Unified Operations | 100% | `1.00` | | Enterprise Support | 75% | `0.75` | | Business Support+ | 30% | `0.30` | | Business, Developer, Basic | none | — | The plan and charge are **deployment values, not skill values.** This skill ships none of them, and nothing in it infers them from bill line items or credit size. Read them from the operator's configuration if supplied — `SUPPORT_PLAN` and `PRIOR_MONTH_SUPPORT_CHARGE` — and treat either that is absent as `unknown`. An unknown plan yields no credit, which is safer than a wrong one. The credit rate is not a separate input: derive it from `SUPPORT_PLAN` using the tier table above. Both supplied values come from the Billing console: *Support plans*, and the `AWS Support` line on the previous month's bill, which must be the month immediately before the reported month. Then apply the **scope gate** before subtracting anything. **Credit subtraction is gated by scope.** Produce a subtracted `Net Direct Cost` only in a full-calendar-month, account-wide report in which the Step 0 reconciliation found no retired spaces inside the window. A retired space's usage is billed but cannot be attributed to a named space or role, so a window containing one has an incomplete inventory by definition and this branch does not apply. Account-wide scope cannot be established without Step 0: when Step 0 was skipped because scope was already fixed to one space or role, this branch does not apply and there is nothing to evaluate — report `Net Direct Cost` as `unavailable at this scope`. Otherwise: ``` monthly_credits = support_charge × credit_rate net_direct_cost = max(0, direct_cost − monthly_credits) ``` The base is the Support charge of the calendar month immediately preceding the report month (that is the most recent complete charge when credits are issued on the 10th). Use `credit_rate` as the decimal from the tier table (e.g. `0.75` for Enterprise Support), not the percentage; multiplying by `75` overstates the credit hundredfold. **At a narrower scope, do not subtract.** For exact-role, single-space, partial-month, rolling, weekly, or per-investigation reports, show the plan, rate, and monthly credit as account-level reference only, and report `Net Direct Cost` as `unavailable at this scope`. Scope precedence treats exact-role single-space as the common path, so this branch is the default rather than the exception. Guardrails: - Compute `monthly_credits` only when `SUPPORT_PLAN` is a crediting tier and `PRIOR_MONTH_SUPPORT_CHARGE` is set. The charge is taken to be that of the calendar month immediately before the reported month; supply the current month's figure, since a stale charge carried over from an earlier run computes a wrong credit. - If the Support plan is unset, the plan is `unknown` and the rate is `unavailable`. Show direct cost and prompt: *"Provide your monthly AWS Support charge to calculate your credit offset."* - Floor `net_direct_cost` at zero, and never print `$0.00` for an unknown credit. The entitlement computed here is not the applied amount — confirm it against the credit on the bill before relying on a net figure. --- ## Step 3: Downstream Cost Attribution (with query-level traceability) Source authority depends on the category. Work down this order, and stop at the first source that owns the figure: 1. **Service history APIs.** `describe_queries` is exclusive for Logs count, role, bytes, and cost (the **Logs source rule**); Athena history supplies exact bytes. 2. **Journal records.** Primary tool-call inventory and cross-check. Never a replacement for `describe_queries` Logs totals.