gcp

name	gcp
description	Use when working with Gcp — cost anti-hallucination rules, MANDATORY parallel execution patterns (30x speedup), monitoring aligners, reusable billing/pricing scripts, VAT/tax handling, and filtering/pagination.
connection_type	gcp
preload	false

GCP CLI Skill

Execute GCP CLI commands with proper credential injection.

CRITICAL: Billing Data Interpretation Rules (Anti-Hallucination)

These rules are MANDATORY when analyzing GCP billing data. Violating them produces wildly incorrect cost reports.

Rule 1: NET COST is the Only Real Cost

The cost column in billing exports is NOT your actual bill. It shows usage priced at contract/on-demand rates before credits are applied. Credits (promotional, SUDs, CUDs, free tier) are stored separately in the credits array with negative amounts.

-- Pre-tax net cost (filter cost_type = 'regular'):
NET COST = SUM(cost WHERE cost_type='regular') + SUM(credits.amount)

-- Tax-inclusive net cost (include all cost_types):
NET COST WITH TAX = SUM(cost) + SUM(credits.amount)

NEVER report SUM(cost) alone as the cost. Always compute net cost. In actual SQL, always use CAST(... AS NUMERIC) — see Rule 8.

Rule 2: ALWAYS Filter by project.id

The billing export table is at the billing account level and contains costs for ALL projects under that billing account. If you query without filtering by project.id, you aggregate costs across 10+ unrelated projects.

-- WRONG: Aggregates ALL projects in the billing account
SELECT service.description, SUM(cost) FROM `{BILLING_TABLE}` GROUP BY 1

-- CORRECT: Scoped to the target project with net cost
SELECT service.description,
  SUM(CAST(cost AS NUMERIC))
    + SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) c), 0)) AS net_cost
FROM `{BILLING_TABLE}`
WHERE project.id = '{PROJECT_ID}'
GROUP BY 1

Rule 3: NEVER Aggregate SUM(cost) Alongside LEFT JOIN UNNEST(credits)

If a row has 3 credit entries, LEFT JOIN UNNEST(credits) duplicates that row 3 times, tripling the SUM(cost). This is the most common cause of inflated cost reports.

-- WRONG: Inflates cost by N times (N = number of credits per row)
SELECT SUM(cost), SUM(credits.amount)
FROM `{BILLING_TABLE}` LEFT JOIN UNNEST(credits) AS credits

-- CORRECT: Subquery aggregates credits without duplicating cost rows
SELECT
  SUM(CAST(cost AS NUMERIC)) AS gross_cost,
  SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) c), 0)) AS total_credits
FROM `{BILLING_TABLE}`
WHERE project.id = '{PROJECT_ID}'

Note: LEFT JOIN UNNEST(credits) is safe when you are ONLY aggregating credits.amount and NOT also aggregating cost — e.g., when filtering by credit type. The danger is combining it with SUM(cost) in the same query.

Rule 4: Sanity-Check Costs Against Known GCP Pricing

Before reporting any cost figure, verify it's physically possible.

Step 1: Check the currency. GCP billing accounts can use ANY currency (USD, VND, EUR, BRL, JPY, etc.). Run gcp_billing_currency or check the currency column BEFORE interpreting any numbers. A value of 5,000,000 in VND (~$200 USD) is very different from 5,000,000 in USD.

Step 2: Verify magnitude against known pricing (in the account's currency):

Machine Type	Region	Monthly On-Demand Price (USD)
e2-micro	asia-southeast1	~$8/mo
e2-small	asia-southeast1	~$15/mo
e2-medium	asia-southeast1	~$30/mo
e2-standard-2	asia-southeast1	~$60/mo
n1-standard-2	asia-southeast1	~$60/mo (before SUD)
n2-standard-2	asia-southeast1	~$70/mo

Red flags that indicate a query error or currency mismatch:

• A single VM shows >$1,000 USD/week (e2-standard-2 costs ~$60/month)
• Total project cost exceeds $100,000 USD/week for a standard workload
• AI/API costs look 1,000x-25,000x higher than expected (likely VND/JPY reported as USD)

If numbers seem unreasonably high, STOP and verify the currency before reporting. Do NOT present cost numbers to users without confirming the currency unit.

Rule 5: Detect Credit Programs Before Alerting

If net_cost is consistently $0 (or near-zero) while gross_cost is large, the account is on a promotional credit program (free trial, startup credits, enterprise credits). This is normal — not an anomaly.

MANDATORY first query before any billing analysis:

-- Step 0: Detect credit program status
SELECT
  c.type AS credit_type,
  c.full_name AS credit_name,
  COUNT(*) AS line_items,
  SUM(CAST(c.amount AS NUMERIC)) AS total_credit_amount
FROM `{BILLING_TABLE}`, UNNEST(credits) AS c
WHERE DATE(usage_start_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
  AND project.id = '{PROJECT_ID}'
GROUP BY 1, 2
ORDER BY total_credit_amount ASC

Interpretation:

• PROMOTION credits with large amounts → account is on promotional/trial program
• SUSTAINED_USAGE_DISCOUNT → automatic discounts on N1/N2/N2D/C2/M1/M2 instances
• COMMITTED_USAGE_DISCOUNT → organization has CUD commitments
• FREE_TIER → usage within free tier limits

If PROMOTION credits fully offset costs: Report that the account is on a credit program. Do NOT generate alarmist alerts about "projected costs when credits expire" unless the user specifically asks for that analysis.

Rule 6: Detect Anomalies on NET Cost, Not Gross Cost

Gross cost fluctuates when credits are added/removed/adjusted. Only net cost reflects actual spending changes.

-- Pseudocode (not valid SQL — see Anomaly Detection query in BigQuery section for working version)

-- WRONG: Anomaly on gross cost → false positive from credit changes
HAVING SUM(cost) > threshold

-- CORRECT: Anomaly on net cost → real spending change
HAVING (SUM(cost) + SUM(credits_subquery)) > threshold

Rule 7: Verify Services Actually Exist Before Alerting

If billing data shows charges for a service (e.g., App Engine, Load Balancer) but gcloud commands show that service doesn't exist in the project, the charges are likely from another project in the same billing account that leaked into your unfiltered query. Go back to Rule 2 and add WHERE project.id = ....

Rule 8: Use NUMERIC Casting for Financial Precision

The cost and credits.amount fields are Float type. Summing millions of rows accumulates floating-point errors. Always cast:

SUM(CAST(cost AS NUMERIC)) -- not SUM(cost)

Rule 9: Understand invoice.month vs usage_start_time

• invoice.month (YYYYMM): The invoice this line item belongs to. Use for invoice reconciliation.
• usage_start_time: When usage actually occurred. Use for trend analysis.
• These can differ: late-reported usage from month N may appear on month N+1's invoice.
• See the Invoice Reconciliation query in the BigQuery Billing Export Patterns section for a working example.

Rule 10: Data Has 24-48 Hour Delay

Billing export data takes up to 24-48 hours to fully propagate. Do NOT alert on "missing data" for the current day or yesterday.

Rule 11: Always Detect and Report Currency

GCP billing accounts can be configured in any currency (USD, EUR, VND, BRL, JPY, GBP, etc.). The currency column in the billing export table identifies the billing currency. NEVER assume USD.

MANDATORY: Run gcp_billing_currency (or check SELECT DISTINCT currency FROM TABLE) as part of the first billing query. Include the currency in every cost report.

-- Check billing currency
SELECT DISTINCT currency FROM `{BILLING_TABLE}` WHERE project.id = '{PROJECT_ID}'

Reporting rules:

• ALWAYS include the currency code when presenting costs (e.g., "5,368,844 VND" not "$5,368,844")
• NEVER use $ symbol without confirming the currency is USD
• If the user needs USD conversion, state the approximate exchange rate used and note it may not be current
• Do NOT hardcode exchange rates in SQL queries. Report costs in their native currency and provide approximate USD equivalent separately if needed

Common non-USD currencies and approximate rates (for sanity-checking only):

• VND: ~25,000 VND = 1 USD (costs appear 25,000x larger than USD equivalent)
• JPY: ~150 JPY = 1 USD (costs appear 150x larger)
• EUR: ~0.92 EUR = 1 USD
• BRL: ~5 BRL = 1 USD

Mandatory Pre-Analysis Checklist

Before writing ANY billing query, verify ALL of the following:

• Currency has been detected via gcp_billing_currency or equivalent query
• Query includes WHERE project.id = '{PROJECT_ID}' filter
• Net cost is computed as SUM(CAST(cost AS NUMERIC)) + SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) c), 0))
• NO LEFT JOIN UNNEST(credits) used alongside SUM(cost)
• Cost fields use CAST(... AS NUMERIC) for precision
• Credit program detection query has been run FIRST
• Any per-resource cost is sanity-checked against known GCP pricing in the detected currency
• Anomaly detection uses net cost, not gross cost
• cost_type is considered (regular vs tax vs adjustment vs rounding_error)
• All cost figures in the report include the currency unit

CLI Tips

Parallel Execution Requirement (CRITICAL)

ALL independent operations MUST run in parallel using background jobs (&) and wait

ENFORCEMENT RULES:

• FORBIDDEN: Sequential loops like for item in $items; do cmd $item; done (causes O(n) runtime)
• MANDATORY: Every independent operation spawns a background job: { cmd1 } & { cmd2 } & { cmd3 } & wait
• DETECTION: If your script processes N resources/metrics/regions and N > 1, the script MUST contain at least N background jobs
• TIME IMPACT: Sequential execution with 30 instances x 2 seconds per call = 60 seconds. Parallel = 2 seconds (30x faster)
• VALIDATION CHECKLIST (agent must mentally verify before output):
  • Count independent operations: ___
  • Count background jobs (&): ___
  • These numbers MUST match, or script will be REJECTED
  • Do all operations depend on each other? (Only valid exception to parallel requirement)

PARALLEL PATTERN (CORRECT):

for instance in $instances; do
  operation "$instance" &  # ← Spawn as background job
done
wait  # ← Wait for all to complete

SEQUENTIAL PATTERN (FORBIDDEN - ONLY if operations have data dependencies):

result=$(operation1)
operation2 "$result"  # ← Only valid if operation2 requires operation1's output

Agent Output Rules

• The script output is for the agent itself to read and process, NOT for human reading
• Do NOT add visual formatting, icons, or decorative elements (no emojis, borders, or separators)
• NEVER USE echo statements for section breaks, headers, or formatting (no "--------", "====", or similar)
• Focus on raw data extraction and minimal, parseable output
• Use plain text format with consistent delimiters for easy parsing
• Prioritize machine-readability over human presentation
• NEVER run commands or scripts that print, log, or expose environment variables, credentials, or GCP keys (e.g., GOOGLE_APPLICATION_CREDENTIALS)

Execution Guidelines

• PARALLEL EXECUTION IS MANDATORY: Always use background jobs (&) and wait for independent operations
  • Process multiple instances/resources in parallel: { ... } & with wait at the end
  • Fetch multiple metrics for the same resource in parallel, then wait before processing
  • Sequential loops are FORBIDDEN unless operations have strict dependencies
  • Parallel execution reduces runtime from O(n * time_per_operation) to O(max_operation_time) - use it always
• GCP region and project ID are already preconfigured in the environment - no need to set them manually
• Always consolidate related steps into single CLI Bash script if possible
• Only use read-only commands (e.g., list, describe, get) - never modify resources
• Always format CLI output as plain text (never JSON or table) so that it's easy for the agent to parse. For GCP use --format=text or --format="value(...)" as appropriate. Also use filtering/query flags to limit output to what is needed. These practices are crucial for efficiency and accuracy.
• NON-INTERACTIVE MODE: Use --quiet (-q) flag or export CLOUDSDK_CORE_DISABLE_PROMPTS=1 to disable prompts in scripts

Filtering Guidance

FILTERING ORDER MATTERS - Understand server-side vs client-side

1. --filter (VARIES BY COMMAND) - Can be server-side OR client-side

   • Some commands send filter to API (server-side) → reduces network payload
   • Other commands filter locally (client-side) → full data still transferred
   • Use --log-http to verify: if filter appears in API request, it's server-side
   • Server-side is MORE efficient for large datasets

2. --format (ALWAYS CLIENT-SIDE) - Formatting after data retrieval

   • Use --format="value(...)" for clean, parseable output
   • Use projections to select specific fields
   • Always applied AFTER --filter

PERFORMANCE IMPACT:

• Server-side --filter: API returns only matching records
• Client-side --filter: API returns ALL records, filtered locally
• Use --log-http to check which mode your command uses

EXAMPLES:

# Efficient: --filter with --format for minimal output
gcloud compute instances list --filter="status=RUNNING" \
    --format="value(name,zone.scope(zones),machineType.scope(machineTypes))"

# Verify if filter is server-side (look for filter in HTTP request)
gcloud compute instances list --filter="status=RUNNING" --log-http 2>&1 | grep -i filter

# Multiple filter conditions
gcloud compute instances list \
    --filter="status=RUNNING AND machineType~n1-standard" \
    --format="value(name,zone)"

COMMON FILTER OPERATORS:

• = exact match, != not equal, ~ regex match, !~ regex not match
• : substring match (HAS operator)
• >, >=, <, <= for comparisons
• AND, OR, NOT for boolean logic

Pagination Guidelines

PAGINATION FOR LARGE DATASETS - Prevent timeouts and memory issues

KEY PARAMETERS:

• --limit=N: Maximum total items to return (stops early)
• --page-size=N: Items per API call (internal pagination, still returns all unless --limit set)
• --sort-by=FIELD: Sort results (prefix with ~ for descending)

ORDER OF OPERATIONS (gcloud applies in this order):

1. --flatten → 2. --sort-by → 3. --filter → 4. --limit

EXAMPLES:

# Get first 10 instances only
gcloud compute instances list --limit=10 --format="value(name,zone)"

# Paginate with smaller chunks (memory efficiency)
gcloud compute instances list --page-size=50 --limit=200 \
    --format="value(name,zone)"

# Sort by creation time, newest first
gcloud compute instances list --sort-by=~creationTimestamp --limit=5 \
    --format="value(name,creationTimestamp)"

BEST PRACTICE: Combine --filter + --limit to minimize data transfer:

# Filter server-side, limit client-side
gcloud compute instances list --filter="status=RUNNING" --limit=100 \
    --format="value(name,zone)"

Format Projections

USEFUL PROJECTION FUNCTIONS - Reduce post-processing with built-in transforms

EXTRACTION FUNCTIONS:

• .scope(segment) - Extract last URL segment (e.g., zone name from full URL)
• .basename() - Get filename from path
• .segment(n) - Get nth segment from URL

DATE/TIME FUNCTIONS:

• .date(format) - Format timestamp (e.g., .date('%Y-%m-%d'))
• .date(tz=LOCAL) - Convert to local timezone

STRING FUNCTIONS:

• .yesno(yes, no) - Convert boolean to custom strings
• .list() - Format as comma-separated list

EXAMPLES:

# Extract zone name from full URL
gcloud compute instances list \
    --format="value(name,zone.scope(zones),machineType.scope(machineTypes))"
# Output: my-vm  us-central1-a  n1-standard-1

# Format creation date
gcloud compute instances list \
    --format="table(name,creationTimestamp.date('%Y-%m-%d'),status)"

# Boolean formatting
gcloud compute instances list \
    --format="value(name,scheduling.preemptible.yesno('preemptible','on-demand'))"

# Get single value (no headers)
gcloud config get-value project --format="value(.)"

REFERENCE: Run gcloud topic projections for full documentation

Efficient CLI Script Example

ANTI-PATTERN EXAMPLE (SEQUENTIAL - SLOW - UNACCEPTABLE)

#!/bin/bash
# RUNTIME: ~60 seconds for 30 instances (2 sec per call x 30)
END_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
START_TIME=$(date -u -d "30 days ago" +"%Y-%m-%dT%H:%M:%SZ")

echo "GCP VM Metrics Summary ($START_TIME to $END_TIME)"

PROJECT_ID=$(gcloud config get-value project)
echo "Project: $PROJECT_ID"

# Get zones and instances
gcloud compute zones list --format="value(name)" | while read zone; do
    instances=$(gcloud compute instances list --zones="$zone" --format="value(name)" --project="$PROJECT_ID")
    if [ -n "$instances" ]; then
        echo "Zone: $zone"

        # This SEQUENTIAL loop is FORBIDDEN
        echo "$instances" | while read instance_name; do
            echo "  Instance: $instance_name"

            # Sequential metric fetches - UNACCEPTABLE
            gcloud monitoring time-series list \
                --filter="resource.labels.project_id='$PROJECT_ID' AND resource.labels.zone='$zone' AND resource.labels.instance_id='$instance_name' AND metric.type='compute.googleapis.com/instance/cpu/utilization'" \
                --interval.start-time="$START_TIME" \
                --interval.end-time="$END_TIME" \
                --aggregation.alignment-period=3600s \
                --aggregation.per-series-aligner="ALIGN_MEAN" \
                --format="value(points[].value.doubleValue)" \
                --project="$PROJECT_ID"

            gcloud monitoring time-series list \
                --filter="resource.labels.project_id='$PROJECT_ID' AND resource.labels.zone='$zone' AND resource.labels.instance_id='$instance_name' AND metric.type='compute.googleapis.com/instance/network/received_bytes_count'" \
                --interval.start-time="$START_TIME" \
                --interval.end-time="$END_TIME" \
                --aggregation.alignment-period=3600s \
                --aggregation.per-series-aligner="ALIGN_RATE" \
                --format="value(points[].value.doubleValue)" \
                --project="$PROJECT_ID"
        done
    fi
done
# TOTAL TIME: ~60 seconds (UNACCEPTABLE for 30+ instances)

CORRECT EXAMPLE (PARALLEL - FAST - REQUIRED)

#!/bin/bash
# RUNTIME: ~2 seconds for 30 instances (all run simultaneously)
END_TIME=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
START_TIME=$(date -u -d "30 days ago" +"%Y-%m-%dT%H:%M:%SZ")
PROJECT_ID=$(gcloud config get-value project 2>/dev/null)

# Fetch a single metric (called in parallel)
get_metric() {
    local instance=$1 zone=$2 metric=$3 aligner=$4
    gcloud monitoring time-series list \
        --filter="resource.labels.instance_id='$instance' AND metric.type='$metric'" \
        --interval.start-time="$START_TIME" --interval.end-time="$END_TIME" \
        --aggregation.alignment-period=3600s \
        --aggregation.per-series-aligner="$aligner" \
        --format="value(points[].value.doubleValue)" \
        --project="$PROJECT_ID" \
        | awk -v i="$instance" -v m="$metric" '{sum+=$1; count++} END {if(count>0) printf "%s\t%s\t%.2f\n", i, m, sum/count}'
}

# Process one instance: fetch multiple metrics in parallel
process_instance() {
    local instance=$1 zone=$2
    get_metric "$instance" "$zone" "compute.googleapis.com/instance/cpu/utilization" "ALIGN_MEAN" &
    get_metric "$instance" "$zone" "compute.googleapis.com/instance/network/received_bytes_count" "ALIGN_RATE" &
    wait  # Wait for all metrics of this instance
}

# Process ALL instances in parallel
instances=$(gcloud compute instances list --format="value(name,zone.scope(zones))" --project="$PROJECT_ID")
echo "$instances" | while read instance zone; do
    process_instance "$instance" "$zone" &
done
wait  # Wait for all instances to complete

PERFORMANCE COMPARISON TABLE

Pattern	Instances	Time/Call	Total Time	Speed
Sequential (❌)	30	2 sec	~60 sec	Baseline
Parallel (✅)	30	2 sec	~2 sec	30x faster
Sequential (❌)	100	2 sec	~200 sec	Baseline
Parallel (✅)	100	2 sec	~2 sec	100x faster

KEY DIFFERENCES IN THIS SCRIPT (What Makes It Parallel)

• In the main loop: process_instance "$instance" "$zone" & - spawns each instance as background job
• After the loop: wait - waits for all instances to complete
• Inside process_instance: metrics are fetched in parallel (with & and inner wait)
• Uses --format="value(name,zone.scope(zones))" for efficient extraction

VALIDATION CHECKLIST FOR AGENT Before outputting ANY script, check every item:

• Count number of independent resources/metrics/zones to process: ___
• Count number of & background job spawns in script: ___
• If these counts don't match, the script is WRONG - REJECT it and rewrite
• Verify each background job block is followed by a wait statement
• Check that NO sequential loops exist for independent operations
• Confirm expected runtime is ~2-10 seconds (not ~30+ seconds)
• Verify GCP monitoring aligners are correct (ALIGN_MEAN, ALIGN_MAX, ALIGN_RATE, etc.)
• Confirm script uses --format=text or --format="value(...)" with proper filtering

Common GCP CLI Patterns

• List all projects: gcloud projects list --format="value(projectId,name,lifecycleState)"
• List compute instances: gcloud compute instances list --format="value(name,zone,machineType.scope(machineTypes),status)"
• Get instance details: gcloud compute instances describe myInstance --zone=us-central1-a --format="value(name,machineType,status,scheduling.preemptible)"
• List Cloud Storage buckets: gcloud storage buckets list --format="value(name,location,storageClass)"
• List Cloud SQL instances: gcloud sql instances list --format="value(name,databaseVersion,region,tier,state)"
• Get current project (preconfigured): gcloud config get-value project --format="value(.)"
• List App Engine services: gcloud app services list --format="value(id,split.allocations.keys())"
• List Cloud Functions: gcloud functions list --format="value(name,status,trigger.eventTrigger.eventType)"
• Get billing info: gcloud billing accounts list --format="value(name,displayName,open)"
• List Kubernetes clusters: gcloud container clusters list --format="value(name,location,status,currentMasterVersion)"

GCP Service Naming Patterns

• Compute Engine: e2-micro, n1-standard-1, c2-standard-4, n2-highmem-2
• Cloud Storage: STANDARD, NEARLINE, COLDLINE, ARCHIVE
• Cloud SQL: db-f1-micro, db-n1-standard-1, db-n1-highmem-2
• App Engine: F1, F2, F4, F4_1G (for automatic scaling)
• Cloud Functions: Various memory sizes (128MB, 256MB, 512MB, 1GB, 2GB, 4GB, 8GB)

Billing CLI Commands

BILLING ACCOUNT & BUDGET MANAGEMENT - gcloud billing commands

LIST BILLING ACCOUNTS (accounts you have access to):

# List all billing accounts with key fields
gcloud billing accounts list --format="value(name,displayName,open)"

# Get billing account ID only (for scripting)
gcloud billing accounts list --format="value(name)" --filter="open=true"

DESCRIBE BILLING ACCOUNT:

# Get account details (check if sub-account)
gcloud billing accounts describe BILLING_ACCOUNT_ID \
    --format="value(displayName,masterBillingAccount,open)"
# If masterBillingAccount is set, this is a reseller sub-account

LIST PROJECTS UNDER BILLING ACCOUNT:

# List all projects linked to a billing account
gcloud billing projects list --billing-account=BILLING_ACCOUNT_ID \
    --format="value(projectId,billingEnabled)"

# Filter to only enabled projects
gcloud billing projects list --billing-account=BILLING_ACCOUNT_ID \
    --filter="billingEnabled=true" --format="value(projectId)"

CHECK PROJECT BILLING STATUS:

# Check if specific project has billing enabled
gcloud billing projects describe PROJECT_ID --format="value(billingEnabled)"
# Returns: True or False

# Get billing account linked to project
gcloud billing projects describe PROJECT_ID \
    --format="value(billingAccountName)"

BUDGET MANAGEMENT:

# List all budgets for a billing account
gcloud billing budgets list --billing-account=BILLING_ACCOUNT_ID \
    --format="value(displayName,amount.specifiedAmount.units,amount.specifiedAmount.currencyCode)"

# Describe specific budget details
gcloud billing budgets describe BUDGET_ID --billing-account=BILLING_ACCOUNT_ID \
    --format="value(displayName,amount,budgetFilter,thresholdRules)"

PARALLEL PATTERN FOR MULTI-ACCOUNT ANALYSIS:

# Fetch billing info for multiple accounts in parallel
accounts=$(gcloud billing accounts list --format="value(name)" --filter="open=true")
for account in $accounts; do
    {
        projects=$(gcloud billing projects list --billing-account="$account" \
            --format="value(projectId)" --filter="billingEnabled=true")
        echo "$account: $(echo "$projects" | wc -l) projects"
    } &
done
wait

VAT/Tax Handling

CRITICAL: VAT/TAX AWARENESS - Why your costs may differ from Console

IMPORTANT: GCP Pricing API and BigQuery billing exports return TAX-EXCLUSIVE (pre-tax) prices. The GCP Console dashboard shows TOTAL costs INCLUDING taxes (VAT, GST, sales tax, etc.).

THIS CAUSES DISCREPANCIES between API results and what customers see in their Console!

HOW GCP HANDLES TAXES:

1. Pricing API: Returns base list prices WITHOUT tax
2. BigQuery Billing Export: cost column is PRE-TAX; taxes are separate rows with cost_type = "tax"
3. Console Dashboard: Shows aggregated totals WITH taxes included
4. Invoices: Show taxes as separate line items by project

TAX RATES BY REGION (examples - varies by location and changes over time):

• EU countries: 19-27% VAT (varies by country)
• UK: 20% VAT
• Australia: 10% GST
• India: 18% GST
• Canada: 5-15% (varies by province)
• Bahrain: 10% VAT (since Feb 2022)
• US: State sales tax varies (0-10%+)

TO GET TAX-INCLUSIVE TOTALS FROM BIGQUERY:

-- Get total cost INCLUDING taxes (matches Console dashboard)
-- Note: project.id filter required per Rule 2
SELECT
  invoice.month AS invoice_month,
  SUM(CASE WHEN cost_type != 'tax' THEN CAST(cost AS NUMERIC) ELSE 0 END) AS pre_tax_cost,
  SUM(CASE WHEN cost_type = 'tax' THEN CAST(cost AS NUMERIC) ELSE 0 END) AS tax_amount,
  SUM(CAST(cost AS NUMERIC))
    + SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) c), 0)) AS net_cost_with_tax
FROM `{BILLING_TABLE}`
WHERE project.id = '{PROJECT_ID}'
  AND invoice.month = FORMAT_DATE('%Y%m', CURRENT_DATE())
GROUP BY 1

-- Get tax breakdown by project (intentionally no project.id filter: multi-project breakdown)
SELECT
  project.id AS project_id,
  SUM(CASE WHEN cost_type != 'tax' THEN CAST(cost AS NUMERIC) ELSE 0 END) AS usage_cost,
  SUM(CASE WHEN cost_type = 'tax' THEN CAST(cost AS NUMERIC) ELSE 0 END) AS tax_cost,
  SUM(CAST(cost AS NUMERIC))
    + SUM(IFNULL((SELECT SUM(CAST(c.amount AS NUMERIC)) FROM UNNEST(credits) c), 0)) AS net_total_with_tax
FROM `{BILLING_TABLE}`
WHERE DATE(usage_start_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY 1
ORDER BY net_total_with_tax DESC

-- Get tax types (VAT, GST, sales tax, etc.)
SELECT
  sku.description AS tax_type,
  SUM(CAST(cost AS NUMERIC)) AS tax_amount
FROM `{BILLING_TABLE}`
WHERE project.id = '{PROJECT_ID}'
  AND cost_type = 'tax'
  AND DATE(usage_start_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY 1
ORDER BY 2 DESC

WHEN REPORTING COSTS TO USERS:

• ALWAYS clarify whether costs are pre-tax or post-tax
• If comparing to Console, include taxes: "Total cost (including VAT): $X"
• For budgeting, use tax-inclusive figures to match what they'll be invoiced
• Include tax breakdown when relevant: "Usage: $X + VAT: $Y = Total: $Z"

COST COMPARISON FORMULA (pseudocode — use CAST/UNNEST subquery pattern in actual SQL):

Console Total = Usage Cost + Credits + Taxes
              = SUM(cost WHERE cost_type='regular')
              + SUM(credits.amount)    -- credits are negative
              + SUM(cost WHERE cost_type='tax')
              = SUM(cost) + SUM(credits.amount)   -- simplified: all cost_types

Pricing Script (get_pricing_gcp.sh)

DO NOT read or modify the script file. Only source and call the function.

SETUP (at the start of your script):

source ./_skills/connections/gcp/gcp/scripts/get_pricing_gcp.sh

FUNCTION: get_gcp_cost RESOURCE REGION

Auto-detects the GCP service from the resource name prefix and returns on-demand pricing in TOON format.

NOTE: All prices are TAX-EXCLUSIVE. See VAT/Tax Handling for tax-inclusive calculations.

Supported resource prefixes:

Prefix	Service	Example
e2-, n1-, n2-, c2-, c3-	Compute Engine	e2-standard-2
cloudsql-, db-	Cloud SQL	cloudsql-db-n1-standard-2
gcs-	Cloud Storage	gcs-standard
functions-	Cloud Functions	functions-256mb
cloudrun-	Cloud Run	cloudrun-1cpu-512mb
redis-	Memorystore	redis-basic-1gb
bq-	BigQuery	bq-ondemand
pd-	Persistent Disk	pd-ssd-100gb
lb-	Load Balancer	lb-forwarding-rule
cloudnat-	Cloud NAT	cloudnat-standard
gke-	GKE	gke-standard

Compute Engine detail: GCP bills vCPU and RAM separately. The script has a built-in machine spec table and queries both Core and Ram SKUs, combining them into a total hourly/monthly estimate.

Examples:

source ./_skills/connections/gcp/gcp/scripts/get_pricing_gcp.sh
get_gcp_cost e2-standard-2 asia-southeast1
get_gcp_cost n2-standard-4 us-central1
get_gcp_cost gcs-standard us-central1
get_gcp_cost cloudsql-db-n1-standard-2 asia-southeast1

Monitoring and Metrics

COMMON METRICS:

• CPU utilization: compute.googleapis.com/instance/cpu/utilization
• Network traffic: compute.googleapis.com/instance/network/received_bytes_count, sent_bytes_count
• Disk I/O: compute.googleapis.com/instance/disk/read_bytes_count, write_bytes_count
• Memory usage (with monitoring agent): agent.googleapis.com/memory/percent_used

ALIGNERS (per-series-aligner):

• ALIGN_MEAN: Average value over alignment period (use for utilization metrics)
• ALIGN_MAX: Maximum value (use for peak detection)
• ALIGN_MIN: Minimum value
• ALIGN_RATE: Rate of change (use for counter metrics like bytes_count)
• ALIGN_SUM: Sum of values (use for uptime, request counts)

CRITICAL: alignment-period MUST be >= 60 seconds If you specify a per-series-aligner other than ALIGN_NONE, alignment-period is REQUIRED and must be at least 60 seconds.

CROSS-SERIES REDUCERS (aggregate across multiple resources):

• REDUCE_MEAN: Average across all time series
• REDUCE_MAX: Maximum across all time series
• REDUCE_SUM: Sum across all time series
• REDUCE_COUNT: Count of time series

CROSS-SERIES EXAMPLE (aggregate CPU across all instances in a zone):

gcloud monitoring time-series list \
    --filter="metric.type='compute.googleapis.com/instance/cpu/utilization'" \
    --aggregation.alignment-period=3600s \
    --aggregation.per-series-aligner=ALIGN_MEAN \
    --aggregation.cross-series-reducer=REDUCE_MEAN \
    --aggregation.group-by-fields="resource.labels.zone" \
    --format="value(points[].value.doubleValue)"

Billing Script (get_billing_gcp.sh)

DO NOT read or modify the script file. Only source and call the functions.

SETUP (at the start of your script):

source ./_skills/connections/gcp/gcp/scripts/get_billing_gcp.sh

All functions enforce anti-hallucination rules: WHERE project.id filter, CAST(... AS NUMERIC) on financial fields, net cost via UNNEST subquery (never LEFT JOIN UNNEST + SUM(cost)), cost_type = 'regular' where appropriate. Output is TOON format (tab-separated).

TABLE NAMING CONVENTION:

• Standard usage: dataset.gcp_billing_export_v1_<BILLING_ACCOUNT_ID_NO_DASHES>
• Detailed usage: dataset.gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID_NO_DASHES>
• Billing Account ID 012ABC-456DEF-789GHI becomes table suffix 012ABC456DEF789GHI (dashes removed)

FUNCTION REFERENCE:

Function	Purpose	Signature
gcp_billing_currency	Detect billing currency (MANDATORY first)	TABLE PROJECT_ID
gcp_billing_credits	Credit program detection (MANDATORY second)	TABLE PROJECT_ID
gcp_billing_summary	Top services by net cost	TABLE PROJECT_ID [--days N]
gcp_billing_trend	Daily net cost trend	TABLE PROJECT_ID [--days N]
gcp_billing_anomalies	Z-score anomaly on net cost	TABLE PROJECT_ID
gcp_billing_by_resource	Resource-level breakdown (detailed export)	TABLE PROJECT_ID [--days N]
gcp_billing_by_sku	SKU-level breakdown	TABLE PROJECT_ID [--days N]
gcp_billing_invoice	Invoice reconciliation (uses invoice.month)	TABLE PROJECT_ID [--month YYYYMM]
gcp_billing_compare	Multi-project comparison (no project filter)	TABLE [--days N]

MANDATORY WORKFLOW (every billing analysis):

1. Always run gcp_billing_currency first to detect the billing currency (see Rule 11)
2. Always run gcp_billing_credits second to detect credit programs
3. Interpret credit_coverage_ratio: close to -1.0 = fully covered by credits (do NOT alarm); -0.3 to -0.01 = partial discounts (normal); ~0 = minimal credits
4. Then run gcp_billing_summary or other functions as needed
5. Sanity-check costs against known GCP pricing in the detected currency (see Rule 4)

Examples:

source ./_skills/connections/gcp/gcp/scripts/get_billing_gcp.sh
TABLE="dataset.gcp_billing_export_v1_012ABC456DEF789GHI"
PROJECT="my-project-id"

# Step 0a: Detect billing currency (MANDATORY - see Rule 11)
gcp_billing_currency "$TABLE" "$PROJECT"

# Step 0b: Detect credit programs (MANDATORY)
gcp_billing_credits "$TABLE" "$PROJECT"

# Top services by net cost (last 30 days)
gcp_billing_summary "$TABLE" "$PROJECT" --days 30

# Daily trend
gcp_billing_trend "$TABLE" "$PROJECT" --days 14

# Anomaly detection
gcp_billing_anomalies "$TABLE" "$PROJECT"

# Resource-level breakdown (requires detailed export table)
gcp_billing_by_resource "$TABLE" "$PROJECT" --days 7

# SKU-level breakdown
gcp_billing_by_sku "$TABLE" "$PROJECT" --days 7

# Invoice reconciliation (current month)
gcp_billing_invoice "$TABLE" "$PROJECT"

# Invoice reconciliation (specific month)
gcp_billing_invoice "$TABLE" "$PROJECT" --month 202601

# Multi-project comparison (no project filter)
gcp_billing_compare "$TABLE" --days 7

BILLING EXPORT SCHEMA REFERENCE (key columns):

Column	Description
currency	ISO 4217 currency code (USD, VND, EUR, etc.). Check this FIRST — never assume USD.
cost	Usage cost at contract/on-demand rate, BEFORE credits. NOT your actual bill.
cost_at_list	Cost at public list price (before negotiated discounts).
credits	Array of credit entries. Each has type, amount (always negative), full_name.
credits.type	PROMOTION, SUSTAINED_USAGE_DISCOUNT, COMMITTED_USAGE_DISCOUNT, FREE_TIER, DISCOUNT, etc.
cost_type	"regular", "tax", "adjustment", or "rounding_error".
project.id	GCP project ID. ALWAYS filter by this.
invoice.month	YYYYMM string. Use for invoice reconciliation.
usage_start_time	When usage occurred. Use for trend analysis.
resource.name	Resource identifier (detailed export only).

CREDIT TYPES:

Credit Type	Meaning	Typical Coverage
PROMOTION	Trial credits, startup credits, enterprise promotional	Can be 100% (full offset)
SUSTAINED_USAGE_DISCOUNT	Auto-discount for N1/N2/N2D/C2/M1/M2 running >25% of month	Up to 30% for N1
COMMITTED_USAGE_DISCOUNT	Resource-based CUD commitment	37-55% depending on term
COMMITTED_USAGE_DISCOUNT_DOLLAR_BASE	Spend-based CUD commitment	Varies by commitment
FREE_TIER	Always-free tier usage	Small amounts
DISCOUNT	Other negotiated discounts	Varies
RESELLER_MARGIN	Reseller margin credits	Varies

NOTE on SUDs: E2 machine types are NOT eligible for Sustained Use Discounts. Only N1, N2, N2D, C2, M1, M2 families receive SUDs.

gcloud Topic References

BUILT-IN HELP - Learn filter/format/projection syntax

gcloud topic filters      # Filter expression syntax and operators
gcloud topic formats      # Output format options and projections
gcloud topic projections  # Projection functions (.scope(), .date(), etc.)

Parallel vs Sequential Rules Summary

PARALLEL vs SEQUENTIAL - Quick Reference

• ALWAYS PARALLEL: Multiple instances, metrics, zones, or projects
• ONLY SEQUENTIAL: When operation B requires output from operation A
• PATTERN: for item in $items; do operation "$item" & done; wait

FORBIDDEN ANTI-PATTERNS (see detailed examples above):

• ❌ Sequential loops: for x in $list; do cmd; done → use & done; wait
• ❌ Sequential commands: result1=$(cmd1); result2=$(cmd2) → use cmd1 & cmd2 & wait
• ❌ Individual describe calls when list is available

Output Format

Present results as a structured report:

Gcp Report
══════════
Resources discovered: [count]

Resource       Status    Key Metric    Issues
──────────────────────────────────────────────
[name]         [ok/warn] [value]       [findings]

Summary: [total] resources | [ok] healthy | [warn] warnings | [crit] critical
Action Items: [list of prioritized findings]

Target ≤50 lines of output. Use tables for multi-resource comparisons.

Counter-Rationalizations

Shortcut	Counter	Why
"I'll skip discovery and check known resources"	Always run Phase 1 discovery first	Resource names change, new resources appear — assumed names cause errors
"The user only asked for a quick check"	Follow the full discovery → analysis flow	Quick checks miss critical issues; structured analysis catches silent failures
"Default configuration is probably fine"	Audit configuration explicitly	Defaults often leave logging, security, and optimization features disabled
"Metrics aren't needed for this"	Always check relevant metrics when available	API/CLI responses show current state; metrics reveal trends and intermittent issues
"I don't have access to that"	Try the command and report the actual error	Assumed permission failures prevent useful investigation; actual errors are informative