a365-code-validator

name	a365-code-validator
description	Validates and optionally fixes Agent 365 observability code so an agent actually emits MAC Activity telemetry. Use when an Agent 365 agent's telemetry isn't reaching MAC Activity or Defender, when spans export (HTTP 200 / "sent") but Activity stays empty, when the caller or agent id looks wrong in reporting, or to pre-flight whether an agent will emit MAC Activity before shipping. Diagnoses and fixes existing instrumentation; do NOT use it to add observability from scratch — use instrument-observability for that. Checks exporter activation, runtime agent-identity binding (vs blueprint id), the S2S FMI / OBO token shape, S2S vs OBO endpoint selection, the required semantic spans (invoke_agent/chat/execute_tool/output_messages), Activity attributes, and live blueprint grants and inheritance through read-only a365 Microsoft Graph diagnostics. Read-only by default; asks before applying safe fixes. Python, Node.js, and .NET.
compatibility	["claude-code","vscode-copilot","github-copilot-cli"]
user-invocable	true
argument-hint	Optional: path to agent project, or agent id / blueprint id to include in the report
allowed-tools	Read, Write, Edit, Glob, Grep, Bash, AskUserQuestion, TaskCreate, TaskUpdate, TaskList
model	sonnet
hooks	{"preToolUse":[{"type":"command","command":"node ${CLAUDE_PLUGIN_ROOT}/hooks/preToolUse/path-guard.js","timeout":5000}],"stop":[{"type":"command","command":"node ${CLAUDE_PLUGIN_ROOT}/hooks/stop/validate-a365-code-validator.js","timeout":15000},{"type":"prompt","prompt":"Verify the validation result was presented to the user with:\n1. exporter activation status;\n2. identity binding status (agent id vs blueprint id);\n3. semantic span coverage (invoke_agent/chat/execute_tool/output_messages);\n4. endpoint/token mode (S2S vs OBO);\n5. live blueprint grant and effective-inheritance status, or why the check was skipped;\n6. concrete next debug commands;\n7. if blockers were found, the user was asked whether to fix now, create a fix plan, or stop.\nReturn {\"ok\": false, \"reason\": \"<missing item>\"} if any item is absent.\nOtherwise return {\"ok\": true}.\n","timeout":30000}]}

A365 Code Validator

Trigger phrases — any of these will activate this skill automatically:

• "validate a365 code"
• "run a365 code validator"
• "check a365 observability code"
• "debug a365 activity missing"
• "debug MAC activity for this agent"
• "validate Agent 365 telemetry"
• "check why Agent Activity is empty"
• "check A365 exporter flags"
• "validate gen_ai agent id"
• "check A365 span types"

---

Overview

This skill performs a report-first validation of an existing agent codebase to answer:

"If this agent runs now, will its spans reach the A365 backend and become eligible for MAC Activity?"

It is designed for incident response. It checks the exact classes of bugs that cause silent or confusing A365 Activity failures:

1. Exporter configured but not actually enabled.
2. gen_ai.agent.id set to a Blueprint ID instead of the runtime Agent Identity / Source Agent ID.
3. Generic HTTP/OpenAI spans emitted without A365 semantic operations.
4. S2S/OBO endpoint mismatch.
5. Missing token resolver or wrong S2S token shape.
6. Stale local Blueprint ID or missing Blueprint permissions in the target tenant.
7. Inheritable permission policy configured without grants on the Blueprint service principal.
8. MAC reporting expectations confused with raw ingest success.

No source code is modified during validation. After the report, the skill asks whether to apply safe fixes, create a fix plan only, or stop.

---

Output is support-safe (applies to every phase)

This report may be pasted into a customer or partner thread, so it must be safe to share: never print internal cluster/database names, Kusto queries, private endpoint or route templates, correlation IDs, or real tenant/agent IDs. Report file/function names and public, supportable signals only. This is non-negotiable — it holds even when a user asks for "just the raw details."

---

Phase 0 — Create and Display Task List

Show the user this checklist BEFORE Phase 1. Exactly one task is in progress at a time. Claude Code uses TaskCreate / TaskUpdate; VS Code Copilot Chat / GitHub Copilot CLI should show and update a markdown checklist.

TaskCreate: "Detect stack and A365 artifacts"
TaskCreate: "Run static A365 code validator"
TaskCreate: "Inspect identity binding and semantic spans"
TaskCreate: "Check live blueprint permissions and runtime state"
TaskCreate: "Summarize blockers and next fixes"
TaskCreate: "Offer guided remediation"