waza-docs

description	Reference and links to the official Microsoft/waza documentation. You MUST load this skill when you need detailed guides, best practices, or reference documentation for the Waza tool.
license	MIT
name	waza-docs

Waza Docs

WHEN TO USE

• When configuring or troubleshooting Waza implementations.
• When creating SKILL assessments and evaluating agents using Waza.
• When you need reference material such as Graders, CI/CD integration, or telemetry documentation.

References

To fully utilize this skill, you MUST read at least one of the links relevant to the current context:

• CI/CD Integration Guide This guide explains how to integrate waza evaluations into your CI/CD pipelines using GitHub Actions, Azure DevOps, and other platforms.

• Waza Demo Guide Comprehensive walk-through for demonstrating waza's capabilities. Provides step-by-step instructions for 9 practical demonstrations.

• Getting Started with Waza A complete walkthrough for creating, testing, and validating AI agent skills.

• Waza User Guide A CLI tool for evaluating AI agent skills, including installation, creating evaluations, running benchmarks, and reviewing results.

• Integration Testing with Copilot SDK This guide explains how to run real integration tests using the GitHub Copilot SDK.

• Waza Skills Development Platform - PRD Product Requirements Document for the Waza Skills Development Platform.

• Release Process This document describes how the Waza release process works.

• Skill Development Best Practices Guidance for creating high-quality SKILL.md files that integrate well with waza evaluations and follow industry best practices.

• Waza Integration for microsoft/skills This guide explains how to use waza for evaluating skills in the microsoft/skills repository.

• Runtime Telemetry Capture and analyze metrics from skills running in production.

• Token Limits Configuration Reference for resolving token limits in configuration, such as .waza.yaml.

• Waza Project Configuration Schema JSON Schema for validating .waza.yaml project configuration files.

• Advisory Checks Logic for advisory Waza checks and compliance patterns for SKILL.md files.

• Writing Skill Evals - Tutorial This tutorial walks you through creating evaluations for your Agent Skills.

Design

• A/B Baseline Skill Impact Measurement Design document for A/B Skill Impact Measurement.

Grader Reference

Complete reference for all available grader types in waza.

• Grader Reference Overview Complete reference for all available grader types in waza.

• action_sequence Tool Call Sequence Validation. Validates that the agent's tool calls match an expected action sequence.

• behavior Agent Behavior Validation. Validates agent behavior patterns like tool call counts, token usage, and execution duration.

• code Assertion-Based Grader. Evaluates expressions against the execution context using an inline script runner.

• diff Workspace File Comparison. Compares post-execution workspace files against expected snapshots and/or line fragments.

• file File Existence & Content Grader. Validates file existence, absence, and content patterns in the post-execution workspace.

• human_calibration Calibration Grader. Collects human labels to calibrate LLM graders.

• human Manual Review Grader. Marks tasks for human review.

• json_schema JSON Schema Validation Grader. Validates that the agent output is valid JSON conforming to a given JSON schema.

• llm_comparison Reference Comparison Grader. Compares output against a reference using LLM.

• llm LLM-as-Judge Grader. Uses an AI model to evaluate quality.

• External Program Grader. Runs an external program or script to grade agent output.

Plans

• MCP Server Design Document Design document for the MCP Server implementation.