// Retrieve and analyze Azure DevOps build failures for GitHub PRs. Use when CI fails. CRITICAL: Collect ALL errors from ALL platforms FIRST, write hypotheses to file, then fix systematically.
| name | pr-build-status |
| description | Retrieve and analyze Azure DevOps build failures for GitHub PRs. Use when CI fails. CRITICAL: Collect ALL errors from ALL platforms FIRST, write hypotheses to file, then fix systematically. |
| metadata | {"author":"dotnet-maui","version":"2.0"} |
| compatibility | Requires GitHub CLI (gh) authenticated with access to dotnet/fsharp repository. |
Retrieve and systematically analyze Azure DevOps build failures for GitHub PRs.
DO NOT push fixes until ALL errors are collected and reproduced locally.
LLMs tend to focus on the first error found and ignore others. This causes:
1. COLLECT ALL → Get errors from ALL jobs across ALL platforms
2. DOCUMENT → Write CI_ERRORS.md with hypotheses per platform
3. REPRODUCE → Run each failing test LOCALLY (in isolation!)
4. FIX → Fix each issue, verify locally
5. PUSH → Only after ALL issues verified fixed
All scripts are in .github/skills/pr-build-status/scripts/
pwsh .github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 -PrNumber <PR_NUMBER>
# Get overview of all stages and jobs
pwsh .github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 -BuildId <BUILD_ID>
# Get ONLY failed jobs (use this to see all failing platforms)
pwsh .github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 -BuildId <BUILD_ID> -FailedOnly
# Get ALL errors (build errors + test failures) - USE THIS FIRST
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID>
# Filter to specific job (after getting overview)
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID> -JobFilter "*Linux*"
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID> -JobFilter "*Windows*"
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID> -JobFilter "*MacOS*"
# Get timeline with all jobs
$uri = "https://dev.azure.com/dnceng-public/public/_apis/build/builds/<BUILD_ID>/timeline?api-version=7.1"
Invoke-RestMethod -Uri $uri | Select-Object -ExpandProperty records | Where-Object { $_.result -eq "failed" }
# Get specific log content
$logUri = "https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_apis/build/builds/<BUILD_ID>/logs/<LOG_ID>"
Invoke-RestMethod -Uri $logUri | Select-String "Failed|Error|FAIL"
pwsh .github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 -PrNumber XXXXX
# Note the BuildId with FAILED state
pwsh .github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 -BuildId YYYYY -FailedOnly
IMPORTANT: Note jobs from EACH platform:
# Collect errors from EACH platform separately
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId YYYYY -JobFilter "*Linux*"
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId YYYYY -JobFilter "*Windows*"
pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId YYYYY -JobFilter "*MacOS*"
Create a file in session workspace with ALL findings:
# CI Errors for PR #XXXXX - Build YYYYY
## Failed Jobs Summary
| Platform | Job Name | Error Type |
|----------|----------|------------|
| Linux | ... | Test |
| Windows | ... | Test |
## Hypothesis Per Platform
### Linux/MacOS Failures
- Error: "The type 'int' is not defined"
- Hypothesis: Missing FSharp.Core reference in test setup
- Reproduction: `dotnet test ... -f net10.0`
### Windows Failures
- Error: "Expected cache hits for generic patterns"
- Hypothesis: Flaky test assertion, passes with other tests
- Reproduction: `dotnet test ... --filter "FullyQualifiedName~rigid generic"`
## Reproduction Commands
...
## Fix Verification Checklist
- [ ] Linux error reproduced locally
- [ ] Windows error reproduced locally
- [ ] Fix verified for Linux
- [ ] Fix verified for Windows
- [ ] Tests run IN ISOLATION (not just with other tests)
# Run failing tests IN ISOLATION (critical!)
dotnet test ... --filter "FullyQualifiedName~FailingTestName" -f net10.0
# Run multiple times to check for flakiness
for ($i = 1; $i -le 3; $i++) { dotnet test ... }
Only after ALL issues reproduced:
See Linux error → Fix → Push → Wait → See Windows error → Fix → Push → ...
See Linux error → See Windows error → See MacOS error → Document all →
Fix all → Verify all locally → Push once
dotnet test ... --filter "OverloadCacheTests" # All 8 pass together
dotnet test ... --filter "FullyQualifiedName~specific test name" # May fail alone!
gh (GitHub CLI) - authenticatedpwsh (PowerShell 7+)Performs multi-agent, multi-model code review of F# compiler PRs across 19 dimensions including type checking, IL emission, binary compatibility, and IDE performance. Dispatches parallel assessment agents per dimension, consolidates with cross-model agreement scoring, and filters false positives. Invoke when reviewing compiler changes, requesting expert feedback, or performing pre-merge quality checks.
Always invoke after editing .fs files. Provides fast parse/typecheck feedback without a full dotnet build. Prefer this over dotnet build for iterative changes. Also finds symbol references and inferred type hints.
Detect flaky tests by scanning recent AzDo CI builds for test failures recurring across multiple unrelated PRs. Use when investigating intermittent failures, CI instability, deciding which tests to quarantine, or checking if RunTestCasesInSequence no-ops are causing parallel-safety issues.
Fix F# debugging issues (breakpoints, .pdb, sequence points). Build, run VS integration tests, inspect IL/PDB.
Investigate compiler failures, test errors, or unexpected behavior through systematic minimal reproduction, 3-hypothesis testing, and verification. Always re-run builds and tests after changes.
Fix ILVerify baseline failures when IL shape changes (codegen, new types, method signatures). Use when CI fails on ILVerify job.