run-device-tests

Stars23,289

Forks1,959

UpdatedJuly 1, 2026 at 07:22

Build and run .NET MAUI device tests locally with category filtering. Supports iOS, MacCatalyst, Android on macOS; Android, Windows on Windows. Use TestFilter to run specific test categories.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Source

dotnet

dotnet/maui

View GitHub Repository View Creator Repositories

Download

Related occupationsSOC

Based on SOC occupation classification

Software Quality Assurance Analysts and TestersComputer and Mathematical Occupations·SOC 15-1253

File Explorer

3 files

SKILL.md

readonly

More from this repository

same repository

find-regression-risk

dotnet/maui

Detects potential regression risks in a PR by cross-referencing lines the PR REMOVES against lines ADDED by recent labeled bug-fix PRs (`i/regression`, `t/bug`, `p/0`, `p/1`) touching the same files. Purely mechanical — no AI/LLM. Emits a CLEAN / OVERLAP / REVERT verdict plus structured findings. Triggers on: "does this PR revert a previous fix", "check PR for regression risk", "find regression risks in PR", "is this change reverting a bug fix". Do NOT use for: assessing ship-readiness of a release branch (use release-readiness), investigating CI failures (use azdo-build-investigator), or general code review (use code-review).

2026-07-3023.3k

dependency-flow

dotnet/maui

MAUI-specific dependency flow rules, channel conventions, and feed lookup workflows. Use when asked about darc, BAR, Maestro, feeds for .NET MAUI, build promotion, asset lookup, channel mappings, or dependency flow for dotnet/maui. Wraps the maestro-cli skill and maestro MCP tools with MAUI-specific guardrails.

2026-07-2923.3k

release-readiness

dotnet/maui

Assesses ship-readiness for .NET MAUI release branches — Servicing Releases (SR) and Previews. Surveys CI pipelines, computes what's actually NEW in the branch (commits + source PRs with revert detection), and cross-references open `regressed-in-*` issues against branch contents to identify port candidates, rejected backports, and unresolved regressions. Supports both in-flight and pre-cut (candidate) modes for SR and Preview branches.

2026-07-2923.3k

review-test-failures

dotnet/maui

Classifies PR CI/test failures as likely PR-caused or unrelated, compares against base-branch baseline, and emits an overall merge-readiness verdict. Uses gathered GitHub/AzDO/Helix context and the shared MAUI CI facts.

2026-07-2923.3k

ci-fix

dotnet/maui

Common safety and triage protocol for .NET MAUI CI-fixer workflows. Use whenever investigating, creating, or advancing a `[ci-fix]` or `[ci-fix-net11]` pull request from a `ci-scan` issue, including manual single-issue dispatches and scheduled sweeps.

2026-07-2923.3k

code-review

dotnet/maui

Deep code review of PR or materialized candidate-patch changes for correctness, safety, and MAUI conventions. Uses independence-first assessment (code before narrative) and delegates to the maui-expert-reviewer agent for per-dimension sub-agent evaluation. Triggers on: "review code for PR", "code review PR", "review candidate patch", "analyze code changes", "check PR code quality". Do NOT use for: summarizing PRs, describing what changed, general PR questions, running tests, or fixing code.

2026-07-2823.3k

name	run-device-tests
description	Build and run .NET MAUI device tests locally with category filtering. Supports iOS, MacCatalyst, Android on macOS; Android, Windows on Windows. Use TestFilter to run specific test categories.
metadata	{"author":"dotnet-maui","version":"2.1"}
compatibility	Requires xharness CLI (for iOS/MacCatalyst/Android), Xcode (for Apple platforms), Android SDK (for Android), and .NET SDK with platform workloads.

Run Device Tests Skill

Build and run .NET MAUI device tests locally on iOS simulators, MacCatalyst, Android emulators, or Windows.

Platform Support

Host OS	Supported Platforms
macOS	ios, maccatalyst, android
Windows	android, windows

Tools Required

This skill uses bash together with pwsh (PowerShell 7+) to run the PowerShell scripts. Requires:

xharness - Global dotnet tool for running tests on iOS/MacCatalyst/Android
dotnet - .NET SDK with platform workloads installed
iOS/MacCatalyst: Xcode with iOS simulators
Android: Android SDK with emulator
Windows: Windows SDK

Dependencies

This skill uses shared infrastructure scripts:

.github/scripts/shared/Start-Emulator.ps1 - Detects and boots iOS simulators / Android emulators
.github/scripts/shared/shared-utils.ps1 - Common utility functions

These are automatically loaded by the Run-DeviceTests.ps1 script.

When to Use

User wants to run device tests locally
User wants to verify iOS/MacCatalyst/Android/Windows compatibility
User wants to test on a specific iOS version (e.g., iOS 26)
User asks "run device tests for Controls/Core/Essentials/Graphics"
User asks "test on iOS simulator" or "test on Android emulator"
User asks "run device tests on MacCatalyst"
User wants to run only specific test categories (e.g., "run Button tests")

Available Test Projects

Project	Path
Controls	`src/Controls/tests/DeviceTests/Controls.DeviceTests.csproj`
Core	`src/Core/tests/DeviceTests/Core.DeviceTests.csproj`
Essentials	`src/Essentials/test/DeviceTests/Essentials.DeviceTests.csproj`
Graphics	`src/Graphics/tests/DeviceTests/Graphics.DeviceTests.csproj`
BlazorWebView	`src/BlazorWebView/tests/DeviceTests/MauiBlazorWebView.DeviceTests.csproj`

Scripts

All scripts are in .github/skills/run-device-tests/scripts/

Run Device Tests (Full Workflow)

# Run Controls device tests on iOS simulator (default on macOS)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

# Run Core device tests on MacCatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform maccatalyst

# Run Controls device tests on Android emulator
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

# Run Controls device tests on Windows (default on Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

# Run on specific iOS version
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform ios -iOSVersion 26

# Run with test filter
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Run other test projects
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Essentials -Platform android
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Graphics -Platform maccatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project BlazorWebView -Platform ios

Build Only (No Test Run)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -BuildOnly

List Available Simulators/Emulators

# iOS simulators
xcrun simctl list devices available

# Android emulators
emulator -list-avds

Workflow

Run tests: scripts/Run-DeviceTests.ps1 -Project <name> -Platform <platform> automatically detects/boots device, builds, and runs tests
Check results: Look at the console output or artifacts/log/ directory for detailed test results

Output

Build artifacts: artifacts/bin/<Project>.DeviceTests/<Configuration>/<tfm>/<rid>/
Test logs: artifacts/log/
Test results summary is printed to console

Prerequisites

xharness global tool: dotnet tool install --global Microsoft.DotNet.XHarness.CLI
.NET SDK with platform workloads
iOS/MacCatalyst: Xcode with simulators installed
Android: Android SDK with emulator configured
Windows: Windows SDK
For iOS 26: macOS Tahoe (26) with Xcode 26

Examples

# Quick test run for Controls on iOS (default on macOS)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

# Test on MacCatalyst
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform maccatalyst

# Test on Android emulator (works on both macOS and Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

# Test on Windows (default on Windows)
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

# Test on iOS 26 specifically
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -iOSVersion 26

# Run only Button category tests on iOS
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Build for Android without running tests
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform android -BuildOnly

Notes

The script automatically detects and boots an iOS simulator / Android emulator using the shared Start-Emulator.ps1 infrastructure
Default iOS simulator is iPhone Xs with iOS 18.5 (same as UI tests)
Default Android emulator priority: API 30 Nexus > API 30 > Nexus > First available
MacCatalyst runs directly on the Mac (no simulator needed)
Windows tests run directly on the local machine
Simulator/emulator selection and boot logic is handled by .github/scripts/shared/Start-Emulator.ps1
xharness manages test execution and reporting for iOS/MacCatalyst/Android
Windows runs the built device-test app directly and reads its xUnit XML results, matching eng/devices/windows.cake

Test Filtering

The -TestFilter parameter allows running specific test categories instead of all tests. This is useful for quick iteration during development.

Filter Syntax

Format	Description	Example
`Category=X`	Run only category X	`Category=Button`
`SkipCategories=X,Y,Z`	Skip categories X, Y, Z	`SkipCategories=Shell,CollectionView`

Examples

# Run only Button tests on iOS
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

# Run only Button tests on Android  
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android -TestFilter "Category=Button"

# Skip heavy test categories
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "SkipCategories=Shell,CollectionView,HybridWebView"

How Test Filtering Works

Test filtering is implemented in src/Core/tests/DeviceTests.Shared/DeviceTestSharedHelpers.cs:

Platform	How Filter is Passed	How Filter is Read
iOS/MacCatalyst	`--set-env=TestFilter=...`	`NSProcessInfo.ProcessInfo.Environment["TestFilter"]`
Android	`--arg TestFilter=...`	`MauiTestInstrumentation.Current.Arguments.GetString("TestFilter")`
Windows Controls	App argument selects discovered category index	`ControlsHeadlessTestRunner` category loop

Available Test Categories

Common categories in Controls.DeviceTests:

Button, Label, Entry, Editor - Individual control tests
CollectionView, ListView, CarouselView - Collection controls (heavy)
Shell, Navigation, TabbedPage - Navigation tests (heavy)
Layout, FlexLayout - Layout tests
Memory - Memory leak tests
Accessibility - Accessibility tests
Gesture - Gesture recognition tests

To see all categories, check src/Controls/tests/DeviceTests/TestCategory.cs.

Troubleshooting

Xcode Version Mismatch

If you see errors about Xcode version mismatch (e.g., "Xcode 26.2 installed but 26.0 required"):

# Use -SkipXcodeVersionCheck to bypass validation
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -SkipXcodeVersionCheck

MacCatalyst App Bundle Not Found

MacCatalyst apps use display names (e.g., "Controls Tests.app") not assembly names. The script handles this automatically by searching for .app bundles in the output directory.

Android Package Name Issues

Android package names must be lowercase (e.g., com.microsoft.maui.controls.devicetests). The script has a mapping of project names to correct package names.

Test Filter Not Working

Ensure full rebuild: The test filter code must be compiled into the app. Delete artifacts and rebuild:

rm -rf artifacts/bin/<Project>.DeviceTests
pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android -TestFilter "Category=Button"

Check the console output: Look for TestFilter: Category=Button in the test output to confirm filter was applied.
Verify category exists: Ensure the category name matches exactly (case-sensitive).

XHarness Device Detection

The script automatically handles XHarness device targeting for iOS and Android:

iOS

Simulator Boot: Start-Emulator.ps1 detects and boots appropriate iOS simulator
UDID Extraction: Script extracts simulator UDID from Start-Emulator.ps1 output
iOS Version Detection: Script queries xcrun simctl to get iOS version from booted simulator
XHarness Targeting: Passes both --target ios-simulator-64_VERSION and --device UDID to xharness for explicit targeting

MacCatalyst

Runs directly on the Mac, no device targeting needed
XHarness uses --target maccatalyst

Android

Emulator Boot: Start-Emulator.ps1 detects running device or boots emulator
Device ID Extraction: Script gets device ID (e.g., emulator-5554)
XHarness Targeting: Passes --device-id to xharness android command

Windows

No device/emulator needed
Runs the built device-test app directly and parses TestResults-*.xml

Why both --target and --device for iOS?

XHarness requires --target ios-simulator-64 (or ios-simulator-64_VERSION) to specify platform type
Adding --device UDID explicitly tells xharness which simulator to use
This combination ensures reliable device selection even on ARM64 Macs where automatic detection can fail

Example xharness invocations:

# iOS with test filter
dotnet xharness apple test \
  --app path/to/app \
  --target ios-simulator-64_18.5 \
  --device 56AE278D-60F7-4892-9DE0-6341357CA068 \
  -o artifacts/log \
  --timeout 01:00:00 \
  -v \
  --set-env=TestFilter=Category=Button

# MacCatalyst with test filter
dotnet xharness apple test \
  --app path/to/app \
  --target maccatalyst \
  -o artifacts/log \
  --timeout 01:00:00 \
  -v \
  --set-env=TestFilter=Category=Button

# Android with test filter
dotnet xharness android test \
  --app path/to/app.apk \
  --package-name com.microsoft.maui.controls.devicetests \
  --device-id emulator-5554 \
  -o artifacts/log \
  --timeout 01:00:00 \
  -v \
  --arg TestFilter=Category=Button

This ensures tests run on the correct device with proper version targeting and filtering.