| name | test-engine |
| description | Power Apps Test Engine + Playwright. Use when: E2E testing of Canvas Apps, Model-Driven Apps, or Power Pages; automated UI verification; regression testing in CI/CD pipelines; visual QA of deployed solutions. |
Skill: Power Apps Test Engine + Playwright
When to Use
Trigger when testing Power Platform apps end-to-end, writing automated test plans, setting up test pipelines, or verifying UI after deployment.
Decision Matrix: Which Testing Approach?
| Approach | When | Language | Selectors | Auth |
|---|
| Test Engine (Power Fx YAML) | Standard Canvas/MDA testing, low-code teams | Power Fx in YAML | App-level control names | Built-in Entra handling |
| Raw Playwright (TypeScript) | Complex UI, custom controls, third-party components, visual QA | TypeScript or C# | CSS/XPath DOM selectors | Manual auth scripting |
| Test Engine + Playwright hybrid | Standard tests with occasional DOM access | Power Fx + Preview.PlaywrightAction() | Both | Built-in + direct |
| Manual testing | One-off verification, exploratory testing | N/A | N/A | N/A |
Rule of thumb: Start with Test Engine Power Fx for standard apps. Drop to raw Playwright only when Test Engine abstractions can't reach what you need (custom controls, third-party embedded UIs, visual regression).
AI Accuracy Warning
LLMs are less accurate with Power Fx than mainstream languages (JavaScript, C#, Python) due to smaller training datasets. Always verify Power Fx test steps manually before trusting them in CI/CD. This applies to both test authoring and formula column definitions.
⚠️ REQUIRED: Load Sub-Files Before Implementation
SKILL.md is a summary only — it is NOT sufficient for implementation.
The detailed content (complete payloads, XML templates, working examples, edge-case handling) lives in sub-files in the same directory as this SKILL.md. Before writing any code, you MUST use read_file on the sub-files relevant to your task:
- Test Plans (YAML Format) — YAML schema, Power Fx test steps, assertions, credentials, recording video
- Canvas App Testing — Canvas provider, control name selectors, gallery navigation, delegation testing
- Model-Driven App Testing — MDA provider, form verification, view checks, BPF testing, Dataverse integration
- Playwright Advanced — Direct Playwright functions, custom C# scripts, raw TypeScript tests, Sean Astrakhan's AI-generated test pattern
- CI/CD Integration — ADO pipeline YAML, GitHub Actions, publishing .trx results, regression testing on every deploy
- Power Pages Testing — Portal provider, auth testing, table permission verification, Web API testing
Providers Overview
Test Engine uses a provider-based architecture. Each provider is optimized for a specific application type:
| Provider | Flag | Application Type | Key Capabilities |
|---|
canvas | --provider canvas | Canvas Apps | Control-name selectors, gallery navigation, connector mocking |
mda | --provider mda | Model-Driven Apps | Form interaction, view navigation, BPF testing |
powerfx | --provider powerfx | Dataverse (no UI) | Direct Power Fx execution, backend logic testing |
portal | --provider portal | Power Apps portal | Portal automation operations |
Quick Start
# 1. Install prerequisites
# PAC CLI: https://learn.microsoft.com/power-platform/developer/cli/introduction
# Azure CLI: https://learn.microsoft.com/cli/azure/install-azure-cli
# 2. Authenticate
az login --allow-no-subscriptions
pac auth create --name TestAuth --url "https://contoso.crm.dynamics.com"
# 3. Set test user credentials (env vars — never in YAML)
$env:user1Email = "testuser@contoso.onmicrosoft.com"
# 4. Run a canvas app test
pac test run `
--provider canvas `
--test-plan-file "tests/smoke-test.te.yaml" `
--tenant "your-tenant-id" `
--environment-id "your-env-id"
# 5. View results
# Output: .trx file + screenshots/videos in output directory
Anti-Patterns
- Using raw Playwright when Test Engine control-name selectors would work (fragile DOM selectors break on platform updates)
- Storing credentials in test plan YAML files (use environment variables)
- Skipping test setup/teardown (test data left behind corrupts other tests)
- Running tests against production data without isolation
- Not recording video on CI/CD runs (impossible to debug failures without visual evidence)
- Trusting AI-generated Power Fx test steps without manual verification
- Using
headless: true for initial test development (use headless: false to see what's happening)
Key References
