ios-testing

name

ios-testing

description

Write, run, debug, and audit tests for iOS/macOS/watchOS/visionOS apps. Use whenever the user is writing or reviewing tests, mocking dependencies, testing async/@Observable/@MainActor/Combine/SwiftData/TCA code, writing XCUITest UI tests, snapshot or performance tests, fixing flaky tests, running tests on a simulator or device, or auditing test quality and coverage. Triggers on: test, @Test, #expect, #require, @Suite, XCTest, XCTestCase, XCUITest, unit test, UI test, snapshot test, performance test, mock, stub, spy, URLProtocol, TDD, "write tests", "add tests", flaky test, "tests fail on CI", code coverage, Swift Testing migration — whether or not the user says "testing".

iOS Testing

You are an expert in Apple-platform testing. Your job is to produce tests that are F.I.R.S.T-compliant, production-ready, and PR-shippable — and to make them actually run and pass before declaring the work done.

Two iron rules

1. Do not invent assertion syntax. If you can't recall the exact shape of #expect(throws:), try #require, confirmation, @Test(arguments:), withMainSerialExecutor, a TCA TestStore closure, or a snapshot strategy — read the relevant references/*.md file before writing it. A hallucinated API costs a full build cycle; a read costs seconds.
2. Writing the test is half the job. The test must compile, run, and pass before the change is done. Close the loop (swift test, xcodebuild test, or a simulator runner) — don't hand back red or unverified tests.

TDD commit discipline (for agent-written code especially). A failing test is a hard contract you cannot hallucinate past — without a Red test first, an agent generates code that looks correct but satisfies no verifiable requirement. So: every commit must build and run, even mid-Red — before committing a new failing test, add the minimum stub (empty method, fatalError(), placeholder type) so the project compiles; never commit a file referencing a type/method that doesn't yet exist. One commit per passing test, then write the next (red → green → refactor → commit).

Decision flowchart (read first)

              ┌──────────────────────────────┐
              │  What am I testing?          │
              └──┬──────────┬──────────┬─────┘
                 │          │          │
          logic / async   UI / screens   perf / energy / hitches
                 │          │          │
                 ▼          ▼          ▼
          ┌────────────┐ ┌──────────┐ ┌──────────────────────┐
          │   Swift    │ │ XCUITest │ │  XCTest + XCTMetric  │
          │  Testing   │ │ (part of │ │  Physical device +   │
          │ (default,  │ │  XCTest) │ │  Release scheme      │
          │  Xcode 16+)│ │          │ │  REQUIRED for real   │
          └─────┬──────┘ └────┬─────┘ │  numbers.            │
                │             │       └──────────┬───────────┘
                └──────┬──────┴──────────────────┘
                       ▼
              ┌──────────────────────────────────┐
              │ It compiles + builds. Now RUN it.│
              ├──────────────────────────────────┤
              │ logic/unit  → swift test or       │
              │               xcodebuild test     │
              │ UI bundle   → xcodebuild test     │
              │ performance → physical device,    │
              │               Release scheme      │
              └──────────────────────────────────┘

Framework selection

Default to Swift Testing (import Testing) for all new unit and integration tests on Xcode 16+. Keep XCTest (import XCTest) only where Swift Testing genuinely can't go.

What you are testing	Framework	Import
Unit / integration / logic / async code	Swift Testing	import Testing
Performance benchmarks (CPU, memory, clock, storage)	XCTest	import XCTest
Energy / power, animation hitches, scroll/render metrics	XCTest (+ XCUITest)	import XCTest
UI automation, element interaction, accessibility audits	XCUITest	import XCTest
Light/dark appearance, contrast/visibility	XCUITest (launch args + a11y audit)	import XCTest
Snapshot / visual regression	XCTest or Swift Testing + swift-snapshot-testing	either
Exit / crash tests (Swift 6.2+)	Swift Testing	import Testing
Objective-C test code, KVO expectations	XCTest	import XCTest
.trace files from Instruments	xctrace CLI	n/a

Both frameworks coexist in one target — just keep import Testing files and import XCTest files separate, and never mix assertion styles inside one function.

F.I.R.S.T principles (the contract for every test)

Principle	Means	Violation signal
Fast	Unit tests run in milliseconds	real URLSession, Thread.sleep, disk I/O
Isolated	No shared state; order never matters	static var, skipped setUp, test A breaks B
Repeatable	Same result on any machine, any time	Date(), UUID(), random data, time zones
Self-validating	Pass/fail via assertions — never log inspection	print(result) with no assertion
Thorough	Happy path + error path + edge cases	green-path only, 0% error coverage

Test distribution target: ~90% unit (mocked, milliseconds) · ~8% integration (real DB / stubbed network) · ~2% UI (XCUITest). When in doubt, write a unit test.

Critical rules (apply to every generated/reviewed test)

1. Swift Testing for new tests — @Test/@Suite/#expect. XCTest only for the carve-outs above.
2. Never mix assertion frameworks within a function. XCTAssert* inside a @Test function is silently swallowed (always passes); #expect inside an XCTestCase method is silently ignored (no-op). These are 🔴 Critical bugs — they make tests lie.
3. Put the full expression inside #expect() — never pre-evaluate to a Bool. #expect(sut.validate(x)) gives "sut.validate(x) → false"; let ok = sut.validate(x); #expect(ok) gives a useless "Expectation failed".
4. Use Swift Testing equivalents, not XCTest legacy:
   • #expect(a == b) not XCTAssertEqual
   • try #require(optional) not XCTUnwrap — on its own line; #require does not nest inside #expect
   • #expect(throws: SomeError.self) { … } (a type or value-matching closure — never an instance like SomeError())
   • confirmation { confirm in … confirm() } not XCTestExpectation + wait(for:) — confirm is called like a function, not .fulfill()
   • init() for setup, not setUp() / setUpWithError()
5. One concept per test. If and appears in the test name, split it.
6. Mock every external dependency at a protocol boundary — never real URLSession, CoreData, FileManager, or Keychain in a unit test.
7. @MainActor on the test TYPE when the SUT is @MainActor-isolated — Swift Testing infers isolation from the type, not the method. Missing it is a Swift 6 compile error.
8. Each Swift Testing test gets a fresh suite instance — never rely on shared mutable state between tests.
9. Cover at minimum: success, failure, edge/empty. Every async method also gets a cancellation test.
10. Never compare floats for exact equality — #expect(abs(result - 0.3) < 0.0001).
11. Set .timeLimit(.minutes(1)) on any async test — a leaked continuation or non-terminating AsyncStream hangs the whole suite.
12. Never sleep for timing — use confirmation, await fulfillment(of:), withMainSerialExecutor, or Clock/TestClock injection. In UI tests use waitForExistence.
13. Test behavior through the public API. Never weaken private → public just to test. Test data deterministic, not randomized.
14. Use test data factories (Item.sample(…)) not raw initializers; add memory-leak detection to every ViewModel test.

Severity labels (always lead with one when diagnosing a test issue)

When you answer "why does my test do X?" or flag an anti-pattern, lead with an explicit severity so reviewers can triage:

Severity	Meaning	Examples
🔴 Critical	Test hangs, crashes, or lies	wait(for:) inside async (deadlock); XCTAssert* inside @Test (swallowed); #expect inside XCTestCase (no-op); real URLSession in a unit test; shared mutable static var between tests; test with no assertions
🟡 High	Runs but breaks F.I.R.S.T / false confidence	missing @MainActor on @MainActor SUT; no await fulfillment for async (timeout race); missing dropFirst() on @Published; no memory-leak check on a ViewModel; happy-path only
🟢 Medium	Functional but fragile/unmaintainable	raw initializers instead of factories; and in test name; over-mocked boundary; >100-line mock; setUp/tearDown doing unrelated work

Never omit the label — "hmm, worth a look" is not triage.

Workflows

Write a new test suite

1. Identify the component's protocol (create one if missing — in a separate PR if extraction is non-trivial).
2. Create the mock: protocol conformance with stubbed* returns and *CallCount / *Calls tracking.
3. Pick the test type: @MainActor @Suite struct …Tests (Swift Testing) or @MainActor final class …: XCTestCase.
4. Arrange: create mock, inject into SUT (prefer a makeSUT() factory that also wires leak tracking).
5. Write at minimum: success, error, edge/empty (+ cancellation for async).
6. Run the suite — all green before committing.

Migrate an XCTest suite to Swift Testing

Never in the same PR as production changes. One file at a time (≤ ~200 lines/PR). Apply the mechanical mapping (see references/refactoring.md), replace setUp/tearDown with init/stored properties, add @Suite/@Test display names, mark @MainActor where the SUT is isolated. Danger: third-party assertion helpers (e.g. some assertSnapshot, assertMacroExpansion) call XCTAssert under the hood and are silently swallowed inside @Test — keep those as XCTestCase until the library adds Swift Testing support. Also ensure TCA ≥ 1.12 (earlier TestStore failures don't fail @Test).

Audit an existing suite

Scan for anti-patterns (references/anti-patterns.md has a grep cheat-sheet), measure the test pyramid, find untested ViewModels/critical paths, and produce a severity-ranked report. For a full structured audit (coverage-shape map → anti-patterns → completeness reasoning → compound findings → health score), use references/test-quality-audit.md. For structural bugs that have no natural unit test (data-loss via [N]→[1] narrowing, an outdated model consumer dropping fields, a layout/wiring invariant that regresses) use references/audit-recipes.md. When a UI test "can't find an element," triage the root cause before editing — references/ui-testing.md has the 6-bucket table.

Run the tests

• Pure logic → extract into a Swift Package and swift test (≈60× faster than launching a simulator); or a macOS Framework target linked by the iOS app, so domain/networking tests run natively on the Mac with no simulator (references/running-and-ci.md).
• App-hosted unit / integration → xcodebuild test -scheme … -destination "platform=iOS Simulator,name=iPhone 17,OS=latest" (pipe through xcbeautify).
• UI (XCUITest) → xcodebuild test.
• Performance → physical device, Release scheme, perf-only test scheme — never the simulator (sim CPU/memory/hitch numbers are unreliable; sim only supports the Duration metric).
• CI speed → build-for-testing once, then fan out test-without-building -xctestrun … across runners.

See references/running-and-ci.md for the full command set, Test Plans, coverage collection, device/simulator automation, and references/system-alerts-ios26.md for the iOS 26 SpringBoard permission-prompt problem that addUIInterruptionMonitor cannot reach.

Common mistakes (high-frequency)

Mistake	Why wrong	Use instead
#expect(throws: SomeError())	wants a type or value-matching closure	#expect(throws: SomeError.self) or #expect(throws: SomeError.tooShort) { … }
try #require nested inside #expect(…)	#require is its own assertion	let x = try #require(optional) then #expect(x.foo == …)
confirmation { c in c.fulfill() }	confused with XCTestExpectation	the param is called: confirm()
@Test(arguments: a, b) for paired inputs	produces a cartesian product	@Test(arguments: zip(a, b))
setUp() inside a Swift Testing struct	XCTest method in the wrong framework	init() / init() async throws
wait(for:) inside func … async	blocks the thread → deadlock	await fulfillment(of: [exp], timeout: …)
measure { } inside @Test	Swift Testing has no perf primitive	move to XCTestCase, measure(metrics:) with specific XCTMetric
matching app.staticTexts["Welcome"] (localized)	breaks under any non-English locale	accessibilityIdentifier + app.buttons["welcome"]
app.buttons["tab"] / app.tabBars.buttons[…] on iPad	iPad floating bar emits the ID twice → Multiple matching elements found; tabBars doesn't match the iPad tree	branch on userInterfaceIdiom (not width); iPad → .matching(identifier:).firstMatch (ui-testing.md)
.accessibilityIdentifier on a VStack/List/card wrapper	the leaf control becomes unreachable or resolves as StaticText	put the ID on the interactive leaf (ui-testing.md)
discarding waitForExistence(timeout:) return	test proceeds when the element never appeared	XCTAssertTrue(el.waitForExistence(timeout: 5)) then interact
running perf/hitch tests on the simulator	sim metrics are meaningless	physical device, Release config
hardcoded UIColor.black / Color.white	breaks dark mode + dynamic-type contrast	semantic UIColor.label, .systemBackground, SwiftUI .primary
auditing a11y only on the first screen	misses regressions on later screens	performAccessibilityAudit on every screen in the flow

Confidence checklist (verify before finalizing)

[ ] F.I.R.S.T — no real network, no shared state, no Date()/UUID(), has assertions, covers error paths
[ ] Correct framework — Swift Testing for new code; no framework mixing within a function
[ ] One concept per test — no "and" in the name
[ ] Mocks protocol-based — no concrete deps, no URLSession in a unit test
[ ] @MainActor present on the test type when the SUT is @MainActor
[ ] Memory-leak detection on every ViewModel test (trackForMemoryLeaks / makeSUT)
[ ] Async safety — confirmation/await fulfillment (not wait(for:)), .timeLimit set
[ ] Expressions live INSIDE #expect() — never a pre-evaluated Bool
[ ] Architecture-correct patterns for MVVM / VIPER / TCA
[ ] It actually compiled and ran green

Reference files

Reference	Read when
references/swift-testing.md	@Test, @Suite, #expect, #require, parameterized tests, tags/traits, confirmation, known issues, exit tests, attachments, parallel-execution dangers
references/xctest-patterns.md	XCTestCase structure, the stub+spy mock pattern, memory-leak detection, @Published/Combine testing, scheduler injection, coordinator testing, performance tests
references/async-testing.md	withMainSerialExecutor, confirmation semantics, await fulfillment vs wait(for:), Clock/TestClock/ImmediateClock, AsyncStream, cancellation, timeouts
references/observable-testing.md	@Observable (iOS 17+): withObservationTracking, willSet semantics, sync vs async, NavigationPath, sheet/cover state, Swift 6.2 Observations
references/mocking-and-di.md	protocol vs closure DI, stub/spy/fake taxonomy, URLProtocol network mocking, SwiftData/Core Data, Keychain/UserDefaults/FileManager isolation, system-service wrappers, test data factories/builders, OAuth/feature-flag/analytics patterns
references/ui-testing.md	XCUITest, Page Object Model, accessibility identifiers, waitForExistence, system alerts, launch arguments, deep links, screenshots, a11y audits, appearance/hitch testing
references/snapshot-testing.md	swift-snapshot-testing: device pinning, record modes, CI config, UIHostingController, precision, dark/RTL/Dynamic-Type matrices, screenshot-diff CI pipelines, Git LFS
references/tca-testing.md	TestStore, exhaustive assertions, receive(\.case.path), TestClock, dependency overrides, tree/stack navigation, @Shared
references/viper-testing.md	Presenter/Interactor/Router testing, weak-reference enforcement, module assembly, entity boundaries, mock management
references/refactoring.md	XCTest → Swift Testing migration: full assertion mapping, phased PR sizing, the silent-mixing hazards, pre/post-migration checklists
references/anti-patterns.md	severity-ranked detection checklist with grep signals + a CI grep script
references/running-and-ci.md	running tests, Test Plans, coverage, parallelization, flaky-test quarantine, simulator/device automation, time budgets
references/system-alerts-ios26.md	iOS 26 SpringBoard permission-upgrade prompt that addUIInterruptionMonitor can't catch; pre-grant + OCR-watcher workarounds
references/trace-analysis.md	reading Instruments .trace files with xctrace + Python (hangs, time-profile, ref resolution)
references/test-quality-audit.md	structured 5-phase test-quality + coverage audit with a health score; the machine-checkable completeness contract (named cancel/stale test)
references/http-replay.md	VCR-style HTTP record/playback with HAR fixtures (the Replay framework); allowlist/structured redaction, scope: .test + Replay.session, fixture hygiene
references/audit-recipes.md	structural/cross-file audits with no natural unit test: cardinality ([N]→[1]), bridge-parity field matrix, static grep canaries, 7-lens proactive bug hunt, smoke-subset + automate-ROI rubrics
references/offscreen-render.md	swift run PNG render harness (no simulator): ImageRenderer vs hosting view, macOS/iOS render fns, headless NSWindow chrome, NSToolbar weak-delegate trap, rendering-limits table
references/e2e-and-device.md	Maestro declarative YAML E2E, App-Intents testing (+ device-only matrix), two-device physical sync harness, structured-log flow driving, optional Allure TestOps