testing-tracks

# Testing Tracks: From Internal to Production This skill manages the progressive testing workflow that validates your app with real users before production release. The key principle: start narrow (internal), expand gradually (closed/external), then go wide (open/production). ## Testing Progression ``` Internal (minutes) Closed/External (hours) Open (days) Production 100 testers ──> email lists / groups ──> discoverable ──> everyone no review review on first build review full review instant fast moderate standard ``` ## Google Play Testing Tracks Read `references/google-play-tracks.md` for complete setup instructions. ### Track Comparison | Track | Testers | Review | Speed | Cost (paid apps) | Visibility | |-------|---------|--------|-------|-------------------|------------| | Internal | Up to 100 | None | Seconds | Free for testers | Invite only | | Closed | Email lists, Google Groups | Yes | Hours | Testers must pay | Invite only | | Open | Unlimited | Yes | Hours | Testers must pay | Discoverable on Play | ### Recommended Flow 1. **Internal testing** (first) — validate builds work, basic smoke testing 2. **Pre-launch report** — upload to closed or open track to trigger automated testing on real devices 3. **Closed testing** — invite specific user groups for focused feedback 4. **Open testing** — broader audience, find edge cases, stress test ### Pre-Launch Reports Automatically generated when you upload to closed or open tracks. Reports include: - **Stability:** crashes and ANRs across devices - **Performance:** startup time, rendering - **Accessibility:** unlabeled elements, contrast issues - **Security:** known vulnerability patterns - **Compatibility:** different Android versions and screen sizes **Action items from pre-launch reports are blockers** — fix critical/high severity issues before promoting to production. ## Apple TestFlight Read `references/testflight.md` for complete setup instructions. ### Testing Tiers | Tier | Testers | Review | Build availability | |------|---------|--------|--------------------| | Internal | Up to 100 (App Store Connect users) | None | Immediate after processing | | External | Up to 10,000 | Required (first build per group) | After review approval | ### Key Constraints - Builds expire after **90 days** - External testing requires **App Review** for the first build (subsequent builds often skip full review) - Testers need the free **TestFlight app** installed - Managed Apple Accounts (Apple Business Manager) cannot test ### TestFlight Workflow 1. Upload build via CI/CD (Codemagic publishing or Fastlane) 2. Build processes on Apple's servers (5-30 minutes) 3. Add build to internal testing group — available immediately 4. Add build to external testing group — goes to App Review first 5. Once approved, testers receive email notification 6. Testers install via TestFlight app 7. Feedback submitted through TestFlight or email ## Real Device Testing Read `references/real-device-testing.md` for platform-specific setup. ### When to Use Real Device Testing Use cloud device labs when: - You need to test on devices you do not physically own - Accessibility testing across multiple screen sizes - Performance validation across chipset generations - Regression testing on older OS versions - Your QA team is remote and cannot share physical devices ### Platform Comparison | Feature | BrowserStack | Sauce Labs | |---------|-------------|------------| | Real devices | 20,000+ iOS & Android | Private fleet (API access) | | Flutter support | Appium + Detox (Android) | Framework-agnostic (HTTP/WS) | | CI/CD integration | Azure Pipelines, GitHub Actions | API-based, any CI | | Free tier | No (paid plans) | No (private devices required) | | Unique feature | App Automate, App Live | Real Device Access API | ### Quick Decision - **BrowserStack** — best for most teams: broader device coverage, easier setup, better Flutter support via Appium - **Sauce Labs** — best for teams needing API-level device control or private dedicated devices ## Testing Checklist Before promoting from testing to production: ### Functionality - [ ] Core user flows work end-to-end - [ ] Authentication (login, logout, session expiry) - [ ] Network error handling (offline mode, timeouts) - [ ] Deep links resolve correctly - [ ] Push notifications received and handled ### Platform-Specific - [ ] Android: back button behavior, permissions dialogs - [ ] iOS: swipe gestures, status bar, safe areas - [ ] Tablet/large screen: layout adapts correctly - [ ] Dark mode: all screens render properly ### Performance - [ ] Cold start under 3 seconds - [ ] No visible jank during scrolling - [ ] Memory usage stable (no leaks in long sessions) - [ ] Battery drain reasonable ### Store Compliance - [ ] Google Play pre-launch report: no critical issues - [ ] No crashes in TestFlight crash reports - [ ] Content matches store listing descriptions - [ ] Privacy policy matches actual data collection ## Collecting Feedback ### Google Play Testing - Testers can submit private feedback via Play Console - Feedback visible only to you (does not affect public ratings) - Reply directly to testers through Play Console ### TestFlight - TestFlight 2.3+: feedback via TestFlight app or screenshot annotation - All feedback visible in App Store Connect > TestFlight > Feedback - Older versions: feedback via email to specified address ### Structured Feedback Template Share this with testers to get actionable feedback: > **Device:** [model + OS version] > **Build:** [version + build number] > **What did you try to do?** > **What happened?** > **What did you expect?** > **Screenshot/recording:** [attach if possible]