Comprehensive testing guide for Yoto Smart Stream - covering authentication testing, functional testing, Playwright UI automation, and test-and-fix development loops. Use when writing tests, debugging test failures, or implementing test coverage.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Comprehensive testing guide for Yoto Smart Stream - covering authentication testing, functional testing, Playwright UI automation, and test-and-fix development loops. Use when writing tests, debugging test failures, or implementing test coverage.
Yoto Smart Stream Testing
Comprehensive testing guide covering authentication testing, functional testing patterns, Playwright UI automation, and iterative test-and-fix development loops.
Overview
Testing strategy for Yoto Smart Stream focuses on:
Admin Page: Dark Mode widget visible and functional, all controls responsive, zero errors
Login Page: Authentication form renders correctly, Dark Mode widget operational, zero errors
Dark Mode Widget: "๐ Activate dark mode" control verified on all pages
Browser Console: No errors, warnings, or issues on any tested page
Deployment: v0.3.0 stable on Railway, health checks passing, service running
Quick Start
Prerequisites
# Install development dependencies
pip install -e ".[dev]"# Verify pytest is installed
pytest --version
# Verify Playwright is installed (for UI tests)
playwright --version
Run All Tests
# Run all tests with coverage
pytest --cov=yoto_smart_stream --cov-report=html
# Run specific test categories
pytest tests/test_auth.py # Authentication tests
pytest tests/test_api.py # API endpoint tests
pytest tests/test_login_flows.py # Playwright UI tests# Run with verbose output
pytest -v
# Run with output capture disabled (see print statements)
pytest -s
/\
/UI\ โ Few (critical workflows)
/โโโโ\
/INTEG\ โ Some (component interactions)
/โโโโโโ\
/ UNIT \ โ Many (isolated components)
โโโโโโโโโโ
Best Practices
1. Test Naming
# Good: Descriptive test namesdeftest_login_with_valid_credentials_returns_token():
...
deftest_login_with_invalid_password_returns_401():
...
# Bad: Vague test namesdeftest_login():
...
deftest_case_1():
...
2. Arrange-Act-Assert
deftest_create_user():
# Arrange: Setup test data
user_data = {
"username": "testuser",
"password": "testpass",
"role": "user"
}
# Act: Execute the action
response = client.post("/api/admin/users", json=user_data)
# Assert: Verify the resultsassert response.status_code == 201assert response.json()["username"] == "testuser"assert response.json()["role"] == "user"
3. Test Isolation
# Good: Each test is independent@pytest.fixturedefclean_database():
db.clear()
yield
db.clear()
deftest_create_user(clean_database):
# Test creates user in clean database
...
deftest_list_users(clean_database):
# Test lists users in clean database
...
# Bad: Tests depend on each otherdeftest_create_user():
global user_id
user_id = create_user()
deftest_delete_user():
delete_user(user_id) # Depends on previous test
4. Clear Error Messages
# Good: Descriptive assertionsdeftest_player_status():
response = client.get("/api/players/123")
assert response.status_code == 200, \
f"Expected 200 but got {response.status_code}. Response: {response.text}"
data = response.json()
assert"online"in data, \
f"'online' field missing from response. Got: {list(data.keys())}"# Bad: Bare assertionsdeftest_player_status():
response = client.get("/api/players/123")
assert response.status_code == 200assert"online"in response.json()
5. Fixtures Over Setup/Teardown
# Good: Use fixtures@pytest.fixturedeftest_user(client):
response = client.post("/api/admin/users", json={
"username": "testuser",
"password": "testpass"
})
user_id = response.json()["id"]
yield user_id
client.delete(f"/api/admin/users/{user_id}")
deftest_with_user(client, test_user):
# User automatically created and cleaned up
...
# Bad: Manual setup/teardowndeftest_with_user(client):
# Setup
response = client.post("/api/admin/users", ...)
user_id = response.json()["id"]
try:
# Test
...
finally:
# Teardown
client.delete(f"/api/admin/users/{user_id}")
Running Tests
Local Development
# Run all tests
pytest
# Run specific test file
pytest tests/test_auth.py
# Run specific test
pytest tests/test_auth.py::test_login_success
# Run tests matching pattern
pytest -k "login"# Run with coverage
pytest --cov=yoto_smart_stream --cov-report=html
# Run and stop on first failure
pytest -x
# Run with verbose output
pytest -v
# Run with print statements visible
pytest -s
# Run in parallel (requires pytest-xdist)
pytest -n auto
Playwright Tests
# Run Playwright tests
pytest tests/test_login_flows.py
# Run in headed mode (see browser)
pytest tests/test_login_flows.py --headed
# Run specific browser
pytest tests/test_login_flows.py --browser chromium
pytest tests/test_login_flows.py --browser firefox
pytest tests/test_login_flows.py --browser webkit
# Debug mode (opens Playwright Inspector)
PWDEBUG=1 pytest tests/test_login_flows.py
# Generate trace for debugging
pytest tests/test_login_flows.py --tracing on
CI/CD Testing
# Run in CI mode (non-interactive)
pytest --tb=short --maxfail=3
# Generate JUnit XML for CI reporting
pytest --junitxml=test-results.xml
# Generate coverage reports for CI
pytest --cov=yoto_smart_stream --cov-report=xml --cov-report=term
# Run with environment-specific config
SERVICE_URL=https://yoto-smart-stream-pr-61.up.railway.app pytest
Troubleshooting
Tests Failing Locally
Symptom: Tests pass in CI but fail locally (or vice versa)
Common Causes:
Environment Variables:
# Check required environment variablesecho$SERVICE_URLecho$YOTO_CLIENT_ID# Set for testingexport SERVICE_URL=https://yoto-smart-stream-develop.up.railway.app
Database State:
# Clear test databaserm -f test_database.db
# Or use fixture to ensure clean state
@pytest.fixture(autouse=True)
def clean_db():
db.clear()
yield
db.clear()
Network Issues:
# Test connectivity
curl https://yoto-smart-stream-develop.up.railway.app/api/health
# Check if service is running
railway status
