pythonista-testing

name	pythonista-testing
description	Python testing best practices with pytest, TDD, and mocking. ALWAYS use this skill BEFORE writing or modifying any test code. Triggers on "test", "tests", "testing", "TDD", "test-driven", "pytest", "add tests", "write tests", "test this", "unit test", "integration test", "test coverage", "bug fix", "fix bug", "mock", "fixture", "assert", or when about to create/edit files in tests/ directory.

Python Testing Best Practices

Core Philosophy

Write invariant-based tests that verify what SHOULD be true, not bug-affirming tests that prove bugs existed.

Test-Driven Development (TDD)

ALWAYS use TDD when fixing bugs:

1. Find existing tests for the broken functionality
2. Run them to verify they pass (shouldn't catch bug)
3. Improve tests until they fail (exposing the bug)
4. Fix the code to make tests pass
5. Verify all tests pass

Quick Start

# Run tests
pytest tests/path/to/test.py -v

# Run with coverage
pytest --cov=src --cov-report=term-missing

Critical Rules

Mocking - ALWAYS use patch.object

# CORRECT - refactor-safe
from unittest.mock import patch

@patch.object(MyClass, 'method_name')
def test_with_mock(mock_method):
    ...

# WRONG - breaks silently on refactor
@patch('module.path.MyClass.method_name')
def test_with_mock(mock_method):
    ...

Mock Dependencies, NOT the System Under Test

# CORRECT - Mock dependencies, test real code
generator = NewsPostGenerator()
generator._queries_chain = AsyncMock()      # Dependency - mock it
generator._search_engine = AsyncMock()       # Dependency - mock it
await generator.generate_news_post(...)      # SUT - actually runs

# WRONG - Tests nothing
generator = AsyncMock(spec=NewsPostGenerator)

Test Data - ALWAYS use Pydantic models

# CORRECT - validation, type safety
def create_test_result(channel_id: str) -> VideoModerationResult:
    return VideoModerationResult(
        channel_id=channel_id,
        user_id="test_user",
        timestamp=datetime.now(UTC),
        details=VideoModerationDetails(is_appropriate=True)
    )

# WRONG - no validation, won't catch schema changes
def create_test_data():
    return {"channel_id": "test", "user_id": "user123"}

Constants - NEVER use naked literals

# CORRECT - relationships explicit
DEFAULT_RECHECK_INTERVAL = 60
STALE_AGE = DEFAULT_RECHECK_INTERVAL + MODERATION_DURATION + 10

def test_staleness_detection():
    timestamp = datetime.now(UTC) - timedelta(seconds=STALE_AGE)
    assert is_stale(result)

# WRONG - magic numbers
def test_staleness_detection():
    timestamp = datetime.now(UTC) - timedelta(seconds=120)  # Why 120?

Invariant Testing

# CORRECT - Test what SHOULD be true
def test_drip_selector_populated_with_all_drip_names():
    """INVARIANT: Selector contains all drip names from config."""
    config = make_config_with_drips(["drip1", "drip2", "drip3"])
    page = setup_page_with_config(config)
    assert page.drip_selector.options == ["drip1", "drip2", "drip3"]

# WRONG - Bug-affirming test
def test_bug_123_drip_selector_empty():
    """Bug 123: Drip selector is empty."""
    assert len(drip_selector.options) > 0  # Proves bug, doesn't verify correctness

E2E Testing - Call Production Code

# CORRECT - Call actual production code
async def test_news_with_image_e2e():
    await news_flow.create_news_post(flow_input)
    published_event = mock_kafka.publish_object.call_args.kwargs["output"]
    assert published_event.attachments is not None  # Fails if code forgot attachment

# WRONG - Manually construct state
async def test_news_with_image_e2e():
    activity_event = FeedActivityEvent(
        attachments=[news_post.attachment],  # WE added this, not production code!
    )
    # Test passes even if production code is broken

Access Mock Args Explicitly

# CORRECT - Clear and explicit
flow_input = call_args.args[0]
delay = call_args.kwargs["delay"]

# WRONG - Cryptic
flow_input = call_args[0][0]
delay = call_args[1]["delay"]

Testing Checklist

Before committing:

• All imports at top of file
• Using patch.object, not patch
• Mocking dependencies, not SUT
• No mocking of model classes
• Test data uses Pydantic models
• No naked literals - all values are constants
• Mock args accessed with .args[N] / .kwargs["name"]
• E2E tests call actual production code
• Tests verify invariants, not just "no error"
• 100% coverage of new code

Reference Files

For detailed patterns and examples:

• references/mocking.md - Mocking strategies and when to mock
• references/test-data.md - Test data creation patterns
• references/e2e-testing.md - E2E and user journey patterns
• references/testability.md - Extracting testable code

Related Skills

• For debugging bugs, see /pythonista-debugging
• For type safety, see /pythonista-typing
• For code review, see /pythonista-reviewing