Menu
Comprehensive systematic approach to achieving complete test coverage through structured category-based testing
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
Plan before implementing - understand scope and approach with detailed guidance
Comprehensive guide for implementing code incrementally following established patterns, conventions, and quality standards
Comprehensive checklist for documenting follow-up work and testing needs after implementation
Comprehensive guidelines for when and how to use mocks, stubs, and fakes in tests
| name | test-coverage-categories |
| description | Comprehensive systematic approach to achieving complete test coverage through structured category-based testing |
| languages | ["python","typescript","javascript","go","rust","java","csharp","php","ruby"] |
| subagents | ["test/unit","test/integration","code/feature"] |
| tools_needed | ["read","write"] |
A systematic approach to comprehensive test coverage. Work through these categories in order to ensure your tests catch all failure modes, edge cases, and security issues.
Definition: Expected inputs produce expected outputs with no errors.
Purpose: Verify core functionality works under normal conditions.
What to test:
Example:
def test_create_user_happy_path():
"""Test creating user with valid data succeeds."""
user_data = {
"email": "alice@example.com",
"name": "Alice",
"password": "SecurePass123!",
}
user = create_user(user_data)
# Assert expected return value
assert user.email == "alice@example.com"
assert user.name == "Alice"
assert user.id is not None
# Assert expected side effects
assert db.users.count() == 1
assert user.created_at is not None
Common mistakes:
Definition: Test at the edges of valid ranges (min, max, exactly at limit, one over limit).
Purpose: Catch off-by-one errors, overflow issues, and boundary-related bugs.
What to test:
Example:
def test_username_length_boundaries():
"""Test username validation at boundary values."""
# Minimum length (3 chars) - valid
assert is_valid_username("abc") is True
# Below minimum (2 chars) - invalid
assert is_valid_username("ab") is False
# Maximum length (20 chars) - valid
assert is_valid_username("a" * 20) is True
# Above maximum (21 chars) - invalid
assert is_valid_username("a" * 21) is False
# Empty string - invalid
assert is_valid_username("") is False
def test_pagination_boundaries():
"""Test pagination at boundary values."""
# Page size = 1 (minimum)
results = get_users(page=1, size=1)
assert len(results) == 1
# Page size = 100 (maximum)
results = get_users(page=1, size=100)
assert len(results) <= 100
# Page size = 0 (invalid)
with pytest.raises(ValidationError):
get_users(page=1, size=0)
# Page size = 101 (over max)
with pytest.raises(ValidationError):
get_users(page=1, size=101)
Common boundary conditions:
Definition: Test with absent, null, zero, or empty values for each nullable input.
Purpose: Catch null pointer exceptions, division by zero, and missing data handling bugs.
What to test:
None / null / undefined for nullable fields"")[], {})0, 0.0)Example:
def test_calculate_average_with_empty_list():
"""Test average calculation with empty list."""
# Empty list should raise or return None (depending on design)
with pytest.raises(ValueError, match="Cannot calculate average of empty list"):
calculate_average([])
def test_get_user_by_email_with_none():
"""Test user lookup with None email."""
with pytest.raises(ValidationError, match="Email cannot be None"):
get_user_by_email(None)
def test_format_address_with_missing_fields():
"""Test address formatting with missing optional fields."""
address = {
"street": "123 Main St",
"city": "Springfield",
"state": None, # Optional field
"zip": "", # Empty string
}
# Should handle missing fields gracefully
formatted = format_address(address)
assert "123 Main St" in formatted
assert "Springfield" in formatted
# Should not include state or zip if missing
def test_divide_by_zero():
"""Test division with zero divisor."""
with pytest.raises(ZeroDivisionError):
divide(10, 0)
Checklist for nullable inputs:
None""[]{}0Definition: Test when dependencies fail, resources are unavailable, or external calls raise exceptions.
Purpose: Verify error handling, logging, and graceful degradation.
What to test:
Example:
def test_save_user_when_database_unavailable(mocker):
"""Test user save handles DB connection failure gracefully."""
# Mock database to raise connection error
mocker.patch.object(db, "save", side_effect=ConnectionError("DB unavailable"))
user = User(email="alice@example.com")
# Should raise specific error (not generic exception)
with pytest.raises(DatabaseError, match="Failed to save user"):
save_user(user)
# Should log error
assert "DB unavailable" in captured_logs
def test_fetch_user_when_api_returns_500(mocker):
"""Test user fetch handles API 500 error."""
mocker.patch("requests.get", return_value=MockResponse(status=500))
with pytest.raises(ExternalAPIError):
fetch_user_from_external_api("user123")
def test_process_payment_when_gateway_timeout(mocker):
"""Test payment processing handles gateway timeout."""
mocker.patch.object(
payment_gateway,
"charge",
side_effect=TimeoutError("Gateway timeout"),
)
result = process_payment(amount=100, card_token="tok_123")
# Should return failure result (not raise)
assert result.status == "failed"
assert "timeout" in result.error.lower()
# Should not charge user
assert payment_gateway.charge.call_count == 1 # No retries on timeout
Common error scenarios:
Definition: If function has state or depends on execution order, test with different orderings or concurrent access.
Purpose: Catch race conditions, deadlocks, and state corruption from concurrent access.
What to test:
Example:
import asyncio
import pytest
@pytest.mark.asyncio
async def test_concurrent_counter_increments():
"""Test counter handles concurrent increments correctly."""
counter = Counter()
# Run 100 concurrent increments
tasks = [counter.increment() for _ in range(100)]
await asyncio.gather(*tasks)
# Should be exactly 100 (no lost updates)
assert counter.value == 100
def test_order_state_transitions():
"""Test order state transitions enforce valid sequences."""
order = Order()
# Valid sequence: pending → confirmed → shipped → delivered
order.confirm()
assert order.status == OrderStatus.CONFIRMED
order.ship()
assert order.status == OrderStatus.SHIPPED
order.deliver()
assert order.status == OrderStatus.DELIVERED
def test_order_invalid_state_transition():
"""Test order prevents invalid state transitions."""
order = Order()
# Invalid: skip from pending directly to delivered
with pytest.raises(InvalidStateTransitionError):
order.deliver() # Must confirm and ship first
@pytest.mark.asyncio
async def test_concurrent_stock_updates():
"""Test concurrent stock updates don't oversell."""
product = Product(stock=10)
# 20 concurrent purchase attempts (should only succeed 10 times)
tasks = [product.purchase(quantity=1) for _ in range(20)]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Exactly 10 should succeed
successes = [r for r in results if not isinstance(r, Exception)]
assert len(successes) == 10
# Stock should be 0
assert product.stock == 0
When to test concurrency:
When to skip:
Definition: Test that functions enforce access control (who can call it, what they can access).
Purpose: Catch security vulnerabilities where unauthorized users can access restricted resources.
What to test:
Example:
def test_get_user_requires_authentication():
"""Test user retrieval requires authentication."""
# No auth token provided
with pytest.raises(Unauthenticated):
get_user(user_id="123", auth_token=None)
def test_update_user_requires_ownership():
"""Test user update requires ownership or admin role."""
alice = create_user(email="alice@example.com")
bob = create_user(email="bob@example.com")
# Bob tries to update Alice's profile (should fail)
with pytest.raises(Forbidden):
update_user(
user_id=alice.id,
data={"name": "Hacked"},
auth_user_id=bob.id,
)
def test_delete_order_requires_admin_role():
"""Test order deletion requires admin role."""
user = create_user(email="user@example.com", role="user")
admin = create_user(email="admin@example.com", role="admin")
order = create_order(user_id=user.id)
# Regular user cannot delete order
with pytest.raises(Forbidden):
delete_order(order_id=order.id, auth_user_id=user.id)
# Admin can delete order
delete_order(order_id=order.id, auth_user_id=admin.id)
assert get_order(order.id) is None
def test_list_users_filters_by_organization():
"""Test user list enforces organization boundary."""
org_a = create_organization(name="Org A")
org_b = create_organization(name="Org B")
alice = create_user(email="alice@a.com", org_id=org_a.id)
bob = create_user(email="bob@b.com", org_id=org_b.id)
# Alice should only see users in Org A
users = list_users(auth_user_id=alice.id)
assert len(users) == 1
assert users[0].id == alice.id
Authorization patterns to test:
Definition: Test with malicious or malformed inputs designed to exploit vulnerabilities.
Purpose: Catch security vulnerabilities (SQL injection, XSS, path traversal, etc.) and robustness issues.
What to test:
../../../etc/passwd in file paths\0), newlines, tabs%s, %n in format stringsExample:
def test_search_users_with_sql_injection_attempt():
"""Test user search is protected against SQL injection."""
# Attempt SQL injection
malicious_query = "'; DROP TABLE users; --"
# Should NOT execute SQL, should treat as literal string
results = search_users(query=malicious_query)
# Should return empty results (no match) or error, NOT drop table
assert isinstance(results, list)
# Verify table still exists
assert db.table_exists("users")
def test_create_user_with_xss_in_name():
"""Test user creation sanitizes XSS in name field."""
malicious_name = "<script>alert('XSS')</script>"
user = create_user(email="test@example.com", name=malicious_name)
# Should escape or strip script tags
assert "<script>" not in user.name
assert user.name in [
"<script>alert('XSS')</script>", # Escaped
"alert('XSS')", # Stripped tags
"", # Rejected entirely
]
def test_read_file_with_path_traversal_attempt():
"""Test file read prevents path traversal."""
malicious_path = "../../../etc/passwd"
# Should reject or normalize path
with pytest.raises(SecurityError, match="Invalid file path"):
read_file(malicious_path)
def test_username_with_unicode_and_emoji():
"""Test username handles unicode and emoji correctly."""
unicode_names = [
"用户名", # Chinese characters
"Müller", # Umlaut
"José", # Accented character
"🎉🎊", # Emoji
"café", # Combining characters
]
for name in unicode_names:
# Should either accept or reject gracefully (not crash)
try:
user = create_user(email=f"{name}@example.com", name=name)
assert user.name is not None
except ValidationError as e:
# If rejected, should have clear error message
assert "character" in str(e).lower() or "unicode" in str(e).lower()
def test_comment_with_extremely_long_string():
"""Test comment field handles extremely long input."""
# 10MB string
huge_comment = "a" * (10 * 1024 * 1024)
# Should reject or truncate
with pytest.raises(ValidationError, match="too long|exceeds maximum"):
create_comment(text=huge_comment)
def test_email_with_null_bytes():
"""Test email validation rejects null bytes."""
malicious_email = "test@example.com\x00@evil.com"
# Should reject null bytes
with pytest.raises(ValidationError):
create_user(email=malicious_email)
Adversarial input checklist:
'; DROP TABLE, 1' OR '1'='1)<script>, javascript:, onerror=)../, ..\\, absolute paths);, |, &&, backticks)%s, %n, %x)For the function/module being tested, create a checklist:
## Test Coverage for `create_order()`
- [ ] HAPPY PATH - Valid order creation
- [ ] BOUNDARY VALUES - Min/max item count, price ranges
- [ ] EMPTY/NULL - Empty items list, null user_id
- [ ] ERROR CASES - DB failure, payment gateway error
- [ ] CONCURRENT - Multiple orders from same user
- [ ] AUTHORIZATION - Only authenticated users can create orders
- [ ] ADVERSARIAL - XSS in product name, huge item counts
Work through the checklist, implementing tests for each applicable category.
If a category doesn't apply, document why:
## Test Coverage for `calculate_tax()`
- [x] HAPPY PATH - Valid tax calculation
- [x] BOUNDARY VALUES - 0 amount, max amount
- [x] EMPTY/NULL - null amount
- [x] ERROR CASES - Invalid tax rate
- [ ] CONCURRENT - N/A (pure function, no state)
- [ ] AUTHORIZATION - N/A (internal utility, no auth)
- [ ] ADVERSARIAL - N/A (validated numeric input only)
If you can't test a category, explain why and how the risk is mitigated:
## Test Coverage for `process_payment()`
- [x] HAPPY PATH
- [x] ERROR CASES - Gateway timeout, card declined
- [ ] CONCURRENT - **GAP**: Concurrent payments from same user not tested
- **Mitigation**: Payment gateway handles idempotency
- **Risk**: Low (gateway prevents double-charge)
- **TODO**: Add test when we migrate to new gateway (Q3 2026)
Problem: Tests pass with valid inputs but fail in production with edge cases.
Fix: Use this category framework to ensure comprehensive coverage.
Problem: Off-by-one errors, overflow bugs slip through.
Example:
Fix: Always test min, max, exactly-at-limit, and one-over-limit.
Problem: Production errors crash the app instead of being handled gracefully.
Fix: Mock dependencies to throw exceptions and verify error handling.
Problem: SQL injection, XSS, path traversal vulnerabilities discovered in production.
Fix: Test adversarial inputs for all user-facing functions.
Seven Categories for Comprehensive Coverage:
Usage:
Benefits: