testing-quality

name	testing-quality
description	Apply LogHunter test-first development, pytest structure, coverage targets, Ruff, Pyright, build checks, deterministic fixtures, and CI expectations.
license	MIT
compatibility	opencode
metadata	{"project":"loghunter-cli","quality":"testing-ci"}

Testing and Quality

Test-first rule

All behavior-changing work in the parser, analyzer, detector, CLI, and presentation layers follows:

Failing test
  -> minimal implementation
  -> refactor
  -> focused verification
  -> full verification

A behavior-changing patch without a regression or feature test is incomplete unless the behavior cannot reasonably be automated and the limitation is documented.

Test structure

Recommended layout:

tests/
├── conftest.py
├── fixtures/
│   └── logs/
├── unit/
│   ├── test_models.py
│   ├── test_parser.py
│   ├── test_analyzer.py
│   ├── test_detector.py
│   └── test_exporters.py
├── integration/
│   ├── test_analyze_command.py
│   ├── test_top_ips_command.py
│   ├── test_validate_command.py
│   └── test_export_command.py
└── test_cli.py

Use the smallest appropriate layer. Pure parser logic belongs in unit tests; complete command behavior belongs in integration tests.

Fixtures

• Use synthetic log records.
• Never commit real private authentication logs.
• Keep each fixture focused.
• Prefer inline strings for one-line parser cases.
• Use fixture files for multi-line and end-to-end scenarios.
• Use tmp_path for temporary input and output files.
• Checked-in repository samples belong under samples/ and must be synthetic, deterministic, strict UTF-8, and safe for public release.

Determinism

Tests must not depend on:

• Network access.
• The developer's real system logs.
• Local timezone unless explicitly fixed.
• The current year unless injected.
• File iteration order.
• Random values without a fixed seed.
• Terminal width unless configured in the test.

Parser tests

For every supported event family include:

• Canonical line.
• Valid variation.
• Malformed line.
• Unrelated line.
• IPv4 or IPv6.
• Correct line number.
• Correct enum and fields.

Detector tests

Include:

• Below threshold.
• At threshold.
• Above threshold.
• Exact time boundary.
• Outside time boundary.
• Independent source addresses.
• Severity transitions.
• Duplicate suppression.
• Invalid configuration.

Analyzer tests

Include semantic failed/success/standalone-invalid counts, the failed-invalid distinction, source and username aggregation, true timestamp extrema independent of input order, deterministic ranking ties, generator and single-pass consumption, empty streams, and parser-stat count mismatches.

CLI tests

Use:

from typer.testing import CliRunner

Test:

• Exit code.
• stdout.
• stderr when relevant.
• Created files.
• Machine-readable output validity.
• Absence of tracebacks for expected errors.

For the current analyze slice, cover mixed supported/ignored input, required year, detector defaults and overrides, top-source presentation limits, no-color output, fail-on-detection, missing/non-file/ empty/invalid-UTF-8 inputs, unsupported-only files, numeric validation, chronology inconsistency, IPv6 canonicalization, and generic unexpected-failure redaction. Keep root help/version tests and pure output tests separate. For the current top-ips slice, cover the same input/error translations, deterministic ranking, --limit boundaries, canonical IPv6 output, no-color output, analyzer error translation, detector non-invocation, and generic unexpected-failure redaction. For the current validate slice, cover mixed, full, and zero coverage, report-before-exit-4 behavior, required-year and input errors, no-color output, one collect_parse_stats pass, analyzer/detector/event-stream non-invocation, and generic unexpected-failure redaction.

For the export slice, cover required and invalid options, silent JSON/CSV success, findings still exiting 0, custom threshold/window values, IPv6 canonicalization, exactly three incremental public passes, analyzer and detector inconsistency translation, missing/empty/non-file/invalid-UTF-8 and unsupported-only inputs, exit 5 destination failures, force replacement, source equality, symlinks, terminal-control-safe diagnostics, and generic unexpected failures. Unit exporter tests must assert the exact versioned schemas, scalar types, timestamps, enum values, Unicode, domain ordering, formula-leading username protection, LF-only output, and temporary-write failure preservation.

Coverage

Targets:

• Overall: at least 85 percent.
• Parser: at least 90 percent.
• Detector: at least 90 percent.
• Analyzer: at least 90 percent.

Coverage is a guardrail, not a replacement for meaningful boundary assertions.

Static quality

Run:

uv run ruff format --check .
uv run ruff check .
uv run pyright

Do not silence rules broadly to make checks pass. Prefer a local, justified suppression when necessary.

Test commands

Focused:

uv run pytest tests/unit/test_parser.py
uv run pytest tests/unit/test_analyzer.py
uv run pytest tests/unit/test_detector.py
uv run pytest tests/unit/test_exporters.py
uv run pytest tests/test_cli.py
uv run pytest tests/unit/test_output.py
uv run pytest tests/integration/test_analyze_command.py
uv run pytest tests/integration/test_top_ips_command.py
uv run pytest tests/integration/test_validate_command.py
uv run pytest tests/integration/test_export_command.py

Full:

uv run pytest
uv run pytest --cov=loghunter --cov-report=term-missing

Packaging:

uv build
uv run loghunter --help
uv run loghunter --version
uv run loghunter analyze --help
uv run loghunter top-ips --help
uv run loghunter validate --help
uv run loghunter export --help
uv run python -m loghunter --help
uv run python -m loghunter analyze --help
uv run python -m loghunter top-ips --help
uv run python -m loghunter validate --help
uv run python -m loghunter export --help

Stable-release CI expectations

The release-readiness workflow must cover Python 3.12, 3.13, and 3.14 on pull requests and pushes to main. Grant only contents: read repository permissions and use the lockfile.

Jobs should verify:

• uv sync --locked --dev.
• Ruff formatting.
• Ruff linting.
• Pyright strict mode.
• Pytest with coverage.
• Package build.
• Installed CLI smoke test from the built wheel.

The artifact smoke test must create a fresh environment under the runner's temporary directory, install dist/loghunter_cli-1.0.0-py3-none-any.whl into that environment, change outside the checkout, and invoke the installed loghunter --help, loghunter --version, and python -m loghunter --help commands. It must not accidentally import the source checkout or use the project environment.

Review checklist

• New behavior has a failing-first test or clear regression test.
• Boundaries are asserted.
• Fixtures are synthetic.
• Tests are deterministic.
• Error behavior is tested.
• Public output is tested.
• Reported commands were actually run.