testing

name	testing
description	Run Python and Rust tests for physicsnemo-curator using uv + pytest with coverage reporting via pytest-cov, and cargo-nextest for Rust tests. Includes benchmark workflows with pytest-benchmark and criterion.

Testing Skill

Prerequisites

Ensure dev dependencies are installed:

make install        # or: uv sync --group dev
make develop        # build native Rust extension: uv run maturin develop

Python Tests

Run the full test suite with coverage

uv run pytest test/ --cov --cov-report=term-missing

Or use the Makefile shortcut:

make test

Run a single test file

uv run pytest test/test_example.py -v

Run a single test function

uv run pytest test/test_example.py::test_my_function -v

Useful pytest flags

Flag	Purpose
-v	Verbose output
-x	Stop on first failure
-k "expression"	Run tests matching expression
-m "not slow"	Skip tests marked as slow
--tb=long	Full tracebacks
--pdb	Drop into debugger on failure
-n auto	Parallel execution (requires pytest-xdist)

Coverage reports

# Terminal report with missing lines (default in make test)
uv run pytest test/ --cov --cov-report=term-missing

# HTML report (opens in browser)
uv run pytest test/ --cov --cov-report=html
# Output: htmlcov/index.html

# XML report (for CI integration)
uv run pytest test/ --cov --cov-report=xml
# Output: coverage.xml

# Combined reports
uv run pytest test/ --cov --cov-report=term-missing --cov-report=html --cov-report=xml

Coverage configuration is in pyproject.toml under [tool.coverage.run] and [tool.coverage.report]. The minimum coverage threshold is set to 80%.

Python benchmarks

# Run benchmarks only
uv run pytest test/ --benchmark-only

# Run benchmarks with comparison
uv run pytest test/ --benchmark-compare

# Skip benchmarks during normal test runs
uv run pytest test/ --benchmark-skip

Benchmark tests should be marked with @pytest.mark.benchmark and use the benchmark fixture from pytest-benchmark.

Rust Tests

Run Rust tests with nextest

cargo nextest run --manifest-path src/rust/Cargo.toml

Or use the Makefile shortcut:

make test-rust

Run specific Rust tests

cargo nextest run --manifest-path src/rust/Cargo.toml -E "test(parse)"

Rust benchmarks (criterion)

cargo bench --manifest-path src/rust/Cargo.toml

Benchmark results are saved to src/rust/target/criterion/. HTML reports are generated when the html_reports feature is enabled.

Run everything

make bench    # Python benchmarks + Rust benchmarks
make check    # format-check + lint-check + typecheck + interrogate + deny

ASV Historical Benchmarks

ASV (airspeed velocity) tracks performance across the project's git history, producing an interactive web dashboard.

Configuration

ASV is configured in asv.conf.json at the project root. Benchmarks live in benchmarks/ (separate from the pytest test suite in test/). All ASV artifacts (environments, results, HTML) are stored under .asv/ and gitignored.

Running ASV benchmarks

# Benchmark the current commit
make asv-run
# Equivalent to: uv run asv run --quick HEAD^!

# Dry-run (smoke test, no results saved)
make asv-quick

# Benchmark a range of commits
uv run asv run v0.1.0..HEAD

# Compare two revisions
make asv-compare REF1=main REF2=HEAD
# Equivalent to: uv run asv compare main HEAD

# Find the commit that introduced a regression
uv run asv find v0.1.0..HEAD TimePipelineIteration.time_iterate_all

# Show results for a specific commit
uv run asv show HEAD

Publishing the dashboard

# Build static HTML dashboard
make asv-publish
# Output: .asv/html/

# Preview locally
make asv-preview

# Push to GitHub Pages
uv run asv gh-pages

Writing ASV benchmarks

ASV benchmarks use magic name prefixes:

Prefix	Measures
time_	Wall-clock execution time
mem_	Memory footprint of returned object
peakmem_	Peak resident memory
track_	Arbitrary numeric value
timeraw_	Execution time in a fresh subprocess

Benchmarks support setup(), teardown(), setup_cache() lifecycle methods and parameterization via params / param_names class attributes.

See benchmarks/bench_pipeline.py for working examples.