regression-test

name	regression-test
description	Run regression tests for voxtype releases. Use before major releases to verify core functionality, CLI commands, and configuration handling.
user-invocable	true
allowed-tools	["Bash","Read","Glob","Grep"]

Regression Test

Comprehensive testing checklist for voxtype releases.

Note: For detailed manual smoke tests (recording cycles, GPU isolation, output drivers, etc.), see docs/SMOKE_TESTS.md. Run those tests for thorough pre-release validation.

Quick Smoke Test

# Build and basic checks
cargo build --release
./target/release/voxtype --version
./target/release/voxtype --help
./target/release/voxtype setup --help

Unit Tests

cargo test

Key test modules:

• text:: - Spoken punctuation and replacements
• cli:: - Command-line argument parsing
• state:: - State machine transitions

CLI Command Tests

Setup Commands

# List available models
./target/release/voxtype setup --list-models

# Show current configuration
./target/release/voxtype setup --show-config

# Check GPU detection (if available)
./target/release/voxtype setup gpu --status

Status Commands

# Check daemon status (will fail if not running, that's ok)
timeout 2 ./target/release/voxtype status || echo "Daemon not running (expected)"

# JSON output format
timeout 2 ./target/release/voxtype status --format json || echo "Daemon not running (expected)"

Transcription Test

# Test with a sample audio file (if available)
./target/release/voxtype transcribe test.wav

Configuration Tests

Default Config Loading

# Should not error with missing config
rm -f ~/.config/voxtype/config.toml
./target/release/voxtype --help

# Should load config without errors
mkdir -p ~/.config/voxtype
cp config/default.toml ~/.config/voxtype/config.toml
./target/release/voxtype setup --show-config

Config Backwards Compatibility

Test that old config files still work:

# Create minimal old-style config
cat > /tmp/test-config.toml << 'EOF'
[hotkey]
key = "SCROLLLOCK"

[whisper]
model = "base.en"
EOF

# Should not error
./target/release/voxtype --config /tmp/test-config.toml --help

Binary Variant Tests

For each binary variant, verify:

VERSION=0.4.14

# AVX2
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-avx2 --version
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-avx2 --help

# AVX-512
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-avx512 --version
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-avx512 --help

# Vulkan
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-vulkan --version
releases/${VERSION}/voxtype-${VERSION}-linux-x86_64-vulkan --help

Integration Tests

Daemon Lifecycle

# Start daemon
./target/release/voxtype &
DAEMON_PID=$!
sleep 2

# Check it's running
./target/release/voxtype status

# Stop daemon
kill $DAEMON_PID

Signal Handling

./target/release/voxtype &
DAEMON_PID=$!
sleep 1

# SIGUSR1 should start recording (will fail without audio, ok)
kill -USR1 $DAEMON_PID

# SIGTERM should graceful shutdown
kill -TERM $DAEMON_PID

Package Tests

Debian Package

# Validate structure
dpkg-deb --info releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb

# List contents
dpkg-deb --contents releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb

# Check for required files
dpkg-deb --contents releases/${VERSION}/voxtype_${VERSION}-1_amd64.deb | grep -E 'voxtype-avx2|voxtype-avx512|config.toml|voxtype.service'

RPM Package

rpm -qp --info releases/${VERSION}/voxtype-${VERSION}-1.x86_64.rpm
rpm -qp --list releases/${VERSION}/voxtype-${VERSION}-1.x86_64.rpm

Checklist for Major Releases

• cargo test passes
• cargo clippy has no warnings
• All binary variants build successfully
• Binary instruction validation passes (no AVX-512 in AVX2/Vulkan)
• Version numbers match across all binaries
• CLI --help output is correct
• Default config loads without errors
• Old configs still work (backwards compatibility)
• Packages install correctly on target distros
• Daemon starts and stops cleanly
• Recording/transcription works end-to-end (manual test)

Known Test Limitations

• Audio capture requires real audio hardware (can't fully test in CI)
• evdev hotkey detection requires /dev/input access
• Transcription requires downloaded Whisper model
• GPU acceleration requires Vulkan-capable hardware