tmux-freeze-test

name

tmux-freeze-test

description

TUI/CLI application visual testing via tmux sessions and freeze screenshots. Use this skill when the user wants to: test a TUI app's visual output, capture terminal screenshots for comparison, run side-by-side UI comparisons between applications (e.g., "compare our TUI with Claude Code"), verify slash command output, test tab completion behavior, do black-box E2E testing of terminal applications, or capture evidence of TUI behavior. Triggers on: "tmux test", "freeze screenshot", "visual test", "capture terminal output", "compare UI", "TUI test", "black-box test", "E2E terminal test", screenshots of CLI output.

tmux + freeze Visual Testing

A methodology for testing TUI/CLI applications by running them in controlled tmux sessions, interacting programmatically, and capturing visual output via freeze (SVG) or tmux capture-pane (text fallback).

This approach was battle-tested across 5 rounds of E2E testing (12 black-box agents) on the DeepSeekAgents project. It works for any terminal application.

When to use this

• Verifying TUI rendering (help screens, tab completion, slash commands)
• Side-by-side comparison of multiple CLI tools
• Black-box E2E testing where you act as a real user
• Capturing visual evidence of terminal behavior
• Regression testing of UI changes

Core workflow

1. Create tmux session  -->  2. Run application  -->  3. Send input
       |                           |                        |
       v                           v                        v
4. Wait for render  -->  5. Capture output  -->  6. Analyze / Compare

Step 1: Create a controlled tmux session

Always set explicit dimensions. Different terminal sizes produce different layouts -- inconsistent sizes make comparisons meaningless.

# Standard test session: 120 columns x 40 rows
tmux new-session -d -s SESSION_NAME -x 120 -y 40 "COMMAND_TO_RUN"

Naming convention: Use descriptive session names that encode the test purpose.

tmux new-session -d -s t1-help ...     # Test 1: help display
tmux new-session -d -s t2-tab ...      # Test 2: tab completion
tmux new-session -d -s compare-claude  # Comparison: Claude Code

Why -d: Detached mode. The session runs in background so you can interact programmatically without needing a visible terminal.

Step 2: Send input and wait

Terminal applications need time to render. Always wait after sending input.

# Send a command (Enter submits it)
tmux send-keys -t SESSION_NAME "COMMAND" Enter

# Wait for the application to process and render
sleep 2

# Send partial input (no Enter) -- useful for testing tab completion
tmux send-keys -t SESSION_NAME "/co"
sleep 1
tmux send-keys -t SESSION_NAME Tab
sleep 1

Wait times:

Action	Recommended wait
Application startup	3-5 seconds
Simple command (/help, /clear)	2 seconds
Tab completion trigger	1 second
API-dependent action (chat)	5-10 seconds
Heavy operation (build, test)	10-30 seconds

These are minimums. Increase for slower machines or network-dependent actions.

Step 3: Capture output

Two capture methods, in order of preference:

Method A: freeze (SVG -- preferred)

freeze renders terminal output as a clean SVG image. Best for visual comparison and archival.

freeze \
  --execute "tmux capture-pane -t SESSION_NAME -p" \
  --output OUTPUT_PATH.svg \
  --window \
  --width 120 \
  --height 40

Method B: tmux capture-pane (text fallback)

If freeze is not installed, capture raw text. Still useful for content verification, just not as pretty.

tmux capture-pane -t SESSION_NAME -p > OUTPUT_PATH.txt

Robust capture pattern (try freeze, fall back to text)

freeze --execute "tmux capture-pane -t SESSION_NAME -p" \
  --output OUTPUT.svg --window --width 120 --height 40 \
  2>/dev/null \
  || tmux capture-pane -t SESSION_NAME -p > OUTPUT.txt

Step 4: Clean up

Always kill sessions when done. Leaked tmux sessions consume resources and can interfere with subsequent tests.

tmux kill-session -t SESSION_NAME 2>/dev/null

Organizing test output

Use a structured directory hierarchy. This makes comparison reports and historical tracking straightforward.

.agents/docs/review/tui-test/
  help-display.svg          # Our app: /help output
  tab-completion.svg        # Our app: tab completion
  cost-output.txt           # Our app: /cost command
  claude-help.svg           # Claude Code: /help output
  claude-tab.txt            # Claude Code: tab completion
  codex-help.txt            # Codex: help output
  comparison-report.md      # Side-by-side analysis

For multi-round testing:

.agents/docs/review/e2e/
  round1/
    t1-rust-calculator/     # Test 1
    t2-static-site/         # Test 2
    t3-file-monitor/        # Test 3
  round2/
    t4-skill-creator/
    ...

Side-by-side comparison pattern

When comparing two or more applications, run them all with the same window dimensions and the same commands.

# Application A
tmux new-session -d -s app-a -x 120 -y 40 "our-app"
sleep 3
tmux send-keys -t app-a "/help" Enter
sleep 2
freeze --execute "tmux capture-pane -t app-a -p" --output a-help.svg ...
tmux kill-session -t app-a

# Application B (same dimensions, same command)
tmux new-session -d -s app-b -x 120 -y 40 "their-app"
sleep 3
tmux send-keys -t app-b "/help" Enter
sleep 2
freeze --execute "tmux capture-pane -t app-b -p" --output b-help.svg ...
tmux kill-session -t app-b

Then write a comparison report analyzing differences in:

• Command count and completeness
• Visual formatting and layout
• Parameter hints and help text quality
• Error message clarity
• Overall polish

Black-box E2E testing pattern

For testing an AI agent as a real user would, each test agent should:

1. Have a realistic persona and task -- not "test this feature" but "I'm a junior developer building a REST API"
2. Work in an isolated directory -- avoid cross-contamination
3. Verify file system changes -- after each step, check that expected files exist with expected content
4. Grade the experience -- rate on dimensions like correctness, helpfulness, safety boundary compliance
5. Report bugs with evidence -- file:line, reproduction steps, severity

# Create isolated test environment
mkdir -p /tmp/e2e-test/t1-project && cd /tmp/e2e-test/t1-project

# Start application in that directory
tmux new-session -d -s t1-e2e -x 120 -y 40 \
  "cd /tmp/e2e-test/t1-project && target/release/app code --yolo"

# Interact as a user would
sleep 5
tmux send-keys -t t1-e2e "Create a Rust calculator project with add, subtract, multiply, divide" Enter
sleep 30

# Verify results
ls /tmp/e2e-test/t1-project/src/
cat /tmp/e2e-test/t1-project/Cargo.toml

# Capture final state
tmux capture-pane -t t1-e2e -p > evidence.txt
tmux kill-session -t t1-e2e

Multi-round testing strategy

Break testing into rounds to manage concurrency and prevent resource exhaustion. Each round runs 2-3 tests in parallel.

Round	Tests	Focus
R1	3 parallel	Core functionality (file ops, code gen)
R2	3 parallel	Edge cases (skills, complex projects)
R3	3 parallel	Stress (concurrency, timeouts, errors)
R4	2 parallel	Mode-specific (chat sandbox, friend safety)
R5	2 parallel	Adversarial (path escape, prompt injection)

Common pitfalls

Pitfall: Forgetting to wait after sending input. Fix: Always sleep after tmux send-keys. The application needs render time.

Pitfall: Inconsistent terminal dimensions across captures. Fix: Always pass -x WIDTH -y HEIGHT to tmux new-session.

Pitfall: Leaked tmux sessions from failed tests. Fix: Wrap test sequences in a function that always kills the session in a trap or finally block. List leaked sessions with tmux list-sessions.

Pitfall: freeze not installed. Fix: Fall back to tmux capture-pane -p text capture. Install freeze with go install github.com/charmbracelet/freeze@latest when possible.

Pitfall: Application crashes mid-test and session becomes unresponsive. Fix: Set a timeout. If tmux capture-pane shows no change after the expected wait time, kill the session and log a crash.

Pitfall: API-dependent tests fail due to auth issues. Fix: Verify API keys are valid before starting test rounds. Run a quick smoke test first.

Checklist

Before starting a test session:

• tmux is installed and accessible
• freeze is installed (or text fallback is acceptable)
• Application binary is built (release mode recommended for realistic timing)
• Output directory exists
• API keys are valid (if testing AI features)
• No conflicting tmux sessions from previous runs (tmux list-sessions)

After completing tests:

• All tmux sessions killed
• All captures saved to structured output directory
• Comparison report written (if doing side-by-side)
• Bugs documented with severity and evidence
• Results committed to version control