| name | feature-test |
| description | Guide for creating E2E feature tests with VHS GIF generation and Zola feature page auto-discovery for new visible UI features. |
Feature Test Skill
Use this skill when adding a new visible UI feature that is user-facing and demonstrable
in a PTY scenario. The workflow produces four artifacts from one test:
- An E2E test in
crates/agentty/tests/e2e/.
- A feature GIF in
docs/site/static/features/.
- A static PNG poster in
docs/site/static/features/.
- A Zola content page in
docs/site/content/features/.
When to Use
A feature test is warranted when all three criteria are met:
- Visible UI behavior change — the feature renders something new or different on
screen.
- User-facing — an end user can trigger or observe the behavior through normal
interaction.
- Demonstrable in a scenario — the behavior can be captured in a short PTY recording
without live agent backends.
Skip this skill for internal refactors, backend-only changes, or features that require a
running agent to demonstrate.
Naming Convention
A single name flows through the entire pipeline:
| Artifact | Path |
|---|
| Test function | test_{name} in crates/agentty/tests/e2e/{module}.rs |
| GIF file | docs/site/static/features/{name}.gif |
| PNG poster | docs/site/static/features/{name}.png |
| Zola page | docs/site/content/features/{name}.md |
Choose a short, descriptive snake_case name that describes the feature (e.g.,
session_creation, help_overlay, tab_switch).
Workflow
1. Choose the test module
Place the test in the E2E module that best matches the feature area:
session.rs — session lifecycle and prompt interactions.navigation.rs — tab cycling, help overlay, quit dialog.confirmation.rs — confirmation dialogs.project.rs — project page and project-related flows.
If no existing module fits, create a new one and register it in
crates/agentty/tests/e2e/main.rs.
2. Write the test using FeatureTest
Use the FeatureTest builder from crates/agentty/tests/e2e/common.rs. This is the
preferred pattern — it handles TempDir and BuilderEnv creation, scenario execution,
GIF generation with content-hash caching, and optional Zola page creation in a single
declarative chain.
use testty::assertion;
use testty::region::Region;
use testty::scenario::Scenario;
use crate::common;
use crate::common::FeatureTest;
#[test]
fn test_{name}() {
FeatureTest::new("{name}")
.with_git()
.zola(
"Human-readable title",
"One-line description for the feature card.",
50,
)
.run(
|scenario| {
scenario
.compose(&common::wait_for_agentty_startup())
.compose(&common::switch_to_tab("Sessions"))
.viewing_pause_ms(1500)
.press_key("a")
.wait_for_stable_frame(300, 5000)
.viewing_pause_ms(1500)
.capture_labeled("label", "Description of captured state")
},
|frame, _report| {
let full = Region::full(frame.cols(), frame.rows());
assertion::assert_text_in_region(frame, "expected text", &full);
},
);
}
FeatureTest builder methods
new(name) — set the feature name (used for GIF filename and Zola page)..with_git() — initialize a git repo in the workdir (required for
session/worktree features)..zola(title, description, weight) — enable Zola page auto-generation with the
given frontmatter fields. The page is written only if it does not already exist..run(build_scenario, assert) — execute the scenario, run assertions, and
generate the GIF.
Common Journey helpers
Reuse the shared journey builders from common.rs instead of repeating step sequences:
wait_for_agentty_startup() — wait for the initial TUI frame.switch_to_tab(name) — press Tab and wait for stability.switch_to_tab_reverse(name) — press BackTab and wait.open_quit_dialog() — press q and wait.open_help_overlay() — press ? and wait.create_session_and_return_to_list() — full session creation flow.create_session_with_prompt_and_return_to_list(prompt) — session creation with a
custom prompt.
3. Verify the Zola page
If you used .zola(...), FeatureTest auto-generates the content page at
docs/site/content/features/{name}.md on first run. The generated page uses this
frontmatter:
+++
title = "Feature title"
description = "One-line description shown on the card."
weight = 50
[extra]
gif = "{name}.gif"
+++
The features.html template auto-discovers all pages in content/features/ sorted by
weight. No manual template edits are needed.
4. Create or refresh the PNG poster
Every feature GIF needs a same-named PNG poster for the site's prefers-reduced-motion
and noscript fallbacks. Create the poster when adding a GIF, and regenerate it
whenever the GIF changes. TESTTY_GIF_MODE=check verifies GIF freshness but does not
verify or update the poster.
First inspect the finished GIF and choose a timestamp that clearly communicates the
feature's result. Prefer a stable end state with the important UI visible. Do not
automatically use the first or midpoint frame: it may show an empty, loading, or
transitional state.
Check the GIF duration before choosing a timestamp:
ffprobe -v error \
-show_entries format=duration \
-of default=noprint_wrappers=1 \
docs/site/static/features/<name>.gif
Extract exactly one frame with ffmpeg, preserving the aspect ratio and capping the
width at 1600 pixels without upscaling:
ffmpeg -y \
-i docs/site/static/features/<name>.gif \
-ss <timestamp> \
-vf "scale=w='min(1600\,iw)':h=-1" \
-frames:v 1 \
-compression_level 9 \
docs/site/static/features/<name>.png
Use a timestamp accepted by ffmpeg, such as 00:00:01.500, that falls within the
reported duration. Open the resulting PNG and verify that it is sharp, legible, free of
transitional artifacts, and still represents the current GIF. If it does not, choose a
better timestamp and rerun the command.
Check that every GIF declared by a feature page has a nonempty poster before finalizing:
for feature_page in docs/site/content/features/*.md; do
gif_name=$(sed -n 's/^gif = "\(.*\)"/\1/p' "${feature_page}")
test -z "${gif_name}" && continue
poster_path="docs/site/static/features/${gif_name%.gif}.png"
test -s "${poster_path}" || {
echo "missing PNG poster: ${poster_path}"
exit 1
}
done
5. Run and verify
TESTTY_GIF_MODE=check cargo nextest run --locked --profile ci -p agentty --test e2e test_{name}
prek run zola-check --all-files --hook-stage manual
Routine agent validation must use TESTTY_GIF_MODE=check so the semantic PTY assertions
still run while GIF freshness is checked without invoking VHS or launching Chrome. Use
TESTTY_GIF_MODE=force only when intentionally regenerating GIF assets.
Record in the canonical container
Committed hash sidecars must be produced by the same environment that CI verifies them
in. container/e2e.Containerfile defines that environment: a pinned linux/amd64 image
with the Rust toolchain, prek, cargo-nextest, and the full VHS recording stack
(vhs, ttyd, Chromium, ffmpeg, JetBrains Mono). Presubmit, postsubmit, and release
checks pull this image from GHCR by digest and run the test-agentty-e2e hook inside it
against a read-only checkout. Pull the same immutable digest for local recording instead
of building or recording on the host. The host needs a running Podman environment only —
no local Chrome or VHS — and the localhost-socket sandbox restriction below does not
apply inside the container.
On macOS or Windows, initialize the Podman machine once with podman machine init, then
start it with podman machine start before pulling or running the image. Linux hosts
run Podman directly without a machine.
Record or refresh feature artifacts with a writable workspace mount and generate mode,
which re-records only missing or stale GIFs. Run the local container as the host user so
Linux bind mounts remain writable. A host-owned cache directory provides writable home,
Cargo, prek, and build locations while preserving them between runs:
e2e_image=ghcr.io/agentty-xyz/agentty-e2e@sha256:d348629b6c449d33f5285c9ef2b7ea0ac3c47fcf264b1ec7f3f4c58a6952a696
e2e_cache_root="${XDG_CACHE_HOME:-${HOME}/.cache}/agentty-e2e"
mkdir -p \
"${e2e_cache_root}/home" \
"${e2e_cache_root}/cargo" \
"${e2e_cache_root}/prek" \
"${e2e_cache_root}/target"
podman pull "${e2e_image}"
podman run --rm --platform linux/amd64 \
--user "$(id -u):$(id -g)" \
--mount type=bind,source="$PWD",target=/workspace \
--mount type=bind,source="${e2e_cache_root}",target=/cache \
--env HOME=/cache/home \
--env CARGO_HOME=/cache/cargo \
--env PREK_HOME=/cache/prek \
--env CARGO_TARGET_DIR=/cache/target \
--env TESTTY_GIF_MODE=generate \
"${e2e_image}" \
cargo nextest run --locked --profile ci -p agentty --test e2e test_{name}
The --user override is only for writable local recording. CI does not use it: the
image retains its unprivileged UID 1001 default, and workflows mount the checkout
read-only for check mode.
Review the changed GIF and .{name}.hash sidecar, then refresh the PNG poster for every
regenerated GIF (section 4) before committing all three together. Successful generation
removes the previous same-named PNG intentionally so a stale poster cannot pass the
nonempty-poster integrity check; a failed recording preserves the last valid poster.
Refresh the canonical image
Only maintainers changing container/e2e.Containerfile should build and publish a
replacement image. Build the linux/amd64 candidate, run the affected focused feature
tests with the local tag, then push latest after authenticating Podman to GHCR with
package-write permission:
podman build --platform linux/amd64 \
--file container/e2e.Containerfile \
--tag ghcr.io/agentty-xyz/agentty-e2e:latest container
podman push ghcr.io/agentty-xyz/agentty-e2e:latest
Copy the digest reported by podman push into every workflow E2E_IMAGE value and the
local recording command above. Verify that the digest can be pulled without registry
credentials so forked pull-request CI can use it. Re-record every feature affected by a
tool, browser, font, or rendering change and refresh its PNG poster before committing
the new digest and artifacts together.
Host recording caveats
force mode must run from an unsandboxed shell — a normal user terminal, or a single
agent command explicitly approved to bypass the sandbox. VHS records through localhost
sockets (ttyd plus the Chrome DevTools protocol), and sandboxed agent shells deny all
network, so vhs segfaults in randomPort() before Chrome ever launches. That failure
is easy to misdiagnose as a missing browser binary: having Chrome, vhs, ttyd, and
ffmpeg installed is not enough — the shell must also allow localhost sockets.
In default generate-if-stale mode, machines without VHS still run and assert the test
correctly, then gracefully skip GIF generation. In force mode, VHS must be installed
because regeneration was explicitly requested.
The TESTTY_GIF_MODE env var selects the freshness mode used by FeatureTest:
- unset /
generate / generate-if-stale (default) — regenerate when the on-disk hash
sidecar is missing or stale, otherwise reuse the committed GIF.
check / check-only — compute the would-be hash and compare it to the on-disk
sidecar without invoking VHS or touching the GIF output directory. The harness fails
the test when a committed sidecar has drifted, an existing sidecar is invalid, or the
GIF itself is missing, and surfaces the current/committed hashes plus sidecar errors
so CI catches drift. Existing GIFs that predate sidecars are tolerated until a
recording run creates their baseline. .zola(...) tests without any committed docs
page, GIF, or sidecar are treated as unpublished and skipped by check mode until a
recording run publishes their artifacts.force / always / always-generate — bypass the hash cache and re-run VHS
unconditionally. VHS must be installed: a missing VHS binary fails the test instead of
being silently skipped, because regeneration was explicitly requested.
Run prek run zola-check --all-files --hook-stage manual after the test to catch broken
frontmatter or template integration before the page reaches CI. This requires Zola to be
installed locally — if unavailable, CI catches it via the pages.yml workflow.
Freshness hash determinism
The hash only means "the UI moved" if the same UI hashes the same way every run — and on
every machine, because sidecars are committed locally and checked on Linux CI. The
harness already neutralizes the known variance:
- Temp paths are normalized by testty, and
BuilderEnv keeps every painted directory
under the test HOME so paths render home-collapsed (~/test-project,
~/.agentty/wt/<hash>) with a platform-independent length.
FeatureTest pins the wall clock and UTC offset, redacts the wt/<hash> worktree
name a session derives from its generated UUID (see
common::session_worktree_redaction), and redacts the Agentty v<version> header so
release bumps do not stale every GIF.BuilderEnv stubs every supported agent CLI, so agent availability — and the default
agent a new session resolves — does not depend on which real CLIs a machine has.
Anything else volatile a scenario puts on screen — another generated id, a live
duration, a random port — makes every run look stale and re-records the GIF for nothing.
Freeze it in the app under test, or declare it with FeatureDemo::redact.
Legacy Pattern
Older tests manage TempDir, BuilderEnv, Scenario, and save_feature_gif manually
instead of using FeatureTest. This pattern still works but is not recommended for new
tests. Prefer FeatureTest for all new feature tests.
#[test]
fn legacy_example() {
let temp = tempfile::TempDir::new().expect("failed to create temp dir");
let env = BuilderEnv::new(temp.path()).expect("failed to create builder env");
let scenario = Scenario::new("{name}")
.compose(&common::wait_for_agentty_startup())
.capture_labeled("label", "description");
let (frame, report) = scenario
.run_with_proof(env.builder())
.expect("scenario execution failed");
let full = Region::full(frame.cols(), frame.rows());
assertion::assert_text_in_region(&frame, "expected", &full);
common::save_feature_gif(&scenario, &report, &env, "{name}");
}
When using the legacy pattern, create the Zola page manually at
docs/site/content/features/{name}.md.
Checklist
