com um clique
test-fixer
// Agent for diagnosing and fixing flaky terminal UI tests in the Hex1b test suite. Use when tests pass locally but fail in CI, or when tests exhibit timing-sensitive behavior.
// Agent for diagnosing and fixing flaky terminal UI tests in the Hex1b test suite. Use when tests pass locally but fail in CI, or when tests exhibit timing-sensitive behavior.
Guidelines for reviewing API design in the Hex1b codebase. Use when evaluating public APIs, reviewing accessibility modifiers, or assessing whether new APIs follow project conventions.
Step-by-step guide for creating new widgets in the Hex1b TUI library. Use when implementing new widgets from scratch, including widget records, nodes, extension methods, theming, reconciliation, and tests.
Guidelines for producing accurate and maintainable documentation for the Hex1b TUI library. Use when writing XML API documentation comments, creating end-user guides, or updating existing documentation.
Guidelines for writing unit tests in the Hex1b TUI library. Use when creating new tests for widgets, nodes, or terminal functionality.
Agent for validating Hex1b documentation against actual library behavior. Use when auditing documentation accuracy, testing interactive examples, or identifying discrepancies between documentation and implementation.
Guidelines for running and interpreting Surface API performance benchmarks. Use when modifying code in src/Hex1b/Surfaces/ to ensure performance is not regressed.
| name | test-fixer |
| description | Agent for diagnosing and fixing flaky terminal UI tests in the Hex1b test suite. Use when tests pass locally but fail in CI, or when tests exhibit timing-sensitive behavior. |
This skill provides guidelines for AI agents to diagnose and fix flaky tests in the Hex1b TUI library test suite. These tests use Hex1bTerminalInputSequenceBuilder to simulate user interactions with terminal applications.
| Pattern | Symptom | Fix |
|---|---|---|
| Snapshot After Exit | Test passes locally, fails on Linux CI | Move WaitUntil before Capture, ensure Capture is before exit |
| Missing WaitUntil | Intermittent assertion failures | Add WaitUntil for expected state before Capture |
| Race with Ctrl+C | Snapshot missing expected content | Add WaitUntil between last action and Capture |
| Task.WhenAny Race | Test sometimes times out | Replace with proper WaitUntil or increase timeout |
| Test Interference | Pass isolated, fail in suite | Check for shared state, file locks, or parallel execution issues |
| Platform-Specific | Fails consistently on Windows/Linux | Add platform skip trait or fix platform-specific code |
| Task.Delay for Async Events | Flaky on slower CI runners | Replace Task.Delay with TaskCompletionSource signal |
| Helper Partial Wait | Tests using multi-line helpers fail intermittently | Wait for all/last content, not just first line |
Assert.True() Failure or An item should be selected with indicatorThe ApplyWithCaptureAsync method returns terminal.CreateSnapshot() after all steps complete, not at the point where .Capture() is called. When the sequence includes Ctrl+C to exit the app, the terminal buffer may be cleared before the final snapshot is taken.
Platform difference: Windows terminal buffers persist longer after app exit; Linux clears them more aggressively.
// ❌ BROKEN: Snapshot is taken AFTER Ctrl+C exits the app
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("Counter: 3"), TimeSpan.FromSeconds(2))
.Capture("final") // This saves SVG but doesn't capture for return!
.Ctrl().Key(Hex1bKey.C) // App exits, terminal buffer may be cleared
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
// snapshot is from AFTER Ctrl+C, not from Capture step!
Assert.True(snapshot.ContainsText("Counter: 3")); // May fail on Linux!
Hex1bTerminalInputSequenceBuilder builds a sequence of TestStep objectsApplyWithCaptureAsync executes all steps, then calls terminal.CreateSnapshot() at the endCaptureStep only saves SVG/HTML files—it does NOT store the snapshot for returnCtrl+C triggers app exit, the terminal may clear its buffer before the final snapshotOption A: Ensure WaitUntil is immediately before Capture (Recommended)
// ✅ FIXED: WaitUntil confirms state, then Capture, then exit
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.Key(Hex1bKey.A)
.Key(Hex1bKey.B)
.WaitUntil(s => s.ContainsText("Counter: 3"), TimeSpan.FromSeconds(2))
.Capture("final")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
await runTask;
// If the WaitUntil passed, we know the content was there at Capture time
// The assertion is now checking what WaitUntil already verified
Assert.True(snapshot.ContainsText("Counter: 3"));
Option B: Don't use snapshot for content assertions
// ✅ ALTERNATIVE: Use render counters or other non-snapshot assertions
var runTask = app.RunAsync(TestContext.Current.CancellationToken);
await new Hex1bTerminalInputSequenceBuilder()
.Key(Hex1bKey.A)
.Key(Hex1bKey.B)
.Capture("final")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
await runTask;
// Assert on counters/state captured during execution, not snapshot content
Assert.Equal(1, staticRenderCount);
Option C: Use WaitUntil as the assertion itself
// ✅ ALTERNATIVE: WaitUntil IS the assertion - if it passes, test passes
await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("> Item 15"), TimeSpan.FromSeconds(2))
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyAsync(terminal, TestContext.Current.CancellationToken);
await runTask;
// No additional assertions needed - WaitUntil already verified the content
User input (key press, mouse click) triggers async rendering. Without WaitUntil, the snapshot may be captured before the render completes.
// ❌ BROKEN: No wait after Down() for selection to update
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("First"), TimeSpan.FromSeconds(2))
.Down() // Triggers async render
.Capture("final") // May capture before render completes!
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
Assert.True(snapshot.ContainsText("> Second")); // Flaky!
// ✅ FIXED: Wait for expected state after action
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("First"), TimeSpan.FromSeconds(2))
.Down()
.WaitUntil(s => s.ContainsText("> Second"), TimeSpan.FromSeconds(2)) // Wait for render!
.Capture("final")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
Assert.True(snapshot.ContainsText("> Second")); // Reliable
Ctrl+C can be processed before the previous action's render completes, especially if the action triggers complex layout recalculation.
// ❌ BROKEN: Scroll may not complete before Ctrl+C
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("Item 01"), TimeSpan.FromSeconds(2))
.ScrollDown(5)
.Capture("after_scroll")
.Ctrl().Key(Hex1bKey.C) // May interrupt scroll processing!
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
Assert.True(snapshot.ContainsText("> Item 06")); // Flaky!
// ✅ FIXED: Verify scroll completed before exiting
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("Item 01"), TimeSpan.FromSeconds(2))
.ScrollDown(5)
.WaitUntil(s => s.ContainsText("> Item 06"), TimeSpan.FromSeconds(2)) // Verify scroll!
.Capture("after_scroll")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
Assert.True(snapshot.ContainsText("> Item 06")); // Reliable
| Component | Location | Purpose |
|---|---|---|
Hex1bTerminalInputSequenceBuilder | src/Hex1b/Automation/ | Fluent builder for test sequences |
Hex1bTerminalInputSequence | src/Hex1b/Automation/ | Executes steps and returns final snapshot |
CaptureStep | tests/Hex1b.Tests/CaptureStep.cs | Saves SVG/HTML at capture point |
WaitUntilStep | src/Hex1b/Automation/WaitUntilStep.cs | Polls until condition met or timeout |
TestSequenceExtensions | tests/Hex1b.Tests/TestSequenceExtensions.cs | Adds .Capture() extension method |
// From Hex1bTerminalInputSequence.cs
public async Task<Hex1bTerminalSnapshot> ApplyAsync(Hex1bTerminal terminal, CancellationToken ct = default)
{
foreach (var step in _steps)
{
ct.ThrowIfCancellationRequested();
await step.ExecuteAsync(terminal, _options, ct);
}
return terminal.CreateSnapshot(); // ⚠️ Snapshot is ALWAYS at the end!
}
The CaptureStep only saves files—it does NOT affect what ApplyAsync returns:
// From CaptureStep.cs
internal override Task ExecuteAsync(...)
{
TestCaptureHelper.Capture(terminal, Name); // Saves SVG/HTML only
return Task.CompletedTask;
}
# Get CI logs
gh run view <run-id> --log-failed
# Look for test names and error messages
# Common patterns:
# - Assert.True() Failure
# - Expected: True / Actual: False
# - "An item should be selected with indicator"
# Run specific test locally
dotnet test tests/Hex1b.Tests --filter "FullyQualifiedName~<TestName>"
# Run multiple times to check for flakiness
for ($i = 1; $i -le 10; $i++) {
dotnet test tests/Hex1b.Tests --filter "FullyQualifiedName~<TestName>" --no-build 2>&1 |
Select-String -Pattern "(Passed|Failed)"
}
Look for these anti-patterns:
// Anti-pattern 1: Snapshot assertion after Ctrl+C
.Capture("final")
.Ctrl().Key(Hex1bKey.C)
// ... later ...
Assert.True(snapshot.ContainsText("expected")); // ❌
// Anti-pattern 2: Action without WaitUntil before Capture
.Down()
.Capture("final") // ❌ No WaitUntil after Down()
// Anti-pattern 3: Complex action right before Ctrl+C
.ScrollDown(10)
.Capture("final")
.Ctrl().Key(Hex1bKey.C) // ❌ No WaitUntil to verify scroll completed
WaitUntil after any action that changes stateWaitUntil checks for the exact condition being assertedWaitUntil immediately before Capture[Fact]
public async Task Navigation_DownArrow_SelectsNextItem()
{
using var workload = new Hex1bAppWorkloadAdapter();
using var terminal = Hex1bTerminal.CreateBuilder()
.WithWorkload(workload)
.WithHeadless()
.WithDimensions(40, 10)
.Build();
using var app = new Hex1bApp(
ctx => Task.FromResult<Hex1bWidget>(ctx.List(["First", "Second", "Third"])),
new Hex1bAppOptions { WorkloadAdapter = workload }
);
var runTask = app.RunAsync(TestContext.Current.CancellationToken);
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("> First"), TimeSpan.FromSeconds(2)) // Initial state
.Down()
.WaitUntil(s => s.ContainsText("> Second"), TimeSpan.FromSeconds(2)) // After action
.Capture("after_down")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
await runTask;
// This assertion is technically redundant since WaitUntil verified it
Assert.True(snapshot.ContainsText("> Second"));
}
[Fact]
public async Task StaticWidget_OnlyRendersOnce()
{
using var workload = new Hex1bAppWorkloadAdapter();
using var terminal = Hex1bTerminal.CreateBuilder()
.WithWorkload(workload)
.WithHeadless()
.Build();
var renderCount = 0;
var staticWidget = new TestWidget().OnRender(_ => renderCount++);
using var app = new Hex1bApp(
ctx => Task.FromResult<Hex1bWidget>(staticWidget),
new Hex1bAppOptions { WorkloadAdapter = workload }
);
var runTask = app.RunAsync(TestContext.Current.CancellationToken);
await new Hex1bTerminalInputSequenceBuilder()
.Key(Hex1bKey.A)
.Key(Hex1bKey.B)
.Capture("final")
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
await runTask;
// Assert on counter, not snapshot content
Assert.Equal(1, renderCount);
}
[Fact]
public async Task Render_AllItems_AreVisible()
{
using var workload = new Hex1bAppWorkloadAdapter();
using var terminal = Hex1bTerminal.CreateBuilder()
.WithWorkload(workload)
.WithHeadless()
.Build();
var context = CreateContext(workload);
var node = new ListNode { Items = ["Item 1", "Item 2", "Item 3"] };
node.Arrange(new Rect(0, 0, 20, 5));
node.Render(context);
// No app, no Ctrl+C needed
var snapshot = await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s =>
s.ContainsText("Item 1") &&
s.ContainsText("Item 2") &&
s.ContainsText("Item 3"),
TimeSpan.FromSeconds(2))
.Capture("final")
.Build()
.ApplyWithCaptureAsync(terminal, TestContext.Current.CancellationToken);
Assert.True(snapshot.ContainsText("Item 1"));
Assert.True(snapshot.ContainsText("Item 2"));
Assert.True(snapshot.ContainsText("Item 3"));
}
Task.WhenAny to check if app exitedUsing Task.WhenAny(runTask, Task.Delay(...)) creates a race condition. The delay may win even when the app is about to exit, or the exit may happen but not be detected in time.
// ❌ BROKEN: Race condition with Task.WhenAny
var runTask = app.RunAsync(TestContext.Current.CancellationToken);
await renderTest.Task.WaitAsync(TimeSpan.FromSeconds(1), TestContext.Current.CancellationToken);
await new Hex1bTerminalInputSequenceBuilder()
.Key(Hex1bKey.C, Hex1bModifiers.Control)
.Build()
.ApplyAsync(terminal);
var completed = await Task.WhenAny(runTask, Task.Delay(2000));
Assert.True(completed == runTask, "Expected CTRL-C to exit"); // Flaky!
// ✅ FIXED: Use WaitAsync with timeout instead of Task.WhenAny
var runTask = app.RunAsync(TestContext.Current.CancellationToken);
await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText("expected content"), TimeSpan.FromSeconds(2))
.Ctrl().Key(Hex1bKey.C)
.Build()
.ApplyAsync(terminal, TestContext.Current.CancellationToken);
// Use WaitAsync - throws TimeoutException if it takes too long
await runTask.WaitAsync(TimeSpan.FromSeconds(5), TestContext.Current.CancellationToken);
dotnet test --filter "FullyQualifiedName~TestName"Tests may interfere with each other through:
# Run test in isolation
dotnet test --filter "FullyQualifiedName~TestName" --no-build
# Run test with suspected interfering tests
dotnet test --filter "FullyQualifiedName~TestName|FullyQualifiedName~OtherTest" --no-build
using statements on terminals/workloads// Use unique temp file names
var tempPath = Path.Combine(Path.GetTempPath(), $"hex1b_test_{Guid.NewGuid()}.cast");
// Or include test name
var tempPath = Path.Combine(Path.GetTempPath(), $"hex1b_{nameof(MyTestMethod)}.cast");
| Feature | Windows | Linux |
|---|---|---|
| PTY support | Limited (ConPTY) | Native |
| Terminal buffer cleanup | Delayed | Immediate |
| File locking | Strict | Flexible |
| Path separators | \ | / |
// Skip on Windows - PTY tests require Unix
[Fact]
[Trait("Category", "LinuxOnly")]
public async Task WithPtyProcess_ExecutesProcess()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
// Skip on Windows - PTY not fully supported
return;
}
// ... test code
}
// Or use Skip property
[Fact(Skip = "Requires Unix PTY support")]
public async Task PtyTest() { }
// Or conditional skip with xUnit
public class LinuxOnlyFactAttribute : FactAttribute
{
public LinuxOnlyFactAttribute()
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
Skip = "This test requires Linux";
}
}
}
Hex1bTerminalBuilderTests.WithPtyProcess_* - Require PTYNanoExploratoryTests.* - Require nano and PTYWithPtyProcess() builder methodTask.Delay(100)) for an async event like input binding firingUsing Task.Delay to wait for async events like input bindings firing is unreliable. The fixed delay may not be long enough on slower CI runners, or may be unnecessarily long on fast machines.
Example: Input binding tests that send a key and wait for the binding callback to fire.
// ❌ BROKEN: Fixed delay may not be long enough on slow CI runners
var bindingFired = false;
using var app = new Hex1bApp(
ctx =>
{
var vstack = new VStackWidget([ctx.Test()])
.InputBindings(bindings =>
{
bindings.Shift().Key(key).Action(_ =>
{
bindingFired = true; // Sets flag when binding fires
return Task.CompletedTask;
}, $"Test Shift+{key}");
});
return Task.FromResult<Hex1bWidget>(vstack);
},
new Hex1bAppOptions { WorkloadAdapter = workload }
);
// ... wait for render ...
await new Hex1bTerminalInputSequenceBuilder()
.Shift().Key(key)
.Build()
.ApplyAsync(terminal, TestContext.Current.CancellationToken);
await Task.Delay(100); // ❌ Fixed delay - may not be long enough!
Assert.True(bindingFired); // Flaky!
Replace the boolean flag and Task.Delay with a TaskCompletionSource that signals when the event occurs:
// ✅ FIXED: Use TaskCompletionSource to wait for the async event
var bindingFired = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
using var app = new Hex1bApp(
ctx =>
{
var vstack = new VStackWidget([ctx.Test()])
.InputBindings(bindings =>
{
bindings.Shift().Key(key).Action(_ =>
{
bindingFired.TrySetResult(); // Signal completion
return Task.CompletedTask;
}, $"Test Shift+{key}");
});
return Task.FromResult<Hex1bWidget>(vstack);
},
new Hex1bAppOptions { WorkloadAdapter = workload }
);
// ... wait for render ...
await new Hex1bTerminalInputSequenceBuilder()
.Shift().Key(key)
.Build()
.ApplyAsync(terminal, TestContext.Current.CancellationToken);
// Wait for the event with a timeout - will fail fast if binding doesn't fire
await bindingFired.Task.WaitAsync(TimeSpan.FromSeconds(2), TestContext.Current.CancellationToken);
// If we got here, the binding fired (the wait would have timed out otherwise)
Assert.True(bindingFired.Task.IsCompleted); // Reliable
TaskCompletionSource instead of a boolean flagTrySetResult() in the event handler to signal completionWaitAsync with timeout instead of Task.DelayTaskCreationOptions.RunContinuationsAsynchronously to avoid potential deadlocksTest helper methods that write multiple lines of content may only wait for the first line to appear before taking a snapshot. On faster CI systems, the snapshot may be captured before all lines are processed by the output pump.
Example: A helper writes lines ["A", "B"] but only waits for "A" to appear. Tests that expect "B" on a separate line may fail because the snapshot is taken before "B" is processed.
// ❌ BROKEN: Only waits for first line
private static async Task<Hex1bTerminalSnapshot> CreateSnapshotAsync(string[] lines)
{
using var workload = new Hex1bAppWorkloadAdapter();
using var terminal = Hex1bTerminal.CreateBuilder().WithWorkload(workload).Build();
foreach (var line in lines)
{
workload.Write(line + "\r\n");
}
// BUG: Only waits for first line!
var firstLine = lines.Length > 0 ? lines[0] : "";
await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => s.ContainsText(firstLine), TimeSpan.FromSeconds(1))
.Build()
.ApplyAsync(terminal);
return terminal.CreateSnapshot(); // May miss content after first line!
}
// ✅ FIXED: Wait for last line to ensure all content is processed
private static async Task<Hex1bTerminalSnapshot> CreateSnapshotAsync(string[] lines)
{
using var workload = new Hex1bAppWorkloadAdapter();
using var terminal = Hex1bTerminal.CreateBuilder().WithWorkload(workload).Build();
foreach (var line in lines)
{
workload.Write(line + "\r\n");
}
// Wait for the LAST line to ensure all lines are written
var lastLine = lines.Length > 0 ? lines[^1] : "";
await new Hex1bTerminalInputSequenceBuilder()
.WaitUntil(s => string.IsNullOrEmpty(lastLine) || s.ContainsText(lastLine),
TimeSpan.FromSeconds(1), "last line content")
.Build()
.ApplyAsync(terminal);
return terminal.CreateSnapshot(); // Now contains all lines
}
WaitUntil condition only checks for initial/first contentBefore committing test changes, verify:
WaitUntilWaitUntil condition matches what will be assertedCapture is placed after WaitUntil, before Ctrl+CWaitUntil, not Wait)Task.WhenAny race conditions - use WaitAsync insteadTask.Delay for async events - use TaskCompletionSource instead