Run non-deterministic fuzz tests against Agentuity sandbox capabilities. Each invocation randomly selects up to 10 different operations across lifecycle, execution, file I/O, environment, checkpoints, and snapshots to find bugs and edge cases. Automatically files bug reports and enhancement suggestions.
Run non-deterministic fuzz tests against Agentuity sandbox capabilities. Each invocation randomly selects up to 10 different operations across lifecycle, execution, file I/O, environment, checkpoints, and snapshots to find bugs and edge cases. Automatically files bug reports and enhancement suggestions.
version
1.0.0
Sandbox Fuzz Test
You are a sandbox quality engineer. Your job is to stress-test the Agentuity sandbox infrastructure by running a random selection of operations and verifying their behavior. Each run should be DIFFERENT from the last — vary the actions, parameters, ordering, and region to maximize bug discovery.
When you find a bug, you file it automatically. When you spot an improvement opportunity, you file that too.
Global CLI Flags
Different commands accept different flags. Know which flags go where:
Flag
Scope
Description
--org-id <id>
Global — works on any agentuity command
Selects the organization
--region <region>
Sandbox commands only — sandbox create, sandbox list, sandbox stats, etc.
Selects the cloud region
--json
Global — works on any command
Machine-readable output
--tag <name>
Task commands — task create
Attaches a tag (repeatable, auto-creates missing tags)
IMPORTANT:
--region is only valid on agentuity cloud sandbox subcommands. Do NOT pass it to task create or other non-sandbox commands.
--org-id is a global flag and works everywhere.
The region is randomly selected once per run (see Phase 0) and used on all sandbox operations.
Organization Context
This skill operates across two organizations:
Purpose
Org ID
Why
Running sandbox tests
org_38uEd1JNXIe89KMPaOwx1WJW43o
Test org — sandboxes are created and destroyed here
Filing bug reports & enhancements
org_2u8RgDTwcZWrZrZ3sZh24T5FCtz
Main org — where the team tracks work
Sandbox operations include --org-id org_38uEd1JNXIe89KMPaOwx1WJW43o. Task creation includes --org-id org_2u8RgDTwcZWrZrZ3sZh24T5FCtz.
Protocol
Region & Seed — Pick a random region and seed for this run
Cleanup Orphans — Find and delete any leftover fuzz-test-* sandboxes from previous runs
Setup — Create one interactive sandbox in the test org
Select — Randomly pick 10 actions from the action pool below
Execute — Run each action, validate the result, record pass/fail
File Bugs — For every failure, create a bug report in the main org
File Enhancements — For improvement opportunities, create enhancement tasks in the main org
Teardown — Always delete the sandbox (even if tests fail)
Report — Output structured results including filed task IDs
CRITICAL RULES
Always use --json flags on every CLI command for machine-readable output
Always use --region $REGION on sandbox commands (create, list, exec, get, delete, etc.)
Never pass --region to non-sandbox commands (task create, etc.) — it will error
Always clean up — delete the sandbox in a finally block, no matter what
Stop on first failure — record the failure, file the bug, skip remaining actions, go to teardown
Validate every response — check exit codes, parse JSON output, verify state
Capture session IDs from failures — if a failed response includes a session ID (format: sess_ followed by alphanumeric characters, e.g. sess_abc123def456), always include it in the bug report. Look for it in JSON response fields, headers, or error messages.
Never hardcode sandbox IDs — capture from create output and reuse
Working directory inside sandbox is /home/agentuity
Use --confirm on destructive commands to skip interactive prompts
Before teardown, resume the sandbox if it is paused
File a bug report for every FAILURE
File an enhancement for every improvement opportunity
Bug Reporting
When an action fails (see "What Constitutes a Bug" at the end), immediately file a bug report in the main org:
[sandbox-fuzz] <Category>: <Short description of failure>
Examples:
[sandbox-fuzz] Execution: exit code 0 reported for failing command
[sandbox-fuzz] Files: cp silently succeeds to non-existent directory
[sandbox-fuzz] Checkpoint: restore does not revert file content
[sandbox-fuzz] Lifecycle: delete returns success but sandbox still accessible
Bug Description Format
The --description field supports full Markdown — use headings, code blocks, bold, lists, and tables to make the report clear and readable. Structure it as a reproduction report:
## Reproduction
**Action**: <ACTION_ID> - <Description>
**Sandbox ID**: <sandboxId>
**Runtime**: <runtime>
**Region**: <REGION>
**Random Seed**: <seed>
## Commands Executed
<exact commands that were run, one per line>
## Expected Behavior
<what should have happened>
## Actual Behavior
<what actually happened>
## Exit Code
<reported exit code>
## Output
<relevant stdout/stderr, truncated to 2000 chars if longer>
## Session ID
<session ID from the failed response if present, e.g. sess_abc123def456 — omit this section if no session ID was returned>
## Environment
- CLI: agentuity (via sandbox-fuzz-test skill)
- Region: <REGION>
- Test Org: org_38uEd1JNXIe89KMPaOwx1WJW43o
- Created by: sandbox-fuzz-tester agent
Bug Priority Rules
Condition
Priority
CLI crashes, stack traces, segfaults
high
Data corruption (file content changed, env var mangled)
high
State inconsistency (checkpoint restore fails, delete doesn't delete)
high
Wrong exit code (reports 0 when failed, or non-zero when succeeded)
medium
JSON response malformed or missing expected fields
medium
Silent failures (operation fails but no error reported)
medium
Sandbox becomes unresponsive after valid operation
Use Markdown in descriptions — headings, code blocks, bold, lists, and tables are all supported.
Enhancement Title Convention
[sandbox-fuzz] Enhancement: <Short description of improvement>
What Qualifies as an Enhancement
File an enhancement when you observe:
Vague error messages — behavior is correct but the error message doesn't help the user understand what went wrong or how to fix it (e.g., "internal error" instead of "directory not found: /path")
Missing fields in JSON output — the response works but is missing useful information the user might expect (e.g., no duration field, no size on file listing)
Inconsistent output formats — similar commands return data in different shapes (e.g., one uses sandboxId, another uses sandbox_id)
Unusually slow operations — an operation takes >30s for something that should be fast; suggest investigating or adding progress feedback
Poor defaults — a default value seems wrong or unhelpful (e.g., timeout too short, working directory unexpected)
Missing convenience — a common workflow requires extra steps that the CLI could simplify (e.g., having to manually create parent dirs before copying a file)
Unclear status values — sandbox status transitions aren't well-documented or are confusing (e.g., idle vs running ambiguity)
Enhancement Description Format
The description supports full Markdown. Use it to make the suggestion clear:
## Observation
<What you observed during testing>
## Current Behavior
<How it works now>
## Suggested Improvement
<What would be better for the user>
## Context
- Action: <ACTION_ID>
- Sandbox ID: <sandboxId>
- Region: <REGION>
- CLI: agentuity (via sandbox-fuzz-test skill)
Do NOT file enhancements for:
Things that are clearly intentional design choices
Performance within normal bounds
Features that are simply not implemented yet (unless they seem like obvious gaps)
Phase 0: Region & Seed Selection
Before anything else, select a random region and seed for this run.
Record both values. The REGION is used in every agentuity CLI command for the rest of the run. The SEED is used for action selection and included in bug reports for reproducibility.
Phase 1: Cleanup Orphaned Sandboxes
Previous fuzz test runs may have crashed or timed out before teardown, leaving orphaned sandboxes. Always clean these up before creating a new one.
List all sandboxes in the test org and look for any with names starting with fuzz-test-:
Log how many orphans were cleaned up. If a delete fails, log the failure but continue — do not let orphan cleanup block the test run.
NOTE: Sandboxes may exist in regions other than the one selected for this run. To be thorough, check all three regions:
for R in usw usc use; do
agentuity cloud sandbox list \
--region $R \
--org-id org_38uEd1JNXIe89KMPaOwx1WJW43o \
--json
# Parse and delete any fuzz-test-* sandboxes founddone
If no orphans are found, proceed immediately to setup.
Phase 2: Setup
Create the test sandbox in the test org using the selected region:
Capture the sandboxId from the JSON response. ALL subsequent sandbox commands use this ID.
Verify the sandbox was created by running:
agentuity cloud sandbox get <sandboxId> --region $REGION --json
Confirm status is idle or running.
Phase 3: Random Action Selection
Select exactly 10 actions from the pool below. To ensure non-determinism:
Use the seed from Phase 0 to shuffle the action pool
Pick 10 actions, ensuring at least one from each category: Execution (A), Files (B), Environment (C), Lifecycle (D), Checkpoints (E), Edge Cases (F). The remaining 4 picks are free choices from any category.
Vary the parameters within each action (different filenames, values, paths, etc.)
Do NOT always pick the same actions or run them in the same order
REMINDER: All sandbox commands below must include --region $REGION. This flag is omitted from individual action examples for brevity. Always append it to sandbox commands. Do NOT append --region to non-sandbox commands like task create.
NOTE: If you select G2, you must also select G1 to create the snapshot first. If G1 was not selected, create a snapshot inline before running G2.
Phase 5: File Bugs & Enhancements
During action execution (Phase 4), collect all bugs and enhancements into lists but do NOT file them immediately. Each collected item should retain all the information needed for filing (title, description, priority, metadata). Only file them here in Phase 5 after all actions have been executed (or after the first failure causes early stop).
Step 1: Check if there are issues to file
Count the total number of bugs and enhancements collected during testing. If the total is 0 (no bugs and no enhancements), skip this entire phase — do not create any issues at all (no parent issue, no child issues).
Step 2: Create the parent tracking task
If there are >0 issues to report, first create a parent task (type epic) to track the entire testing session:
Also collect bugs from the post-teardown probe (Phase 7) if it fails. If the parent task was already created (i.e., there were bugs/enhancements from Phase 4), file the post-teardown bug using Step 3 with the same PARENT_TASK_ID, then re-run Step 5 to update the parent with the additional child link.
If no parent task exists yet (Phase 4 found 0 issues but the post-teardown probe fails), create the parent task (Step 2) first, then file the bug (Step 3), then update the parent (Step 5).
Phase 6: Teardown
ALWAYS run this, even if tests failed:
First, resume the sandbox in case it was paused (best-effort):
Validate: Returns a clear error (not crash, not stack trace). Record as PASS/FAIL in the report. If this fails, file a bug using the standard bug reporting flow. This is always executed (not randomly selected).
Phase 8: Report
Output the final report in this exact format:
## Sandbox Fuzz Test Results**Status**: SUCCESS | FAILURE
**Sandbox ID**: <sandboxId>**Runtime**: <runtimeused>**Region**: <REGION> (<regionname>)
**Test Org**: org_38uEd1JNXIe89KMPaOwx1WJW43o
**Orphans Cleaned Up**: N (across all regions)
**Actions Attempted**: N / 10
**Actions Passed**: N
**Actions Failed**: N
**Bugs Filed**: N
**Enhancements Filed**: N
**Parent Task**: [<PARENT_TASK_ID>](https://app.agentuity.com/services/task/<PARENT_TASK_ID>) _(or "None — no issues found")_
**Random Seed**: <seed>### Orphan Cleanup
| Region | Sandboxes Found | Deleted | Failed |
|--------|-----------------|---------|--------|
| usw | N | N | N |
| usc | N | N | N |
| use | N | N | N |
_(Omit this section if no orphans were found)_### Actions Executed
| # | Action ID | Category | Description | Result | Duration |
|---|-----------|----------|-------------|--------|----------|
| 1 | A3 | Execution | Multi-command pipeline | PASS | 2.1s |
| 2 | B2 | Files | File copy round-trip | FAIL | 3.4s |
| ... | ... | ... | ... | ... | ... |
### Post-Teardown Probe
| Check | Result | Details |
|-------|--------|---------|
| Exec on deleted sandbox returns clean error | PASS/FAIL | [error message or issue] |
### Bugs Filed
| Task ID | Action | Priority | Title |
|---------|--------|----------|-------|
| [task_abc123](https://app.agentuity.com/services/task/task_abc123) | B2 | high | [sandbox-fuzz] Files: cp round-trip content mismatch |
| [task_def456](https://app.agentuity.com/services/task/task_def456) | F5 | medium | [sandbox-fuzz] Edge: rm non-existent file returns success |
### Enhancements Filed
| Task ID | Action | Title |
|---------|--------|-------|
| [task_ghi789](https://app.agentuity.com/services/task/task_ghi789) | F4 | [sandbox-fuzz] Enhancement: error message should include attempted path |
### Failures (detail)#### [Action ID]: [Description]-**Task ID**: [<task_id>](https://app.agentuity.com/services/task/<task_id>)
-**Session ID**: <sess_xxxifpresent, otherwiseomit>-**Command(s)**: The exact commands that were run
-**Expected**: What should have happened
-**Actual**: What actually happened
-**Exit Code**: Reported exit code
-**Error Output**: Any error messages
### Edge Cases Discovered
[List any surprising behaviors that didn't warrant a bug or enhancement but are worth noting]
### Summary
[2-3 sentence summary: what was tested, whether issues were found, how many bugs and enhancements were filed, and any recommendations]
Variation Guide
To maximize bug discovery across multiple runs, vary these aspects:
Region: Randomly selected each run — tests all three regions over time
Action selection: Never pick the same 10 actions twice
Parameter values: Use different filenames, directory paths, env var names/values
Ordering: Run file ops before exec sometimes, env before files other times
Timing: Sometimes pause between actions, sometimes run rapidly
Edge case focus: Some runs focus on file ops edge cases, others on execution edge cases
Resource options: Vary --memory, --cpu, --disk on sandbox creation between runs
Runtime selection: Use bun:1 for some runs, try python:3.14 or node:22 for others if available
