Run any Skill in Manus with one click

ferrogate-test-strategy

Stars20

Forks4

UpdatedJuly 13, 2026 at 05:46

Use when working in the FerroGate repo and deciding how to test a change — which test layer a change needs, adding coverage for a feature or bugfix, whether the harness is sufficient, or when a test run surfaces a bug or new feature that needs tracking. For running the harness CLI itself, use the ferrogate-test skill.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

lianluo-esign

lianluo-esign/ferrogate

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software Quality Assurance Analysts and TestersComputer and Mathematical Occupations·SOC 15-1253

SKILL.md

readonly

name	ferrogate-test-strategy
description	Use when working in the FerroGate repo and deciding how to test a change — which test layer a change needs, adding coverage for a feature or bugfix, whether the harness is sufficient, or when a test run surfaces a bug or new feature that needs tracking. For running the harness CLI itself, use the ferrogate-test skill.

FerroGate Test Strategy

The guiding direction for FerroGate's layered test system: which layer a change needs, how findings feed the issue queue, and how the tooling must grow to keep up. The binding source of truth is AGENTS.md → ## Testing Architecture and docs/testing/testing-architecture.md; this skill is the on-the-spot pointer, kept thin so it does not drift from those. To operate the E2E harness CLI, use the ferrogate-test skill.

Pick the layer for the change

"Unit tests pass" never proves a cross-cutting or runtime-wiring change.

Pure logic → Static gate + Unit (+ Property if it is a state machine/invariant).
Unit test implementations live in dedicated sibling *_test.rs files (or crates/*/tests/*.rs), never inline in business-logic files. A production module may keep only the minimal cfg(test) path declaration needed for private-item access.
Wiring inside a crate → add Crate integration (crates/*/tests/*.rs).
Provider / guardrail / policy scope / quota surface → add Contract/compliance: prove what it writes is what the runtime reads, and that it emits the telemetry/audit evidence it claims. A 200 with the runtime ignoring the value is a failure, not a pass (the #188 asset-quota-scope failure mode).
Producer→consumer handoff (billing, evidence, exports) → add Cross-component chain (ferrogate-test gateway-billing-chain / guardrail-supabase).
Persistence / schema / migration → add Durability (ferrogate-test *-restart).
Any feature → the E2E harness (ferrogate-test ci) must close the loop.
Hot-path performance → run the Performance layer, record the number; never a silent PR gate.

Property tests (proptest) belong on state machines/invariants (routing fallback, quota reserve/settle/rollback, streaming stage transitions, ack/settlement ordering), not ordinary branch logic. Streaming and concurrency are correctness surfaces — add a focused async test at the lowest layer that can reproduce the race instead of leaving it to a slow E2E run.

Feed findings back to the issue queue

A bug or missing capability a test surfaces, and that is not fixed in the same change, becomes a house-style GitHub issue before the change lands — never a silent #[ignore], skipped scenario, or inline TODO.

Real bug, not fixed now → file a bug issue and add a failing regression test at the lowest layer that reproduces it; the fix makes it pass.
Flaky test → confirm against main, open or link a tracking issue, record it in the affected suite. No blind retries that hide a real race.
New feature/capability idea → file it and prioritize it in the queue; do not scope-creep it into the current change.
House style: [area][priority] title; Problem → Scope → Acceptance (checkboxes) → Non-goals body; reuse existing labels; link the issue from the failing suite and from the commit that acts on it.

The harness must keep up with the layers

The methodology is fixed; the tooling is not yet complete. ferrogate-test does not yet support every layer above — treat it as a living harness that must be iterated to satisfy the test system, not a fixed set of scenarios the methodology bends to.

If a change needs a layer the harness cannot yet express, grow the harness (new subcommand, shared assertion, fixture) and wire it into ferrogate-test ci — do not skip the layer or downgrade the claim to fit the current tool.
A missing tool never silently shrinks coverage. A layer you cannot yet automate is still owed a manual proof at the affected surface plus a filed tooling issue.
Quota, canonical provider settlement, and Guardrail allow/block evidence use the reusable ComponentContract executor; quota and Guardrail also have live-Supabase proof. Runtime and the harness share one canonical provider adapter-family registry, and exact matrix equality fails closed before E2E. #214 completed that matrix and #213 completed distinct, idempotent settlement identities for multiple billable attempts in one logical request.
Live Supabase proof is isolated per scenario: one unique schema is reused for that scenario's restarts and removed through exact-name RAII cleanup. Use --keep-supabase-schema only when retained state is deliberately required for debugging.

Run the narrowest proof

Local module gate (mirrors CI): scripts/local-test-modules.sh <module> (quality, core-policy, control-plane, agentic-gateway, ai-proxy, cli-tooling, gateway-runtime, e2e-harness, supabase-storage).
Workspace units: cargo +1.88.0 test --workspace --all-features.
Static: cargo fmt --all -- --check; cargo clippy --workspace --all-targets --all-features -- -D warnings; python3 scripts/check-openapi.py.
E2E: the ferrogate-test harness (see the ferrogate-test skill).

Read the output. Do not claim production readiness from unit tests alone when the change touches runtime wiring, live reload, TLS/ACME, provider streaming, or billing settlement.

GitHub Actions are release-only (release: published) and never a per-commit fallback. Missing local infrastructure means the proof remains explicitly unverified; it does not justify adding a push/PR/manual cloud trigger.

More from this repository

same repository

ferrogate-code-review

lianluo-esign/ferrogate

Use when running the FerroGate code-review role of the three-agent board loop — the session that watches the GitHub Project "In review" lane, judges what the dev agent handed off, and either advances the item to "Testing" or bounces it back to "Ready" with findings (e.g. "run the review agent", "work the In review lane", "review what the dev loop landed"). Covers the fixed lane/edges, the board handles, the review methodology (acceptance-box audit first, static verification over rebuild, report-never-edit, `review-rejected` on bounce), and the shared discipline (GraphQL quota rationing, cached lane tooling, worktree isolation, never build in the main worktree). Neighbouring roles: ferrogate-dev-loop (upstream) and ferrogate-test (downstream); all three at once: ferrogate-multi-agent-loop.

2026-07-2520

ferrogate-test

lianluo-esign/ferrogate

Use when working in the FerroGate repo and the task involves the ferrogate-test Rust CLI itself — running Admin API or gateway API E2E coverage, CI harness verification, Docker-backed cluster scenarios, editing tools/ferrogate-test, or release-grade end-to-end validation. For deciding which test layer a change needs and how findings become issues, use the ferrogate-test-strategy skill.

2026-07-2520

ferrogate-multi-agent-loop

lianluo-esign/ferrogate

Use when running FerroGate development as THREE cooperating autonomous agents — a code-generation-only dev driver that fans slices out across worktree-isolated subagents, a code-review agent, and a test gate — coordinating through the GitHub Project board (e.g. "run the dev loop, the review and the test gate", "keep the board moving with all three agents", setting up the dev/review/test split, or checking what a neighbouring role will do with your output). Covers the choreography, the board handoff (Ready → In progress → In review → Testing → Done, with every failure bouncing back to Ready), the seven Status option IDs, and the shared GraphQL quota discipline. For one role only, use ferrogate-dev-loop (dev), ferrogate-code-review (review), or ferrogate-test / ferrogate-test-strategy (gate).

2026-07-2520

ferrogate-dev-loop

lianluo-esign/ferrogate

Use when running FerroGate development as an autonomous, continuously-iterating code-generation loop that fans work out across parallel subagents against the GitHub Project board (e.g. "keep iterating on the board", "持续迭代开发", a /loop that advances issues). This is the DEV role of the three-agent loop — code generation only. Covers the binding constraints — advance sub-issues only up to "In review" (never Testing, never Done), Ready is also the rework inbox for bounced issues, Projects GraphQL quota discipline, the max-3 parallel-subagent ceiling, worktree isolation + slice separation, the cherry-pick integration flow, and mandatory worktree cleanup to bound disk. For the neighbouring roles see ferrogate-code-review and ferrogate-test; for all three at once see ferrogate-multi-agent-loop. Not for a single named-issue slice (use docs/dynamic-workflow.md).

2026-07-2520

ferrogate-release

lianluo-esign/ferrogate

Use when cutting a FerroGate release — building the container image and publishing it to GHCR. Covers the standing directive to NOT use GitHub Actions (too slow) and instead gate on the local ferrogate-test suite then push locally, including the no-sudo/no-Docker host path (musl-static + crane). Use for "release", "publish image", "push to GHCR", "cut a version".

2026-07-1920