Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

release

Estrellas0

Forks0

Actualizado16 de junio de 2026 a las 15:02

Cut a release of the Eval Containers image fleet — pass the readiness gate, then build, tag, and push every benchmark, agent, model, gateway, and core image to the registry in bulk. Use when tagging a release or pushing the fleet to ghcr.io/exgentic. For building images during local dev (one artifact at a time, no push, no readiness gate), use the `build` skill instead; this skill is the full release flow that wraps it.

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

Exgentic

Exgentic/eval-containers

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Desarrolladores de softwareOcupaciones informáticas y matemáticas·SOC 15-1252

Explorador de archivos

2 archivos

SKILL.md

readonly

Más de este repositorio

mismo repositorio

audit-rules-drift

Exgentic/eval-containers

Walk every normative RULES principle against the actual repository state and report, per rule, whether the code still honours it — compliant, partial drift, or drifted — with file:line evidence and a proposed mechanical check for each gap. Use this for "audit RULES drift", "do the rules still match the code", before a release (step 29 of the `verify` walk), or after a RULES rename that the code may not have followed. This is the rule-text-vs-code consistency pass; for the whole-repo build/fixture/count audit use audit-fleet, and to apply a single topic's rules to one artifact use the meta `review` skill.

2026-06-210

build

Exgentic/eval-containers

Build the Eval Containers image fleet — or any single artifact or eval combination — with `docker buildx bake`, and write the per-artifact `docker-bake.hcl` files the build graph requires. Use when building an image locally, composing a benchmark × agent × model eval, or adding a bake file for a new artifact. For cutting and pushing a tagged release of the whole fleet (with the readiness gate), use the `release` skill, which wraps this one.

2026-06-210

add-benchmark

Exgentic/eval-containers

Use when adding a new benchmark image to the fleet — a Dockerized task plus verifier that evaluates an agent. Walks the directory layout, the five authored files, the two version knobs, required labels, and the three deployment surfaces. For packaging the AI system that runs *inside* a benchmark (install + entrypoint, LLM access), use add-agent instead.

2026-06-170

audit-fleet

Exgentic/eval-containers

Run the whole-repository health audit that a release manager reads before cutting a tag — run every mechanical gate, reconcile counts against the README, walk the ten fleet questions (does every benchmark build, does every released benchmark have a fixture, do the docs match disk, are agent versions pinned, is CI real), and classify a red / yellow / green verdict. Use this for "audit the fleet", "is the repo release-ready", a quarterly health check, or as step 25 of the `verify` release walk. This is the whole-repo cross-cutting pass; for a single Dockerfile use audit-dockerfile, for a single trajectory use audit-trajectory, and for RULES-text-vs-code drift use audit-rules-drift.

2026-06-160

audit-trajectory

Exgentic/eval-containers

Run a judgment-level health review of a recorded evaluation trajectory in two halves — was the task the agent saw well-formed (the prompt), and was the actual run sane (refusals, retries, wrong answers, API errors, cost). This is the procedural layer that catches content-filter refusals, factually wrong answers on verifiable benchmarks, and tasks delegated to an external file that the mechanical trajectory rule catalog cannot see. Use this for "audit the fixtures/trajectories", when the mechanical catalog flags a yellow, when a new benchmark batch lands, or as step 24 of the `verify` release walk. This is the per-fixture pass; for per-Dockerfile static health use audit-dockerfile, and for the whole-repo sweep use audit-fleet.

2026-06-160

verify

Exgentic/eval-containers

Walk the complete release verification of eval-containers end to end — preflight, sanity gates, build/replay/live sweeps, upstream + security scans, the three procedural audits, docs and CI checks, the fleet report, and the tag/publish/announce flow. Use this when cutting a release tag or asked to "run release verification" / "is this release-ready". This is the whole-pipeline driver; for the individual judgment-level passes it invokes, use the narrower audit skills next door (audit-dockerfile, audit-trajectory, audit-fleet, audit-rules-drift).

2026-06-160

name

release

description

Release the image fleet

Releasing means producing the whole fleet — every benchmark, agent, model, gateway, and core image — tagged, labeled, and pushed to ghcr.io/exgentic. The guiding principle: CI builds the fleet; humans build one thing at a time (see the build skill for the single-artifact loop). A release is the one time the entire fleet builds and ships together, so it MUST pass the full readiness gate first.

A release ships the CLI at the same version too: the vX.Y.Z tag also fires .github/workflows/release.yml (cargo-dist), which builds the eval-containers binaries and publishes the crate. This skill covers the fleet half; the unified outcomes — one tag, and which workflow owns what — are .agents/delivery/RULES.md. Because the tag is the version (.agents/RULES.md:9), bump Cargo.toml, benchmarks/_chart/Chart.yaml, and CHANGELOG.md to the release version before tagging, or the release aborts on the version-agreement gate (.agents/delivery/RULES.md:6). Curate CHANGELOG.md's [Unreleased] section here — drawn from the commit log, in the Keep a Changelog sections, for consumer-visible changes only; it is release-owner-curated, never edited per PR (.agents/delivery/RULES.md:8–10).

Serves: .agents/RULES.md:1 (the image is the product), .agents/RULES.md:2 (standalone artifacts), .agents/RULES.md:14 (verification is normative — no release ships red), and .agents/RULES.md:15 (the bake graph is the build artifact).

Steps

Confirm release verification has run, not just contribution verification. A release MUST pass every contribution gate plus the live fleet sweep, the procedural audits, and the upstream reachability check. These are two different processes triggered at different points — never conflate them (.agents/verification/RULES.md:1, .agents/verification/RULES.md:2). Why: contribution gates run offline on every PR; the release-only gates (live, upstream, audits) are what certify the fleet actually produces trajectories end-to-end against a real model.
Run the release-only gates out of band first. The fleet report is a pure aggregator — it probes each category's log, it does not re-run them (tests/run/fleet/RULES.md:1, tests/run/fleet/RULES.md:2). So run, in any order:
- cargo test --test build -- --ignored — the build sweep (tests/build/RULES.md). On macOS/podman set DOCKER_HOST to the podman machine socket (tests/build/RULES.md:9); CI uses the default.
- cargo test --test upstream -- --ignored — every pinned FROM, upstream_base label, and HF/GitHub URL still resolves (tests/run/upstream/RULES.md). Any 404 is red (tests/run/upstream/RULES.md:9).
- cargo test --test live -- --ignored — the live fleet sweep: every buildable benchmark, ≥3 tasks each, against the model of record (gpt-5.4) with the reference agent (claude-code) (tests/run/live/RULES.md:3, tests/run/live/RULES.md:4, tests/run/live/RULES.md:5). Respect EVAL_LIVE_BUDGET_USD (tests/run/live/RULES.md:7). Why first: the fleet report reads these logs; stale logs yield a stale verdict.
Render the fleet report and read the verdict. Run cargo test --test fleet -- --ignored to regenerate .agents/verification/fleet/report.md from scratch — never hand-edit it (tests/run/fleet/RULES.md:7). The verdict is red / yellow / green (tests/run/fleet/RULES.md:4). No release MAY ship with a red verdict (.agents/RULES.md:14).
Walk the go/no-go readiness checklist. Confirm every gate is green or justified-yellow, and that each outstanding yellow has a documented root cause and a reason it is not release-blocking. The full checklist — verdict classification, gate matrix, and outstanding-findings policy — is in references/readiness.md. Why: a green report is necessary but not sufficient; a yellow ships only when the gaps are enumerated and understood.
Promote any new live-sweep trajectories to fixtures. Each passing live run becomes a replay fixture so contribution verification can re-run it at zero cost (tests/run/live/RULES.md:2, tests/run/live/RULES.md:11). Rename output/<bench>/<task>/model/trajectory.jsonl (converted to OTLP) → .agents/verification/replay/fixtures/<bench>-<task>-claude-code.traces.jsonl and add a provenance.json entry recording model, agent version, benchmark data_revision, timestamp, and the release tag (tests/run/replay/RULES.md:6). A run that fails an inspection rule is NOT promoted — record it in .agents/verification/live/known-broken.md instead (tests/run/live/RULES.md:12). Why: fixtures are the immutable ground truth that makes the next cycle's offline replay meaningful (tests/run/replay/RULES.md:4).
Build and push the fleet via Docker Bake. The build graph is an artifact in the tree: every core/, agents/, benchmarks/, models/, gateways/ directory ships a docker-bake.hcl next to its Dockerfile (.agents/RULES.md:15). Release builds go through docker buildx bake — the same canonical invocation tests and the CLI use. Tag the fleet via the fleet-wide TAG variable (.agents/RULES.md:15, sub-rule b) and push:
```
# Build + push the whole fleet at the release tag (the actual release step)
TAG=v1.2.0 docker buildx bake --push

# Override the registry for a staging push
REGISTRY=ghcr.io/eval-containers TAG=v1.2.0 docker buildx bake --push
```
REGISTRY (default ghcr.io/exgentic) and TAG (default latest) are declared once at the repo root ./docker-bake.hcl and picked up by auto-discovery (.agents/RULES.md:15, sub-rules b–c). The build skill covers composing and building individual targets, groups, and eval combinations; this step is the fleet-wide push. Why bake: it is Docker's native declarative multi-image build, and keeping every consumer (CI, CLI, tests, OC translators) on one invocation is what makes the build graph reusable (.agents/RULES.md:15, sub-rule d).
Prefer letting CI build the fleet. .github/workflows/release-images.yml runs bake on every push to main (tag: latest) and every v* tag (tag: the git tag), setting GIT_SHA and BUILD_DATE, then bake --pushes the result. CI runs on real Docker on Linux, where the full sweep is clean; local podman-on-macOS chokes the parallel fleet build on network contention (a documented, non-structural caveat — see references/readiness.md and .agents/verification/build/known-broken.md). Why: humans building 100+ images locally is slow and flaky; CI is the authoritative fleet builder.
Commit the fleet report alongside the release tag. When cutting the tag, commit the final .agents/verification/fleet/report.md so the release artifact carries its own verification record (tests/run/fleet/RULES.md:8). Why: a release that ships without its certifying report cannot be audited after the fact.

Releasing the CLI alongside the fleet

The CLI half is automatic on the tag — no bake, no fleet gate. Once Cargo.toml, benchmarks/_chart/Chart.yaml, and CHANGELOG.md carry the release version and the vX.Y.Z tag is pushed, .github/workflows/release.yml (cargo-dist) builds the cross-platform eval-containers binaries, generates the installers, publishes the crate to crates.io, and creates the tag's GitHub Release (.agents/delivery/RULES.md:2, .agents/delivery/RULES.md:4). The fleet workflow (release-images.yml) publishes only to the registry and MUST NOT touch that GitHub Release, so the two halves compose on one tag instead of clobbering each other (.agents/delivery/RULES.md:3–4). crates.io versions are immutable — never reuse a number, so the first unified tag after a manual crate publish starts one patch above it (.agents/delivery/RULES.md:7).

What this skill does NOT cover

Building one artifact for local dev — that's the build skill.
Running tests or agents — bake only builds. Use cargo test --test replay / --test live, or docker compose up / eval-containers run.
Verifying labels post-build — that's the build sweep (tests/build/RULES.md).

References

Delivery rules — the version/release outcomes (one tag, workflow ownership, guards) this procedure satisfies.
references/readiness.md — the go/no-go gate.
Docker Bake docs
The build skill (.agents/build/SKILL.md) — single-artifact and eval-combination builds.