| name | agilab-evidence-contracts |
| description | Maintain AGILAB evidence, proof, replay, and verification contracts. Use when code, docs, tests, or workflows touch run manifests, artifact hashes, first-proof or release-proof evidence, proof capsules, notebook exports, agent-run traces, MLflow handoff, replay commands, or claims about reproducibility and attestation. |
| license | BSD-3-Clause (see repo LICENSE) |
| metadata | {"updated":"2026-06-01T00:00:00.000Z"} |
AGILAB Evidence Contracts
Use this skill when a change affects AGILAB's evidence-first value: proving that
an experiment, app run, notebook migration, agent action, or release can be
reviewed and replayed. The default stance is simple: a feature is not validated
unless it leaves evidence with a schema, path contract, artifact hashes when
possible, and a verification or replay command.
Evidence Surfaces
Classify the touched surface before editing:
- Run evidence:
run_manifest.json, app/stage metadata, produced artifacts,
timing, environment context, status, and error summaries.
- Proof evidence:
agilab first-proof, release proof, proof capsules,
signatures, replay/verify commands, and adoption reports.
- Notebook handoff: imported notebooks, exported notebooks, stage manifests,
lab_stages.toml, generated analysis views, and exit-path artifacts.
- Agent evidence:
agilab.agent_run.v1 manifests, event streams,
stdout/stderr artifacts, lineage, handoff, and redaction behavior.
- Supply-chain evidence: SBOM,
pip-audit, PyPI provenance, release assets,
dataset release manifests, version alignment, and trusted-publisher outputs.
- UI robot evidence: screenshots, robot JSON, route coverage, widget actions,
and public demo proof artifacts.
- Workflow dry-run evidence: static validation reports for
lab_stages.toml,
notebook-import previews, capability manifests, and other contract checks
that prove readiness without executing user code.
Contract Checklist
For every new or changed evidence output, verify:
- It has a stable
schema or explicit version field.
- It records producer, command, timestamp or run id, version, and target context.
- It records artifact paths relative to a declared root, not machine-specific
private absolute paths unless explicitly marked local-only.
- It includes file size and
sha256 for persisted artifacts when feasible.
- It distinguishes missing, skipped, failed, and passed states.
- It redacts secrets and stores command/env values safely.
- It has a documented verification, replay, or inspection command.
- It has at least one regression test that catches schema or path drift.
- Public docs describe what the evidence proves and what it does not prove.
For GitHub dataset release evidence, also verify that the manifest records every
tracked dataset path, size, sha256, LFS-tracking status, and whether the file
is packaged into PyPI artifacts. The release archive must contain materialized
dataset contents, not Git LFS pointer files, and it must remain separate from
the normal source/code GitHub Release assets.
Anti-Patterns
- Claiming "validated", "certified", "reproducible", or "release-ready" without
a linked command or artifact.
- Writing evidence under a hidden or unstable location without documenting the
path contract.
- Storing only screenshots or prose when a machine-readable manifest is needed.
- Hashing transient paths but not the artifacts users actually inspect.
- Treating UI success as run success without backend evidence.
- Mixing local developer state into public release proof.
- Updating docs, badges, or scorecards without checking the current evidence.
External Inspiration Gate
When a user points to an external product, framework, repository, or docs URL as
inspiration, do not convert the idea directly into roadmap prose. First extract
the transferable pattern, reject parts that do not fit AGILAB's evidence-first
positioning, then implement the smallest AGILAB-native evidence primitive when
the idea is worth adopting.
A valid adopted pattern should normally include:
- a stable schema such as
agilab.<feature>.v1;
- a producer command, report, manifest, validator, or dry-run checker;
- a reader or UI inspection surface when humans need to review the evidence;
- tests that fail on schema, path, or stale-output drift;
- public docs stating what the evidence proves and what it does not prove.
Examples: a declarative workflow idea should become a lab_stages.toml
validator or graph contract, not a new unbounded DSL; an agent-platform catalog
idea should become a machine-readable AGILAB capability index, not marketing
copy.
Review Workflow
- Inspect the current evidence producer before changing docs or claims.
- Identify the consumer: human review, replay command, release gate, robot,
notebook export, MLflow handoff, or downstream app.
- Keep the manifest additive when possible; if removing or renaming fields,
update readers, tests, docs, and migration notes in the same change.
- Prefer a small verifier over manual inspection for critical evidence.
- For docs, state the exact command and path that produce the evidence.
- For release work, compare
CHANGELOG.md, release proof, workflow output,
PyPI/GitHub code release state, GitHub dataset release state, docs, and
Hugging Face sync before claiming done.
- For external inspiration, record the AGILAB-native evidence primitive that
was shipped, or state explicitly why no implementation was adopted.
Validation Commands
Start with impact triage:
uv --preview-features extra-build-dependencies run python tools/impact_validate.py --staged
Use the narrow evidence checks that match the change:
uv --preview-features extra-build-dependencies run pytest -q -o addopts='' test/test_evidence_contract.py
uv --preview-features extra-build-dependencies run pytest -q -o addopts='' test/test_agent_trace.py test/test_agent_run.py
uv --preview-features extra-build-dependencies run python tools/release_proof_check.py
uv --preview-features extra-build-dependencies run python tools/workflow_parity.py --profile skills
When touching proof capsules or release publication, inspect the current release
plan instead of relying on memory:
./dev --print-only release
uv --preview-features extra-build-dependencies run python tools/pypi_project_preflight.py
Report residual risk when evidence depends on external services, hosted demos,
clusters, GPU hardware, cloud credentials, or a workflow that was not rerun.
