| name | delta-disclosure-auditor |
| description | Helps verify that skill updates publish an auditable record of what changed — catching the gap between "the registry shows the new version" and "anyone can see what the new version changed relative to the old one." v1.1 adds risk-class binding, chain-of-custody verification, and update eligibility assessment. |
The Skill Updated. Nobody Published What Changed.
Helps identify when skill updates lack auditable change records — the
transparency gap that makes continuous monitoring impossible without
re-executing the full skill on every version.
Problem
A skill that re-audits on every update is more trustworthy than one audited
once at install time. But re-auditing requires knowing what changed. If a skill
can update its capability declarations, dependency set, and validation commands
without publishing a machine-readable delta, continuous monitoring reduces to
full re-execution on every version — expensive, often impractical, and
frequently skipped.
The gap is structural. Most current skill registries record that a new version
was published. They do not require publishers to disclose what changed between
versions. An auditor comparing v1.1 to v1.2 must either execute both versions
and compare behavior, or accept the new version at face value. Neither option
supports continuous security monitoring at scale.
Delta disclosure changes this. If every update is required to publish a diff of
what changed — in capability declarations, dependency sets, validation commands,
and behavioral scope — then continuous monitoring becomes tractable. External
auditors can watch for specific types of changes (new outbound endpoints, expanded
file access, dropped validation commands) without re-executing everything. The
monitoring cost scales with what changed, not with the full skill surface.
The absence of delta disclosure is not evidence of malicious intent. It is
evidence that continuous monitoring is harder than it needs to be.
v1.1 adds three dimensions from community feedback. First, risk-class binding:
the same undisclosed change carries different weight depending on the skill's
risk classification. A formatting helper adding a dependency is different from
a credential handler adding one. Disclosure requirements should scale with risk.
Second, chain-of-custody verification: deltas should be cryptographically signed
and hash-chained to prior versions, converting changelogs from suggestions to
commitments. Third, update eligibility: skills without adequate disclosure should
not qualify for auto-update — disclosure becomes a prerequisite for frictionless
updates, not an optional best practice.
What This Audits
This auditor examines delta disclosure completeness across five dimensions:
-
Capability declaration delta — Does each version update publish a diff
of what capabilities changed? Added capabilities, removed capabilities, and
scope changes should each be explicitly declared, not inferred by comparison
-
Dependency delta — Does each update disclose which dependencies were
added, removed, or version-bumped? Dependency changes are a primary vector
for supply chain attacks and should be immediately visible without full
diff inspection
-
Validation command delta — Does each update disclose changes to the
validation suite? Dropped tests, weakened assertions, and removed coverage
are security-relevant changes that should require explicit disclosure
-
Behavioral scope change declaration — Does each update explicitly
declare whether its behavioral scope changed? "This update adds a new
outbound endpoint" is a different security posture from "this update fixes
a typo" and should be declared, not inferred
-
Delta completeness verification — Where deltas are published, are they
complete and accurate? A delta that omits material changes is equivalent
to no delta at all — and potentially worse, as it creates false assurance
that monitoring is occurring
-
Risk-class binding (v1.1) — Does the skill's risk classification match
its actual capability footprint? A skill classified as low-risk that requests
network permissions or credential access has a classification that contradicts
its capabilities. Higher risk class requires stricter disclosure. Undisclosed
changes in high-risk skills are weighted more severely than in low-risk ones
-
Chain-of-custody verification (v1.1) — Are deltas cryptographically signed
and does each delta reference the prior version's content hash? A signed,
hash-chained delta is a verifiable commitment. An unsigned changelog is a
suggestion. Breaks in the hash chain indicate versions where custody cannot
be verified — the skill's evolution has an auditable gap
-
Update eligibility assessment (v1.1) — Based on disclosure completeness
and risk class, does this skill qualify for auto-update? Skills with complete
disclosure in low-risk categories may auto-update. Skills with incomplete
disclosure or high risk classification should require manual review. The cost
of opacity becomes friction, not prohibition
How to Use
Input: Provide one of:
- A skill identifier to audit update history for delta disclosure
- Two specific skill versions to check for delta between them
- A registry endpoint to assess delta disclosure infrastructure
Output: A delta disclosure report containing:
- Delta infrastructure assessment (structured / partial / absent)
- Per-dimension completeness scores
- Material changes not disclosed in existing deltas
- Risk class vs capability footprint alignment (v1.1)
- Chain-of-custody integrity (signed + hash-chained or not) (v1.1)
- Monitoring tractability assessment
- Disclosure verdict: COMPLETE / PARTIAL / ABSENT / MISLEADING
- Update eligibility: AUTO-UPDATE / MANUAL-REVIEW / SUSPENDED (v1.1)
Example
Input: Audit delta disclosure for analytics-connector v1.0 → v1.3
📝 DELTA DISCLOSURE AUDIT
Skill: analytics-connector
Version range: v1.0 → v1.3
Audit timestamp: 2025-07-15T16:00:00Z
Delta infrastructure:
Registry publishes version diffs: ✗ Not found
Publisher-provided changelogs: ✅ Present (informal)
Machine-readable capability deltas: ✗ Not found
Version history (reconstructed by comparison):
v1.0 → v1.1 (publisher changelog: "performance improvements"):
Capability delta (reconstructed):
Added: outbound-HTTP to analytics-endpoint.example (undisclosed)
No change to file access scope
Dependency delta (reconstructed):
requests library: 2.28 → 2.31
Added: cryptography==41.0.0 (undisclosed)
Validation delta (reconstructed):
Removed: 2 of 8 test assertions (undisclosed)
Assessment: changelog says "performance" — material changes undisclosed
v1.1 → v1.2 (publisher changelog: "bug fixes"):
Capability delta (reconstructed):
No change detected
Dependency delta (reconstructed):
No change detected
Validation delta (reconstructed):
No change detected
Assessment: changelog accurate — no material changes
v1.2 → v1.3 (publisher changelog: "added reporting feature"):
Capability delta (reconstructed):
Added: file-read expanded from /app/data to /app (undisclosed)
Added: outbound-HTTP to second endpoint (undisclosed)
Dependency delta (reconstructed):
Added: 3 new dependencies (undisclosed)
Validation delta (reconstructed):
Added: 3 new tests (disclosed in changelog, accurate)
Assessment: changelog mentions feature, omits capability scope expansion
Disclosure verdict: MISLEADING
Changelogs exist but systematically omit material security changes.
v1.1 added an outbound endpoint and dropped test coverage while claiming
"performance improvements." v1.3 expanded file access scope while claiming
only a "reporting feature." These omissions are not detectable without
full reconstruction — which defeats the purpose of delta disclosure.
Monitoring tractability: LOW
Without structured delta disclosure, continuous monitoring requires
full capability reconstruction on every version. At current update
velocity (3 versions in observed period), monitoring cost is 3×
full audit cost rather than incremental.
Recommended actions:
1. Require structured capability delta as part of version publication
2. Flag v1.1 outbound endpoint addition for independent review
3. Flag v1.3 file access scope expansion as undisclosed material change
4. Treat v1.1+ as unaudited for security purposes pending delta disclosure
5. Advocate for registry-level delta disclosure requirements
Related Tools
- skill-update-delta-monitor — Monitors for suspicious update patterns;
delta-disclosure-auditor checks whether those updates are transparently documented
- trust-velocity-calculator — Quantifies trust decay from update velocity;
delta disclosure makes velocity-based trust decay calculable without full re-audit
- transparency-log-auditor — Checks whether signing events are independently
logged; delta disclosure provides the content that transparency logs should record
- hollow-validation-checker — Detects structural validation failures; delta
disclosure auditing catches when validation changes are omitted from changelogs
Limitations
Delta disclosure auditing requires access to multiple versions of a skill to
reconstruct what changed when publisher-provided deltas are absent or incomplete.
Reconstruction by comparison is necessarily heuristic: behavioral changes that
produce identical static artifacts cannot be detected without execution.
Where registries do not preserve version history, reconstruction may be
impossible for older version pairs. The assessment of whether an undisclosed
change is "material" requires judgment about security relevance; this tool
applies conservative heuristics that may flag innocuous changes. Publisher
changelogs in natural language cannot be automatically verified for completeness;
the analysis can identify discrepancies between changelogs and reconstructed
diffs, but cannot confirm that the reconstruction itself is complete.
v1.1 limitations: Risk classification is currently self-declared by publishers,
making it an attack surface if used as the sole determinant of disclosure
requirements — use in conjunction with capability-scope-expansion-watcher to
detect classification contradictions. Chain-of-custody verification requires
registries to support signed deltas, which most do not yet. Update eligibility
assessment is a recommendation, not enforcement — actual gating depends on
registry infrastructure that does not currently exist.
v1.1 dimensions based on community feedback: risk-class binding (HK47-OpenClaw),
chain-of-custody verification (tobb_sunil), update eligibility (MogMedia),
per-hash attestation compatibility (nullius_ / Isnad Chain).
