ecosystem-health

name	ecosystem-health
description	Analyzes Claude Code ecosystem health by tracking all 27 extensibility components across 6 tiers - including plugin components, core configuration, environment/CLI, authentication, session features, and integrations. Use when checking if Claude Code components are up-to-date, orchestrating audits efficiently, tracking documentation coverage, applying updates from new Claude Code versions, or getting an overview of ecosystem component staleness.
user-invocable	false
allowed-tools	Read, Write, Glob, Grep, Skill

Ecosystem Health

MANDATORY: docs-management Delegation

CRITICAL: This skill follows the anti-duplication principle. ALL component details MUST be queried from docs-management at runtime.

What This Skill Hardcodes (Static - Changes Rarely)

Data	Why Static
Tier structure (1-6)	Design decision, architectural choice
Audit commands (/audit-*)	OUR commands in claude-ecosystem plugin
Audit types (automated/manual/documentation)	Classification policy
Scoring/prioritization logic	Policy decisions

What MUST Be Delegated (Dynamic - Changes With Releases)

Data	Why Dynamic	How to Get
CLI flags list	New flags every release	Query: docs-management: cli-reference.md CLI flags
Environment variables	New env vars frequently	Query: docs-management: settings.md environment variables
Authentication methods	New providers added	Query: docs-management: iam.md authentication methods
Permission modes	Modes evolve	Query: docs-management: iam.md permission modes
Cloud providers	New providers added	Query: docs-management: setup.md cloud providers
IDE integrations	New integrations added	Query: docs-management: third-party-integrations.md IDE
Change keywords	Terminology evolves	Derive from docs-management index keywords
File patterns	Locations can change	Query official docs for current patterns

Delegation Rules

1. NEVER use hardcoded lists for CLI flags, env vars, auth methods, cloud providers, IDE integrations, permission modes, or any frequently-changing data
2. ALWAYS query docs-management when you need component details
3. Use claude-code-guide agent for live verification during --discover and --check modes
4. If docs-management returns empty, that's a signal to check if the component still exists

Query Patterns for docs-management

# Tier 3: Environment & CLI
"cli-reference.md CLI flags"              → Get current CLI flags list
"settings.md environment variables"       → Get current env vars list
"iam.md permission modes"                 → Get current permission modes

# Tier 4: Authentication & Access
"iam.md authentication methods"           → Get current auth methods
"iam.md configuring permissions"          → Get permission rule patterns
"iam.md credential management"            → Get credential features

# Tier 5: Session & Runtime
"cli-reference.md session features"       → Get session features (resume, checkpoints)
"security.md sandbox configuration"       → Get sandbox settings

# Tier 6: Integration
"third-party-integrations.md IDE"         → Get IDE integrations list
"setup.md cloud providers"                → Get cloud provider list
"common-workflows.md CI/CD"               → Get CI/CD platforms

# Changelog for change categorization
"CHANGELOG recent changes"                → Get changelog entries

---

Overview

This skill tracks Claude Code ecosystem health across ALL extensibility points - not just plugin components. It monitors 27 component types across 6 tiers.

Schema v2.2 introduces the Tiered Validation Model for changelog-triggered audit staleness:

• Minor changes (features, deprecations) → Targeted keyword validation (cheap)
• Major changes (behavior_change, security) → Full audit required (no shortcuts)
• Bugfixes → No validation needed

This approach provides 60-96% token savings while maintaining strict accuracy requirements.

Component Tiers

Tier	Category	Components	Audit Type	Description
1	Core Configuration	4	Mixed	User, project, and enterprise settings
2	Plugin Components	12	Automated	Components packaged in plugins
3	Environment & CLI	3	Documentation	Env vars, CLI flags, permission modes
4	Authentication & Access	3	Mixed	Auth methods, permission rules
5	Session & Runtime	2	Mixed	Session features, sandbox config
6	Integration	3	Documentation	IDEs, cloud providers, CI/CD

Audit Types Explained

Type	Description	Has Audit Command?	Tracking Method
automated	Full audit via /audit-*	Yes	Pass rate, component count
manual	Requires human review	No	Human review tracking
documentation	Tracks doc coverage only	No	Doc coverage via docs-management queries

---

Tiered Validation Model (v2.2)

Schema v2.2 introduces a three-tier validation model that automatically invalidates audits when changelog changes affect components, while using the most token-efficient validation method appropriate for each change type.

Change Severity Classification

Change Type	Severity	Validation Requirement	Rationale
feature	Minor	Targeted validation (keyword check)	New features can be verified by checking keywords exist
deprecation	Minor	Targeted validation (keyword check)	Deprecations can be verified by checking warnings documented
bugfix	None	No validation needed	Bugfixes don't affect plugin documentation/compliance
behavior_change	Major	Full audit required	Behavior changes may have wide-ranging impacts
security	Major	Full audit required	Security changes require comprehensive review

Validation Tiers

Tier 1: Targeted Validation (Minor Changes)

For feature and deprecation changes:

• Method: Grep-based keyword verification
• Evidence: File path, line number, matched text
• Cost: ~100-500 tokens per change
• Status on pass: VALIDATED

Algorithm:

1. Load validation spec from granular_changelog change entry
2. For each keyword in validation.keywords:
   a. Run grep against target_skill directory
   b. If match found, record evidence (file, line, match text)
3. If matches >= required_matches:
   a. Set validation_result.validated = true
   b. Set confidence based on match count:
      - 1 match = "medium"
      - 2+ matches = "high"
4. Update component_coverage validation tracking

Tier 2: Full Audit (Major Changes)

For behavior_change and security changes:

• Method: Spawns auditor agent(s)
• Audit command: Specified in validation.audit_command
• Cost: ~3,000-8,000 tokens
• Status on pass: AUDITED

Important: Major changes cannot be validated via targeted validation. The system must enforce this - no shortcuts.

Tier 3: Periodic Review

Regardless of validation status:

• Threshold: >90 days since last full audit
• Action: Triggers full audit
• Rationale: Ensures nothing drifts over time

Validation Spec Schema (v2.2)

Each change in granular_changelog can have a validation spec:

validation:
  method: "keyword_check"           # keyword_check | full_audit | manual | none
  target_skill: "hook-management"   # Which skill to validate (null for memory files)
  keywords:                         # Manual list - NEVER auto-extracted
    - "additionalContext"
    - "additional context"
    - "PreToolUse.*additionalContext"  # Regex supported
  required_matches: 1               # How many keywords must match
  reason: "..."                     # Optional explanation (esp. for method: none)

# For full_audit method:
validation:
  method: "full_audit"
  target_skill: "permission-management"
  audit_command: "/audit-settings"
  reason: "Security fix - targeted validation insufficient"

Validation Result Schema

After validation runs, results are recorded:

validation_result:
  validated: true                   # Did validation pass?
  validated_date: "2026-01-16"      # When validated
  evidence:                         # For keyword_check method
    - file: "path/to/file.md"
      line: 142
      match: "matched text"
  audit_performed: true             # For full_audit method
  audit_date: "2026-01-12"          # When audit was run
  confidence: "high"                # high | medium | low

Status Calculation (Conservative)

Component status is determined by this priority order (first match wins):

Priority	Condition	Status
1	Has pending_major_changes	NEEDS AUDIT ⚠️
2	Has pending_minor_changes	NEEDS VALIDATION
3	days_since(last_audit) > 90	STALE
4	validation_version == latest AND validation_confidence == "high"	VALIDATED
5	last_audit recent AND no pending changes	OK
6	Never audited	UNKNOWN

Conservative Rule: If ANY doubt exists, escalate to higher requirement.

---

Validation Accuracy Rules (CRITICAL)

The system must NEVER report "validated" unless certain. These rules are non-negotiable.

Rule 1: Keyword Matches Must Be Contextually Correct

❌ Finding "context" when looking for "context: fork" is NOT a match
✅ Must match exact keyword or regex pattern

Rule 2: Multiple Evidence Preferred

Evidence Count	Confidence Level
0 matches	validation_failed: true
1 match	confidence: "medium"
2+ matches	confidence: "high"

Rule 3: Human Confirmation for Edge Cases

If confidence is "low":

• Flag for manual review
• Do NOT auto-mark as validated
• Include in --apply output for user decision

Rule 4: Evidence is Mandatory

• No validation without captured evidence
• Evidence must include file path and line number
• Empty evidence = validation failed

Rule 5: Major Changes Cannot Use Targeted Validation

❌ security change → keyword_check (FORBIDDEN)
❌ behavior_change → keyword_check (FORBIDDEN)
✅ security change → full_audit (REQUIRED)
✅ behavior_change → full_audit (REQUIRED)

The system must reject attempts to use targeted validation for major changes.

The skill provides:

1. Parsing changelogs to identify new features and changes
2. Tracking audit coverage across all 27 component types
3. Documentation coverage for non-auditable components (queried from docs-management)
4. Identifying pending updates needed for compliance
5. Orchestrating audits efficiently (avoiding token waste)
6. Helping apply updates from new Claude Code versions

When to Use This Skill

Use this skill when:

• Checking if ANY Claude Code extensibility component is up-to-date
• Getting an overview of audit coverage and staleness across all tiers
• Tracking documentation coverage for non-auditable components
• Planning which audits to run (token-efficient approach)
• Applying updates after Claude Code releases new versions
• Preparing a plugin release
• Detecting new/deprecated Claude Code features

Tracking File

Location: .claude/ecosystem-health.yaml

This file persists ecosystem health state across sessions. It stores audit metadata ONLY - not component details (which are delegated to docs-management).

Schema v2.1 Structure:

schema_version: "2.1"

last_check:
  date: "YYYY-MM-DD"
  claude_code_version: "X.Y.Z"
  changelog_hash: "sha256:..."

component_coverage:
  # Tier 1: Core Configuration
  tier1_configuration:
    user_settings:
      last_audit: null
      components_audited: 0
      pass_rate: null
      audit_type: "manual"
      # Query docs-management for: settings.md user configuration
    project_settings:
      last_audit: "YYYY-MM-DD"
      components_audited: N
      pass_rate: 0.XX
      audit_type: "automated"
      audit_command: "/audit-settings"
    managed_settings: { ... }
    memory_system: { ... }

  # Tier 2: Plugin Components (12 components)
  tier2_plugins:
    skills: { audit_type: "automated", audit_command: "/audit-skills" }
    agents: { audit_type: "automated", audit_command: "/audit-agents" }
    # ... (12 components, each with audit_type and audit_command)

  # Tier 3-6: Documentation-tracked (DELEGATE to docs-management)
  tier3_environment:
    environment_variables:
      audit_type: "documentation"
      # DELEGATE: Query docs-management for: settings.md environment variables
    cli_flags: { ... }
    permission_modes: { ... }

  tier4_authentication: { ... }
  tier5_session: { ... }
  tier6_integration: { ... }

changelog_versions_checked:
  - version: "X.Y.Z"
    checked_date: "YYYY-MM-DD"
    changes_applied: true/false

pending_updates:
  - feature: "feature name"
    since_version: "X.Y.Z"
    affects: ["skills", "commands"]
    status: "pending" | "applied" | "skipped"

last_discovery:
  date: "YYYY-MM-DD"
  docs_scanned: [...]
  changelog_version: "X.Y.Z"
  components_detected: 27
  tiers_scanned: 6
  gaps_found: 0

Key Design Decision: The tracking file does NOT contain tracked_* arrays (no hardcoded lists of env vars, CLI flags, auth methods, etc.). All such data must be queried from docs-management at runtime.

Changelog Access

MANDATORY: Access changelog via docs-management skill.