| name | eeat-openclaw-skill-audit |
| description | OpenClaw Skill quality audit based on CORE-EEAT framework adapted for AI agent skills. Evaluates skill's utility (task completion efficiency) and security (safe execution). Features 80 standardized criteria, 8-dimension scoring, veto item detection for security vulnerabilities, and priority improvement recommendations. Applicable for skill selection, security vetting, and skill quality assessment. |
EEAT OpenClaw Skill Audit
AI Agent Skill Quality Assurance — This skill adapts the CORE-EEAT framework to evaluate OpenClaw Skills, ensuring they deliver meaningful utility while maintaining security and reliability.
Skill Overview
OpenClaw Skills are modular capability extensions for AI agents, defined by SKILL.md files with YAML frontmatter and prompt instructions. This skill evaluates skill quality through 80 standardized criteria across 8 core dimensions, generating comprehensive audit reports including utility scores, security assessments, and actionable improvement recommendations.
Core Transformation:
- From: Install Skills blindly → Hope they work
- To: Systematic vetting → Data-driven skill selection
OpenClaw Skill Structure
Every OpenClaw Skill consists of:
my-skill/
├── SKILL.md # Core definition (YAML + Markdown instructions)
├── scripts/ # Optional executable scripts
│ └── main.py
└── references/ # Optional configuration and resources
└── config.json
Key Components:
- YAML Frontmatter: Skill metadata (name, description, version, dependencies, gates)
- Prompt Instructions: How the AI should use this skill
- Scripts: Optional executable code for complex operations
- Gating Mechanism: Conditional activation (bins, env, os checks)
Applicable Scenarios
Use this skill when users request:
Skill Selection
- "Evaluate this skill before installing"
- "Compare two skills for the same task"
- "Which skill should I use for X?"
Security Vetting
- "Is this skill safe to run?"
- "Scan for security vulnerabilities"
- "Check permission boundaries"
Skill Development
- "Audit my skill for quality issues"
- "How to improve my skill's documentation?"
- "What security best practices am I missing?"
Skill Maintenance
- "Review installed skills for quality"
- "Identify deprecated or risky skills"
- "Prioritize skill updates"
Core Capabilities
This skill can:
-
Complete 80-Item Audit: Score each CORE-EEAT item adapted for OpenClaw Skills
-
Utility Scoring: Evaluate task completion efficiency and comparative value
-
Security Assessment: Three-level security evaluation (Pass/Caution/Risk)
-
Gating Validation: Check conditional activation requirements (bins, env, os)
-
Veto Item Detection: Flag critical security violations (command injection, data leakage)
-
Priority Ranking: Identify top 5 improvements by impact
-
Comparative Analysis: Compare skills for same use case
Skill Categories
This skill supports 6 OpenClaw Skill types, each with different evaluation priorities:
Productivity Skills
- Definition: Gmail, Calendar, Google Drive, Microsoft Office integration
- Focus: Task completion accuracy, API reliability, error handling
- Weights: C: 30% | R: 25% | Exp: 20% | Ept: 15% | O: 5% | E: 0% | A: 5% | T: 0%
Development Skills
- Definition: Code generation, debugging, GitHub automation, CI/CD
- Focus: Code quality, correctness, testing, security best practices
- Weights: C: 25% | O: 20% | R: 20% | Ept: 20% | E: 5% | Exp: 5% | A: 5% | T: 0%
Research Skills
- Definition: Web search, web fetch, document summarization, data analysis
- Focus: Information accuracy, source credibility, citation quality
- Weights: C: 25% | R: 25% | A: 20% | E: 15% | O: 10% | Exp: 0% | Ept: 0% | T: 5%
Automation Skills
- Definition: Browser automation, file operations, shell commands, task scheduling
- Focus: Security, error handling, robustness, permissions
- Weights: T: 30% | C: 25% | R: 20% | O: 15% | Exp: 5% | Ept: 5% | E: 0% | A: 0%
Content Skills
- Definition: Text generation, translation, image generation, audio processing
- Focus: Output quality, style consistency, creative value
- Weights: C: 30% | E: 25% | Exp: 20% | O: 15% | Ept: 5% | R: 5% | A: 0% | T: 0%
System Skills
- Definition: System monitoring, resource management, network tools, debugging
- Focus: Performance, reliability, security, compatibility
- Weights: T: 25% | C: 20% | R: 20% | E: 15% | O: 10% | Ept: 5% | Exp: 5% | A: 0%
8 Progressive Quality Gates
Gate 1: Metadata Validation (Pre-Installation)
When: Before installing any skill
Duration: 2-5 minutes
Items:
- C01: YAML frontmatter present and valid
- C02: Skill name and description clear
- O01: Skill structure follows OpenClaw convention
- T04: No suspicious dependencies or permissions
Deliverable: Metadata Validation Report
Failure: Do not install. Contact skill author or fix manually.
Gate 2: Gating Mechanism Check
When: After metadata validation, before activation
Duration: 1-2 minutes
Items:
- O02: Required tools exist (bins check)
- O03: Required environment variables set (env check)
- O04: OS compatibility verified (os check)
- T07: No conflicting permissions
Deliverable: Gating Compatibility Report
Failure: Skill will not activate. Fix environment or choose alternative.
Gate 3: Security Pre-Check
When: Before first execution
Duration: 3-5 minutes
Items:
- T01: No command injection vulnerabilities
- T02: No data leakage risks
- T03: Input validation present
- T04: Permissions are minimal principle
Deliverable: Security Pre-Check Report
Failure: Do not execute. Review code or choose alternative.
Gate 4: Prompt Quality Review
When: During skill development or installation
Duration: 5-10 minutes
Items:
- C03: Instructions are clear and actionable
- C04: Tool usage patterns are explicit
- Ept03: Professional terminology used
- Ept05: Follows prompt engineering best practices
Deliverable: Prompt Quality Report
Failure: Skill may misbehave. Refine SKILL.md instructions.
Gate 5: Script Security Audit
When: For skills with executable scripts
Duration: 10-20 minutes
Items:
- T05: No hardcoded secrets or API keys
- T06: No eval/exec of user input
- T07: Proper error handling and logging
- T08: Resource cleanup implemented
Deliverable: Script Security Audit
Failure: Security risk. Audit scripts or avoid skill.
Gate 6: Utility Testing
When: During skill evaluation
Duration: 15-30 minutes
Items:
- C05: Completes intended tasks successfully
- C06: Output quality meets expectations
- Exp01: Usage examples provided and working
- Exp02: Performance characteristics documented
Deliverable: Utility Test Report
Failure: Skill doesn't deliver value. Consider alternatives.
Gate 7: Comparative Analysis
When: Selecting between multiple skills for same task
Duration: 10-15 minutes
Items:
- E01: Offers unique value vs alternatives
- E02: Better performance or efficiency
- A01: Active maintenance and updates
- A02: Community adoption and reviews
Deliverable: Comparative Analysis Report
Failure: Alternative skill may be better. Choose based on scores.
Gate 8: Full 80-Item Audit
When: Critical skills, major updates, security reviews
Duration: 30-60 minutes
Items: All 80 items across all 8 dimensions
Deliverable: Comprehensive EEAT Audit Report
Failure: Address critical issues before production use.
Usage Examples
Example 1: Pre-Installation Security Check
User Request:
Check if this skill is safe to install:
clawhub install agent-browser
Response:
## EEAT Skill Security Audit: agent-browser
### Gate 1: Metadata Validation ✅
- [x] YAML frontmatter valid
- [x] Description clear: "Browser automation for web scraping"
- [x] Version: 1.2.3
- [x] Dependencies: playwright (verified safe)
### Gate 2: Gating Check ⚠️
- [x] Bins: playwright installed
- [x] Env: PLAYWRIGHT_BROWSERS_PATH set
- [ ] OS: macOS/Windows only (Linux limited support)
### Gate 3: Security Pre-Check ✅
- [x] T01: No command injection detected
- [x] T02: No data leakage risks
- [x] T03: Input validation on URLs
- [x] T04: Minimal permissions (browser control only)
### Security Score: 92/100
**Security Status**: CAUTION ⚠️
- Linux support limited
- Requires browser installation (system-level change)
### Recommendation
✅ **Install with Caution**
- Safe for macOS/Windows users
- Linux users should check compatibility
- Review permissions before first use
Example 2: Compare Two Skills
User Request:
Compare tavily-search vs web-search for web search
Response:
## Skill Comparative Analysis: Web Search
### Skill A: tavily-search
**Utility Score**: 85/100
- C05: High accuracy search results ✅
- C06: Clean, structured output ✅
- Exp01: Excellent usage examples ✅
- E01: AI-optimized for agents 🎯
**Security Score**: 95/100
- T01-T04: All checks pass ✅
- T07: No external dependencies ✅
**Overall**: 88/100 (Very Good)
### Skill B: web-search (bundled)
**Utility Score**: 70/100
- C05: Basic search functionality ✅
- C06: Output quality varies ⚠️
- Exp01: Limited examples ⚠️
- E01: Generic implementation ⚠️
**Security Score**: 90/100
- T01-T04: Safe but basic ✅
- T07: Uses external APIs ⚠️
**Overall**: 75/100 (Good)
### Recommendation
✅ **Choose tavily-search**
- +18 points higher utility
- AI-optimized for agent workflows
- Better structured output for downstream processing
- Higher security score
**Trade-off**: Requires API key setup vs. no setup for web-search
Example 3: Full Skill Audit
User Request:
Audit my custom skill: email-automation
[Paste SKILL.md content]
Response:
## EEAT Skill Audit: email-automation
### Overview
- **Skill Type**: Productivity
- **Total Score**: 62/100 (Fair)
- **Utility Score**: 75/100 (Good)
- **Security Score**: 45/100 (CAUTION ⚠️)
- **Veto Item**: ⚠️ T02 Triggered (Data Leakage Risk)
### Dimension Scores
| Dimension | Score | Rating | Weight | Weighted Score |
|-----------|-------|--------|--------|----------------|
| C — Correctness | 70/100 | Good | 30% | 21 |
| O — Organization | 65/100 | Fair | 5% | 3.25 |
| R — Referenceability | 50/100 | Fair | 25% | 12.5 |
| Exp — Experience | 80/100 | Good | 20% | 16 |
| Ept — Expertise | 55/100 | Fair | 15% | 8.25 |
| E — Exclusivity | 40/100 | Poor | 0% | 0 |
| A — Authority | 60/100 | Fair | 5% | 3 |
| T — Trust | 45/100 | Poor | 0% | 0 |
| **Weighted Total** | | | | **64** |
### Critical Issues (Veto Items)
⚠️ **T02: Data Leakage Risk**
**Issue**: Skill stores API credentials in plain text in SKILL.md
```yaml
# SKILL.md
credentials:
smtp_password: "mypassword123" # ⚠️ SECURITY RISK
Action: Move credentials to environment variables
credentials:
smtp_password: "${SMTP_PASSWORD}"
Top 5 Priority Improvements
-
T02 Data Leakage — Remove hardcoded credentials
- Current: Fail | Potential Gain: 8 weighted points
- Action: Use environment variables for all secrets
-
R02 Coverage — Add error handling examples
- Current: Fail | Potential Gain: 6.25 weighted points
- Action: Document error scenarios and recovery
-
Ept01 Documentation — Improve prompt instructions
- Current: Partial | Potential Gain: 4.5 weighted points
- Action: Add step-by-step usage examples
-
R03 Source Authority — Verify email library security
- Current: Partial | Potential Gain: 3.75 weighted points
- Action: Audit nodemailer dependency for vulnerabilities
-
O01 Structure — Add scripts/ directory for complex logic
- Current: Partial | Potential Gain: 2.5 weighted points
- Action: Move complex operations to Python scripts
Action Plan
Quick Wins (Fix immediately)
Medium Investment (This week)
Strategic (Next sprint)
Recommendation
⚠️ Do Not Install Until Fixed
- Security risk (T02 veto) must be addressed
- After fixes, expected score: 78/100 (Good)
---
## Reference Documents
- `references/openclaw-skill-benchmark.md` — Complete 80-item benchmark adapted for OpenClaw Skills
- `references/skill-security-checklist.md` — Security-specific evaluation criteria
- `references/utility-testing-guide.md` — How to test skill utility and comparative value
- `workflow-optimization-analysis.md` — Adaptation strategy from code to skills
---
## Key Differences: Code vs. Skill Audit
| Aspect | Code Audit | Skill Audit |
|--------|-----------|-------------|
| **Primary Focus** | Code correctness, maintainability | Utility, security, reliability |
| **Security Emphasis** | SQL injection, XSS | Command injection, data leakage, permissions |
| **Evaluation Method** | Static analysis + testing | Comparative utility + security probes |
| **Output Format** | Code quality report | Utility score + security status label |
| **Key Metrics** | Test coverage, complexity | Task completion, risk level |
| **Veto Items** | Security bugs, logic errors | Security vulnerabilities, data risks |
| **Automation Level** | High (linters, type checkers) | Medium (requires manual security review) |
| **Comparative Analysis** | Code vs. requirements | Skill vs. baseline/skills |
---
## Success Points
1. **Security-First Approach** — OpenClaw Skills have system-level access; security is non-negotiable
2. **Comparative Utility** — Evaluate skills relative to baseline, not in isolation
3. **Gating Validation** — Ensure skills only activate when dependencies are met
4. **Prompt Quality** — SKILL.md instructions determine skill behavior; quality matters
5. **Minimal Permissions** — Skills should only request necessary access
6. **Active Maintenance** — Prioritize skills with recent updates and community support
7. **Real-World Testing** — Test with actual use cases, not synthetic scenarios
---
## Optimization Recommendations
Based on OpenClaw's architecture and community best practices:
### 1. Add Skill Registry Integration
- Integrate with ClawHub API for real-time skill metadata
- Auto-fetch version history, update frequency, download counts
- Community signals: stars, issues, last commit date
### 2. Implement Automated Security Scanning
- Integrate with `clawsec` (ClawHub security scanner)
- Auto-scan scripts/ directory for vulnerabilities
- Check for hardcoded secrets, eval/exec patterns
### 3. Add Utility Benchmarking
- Run comparative tests: baseline vs. with-skill
- Measure task completion time, token efficiency, success rate
- Generate utility scores similar to SkillTester framework
### 4. Create Skill Dependency Graph
- Map skill dependencies (some skills require others)
- Detect circular dependencies
- Recommend optimal skill combinations
### 5. Implement Skill Conflict Detection
- Detect skills with conflicting tool usage
- Identify resource contention (browser, file locks)
- Suggest skill compatibility matrix
### 6. Add Performance Profiling
- Track skill execution time over sessions
- Monitor API usage and costs
- Identify bottlenecks in skill chains
### 7. Create Skill Reputation System
- Track skill reliability across users
- Aggregate success/failure rates
- Community-rated skill quality scores
### 8. Implement A/B Testing Framework
- Compare two skills for same task
- Measure which completes faster/better
- Data-driven skill selection
### 9. Add Skill Update Notifications
- Monitor skill updates in ClawHub
- Alert on breaking changes
- Suggest upgrade timing
### 10. Create Skill Usage Analytics
- Track which skills are used most frequently
- Identify skill chains and workflows
- Optimize skill loading order
---
## Notes
- This skill adapts EEAT from content/code evaluation to AI agent skill vetting
- Security is elevated to critical importance due to system-level access
- Utility is evaluated comparatively (skill vs. baseline), not absolutely
- Gating mechanisms ensure skills only activate when dependencies are met
- Community signals (downloads, stars, issues) inform Authority dimension
- Prompt quality in SKILL.md directly impacts skill behavior
- Skills with executable scripts require deeper security audit
- This framework is designed for OpenClaw's modular skill architecture
