Skip to main content Startseite Ersteller adu2021 skillxiv agent-skills-security-analysis
agent-skills-security-analysis Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Zur Installation springen Skills Marktplatz Entdecken und erkunden Sie KI-Skills, die von der Community erstellt wurden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Prompt-Details anzeigen Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
npx skills add https://github.com/ADu2021/skillXiv --skill agent-skills-security-analysisDer Befehl bleibt in einer Zeile. Scrollen Sie horizontal, um ihn vor dem Kopieren vollständig zu prüfen.
Sie bevorzugen eine lokale Kopie? Laden Sie die Dateien herunter, die SkillsMP derzeit vorliegen.
ZIP herunterladen Herunterladen... Mehr aus diesem Repository
Verwandte Berufe SOC
Basierend auf der SOC-Berufsklassifikation
name agent-skills-security-analysis title Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2601.10338 keywords ["agent-security","skill-vetting","vulnerability-detection","threat-analysis","skill-marketplace"] description Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Overview
Conduct security analysis of modular agent skills before deployment. Use multi-stage detection combining static analysis and LLM-based semantic classification to identify vulnerabilities such as data exfiltration, privilege escalation, prompt injection, and supply chain risks.
When to Use
When integrating third-party agent skills into your systems
For skill marketplaces or skill package management
To build automated vetting pipelines before skill deployment
When auditing existing skill collections for vulnerabilities
When NOT to Use
For single-use, internally-developed skills
In fully sandboxed environments with no skill communication
For read-only skills with no side effects
In low-risk applications where skill compromise has minimal impact
Key Technical Components
SkillScan Detection Framework
Implement multi-stage detection pipeline combining static and semantic analysis.
class SkillScan :
def __init__ (self ):
self .static_analyzer = StaticAnalyzer()
self .semantic_classifier = SemanticClassifier()
def scan_skill (self, skill_code, skill_metadata ):
"""Comprehensive vulnerability detection"""
results = {
"static_findings" : self .static_analyzer.analyze(skill_code),
"semantic_findings" : self .semantic_classifier.classify(skill_code),
"metadata_issues" : self .check_metadata(skill_metadata),
"vulnerability_patterns" : []
}
results["vulnerability_patterns" ] = self .consolidate_findings(results)
results["risk_score" ] = self
return
def
consolidate_findings
self, findings
"""Identify distinct vulnerability patterns"""
for
in
"static_findings"
self
return
list
set
Static Analysis Component Detect vulnerabilities through code pattern matching.
class StaticAnalyzer :
VULNERABILITY_PATTERNS = {
"credential_exposure" : [
r"api[_]?key\s*=" ,
r"password\s*=" ,
r"token\s*="
],
"file_access_risk" : [
r"open\s*\(" ,
r"read\s*file" ,
r"write\s*file" ,
r"os\.remove"
],
"network_calls" : [
r"requests\." ,
r"urllib" ,
r"socket\." ,
r"send.*http"
],
"process_execution" : [
r"subprocess\." ,
r"os\.system" ,
r"popen"
]
}
def analyze (self, code ):
"""Find suspicious patterns in code"""
findings = []
for pattern_type, patterns in self .VULNERABILITY_PATTERNS.items():
for pattern in patterns:
matches = re.findall(pattern, code, re.IGNORECASE)
if matches:
findings.append({
"type" : pattern_type,
"pattern" : pattern,
"match_count" : len (matches),
"severity" : self .estimate_severity(pattern_type)
})
return findings
def estimate_severity (self, pattern_type ):
"""Assess severity of vulnerability pattern"""
severity_map = {
"credential_exposure" : "critical" ,
"process_execution" : "high" ,
"network_calls" : "medium" ,
"file_access_risk" : "medium"
}
return severity_map.get(pattern_type, "low" )
LLM-Based Semantic Classification Use language models to understand vulnerability intent.
class SemanticClassifier :
def classify (self, code ):
"""Identify vulnerability intent through semantic analysis"""
intent_categories = [
"data_exfiltration" ,
"privilege_escalation" ,
"prompt_injection" ,
"supply_chain_risk" ,
"benign_risky_pattern"
]
classifications = {}
for category in intent_categories:
prompt = f"""Analyze this code for {category} intent:
{code}
Is there evidence of {category} ? (yes/no/unclear)
Confidence: 0-1
"""
result = self .llm_classify(prompt)
classifications[category] = {
"detected" : result["answer" ],
"confidence" : result["confidence" ]
}
return classifications
def llm_classify (self, prompt ):
"""LLM-based semantic analysis"""
pass
Vulnerability Category Framework Organize vulnerabilities into actionable categories.
class VulnerabilityCategory :
CATEGORIES = {
"data_exfiltration" : {
"description" : "Attempt to send data outside system" ,
"prevalence" : 0.133 ,
"examples" : ["send_logs" , "collect_files" , "api_exfil" ]
},
"privilege_escalation" : {
"description" : "Attempt to gain higher permissions" ,
"prevalence" : 0.118 ,
"examples" : ["sudo_access" , "admin_check" , "permission_bypass" ]
},
"malicious_intent" : {
"description" : "Clear evidence of deliberate harm" ,
"prevalence" : 0.052 ,
"examples" : ["backdoor" , "ransomware_pattern" , "botnet" ]
},
"prompt_injection" : {
"description" : "Vulnerability to LLM prompt injection" ,
"prevalence" : 0.045 ,
"examples" : ["unescaped_input" , "eval_user_input" ]
},
"supply_chain" : {
"description" : "Risk to dependency management" ,
"prevalence" : 0.035 ,
"examples" : ["typosquatting" , "dependency_confusion" ]
}
}
@staticmethod
def get_risk_level (vulnerability ):
"""Map vulnerability to risk level"""
if vulnerability in ["data_exfiltration" , "privilege_escalation" , "malicious_intent" ]:
return "high"
elif vulnerability in ["prompt_injection" ]:
return "medium"
else :
return "low"
Risk Scoring System Assign quantitative risk scores to skills.
class RiskScorer :
def compute_score (self, findings ):
"""Compute overall risk score 0-1"""
if not findings:
return 0.0
critical_count = len ([f for f in findings if f.get("severity" ) == "critical" ])
high_count = len ([f for f in findings if f.get("severity" ) == "high" ])
medium_count = len ([f for f in findings if f.get("severity" ) == "medium" ])
score = (
critical_count * 0.5 +
high_count * 0.3 +
medium_count * 0.1
) / len (findings)
return min (1.0 , score)
def requires_vetting (self, risk_score ):
"""Determine if skill requires manual review"""
return risk_score > 0.3
Skill Bundling Analysis Identify that executable scripts increase vulnerability risk.
class BundlingAnalysis :
def assess_bundling_risk (self, skill ):
"""Check if skill includes executable scripts"""
has_scripts = any (
script_ext in skill["files" ]
for script_ext in [".py" , ".sh" , ".js" , ".exe" ]
)
if has_scripts:
return {"has_scripts" : True , "risk_multiplier" : 2.12 }
else :
return {"has_scripts" : False , "risk_multiplier" : 1.0 }
Performance Characteristics
Detection coverage: 31,132 skills analyzed
Vulnerability prevalence: 26.1% contain at least one vulnerability
Detection precision: 86.7%
Detection recall: 82.5%
14 distinct vulnerability patterns identified
Deployment Recommendations
Scan all incoming skills before adding to marketplaceUse tiered trust levels : vetted, pending-review, blockedImplement capability-based permissions to limit vulnerable skillsMaintain signature database of known vulnerable patternsRequire security attestation from skill publishersArchive and track all security findings for auditing
Vetting Checklist
References
26.1% of agent skills contain exploitable vulnerabilities
Executable script bundling increases vulnerability risk 2.12x
Multi-stage detection (static + semantic) required for comprehensive coverage
Mandatory vetting essential before production deployment
