Skip to main content Inicio Creadores adu2021 skillxiv agent-skills-security-analysis
agent-skills-security-analysis Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Ir a la instalación Skills Marketplace Descubre y explora habilidades de IA creadas por la comunidad.
Ocupaciones relacionadas SOC
Basado en la clasificación ocupacional SOC
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Mostrar detalles del prompt Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/ADu2021/skillXiv --skill agent-skills-security-analysisEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Descargar Zip Descargando... Más de este repositorio meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
name agent-skills-security-analysis title Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2601.10338 keywords ["agent-security","skill-vetting","vulnerability-detection","threat-analysis","skill-marketplace"] description Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Overview
Conduct security analysis of modular agent skills before deployment. Use multi-stage detection combining static analysis and LLM-based semantic classification to identify vulnerabilities such as data exfiltration, privilege escalation, prompt injection, and supply chain risks.
When to Use
When integrating third-party agent skills into your systems
For skill marketplaces or skill package management
To build automated vetting pipelines before skill deployment
When auditing existing skill collections for vulnerabilities
When NOT to Use
For single-use, internally-developed skills
In fully sandboxed environments with no skill communication
For read-only skills with no side effects
In low-risk applications where skill compromise has minimal impact
Key Technical Components
SkillScan Detection Framework
Implement multi-stage detection pipeline combining static and semantic analysis.
class SkillScan :
def __init__ (self ):
self .static_analyzer = StaticAnalyzer()
self .semantic_classifier = SemanticClassifier()
def scan_skill (self, skill_code, skill_metadata ):
"""Comprehensive vulnerability detection"""
results = {
"static_findings" : self .static_analyzer.analyze(skill_code),
"semantic_findings" : self .semantic_classifier.classify(skill_code),
"metadata_issues" : self .check_metadata(skill_metadata),
"vulnerability_patterns" : []
}
results["vulnerability_patterns" ] = self .consolidate_findings(results)
results["risk_score" ] = self
return
def
consolidate_findings
self, findings
"""Identify distinct vulnerability patterns"""
for
in
"static_findings"
self
return
list
set
Static Analysis Component Detect vulnerabilities through code pattern matching.
class StaticAnalyzer :
VULNERABILITY_PATTERNS = {
"credential_exposure" : [
r"api[_]?key\s*=" ,
r"password\s*=" ,
r"token\s*="
],
"file_access_risk" : [
r"open\s*\(" ,
r"read\s*file" ,
r"write\s*file" ,
r"os\.remove"
],
"network_calls" : [
r"requests\." ,
r"urllib" ,
r"socket\." ,
r"send.*http"
],
"process_execution" : [
r"subprocess\." ,
r"os\.system" ,
r"popen"
]
}
def analyze (self, code ):
"""Find suspicious patterns in code"""
findings = []
for pattern_type, patterns in self .VULNERABILITY_PATTERNS.items():
for pattern in patterns:
matches = re.findall(pattern, code, re.IGNORECASE)
if matches:
findings.append({
"type" : pattern_type,
"pattern" : pattern,
"match_count" : len (matches),
"severity" : self .estimate_severity(pattern_type)
})
return findings
def estimate_severity (self, pattern_type ):
"""Assess severity of vulnerability pattern"""
severity_map = {
"credential_exposure" : "critical" ,
"process_execution" : "high" ,
"network_calls" : "medium" ,
"file_access_risk" : "medium"
}
return severity_map.get(pattern_type, "low" )
LLM-Based Semantic Classification Use language models to understand vulnerability intent.
class SemanticClassifier :
def classify (self, code ):
"""Identify vulnerability intent through semantic analysis"""
intent_categories = [
"data_exfiltration" ,
"privilege_escalation" ,
"prompt_injection" ,
"supply_chain_risk" ,
"benign_risky_pattern"
]
classifications = {}
for category in intent_categories:
prompt = f"""Analyze this code for {category} intent:
{code}
Is there evidence of {category} ? (yes/no/unclear)
Confidence: 0-1
"""
result = self .llm_classify(prompt)
classifications[category] = {
"detected" : result["answer" ],
"confidence" : result["confidence" ]
}
return classifications
def llm_classify (self, prompt ):
"""LLM-based semantic analysis"""
pass
Vulnerability Category Framework Organize vulnerabilities into actionable categories.
class VulnerabilityCategory :
CATEGORIES = {
"data_exfiltration" : {
"description" : "Attempt to send data outside system" ,
"prevalence" : 0.133 ,
"examples" : ["send_logs" , "collect_files" , "api_exfil" ]
},
"privilege_escalation" : {
"description" : "Attempt to gain higher permissions" ,
"prevalence" : 0.118 ,
"examples" : ["sudo_access" , "admin_check" , "permission_bypass" ]
},
"malicious_intent" : {
"description" : "Clear evidence of deliberate harm" ,
"prevalence" : 0.052 ,
"examples" : ["backdoor" , "ransomware_pattern" , "botnet" ]
},
"prompt_injection" : {
"description" : "Vulnerability to LLM prompt injection" ,
"prevalence" : 0.045 ,
"examples" : ["unescaped_input" , "eval_user_input" ]
},
"supply_chain" : {
"description" : "Risk to dependency management" ,
"prevalence" : 0.035 ,
"examples" : ["typosquatting" , "dependency_confusion" ]
}
}
@staticmethod
def get_risk_level (vulnerability ):
"""Map vulnerability to risk level"""
if vulnerability in ["data_exfiltration" , "privilege_escalation" , "malicious_intent" ]:
return "high"
elif vulnerability in ["prompt_injection" ]:
return "medium"
else :
return "low"
Risk Scoring System Assign quantitative risk scores to skills.
class RiskScorer :
def compute_score (self, findings ):
"""Compute overall risk score 0-1"""
if not findings:
return 0.0
critical_count = len ([f for f in findings if f.get("severity" ) == "critical" ])
high_count = len ([f for f in findings if f.get("severity" ) == "high" ])
medium_count = len ([f for f in findings if f.get("severity" ) == "medium" ])
score = (
critical_count * 0.5 +
high_count * 0.3 +
medium_count * 0.1
) / len (findings)
return min (1.0 , score)
def requires_vetting (self, risk_score ):
"""Determine if skill requires manual review"""
return risk_score > 0.3
Skill Bundling Analysis Identify that executable scripts increase vulnerability risk.
class BundlingAnalysis :
def assess_bundling_risk (self, skill ):
"""Check if skill includes executable scripts"""
has_scripts = any (
script_ext in skill["files" ]
for script_ext in [".py" , ".sh" , ".js" , ".exe" ]
)
if has_scripts:
return {"has_scripts" : True , "risk_multiplier" : 2.12 }
else :
return {"has_scripts" : False , "risk_multiplier" : 1.0 }
Performance Characteristics
Detection coverage: 31,132 skills analyzed
Vulnerability prevalence: 26.1% contain at least one vulnerability
Detection precision: 86.7%
Detection recall: 82.5%
14 distinct vulnerability patterns identified
Deployment Recommendations
Scan all incoming skills before adding to marketplaceUse tiered trust levels : vetted, pending-review, blockedImplement capability-based permissions to limit vulnerable skillsMaintain signature database of known vulnerable patternsRequire security attestation from skill publishersArchive and track all security findings for auditing
Vetting Checklist
References
26.1% of agent skills contain exploitable vulnerabilities
Executable script bundling increases vulnerability risk 2.12x
Multi-stage detection (static + semantic) required for comprehensive coverage
Mandatory vetting essential before production deployment
