Skip to main content Início Criadores adu2021 skillxiv agent-skills-security-analysis
agent-skills-security-analysis Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Ir para a instalação Skills Marketplace Descubra e explore skills de IA criadas pela comunidade.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Mostrar detalhes do prompt Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
npx skills add https://github.com/ADu2021/skillXiv --skill agent-skills-security-analysisO comando permanece em uma só linha. Role horizontalmente para revisá-lo antes de copiar.
Prefere uma cópia local? Baixe os arquivos disponíveis atualmente no SkillsMP.
Baixar Zip Baixando... Mais deste repositório meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
Ocupações relacionadas SOC
Baseado na classificação ocupacional SOC
name agent-skills-security-analysis title Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2601.10338 keywords ["agent-security","skill-vetting","vulnerability-detection","threat-analysis","skill-marketplace"] description Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Overview
Conduct security analysis of modular agent skills before deployment. Use multi-stage detection combining static analysis and LLM-based semantic classification to identify vulnerabilities such as data exfiltration, privilege escalation, prompt injection, and supply chain risks.
When to Use
When integrating third-party agent skills into your systems
For skill marketplaces or skill package management
To build automated vetting pipelines before skill deployment
When auditing existing skill collections for vulnerabilities
When NOT to Use
For single-use, internally-developed skills
In fully sandboxed environments with no skill communication
For read-only skills with no side effects
In low-risk applications where skill compromise has minimal impact
Key Technical Components
SkillScan Detection Framework
Implement multi-stage detection pipeline combining static and semantic analysis.
class SkillScan :
def __init__ (self ):
self .static_analyzer = StaticAnalyzer()
self .semantic_classifier = SemanticClassifier()
def scan_skill (self, skill_code, skill_metadata ):
"""Comprehensive vulnerability detection"""
results = {
"static_findings" : self .static_analyzer.analyze(skill_code),
"semantic_findings" : self .semantic_classifier.classify(skill_code),
"metadata_issues" : self .check_metadata(skill_metadata),
"vulnerability_patterns" : []
}
results["vulnerability_patterns" ] = self .consolidate_findings(results)
results["risk_score" ] = self
return
def
consolidate_findings
self, findings
"""Identify distinct vulnerability patterns"""
for
in
"static_findings"
self
return
list
set
Static Analysis Component Detect vulnerabilities through code pattern matching.
class StaticAnalyzer :
VULNERABILITY_PATTERNS = {
"credential_exposure" : [
r"api[_]?key\s*=" ,
r"password\s*=" ,
r"token\s*="
],
"file_access_risk" : [
r"open\s*\(" ,
r"read\s*file" ,
r"write\s*file" ,
r"os\.remove"
],
"network_calls" : [
r"requests\." ,
r"urllib" ,
r"socket\." ,
r"send.*http"
],
"process_execution" : [
r"subprocess\." ,
r"os\.system" ,
r"popen"
]
}
def analyze (self, code ):
"""Find suspicious patterns in code"""
findings = []
for pattern_type, patterns in self .VULNERABILITY_PATTERNS.items():
for pattern in patterns:
matches = re.findall(pattern, code, re.IGNORECASE)
if matches:
findings.append({
"type" : pattern_type,
"pattern" : pattern,
"match_count" : len (matches),
"severity" : self .estimate_severity(pattern_type)
})
return findings
def estimate_severity (self, pattern_type ):
"""Assess severity of vulnerability pattern"""
severity_map = {
"credential_exposure" : "critical" ,
"process_execution" : "high" ,
"network_calls" : "medium" ,
"file_access_risk" : "medium"
}
return severity_map.get(pattern_type, "low" )
LLM-Based Semantic Classification Use language models to understand vulnerability intent.
class SemanticClassifier :
def classify (self, code ):
"""Identify vulnerability intent through semantic analysis"""
intent_categories = [
"data_exfiltration" ,
"privilege_escalation" ,
"prompt_injection" ,
"supply_chain_risk" ,
"benign_risky_pattern"
]
classifications = {}
for category in intent_categories:
prompt = f"""Analyze this code for {category} intent:
{code}
Is there evidence of {category} ? (yes/no/unclear)
Confidence: 0-1
"""
result = self .llm_classify(prompt)
classifications[category] = {
"detected" : result["answer" ],
"confidence" : result["confidence" ]
}
return classifications
def llm_classify (self, prompt ):
"""LLM-based semantic analysis"""
pass
Vulnerability Category Framework Organize vulnerabilities into actionable categories.
class VulnerabilityCategory :
CATEGORIES = {
"data_exfiltration" : {
"description" : "Attempt to send data outside system" ,
"prevalence" : 0.133 ,
"examples" : ["send_logs" , "collect_files" , "api_exfil" ]
},
"privilege_escalation" : {
"description" : "Attempt to gain higher permissions" ,
"prevalence" : 0.118 ,
"examples" : ["sudo_access" , "admin_check" , "permission_bypass" ]
},
"malicious_intent" : {
"description" : "Clear evidence of deliberate harm" ,
"prevalence" : 0.052 ,
"examples" : ["backdoor" , "ransomware_pattern" , "botnet" ]
},
"prompt_injection" : {
"description" : "Vulnerability to LLM prompt injection" ,
"prevalence" : 0.045 ,
"examples" : ["unescaped_input" , "eval_user_input" ]
},
"supply_chain" : {
"description" : "Risk to dependency management" ,
"prevalence" : 0.035 ,
"examples" : ["typosquatting" , "dependency_confusion" ]
}
}
@staticmethod
def get_risk_level (vulnerability ):
"""Map vulnerability to risk level"""
if vulnerability in ["data_exfiltration" , "privilege_escalation" , "malicious_intent" ]:
return "high"
elif vulnerability in ["prompt_injection" ]:
return "medium"
else :
return "low"
Risk Scoring System Assign quantitative risk scores to skills.
class RiskScorer :
def compute_score (self, findings ):
"""Compute overall risk score 0-1"""
if not findings:
return 0.0
critical_count = len ([f for f in findings if f.get("severity" ) == "critical" ])
high_count = len ([f for f in findings if f.get("severity" ) == "high" ])
medium_count = len ([f for f in findings if f.get("severity" ) == "medium" ])
score = (
critical_count * 0.5 +
high_count * 0.3 +
medium_count * 0.1
) / len (findings)
return min (1.0 , score)
def requires_vetting (self, risk_score ):
"""Determine if skill requires manual review"""
return risk_score > 0.3
Skill Bundling Analysis Identify that executable scripts increase vulnerability risk.
class BundlingAnalysis :
def assess_bundling_risk (self, skill ):
"""Check if skill includes executable scripts"""
has_scripts = any (
script_ext in skill["files" ]
for script_ext in [".py" , ".sh" , ".js" , ".exe" ]
)
if has_scripts:
return {"has_scripts" : True , "risk_multiplier" : 2.12 }
else :
return {"has_scripts" : False , "risk_multiplier" : 1.0 }
Performance Characteristics
Detection coverage: 31,132 skills analyzed
Vulnerability prevalence: 26.1% contain at least one vulnerability
Detection precision: 86.7%
Detection recall: 82.5%
14 distinct vulnerability patterns identified
Deployment Recommendations
Scan all incoming skills before adding to marketplaceUse tiered trust levels : vetted, pending-review, blockedImplement capability-based permissions to limit vulnerable skillsMaintain signature database of known vulnerable patternsRequire security attestation from skill publishersArchive and track all security findings for auditing
Vetting Checklist
References
26.1% of agent skills contain exploitable vulnerabilities
Executable script bundling increases vulnerability risk 2.12x
Multi-stage detection (static + semantic) required for comprehensive coverage
Mandatory vetting essential before production deployment
