Skip to main content 首页 创作者 adu2021 skillxiv agent-skills-security-analysis
agent-skills-security-analysis Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
跳到安装 Skills Marketplace 发现并探索由社区构建的 Agent Skills
用 Codex 或 Claude 帮你安装 复制这段 Prompt,粘贴到 Codex、Claude 或其他助手里,让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt;运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill agent-skills-security-analysis命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地?可下载 SkillsMP 当前能够提供的文件。
下载 Zip 下载中... 同仓库更多 Skills meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
name agent-skills-security-analysis title Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2601.10338 keywords ["agent-security","skill-vetting","vulnerability-detection","threat-analysis","skill-marketplace"] description Empirically analyzes 31,132 agent skills to identify 14 distinct vulnerability patterns, finding 26.1% contain security flaws including data exfiltration, privilege escalation, and malicious intent risks that require mandatory vetting.
Overview
Conduct security analysis of modular agent skills before deployment. Use multi-stage detection combining static analysis and LLM-based semantic classification to identify vulnerabilities such as data exfiltration, privilege escalation, prompt injection, and supply chain risks.
When to Use
When integrating third-party agent skills into your systems
For skill marketplaces or skill package management
To build automated vetting pipelines before skill deployment
When auditing existing skill collections for vulnerabilities
When NOT to Use
For single-use, internally-developed skills
In fully sandboxed environments with no skill communication
For read-only skills with no side effects
In low-risk applications where skill compromise has minimal impact
Key Technical Components
SkillScan Detection Framework
Implement multi-stage detection pipeline combining static and semantic analysis.
class SkillScan :
def __init__ (self ):
self .static_analyzer = StaticAnalyzer()
self .semantic_classifier = SemanticClassifier()
def scan_skill (self, skill_code, skill_metadata ):
"""Comprehensive vulnerability detection"""
results = {
"static_findings" : self .static_analyzer.analyze(skill_code),
"semantic_findings" : self .semantic_classifier.classify(skill_code),
"metadata_issues" : self .check_metadata(skill_metadata),
"vulnerability_patterns" : []
}
results["vulnerability_patterns" ] = self .consolidate_findings(results)
results["risk_score" ] = .compute_risk_score(results)
results
( ):
patterns = []
finding findings[ ]:
pattern = .classify_pattern(finding)
patterns.append(pattern)
( (patterns))
self
return
def
consolidate_findings
self, findings
"""Identify distinct vulnerability patterns"""
for
in
"static_findings"
self
return
list
set
Static Analysis Component Detect vulnerabilities through code pattern matching.
class StaticAnalyzer :
VULNERABILITY_PATTERNS = {
"credential_exposure" : [
r"api[_]?key\s*=" ,
r"password\s*=" ,
r"token\s*="
],
"file_access_risk" : [
r"open\s*\(" ,
r"read\s*file" ,
r"write\s*file" ,
r"os\.remove"
],
"network_calls" : [
r"requests\." ,
r"urllib" ,
r"socket\." ,
r"send.*http"
],
"process_execution" : [
r"subprocess\." ,
r"os\.system" ,
r"popen"
]
}
def analyze (self, code ):
"""Find suspicious patterns in code"""
findings = []
for pattern_type, patterns in self .VULNERABILITY_PATTERNS.items():
for pattern in patterns:
matches = re.findall(pattern, code, re.IGNORECASE)
if matches:
findings.append({
"type" : pattern_type,
"pattern" : pattern,
"match_count" : len (matches),
"severity" : self .estimate_severity(pattern_type)
})
return findings
def estimate_severity (self, pattern_type ):
"""Assess severity of vulnerability pattern"""
severity_map = {
"credential_exposure" : "critical" ,
"process_execution" : "high" ,
"network_calls" : "medium" ,
"file_access_risk" : "medium"
}
return severity_map.get(pattern_type, "low" )
LLM-Based Semantic Classification Use language models to understand vulnerability intent.
class SemanticClassifier :
def classify (self, code ):
"""Identify vulnerability intent through semantic analysis"""
intent_categories = [
"data_exfiltration" ,
"privilege_escalation" ,
"prompt_injection" ,
"supply_chain_risk" ,
"benign_risky_pattern"
]
classifications = {}
for category in intent_categories:
prompt = f"""Analyze this code for {category} intent:
{code}
Is there evidence of {category} ? (yes/no/unclear)
Confidence: 0-1
"""
result = self .llm_classify(prompt)
classifications[category] = {
"detected" : result["answer" ],
"confidence" : result["confidence" ]
}
return classifications
def llm_classify (self, prompt ):
"""LLM-based semantic analysis"""
pass
Vulnerability Category Framework Organize vulnerabilities into actionable categories.
class VulnerabilityCategory :
CATEGORIES = {
"data_exfiltration" : {
"description" : "Attempt to send data outside system" ,
"prevalence" : 0.133 ,
"examples" : ["send_logs" , "collect_files" , "api_exfil" ]
},
"privilege_escalation" : {
"description" : "Attempt to gain higher permissions" ,
"prevalence" : 0.118 ,
"examples" : ["sudo_access" , "admin_check" , "permission_bypass" ]
},
"malicious_intent" : {
"description" : "Clear evidence of deliberate harm" ,
"prevalence" : 0.052 ,
"examples" : ["backdoor" , "ransomware_pattern" , "botnet" ]
},
"prompt_injection" : {
"description" : "Vulnerability to LLM prompt injection" ,
"prevalence" : 0.045 ,
"examples" : ["unescaped_input" , "eval_user_input" ]
},
"supply_chain" : {
"description" : "Risk to dependency management" ,
"prevalence" : 0.035 ,
"examples" : ["typosquatting" , "dependency_confusion" ]
}
}
@staticmethod
def get_risk_level (vulnerability ):
"""Map vulnerability to risk level"""
if vulnerability in ["data_exfiltration" , "privilege_escalation" , "malicious_intent" ]:
return "high"
elif vulnerability in ["prompt_injection" ]:
return "medium"
else :
return "low"
Risk Scoring System Assign quantitative risk scores to skills.
class RiskScorer :
def compute_score (self, findings ):
"""Compute overall risk score 0-1"""
if not findings:
return 0.0
critical_count = len ([f for f in findings if f.get("severity" ) == "critical" ])
high_count = len ([f for f in findings if f.get("severity" ) == "high" ])
medium_count = len ([f for f in findings if f.get("severity" ) == "medium" ])
score = (
critical_count * 0.5 +
high_count * 0.3 +
medium_count * 0.1
) / len (findings)
return min (1.0 , score)
def requires_vetting (self, risk_score ):
"""Determine if skill requires manual review"""
return risk_score > 0.3
Skill Bundling Analysis Identify that executable scripts increase vulnerability risk.
class BundlingAnalysis :
def assess_bundling_risk (self, skill ):
"""Check if skill includes executable scripts"""
has_scripts = any (
script_ext in skill["files" ]
for script_ext in [".py" , ".sh" , ".js" , ".exe" ]
)
if has_scripts:
return {"has_scripts" : True , "risk_multiplier" : 2.12 }
else :
return {"has_scripts" : False , "risk_multiplier" : 1.0 }
Performance Characteristics
Detection coverage: 31,132 skills analyzed
Vulnerability prevalence: 26.1% contain at least one vulnerability
Detection precision: 86.7%
Detection recall: 82.5%
14 distinct vulnerability patterns identified
Deployment Recommendations
Scan all incoming skills before adding to marketplace
Use tiered trust levels : vetted, pending-review, blocked
Implement capability-based permissions to limit vulnerable skills
Maintain signature database of known vulnerable patterns
Require security attestation from skill publishers
Archive and track all security findings for auditing
Vetting Checklist
References
26.1% of agent skills contain exploitable vulnerabilities
Executable script bundling increases vulnerability risk 2.12x
Multi-stage detection (static + semantic) required for comprehensive coverage
Mandatory vetting essential before production deployment