菜单
Implementing safety filters, content moderation, and guardrails for AI system inputs and outputs
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/pluginagentmarketplace/custom-plugin-ai-red-teaming --skill input-output-guardrails命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
CI/CD integration and automation frameworks for continuous AI security testing
Standard datasets and benchmarks for evaluating AI security, robustness, and safety
Professional certifications, CTF competitions, and training resources for AI security practitioners
基于 SOC 职业分类
正在显示 SKILL.md
| name | input-output-guardrails |
| version | 2.0.0 |
| description | Implementing safety filters, content moderation, and guardrails for AI system inputs and outputs |
| sasmp_version | 1.3.0 |
| bonded_agent | 05-defense-strategy-developer |
| bond_type | SECONDARY_BOND |
| input_schema | {"type":"object","required":["guardrail_type"],"properties":{"guardrail_type":{"type":"string","enum":["input","output","both"]},"strictness":{"type":"string","enum":["permissive","balanced","strict"],"default":"balanced"}}} |
| output_schema | {"type":"object","properties":{"blocked_requests":{"type":"integer"},"filtered_outputs":{"type":"integer"},"false_positive_rate":{"type":"number"}}} |
| owasp_llm_2025 | ["LLM01","LLM02","LLM05","LLM07"] |
| nist_ai_rmf | ["Manage"] |
Implement multi-layer safety systems to filter malicious inputs and harmful outputs.
Skill: input-output-guardrails
Agent: 05-defense-strategy-developer
OWASP: LLM01 (Injection), LLM02 (Disclosure), LLM05 (Output), LLM07 (Leakage)
NIST: Manage
Use Case: Production safety filtering
User Input → [Input Guardrails] → [AI Model] → [Output Guardrails] → Response
↓ ↓
[Blocked/Modified] [Blocked/Modified]
↓ ↓
[Fallback Response] [Safe Alternative]
Category: prompt_injection
Latency: <10ms
Block Rate: 95%+
class InputGuardrails:
INJECTION_PATTERNS = [
r'ignore\s+(previous|prior|all)\s+(instructions?|guidelines?)',
r'you\s+are\s+(now|an?)\s+(unrestricted|evil)',
r'(developer|admin|debug)\s+mode',
r'bypass\s+(safety|security|filter)',
r'pretend\s+(you|to)\s+(are|be)',
r'what\s+(is|are)\s+your\s+(instructions?|prompt)',
]
def ():
.patterns = [re.(p, re.I) p .INJECTION_PATTERNS]
.max_length = config.get(, )
.pii_detector = PIIDetector()
() -> [, ]:
(user_input) > .max_length:
,
user_input.strip():
,
pattern .patterns:
pattern.search(user_input):
,
.pii_detector.contains_pii(user_input):
, .pii_detector.redact(user_input)
, user_input
class PIIDetector:
PATTERNS = {
'ssn': r'\b\d{3}-\d{2}-\d{4}\b',
'credit_card': r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b',
'email': r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}',
'phone': r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b',
'api_key': r'(sk|pk)[-_][a-zA-Z0-9]{20,}',
}
def contains_pii(self, text: str) -> bool:
for pattern in self.PATTERNS.values():
if re.search(pattern, text):
return True
return False
def redact(self, text: str) -> str:
for name, pattern in self.PATTERNS.items():
text = re.sub(pattern, f'[REDACTED_{name.upper()}]', text)
return text
Limits:
max_tokens_input: 4096
max_requests_per_minute: 60
max_concurrent: 5
cost_limit_per_hour: $10
Actions:
exceeded_tokens: truncate
exceeded_rate: queue (5s backoff)
exceeded_concurrent: reject
exceeded_cost: block
class OutputGuardrails:
def __init__(self, config):
self.toxicity_threshold = config.get('toxicity', 0.3)
self.toxicity_model = load_toxicity_classifier()
self.blocklist = self._load_blocklist()
def filter(self, response: str) -> tuple[str, dict]:
metadata = {'filtered': False, 'reasons': []}
# Toxicity check
toxicity = self.toxicity_model.predict(response)
if toxicity > self.toxicity_threshold:
metadata['filtered'] = True
metadata['reasons'].append('toxicity')
return self._safe_response(), metadata
# Blocklist check
for term in self.blocklist:
if term.lower() in response.lower():
metadata['filtered'] = True
metadata['reasons'].append('blocklist')
return self._safe_response(), metadata
# System prompt leak detection
if self._detects_system_leak(response):
metadata['filtered'] = True
metadata['reasons'].append('system_leak')
response = self._redact_system_content(response)
return response, metadata
def _detects_system_leak(self, response: str) -> bool:
leak_indicators = [
'you are a helpful',
'your instructions are',
'system prompt:',
]
return any(ind in response.lower() for ind in leak_indicators)
class OutputRedactor:
SENSITIVE_PATTERNS = {
'api_key': r'[a-zA-Z0-9_-]{20,}(?:key|token|secret)',
'password': r'password["\']?\s*[:=]\s*["\']?[^\s"\']+',
'connection_string': r'(mongodb|mysql|postgres)://[^\s]+',
'ip_address': r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b',
}
def redact(self, response: str) -> str:
for name, pattern in self.SENSITIVE_PATTERNS.items():
response = re.sub(pattern, '[REDACTED]', response, flags=re.I)
return response
Factuality Checks:
major_claims:
action: flag_for_verification
threshold: confidence < 0.8
citations:
action: verify_source_exists
block_if: source_not_found
uncertainty:
action: add_disclaimer
phrases: ["I'm not certain", "might be", "could be"]
# guardrails_config.yaml
input:
injection_detection: true
pii_redaction: true
max_length: 4096
rate_limit: 60/min
output:
toxicity_threshold: 0.3
blocklist_enabled: true
sensitive_redaction: true
system_leak_detection: true
fallback:
input_blocked: "I cannot process this request."
output_blocked: "I cannot provide this information."
logging:
log_blocked: true
log_filtered: true
include_reason: false # Privacy
┌──────────────────┬─────────┬────────┬──────────┐
│ Metric │ Target │ Actual │ Status │
├──────────────────┼─────────┼────────┼──────────┤
│ Injection Block │ >95% │ 97% │ ✓ PASS │
│ False Positive │ <2% │ 1.5% │ ✓ PASS │
│ Latency Impact │ <50ms │ 35ms │ ✓ PASS │
│ Toxicity Block │ >90% │ 92% │ ✓ PASS │
│ PII Redaction │ >99% │ 99.5% │ ✓ PASS │
└──────────────────┴─────────┴────────┴──────────┘
Issue: High false positive rate
Solution: Tune patterns, add allowlist, use context
Issue: Latency too high
Solution: Optimize regex, use compiled patterns, cache
Issue: Bypassed by encoding
Solution: Normalize unicode, decode before checking
| Component | Purpose |
|---|---|
| Agent 05 | Implements guardrails |
| /defend | Configuration recommendations |
| CI/CD | Automated testing |
| Monitoring | Alert on filter triggers |
Protect AI systems with comprehensive input/output guardrails.