メニュー
Implementing safety filters, content moderation, and guardrails for AI system inputs and outputs
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/pluginagentmarketplace/custom-plugin-ai-red-teaming --skill input-output-guardrailsコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
| name | input-output-guardrails |
| version | 2.0.0 |
| description | Implementing safety filters, content moderation, and guardrails for AI system inputs and outputs |
| sasmp_version | 1.3.0 |
| bonded_agent | 05-defense-strategy-developer |
| bond_type | SECONDARY_BOND |
| input_schema | {"type":"object","required":["guardrail_type"],"properties":{"guardrail_type":{"type":"string","enum":["input","output","both"]},"strictness":{"type":"string","enum":["permissive","balanced","strict"],"default":"balanced"}}} |
| output_schema | {"type":"object","properties":{"blocked_requests":{"type":"integer"},"filtered_outputs":{"type":"integer"},"false_positive_rate":{"type":"number"}}} |
| owasp_llm_2025 | ["LLM01","LLM02","LLM05","LLM07"] |
| nist_ai_rmf | ["Manage"] |
Implement multi-layer safety systems to filter malicious inputs and harmful outputs.
Skill: input-output-guardrails
Agent: 05-defense-strategy-developer
OWASP: LLM01 (Injection), LLM02 (Disclosure), LLM05 (Output), LLM07 (Leakage)
NIST: Manage
Use Case: Production safety filtering
User Input → [Input Guardrails] → [AI Model] → [Output Guardrails] → Response
↓ ↓
[Blocked/Modified] [Blocked/Modified]
↓ ↓
[Fallback Response] [Safe Alternative]
Category: prompt_injection
Latency: <10ms
Block Rate: 95%+
class InputGuardrails:
INJECTION_PATTERNS = [
r'ignore\s+(previous|prior|all)\s+(instructions?|guidelines?)',
r'you\s+are\s+(now|an?)\s+(unrestricted|evil)',
r'(developer|admin|debug)\s+mode',
r'bypass\s+(safety|security|filter)',
r'pretend\s+(you|to)\s+(are|be)',
r'what\s+(is|are)\s+your\s+(instructions?|prompt)',
]
def ():
.patterns = [re.(p, re.I) p .INJECTION_PATTERNS]
.max_length = config.get(, )
.pii_detector = PIIDetector()
() -> [, ]:
(user_input) > .max_length:
,
user_input.strip():
,
pattern .patterns:
pattern.search(user_input):
,
.pii_detector.contains_pii(user_input):
, .pii_detector.redact(user_input)
, user_input
class PIIDetector:
PATTERNS = {
'ssn': r'\b\d{3}-\d{2}-\d{4}\b',
'credit_card': r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b',
'email': r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}',
'phone': r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b',
'api_key': r'(sk|pk)[-_][a-zA-Z0-9]{20,}',
}
def contains_pii(self, text: str) -> bool:
for pattern in self.PATTERNS.values():
if re.search(pattern, text):
return True
return False
def redact(self, text: str) -> str:
for name, pattern in self.PATTERNS.items():
text = re.sub(pattern, f'[REDACTED_{name.upper()}]', text)
return text
Limits:
max_tokens_input: 4096
max_requests_per_minute: 60
max_concurrent: 5
cost_limit_per_hour: $10
Actions:
exceeded_tokens: truncate
exceeded_rate: queue (5s backoff)
exceeded_concurrent: reject
exceeded_cost: block
class OutputGuardrails:
def __init__(self, config):
self.toxicity_threshold = config.get('toxicity', 0.3)
self.toxicity_model = load_toxicity_classifier()
self.blocklist = self._load_blocklist()
def filter(self, response: str) -> tuple[str, dict]:
metadata = {'filtered': False, 'reasons': []}
# Toxicity check
toxicity = self.toxicity_model.predict(response)
if toxicity > self.toxicity_threshold:
metadata['filtered'] = True
metadata['reasons'].append('toxicity')
return self._safe_response(), metadata
# Blocklist check
for term in self.blocklist:
if term.lower() in response.lower():
metadata['filtered'] = True
metadata['reasons'].append('blocklist')
return self._safe_response(), metadata
# System prompt leak detection
if self._detects_system_leak(response):
metadata['filtered'] = True
metadata['reasons'].append('system_leak')
response = self._redact_system_content(response)
return response, metadata
def _detects_system_leak(self, response: str) -> bool:
leak_indicators = [
'you are a helpful',
'your instructions are',
'system prompt:',
]
return any(ind in response.lower() for ind in leak_indicators)
class OutputRedactor:
SENSITIVE_PATTERNS = {
'api_key': r'[a-zA-Z0-9_-]{20,}(?:key|token|secret)',
'password': r'password["\']?\s*[:=]\s*["\']?[^\s"\']+',
'connection_string': r'(mongodb|mysql|postgres)://[^\s]+',
'ip_address': r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b',
}
def redact(self, response: str) -> str:
for name, pattern in self.SENSITIVE_PATTERNS.items():
response = re.sub(pattern, '[REDACTED]', response, flags=re.I)
return response
Factuality Checks:
major_claims:
action: flag_for_verification
threshold: confidence < 0.8
citations:
action: verify_source_exists
block_if: source_not_found
uncertainty:
action: add_disclaimer
phrases: ["I'm not certain", "might be", "could be"]
# guardrails_config.yaml
input:
injection_detection: true
pii_redaction: true
max_length: 4096
rate_limit: 60/min
output:
toxicity_threshold: 0.3
blocklist_enabled: true
sensitive_redaction: true
system_leak_detection: true
fallback:
input_blocked: "I cannot process this request."
output_blocked: "I cannot provide this information."
logging:
log_blocked: true
log_filtered: true
include_reason: false # Privacy
┌──────────────────┬─────────┬────────┬──────────┐
│ Metric │ Target │ Actual │ Status │
├──────────────────┼─────────┼────────┼──────────┤
│ Injection Block │ >95% │ 97% │ ✓ PASS │
│ False Positive │ <2% │ 1.5% │ ✓ PASS │
│ Latency Impact │ <50ms │ 35ms │ ✓ PASS │
│ Toxicity Block │ >90% │ 92% │ ✓ PASS │
│ PII Redaction │ >99% │ 99.5% │ ✓ PASS │
└──────────────────┴─────────┴────────┴──────────┘
Issue: High false positive rate
Solution: Tune patterns, add allowlist, use context
Issue: Latency too high
Solution: Optimize regex, use compiled patterns, cache
Issue: Bypassed by encoding
Solution: Normalize unicode, decode before checking
| Component | Purpose |
|---|---|
| Agent 05 | Implements guardrails |
| /defend | Configuration recommendations |
| CI/CD | Automated testing |
| Monitoring | Alert on filter triggers |
Protect AI systems with comprehensive input/output guardrails.
CI/CD integration and automation frameworks for continuous AI security testing
Standard datasets and benchmarks for evaluating AI security, robustness, and safety
Professional certifications, CTF competitions, and training resources for AI security practitioners
SOC 職業分類に基づく