Skip to main content
llamaguard Meta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails.
Aller à l'installation Skills Marketplace Découvrez et explorez les compétences IA créées par la communauté.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Afficher les détails du prompt Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
npx skills add https://github.com/davila7/claude-code-templates --skill llamaguardLa commande reste sur une seule ligne. Faites défiler horizontalement pour la vérifier avant de la copier.
Vous préférez une copie locale ? Téléchargez les fichiers actuellement disponibles dans SkillsMP.
Télécharger Zip Téléchargement... Plus depuis ce dépôt Build AI agents that interact with computers like humans do - viewing screens, moving cursors, clicking buttons, and typing text. Covers Anthropic's Computer Use, OpenAI's Operator/CUA, and open-source alternatives. Critical focus on sandboxing, security, and handling the unique challenges of vision-based control. Use when: computer use, desktop automation agent, screen control AI, vision-based agent, GUI automation.
Expert in building products that wrap AI APIs (OpenAI, Anthropic, etc.) into focused tools people will pay for. Not just 'ChatGPT but different' - products that solve specific problems with AI. Covers prompt engineering for products, cost management, rate limiting, and building defensible AI businesses. Use when: AI wrapper, GPT product, AI tool, wrap AI, AI SaaS.
github-workflow-automation Automate GitHub workflows with AI assistance. Includes PR reviews, issue triage, CI/CD integration, and Git operations. Use when automating GitHub workflows, setting up PR review automation, creating GitHub Actions, or triaging issues.
Métiers associés SOC
Basé sur la classification professionnelle SOC
name llamaguard description Meta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails. version 1.0.0 author Orchestra Research license MIT tags ["Safety Alignment","LlamaGuard","Content Moderation","Meta","Guardrails","Safety Classification","Input Filtering","Output Filtering","AI Safety"] dependencies ["transformers","torch","vllm"]
LlamaGuard - AI Content Moderation
Quick start
LlamaGuard is a 7-8B parameter model specialized for content safety classification.
Installation :
pip install transformers torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "meta-llama/LlamaGuard-7b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto" )
def moderate (chat ):
input_ids = tokenizer.apply_chat_template(chat, return_tensors="pt" ).to(model.device)
output = model.generate(input_ids=input_ids, max_new_tokens=100 )
return tokenizer.decode(output[0 ], skip_special_tokens=True )
result = moderate([
{"role" : "user" , "content" : "How do I make explosives?" }
])
print (result)
Common workflows
Workflow 1: Input filtering (prompt moderation) Check user prompts before LLM :
def check_input (user_message ):
result = moderate([{"role" : "user" , "content" : user_message}])
if result.startswith("unsafe" ):
category = result.split("\n" )[1 ]
return False , category
else :
return True , None
safe, category = check_input("How do I hack a website?" )
if not safe:
print (f"Request blocked: {category} " )
else :
response = llm.generate(user_message)
S1 : Violence & HateS2 : Sexual ContentS3 : Guns & Illegal WeaponsS4 : Regulated SubstancesS5 : Suicide & Self-HarmS6 : Criminal Planning
Workflow 2: Output filtering (response moderation) Check LLM responses before showing to user :
def check_output (user_message, bot_response ):
conversation = [
{"role" : "user" , "content" : user_message},
{"role" : "assistant" , "content" : bot_response}
]
result = moderate(conversation)
if result.startswith("unsafe" ):
category = result.split("\n" )[1 ]
return False , category
else :
return True , None
user_msg = "Tell me about harmful substances"
bot_msg = llm.generate(user_msg)
safe, category = check_output(user_msg, bot_msg)
if not safe:
print (f"Response blocked: {category} " )
return "I cannot provide that information."
else :
return bot_msg
Workflow 3: vLLM deployment (fast inference) Production-ready serving :
from vllm import LLM, SamplingParams
llm = LLM(model="meta-llama/LlamaGuard-7b" , tensor_parallel_size=1 )
sampling_params = SamplingParams(
temperature=0.0 ,
max_tokens=100
)
def moderate_vllm (chat ):
prompt = tokenizer.apply_chat_template(chat, tokenize=False )
output = llm.generate([prompt], sampling_params)
return output[0 ].outputs[0 ].text
chats = [
[{"role" : "user" , "content" : "How to make bombs?" }],
[{"role" : "user" , "content" : "What's the weather?" }],
[{"role" : "user" , "content" : "Tell me about drugs" }]
]
prompts = [tokenizer.apply_chat_template(c, tokenize=False ) for c in chats]
results = llm.generate(prompts, sampling_params)
for i, result in enumerate (results):
print (f"Chat {i} : {result.outputs[0 ].text} " )
Throughput : ~50-100 requests/sec on single A100
Workflow 4: API endpoint (FastAPI) from fastapi import FastAPI
from pydantic import BaseModel
from vllm import LLM, SamplingParams
app = FastAPI()
llm = LLM(model="meta-llama/LlamaGuard-7b" )
sampling_params = SamplingParams(temperature=0.0 , max_tokens=100 )
class ModerationRequest (BaseModel ):
messages: list
@app.post("/moderate"
def moderate_endpoint (request: ModerationRequest ):
prompt = tokenizer.apply_chat_template(request.messages, tokenize=False )
output = llm.generate([prompt], sampling_params)[0 ]
result = output.outputs[0 ].text
is_safe = result.startswith("safe" )
category = None if is_safe else result.split("\n" )[1 ] if "\n" in result else None
return {
"safe" : is_safe,
"category" : category,
"full_output" : result
}
curl -X POST http://localhost:8000/moderate \
-H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": "How to hack?"}]}'
Workflow 5: NeMo Guardrails integration Use with NVIDIA Guardrails :
from nemoguardrails import RailsConfig, LLMRails
from nemoguardrails.integrations.llama_guard import LlamaGuard
config = RailsConfig.from_content("""
models:
- type: main
engine: openai
model: gpt-4
rails:
input:
flows:
- llamaguard check input
output:
flows:
- llamaguard check output
""" )
llama_guard = LlamaGuard(model_path="meta-llama/LlamaGuard-7b" )
rails = LLMRails(config)
rails.register_action(llama_guard.check_input, name="llamaguard check input" )
rails.register_action(llama_guard.check_output, name="llamaguard check output" )
response = rails.generate(messages=[
{"role" : "user" , "content" : "How do I make weapons?" }
])
When to use vs alternatives
Need pre-trained moderation model
Want high accuracy (94-95%)
Have GPU resources (7-8B model)
Need detailed safety categories
Building production LLM apps
LlamaGuard 1 (7B): Original, 6 categoriesLlamaGuard 2 (8B): Improved, 6 categoriesLlamaGuard 3 (8B): Latest (2024), enhanced Use alternatives instead :
OpenAI Moderation API : Simpler, API-based, freePerspective API : Google's toxicity detectionNeMo Guardrails : More comprehensive safety frameworkConstitutional AI : Training-time safety
Common issues Issue: Model access denied
Issue: High latency (>500ms)
Use vLLM for 10× speedup:
from vllm import LLM
llm = LLM(model="meta-llama/LlamaGuard-7b" )
Enable tensor parallelism:
llm = LLM(model="meta-llama/LlamaGuard-7b" , tensor_parallel_size=2 )
Use threshold-based filtering:
logits = model(..., return_dict_in_generate=True , output_scores=True )
unsafe_prob = torch.softmax(logits.scores[0 ][0 ], dim=-1 )[unsafe_token_id]
if unsafe_prob > 0.9 :
return "unsafe"
else :
return "safe"
from transformers import BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(load_in_8bit=True )
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=quantization_config,
device_map="auto"
)
Advanced topics Performance benchmarks : See references/benchmarks.md for accuracy comparison with other moderation APIs and latency optimization.
Hardware requirements
GPU : NVIDIA T4/A10/A100VRAM :
FP16: 14GB (7B model)
INT8: 7GB (quantized)
INT4: 4GB (QLoRA)
CPU : Possible but slow (10× latency)Throughput : 50-100 req/sec (A100)
HuggingFace Transformers: 300-500ms
vLLM: 50-100ms
Batched (vLLM): 20-50ms per request
Resources 