Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Instruções da origem · Visualização somente leitura
name
ai-scanner-garak
description
AI model safety scanner built on NVIDIA garak for testing LLMs against 179 security probes across 35 vulnerability families
triggers
["scan an AI model for vulnerabilities","test LLM security with garak","run AI safety assessment","set up ai-scanner for penetration testing","configure AI model security scanning","check LLM for OWASP top 10 vulnerabilities","schedule recurring AI security scans","export AI scan results to PDF or SIEM"]
AI Scanner is an open-source Ruby on Rails web application for AI model security assessments, wrapping NVIDIA garak with a multi-tenant UI, scheduling, PDF reports, and SIEM integration. It runs 179 community probes across 35 vulnerability families aligned with the OWASP LLM Top 10.
Probe Families (35 total, aligned to OWASP LLM Top 10)
Key probe families available:
# List all available probe familiesGarak::ProbeRegistry.families
# => ["prompt_injection", "jailbreak", "data_leakage", "insecure_output",# "supply_chain", "sensitive_info", "excessive_agency", "overreliance",# "model_theft", "malicious_plugins", ...]# Get probes within a familyGarak::ProbeRegistry.probes_for("prompt_injection")
# => 179 total probes across all families
SiemIntegration.configure do |config|
config.provider = :rsyslog
config.rsyslog_host = ENV["RSYSLOG_HOST"]
config.rsyslog_port = ENV["RSYSLOG_PORT"].to_i
end
Multi-Tenant Organization Management
# Create a new organization
org = Organization.create!(name:"Security Team Alpha")
# Invite a user
user = User.invite!(
email:"analyst@example.com",
organization: org,
role:"analyst"# roles: "admin", "analyst", "viewer"
)
# Data is encrypted at rest per organization
org.encryption_key # => managed automatically
# Full test suite
bundle exec rspec
# Specific area
bundle exec rspec spec/models/scan_spec.rb
bundle exec rspec spec/jobs/scan_job_spec.rb
# Lint
bundle exec rubocop
Common Patterns
Testing a New LLM Before Deployment
# Comprehensive pre-deployment scan
target = Target.create!(
name:"New Model v2 - Pre-deploy",
target_type:"api",
api_endpoint:ENV["NEW_MODEL_ENDPOINT"],
api_key:ENV["NEW_MODEL_API_KEY"],
model_name:"new-model-v2",
organization: current_organization
)
# Run all 35 probe families
scan = Scan.create!(
target: target,
probe_families:Garak::ProbeRegistry.families,
organization: current_organization
)
ScanJob.perform_now(scan.id)
if scan.reload.asr_score > 0.15
puts "WARNING: ASR #{scan.asr_score} exceeds threshold. Review before deploying."else
puts "PASS: Model meets security threshold."end
Using the Mock LLM for Testing Scanner Setup
The built-in Mock LLM lets you validate your scanner configuration without hitting real APIs:
target = Target.create!(
name:"Mock LLM",
target_type:"mock",
organization: current_organization
)
# Run a quick scan to verify everything works end-to-end
Webhook on Scan Completion
# config/initializers/scan_hooks.rbActiveSupport::Notifications.subscribe("scan.completed") do |_, _, _, _, payload|
scan = Scan.find(payload[:scan_id])
if scan.asr_score > 0.20SlackNotifier.alert(
channel:"#security-alerts",
message:"High ASR detected: #{scan.asr_score} on #{scan.target.name}"
)
endend
Docker Compose Production Tips
# docker-compose.override.yml — production additionsservices:web:environment:RAILS_ENV:productionFORCE_SSL:"true"labels:-"traefik.enable=true"-"traefik.http.routers.scanner.rule=Host(`scanner.example.com`)"-"traefik.http.routers.scanner.tls.certresolver=letsencrypt"
# Upgrade
docker compose pull
docker compose up -d
docker compose exec web rails db:migrate
Troubleshooting
Problem
Solution
Scan stuck in "running"
Check docker compose logs worker — garak Python process may have crashed
SECRET_KEY_BASE error on start
Run openssl rand -hex 64 and set in .env
Can't connect to target API
Verify API key env var is set; check firewall allows outbound from container
Browser target scan fails
Ensure Playwright/Chrome is available in the worker container
PDF export blank
Check wkhtmltopdf is installed in the web container
SIEM not receiving events
Verify SPLUNK_HEC_URL includes full path /services/collector
# View all service logs
docker compose logs -f
# Check worker specifically (runs garak)
docker compose logs -f worker
# Rails console for debugging
docker compose exec web rails console
# Check garak is working
docker compose exec worker python -c "import garak; print(garak.__version__)"
Key Files (for Contributors)
app/
models/
scan.rb # Core scan model, ASR calculation
target.rb # Target types and validation
probe_result.rb # Per-probe result storage
jobs/
scan_job.rb # Async job that invokes garak
services/
garak_runner.rb # Ruby wrapper around garak CLI
report_exporter.rb
siem_integration.rb
lib/
garak/
probe_registry.rb # 179 probes, 35 families
dist/
docker-compose.yml # Production compose file
scripts/
install.sh # One-line installer
