Secrets Detection with Gitleaks
Overview
Gitleaks is a secret detection tool that scans git repositories, files, and directories for hardcoded credentials including passwords, API keys, tokens, and other sensitive information. It uses regex-based pattern matching combined with Shannon entropy analysis to identify secrets that could lead to unauthorized access if exposed.
This skill provides comprehensive guidance for integrating Gitleaks into DevSecOps workflows, from pre-commit hooks to CI/CD pipelines, with emphasis on preventing secret leakage before code reaches production.
Quick Start
Scan current repository for secrets:
brew install gitleaks
gitleaks detect -v
gitleaks detect --source /path/to/code -v
gitleaks detect --report-path gitleaks-report.json --report-format json
Core Workflows
1. Repository Scanning
Scan existing repositories to identify exposed secrets:
gitleaks detect -v --source /path/to/repo
gitleaks detect --config .gitleaks.toml -v
gitleaks detect --report-path findings.json --report-format json
gitleaks detect --report-path findings.sarif --report-format sarif
When to use: Initial security audit, compliance checks, incident response.
2. Pre-Commit Hook Protection
Prevent secrets from being committed in the first place:
cat << 'EOF' > .git/hooks/pre-commit
gitleaks protect --verbose --redact --staged
EOF
chmod +x .git/hooks/pre-commit
Use the bundled script for automated hook installation:
./scripts/install_precommit.sh
When to use: Developer workstation setup, team onboarding, mandatory security controls.
3. CI/CD Pipeline Integration
GitHub Actions
name: gitleaks
on: [push, pull_request]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GitLab CI
gitleaks:
image: zricethezav/gitleaks:latest
stage: test
script:
- gitleaks detect --report-path gitleaks.json --report-format json --verbose
artifacts:
paths:
- gitleaks.json
when: always
allow_failure: false
When to use: Automated security gates, pull request checks, release validation.
4. Baseline and Incremental Scanning
Establish security baseline and track only new secrets:
gitleaks detect --report-path baseline.json --report-format json
gitleaks detect --baseline-path baseline.json --report-path new-findings.json -v
When to use: Legacy codebase remediation, phased rollout, compliance tracking.
5. Configuration Customization
Create custom .gitleaks.toml configuration:
title = "Custom Gitleaks Configuration"
[extend]
useDefault = true
[[rules]]
id = "custom-api-key"
description = "Custom API Key Pattern"
regex = '''(?i)(custom_api_key|custom_secret)[\s]*[=:][\s]*['"][a-zA-Z0-9]{32,}['"]'''
tags = ["api-key", "custom"]
[allowlist]
description = "Global allowlist"
paths = [
'''\.md$''',
'''test/fixtures/''',
]
stopwords = [
'''EXAMPLE''',
'''PLACEHOLDER''',
]
Use bundled configuration templates in assets/:
assets/config-strict.toml - Strict detection (low false negatives)assets/config-balanced.toml - Balanced detection (recommended)assets/config-custom.toml - Template for custom rules
When to use: Reducing false positives, adding proprietary secret patterns, organizational standards.
Security Considerations
Sensitive Data Handling
- Secret Redaction: Always use
--redact flag in logs and reports to prevent accidental secret exposure
- Report Security: Gitleaks reports contain detected secrets - treat as confidential, encrypt at rest
- Git History: Detected secrets in git history require complete removal using tools like
git filter-repo or BFG Repo-Cleaner
- Credential Rotation: All exposed secrets must be rotated immediately, even if removed from code
Access Control
- CI/CD Permissions: Gitleaks scans require read access to repository content and git history
- Report Access: Restrict access to scan reports containing sensitive findings
- Baseline Files: Baseline JSON files contain secret metadata - protect with same controls as findings
Audit Logging
Log the following for compliance and incident response:
- Scan execution timestamps and scope (repository, branch, commit range)
- Number and types of secrets detected
- Remediation actions taken (credential rotation, commit history cleanup)
- False positive classifications and allowlist updates
Compliance Requirements
- PCI-DSS 3.2.1: Requirement 6.5.3 - Prevent hardcoded credentials in payment applications
- SOC2: CC6.1 - Logical access controls prevent unauthorized credential exposure
- GDPR: Article 32 - Appropriate security measures for processing personal data credentials
- CWE-798: Use of Hard-coded Credentials
- CWE-259: Use of Hard-coded Password
- OWASP A07:2021: Identification and Authentication Failures
Bundled Resources
Scripts (scripts/)
install_precommit.sh - Automated pre-commit hook installation with configuration promptsscan_and_report.py - Comprehensive scanning with multiple output formats and severity classificationbaseline_manager.py - Baseline creation, comparison, and incremental scan management
References (references/)
detection_rules.md - Comprehensive list of built-in Gitleaks detection rules with CWE mappingsremediation_guide.md - Step-by-step secret remediation procedures including git history cleanupfalse_positives.md - Common false positive patterns and allowlist configuration strategiescompliance_mapping.md - Detailed mapping to PCI-DSS, SOC2, GDPR, and OWASP requirements
Assets (assets/)
config-strict.toml - High-sensitivity configuration (maximum detection)config-balanced.toml - Production-ready balanced configurationconfig-custom.toml - Template with inline documentation for custom rulesprecommit-config.yaml - Pre-commit framework configurationgithub-action.yml - Complete GitHub Actions workflow templategitlab-ci.yml - Complete GitLab CI pipeline template
Common Patterns
Pattern 1: Initial Repository Audit
First-time secret scanning for security assessment:
git clone --mirror https://github.com/org/repo.git audit-repo
cd audit-repo
gitleaks detect --report-path audit-report.json --report-format json -v
./scripts/scan_and_report.py --input audit-report.json --format markdown --output audit-report.md
cp audit-report.json baseline.json
Pattern 2: Developer Workstation Setup
Protect developers from accidental secret commits:
brew install gitleaks
./scripts/install_precommit.sh
echo "api_key = 'EXAMPLE_KEY_12345'" > test.txt
git add test.txt
git commit -m "test"
git reset HEAD~1
rm test.txt
Pattern 3: CI/CD Pipeline with Baseline
Progressive secret detection in continuous integration:
if [ -f ".gitleaks-baseline.json" ]; then
gitleaks detect \
--baseline-path .gitleaks-baseline.json \
--report-path new-findings.json \
--report-format json \
--exit-code 1
else
gitleaks detect \
--report-path .gitleaks-baseline.json \
--report-format json \
--exit-code 0
fi
if [ -f "new-findings.json" ] && [ -s "new-findings.json" ]; then
gitleaks detect \
--baseline-path .gitleaks-baseline.json \
--report-path results.sarif \
--report-format sarif
fi
Pattern 4: Custom Rule Development
Add organization-specific secret patterns:
[[rules]]
id = "acme-corp-api-key"
description = "ACME Corp Internal API Key"
regex = '''(?i)acme[_-]?api[_-]?key[\s]*[=:][\s]*['"]?([a-f0-9]{40})['"]?'''
secretGroup = 1
tags = ["api-key", "acme-internal"]
[[rules]]
id = "acme-corp-database-password"
description = "ACME Corp Database Password Format"
regex = '''(?i)(db_pass|database_password)[\s]*[=:][\s]*['"]([A-Z][a-z0-9@#$%]{15,})['"]'''
secretGroup = 2
tags = ["password", "database", "acme-internal"]
Integration Points
CI/CD Integration
- GitHub Actions: Use
gitleaks/gitleaks-action@v2 for native integration with Security tab
- GitLab CI: Docker-based scanning with artifact retention for audit trails
- Jenkins: Execute via Docker or installed binary in pipeline stages
- CircleCI: Docker executor with orb support
- Azure Pipelines: Task-based integration with results publishing
Security Tools Ecosystem
- SIEM Integration: Export JSON findings to Splunk, ELK, or Datadog for centralized monitoring
- Vulnerability Management: Import SARIF reports into Snyk, SonarQube, or Checkmarx
- Secret Management: Integrate findings with HashiCorp Vault or AWS Secrets Manager rotation workflows
- Ticketing Systems: Automated Jira/ServiceNow ticket creation for remediation tracking
SDLC Integration
- Design Phase: Include secret detection requirements in security architecture reviews
- Development: Pre-commit hooks provide immediate feedback to developers
- Code Review: PR/MR checks prevent secrets from reaching main branches
- Testing: Scan test environments and infrastructure-as-code
- Deployment: Final validation gate before production release
- Operations: Periodic scanning of deployed configurations and logs
Troubleshooting
Issue: Too Many False Positives
Symptoms: Legitimate code patterns flagged as secrets (test fixtures, examples, placeholders)
Solution:
- Review findings to identify patterns:
grep -i "example\|test\|placeholder" gitleaks-report.json
- Add to allowlist in
.gitleaks.toml:
[allowlist]
paths = ['''test/''', '''examples/''', '''\.md$''']
stopwords = ["EXAMPLE", "PLACEHOLDER", "YOUR_API_KEY_HERE"]
- Use commit allowlists for specific false positives:
[allowlist]
commits = ["commit-sha-here"]
- Consult
references/false_positives.md for common patterns
Issue: Performance Issues on Large Repositories
Symptoms: Scans taking excessive time (>10 minutes), high memory usage
Solution:
- Use
--log-opts to limit git history: gitleaks detect --log-opts="--since=2024-01-01"
- Scan specific branches:
gitleaks detect --log-opts="origin/main"
- Use baseline approach to scan only recent changes
- Consider shallow clone for initial scans:
git clone --depth=1000
- Parallelize scans across multiple branches or subdirectories
Issue: Pre-commit Hook Blocking Valid Commits
Symptoms: Developers unable to commit code with legitimate patterns
Solution:
- Add inline comment to bypass hook:
# gitleaks:allow
- Update
.gitleaks.toml allowlist for the specific pattern
- Use
--redact to safely review findings: gitleaks protect --staged --redact
- Temporary bypass (use with caution):
git commit --no-verify
- Review with security team if pattern is genuinely needed
Issue: Secrets Found in Git History
Symptoms: Secrets detected in old commits, already removed from current code
Solution:
- Rotate compromised credentials immediately (highest priority)
- For public repositories, consider full history rewrite using:
git filter-repo (recommended): git filter-repo --path-glob '*.env' --invert-paths- BFG Repo-Cleaner:
bfg --delete-files credentials.json
- Force-push cleaned history:
git push --force
- Notify all contributors to rebase/re-clone
- See
references/remediation_guide.md for detailed procedures
- Document incident in security audit log
Issue: Custom Secret Patterns Not Detected
Symptoms: Organization-specific secrets not caught by default rules
Solution:
- Develop regex pattern: Test at regex101.com with sample secrets
- Add custom rule to
.gitleaks.toml:
[[rules]]
id = "custom-secret-id"
description = "Description"
regex = '''your-pattern-here'''
secretGroup = 1
- Test pattern:
gitleaks detect --config .gitleaks.toml -v --no-git
- Consider entropy threshold if pattern is ambiguous:
[[rules.Entropies]]
Min = "3.5"
Max = "7.0"
Group = "1"
- Validate with known true positives and negatives
Advanced Configuration
Entropy-Based Detection
For secrets without clear patterns, use Shannon entropy analysis:
[[rules]]
id = "high-entropy-strings"
description = "High entropy strings that may be secrets"
regex = '''[a-zA-Z0-9]{32,}'''
entropy = 4.5
secretGroup = 0
Composite Rules (v8.28.0+)
Detect secrets spanning multiple lines or requiring context:
[[rules]]
id = "multi-line-secret"
description = "API key with usage context"
regex = '''api_key[\s]*='''
[[rules.composite]]
pattern = '''initialize_client'''
location = "line"
distance = 5
Global vs Rule-Specific Allowlists
[allowlist]
description = "Organization-wide exceptions"
paths = ['''vendor/''', '''node_modules/''']
[[rules]]
id = "generic-api-key"
[rules.allowlist]
description = "Exceptions only for this rule"
regexes = ['''key\s*=\s*EXAMPLE''']
References
