| name | hestiacp-devops |
| description | Autonomous HestiaCP server management with AI agents. Two-brain Orchestrator+Sentinel architecture for VPS monitoring, troubleshooting, security hardening, PHP-FPM tuning, Exim mail forensics, and post-compromise detection. Use when: HestiaCP, VPS, server monitoring, DevOps agent, Linux admin, Nginx/Apache, PHP-FPM, Exim, Fail2Ban, ClamAV, SSL, backup. |
HestiaCP DevOps AI Skill
Source: webxtekstudio/hestiacp-useful-tools (GitHub, MIT License)
Architecture: Two-brain Orchestrator + Sentinel pattern
Platform: HestiaCP on Debian 12 (Bookworm)
Blueprint: See hestiacp_two_brain_blueprint.md in session artifacts
When to Use
- Server health monitoring and automated reporting
- HestiaCP CLI operations (
v-* commands)
- PHP-FPM pool tuning and troubleshooting (502/504 errors)
- Exim4 mail queue forensics and spam mitigation
- Fail2Ban management and attack response
- SSL certificate management (Let's Encrypt)
- Post-compromise detection (rogue ports, temp executables, webshells)
- Backup verification and disaster recovery
- Nginx/Apache reverse proxy debugging
- CMS permission hardening (WordPress, Laravel, Drupal, Joomla, Magento)
Architecture: Two-Brain Model
Brain 1: Sentinel (Passive Monitor)
- File:
prompts/System-Monitor-Prompt.md
- Input: CRON-injected telemetry (RAM, CPU, Disk, Services, Logs)
- Output: Structured health report (HEALTHY ✅ / WARNING ⚠️ / CRITICAL 🔴)
- Tool:
run_ssh_command (Round 3 deep dives only)
- Key Innovation: Post-compromise detection, attack-is-root-cause principle
Brain 2: Orchestrator (Active DevOps)
- File:
prompts/DevOps-Agent-Prompt.md
- Input: User request or Sentinel alert trigger
- Output: Diagnosis + remediation + verification report
- Tools:
run_ssh_command + ask_knowledge_expert (RAG)
- Constraint: Max 10 ReAct steps
Decision Heuristics (Mandatory)
1. LOOK BEFORE LEAP — cp config.conf config.conf.bak
2. LEAST DESTRUCTIVE — nginx -t BEFORE systemctl restart
3. SILENT EXECUTION — Act safely, report async (no permission prompts in automation)
4. FALSIFIABILITY TEST — After "fixing", run the command that PROVES it's fixed
Noise Filter (False Positive Suppression)
Ignore these — they are NORMAL in HestiaCP:
cloud-init, cloud-config, cloud-final service failures- Apache on port 8080 (backend behind Nginx — this IS the architecture)
- SSH brute-force in auth logs (Fail2Ban handles it)
- High memory if mostly buffers/cache
- Zombie count ≤ 5
- Load spikes during
v-backup-users
- Transient PHP-FPM CPU spikes during backup windows
Severity Classification
| Level | Trigger |
|---|
| ✅ HEALTHY | Zero thresholds breached, or only marginal |
| ⚠️ WARNING | Clear threshold breach with operational concern |
| 🔴 CRITICAL | Service down, disk >90%, backup FAILED, active attack, OOM kills, compromise indicators |
Attack Principle: When PHP-FPM CPU is high AND attack vectors are present, the ROOT CAUSE is the attack — not PHP-FPM. Report as attack, not "busy workload."
Critical Thresholds
| Resource | Warning | Critical |
|---|
Disk (/ or /home) | >80% | >90% |
| Load (1min) | > nproc | > nproc × 2 |
| RAM (excl. cache) | >85% | — |
| Swap | >1GB | >2GB (OOM risk) |
| Mail Queue (Exim) | >10 messages | >50 messages |
| Backup Age | >8 days | STATUS=FAILED |
| SSL Expiry | <14 days | <7 days |
Post-Compromise Detection (6 Vectors)
| Vector | Check | Critical |
|---|
| Rogue Ports | ss -tlnp vs expected | 4444, 5555, 6666, 8888 → 🔴 |
| Temp Executables | /tmp, /var/tmp, /dev/shm | ANY executable → 🔴 |
| User Audit | UID ≥ 1000 with /bin/bash | Unknown user → 🔴 |
| Recent PHP Mods | Files modified <6hrs | shell.php, c99.php → 🔴 |
| Outbound C2 | ss -tnp state established | IRC 6667, crypto 3333 → 🔴 |
| Suspicious Cron | curl, wget, base64, /tmp/ | 🔴 possible backdoor |
Service Name Gotchas
| Service | Correct | WRONG |
|---|
| Mail | exim4 | exim |
| Database | mariadb | mysql |
| PHP | php8.2-fpm | php-fpm |
| DB CLI | mariadb | mysql |
PHP-FPM Tuning Formula
Max Total Workers = (Total RAM - 2GB) / 60MB
Low Traffic: 2-5 workers
Medium Traffic: 10-20 workers
High Traffic: 30-50 workers
Exim Log Decoder
<= Email arrived at server
=> Normal delivery (local or external)
-> Forwarded or aliased delivery
** Delivery permanently FAILED (bounced)
== Delivery temporarily DELAYED (queued)
CRITICAL: mainlog does NOT store email subjects. To get subjects, grep the physical Maildir:
/home/<user>/mail/<domain>/<account>/new/ (unread)
/home/<user>/mail/<domain>/<account>/cur/ (read)
False Path Map (Never Use These)
| Generic Path | HestiaCP Reality |
|---|
/etc/letsencrypt/live | /home/[USER]/conf/web/[DOMAIN]/ssl/ |
/etc/nginx/sites-enabled | /etc/nginx/conf.d/domains/ |
/var/www/html | /home/[USER]/web/[DOMAIN]/public_html/ |
Knowledge Base (16 files in knowledge/)
| # | File | Size | Domain |
|---|
| 01 | hestia-system-paths.md | 11.5KB | Complete path map |
| 02 | hestia-troubleshooting-logs.md | 5.7KB | Diagnostic commands |
| 03 | hestia-cron-guide.md | 2.7KB | CRON management |
| 04 | hestia-php-tuning.md | 4.2KB | PHP-FPM optimization |
| 05 | hestia-database-guide.md | 2.6KB | MariaDB config |
| 06 | hestia-fail2ban-guide.md | 2.0KB | Jail management |
| 07 | hestia-ssl-guide.md | 2.2KB | SSL lifecycle |
| 08 | hestia-clamav-guide.md | 2.1KB | Antivirus config |
| 09 | hestia-exim-troubleshooting.md | 5.0KB | Mail queue forensics |
| 10 | hestia-nginx-templates.md | 1.8KB | Proxy templates |
| 11 | hestia-backups.md | 3.1KB | Backup/restore |
| 12 | hestia-cli-reference.md | 23.2KB | 400+ v-commands |
| 13 | hestia-common-issues.md | 3.1KB | Decision trees |
| 14 | hestia-dovecot-guide.md | 4.4KB | IMAP/POP3 debug |
| 15 | exim-forensics.md | 2.8KB | Maildir forensics |
| 16 | hestia-custom-tools-extended.md | 6.3KB | v-fix-web-permissions, v-security-audit |
Integration
- Skill Router: Category 4 (Deploy & Ops)
- Related Skills:
@gcp-cloud-run, @docker-expert, @security-auditor, @systematic-debugging
- MRL Index: Rebuild
mrl_index.pkl to include these 19 files for semantic search
