| name | afrexai-self-hosting-mastery |
| description | Complete self-hosting and homelab operating system. Deploy, secure, monitor, and maintain self-hosted services with production-grade reliability. Use when setting up home servers, Docker infrastructure, reverse proxies, backups, monitoring, or evaluating self-hosted alternatives to SaaS. |
Self-Hosting Mastery
Complete system for building and operating reliable self-hosted infrastructure — from first server to multi-node homelab.
Phase 1: Infrastructure Assessment
Server Profile YAML
server_profile:
name: ""
hardware:
cpu: ""
ram_gb: 0
storage:
- device: ""
type: ""
size_gb: 0
role: ""
network: ""
os: ""
location: ""
power:
ups: false
wattage_idle: 0
wattage_load: 0
monthly_cost_estimate: ""
network:
public_ip: ""
domain: ""
dns_provider: ""
isp_ports_open: true
goals:
- ""
budget_monthly: ""
Hardware Decision Matrix
| Budget | RAM | Storage | Good For | Example Hardware |
|---|
| $0 | 4-8GB | 64GB+ | Pi-hole, AdGuard, small tools | Raspberry Pi 4/5 |
| $50-150 | 8-16GB | 256GB+ | Docker host, 5-10 services | Used SFF PC (Dell Optiplex, Lenovo Tiny) |
| $150-400 | 16-32GB | 1TB+ | NAS + services, media server | Mini PC (Intel NUC, Beelink) |
| $400-800 | 32-64GB | 4TB+ | Full homelab, VMs + containers | Used enterprise (Dell R720, HP DL380) |
| $800+ | 64GB+ | 10TB+ | Multi-node, Proxmox cluster | Multiple nodes, dedicated NAS |
Self-Host vs SaaS Decision
Ask before self-hosting anything:
- Data sensitivity — Does keeping data local matter? (passwords, health, finance = yes)
- Reliability need — Can you tolerate occasional downtime? (email = risky, media = fine)
- Maintenance budget — Do you have 2-4 hours/month for updates?
- Skill level — Can you debug Docker/networking issues?
- Cost comparison — Is the SaaS < $10/mo? Often not worth self-hosting for trivial savings.
Always self-host: Password manager, DNS/ad-blocking, VPN, bookmarks, notes
Usually self-host: Media server, file sync, photo backup, monitoring, git
Think twice: Email (deliverability hell), calendar (sync complexity), chat (uptime expectations)
Rarely worth it: Search engine (resource hungry), social media (no network effect)
Phase 2: OS & Virtualization
OS Selection Guide
| OS | Best For | Learning Curve | Notes |
|---|
| Debian 12 | Docker-only host | Low | Stable, minimal, just works |
| Ubuntu Server 24.04 | Beginners, wide docs | Low | More packages, snap controversy |
| Proxmox VE | VMs + containers | Medium | Free, enterprise features, ZFS |
| Unraid | NAS + Docker + VMs | Medium | $59-129, great UI, parity array |
| TrueNAS Scale | ZFS NAS + Docker | Medium | Free, ZFS-first, apps improving |
| NixOS | Reproducible configs | High | Declarative, steep learning curve |
Proxmox Quick Setup
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/pve-enterprise.list
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
apt update && apt upgrade -y
apt install -y curl
curl -fsSL https://get.docker.com | sh
VM vs LXC vs Docker Decision
| Factor | VM | LXC | Docker |
|---|
| Isolation | Full (own kernel) | Partial (shared kernel) | Process-level |
| Overhead | High (1-2GB base) | Low (50-200MB) | Minimal |
| Use when | Different OS, GPU passthrough, untrusted workloads | Dedicated service host, ZFS datasets | Most services |
| Avoid when | RAM-constrained | Need Windows, custom kernel | Stateful databases (use LXC/VM) |
Rule: Docker for 90% of services. LXC for Docker hosts or isolated environments. VM for Windows, different kernel needs, or GPU passthrough.
Phase 3: Docker Infrastructure
Docker Compose Project Structure
/opt/stacks/ # or ~/docker/
├── traefik/
│ ├── docker-compose.yml
│ ├── .env
│ ├── config/
│ │ └── traefik.yml
│ └── data/
│ ├── acme.json # chmod 600
│ └── dynamic/
├── monitoring/
│ ├── docker-compose.yml
│ ├── .env
│ └── config/
├── media/
│ ├── docker-compose.yml
│ ├── .env
│ └── config/
├── productivity/
│ ├── docker-compose.yml
│ ├── .env
│ └── config/
└── scripts/
├── backup.sh
├── update-all.sh
└── health-check.sh
Docker Compose Best Practices
services:
app:
image: vendor/app:1.2.3
container_name: app
restart: unless-stopped
networks:
- proxy
- internal
volumes:
- ./config:/config
- app-data:/data
environment:
- TZ=Europe/London
- PUID=1000
- PGID=1000
env_file:
- .env
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`app.example.com`)"
- "traefik.http.routers.app.tls.certresolver=letsencrypt"
- "traefik.http.services.app.loadbalancer.server.port=8080"
[, , , ]
Docker Security Checklist
Phase 4: Reverse Proxy & SSL
Reverse Proxy Selection
| Proxy | Best For | SSL | Config Style | Learning Curve |
|---|
| Traefik | Docker-native, auto-discovery | Auto (ACME) | Labels + YAML | Medium |
| Caddy | Simplicity, auto-SSL | Auto (built-in) | Caddyfile | Low |
| Nginx Proxy Manager | GUI preference | Auto (UI) | Web UI | Very Low |
| Nginx (manual) | Maximum control | Manual/certbot | Config files | High |
Recommendation: Traefik for Docker power users. Caddy for simplicity. NPM for beginners.
Traefik Production Config
api:
dashboard: true
insecure: false
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: ":443"
http:
tls:
certResolver: letsencrypt
certificatesResolvers:
letsencrypt:
acme:
email: you@example.com
storage: /data/acme.json
httpChallenge:
entryPoint: web
providers:
docker:
exposedByDefault: false
network: proxy
file:
directory: /data/dynamic
watch: true
log:
level: WARN
accessLog:
Cloudflare Tunnel (Zero Port Forwarding)
For CGNAT or ISPs blocking ports — expose services without opening firewall:
services:
cloudflared:
image: cloudflare/cloudflared:2024.1.0
container_name: cloudflared
restart: unless-stopped
command: tunnel run
environment:
- TUNNEL_TOKEN=${CF_TUNNEL_TOKEN}
networks:
- proxy
When to use Cloudflare Tunnel vs port forwarding:
- CGNAT (no public IP) → Tunnel (only option)
- ISP blocks 80/443 → Tunnel or DNS challenge + non-standard ports
- Security-first → Tunnel (no open ports)
- Performance-first → Direct (lower latency)
- LAN-only access → Neither (use Tailscale/WireGuard)
Phase 5: Essential Services Stack
Tier 1 — Deploy First (Foundation)
| Service | Purpose | Image | RAM | Notes |
|---|
| Traefik/Caddy | Reverse proxy + SSL | traefik:v3.0 | 64MB | Gateway to everything |
| Pi-hole/AdGuard | DNS + ad blocking | pihole/pihole | 128MB | Network-wide ad blocking |
| Authelia/Authentik | SSO + 2FA | authelia/authelia | 128MB | Protect services without built-in auth |
| Uptime Kuma | Monitoring | louislam/uptime-kuma | 128MB | Know when things break |
| Watchtower | Auto-updates | containrrr/watchtower | 32MB | Optional — some prefer manual |
Tier 2 — Core Services
| Service | Purpose | Alt | RAM |
|---|
| Vaultwarden | Password manager | Bitwarden | 64MB |
| Nextcloud | File sync + office | Seafile (lighter) | 512MB |
| Immich | Photo backup | PhotoPrism | 1-4GB |
| Jellyfin | Media server | Plex (less free) | 512MB-2GB |
| Paperless-ngx | Document management | - | 256MB |
| Home Assistant | Smart home | - | 512MB |
Tier 3 — Power User
| Service | Purpose | RAM |
|---|
| Gitea/Forgejo | Git hosting | 256MB |
| n8n | Workflow automation | 256MB |
| Grafana + Prometheus | Metrics & dashboards | 512MB |
| Tandoor | Recipe management | 256MB |
| Mealie | Meal planning | 128MB |
| Linkwarden/Hoarder | Bookmark manager | 256MB |
| Stirling PDF | PDF tools | 512MB |
| IT-Tools | Developer utilities | 64MB |
RAM Planning
Total RAM needed ≈ OS base (1-2GB) + sum of service RAM + 20% headroom
Example 16GB server:
OS + Docker: 2 GB
Traefik: 0.1 GB
Pi-hole: 0.1 GB
Authelia: 0.1 GB
Uptime Kuma: 0.1 GB
Vaultwarden: 0.1 GB
Nextcloud: 0.5 GB
Immich: 2.0 GB
Jellyfin: 1.0 GB
Paperless: 0.3 GB
Home Assistant: 0.5 GB
──────────────────────
Total: 6.8 GB → 8.2 GB with headroom
Available: ~7.8 GB free for more services
Phase 6: Networking & DNS
DNS Architecture
Internet → Cloudflare DNS → Your Public IP → Router → Server
↓
Reverse Proxy (Traefik)
↓
┌──────────────────┼──────────────────┐
↓ ↓ ↓
app.domain.com files.domain.com media.domain.com
Split DNS (Access Services Locally Without Hairpin NAT)
# Pi-hole/AdGuard: Local DNS rewrites
# Point *.home.example.com → 192.168.1.100 (server LAN IP)
# External: Cloudflare points to public IP
# Result: LAN traffic stays local, external goes through internet
VPN for Remote Access
| Solution | Type | Best For | Complexity |
|---|
| Tailscale | Mesh VPN | Easiest setup, multi-device | Very Low |
| WireGuard | Point-to-point | Performance, full control | Medium |
| Headscale | Self-hosted Tailscale | Privacy, no vendor lock | Medium-High |
Recommendation: Start with Tailscale (free for 3 users). Move to Headscale when you want full control.
Firewall Rules (UFW)
ufw default deny incoming
ufw default allow outgoing
ufw allow 2222/tcp comment 'SSH'
ufw allow 80/tcp comment 'HTTP redirect'
ufw allow 443/tcp comment 'HTTPS'
ufw allow from 192.168.1.0/24 comment 'LAN'
ufw enable
Phase 7: Backup Strategy
3-2-1 Rule Implementation
3 copies: Live data + Local backup + Remote backup
2 media: SSD/HDD (server) + External drive or NAS
1 offsite: Cloud (Backblaze B2, Wasabi) or second location
Backup Script Template
#!/bin/bash
set -euo pipefail
BACKUP_DIR="/mnt/backup/docker"
STACKS_DIR="/opt/stacks"
DATE=$(date +%Y-%m-%d_%H%M)
RETENTION_DAYS=30
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"; }
log "Stopping database services..."
cd "$STACKS_DIR/productivity" && docker compose stop db
log "Backing up volumes..."
for vol in $(docker volume ls -q); do
docker run --rm \
-v "$vol":/source:ro \
-v "$BACKUP_DIR/volumes":/backup \
alpine tar czf "/backup/${vol}_${DATE}.tar.gz" -C /source .
done
log "Backing up configs..."
tar czf "$BACKUP_DIR/configs/stacks_${DATE}.tar.gz" \
--exclude='*.log' \
--exclude='node_modules' \
"$STACKS_DIR"
log "Restarting services..."
cd "/productivity" && docker compose start db
find -name -mtime + -delete
BACKUP_SIZE=$( -sh | -f1)
Backup Schedule
| What | Frequency | Retention | Method |
|---|
| Docker volumes | Daily 3 AM | 30 days | Script + cron |
| Compose files + configs | Daily 3 AM | 90 days | Script + cron |
| Database dumps | Every 6 hours | 7 days | pg_dump/mysqldump |
| Full disk image | Monthly | 3 months | Clonezilla/dd |
| Offsite sync | Daily 5 AM | 60 days | rclone to B2/Wasabi |
Backup Verification (Monthly)
Phase 8: Monitoring & Alerting
Monitoring Stack (Docker Compose)
services:
uptime-kuma:
image: louislam/uptime-kuma:1
container_name: uptime-kuma
restart: unless-stopped
volumes:
- uptime-data:/app/data
labels:
- "traefik.enable=true"
- "traefik.http.routers.uptime.rule=Host(`status.example.com`)"
prometheus:
image: prom/prometheus:v2.49.0
container_name: prometheus
restart: unless-stopped
volumes:
- ./config/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- prometheus-data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.retention.time=30d'
grafana:
image: grafana/grafana:10.3.0
container_name: grafana
restart: unless-stopped
volumes:
- grafana-data:/var/lib/grafana
environment:
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
node-exporter:
image:
Alert Rules
| Metric | Warning | Critical | Action |
|---|
| Disk usage | >80% | >90% | Cleanup or expand |
| RAM usage | >85% | >95% | Identify memory leak, add RAM |
| CPU sustained | >80% 5min | >95% 5min | Check runaway process |
| Container restart | >2/hour | >5/hour | Check logs, fix root cause |
| SSL cert expiry | <14 days | <3 days | Renew cert |
| Backup age | >26 hours | >48 hours | Check backup script/cron |
| Service down | >2 min | >10 min | Investigate, restart |
Notification Channels
| Channel | Service | Best For |
|---|
| Push notification | ntfy.sh (self-hosted) | Mobile alerts |
| Chat | Discord/Slack webhook | Team alerts |
| Email | Uptime Kuma built-in | Formal notifications |
| Dashboard | Grafana + Uptime Kuma | Visual monitoring |
Phase 9: Security Hardening
Server Hardening Checklist
Port 2222
PermitRootLogin no
PasswordAuthentication no
MaxAuthTries 3
AllowUsers yourusername
apt install fail2ban -y
systemctl enable fail2ban
apt install unattended-upgrades -y
dpkg-reconfigure -plow unattended-upgrades
systemctl list-unit-files --state=enabled
Authentication Architecture
Internet → Traefik → Authelia/Authentik → Service
↓
Check: authenticated?
Yes → Forward to service
No → Redirect to login page + 2FA
Authelia (lightweight, YAML config) — good for smaller setups
Authentik (full IdP, web UI) — good for many users/services, SAML/OIDC
Security Scoring (0-100)
| Dimension | Weight | Score Guide |
|---|
| SSH hardened (keys, non-root, non-22) | 15 | 0=default, 15=fully hardened |
| Firewall active (deny-by-default) | 15 | 0=none, 15=UFW/iptables configured |
| Reverse proxy (no direct port exposure) | 15 | 0=ports exposed, 15=all behind proxy |
| SSL/TLS on all services | 10 | 0=HTTP, 10=HTTPS everywhere |
| Auth on all public services | 15 | 0=open, 15=SSO/2FA on everything |
| Container security (non-root, limits) | 10 | 0=default, 10=hardened |
| Auto-updates enabled | 10 | 0=manual, 10=automated |
| Secrets management (.env, not hardcoded) | 10 | 0=in compose, 10=.env + restricted perms |
Score: 0-40 = Vulnerable, 41-70 = Acceptable, 71-90 = Good, 91-100 = Hardened
Phase 10: Maintenance & Updates
Update Strategy
Option A: Manual (Recommended for critical services)
set -euo pipefail
STACKS_DIR="/opt/stacks"
LOG="/var/log/docker-updates.log"
for stack in "$STACKS_DIR"/*/; do
if [ -f "$stack/docker-compose.yml" ]; then
echo "[$(date)] Updating $(basename $stack)..." | tee -a "$LOG"
cd "$stack"
docker compose pull 2>&1 | tee -a "$LOG"
docker compose up -d 2>&1 | tee -a "$LOG"
fi
done
docker image prune -f | tee -a "$LOG"
echo "[$(date)] Update complete" | tee -a "$LOG"
Option B: Watchtower (Automated — use with caution)
services:
watchtower:
image: containrrr/watchtower:1.7.1
container_name: watchtower
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_SCHEDULE=0 0 4 * * MON
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_NOTIFICATIONS=shoutrrr
- WATCHTOWER_NOTIFICATION_URL=discord://webhook
- WATCHTOWER_LABEL_ENABLE=true
Weekly Maintenance Checklist
Monthly Maintenance
Phase 11: Advanced Patterns
Multi-Node Architecture
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Node 1 │ │ Node 2 │ │ Node 3 │
│ (Proxy/DNS) │────│ (Services) │────│ (NAS) │
│ Traefik │ │ Apps │ │ TrueNAS │
│ Pi-hole │ │ Databases │ │ NFS/SMB │
│ Authelia │ │ Media │ │ Backup │
└─────────────┘ └─────────────┘ └─────────────┘
↑ ↑ ↑
└───────── Tailscale Mesh ──────────────┘
Docker Compose Includes (Compose v2.20+)
include:
- path: ../common/traefik-labels.yml
- path: ../common/logging.yml
services:
app:
GitOps for Homelab
homelab-configs/ # Git repo
├── .github/
│ └── workflows/
│ └── deploy.yml # CI: lint + push to server
├── stacks/
│ ├── traefik/
│ ├── monitoring/
│ └── media/
├── scripts/
└── README.md
Workflow: Edit compose locally → commit → push → CI deploys to server
Tools: Flux/ArgoCD (overkill), or simple git pull && docker compose up -d via webhook
Hardware Redundancy
| Component | Solution | Cost |
|---|
| Power | UPS (APC Back-UPS 600VA+) | $60-150 |
| Storage | RAID1/ZFS mirror (not RAID0!) | 2x disk cost |
| Network | Dual NIC, managed switch | $30-100 |
| Server | Second node (cold spare or active) | $100-400 |
Rule: RAID is NOT backup. It protects against disk failure only, not ransomware/deletion/corruption.
Phase 12: Troubleshooting
Common Issues Decision Tree
Service not accessible?
├── Can you ping the server? → No → Network/firewall issue
├── Is the container running? (`docker ps`) → No → Check logs: `docker logs <name>`
├── Is the port exposed? (`docker port <name>`) → No → Check compose ports/networks
├── Is Traefik routing? (Check Traefik dashboard) → No → Check labels, network
├── Is DNS resolving? (`dig app.example.com`) → No → Check DNS provider
└── SSL error? → Check acme.json permissions (chmod 600), cert resolver logs
Docker Debug Commands
docker logs <name> --tail 50
docker inspect <name> | jq '.[0].State'
docker network ls
docker network inspect <network>
docker exec <name> ping other-container
docker stats
docker system df
docker volume ls -f dangling=true
docker compose down && docker compose up -d
docker system prune -af --volumes
Performance Optimization
| Symptom | Likely Cause | Fix |
|---|
| Slow file access | HDD for database | Move DB to SSD |
| High CPU idle | Monitoring too frequent | Increase scrape intervals |
| OOM kills | No memory limits | Set deploy.resources.limits.memory |
| Slow Nextcloud | Missing Redis cache | Add Redis container |
| Jellyfin buffering | No hardware transcoding | Enable GPU passthrough |
| Slow Docker builds | No layer caching | Use multi-stage + .dockerignore |
Service Configuration Quick Reference
Vaultwarden (Password Manager)
services:
vaultwarden:
image: vaultwarden/server:1.30.5
container_name: vaultwarden
restart: unless-stopped
volumes:
- vaultwarden-data:/data
environment:
- SIGNUPS_ALLOWED=false
- WEBSOCKET_ENABLED=true
- ADMIN_TOKEN=${ADMIN_TOKEN}
labels:
- "traefik.enable=true"
- "traefik.http.routers.vault.rule=Host(`vault.example.com`)"
Immich (Photo Backup)
Paperless-ngx (Document Management)
services:
paperless:
image: ghcr.io/paperless-ngx/paperless-ngx:2.4
container_name: paperless
restart: unless-stopped
volumes:
- paperless-data:/usr/src/paperless/data
- paperless-media:/usr/src/paperless/media
- ./consume:/usr/src/paperless/consume
- ./export:/usr/src/paperless/export
environment:
- PAPERLESS_OCR_LANGUAGE=eng
- PAPERLESS_TIME_ZONE=Europe/London
- PAPERLESS_ADMIN_USER=${ADMIN_USER}
- PAPERLESS_ADMIN_PASSWORD=${ADMIN_PASS}
Homelab Quality Rubric (0-100)
| Dimension | Weight | 0 (Poor) | 50 (Decent) | 100 (Excellent) |
|---|
| Security | 20% | Default passwords, open ports | Firewall + SSL | Hardened SSH, SSO/2FA, no-new-privileges |
| Backups | 20% | None | Local only, untested | 3-2-1, automated, verified monthly |
| Monitoring | 15% | None | Uptime Kuma only | Full stack: metrics + logs + alerts |
| Documentation | 10% | Nothing written | README per stack | GitOps, full runbook, diagrams |
| Updates | 10% | Never updated | Manual quarterly | Scheduled weekly, changelogs reviewed |
| Reliability | 10% | Frequent crashes | Mostly stable | UPS, auto-restart, health checks |
| Performance | 10% | Slow, OOM kills | Adequate | Resource limits, SSD, HW transcoding |
| Scalability | 5% | Single machine, no plan | Compose organized | Multi-node ready, IaC |
10 Self-Hosting Mistakes
| # | Mistake | Fix |
|---|
| 1 | Using :latest tag | Pin versions: image:1.2.3 |
| 2 | No backups | 3-2-1 backup rule, test restores |
| 3 | Exposing ports directly | Everything behind reverse proxy |
| 4 | Default passwords | Change immediately, use password manager |
| 5 | No monitoring | Uptime Kuma minimum, Grafana for depth |
| 6 | RAID = backup mentality | RAID protects disks, not data |
| 7 | Over-engineering day 1 | Start small, add complexity as needed |
| 8 | No documentation | Document every service, every port, every cron |
| 9 | Ignoring updates | Security patches matter, schedule updates |
| 10 | Running as root | Non-root containers, restricted SSH |
Natural Language Commands
| Say | Agent Does |
|---|
| "Set up a new service" | Guide through compose file creation with security best practices |
| "Audit my homelab security" | Run through security scoring checklist |
| "Plan my backup strategy" | Design 3-2-1 backup plan for your setup |
| "What should I self-host?" | Assess needs and recommend services by tier |
| "My container keeps crashing" | Walk through troubleshooting decision tree |
| "Help me set up Traefik" | Generate production Traefik config with SSL |
| "Compare NAS options" | Compare TrueNAS vs Unraid vs DIY for your needs |
| "Optimize my Docker setup" | Review compose files for security and performance |
| "Set up monitoring" | Deploy Uptime Kuma + Prometheus + Grafana stack |
| "Plan a hardware upgrade" | Assess current usage, recommend hardware by budget |
| "Migrate from cloud to self-hosted" | Plan migration with data export and service mapping |
| "Set up remote access" | Compare and deploy VPN/Tailscale for secure remote access |