server-admin

name	server-admin
description	Server administration knowledge for Evan's personal home server running Arch Linux. Knows all services, their configs, network layout, and operational procedures.

Server Admin Skill

You are running directly on this server as user evan. You can run commands directly on the server.

You have deep knowledge of Evan's personal home server. The server configuration is managed via Ansible in the ~/workspace/ansible-personal repository.

Server Overview

• OS: Arch Linux (bare metal)
• Hostname: server (single inventory host)
• Playbook: play-server.yml → roles/machines/server
• Config management: ansible-playbook play-server.yml (or with --tags <tag> for specific services)
• User: primary user is evan (uid 1000)

Network

• LAN interface: lan0 (internal, 10.0.0.0/24)
• WAN interface: wan0 (external)
• Domain: home.evanpurkhiser.com
• DNS server: dnsmasq on lan0, listening only on LAN
  • DHCP range: 10.0.0.100–10.0.0.254
  • Falls back to systemd-resolved → Cloudflare DNS (1.1.1.1, 1.0.0.1)
• Firewall: ufw — deny all by default, allow all on lan0, allow HTTPS (443) and BitTorrent (51413) on wan0
• Tailscale: enabled (tailscaled), provides VPN access

DHCP Reservations

IP	Hostname	Notes
10.0.0.2	unifi-ap	UniFi Access Point
10.0.0.3	server-bmc	Server BMC
10.0.0.4	macbook-home	Home MacBook
10.0.0.6	macbook-work	Work MacBook
10.0.0.20	lutron-hub	Lutron lighting hub
10.0.0.25	air-purifier	Air purifier
10.0.0.26	apple-tv	Apple TV
10.0.0.27	sonos-amp	Sonos Amp
10.0.0.28	sonos-sub-mini	Sonos Sub Mini
10.0.0.29	apple-homepod-bathroom	Bathroom HomePod
10.0.0.30	apple-homepod-bedroom	Bedroom HomePod
10.0.0.40	bambulab-p1s	Bambu Lab P1S 3D printer
10.0.0.50	kindle	Kindle

Storage

• Primary storage: ZFS pool mounted at /mnt/documents
• ZFS replication: zrepl pushes the documents filesystem to offsite TCP receiver (Tailscale IP)
  • Snapshots: hourly, prefixed zrepl_
  • Retention: 2 weeks daily → 8 weeks weekly → 6 months monthly (receiver)
• NFS export: /mnt/documents exported to 10.0.0.0/24, 100.0.0.0/8 (Tailscale), and fe80::/64 — all with all_squash (maps to evan/users)
• Transmission download dirs: /mnt/documents/downloads/torrents-complete and torrents-incomplete

Nginx / Public Services

All HTTPS traffic proxied through Cloudflare (strict SSL with Cloudflare origin cert). Real IP recovered from CF-Connecting-IP header.

Domain	Backend	Notes
hass.evanpurkhiser.com	127.0.0.1:8123	Home Assistant (WebSocket upgrade supported)
public.evanpurkhiser.com	127.0.0.1:9005	Waitress file share (robots.txt disallows crawlers)
apis.evanpurkhiser.com/meal-log	127.0.0.1:9006	Meal log API
apis.evanpurkhiser.com/instagram-saver	127.0.0.1:9007	Instagram saver webhook
apis.evanpurkhiser.com/atuin-abacus	127.0.0.1:7071	Atuin Abacus stats API
dots.evanpurkhiser.com	GitHub raw (proxied)	Dotfiles installer, proxied from raw.githubusercontent.com

Default server block returns 444 (immediately close) for unknown hosts.

Podman Containers

All containers are managed via systemd units generated by Podman. Auto-update via podman-auto-update.timer (updates containers with io.containers.autoupdate: registry label).

Atuin (Shell History Sync)

• Pod: atuin — ports 7070:8888 (server) and 7071:8000 (abacus)
• Containers:
  • atuin-postgres — docker.io/postgres:14, data at /var/lib/atuin/data
  • atuin-server — ghcr.io/atuinsh/atuin:18.4.0, config at /var/lib/atuin/config, open registration disabled
  • atuin-abacus — ghcr.io/evanpurkhiser/atuin-abacus:latest, stats/analytics API (port 8000), cache TTL 300s
• Systemd units: container-atuin-postgres, container-atuin-server, container-atuin-abacus, pod-atuin

Home Assistant

• Container: home-assistant — ghcr.io/home-assistant/home-assistant:latest
• Network: host networking
• Config: /var/lib/hass/ (configuration.yaml, automations.yaml, scripts.yaml, secrets.yaml, themes/)
• Devices: /dev/zwave (Z-Wave), /dev/zigbee (Zigbee) — GoControl USB adapter, udev rule 99-usb-gocontrol.rules
• Location: NYC, timezone America/New_York
• External URL: https://hass.evanpurkhiser.com
• Integrations: LunchMoney (unreviewed transaction count sensor), YouTube playlist sensors (Morning, Evening Chillout, Nighttime Sleep, Energized), Virtual Doorman REST unlock command, mobile_app, lovelace, history, logbook
• Mosquitto MQTT broker: port 1883, anonymous access, listening 0.0.0.0
• Restart: ansible-playbook play-server.yml --tags hass or sudo systemctl restart container-home-assistant

Waitress (File Share)

• Container: waitress — ghcr.io/evanpurkhiser/waitress:latest
• Port: 9005:80
• Volume: /mnt/documents:/mnt/documents
• Serves: /mnt/documents/public-share → public.evanpurkhiser.com
• Starts after: mnt-documents.mount

Meal Log

• Container: meal-log — ghcr.io/evanpurkhiser/meal-log:latest
• Port: 9006:8888
• Data: /var/lib/meal-log
• Config: /etc/meal-log.conf (OpenAI token, Cloudflare R2 credentials for image storage in meal-log-images bucket)

Instagram Saver

• Container: instagram-saver — ghcr.io/evanpurkhiser/instagram-saver:latest
• Port: 9007:8888
• Config: /etc/instagram-saver.conf (Google Places API key, OpenAI token)

Systemd Timer Services

All timers are monitored via Sentry Cron Monitors.

Service	Schedule	Purpose
auto-system-update.timer	Sat 09:00	Pacman system update with AI summary; uses OpenAI API; config at /etc/auto-system-update.conf
bambulab-lights-off.timer	Every 15 min	Turns off Bambu Lab P1S printer lights when idle; IP 10.0.0.40; config at /etc/bambulab-lights-off.conf
record-file-history.timer	Mon 00:00	Records filenames from /mnt/documents to /mnt/documents/archived/file-history
speedtest-monitor.timer	08:05 and 22:05 (±1hr random)	Runs speedtest --secure --json >> /var/local/speedtest-monitor.json
venmo-auto-cashout.timer	Every minute	Auto-cashes out Venmo balance to bank; config at /etc/venmo-auto-cashout.conf; state DB at /var/lib/venmo-auto-cashout/transactions.db; LunchMoney category z-venmo
venmo-lunchmoney-ai.timer	Every 30 min	AI-assisted Venmo→LunchMoney reconciliation; notifies via Telegram; state at /var/lib/venmo-lunchmoney-ai/state.json; config at /etc/venmo-lunchmoney-ai.conf
weekly-report.timer	Sat 10:00	Runs weekly-report.sh and pipes to purkhiser-bot.sh (sends to Telegram channel)
clean-macfiles.timer	Hourly	Removes macOS ._* and .DS_Store files from /mnt/documents
podman-auto-update.timer	Custom schedule	Updates Podman containers tagged with io.containers.autoupdate: registry

Transmission (BitTorrent)

• Service: transmission (runs as user evan)
• RPC: port 9091, whitelist 127.0.0.1, 10.0.0.*, 100.*.*.* (Tailscale), no auth required
• RPC URL path: /transmission/
• Peer port: 51413 (open on WAN firewall)
• Download dirs: /mnt/documents/downloads/torrents-complete (done), /mnt/documents/downloads/torrents-incomplete (in progress)
• Download queue: 5 simultaneous
• On torrent complete: triggers transmission-helper-trigger.sh → socket-activated transmission-helper@.service
• Transmission Helper: OpenAI + Telegram integration for torrent completion notifications; config at /etc/transmission-helper.conf; socket: transmission-helper.socket

Purkhiser Bot (Telegram)

• Script: /usr/local/bin/purkhiser-bot.sh
• Config: /etc/purkhiser-bot.conf — Telegram token + channel ID
• Used by: weekly-report, venmo-lunchmoney-ai, transmission-helper

SSH / Access

• SSH: password auth disabled, empty password disabled — key-only
• PAM sudo auth: pam-ssh-agent — sudo authenticated via SSH agent key (key stored at /etc/security/authorized_keys)
• Tailscale: for remote access outside LAN
• Sudo: authenticated via PAM SSH agent auth — biometric prompt will appear on the user's controlling device (MacBook) when sudo is invoked. Avoid using sudo unless strictly necessary.
• Note: If sudo fails, it is likely because Evan cannot biometrically authenticate (e.g., he is accessing the server from an iPhone, which cannot handle the biometric prompt). In this case, sudo commands will not be possible until he is back at a MacBook.

Package Management

• Pacman sync: pacsync role — standard Arch repos
• AUR helper: aur role with aur-builder user — packages installed: zrepl, pam-ssh-agent
• Paccache: automatic cache cleanup
• Time sync: systemd-timesyncd

Common Operations

Apply full playbook

cd ~/workspace/ansible-personal
ansible-playbook play-server.yml

Apply specific service

ansible-playbook play-server.yml --tags <tag>
# Tags: zfs, packages, sshd, network, hardware, pam-ssh-agent, nfs, tailscale, nginx,
#       transmission, hass, waitress, meal-log, instagram-saver, venmo-auto-cashout,
#       venmo-lunchmoney-ai, atuin, podman-auto-update, zrepl, record-file-history,
#       purkhiser-bot, weekly-report, speedtest-monitor, auto-system-update, bambulab-lights-off

Check container status

sudo systemctl status container-home-assistant
sudo systemctl status container-atuin-server
sudo systemctl status container-atuin-postgres
sudo systemctl status container-atuin-abacus
sudo systemctl status container-waitress
sudo systemctl status container-meal-log
sudo systemctl status container-instagram-saver

Check timer status

systemctl list-timers
sudo systemctl status auto-system-update.timer
sudo systemctl status venmo-auto-cashout.timer
sudo systemctl status venmo-lunchmoney-ai.timer

ZFS / zrepl

sudo zfs list
sudo zrepl status
sudo systemctl status zrepl

NFS exports

sudo exportfs -v       # show current exports
sudo exportfs -ra      # reload exports

Nginx

sudo nginx -t                    # test config
sudo systemctl reload nginx      # reload without downtime
sudo systemctl status nginx

Transmission

sudo systemctl status transmission
# Web UI accessible at http://server:9091/transmission/web/ on LAN or Tailscale

Logs

journalctl -u container-home-assistant -f
journalctl -u transmission -f
journalctl -u zrepl -f
journalctl -u dnsmasq -f
journalctl -u nginx -f

Config File Locations

Service	Config Path
nginx	/etc/nginx/nginx.conf
dnsmasq	/etc/dnsmasq.conf
systemd-resolved	/etc/systemd/resolved.conf
NFS exports	/etc/exports
Mosquitto	/etc/mosquitto/mosquitto.conf
zrepl	/etc/zrepl/zrepl.yml
Home Assistant	/var/lib/hass/configuration.yaml
Transmission	~evan/.config/transmission-daemon/settings.json
Atuin	/var/lib/atuin/config/server.toml
auto-system-update	/etc/auto-system-update.conf
bambulab-lights-off	/etc/bambulab-lights-off.conf
venmo-auto-cashout	/etc/venmo-auto-cashout.conf
venmo-lunchmoney-ai	/etc/venmo-lunchmoney-ai.conf
meal-log	/etc/meal-log.conf
instagram-saver	/etc/instagram-saver.conf
transmission-helper	/etc/transmission-helper.conf
purkhiser-bot	/etc/purkhiser-bot.conf
udev rules	/etc/udev/rules.d/99-usb-gocontrol.rules
pam sudo	/etc/pam.d/sudo
ssh authorized keys (sudo)	/etc/security/authorized_keys