verbatim-audit-notify

name	verbatim-audit-notify
description	Send Pushover notifications with UUID-linked verbatim JSONL audit trail. TRIGGERS - pushover notify, send pushover, observability alert, verbatim notification, fleet alert, pushover-lookup, audit log notification, push notification with UUID

Pushover Verbatim+UUID Notification

Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.

A two-script skill that solves the "Pushover message hit my phone but I don't remember what it was about" problem for personal automation fleets. Every notification carries a UUID; the full verbatim payload (including everything that didn't fit in Pushover's 1024-char body) lands in a local JSONL audit log keyed by that UUID. You look it up by pasting the UUID back.

Designed for: cron-fired scripts, launchd daemons, hook outputs — any place that wants "fire-and-forget alerting with full context if you ever need to dig in." Personal scale; one Mac; one Pushover account. Not a microservices observability stack.

Why this exists

Pushover messages are limited to 1024 UTF-8 characters in the body and 250 in the title (per pushover.net/api). Real failure events often need thousands of chars of context: stack traces, full env dumps, file paths, the exact failing command. Truncating loses what you actually need to debug.

The fix is the correlation-ID-plus-JSONL pattern: short summary on the device, full verbatim payload in a local newline-delimited JSON file, UUID linking them. When a notification fires, the body contains the UUID and a pushover-lookup command. Run that and you get the complete entry.

Five scripts + three launchd templates

Asset	Role
scripts/pushover-notify.sh	Sender: generates UUID, writes verbatim JSONL, dispatches Pushover with summary+UUID
scripts/pushover-lookup.sh	Retriever: given a UUID (or prefix), prints the pretty-printed JSONL entry
scripts/pushover-prune.sh	Retention pruner: deletes audit-YYYYMMDD.jsonl files older than N days (default 30)
scripts/pushover-quota.sh	Quota monitor: hits Pushover /apps/limits.json, persists JSON, alerts when low (iter 12b)
scripts/pushover-heartbeat.sh	Daily fleet status summary — companion+kokoro+github-notif+quota+disk+failed services (iter 20)
templates/com.terryli.pushover-prune.plist	launchd timer — daily at 04:15, 90-day retention (iter 8)
templates/com.terryli.pushover-quota.plist	launchd timer — daily at 03:30, alerts when remaining <20% (iter 12b)
templates/com.terryli.pushover-heartbeat.plist	launchd timer — daily at 09:03, INFO heartbeat (auto-promotes to WARN on failure) (iter 20)

Add the scripts to your PATH:

ln -sf "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/scripts/pushover-notify.sh" ~/.local/bin/pushover-notify
ln -sf "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/scripts/pushover-lookup.sh" ~/.local/bin/pushover-lookup
ln -sf "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/scripts/pushover-prune.sh" ~/.local/bin/pushover-prune
ln -sf "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/scripts/pushover-quota.sh" ~/.local/bin/pushover-quota
ln -sf "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/scripts/pushover-heartbeat.sh" ~/.local/bin/pushover-heartbeat

Verify the symlinks resolve to THIS skill (iter 13a 2026-05-19 caught the trap where stale symlinks from a legacy pushover-notify in ~/.claude/tools/notifications/ silently masked the new flag-rich script — the legacy didn't understand --service/--level/--extra, so dispatches "succeeded" but wrote no JSONL audit and sent malformed Pushover payloads):

for cmd in pushover-notify pushover-lookup pushover-prune pushover-quota; do
    readlink "$HOME/.local/bin/$cmd" | grep -q "cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify" \
        && echo "✓ $cmd → iter-5 skill" \
        || echo "✗ $cmd → STALE target ($(readlink "$HOME/.local/bin/$cmd" || echo 'not a symlink')) — rerun the ln -sf commands above"
done

Then sanity-fire the alert path once to catch any other silent failures:

pushover-quota --alert-threshold 1.0   # always fires; check phone + audit log
pushover-lookup --recent 2             # confirm WARN + pushover-notify dispatch lines pair up

Install the launchd timers (retention + quota monitor — see each template header for tuning):

# Retention (daily 04:15, 90-day window)
cp "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/templates/com.terryli.pushover-prune.plist" ~/Library/LaunchAgents/
mkdir -p ~/.local/state/launchd-logs/pushover-prune
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.terryli.pushover-prune.plist

# Quota monitor (daily 03:30, alert at <20% remaining)
cp "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/templates/com.terryli.pushover-quota.plist" ~/Library/LaunchAgents/
mkdir -p ~/.local/state/launchd-logs/pushover-quota
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.terryli.pushover-quota.plist

# Daily fleet heartbeat (09:03, INFO; auto-promotes to WARN on failure)
cp "$HOME/.claude/plugins/marketplaces/cc-skills/plugins/pushover-commander/skills/verbatim-audit-notify/templates/com.terryli.pushover-heartbeat.plist" ~/Library/LaunchAgents/
mkdir -p ~/.local/state/launchd-logs/pushover-heartbeat
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.terryli.pushover-heartbeat.plist

Heartbeat sample output (iter 20)

🔔 Fleet daily heartbeat

companion: ok · up 11h 36m · 320MB · audio✓ · bot=watching · tts=ready
kokoro: ok · idle=-1s · queue=0
pushover quota: 611/10000 (6.11%)
disk: launchd-logs=119MB · audit days=1
failed services: com.terryli.maccy-backup=1

Auto-promotes from INFO (silent) to WARN when any subsystem is degraded (companion or kokoro not ok, OR any com.terryli.* launchd service has last_exit != 0). The structured --extra payload captures every dimension as JSON for forensic lookup via pushover-lookup.

Quick start

Send a notification

pushover-notify \
    --title "maccy-backup failure" \
    --message "Maccy DB unreadable for 31 days; backup script needs TCC Full Disk Access" \
    --service maccy-backup \
    --level ERROR \
    --extra '{"db_path":"/Users/terryli/Library/Containers/org.p0deje.Maccy/Data/Library/Application Support/Maccy/Storage.sqlite","last_success":"2026-04-17","days_since":31}'

Optional device targeting + sound override (iter 14, 2026-05-19) — useful for high-priority events that should land on a specific device with an attention-grabbing sound:

pushover-notify \
    --title "Telegram rate-limit" \
    --message "Bot blocked for 4900s" \
    --service telegram-bot \
    --target rate-limit \
    --level ERROR \
    --priority 1 \
    --device iphone_13_mini \
    --sound siren

--device <name> sends only to the named Pushover device (omit to broadcast to all). --sound <name> selects the alert tone (siren, magic, intermission, none, etc.); the chosen device+sound are also persisted into the JSONL audit entry for forensic completeness.

Output (stdout): the UUID, e.g.

3f8c2d9e-4a1b-4c5d-8e7f-1a2b3c4d5e6f

Your phone receives:

[maccy-backup] level=ERROR priority=1
Maccy DB unreadable for 31 days; backup script needs TCC Full Disk Access

lookup: pushover-lookup 3f8c2d9e-4a1b-4c5d-8e7f-1a2b3c4d5e6f

UUID: 3f8c2d9e-4a1b-4c5d-8e7f-1a2b3c4d5e6f

Look it up

Paste the UUID back:

pushover-lookup 3f8c2d9e-4a1b-4c5d-8e7f-1a2b3c4d5e6f

Or pipe the whole Pushover message body:

pbpaste | pushover-lookup

Output: full pretty-printed JSON with every field that was in --extra, plus the canonical schema.

JSONL schema

Each line in ~/.local/state/pushover/audit-YYYYMMDD.jsonl is one event:

{
  "run_id": "3f8c2d9e-4a1b-4c5d-8e7f-1a2b3c4d5e6f",
  "ts": "2026-05-19T07:23:01.123Z",
  "host": "terryli-mbp",
  "service": "maccy-backup",
  "actor": "launchd",
  "target": "Storage.sqlite",
  "level": "ERROR",
  "title": "maccy-backup failure",
  "message": "Maccy DB unreadable...",
  "priority": 1,
  "extra": {
    "db_path": "...",
    "last_success": "2026-04-17",
    "days_since": 31
  }
}

Followups (the Pushover API response, dispatch failures) are appended as separate lines with the same run_id — so jq -c 'select(.run_id == "...")' *.jsonl reconstructs the full timeline.

Credentials

By default, the sender pulls Pushover credentials from 1Password Claude Automation vault, item <pushover-item> (registered in docs/1password-credential-registry.md). It follows the cc-skills canonical pattern:

1. Unset HTTPS_PROXY / HTTP_PROXY (Claude Code OAuth proxy returns 502 on 1P endpoints)
2. Try Service Account token first (~/.claude/.secrets/op-service-account-token)
3. Fall back to biometric (unset OP_SERVICE_ACCOUNT_TOKEN; op read ...) on permission denied

Override for testing / non-1P environments:

PUSHOVER_TOKEN=... PUSHOVER_USER=... pushover-notify ...

Or skip the remote call entirely (write JSONL only):

NO_PUSHOVER=1 pushover-notify ...

Priority and TTL

Level	Default priority	Phone behavior
INFO	-1	Silent (no sound, no vibration); inbox-only
WARN	0	Default sound and vibration
ERROR	1	Bypass quiet hours
—	2 (manual)	Emergency — repeats until acknowledged (retry=30, expire=600)

For low-signal events (heartbeats, "nothing changed" pings) set a TTL so the message self-expires on the phone:

pushover-notify --level INFO --ttl 300 --title heartbeat --message "..." --service some-service

Wrapper patterns for common use cases

Wrap a launchd script (alert on failure only)

#!/bin/bash
set -e
LOG=$(mktemp)
if ! /path/to/your/script.sh > "$LOG" 2>&1; then
    pushover-notify \
        --title "script.sh failed (exit $?)" \
        --message "$(tail -c 400 "$LOG")" \
        --service script-name \
        --level ERROR \
        --extra "$(jq -Rs '{stdout_tail: .}' < "$LOG")"
    exit 1
fi

Use from a pipe

some-long-running-job 2>&1 \
    | tee /tmp/job.log \
    | tail -n 0  # block until job done
pushover-notify \
    --title "job completed" \
    --service my-job \
    --message "$(tail -c 500 /tmp/job.log)" \
    --extra "$(jq -Rs --arg exit "$?" '{exit_code: ($exit | tonumber), log_tail: .}' < /tmp/job.log)"

Operational notes

• Log location: ~/.local/state/pushover/audit-YYYYMMDD.jsonl — one file per UTC day.

• Rotation vs retention (iter 7, 2026-05-19): the per-day filename gives you natural size-rotation for free — every UTC midnight a new file starts, so size never grows unboundedly within a file. Size-based rotation in ~/.config/log-rotation.conf is therefore not needed and intentionally not wired (the conf file documents this explicitly). What IS needed is retention — pruning old days. pushover-prune handles this: default 30-day window, dry-run by default, never deletes today's file. Run manually or wire into a daily launchd timer:

  pushover-prune                  # show what would be pruned (30d default)
  pushover-prune --apply          # delete files older than 30 days
  pushover-prune --keep 7 --apply # tighter 7-day window
  

• Privacy: JSONL is on the local Mac. Pushover only sees what's in the message body (1024 chars max). Secrets should NOT go in --message or --title.

• Tested: Pushover-side validated end-to-end during iter 4 (test UUID C3B649E1-BF34-4346-A211-511EFE7CDCBD delivered). Prune script boundary-tested iter 7 (today's file preserved even at --keep 0).

References

• Pushover API docs — message format, priorities, receipts
• Pushover May 2026 quota changes — per-account 10k msgs/month
• 1Password registry: docs/1password-credential-registry.md
• Companion hook: plugins/devops-tools/hooks/posttooluse-1password-pattern-reminder.sh (reminds Claude of credential pattern)

Post-Execution Reflection

After this skill completes, check before closing:

1. Did the notification deliver? — Pushover returns a receipt token; if delivery silently failed, fix the instruction (auth, rate-limit, malformed body) that caused it.
2. Did the JSONL audit entry write correctly? — pushover-lookup <uuid> should round-trip the full payload. If not, the writer is dropping fields — fix the schema.
3. Was the message truncated? — If the body exceeded 1024 chars, confirm the --extra payload captured everything that didn't fit. Update Usage examples if the truncation boundary moved.
4. Did pushover-lookup find by UUID prefix? — If only the full UUID worked, the prefix-search needs fixing.

Only update if the issue is real and reproducible — not speculative.