audit-first

name	audit-first
description	Audit-before-fix protocol — first find all gaps across 7 planes, prioritize, pin the gap table, and only then fix. Principle: never touch code until the full picture of problems is understood. Gap table → approval → flow-first per gap → library-first → plan → code. Use when: task begins with looking for problems ("find gaps in X", "audit security", "what's wrong with Y"), not with a specific fix

Audit-First Protocol

For tasks where you must first find all the problems, and only then fix. Use when:

• An area is suspicious but the specific gap is unknown
• Need to make sure we're fixing the right problem, not a symptom
• Task sounds like "look at what's wrong in X" rather than "fix specific Y"
• Before an important deploy — a preventive audit

Difference from arch-first:

	arch-first	audit-first
Purpose	Execute a complex task	Discover problems
Phase 0	Understand the task	Find all gaps
Result	Decomposition into blocks	Gap table with priorities
When to fix	Immediately per the plan	Only after table approval
Launch	"Do X"	"Find what's wrong in X"

Input

A fix task where the specific problems are unknown; the module or integration area to audit.

Output

audit-doc.md with a holes table across 7 planes (Security / Data / Logic / Integrations / Observability / UX / Performance), prioritized, plus per-hole reports as fixes close.

Hands off to

Per-hole flow-first cycle for each prioritized hole, then ship-first at task-level closure.

---

Principle 1 — Plane: Security

The most critical plane. HIGH by default. Checked first.

1.1 Auth / Authz

Endpoint accessible without token → HIGH. Internal endpoint protected only by hardcoded token → HIGH. Authorization doesn't check resource-to-tenant ownership (IDOR) → HIGH.

1.2 Injections (SQL, CMD, XSS)

String concatenation in queries, shell=True, innerHTML with user data → grep and flag.

1.3 Encryption

In transit: SMTP without TLS/SSL → HIGH. HTTP for external calls → MED. At rest: passwords in plaintext → HIGH. OAuth tokens in DB without encryption → LOW (single-tenant) / MED (multi-tenant).

1.4 OAuth / Tokens

• CSRF: state = user_code instead of secrets.token_urlsafe() → HIGH
• Token TTL: access_token used without checking expires_at → MED
• Storage: tokens in logs, git, unignored env → HIGH

1.5 Webhook validation

Two mandatory levels: signature (HMAC-SHA256 verified) + timestamp (abs(now - ts) > 300 rejected). Only level 1 → replay possible → MED. Neither → HIGH.

1.6 Secrets in code and configs

Hardcoded secret in code = HIGH always, even for a private repo.

---

Principle 2 — Plane: Integrations architecture

Doesn't break immediately — breaks under load / when external service fails. MED default, HIGH on critical path.

2.1 Rate Limit (429)

Generic except Exception swallows FloodWaitError / RateLimitError → message lost silently. Correct: catch specific error before generic Exception, sleep + one retry.

2.2 Service-specific errors

Every external service has errors that must not fall into generic Exception. Telegram FloodWaitError → sleep + retry. Whisper 413 → check size BEFORE sending. Zoom webhook stale timestamp → reject with 400. SMTP wrong encryption → ValueError, not silent fail.

2.3 In-Memory state

Module-level dict resets on deploy. _oauth_state_store → in-flight OAuth breaks (MED). _token_cache → extra requests after restart (LOW). _active_sessions → everyone logged out on deploy (HIGH).

2.4 Validation BEFORE an expensive network request

File >25MB sent to Whisper without a check → learn about the problem only on 413 → retry loop. Correct: if getsize > limit: raise before opening the file.

2.5 Circuit Breaker

Every external HTTP call has timeout=. ConnectionError handled separately from business errors. Service unavailable → task marked failed, not hanging.

---

Principle 3 — Plane: Data and DB

Rare but irreversible. Especially dangerous in multi-tenant systems.

3.1 Missing transactions (Race Condition)

Trigger for check: read → decide → write based on what was read. Two execute calls in a row without with conn: → concurrent requests can insert a duplicate.

3.2 Missing constraints

The DB must protect integrity independently of the code. Email without UNIQUE → duplicate accounts. FK without ON DELETE → orphans. Status without CHECK → invalid values. Numeric without NOT NULL DEFAULT 0 → NULL surprises.

3.3 N+1 queries

Query inside a loop. 100 records = 101 queries instead of 1. In SQLite less critical than in PostgreSQL, still noticeable on large datasets.

3.4 Plaintext for sensitive fields

User passwords in plaintext → HIGH always. OAuth tokens in DB → LOW / MED. API keys in DB → MED (better in env).

3.5 No TTL and no cleanup of temporary data

In-memory stores without cleanup, tokens/nonce tables without periodic DELETE WHERE expires_at < now(), logs without partitioning → LOW at small scale, grows with load.

---

Principle 4 — Plane: Error handling

Most invisible plane. Gaps here don't obviously break — they make the system unpredictable.

4.1 Silent Failures

Exception swallowed with only a log + return False. Caller ignores False. Email didn't send, no one knows. Rule: either re-raise (critical operations) or return False + explicit caller-side check.

4.2 Retry loops without a limit

Retry on 413 (file too large) — won't help, file won't shrink. Retry on 401 (invalid key) — won't help either. Rule: retry only for transient errors (network dropped, service temporarily unavailable). For deterministic errors → fail fast with a clear message.

4.3 No Fallback

External service unavailable → what happens to user? Options (best to acceptable): degradation (simpler version) / queue with TTL / notify user of deferred / skip with log (non-critical only). Unacceptable: skip without log and without notification.

4.4 Uninformative error messages

HTTPException(500, str(e)) leaks stack trace + internal details. Rule: 400 for invalid input, 401 auth, 403 access, 404 not found, 502 external service, 500 unexpected (no details externally, details in logs).

---

Principle 5 — Plane: UI/UX

Breaks trust, not data. Empty screen instead of error, button that doesn't react, form that accepts garbage.

5.1 No Loading State

User clicked → nothing visual → clicks again → double submission. Rule: button disabled = true + indicator during any async action.

5.2 No Error State

Request failed → blank screen or stale data. Rule: show message (not technical details), give action ("Try again" / "Contact support"), don't hide behind blank.

5.3 No Empty State

List loaded, no data → empty space. Three states any list must have: Loading (skeleton), Empty ("No X yet. Create the first →"), Error ("Failed to load. Try again.").

5.4 No frontend validation

Form accepts invalid input → server returns 400 → user sees unclear error. Validate on frontend in addition to backend: required fields before submission, email/phone/URL format inline, numeric ranges, interdependent fields.

5.5 Broken Edge Cases

Golden path only. Typical failures: title = null → crash on .length; empty list → blank; very long name → overflow; number 0 → if (count) yields false; date in past → "Start" button active for expired.

5.6 Destructive actions without confirmation

Delete, disconnect, reset → without "Are you sure?". Rule: irreversible action requires explicit confirmation describing consequences.

---

Principle 6 — Plane: Observability

Doesn't break — makes incidents impossible to investigate.

6.1 Silent Success without log

Operation done, nothing logged. Week later: "did it send Wednesday?". Every meaningful operation → log with identifiers.

6.2 No Structured Logging

_log.info("sending email") — can't tie to specific user. Include always: user_code + what's happening + relevant IDs (meeting, job, request_id).

6.3 Critical operations without Audit Trail

OAuth connect/disconnect, tenant settings change, data deletion, auth failure, webhook receipt — all must be logged always and immutably. For compliance, incident debugging, disputes.

6.4 No metrics for external calls

Whisper started answering in 30s instead of 3 — anyone notice before user complaints? Log elapsed for slow ops. Warning if slower than expected threshold.

6.5 Logging secrets

Tokens / passwords / API keys in log lines. Rule: log only non-confidential identifiers. Config with password → log only non-secret fields.

---

Principle 7 — Plane: Config in code instead of source of truth

The most invisible. UI "works" (form saves), DB "works" (data stored), tests "pass" — but real effect is missing. User loses trust: "I did change this".

7.1 Hardcoded behavior parameters

DIGEST_SEND_HOUR = 9 hardcoded in code. User sets 18:00 in UI → saved to settings.digest_hour → digest still goes at 9:00. Silent, invisible bug.

Typical cases: HOUR / DAYS / INTERVAL / REMIND / LIMIT / THRESHOLD / TEMPLATE_ID as constants when they should read from DB. Criticality: HIGH if affects money/sending/access. MED for UX (time, threshold). LOW for metadata.

7.2 Unused DB fields (saved but not read)

Settings table has a field. UI writes it. Business logic never reads it — uses default or constant instead. Grep test: field name → INSERT/UPDATE only, no reads outside migrations → confirmed gap.

7.3 "Phantom" settings forms

UI form saves data, backend doesn't apply. Check intersection: "what POST accepts" vs "what business logic reads". Excess on POST side → phantom fields.

---

Output format — Gap table

After scanning all planes, consolidate into one table. Only format used. Approval point before fixes.

| # | Plane | Integration / Module | Gap | File:line | Criticality |
|---|---|---|---|---|---|
| 1 | Security | Zoom OAuth | state = user_code, no CSRF nonce | routes/oauth.py:34 | HIGH |
| 2 | Security | Email (SMTP) | No TLS, plaintext | services/email_service.py:91 | HIGH |
| 3 | Architecture | Telegram | FloodWaitError in generic except | services/telethon_manager.py:44 | MED |

Criticality definitions

Level	When to set
HIGH	Exploitable without privileges OR data loss/corruption/unauthorized access. Examples: CSRF in OAuth, plaintext passwords, webhook without signature, SQL injection, open authorization.
MED	Requires certain conditions OR data loss under a coincidence of factors. Examples: FloodWaitError in except, replay attack, file >25MB without check.
LOW	Technical improvement or potential vector at growing load. Doesn't break now, creates risk. Examples: in-memory cache without TTL, tokens in plaintext single-tenant, missing metrics.

Rules

• One row = one gap (don't group even if in same file)
• File:line = specific code line or nearest related
• Gap = one sentence: what specifically is missing or wrong
• HIGH does not exceed 3 in a normal task. 5+ HIGH → re-check criteria

Final line:

Total: {N} HIGH / {M} MED / {K} LOW — {X} gaps, of which {P0} P0 blocks and {P1} P1 blocks.
Waiting for approval → then I pin the block order and ask the work mode.

---

Workflow — 5 phases

Sequential. Cannot start fixing until all 5 phases are completed.

Step 0 — Check / create the task

{log_dir} set → proceed silently. Not set → form slug {date}-audit-{area-slug}, show "Creating task: {slug}. Rename?", mkdir, INSERT into artifacts. Cannot skip.

Child task branch: triggers "create child task" / "next audit stage" / "continuation of {parent}". Form {parent}-part{N}, register with links (from, to, 'part_of'). Direct mkdir + Write + INSERT bypassing this protocol is prohibited — the part_of link won't register and ship-first won't see unclosed chain areas.

Step 0.5 — Calibrate incoming artifacts

audit-first expects idea-first-*.md with type "fix" or "problem". Type differs → propose switching to arch-first. Artifact missing → allowed (manual invocation), skip calibration, proceed with Phase 0.

Phase 0 — Define the audit area

Do not scan until the area is specified. Never scan the whole project — analog of flow-first "going into code without anchors" → 50 LOW gaps, none important. Area given → use as starting point. Not given → ask one specific question.

Phase 1 — Scan across the 7 planes

Sequential walk. Order matters:

Order	Plane	Why here
1	Security	Only plane HIGH by default
2	Integrations architecture	Gaps manifest under load
3	Data and DB	Data loss irreversible
4	Error handling	Masks gaps in 1-3
5	UI/UX	After backend planes
6	Observability	Last — doesn't affect correctness
7	Config in code	Silent gaps, UI+DB "work" but settings not applied

For each plane: read area files → grep → pin what was found. "Not affected" rule: irrelevant plane → explicit "not applicable" row, don't leave blank.

Phase 2 — Assemble the gap table

All found gaps → one table. HIGH at top, then MED, then LOW. Assign a block to each HIGH and MED. LOW at user's discretion.

Phase 3 — Table approval

Show table + summary line → wait for explicit approval.

Before approval — no code. Even if the gap is obvious and fix is 2 lines. This is the main rule.

User can: approve → Phase 4; remove row (LOW deferred); change criticality; add missed gap. Each edit → updated table → wait for approval again.

After approval — register blocks in task_blocks for each HIGH/MED gap. Sub-blocks use block_num = N*10 + M, parent_block_num = N.

Append ## Blocks section to task.md. Output explicit "Registered N blocks: {list}. Launching Skill('flow-first')."

Phase 3.5 — Batch call to cadence-first

After the INSERT loop and before asking work mode → invoke cadence-first in batch mode so each gap block gets recommended_cadence pre-baked. HIGH gaps usually Tier 1 full cycle; LOW often Tier 3 plan-only — cadence-first formalizes this differentiation.

Why here: cadence is decided at GENERATION time (rich context — audit area, gap table, criticality), not at EXECUTION time.

On failure → log warning, continue. Blocks execute without pre-baked cadence.

Final step of Phase 3 — Ask work mode

Blocks pinned. How do we work?

1. 🔁 Manual — I run each block, wait for your "ok"
2. 🚀 Autopilot — dev-auto-first takes over

Choice 1 → Skill('flow-first') for first HIGH block. Choice 2 → Skill('dev-auto-first').

Save audit-doc.md to log_dir with the approved gap table. Register in task_artifacts. Cannot skip — on session drop, chat table isn't recoverable.

Phase 4 — Fix per-block

Per-block cycle for each gap in order HIGH → MED → LOW: flow-first (understand landscape) → library-first (ready pattern) → plan-first → execute → ship-first (per-block).

Block transition rule: after "go" on ship-first per-block → first action is Skill(flow-first) for the next block. Not Read, not grep, not plan-first directly. flow-first cannot be skipped even if "task is obvious".

Each fix = separate commit with clear description of what it closes.

After all fixes → ship-first task-level updates gap table (✅ Fixed / 🔁 Deferred), does final report, sessions, STATUS, push + sync.

---

Anti-patterns

❌ Shovel the whole project without an area

Grep everywhere → 80 lines of noise. Half false positives, other half LOW. Rule: area set before first grep.

❌ Fix along with scanning

Found a gap → immediately fix without waiting for remaining planes. Fixing first breaks second you haven't found yet. Rule: scan → table → approval → only then code.

❌ Silently skip a plane

"UI isn't relevant" → plane simply not mentioned. User doesn't know: didn't look, or looked and clean? Rule: explicit "not applicable" row for irrelevant planes.

❌ Mix audit and fix in one step

"Found and fixed straight away". User didn't see the table, didn't approve priorities. Rule: table shown before any code change. Approval mandatory.

❌ Everything is HIGH

Lowering the bar — user stops trusting criticality. Rule: HIGH only for exploitable-without-privileges OR data loss/corruption. 3+ HIGH → re-check criteria.

❌ Don't pin LOW in the table

LOW seems insignificant → skip. Six months later LOW is MED, then HIGH, nobody remembers it was known. Rule: all found gaps in the table. Decision what to do with LOW belongs to user.

❌ Don't check settings forms

Form saves → agent considers "Config in code" plane clean. But doesn't check whether field is READ in business logic. Rule: for each settings field grep-check reads (not save/update). Written but not read → gap.

❌ "go" on next block → plan-first instead of flow-first

After block finished, user answered "go" → agent goes directly to plan-first. Rule: "go" on next block = first action Skill(flow-first). flow-first mandatory for each block without exception.

❌ Create child task directly via bash bypassing Step 0

Direct mkdir + Write + INSERT INTO artifacts without the Step 0 child-task branch → task appears without part_of link → ship-first doesn't see unclosed siblings on closure. Rule: task creation only via Step 0 branch. Direct tools prohibited.

❌ One big fix for the whole table

All gaps in a single commit → impossible to roll back specific fix, impossible to review. Rule: one gap = one commit.

❌ Inlining the skill instead of using the Skill() tool

Executed the protocol from memory — outdated version, no accuracy guarantee. Always via Skill('audit-first').

---

Related skills

• idea-first — upstream artifact read in Step 0.5; type "fix"/"problem" → audit-first; other types → arch-first
• flow-first / library-first / plan-first / ship-first — per-block execution chain
• cadence-first — batch-invoked at Phase 3.5 to pre-bake per-block cadence
• arch-first — sibling for feature/product tasks (decomposition instead of gap discovery)

---

Step 99 — Log invocation

sqlite3 {routing_db} \
  "INSERT INTO skill_invocations (task_id, block_num, skill_name, invoked_at)
   VALUES ('{slug}', '{N}', 'audit-first', datetime('now'))" 2>/dev/null || true