fp-doc-audit

name

fp-doc-audit

description

Verify every claim in the project's false-positive document (typically SECURITY-SCAN-FALSE-POSITIVES.md) against the actual code. Use when the user asks to "audit the false positive doc", "check FP doc", "verify false positives", or before submitting an AppExchange security review. Also invoke automatically when the user is about to submit, tag a release, or push an AppExchange package version. A real FlexibleTeamShare rejection was caused by a false claim in this doc — treat it as a load-bearing artifact and validate every assertion.

False-Positive Documentation Auditor

Verify that every justification in the project's false-positive document is grep-provable against the current code. Previously a claim like "All read queries use WITH USER_MODE" was materially false for two inner classes; the Salesforce reviewer verified by grep, rejected the justifications, and the submission failed. This skill prevents that from happening again.

Default doc locations (in order of preference):

1. docs/false-positives.md — the toolkit-default working location during development.
2. appexchange-artifacts/docs/SECURITY-SCAN-FALSE-POSITIVES.md — present once your AppExchange submission process is underway and Salesforce has supplied artifacts.
3. docs/reference/false-positive-template.docx — the official Salesforce template, populated for actual submission.

If none of these exist, the skill reports "no FP doc to audit" — which is itself a finding if your codebase has any without sharing classes, dynamic SOQL, or LWC→Apex DML paths.

When to run

• Before running /appexchange-audit (Category 10.3 delegates here).
• Before any AppExchange submission or package version creation.
• When the user asks about the FP doc's accuracy.
• When SECURITY-SCAN-FALSE-POSITIVES.md changes in a diff.
• When force-app/main/default/classes/ changes in ways that could invalidate past claims (any removal of WITH USER_MODE, addition of without sharing, new @SuppressWarnings('PMD.ApexCRUDViolation')).

Procedure

1. Locate and read the FP doc. Try the locations listed above in order. If none exists, return "no FP doc to audit" plus a recommendation to create one when the codebase has any of: without sharing classes, dynamic SOQL, @SuppressWarnings('PMD.ApexCRUDViolation') annotations, or @AuraEnabled methods with DML. Otherwise read the doc top-to-bottom and build a list of individual claims.

2. Normalize each claim into a verifiable statement. For each justification:

   • Identify the rule it dismisses (e.g. ApexFlsViolation, DatabaseOperationsMustUseWithSharing, AvoidGlobalInstallUninstallHandlers).
   • Identify the file(s) and line(s) in scope.
   • Extract the specific assertion(s) being made. Examples:
     • "All read queries use WITH USER_MODE" → verifiable: grep all SOQL in the cited file, confirm every one has WITH USER_MODE.
     • "Explicit CRUD checks before DML" → verifiable: for every DML in the cited file, confirm a preceding isCreateable()/isUpdateable()/isDeletable() check.
     • "Runs in system context during install (InstallHandler)" → verifiable: class implements InstallHandler and the cited DML is inside a method invoked by onInstall(InstallContext).
     • "Users without permission set cannot access the objects" → not grep-provable on its own. The permission set model is correct but doesn't satisfy Salesforce review requirements; flag as [WEAK JUSTIFICATION].
     • "Users can only reach this via LWC" → not a control. Flag as [WEAK JUSTIFICATION].

3. Verify each assertion against current code. Use Read + grep on the cited file. Specifically:

   • For "uses WITH USER_MODE" — grep every SELECT in that file; fail the claim if any SOQL lacks WITH USER_MODE / WITH SECURITY_ENFORCED / per-field access loop.
   • For "CRUD checks before DML" — for every insert/update/delete/upsert in the file (and Database.insert(/Database.update(/...), trace back to find the preceding CRUD check in the same call path. Missing in any path → claim fails.
   • For "no cited line" / vague justification — classify as [WEAK JUSTIFICATION].

4. Classify each entry.

   • [VERIFIED] — every assertion holds in current code.
   • [STALE FP DOC] — at least one assertion is materially false. BLOCKER.
   • [WEAK JUSTIFICATION] — assertion is a vague claim (UI-only reachability, permission-set inference, etc.) that Salesforce reviewers have historically rejected. BLOCKER.
   • [MISSING] — a known scan finding is not addressed in the FP doc at all. BLOCKER. Source for known scan findings: appexchange-artifacts/security-scan/security-scan-results.csv if present, otherwise the most recent local Code Analyzer run output (docs/scan-appexchange.html or whatever the latest run produced).

5. Produce a report. Structure:

   # FP Doc Audit — <date>
   
   ## Summary
   - Verified: X
   - Stale: X  (must fix)
   - Weak: X   (must strengthen)
   - Missing: X (must add)
   
   ## Findings
   
   ### [STALE FP DOC] <Rule> — <File>:<Line>
   Claim: "<verbatim quote from FP doc>"
   Counter-example in code: <file>:<line>
   ```apex
   <code snippet showing the mismatch>
   

   Fix: <what to do — usually fix the code to match the claim, OR rewrite the claim to match what the code actually does>.

   [WEAK JUSTIFICATION] ...

   [MISSING] ...

   
   Save to `appexchange-artifacts/security-review/fp-doc-audit-YYYY-MM-DD.md` if the artifacts folder exists; otherwise save to `~/Desktop/fp-doc-audit-YYYY-MM-DD.md`. Tell the user which path was used.
   

Known recurring anti-claims

These have appeared in prior FP docs and must be flagged on sight:

1. "All read queries use WITH USER_MODE" — almost always overstated. Verify by grep over every [SELECT/Database.query in the file. If even one query lacks it, the claim is stale.

2. "Users without the permission set cannot access the objects at all, so FLS checks are redundant" — this is Salesforce-logically defensible but not accepted by reviewers. They want programmatic field-level enforcement regardless. Always flag [WEAK JUSTIFICATION].

3. "Users can only reach this via the UI / LWC" — wrong premise. @AuraEnabled and REST endpoints are callable from any authenticated Apex-capable context, not just your UI. Always [WEAK JUSTIFICATION].

4. "CRUD checks are performed in the calling class" — trace actually. Often true for one call site, false for another. Verify all call sites.

5. "Declared without sharing because system action" — acceptable only for InstallHandler, @InvocableMethod driven by Flow (document why Flow triggers a system-context need), Queueable/Batchable background jobs. For a selector serving @AuraEnabled reads, this justification is not enough — an authorization gate is required.

Heuristics

• If the FP doc's total High+Critical count equals the scanner's total → highly suspicious. Zero real issues is rare; more likely every finding got rationalized.
• If a justification contains the word "only" (as in "users only…", "runs only…"), pause and verify the "only" claim holds across every path.
• If a justification references behaviour of a caller or UI rather than a check in this file, it is a [WEAK JUSTIFICATION] unless the file itself has a concrete gate.

Output contract

Always produce the report file even if everything verifies. The file is the audit trail. Summarize for the user in chat: counts per category, path to the saved report, and one-line verdict (READY / NEEDS WORK / BLOCKED).