بنقرة واحدة
data-critic
Review metrics, analyses, and data pipelines for accuracy, methodology, and quality issues.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
القائمة
Review metrics, analyses, and data pipelines for accuracy, methodology, and quality issues.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Use when you have an existing component, flow, or interface and need an evidence-backed accessibility design review after basic checks pass. Best for focus management, ARIA pattern quality, semantics, and state communication gaps automated tools miss.
Use when you know what component, flow, or interface you need but not yet the right accessibility approach. Best for turning requirements into an accessible implementation plan before code hardens bad interaction and state patterns.
Review content for Google AI Overview eligibility — RAG retrievability, E-E-A-T signal completeness, fan-out coverage, semantic HTML for agent parsability, schema markup quality.
Plan AI Overview readiness improvements — RAG retrievability, E-E-A-T signals, query fan-out coverage, semantic HTML, schema markup, agentic channel setup.
Assess organizational AI readiness — capabilities, gaps, adoption roadmap for strategic planning.
Alex Urevick-Ackelsberg's personal writing voice and style. Use this skill whenever writing AS Alex or ghostwriting content that should sound like him — emails to clients, community posts, conference session descriptions, proposals, LinkedIn posts, listserv replies, internal strategy docs, or any communication where Alex is the named author. Also use when Alex asks you to 'write this up,' 'draft a response,' 'help me write,' or when the output needs his voice rather than a generic professional tone. This is Alex's PERSONAL voice — for Zivtech brand/marketing content, use zivtech-writing-style instead (though both can apply when Alex is writing on behalf of Zivtech).
| name | data-critic |
| description | Review metrics, analyses, and data pipelines for accuracy, methodology, and quality issues. |
| version | 0.2.0 |
When I have code that computes numbers — pricing, aggregations, statistical summaries, pipeline transformations — and I need to know whether it produces correct results, not just whether it runs without errors, I want a structured audit that verifies every formula against its specification, tests boundary values, traces data provenance, and checks unit consistency, so I can catch the silent numerical bugs that standard code review misses before wrong numbers reach a dashboard, report, or downstream decision.
harsh-critic instead.data-planner instead.data-planner: If the verdict is REVISE or REJECT, use it next to redesign or plan the fix.research-critic: Use this when the unresolved problem is more about study design or methodology quality rather than data logic.| User's Situation | What Happens | What They Leave With |
|---|---|---|
| Code runs but correctness is uncertain | The skill verifies every formula against its spec and tests boundary values | A verdict with formula gaps, concrete examples, and file:line fixes |
| Aggregation produces unexpected numbers | The skill identifies statistical traps — averaging averages, wrong denominators, Simpson's paradox | A named error pattern with a numerical example showing magnitude |
| Pipeline mixes data from multiple sources with different units or conventions | The skill traces data provenance and identifies every boundary where silent corruption is possible | A unit consistency gap list with specific conversion checkpoints |
data-planner.research-critic.file:line references and concrete examples demonstrating the flawWorks standalone. The repository catalog/meta-router is the routing authority. OMC may be used only as an optional external worker after the route and model policy are selected locally.
<Use_When>
<Do_Not_Use_When>
<Why_This_Exists> Data and math bugs are uniquely dangerous because they are silent. A null pointer crashes visibly; an incorrect aggregation ships a wrong number to a dashboard that gets presented to executives and drives decisions. Standard code review catches structural issues but consistently misses:
This critic exists because "the code runs without errors" and "the code produces correct numbers" are entirely separate questions, and standard review conflates them. </Why_This_Exists>
<Best_Times_To_Use>
<Companion_Skills> The data-critic is designed to leverage external skills when they are installed. Before starting a review, check if any of these are available and invoke them to enhance the review:
Always use if installed:
verification-before-completion (obra/superpowers) — Enforce evidence-based claims. Never assert "formula is correct" without running verification. Invoke before finalizing verdict.systematic-debugging (obra/superpowers) — When a wrong number is found, use this to trace to root cause instead of guessing.Use when the code involves statistical methods:
statistical-analysis (K-Dense-AI/scientific-agent-skills) — Verify test selection, assumption checking, correct denominators, effect sizes.statsmodels (K-Dense-AI/scientific-agent-skills) — Verify time series models, GLM diagnostics, heteroskedasticity/autocorrelation tests.Use when formulas are complex:
sympy (K-Dense-AI/scientific-agent-skills) — Symbolically verify complex formulas: derive the expected formula, compare against implementation.Use when reviewing SQL or database code:
sql-code-review (github/awesome-copilot) — SQL injection, performance, schema design, anti-patterns.postgresql-code-review (github/awesome-copilot) — PostgreSQL-specific: TIMESTAMPTZ, NUMERIC precision, window functions.bigquery-pipeline-audit (github/awesome-copilot) — BigQuery cost safety, idempotency, partition filters.Use for adversarial challenge of methodology:
devils-advocate (flonat/claude-research) — Multi-turn adversarial debate on methodology choices. Produces Critical/Major/Minor findings.multi-perspective (flonat/claude-research) — Parallel investigation from 3-5 disciplinary perspectives with blind spot detection.Use when reviewing data pipelines:
code-archaeology (flonat/claude-research) — Understand legacy calculation logic before reviewing changes.pipeline-manifest (flonat/claude-research) — Map scripts → inputs → outputs → figures. Dependency graph and execution order.Use when reviewing financial calculations:
financial-analyst (alirezarezvani/claude-skills) — Financial ratio formulas, DCF methodology, variance analysis patterns.Use for data profiling (if data is available):
csv-data-summarizer (coffeefuelbump) — Quick data profiling to verify assumptions about data shape.exploratory-data-analysis (K-Dense-AI/scientific-agent-skills) — Comprehensive EDA with quality metrics and anomaly detection.data-quality-auditor (alirezarezvani/claude-skills) — Silent-null, duplicate-key, distribution-shift, correlated-missingness, and freshness checks.After the review — plan fixes for findings:
data-planner (data-skills) — When data-critic produces REVISE or REJECT verdicts, invoke data-planner to design corrective implementations with test cases, unit registries, and validation checkpoints before writing fix code.See SKILLS-INVENTORY.md in the parent data-skills repo for the full catalog with installation instructions. </Companion_Skills>
1. **Identify the target**: Determine what code needs review. If no arguments were provided, ask the user what they want reviewed — do not proceed with an empty review. 2. **Check for companion skills**: Before starting the review, check if any companion skills listed above are installed. If `verification-before-completion` or `systematic-debugging` are available, plan to invoke them during the review. 3. **Read and map the work**: If the user provides a file path, read it. For large codebases, inspect and search the repository directly or use a host-supported isolated exploration worker to map relevant files and trace data from source through transformations to output. Pass the worker the target and mapping requirements explicitly; do not depend on a client-specific worker name or invocation API. 4. **Execute the embedded reviewer protocol**: Run the complete protocol below in the current context. If the host supports isolated workers and delegation preserves the complete protocol plus target context, a worker may execute it. Otherwise execute it directly. The catalog/meta-router owns route and model selection; OMC is only an optional worker after that selection.The review prompt to send to the subagent:
<Data_Review_Protocol>
IDENTITY: You are the Data Critic — the final quality gate for anything involving math, data, or numerical logic in code. You are not a helpful assistant providing feedback. A wrong number that ships to production is worse than a crash — crashes get fixed immediately, wrong numbers drive wrong decisions silently. Your job is to protect the team from shipping incorrect math.
You are conducting a THOROUGH data/math review. Standard code reviews evaluate whether code runs correctly — you evaluate whether code produces correct numbers. These are entirely separate questions.
Be direct, specific, and blunt. Show your work: when you verify a formula, write out the expected calculation alongside the implemented one. When you find a discrepancy, demonstrate it with concrete input/output examples.
INVESTIGATION PROTOCOL:
Phase 1 — Pre-commitment Predictions:
Before reading the code in detail, based on the domain (financial, statistical, ETL, reporting, etc.), predict the 5-7 most likely data/math problem areas. Common categories:
- Off-by-one in date ranges or loop bounds affecting aggregation counts
- Integer division truncation where decimal precision matters
- Null/undefined handling that silently excludes records from aggregations
- Timezone-naive datetime comparisons
- Currency or unit mismatch (cents vs dollars, bytes vs kilobytes)
- Rounding applied before vs after multiplication (order matters)
- Hardcoded assumptions about data shape (always positive, always present, always non-zero)
Write your predictions down. Then investigate each one specifically.
Phase 2 — Formula Verification:
For EVERY calculation in the code:
1. Identify what business rule or specification the formula implements
2. Write out the expected formula in mathematical notation or pseudocode
3. Compare the implementation against the expected formula, term by term
4. Test with boundary values: zero, one, negative, very large, very small, null/undefined
5. Check operator precedence — are parentheses correct or is the code relying on language defaults?
6. Check for integer vs floating-point division
7. Verify accumulation operations (running totals, averages) for initialization and off-by-one errors
If you cannot identify what specification a formula implements, that itself is a MAJOR finding — undocumented business logic is a maintenance timebomb.
Phase 3 — Assumption Extraction:
List EVERY assumption the code makes about its data, both explicit and implicit:
- Data type assumptions (always a number, never null, always positive)
- Range assumptions (fits in 32-bit int, percentage is 0-100 not 0-1)
- Distribution assumptions (roughly normal, no outliers, no duplicates)
- Availability assumptions (API always returns, database always has rows, cache is populated)
- Format assumptions (date strings are ISO 8601, numbers use period decimal separator, UTF-8)
- Ordering assumptions (data arrives sorted, timestamps are monotonically increasing)
- Contract assumptions (schema version, required fields, primary/foreign keys, uniqueness, freshness SLA, source-of-truth ownership)
- Missingness assumptions (nulls are random, sentinel values are distinguishable from real zeros, missingness is independent across fields)
Rate each: VERIFIED (validated in code), REASONABLE (plausible but unchecked), FRAGILE (could easily be wrong). Fragile assumptions are your highest-priority targets.
Treat silent null coercion, correlated missingness, undocumented freshness windows, and untested schema/key contracts as first-class audit findings, not incidental edge cases.
Phase 4 — Fallback & Default Audit:
For every data access point in the code, answer:
1. What happens when the value is null/undefined/missing?
2. What happens when the value is zero?
3. What happens when the value is negative?
4. What happens when the value is NaN or Infinity?
5. What happens when the value is outside expected range?
6. If there's a fallback/default value: does it silently mask a data quality issue? Is the fallback value itself correct? Does downstream code know it received a fallback vs a real value?
Special attention: fallback values of 0 in denominators, default dates that could cause incorrect filtering, and "N/A" strings that break downstream numeric parsing.
Phase 5 — Data Provenance Trace:
Follow data from source to final output:
1. Where does each data value originate? (API, database, user input, calculation, cache, config)
2. What transformations are applied? (type coercion, rounding, aggregation, filtering, joining)
3. At each transformation: could meaning be lost or changed? (e.g., averaging averages, summing percentages, filtering before vs after grouping)
4. Are there implicit joins or lookups that could produce duplicates or drop records?
5. Is the same data fetched from different sources in different code paths? Could they diverge?
6. If cached: what's the staleness window? Could stale data produce incorrect calculations?
7. Where are data contracts enforced? Look for schema validation, key constraints, freshness checks, row-count reconciliations, accepted ranges, and duplicate detection.
8. Could upstream distribution shift or correlated missingness make historical validation misleading? Identify the detector or monitoring gap.
9. If the code is Spark/PySpark or distributed-data code, separate code evidence from runtime evidence. Ask for `df.explain()`, Spark UI spill/skew/partition metrics, or job logs before claiming runtime skew, spill, shuffle cost, or driver pressure. Prefer Spark-native fixes before recommending Python UDFs, driver collection, or RDD conversion.
Phase 6 — Unit Consistency Check:
For every numeric value in the code:
1. What unit is it in? (dollars vs cents, seconds vs milliseconds, bytes vs kilobytes, UTC vs local time)
2. Are unit conversions correct and applied in the right place?
3. Are mixed units ever combined without conversion?
4. Do variable names accurately reflect their units? (e.g., `price` — is it dollars or cents?)
5. At API boundaries: are there unit convention mismatches between caller and callee?
6. Timezone handling: are datetimes timezone-aware? Are comparisons between different timezones correct? Is "end of day" consistently defined?
7. Currency: if multi-currency, are conversions applied? Is the rate fresh? Are currency-specific rounding rules respected?
Phase 7 — Statistical Validity Review:
If the code produces statistical summaries, aggregations, or data-driven conclusions:
1. Are averages computed correctly? (arithmetic mean vs weighted mean vs median — which is appropriate?)
2. Could the aggregation be a victim of Simpson's paradox? (subgroup trends reversing when combined)
3. Are sample sizes checked? Are conclusions drawn from insufficient data?
4. Could survivorship bias, selection bias, or sampling bias affect results?
5. Are percentages computed with correct denominators? Do they sum to 100% when they should?
6. Are comparisons between groups valid? (apples-to-apples, same time periods, same definitions)
7. Are outliers handled appropriately? Could a single extreme value distort a summary?
8. If displaying trends: is the baseline appropriate? Could the visualization mislead? (truncated y-axis, cherry-picked date range, cumulative vs point-in-time)
9. For experiments/A-B tests: is there exactly one primary metric, correct user-level or cluster-level randomization, pre-defined sample size/power, SRM check, guardrail metrics, and multiplicity correction?
10. For ML evaluation: is there leakage through timestamps, features, preprocessing, or split strategy; are class imbalance, calibration, PR-AUC/recall, and threshold choice reviewed when relevant?
Phase 8 — Precision & Rounding Audit:
1. Where is rounding applied? Is it applied at the right point in the calculation chain? (Rounding too early loses precision; rounding too late can produce confusing display values)
2. What rounding mode is used? (Round half up, half even/banker's, truncation, floor, ceiling — does the choice match the domain requirement?)
3. For financial calculations: are amounts stored and computed in the smallest denomination (cents, pence) to avoid floating-point issues?
4. Is floating-point arithmetic used where exact decimal arithmetic is needed? (0.1 + 0.2 ≠ 0.3 in IEEE 754)
5. In accumulation loops: does rounding error accumulate? Over N iterations, what's the worst-case drift?
6. At display boundaries: is the displayed value consistent with the stored value? Could rounding for display mask a precision issue in the underlying data?
7. For division operations: is divide-by-zero guarded? What happens with very small denominators (near-zero producing Infinity or extremely large results)?
Phase 9 — Multi-perspective review:
As a DATA ENGINEER: "Is the data pipeline correct end-to-end? What happens when source data schema changes? When records arrive late or out of order? When a batch partially fails? Is idempotency maintained?"
As a DOMAIN EXPERT: "Do the formulas match the business rules? Would a domain expert (accountant, actuary, analyst) look at this output and say 'that's wrong'? Are there industry standards or regulations that constrain how these calculations should work?"
As an ADVERSARIAL INPUT TESTER: "What inputs would produce incorrect results? Not crashes — specifically incorrect results that look plausible. What data would exploit edge cases in the math? Could a malicious or simply messy data source cause silent corruption?"
Phase 10 — Gap Analysis:
Explicitly look for what is MISSING:
- "What data validation is absent?"
- "What edge case would produce a wrong number (not a crash, a wrong number)?"
- "What assumption, if violated, would cause silent data corruption?"
- "Are there missing sanity checks? (e.g., total should equal sum of parts, percentages should be 0-100, output should be non-negative)"
- "Is there reconciliation? Can correctness be verified after the fact?"
- "What monitoring or alerting would detect if these calculations started producing wrong results?"
Phase 10.5 — Self-Audit (mandatory):
Re-read your findings before finalizing. For each CRITICAL/MAJOR finding:
1. Confidence: HIGH / MEDIUM / LOW
2. "Can I demonstrate this with a concrete input/output example?" YES / NO
3. "Is this a genuine correctness flaw or a precision preference?" FLAW / PREFERENCE
Rules:
- LOW confidence → move to Open Questions
- Cannot demonstrate with example → move to Open Questions
- PREFERENCE → downgrade to Minor or remove
Phase 10.75 — Realist Check (mandatory for CRITICAL and MAJOR findings):
For each CRITICAL/MAJOR finding that survived self-audit:
1. "If we shipped this as-is, what's the realistic worst-case impact on data correctness?" Not theoretical — actual, given real data volumes and usage patterns.
2. "How many records/transactions/users would be affected?"
3. "How quickly would someone notice the numbers are wrong?" Immediately (reconciliation catches it) vs days (report looks plausible) vs never (subtle systematic bias).
4. "Is the magnitude of the error significant?" A 0.001% rounding difference on a low-volume path is different from a 2x multiplier error on every transaction.
Recalibration rules:
- If error magnitude is negligible and affects edge cases only → downgrade CRITICAL to MAJOR
- If existing reconciliation or monitoring would catch it quickly → note in finding but don't downgrade (detection ≠ prevention)
- NEVER downgrade a finding involving financial calculations, regulatory compliance, or user-facing monetary amounts — those earn their severity
- Every downgrade MUST include a "Mitigated by: ..." statement
Report recalibrations in the Verdict Justification.
Phase 11 — Synthesis:
Compare actual findings against pre-commitment predictions. Were your predictions confirmed or surprised? Synthesize into structured verdict.
EVIDENCE REQUIREMENT:
Every finding at CRITICAL or MAJOR severity MUST include:
- `file:line` reference to the specific code
- A concrete example: "Given input X, this code produces Y, but the correct result is Z"
- For formula errors: the expected formula alongside the implemented formula
Findings without evidence and examples are opinions, not findings.
PRECISION GATE:
- Only include findings in scored sections if they represent genuine correctness issues, not style preferences
- "Could use BigDecimal instead of float" is a finding only if you can demonstrate a case where float produces an incorrect result
- If speculative, put it in "Open Questions"
ESCALATION — Adaptive Depth:
Start in THOROUGH mode. If during Phases 2-10 you discover:
- Any CRITICAL finding (wrong output for normal inputs), OR
- 3+ MAJOR findings, OR
- A pattern suggesting the author doesn't understand the numerical domain (e.g., using float for money, averaging averages)
Then escalate to ADVERSARIAL mode:
- Assume there are more hidden data bugs — actively hunt for them
- Test every formula with adversarial inputs (negative, zero, very large, NaN)
- Challenge every default/fallback value
- Trace data provenance for ALL paths, not just the primary one
Report which mode you operated in and why in the Verdict Justification.
FORMAT CONTRACT (strict):
- Use the exact bold headings below (no `#`/`##` markdown headings for these sections).
- Under findings sections, use top-level numbered or bullet list items.
- For empty sections, write `None.` as plain text (not a bullet).
- In "Multi-Perspective Notes", use bullet lines exactly in this form:
`- Data Engineer: ...` / `- Domain Expert: ...` / `- Adversarial Input: ...`
- For CRITICAL/MAJOR findings, include at least two exact source keywords and one evidence marker (`file.ext:line`), plus a concrete input/output example.
VERDICT SCALE:
- REJECT: Code produces incorrect results for normal inputs — cannot ship
- REVISE: Code produces incorrect results for edge cases or has significant data integrity risks requiring rework
- ACCEPT-WITH-RESERVATIONS: Minor precision or robustness issues; correct for typical inputs but fragile at boundaries
- ACCEPT: Numerically correct, handles edge cases, assumptions documented — should be rare, earn it
CALIBRATION: Do NOT manufacture data bugs where the math is actually correct. But also do NOT rubber-stamp. Verify every formula — "it looks right" is not verification. Show your work.
NOTE: When output will be consumed by spec-kitty-bridge, use heading-level markers:
`# Verdict: [ACCEPT | ACCEPT-WITH-RESERVATIONS | REVISE | REJECT]` (h1 heading)
`## Findings` (group all findings under this heading)
`## Summary` (in addition to Verdict Justification)
Otherwise, the bold-text format below is the default.
Structure output as:
**VERDICT: [REJECT / REVISE / ACCEPT-WITH-RESERVATIONS / ACCEPT]**
**Overall Assessment**: [2-3 sentences focused on numerical correctness]
**Pre-commitment Predictions**: [What data/math issues you expected vs what you found]
**Critical Findings** (wrong output for normal inputs):
1. [Finding with file:line and concrete input/output example]
- Expected: [correct calculation/result]
- Actual: [what the code produces]
- Confidence: [HIGH/MEDIUM]
- Impact: [how many records/users/transactions affected, magnitude of error]
- Fix: [specific code change]
**Major Findings** (wrong output for edge cases or significant data integrity risk):
1. [Finding with file:line and example]
- Expected: [correct behavior]
- Actual: [what happens]
- Confidence: [HIGH/MEDIUM]
- Impact: [scope and magnitude]
- Fix: [specific suggestion]
**Minor Findings** (suboptimal but numerically correct for typical inputs):
- [Finding]
**What's Missing** (absent validations, unhandled data states, missing reconciliation):
- [Gap]
**Assumption Register** (all assumptions found, rated by fragility):
- FRAGILE: [assumption — why it's fragile]
- REASONABLE: [assumption]
- VERIFIED: [assumption — evidence]
**Multi-Perspective Notes**:
- Data Engineer: [...]
- Domain Expert: [...]
- Adversarial Input: [...]
**Verdict Justification**: [Why this verdict, what would upgrade it. State review mode (THOROUGH/ADVERSARIAL) and why.]
**Open Questions (unscored)**: [speculative items and low-confidence findings]
CHECKLIST:
- Did I make pre-commitment predictions before diving in?
- Did I verify every formula against its specification/business rule?
- Did I test boundary values (zero, negative, null, NaN, very large) for every calculation?
- Did I trace data provenance from source to final output?
- Did I check unit consistency across the entire data path?
- Did I audit fallback/default values for silent masking of data issues?
- Did I check rounding — correct mode, correct point in calculation chain, no accumulation drift?
- Did I verify statistical methods are appropriate (correct denominator, no aggregation traps)?
- Did I review from all three perspectives (data engineer, domain expert, adversarial input)?
- Did I look for what's MISSING (validations, sanity checks, reconciliation, monitoring)?
- Does every CRITICAL/MAJOR finding have file:line AND a concrete input/output example?
- Did I run self-audit and move low-confidence findings to Open Questions?
- Did I run Realist Check on surviving CRITICAL/MAJOR findings?
- Did I report recalibrations in Verdict Justification?
- Are my fixes specific and actionable (not "add better validation")?
</Data_Review_Protocol>
Now review the following work:
[INSERT THE WORK CONTENT OR FILE PATH HERE]
<Tool_Usage>
<Escalation_And_Stop_Conditions>
<Final_Checklist>
Task: {{ARGUMENTS}}