| name | portaljs-check-data-quality |
| description | Audit a local or remote tabular file (CSV/TSV) for common data quality issues — schema, nulls, types, duplicates. Read-only. Use when a dataset needs a quality check before publishing, or a showcase renders wrong (blank cells, garbled numbers, an unsortable date column) and the cause needs isolating. |
| allowed-tools | Bash(curl:*), Bash(awk:*), Bash(sort:*), Bash(head:*), Bash(wc:*) |
| version | 1.0.0 |
| author | Datopian <hello@datopian.com> |
| license | MIT |
| compatibility | Claude Code with PortalJS portals (Next.js 14, React 18, Node 18+). Runs from any project via the plugin, a personal ~/.claude/commands install, or a portaljs clone. |
| tags | ["portaljs","data-portal","data-quality","audit","csv","validation"] |
PortalJS — Check Data Quality
Overview
Run a read-only quality audit of one CSV or TSV file, local or remote, and return a
structured JSON report. The audit profiles every column — null/blank counts, inferred
value types, numeric ranges, likely year/date fields — and flags duplicate rows,
duplicate values in identifier-like columns, ambiguous overlapping year columns (e.g.
calendar year vs fiscal year), and mixed-type columns. It never edits the source
file, datasets.json, or any other project file; it only reads the target file (a
remote URL is downloaded to a temp file that is deleted before the run ends) and
prints a report. Use it before publishing a dataset with portaljs-add-dataset, or to
diagnose why a showcase renders wrong.
Prerequisites
python3 on PATH — the audit logic runs as an embedded Python script; nothing is
installed.- One CSV or TSV file, given as a local path or an
http/https URL. Only one file
per run.
Instructions
The canonical, full step-by-step workflow is
.claude/commands/portaljs-check-data-quality.md —
the single source of truth. Read and follow it when executing. Summary:
- Gather input — the file path or URL to audit. If missing, ask for it; never dead-end.
- Resolve the source: if it's an
http/https URL, download it to a temp file first;
otherwise use the local path as given.
- Validate the extension is
.csv or .tsv. If not, or the file is missing, or the
header row is empty, stop and surface the error JSON as-is — do not guess a fix.
- Profile every column: null/blank counts, distinct values, sample values, inferred
per-value type (boolean/integer/float/date/string), numeric min/max, and year
range for columns whose name looks year-like.
- Derive findings from the profiles — duplicate rows, missing-value ratios, invalid
year values, mixed types, suspect negative values, duplicate identifier values, and
ambiguous overlapping year columns — each tagged
critical, warning, or info.
- Assemble the JSON report (
status, file metadata, findings, recommendations,
column_profiles), print it, and clean up the temp file if one was created.
- Relay the report to the user as-is; do not modify the source file,
datasets.json,
or any other project file based on the findings — that's a separate, explicit step.
Output
A single JSON object printed to stdout:
status — ok, warning, or critical.file, file_name, source_type (local or url), row_count, column_count.findings — structured issues, most severe first.recommendations — de-duplicated suggested next steps.column_profiles — per-column summary (nulls, blanks, distinct count, sample
values, inferred types, numeric/year ranges).
No files are created or modified. A remote URL's temp download is removed on exit,
success or failure alike.
Error Handling
| Symptom | Cause | Fix |
|---|
"File ... is not available." | Local path is wrong, or the URL download failed | Verify the path or URL is reachable and retry. |
"Only CSV and TSV files are supported right now." | File extension isn't .csv/.tsv | Convert the file, or point to its tabular source instead. |
"... does not contain tabular headers." | File is empty or the header row is malformed | Open the file and confirm it has a valid, non-empty header line. |
| Command hangs on a URL | Remote host is slow or blocks non-browser requests | Download the file manually and audit the local copy instead. |
python3: command not found | Python 3 isn't installed or not on PATH | Install Python 3, or run the audit where it's available. |
| Report looks truncated in the terminal | Large report wrapped/paginated by the shell | Redirect to a file (> report.json) and open it separately. |
Examples
Example 1 — Audit a local CSV before publishing
/portaljs-check-data-quality ./public/data/trash.csv
Example 2 — Audit a remote CSV over HTTPS
/portaljs-check-data-quality https://example.com/trash.csv
Example 3 — Audit a TSV and save the report for review
bash scripts/check-data-quality.sh ./data/emissions.tsv > /tmp/emissions-quality.json
Example 4 — Read a critical status report
{
"status": "critical",
"findings": [
{ "severity": "critical", "check": "duplicate_rows", "message": "42 duplicate rows found." }
],
"recommendations": ["Review and deduplicate repeated rows if they are not intentional."]
}
Fix the flagged rows/columns, then re-run the audit before publishing.
Resources
