一键在 Manus 中运行任何 Skill

data-validation

星标23

分支7

更新时间2026年5月21日 12:48

Validate tabular data against JSON schemas and configurable business rules — column presence, type checks, nullable/unique constraints, range rules, referential integrity against lookup tables, and temporal consistency.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

在 Manus 中运行

来源

Mosi-AI

Mosi-AI/LiveClawBench

打开 GitHub 仓库查看创作者相关仓库

下载

在 Manus 中运行

Data Validation

Skill Summary

Validates dataset records against both structural schema definitions and domain-specific business rules. Checks column presence, data types, nullable/unique constraints, regex patterns (schema layer), plus numeric range checks, cross-column constraints, referential integrity against external catalog/lookup tables, and date range validation (business rule layer). Records failing validation are flagged, rejected, or separated.

Merges functionality from: schema_validator, business_rule_checker.

Validation Layers

Schema Validation

Required columns: All declared columns must exist
Type checking: Values must match declared type (string, int, float, date, bool)
Nullable constraints: Non-nullable columns must have zero nulls
Uniqueness: Unique columns must have no duplicate values
Pattern matching: String values must match declared regex
Strict mode: Flag extra columns not in schema

Business Rule Validation

Range checks: Numeric columns within [min, max] or > 0
Cross-column constraints: Expressions involving multiple columns (e.g., end_date >= start_date)
Referential integrity: Column values must exist in external lookup table (e.g., product_id in product catalog)
Date range: Date columns within expected fiscal period
Custom regex: String columns matching business patterns

Inputs

--input / -i: Path to dataset (CSV or Parquet) (required)
--schema / -s: Path to JSON schema file
--rules / -r: Path to business rules config (JSON or YAML)
--lookup-dir: Directory containing lookup tables for referential checks
--output / -o: Output directory
--reject-action: flag, reject, or separate (default: flag)
--strict: Fail on extra columns beyond schema
--max-errors: Stop after N errors (default: unlimited)

Output

valid_records.csv: Records passing all checks
rejected_records.json: Failed records with violation details and reasons
validation_report.json: Overall pass/fail, per-column results, per-rule counts
rule_summary.json: Per-rule pass/fail percentages

Implementation

Command: python3 ./skills/data_validation/validate.py -i <file> -s <schema> -r <rules> -o <output_dir>
Dependencies: pandas, numpy, jsonschema

同仓库更多 Skills

同仓库

business-rule-checker

Mosi-AI/LiveClawBench

Validate tabular data against configurable business rules — range checks, cross-column constraints, referential integrity against lookup tables, and temporal consistency.

2026-05-2123

csv-data-loader

Mosi-AI/LiveClawBench

Load and parse CSV files into structured data frames with configurable encoding, delimiter, and date parsing options.

2026-05-2123

csv-export-writer

Mosi-AI/LiveClawBench

Export pandas DataFrames or processed datasets to well-formatted CSV files with configurable encoding, delimiter, quoting, and column ordering.

2026-05-2123

data-aggregator

Mosi-AI/LiveClawBench

Aggregate tabular data by one or more dimensions with configurable metrics — sum, mean, count, min, max, median. Supports multi-level grouping and pivot table generation.

2026-05-2123

data-cleaning-toolkit

Mosi-AI/LiveClawBench

Comprehensive data cleaning toolkit — handles missing values, duplicate removal, type coercion, whitespace trimming, and outlier detection in a single pass over the dataset.

2026-05-2123

data-deduplicator

Mosi-AI/LiveClawBench

Identify and remove duplicate records from tabular datasets using exact-match, fuzzy-match, or composite-key-based deduplication strategies.

2026-05-2123

name	data-validation
description	Validate tabular data against JSON schemas and configurable business rules — column presence, type checks, nullable/unique constraints, range rules, referential integrity against lookup tables, and temporal consistency.
version	1.0.0

Data Validation

Skill Summary

Merges functionality from: schema_validator, business_rule_checker.

Validation Layers

Schema Validation

Required columns: All declared columns must exist
Type checking: Values must match declared type (string, int, float, date, bool)
Nullable constraints: Non-nullable columns must have zero nulls
Uniqueness: Unique columns must have no duplicate values
Pattern matching: String values must match declared regex
Strict mode: Flag extra columns not in schema

Business Rule Validation

Range checks: Numeric columns within [min, max] or > 0
Cross-column constraints: Expressions involving multiple columns (e.g., end_date >= start_date)
Referential integrity: Column values must exist in external lookup table (e.g., product_id in product catalog)
Date range: Date columns within expected fiscal period
Custom regex: String columns matching business patterns

Inputs

--input / -i: Path to dataset (CSV or Parquet) (required)
--schema / -s: Path to JSON schema file
--rules / -r: Path to business rules config (JSON or YAML)
--lookup-dir: Directory containing lookup tables for referential checks
--output / -o: Output directory
--reject-action: flag, reject, or separate (default: flag)
--strict: Fail on extra columns beyond schema
--max-errors: Stop after N errors (default: unlimited)

Output

valid_records.csv: Records passing all checks
rejected_records.json: Failed records with violation details and reasons
validation_report.json: Overall pass/fail, per-column results, per-rule counts
rule_summary.json: Per-rule pass/fail percentages

Implementation

Command: python3 ./skills/data_validation/validate.py -i <file> -s <schema> -r <rules> -o <output_dir>
Dependencies: pandas, numpy, jsonschema