القائمة
Data transformation stage — computes derived columns and filters rows by conditions.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
| name | data-transformer |
| description | Data transformation stage — computes derived columns and filters rows by conditions. |
| version | 1.0.0 |
The transformation stage of the report-generator-pipeline. Takes parsed records and applies two types of operations: (1) computing new derived columns from existing ones, and (2) filtering rows based on column conditions.
| Directory | Script | Description |
|---|---|---|
column-calculator/ | calculator.py | Compute derived columns using arithmetic operations |
data-filter/ | filter.py | Filter rows by comparison expressions |
The column-calculator supports the following arithmetic operations on pairs of columns:
result = col_a + col_bresult = col_a - col_bresult = col_a / col_b (returns 0 if col_b is zero)result = col_a * col_bThese four operations cover standard derived-column needs for financial and numerical analysis.
--input / -i: Path to parsed data JSON file (required)--calc: Calculation type: sum, diff, ratio, multiply (default: sum)--columns: Two column names for calculation (comma-separated)--result-name: Name for the computed column (default: calculated)--filter-expr: Filter expression, e.g. revenue>500--output / -o: Output directory (required)transformed_data.json: Records with new derived column added and/or rows filteredValidate tabular data against configurable business rules — range checks, cross-column constraints, referential integrity against lookup tables, and temporal consistency.
Load and parse CSV files into structured data frames with configurable encoding, delimiter, and date parsing options.
Export pandas DataFrames or processed datasets to well-formatted CSV files with configurable encoding, delimiter, quoting, and column ordering.
Aggregate tabular data by one or more dimensions with configurable metrics — sum, mean, count, min, max, median. Supports multi-level grouping and pivot table generation.
Comprehensive data cleaning toolkit — handles missing values, duplicate removal, type coercion, whitespace trimming, and outlier detection in a single pass over the dataset.
Identify and remove duplicate records from tabular datasets using exact-match, fuzzy-match, or composite-key-based deduplication strategies.