data-transformer

name	data-transformer
description	Data transformation stage — computes derived columns and filters rows by conditions.
version	1.0.0

Data Transformer

Skill Summary

The transformation stage of the report-generator-pipeline. Takes parsed records and applies two types of operations: (1) computing new derived columns from existing ones, and (2) filtering rows based on column conditions.

Sub-Modules

Directory	Script	Description
column-calculator/	calculator.py	Compute derived columns using arithmetic operations
data-filter/	filter.py	Filter rows by comparison expressions

Available Calculation Types

The column-calculator supports the following arithmetic operations on pairs of columns:

• sum: result = col_a + col_b
• diff: result = col_a - col_b
• ratio: result = col_a / col_b (returns 0 if col_b is zero)
• multiply: result = col_a * col_b

These four operations cover standard derived-column needs for financial and numerical analysis.

Inputs

• --input / -i: Path to parsed data JSON file (required)
• --calc: Calculation type: sum, diff, ratio, multiply (default: sum)
• --columns: Two column names for calculation (comma-separated)
• --result-name: Name for the computed column (default: calculated)
• --filter-expr: Filter expression, e.g. revenue>500
• --output / -o: Output directory (required)

Output

• transformed_data.json: Records with new derived column added and/or rows filtered