| name | data_processing |
| description | Process and analyze structured data. Use when working with CSV, JSON, or other structured data formats for transformation, filtering, or analysis. |
Data Processing Skill
This skill enables you to process, transform, and analyze structured data in various formats including CSV, JSON, and text files.
When to Use This Skill
Use this skill when:
- Reading or parsing CSV, JSON, or structured data files
- Transforming data between formats
- Filtering or sorting data records
- Calculating statistics or aggregations
- Generating summaries or reports from data
- Handling large datasets efficiently
Capabilities
- Read and parse CSV, JSON, and other structured formats
- Transform data between formats
- Filter and sort data
- Calculate statistics and aggregations
- Generate summaries and reports
- Handle large datasets efficiently
Guidelines
- First understand the data structure by reading a sample
- Use appropriate parsing for each format:
- CSV: Split by delimiter, handle quoted fields
- JSON: Parse with JSON library or tools
- Fixed-width: Extract by character positions
- For large files, read in chunks or use line-based processing
- Validate data before processing
- Report data quality issues (missing values, format errors)
- Provide summary statistics when appropriate
Common Operations
Data Filtering
- Filter rows by condition
- Select specific columns
- Remove duplicates
Data Transformation
- Convert between formats
- Rename columns
- Calculate derived fields
Data Analysis
- Count records
- Calculate averages/sums
- Find min/max values
- Identify trends
Example Usage
User: "Process this CSV and count unique values in column 2"
Your approach:
- Read the CSV file
- Extract column 2 values
- Use shell command to count unique values
- Present results
User: "Convert this JSON to CSV"
Your approach:
- Read the JSON file
- Parse JSON structure
- Transform to CSV format
- Write to output file
Limitations
- Large files may require chunked processing
- Complex transformations may need multiple steps
- No built-in statistical analysis library
- Shell commands limited to available tools
