Performs exploratory data analysis (EDA) on datasets from CKAN portals and CSV files. Use when analyzing datasets, checking data quality, exploring CSV files, or when the user asks to examine, analyze, or validate data.
Performs exploratory data analysis (EDA) on datasets from CKAN portals and CSV files. Use when analyzing datasets, checking data quality, exploring CSV files, or when the user asks to examine, analyze, or validate data.
This skill provides comprehensive capabilities for Exploratory Data Analysis (EDA) on datasets from CKAN portals and direct CSV files. It focuses on understanding data structure, assessing quality, identifying patterns, and generating insights.
Capabilities
1. Dataset Discovery & Metadata Analysis
CKAN Dataset Exploration: Search and retrieve datasets from CKAN portals
Organization Analysis: Examine publishers and their datasets
Metadata Validation: Assess completeness and quality of metadata
Resource Evaluation: Analyze available data resources and formats
2. Structural Analysis
Schema Discovery: Identify columns, data types, and relationships
Data Profiling: Generate statistical summaries and distributions
Format Assessment: Evaluate data formatting and standards compliance
Field Analysis: Examine individual columns and their characteristics
3. Quality Assessment
Completeness Checking: Identify missing values and null patterns
User: "Analyze this dataset from dati.gov.it"
1. Retrieve dataset metadata using ckan_package_show
2. Download and examine CSV structure
3. Generate statistical summary
4. Check data quality metrics
5. Identify key insights and patterns
6. Produce comprehensive report
Quality Assessment
User: "Check the quality of this CSV file"
1. Examine file structure and schema
2. Check for missing values
3. Validate data types and formats
4. Verify internal consistency
5. Generate quality score (0-10)
6. Provide improvement recommendations
Comparative Analysis
User: "Compare these two datasets"
1. Retrieve both datasets
2. Analyze schemas and structures
3. Compare statistical profiles
4. Identify similarities and differences
5. Highlight quality differences
6. Generate comparison report
Best Practices
Data Access
Verify URLs: Always check that data URLs are accessible
Handle Errors: Gracefully manage missing data and timeouts
Respect Limits: Be aware of API rate limits and file sizes
Use Raw URLs: Prefer direct file URLs over GitHub web interfaces
Analysis Approach
Start with Metadata: Understand the dataset before diving into data
Profile First: Generate statistics before detailed analysis
Validate Early: Check data quality before drawing conclusions
Document Findings: Keep clear records of analysis steps
Quality Assessment
Be Systematic: Follow a consistent quality checklist
Check Calculations: Verify that totals and percentages are correct
Look for Patterns: Identify temporal and categorical patterns