data-analysis

name

data-analysis

description

Comprehensive data analysis workflow for CSV files with interactive guidance and flexible output formats. Use this skill whenever the user mentions: analyzing data, CSV files, data insights, generating reports, data cleaning, exploratory analysis, business metrics, sales analysis, user behavior analysis, data visualization, creating dashboards, or asks questions like "what does this data tell us?" or "analyze this dataset". Also trigger when the user provides a CSV file path and asks for any kind of analysis or summary. This skill provides a professional 7-step workflow with quality checks, interactive cleaning strategy selection, and multiple output formats (Markdown report, interactive HTML, or full dashboard).

Data Analysis Skill

A comprehensive, interactive data analysis workflow that transforms CSV data into actionable business insights. This skill guides you through professional data analysis from initial exploration to final deliverable, with quality gates and user confirmations at key decision points.

Core Workflow

This skill follows a 7-step methodology with 3 interaction points:

Input → Business Understanding → Data Inspection → Cleaning Strategy → EDA → Deep Analysis → Insights → Output
  ↓           ↓                      ↓                ↓                ↓           ↓          ↓
Required   Interaction 1        Quality Gate     Interaction 2    Auto-run    Auto-run   Interaction 3

---

Input Requirements

Required:

• CSV file path (absolute or relative)
• Business question or analysis goal

Optional:

• Data dictionary (field descriptions)
• Analysis depth: --quick (basic stats), --standard (default), or --deep (advanced modeling)
• Auto mode: --auto (skip interactions, use recommended strategies)
• Output preference: --format=markdown|html|dashboard

Usage examples:

"Analyze sales_data.csv - I want to know which channels have the best conversion rates"
"Help me understand customer_behavior.csv, specifically looking at retention patterns"
"Quick analysis of Q4_results.csv --quick --auto"

---

Step 1: Business Understanding (Interaction Point 1)

Your Actions

1. Parse the business question and identify:

   • Key metrics mentioned (revenue, conversion rate, churn, etc.)
   • Analysis type needed (trend analysis, comparison, attribution, prediction)
   • Expected dimensions (time, geography, customer segments, channels)
   • Chart types that would best illustrate the answer

2. Generate an analysis plan in this format:

   ## Analysis Plan
   
   **Core Question:** [Restate the user's goal in one sentence]
   
   **Key Metrics to Calculate:**
   - [Metric 1: e.g., Monthly conversion rate by channel]
   - [Metric 2: e.g., Average order value trend]
   
   **Analysis Dimensions:**
   - [e.g., Channel, Time period, Customer segment]
   
   **Expected Deliverables:**
   - [e.g., Comparison chart showing channel performance]
   - [e.g., Trend line with annotations for key events]
   

Interaction Point 1

Present your analysis plan and ask:

Does this match what you're looking for?
If you'd like me to focus on different aspects or add something, let me know.

If the business goal is unclear, offer templates:

I can help with common scenarios:
1. Sales Analysis (channel comparison, trend forecasting, top products)
2. User Behavior (funnel analysis, retention cohorts, churn prediction)
3. Operations (ROI calculation, campaign effectiveness, resource allocation)

Which best describes what you need, or would you like to describe it differently?

Wait for user confirmation before proceeding.

---

Step 2: Data Inspection (Auto-run with Quality Gate)

Your Actions

1. Load the CSV file:

   import pandas as pd
   import numpy as np
   
   # Try UTF-8 first, fall back to other encodings if needed
   try:
       df = pd.read_csv(file_path, encoding='utf-8')
   except UnicodeDecodeError:
       df = pd.read_csv(file_path, encoding='latin-1')
   

2. Generate a data overview report:

   ## Data Overview
   
   📊 Dimensions: {rows:,} rows × {cols} columns
   💾 Memory: {size} MB
   
   📋 Columns:
   | Column Name | Data Type | Sample Value |
   |-------------|-----------|--------------|
   | ...         | ...       | ...          |
   
   🔍 Preview (first 5 rows):
   [Display formatted table]
   

3. Perform quality checks:

   • Missing values: Count and percentage per column
   • Duplicates: Check for fully duplicate rows
   • Data types: Verify numeric columns aren't stored as strings, dates are parseable
   • Outliers (quick check): Flag columns with extreme values using IQR method

4. Calculate a data quality score (0-100):

   Score = 100 - (missing_penalty + duplicate_penalty + type_mismatch_penalty)
   
   Where:
   - missing_penalty = min(40, missing_rate * 100)
   - duplicate_penalty = min(20, duplicate_rate * 100)
   - type_mismatch_penalty = 10 per column with wrong type
   

Quality Gate 1

Based on the quality score, present findings:

Score ≥ 80 (Good):

✅ Data quality looks good (Score: {score}/100)
Minor issues found: [list if any]
Proceeding to analysis...

Score 60-79 (Fair):

⚠️ Data has some quality issues (Score: {score}/100)
Issues found: [list]
I can still analyze this, but results may be limited. Continue?

Score < 60 (Poor):

🚨 Data quality is concerning (Score: {score}/100)
Major issues:
- [Issue 1 with impact]
- [Issue 2 with impact]

Recommendation: Contact the data provider or provide a data dictionary.
Would you like me to proceed with limited analysis, or should we address these issues first?

---

Step 3: Data Cleaning Strategy (Interaction Point 2)

Your Actions

If quality score ≥ 80 and issues are minor, apply automatic fixes and report:

🧹 Applied automatic cleaning:
- Standardized date format in 'OrderDate' column
- Trimmed whitespace from text fields
Ready to analyze!

If quality score < 80, present issues with specific strategy options:

## Data Cleaning Recommendations

### Issue 1: Missing Values in 'Age' Column (20% missing)
**Strategy options:**
A. Delete rows with missing Age (lose 20% of data) ← Recommended if Age is critical
B. Fill with median age (35 years)
C. Fill with group average (median by Gender)
D. Keep as-is and exclude Age from analysis

### Issue 2: Outliers in 'Price' Column (3 negative values)
**Strategy options:**
A. Remove the 3 rows ← Recommended
B. Set negative values to 0
C. Set to the minimum valid price

### Issue 3: Date Format Inconsistency in 'PurchaseDate'
**Strategy options:**
A. Standardize to YYYY-MM-DD format ← Recommended (automatic)

Interaction Point 2

Ask the user:

Please choose a strategy for each issue (e.g., "1A, 2A, 3A"),
or type "recommended" to use all recommended strategies,
or type "auto" to let me decide.

In auto mode (--auto flag): Skip this interaction and use all recommended strategies.

Execute Cleaning

1. Apply the chosen strategies

2. Log all changes made

3. Report the results:

   ✅ Cleaning completed:
   - Age: Filled 1,234 missing values with median (35)
   - Price: Removed 3 rows with negative values
   - PurchaseDate: Standardized format for all 6,000 rows
   
   📊 Final dataset: {final_rows:,} rows × {cols} columns (was {original_rows:,} rows)
   

4. Save the cleaned data:

   cleaned_path = output_dir / 'cleaned_data.csv'
   df_clean.to_csv(cleaned_path, index=False)
   

---

Step 4: Exploratory Data Analysis (Auto-run)

Your Actions

1. Descriptive statistics:

   • For numeric columns: mean, median, std, min, max, quartiles
   • For categorical columns: value counts, unique values, mode

2. Single-variable analysis:

   • Numeric: Generate histograms, identify distribution shape (normal, skewed, multimodal)
   • Categorical: Generate bar charts showing frequency distribution

3. Multi-variable analysis:

   • Correlation matrix: For all numeric columns (use heatmap visualization)
   • Cross-tabulation: For key categorical dimensions from Step 1
   • Scatter plots: For top 3 correlated pairs related to the business question

4. Generate initial insights: Extract Top 3-5 preliminary findings, such as:

   • "Channel A has 3x the conversion rate of Channel B"
   • "Sales show a strong upward trend since March"
   • "Age and purchase amount have a weak negative correlation (-0.23)"

Output

Save all visualizations as PNG files:

import matplotlib.pyplot as plt
import seaborn as sns

# Example
fig, ax = plt.subplots(figsize=(10, 6))
sns.histplot(data=df, x='Age', bins=30, ax=ax)
plt.title('Age Distribution')
plt.savefig(output_dir / 'age_distribution.png', dpi=300, bbox_inches='tight')
plt.close()

Report the findings in a structured format:

## Exploratory Findings

### Distribution Overview
- [Key observation about data distribution]

### Preliminary Insights
1. **[Insight 1]:** [Data supporting it]
2. **[Insight 2]:** [Data supporting it]
3. **[Insight 3]:** [Data supporting it]

📊 Visualizations saved: age_distribution.png, correlation_heatmap.png, channel_comparison.png

---

Step 5: Deep Analysis (Auto-run)

Your Actions

Based on the business question type identified in Step 1, automatically choose and apply the appropriate analysis method:

Business Question Type	Analysis Method
Trend over time	Time series analysis with moving averages, seasonality detection
Attribution/cause	Grouped comparison, contribution breakdown (e.g., which factor drives 80% of variance)
User behavior	Funnel analysis (conversion at each step), cohort retention analysis
Customer value	RFM model (Recency, Frequency, Monetary), clustering into segments
Forecasting	Simple linear regression or exponential smoothing for trend extrapolation

Example: Trend Analysis

# Time series with moving average
df['Date'] = pd.to_datetime(df['Date'])
df = df.sort_values('Date')
df['7d_MA'] = df['Revenue'].rolling(window=7).mean()

# Detect trend
from scipy.stats import linregress
slope, intercept, r_value, p_value, std_err = linregress(
    df['Date'].map(pd.Timestamp.toordinal),
    df['Revenue']
)

if p_value < 0.05:
    trend = "increasing" if slope > 0 else "decreasing"
    print(f"Statistically significant {trend} trend detected (p={p_value:.4f})")

Example: RFM Analysis

# Calculate RFM scores
current_date = df['PurchaseDate'].max()
rfm = df.groupby('CustomerID').agg({
    'PurchaseDate': lambda x: (current_date - x.max()).days,  # Recency
    'OrderID': 'count',  # Frequency
    'Amount': 'sum'  # Monetary
}).rename(columns={
    'PurchaseDate': 'Recency',
    'OrderID': 'Frequency',
    'Amount': 'Monetary'
})

# Score customers (1-5 scale)
rfm['R_Score'] = pd.qcut(rfm['Recency'], 5, labels=[5, 4, 3, 2, 1])
rfm['F_Score'] = pd.qcut(rfm['Frequency'].rank(method='first'), 5, labels=[1, 2, 3, 4, 5])
rfm['M_Score'] = pd.qcut(rfm['Monetary'], 5, labels=[1, 2, 3, 4, 5])

# Segment customers
rfm['Segment'] = rfm['R_Score'].astype(str) + rfm['F_Score'].astype(str) + rfm['M_Score'].astype(str)

Output

Report deep analysis results with specific numbers:

## Deep Analysis Results

### [Analysis Type: e.g., "Channel Performance Attribution"]

**Key Metric Calculated:** [e.g., Conversion Rate by Channel]

| Channel | Orders | Conversion Rate | Contribution to Revenue |
|---------|--------|-----------------|-------------------------|
| A       | 5,234  | 8.5%            | 45%                     |
| B       | 3,102  | 3.7%            | 28%                     |
| C       | 1,876  | 2.1%            | 27%                     |

**Statistical Finding:**
Channel A's conversion rate is 2.3x higher than Channel B (p < 0.001), indicating significantly better targeting or user experience.

📊 Visualization saved: channel_performance.png

---

Step 6: Insights Generation (Auto-run)

Your Actions

Synthesize all findings into a structured narrative following the What → So What → Now What framework:

## Analysis Report

### 🔍 Core Findings (What)
Objective facts from the data:
1. **[Finding 1]:** [Specific numbers and context]
2. **[Finding 2]:** [Specific numbers and context]
3. **[Finding 3]:** [Specific numbers and context]

### 💡 Business Insights (So What)
Interpretation and implications:
1. **[Insight 1]:** Why this matters for the business