data-analysis

name	data-analysis
version	1.0.0
author	G-HunterAi
license	MIT
description	Turn raw data into decisions with statistical rigor, proper methodology, and awareness of analytical pitfalls.
tags	["data","analysis","statistics","visualization","hypothesis-testing","insights"]
platforms	["all"]
category	research
emoji	📊
requires_bins	["python3"]

Data Analysis

Turn raw data into decisions with statistical rigor, proper methodology, and awareness of common analytical pitfalls. This skill focuses on extracting insights from data while avoiding common mistakes.

When to Use

Use this skill when you need to:

• Statistical analysis: Compare groups, test hypotheses, find significance
• Trend identification: What's changing? In which direction? How fast?
• Pattern discovery: Segment users, identify behaviors, find correlations
• Anomaly detection: What's unusual? What warrants investigation?
• Decision support: Data-driven recommendations backed by analysis
• A/B testing: Evaluate whether changes actually work
• Cohort analysis: How do different user groups behave?
• Root cause analysis: Why did something happen?

Examples:

• "Did the new feature increase retention?"
• "Which customer segments are most profitable?"
• "What predicts customer churn?"
• "Is this metric trend meaningful or just noise?"
• "Are these two groups actually different?"

When NOT to Use

Do NOT use this skill for:

• Simple data transformations: Use spreadsheets (Excel, Sheets)
• Data pipeline engineering: Use dbt, Apache Airflow, or data warehouse tools
• ML model training: Use MLflow, scikit-learn, etc. (this is modeling, not analysis)
• Visualization only: Use BI tools (Tableau, Looker, Power BI)
• Data collection: Use surveys, logging, or analytics platforms

Prerequisites

Python 3.10+

Verify version:

python3 --version

Required Libraries

pip install pandas matplotlib scipy numpy

Optional (recommended)

pip install seaborn scikit-learn statsmodels jupyter

Verify:

python3 -c "import pandas; import numpy; print('Ready')"

Quick Start

1. Setup

pip install pandas matplotlib scipy numpy

2. Load Data

import pandas as pd

# From CSV
df = pd.read_csv("data.csv")

# Inspect
print(df.head())
print(df.describe())
print(df.info())

3. Analyze

# Compare two groups
group_a = df[df['group'] == 'A']['metric']
group_b = df[df['group'] == 'B']['metric']

# Statistical test
from scipy import stats
t_stat, p_value = stats.ttest_ind(group_a, group_b)
print(f"p-value: {p_value:.4f}")

4. Visualize

import matplotlib.pyplot as plt

plt.hist([group_a, group_b], label=['A', 'B'])
plt.legend()
plt.show()

Core Principle

Analysis without a decision is just arithmetic.

Always clarify: What would change if this analysis shows X vs Y?

This forces clarity on what analysis actually matters and avoids vanity metrics.

Methodology First

Before touching any data:

1. What decision is this analysis supporting?
2. What would change your mind? (the real question)
3. What data do you actually have vs what you wish you had?
4. What timeframe is relevant?

Decision-Focused Analysis

• Don't analyze "just because"
• Ask: "If this shows X, what do we do? If it shows Y, what do we do?"
• If the answer is "do the same thing either way," you don't need the analysis

Statistical Rigor Checklist

Before concluding anything:

• Sample size sufficient? (Small N = wide confidence intervals, low power)
• Comparison groups fair? (Same time period? Similar conditions? Matched?)
• Multiple comparisons? (20 tests = 1 "significant" by chance alone)
• Effect size meaningful? (Statistically significant ≠ practically important)
• Uncertainty quantified? ("12-18% lift" not just "15% lift")
• Confounders controlled? (What else might explain the difference?)

Analytical Pitfalls to Catch

Common mistakes that invalidate analysis:

Pitfall	What it looks like	Fix
Simpson's Paradox	Trend reverses when you segment	Always check by key dimensions
Survivorship Bias	Only analyzing current users	Include churned/failed in dataset
Comparing Unequal Periods	Feb (28d) vs March (31d) revenue	Normalize per-day or same-length windows
p-Hacking	Testing until something is "significant"	Pre-register hypotheses or adjust for multiple comparisons
Spurious Correlation	Both went up = "related"	Detrend, check if controlling for time removes relationship
Aggregating Percentages	Averaging percentages directly	Recalculate from underlying totals
Selection Bias	Treatment and control differ before test	Verify pre-experiment metrics are balanced
Confusing Causation	X correlates with Y = X causes Y	Use experiment, natural experiment, or control for confounders

See references/pitfalls.md for detailed examples of each pitfall.

Approach Selection

Match your analysis to the question:

Question type	Approach	Key output
"Is X different from Y?"	Hypothesis test	p-value + effect size + CI
"What predicts Z?"	Regression/correlation	Coefficients + R² + residual check
"How do users behave over time?"	Cohort analysis	Retention curves by cohort
"Are these groups different?"	Segmentation	Profiles + statistical comparison
"What's unusual?"	Anomaly detection	Flagged points + context

See references/techniques.md for detailed when/how for each technique.

Analysis Techniques Reference

Hypothesis Testing

Use when: Comparing two groups to determine if a difference is real or random chance.

from scipy import stats

# Test if two groups are different
t_stat, p_value = stats.ttest_ind(group_a, group_b)

# Interpret:
# p-value < 0.05: Difference is statistically significant
# p-value > 0.05: Could be random variation

Key outputs: p-value, effect size (Cohen's d), confidence interval

Watch out for: Large samples make everything "significant" (focus on effect size)

Cohort Analysis

Use when: Understanding how user behavior changes over time, segmented by when they started.

# Build retention matrix
# Rows: signup cohort (week/month)
# Cols: days since signup
# Values: % of cohort still active

import pandas as pd
retention = pd.pivot_table(
    df,
    values='user_id',
    index='signup_week',
    columns='days_since_signup',
    aggfunc='count'
)

# Normalize by first week (% retained)
retention_pct = retention.div(retention.iloc[:, 0], axis=0) * 100

Key outputs: Retention curves (line chart by cohort), cohort comparison

Watch out for: Cohort size differences, seasonality effects, definition consistency

Regression Analysis

Use when: Understanding what predicts an outcome, controlling for other factors.

from sklearn.linear_model import LinearRegression

# Linear: predicting continuous outcome
model = LinearRegression()
model.fit(X, y)
print(model.coef_)  # Effect of each variable
print(model.score(X, y))  # R² (variance explained)

# Logistic: predicting binary outcome (churn/retention)
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(X, y_binary)

Key outputs: Coefficients, R², p-values per variable, residual plots

Watch out for: Multicollinearity, omitted variable bias, extrapolation beyond data range

Segmentation/Clustering

Use when: Discovering natural groups in your data.

from sklearn.cluster import KMeans

# Normalize features first
from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)

# Find clusters
kmeans = KMeans(n_clusters=3)
clusters = kmeans.fit_predict(X_scaled)

# Profile each cluster
for cluster_id in range(3):
    mask = clusters == cluster_id
    print(f"Cluster {cluster_id}: {X[mask].mean()}")

Key outputs: Cluster profiles, segment sizes, distinguishing characteristics

Watch out for: Garbage in garbage out, cluster count is subjective, stability

Anomaly Detection

Use when: Finding unusual data points that warrant investigation.

# Simple: beyond 2-3 standard deviations
import numpy as np

mean = df['metric'].mean()
std = df['metric'].std()

anomalies = df[np.abs(df['metric'] - mean) > 3 * std]

# Or: IQR method
Q1 = df['metric'].quantile(0.25)
Q3 = df['metric'].quantile(0.75)
IQR = Q3 - Q1

anomalies = df[(df['metric'] < Q1 - 1.5*IQR) | (df['metric'] > Q3 + 1.5*IQR)]

Key outputs: Flagged records with anomaly scores, context for why unusual

Watch out for: Seasonality (Black Friday isn't an anomaly), trends

Time Series Analysis

Use when: Understanding patterns in data over time.

import pandas as pd

# Decompose trend, seasonality, residual
from statsmodels.tsa.seasonal import seasonal_decompose

ts = df.set_index('date')['metric']
decomposition = seasonal_decompose(ts, period=365)

# Visualize components
decomposition.plot()

# Year-over-year comparison
df['last_year'] = df['metric'].shift(365)
df['yoy_change'] = (df['metric'] / df['last_year'] - 1) * 100

Key outputs: Trend direction/strength, seasonal patterns, forecast with uncertainty

Watch out for: Different period lengths, holidays/events, structural breaks

Worked Examples

Example 1: A/B Test Evaluation

Scenario: Ran A/B test on checkout flow. Control: 1000 users, 120 conversions (12%). Treatment: 1000 users, 145 conversions (14.5%).

Question: Did the new flow actually increase conversions?

from scipy.stats import chi2_contingency

# Contingency table
contingency = [[880, 120], [855, 145]]  # [not converted, converted]

chi2, p_value, dof, expected = chi2_contingency(contingency)

print(f"p-value: {p_value:.4f}")
print(f"If p < 0.05: Difference is statistically significant")

# Calculate effect size (odds ratio)
odds_control = 120 / 880
odds_treatment = 145 / 855
odds_ratio = odds_treatment / odds_control
print(f"Odds ratio: {odds_ratio:.2f}")
print(f"Interpretation: Treatment is {(odds_ratio - 1)*100:.1f}% more likely to convert")

Conclusion:

• p-value = 0.025 (significant!)
• Effect size: 23% relative improvement
• 95% CI: 12% to 36% improvement
• Recommendation: Deploy treatment

Example 2: Cohort Retention Analysis

Scenario: Comparing retention of users by signup cohort.

import pandas as pd

# Data: user_id, signup_date, last_active_date
# Build retention table

df['days_active'] = (df['last_active_date'] - df['signup_date']).dt.days
df['signup_week'] = df['signup_date'].dt.to_period('W')

# Retention at different milestones
retention = df.groupby('signup_week').apply(lambda x: {
    'day_1': (x['days_active'] >= 1).sum() / len(x),
    'day_7': (x['days_active'] >= 7).sum() / len(x),
    'day_30': (x['days_active'] >= 30).sum() / len(x),
    'day_90': (x['days_active'] >= 90).sum() / len(x),
})

# Convert to DataFrame
retention_df = pd.DataFrame(retention).T
retention_df *= 100  # Convert to percentages

print(retention_df)

Analysis:

• Week 1 cohort: 80% → 60% → 40% → 25% retention
• Week 2 cohort: 85% → 65% → 45% → 30% retention
• Newer cohorts retaining better (+5-10%)
• Recommendation: Changes made in Week 2 are working

Example 3: Churn Prediction

Scenario: Identify which customers are most likely to churn.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, classification_report

# Features: days_since_login, emails_opened, support_tickets, account_age
# Target: churned (1) or retained (0)

model = LogisticRegression()
model.fit(X_train, y_train)

# Evaluate
y_pred = model.predict(X_test)
y_pred_proba = model.predict_proba(X_test)[:, 1]

print("ROC AUC:", roc_auc_score(y_test, y_pred_proba))
print(classification_report(y_test, y_pred))

# Feature importance
coefs = pd.DataFrame({
    'feature': X_train.columns,
    'coef': model.coef_[0]
}).sort_values('coef', ascending=False)

print(coefs)

Findings:

• ROC AUC = 0.82 (decent model)
• Top churn signals: No logins in 30+ days, support tickets, low email engagement
• Recommendation: Target retention efforts at high-risk users

Output Standards

1. Lead with the insight, not the methodology
2. Quantify uncertainty — ranges, not point estimates
3. State limitations — what this analysis can't tell you
4. Recommend next steps — what would strengthen the conclusion

Error Handling

Issue	Cause	Solution
Dataset too large for memory	CSV/JSON exceeds available RAM	Use chunked reading (pandas.read_csv(chunksize=10000)) or stream processing
Missing values distort results	NaN/null in critical columns	Document missing data strategy upfront: drop, impute (mean/median), or flag
Encoding errors on import	Non-UTF-8 characters in data	Try encoding='latin-1' or encoding='cp1252'; inspect with chardet
Correlation treated as causation	Statistical relationship without causal evidence	Always note "correlation, not causation" and suggest experimental design
Outliers skewing summary stats	Extreme values in dataset	Use median/IQR instead of mean/std; document outlier handling decision

Red Flags to Escalate

• User wants to "prove" a predetermined conclusion
• Sample size too small for reliable inference
• Data quality issues that invalidate analysis
• Confounders that can't be controlled for
• Results too surprising without obvious explanation

Works Well With

• in-depth-research: Provide data-driven answers to research questions
• postgres-advanced: Query complex databases for analysis
• web-scraper: Collect data to analyze
• workflow-orchestrator: Schedule recurring analyses

---

References: see references/ directory for pitfalls and techniques