| name | pandas-eda-workflow |
| description | Pandas EDA skill - Covers data loading, cleaning, transformation, aggregation, and visualization workflows used in real-world data analysis. |
Overview
Pandas is a powerful data analysis library in Python used for handling structured data.
This skill walks through a practical Exploratory Data Analysis (EDA) workflow using Pandas, including inspecting data, cleaning it, transforming features, and generating basic visual insights.
Use this when you're working on:
- Data analysis projects
- Machine learning preprocessing
- CSV / tabular datasets
- Understanding and preparing features
Setup
pip install pandas matplotlib seaborn
Getting Started
import pandas as pd
df = pd.read_csv("resources/dataset.csv")
df.head()
df.info()
df.describe()
Understanding the Dataset
Data Structure Basics
- Series → single column
- DataFrame → table of data
df.shape
df.columns
df.dtypes
Exploring the Data
df.head()
df.tail()
df.sample(5)
df.info()
df.describe()
Dealing with Missing Data
df.isnull().sum()
Removing missing values
df = df.dropna()
Filling missing values
df.fillna(df.mean(), inplace=True)
Cleaning and Preparing Data
Renaming columns
df.rename(columns={"old_name": "new_name"}, inplace=True)
Removing duplicates
df.drop_duplicates(inplace=True)
Fixing data types
df["column"] = df["column"].astype(int)
Filtering and Selecting Data
df[df["age"] > 25]
df[(df["age"] > 25) & (df["salary"] > 50000)]
Summarizing Data
df.groupby("department")["salary"].mean()
df.groupby("department").agg({
"salary": "mean",
"age": "max"
})
Ordering Data
df.sort_values(by="salary", ascending=False)
Creating New Features
df["bonus"] = df["salary"] * 0.1
Visualizing Insights
Using Matplotlib and Seaborn:
Distribution plot
df["age"].hist()
Salary spread
import seaborn as sns
sns.boxplot(x=df["salary"])
Feature relationships
sns.heatmap(df.corr(), annot=True)
Handy Operations
Selecting specific columns
df[["name", "salary"]]
Applying transformations
df["salary"] = df["salary"].apply(lambda x: x * 1.1)
Best Practices
- Prefer vectorized operations over loops
- Use
.loc and .iloc correctly
- Avoid unnecessary modifications
- Work on copies when needed
Tool Selection Guide
- Pandas → tabular data handling
- NumPy → numerical operations
- Seaborn → statistical plots
- Matplotlib → custom visualizations
References
