career-growth

name	career-growth
description	Portfolio building, technical interviews, job search strategies, and continuous learning
sasmp_version	1.3.0
bonded_agent	01-data-engineer
bond_type	SUPPORT_BOND
skill_version	2.0.0
last_updated	2025-01
complexity	foundational
estimated_mastery_hours	40
prerequisites	[]
unlocks	[]

Career Growth

Professional development strategies for data engineering career advancement.

Quick Start

# Data Engineer Portfolio Checklist

## Required Projects (Pick 3-5)
- [ ] End-to-end ETL pipeline (Airflow + dbt)
- [ ] Real-time streaming project (Kafka/Spark Streaming)
- [ ] Data warehouse design (Snowflake/BigQuery)
- [ ] ML pipeline with MLOps (MLflow)
- [ ] API for data access (FastAPI)

## Documentation Template
Each project should include:
1. Problem statement
2. Architecture diagram
3. Tech stack justification
4. Challenges & solutions
5. Results/metrics
6. GitHub link with clean code

Core Concepts

1. Technical Interview Preparation

# Common coding patterns for data engineering interviews

# 1. SQL Window Functions
"""
Write a query to find the running total of sales by month,
and the percentage change from the previous month.
"""
sql = """
SELECT
    month,
    sales,
    SUM(sales) OVER (ORDER BY month) AS running_total,
    100.0 * (sales - LAG(sales) OVER (ORDER BY month))
        / NULLIF(LAG(sales) OVER (ORDER BY month), 0) AS pct_change
FROM monthly_sales
ORDER BY month;
"""

# 2. Data Processing - Find duplicates
def find_duplicates(data: list[dict], key: str) -> list[dict]:
    """Find duplicate records based on a key."""
    seen = {}
    duplicates = []
    for record in data:
        k = record[key]
        if k in seen:
            duplicates.append(record)
        else:
            seen[k] = record
     duplicates


 collections  defaultdict
 time

 :
     ():
        .max_requests = max_requests
        .window = window_seconds
        .requests = defaultdict()

     () -> :
        now = time.time()
        
        .requests[user_id] = [
            t  t  .requests[user_id]
             now - t < .window
        ]
         (.requests[user_id]) < .max_requests:
            .requests[user_id].append(now)