senior-data-scientist

# Senior Data Scientist Expert data science for statistical modeling, experimentation, ML deployment, and data-driven decision making — A/B test design and analysis, feature engineering, model training/evaluation, production deployment, and causal inference. ## Keywords data-science, machine-learning, statistics, a-b-testing, causal-inference, feature-engineering, mlops, experiment-design, model-deployment, python, scikit-learn, pytorch, tensorflow, spark, airflow ## Core Capabilities - **Experiment design & analysis** — hypothesis framing, power analysis and sample sizing, randomization, SRM monitoring, and post-hoc significance testing. - **Feature engineering** — profiling, candidate generation (temporal/aggregation/interaction/text), selection (variance, correlation, SHAP/RFE), and leakage validation. - **Model training & evaluation** — stratified/temporal splits, baselines, hyperparameter tuning, cross-validation, calibration, and fairness checks. - **Production deployment** — containerized serving, input/output drift monitoring (KS/PSI), canary rollouts, and latency/error SLAs. - **Causal inference** — propensity score matching, difference-in-differences, regression discontinuity, instrumental variables, and assumption/placebo testing. ## When to Use - Designing or analyzing an A/B test. - Building a feature engineering pipeline. - Training, evaluating, or deploying an ML model. - Estimating treatment effects from observational data. ## Clarify First Before running an analysis or pipeline, confirm these inputs. If any is unknown or vague, ASK — do not assume: - [ ] **Task** — A/B test design / feature engineering / model evaluation / causal inference (selects the script and workflow) - [ ] **Dataset & target variable** — what you are modeling or measuring (drives feature generation and leakage validation) - [ ] **Decision metric & minimum effect** — the metric and the smallest effect worth detecting (drives power analysis and sample size) Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact. ## Tools | Script | Purpose | |--------|---------| | `scripts/experiment_designer.py` | A/B test design, power analysis, sample size calculation | | `scripts/feature_engineering_pipeline.py` | Automated feature generation, correlation analysis, feature selection | | `scripts/model_evaluation_suite.py` | Model comparison, cross-validation, deployment readiness checks | > For hypothesis testing, causal inference, and regression analysis, use inline scipy/statsmodels — see [references/statistical_methods_advanced.md](references/statistical_methods_advanced.md). ## References Load the reference that matches the task — keep this file lean and pull detail on demand: - **[references/ds-workflows.md](references/ds-workflows.md)** — quick-start commands, tech stack, the five end-to-end workflows (A/B testing, feature pipeline, train/evaluate, deploy, causal inference) with Python snippets, performance targets, and common commands. Read when executing any data-science task. - **[references/ds-operations.md](references/ds-operations.md)** — troubleshooting table, success criteria, and the full CLI flag reference for each script. Read when diagnosing issues or running the tools. - **[references/statistical_methods_advanced.md](references/statistical_methods_advanced.md)** — advanced statistical methods reference (hypothesis testing, causal inference, regression). Read for statistical depth. - **[references/experiment_design_frameworks.md](references/experiment_design_frameworks.md)** — experiment design frameworks and power-analysis foundations. Read when designing rigorous experiments. - **[references/feature_engineering_patterns.md](references/feature_engineering_patterns.md)** — feature engineering patterns and selection techniques. Read when building features. ## Scope & Limitations **This skill covers:** - End-to-end experiment design including power analysis, randomization, and post-hoc analysis - Feature engineering pipelines with profiling, generation, selection, and validation - Model training evaluation including cross-validation, calibration, and fairness checks - Production model deployment with monitoring, drift detection, and canary rollouts **This skill does NOT cover:** - Data engineering infrastructure (ETL orchestration, pipeline scheduling, data lake management) -- see `senior-data-engineer` - Deep learning model architecture design and training at scale (distributed GPU training, custom layers) -- see `senior-ml-engineer` - Prompt engineering, RAG systems, and LLM fine-tuning workflows -- see `senior-prompt-engineer` - Computer vision pipelines (object detection, segmentation, video processing) -- see `senior-computer-vision` ## Integration Points | Skill | Integration | Data Flow | |-------|-------------|-----------| | `senior-data-engineer` | Feature pipeline ingests data from ETL outputs; shares data quality validation patterns | Raw data stores --> feature engineering pipeline --> feature store | | `senior-ml-engineer` | Trained models handed off for MLOps deployment; shares model registry and serving configs | Evaluated model artifacts --> deployment pipeline --> production serving | | `senior-prompt-engineer` | Embedding features from LLMs feed into ML pipelines; experiment frameworks apply to prompt A/B tests | LLM embeddings --> feature vectors; experiment designs --> prompt evaluation | | `senior-architect` | Model serving architecture reviewed for scalability; data platform design aligned with training infrastructure | Architecture specs --> deployment topology --> monitoring dashboards | | `senior-backend` | Model inference endpoints integrated into backend services; API contracts defined for prediction requests | REST/gRPC model API --> backend service layer --> client applications | | `senior-devops` | CI/CD pipelines extended for model retraining triggers; containerized model images deployed via infrastructure-as-code | Docker images --> Kubernetes manifests --> production clusters |