experimental-design

# Experimental Design Skill Design rigorous, reproducible experiments across scientific disciplines. ## When to Use - "Design an experiment to test..." - "How many samples do I need?" - "What controls should I include?" - "Help me plan a clinical trial" - "Is this experimental design valid?" - Power analysis and sample size calculation ## When NOT to Use - Running the actual experiment (use code-execution) - Analyzing collected data (use scipy-analysis + statsmodels-stats) - Writing up results (use paper-writing) - Literature review (use literature-search) ## Design Components ### 1. Research Question and Hypotheses - State clear, testable research question - Formulate H0 and H1 (see hypothesis-gen skill) - Define primary and secondary outcomes ### 2. Study Design Selection | Design | When to Use | Strengths | Weaknesses | |--------|------------|-----------|------------| | RCT | Causal inference needed | Gold standard causality | Expensive, ethical limits | | Factorial | Multiple factors | Tests interactions | Complex analysis | | Crossover | Within-subject comparison | Reduced variability | Carryover effects | | Quasi-experimental | Randomization impossible | Practical feasibility | Weaker causality | | Observational (cohort) | Long-term outcomes | Natural setting | Confounding | | Case-control | Rare outcomes | Efficient for rare events | Recall bias | ### 3. Power Analysis ```python # Sample size calculation template (using scipy/statsmodels) from statsmodels.stats.power import TTestIndPower analysis = TTestIndPower() n = analysis.solve_power( effect_size=0.5, # Cohen's d (small=0.2, medium=0.5, large=0.8) alpha=0.05, # Significance level power=0.80, # Statistical power (commonly 0.80 or 0.90) ratio=1.0, # Ratio of group sizes (n2/n1) alternative='two-sided' ) print(f"Required sample size per group: {int(n) + 1}") ``` Key parameters: - **Effect size**: Expected magnitude of difference - **Alpha**: Type I error rate (usually 0.05) - **Power**: 1 - Type II error rate (usually 0.80-0.95) - **Attrition**: Add 10-20% for expected dropout ### 4. Variable Control - **Independent variables**: What you manipulate - **Dependent variables**: What you measure - **Confounding variables**: What could bias results - **Control strategies**: Randomization, blocking, matching, blinding ### 5. Randomization - Simple randomization (coin flip) - Block randomization (balanced groups) - Stratified randomization (balance key covariates) - Cluster randomization (group-level assignment) ### 6. Blinding - Single-blind: Participants unaware of assignment - Double-blind: Participants and researchers unaware - Triple-blind: Including data analysts ## Reproducibility Checklist - [ ] Protocol pre-registered (OSF, ClinicalTrials.gov, PROSPERO) - [ ] All materials/reagents specified with catalog numbers - [ ] Detailed step-by-step procedure written - [ ] Statistical analysis plan pre-specified - [ ] Data management plan documented - [ ] Raw data sharing plan established - [ ] Code availability ensured - [ ] Sample size justified with power analysis - [ ] Randomization method specified - [ ] Blinding procedures documented - [ ] Inclusion/exclusion criteria defined - [ ] Primary endpoint pre-specified ## Output Format ``` ## Experimental Design: [Title] **Research Question**: [Clear question] **Design Type**: [RCT/Factorial/etc.] ### Participants/Samples - Population: [target population] - Inclusion: [criteria] - Exclusion: [criteria] - Sample Size: N=[total] ([n] per group) — Power=[X], alpha=[X], effect=[X] ### Groups - Experimental: [treatment description] - Control: [control description] - Blinding: [single/double/triple/none] ### Variables - IV: [variables] - DV: [primary + secondary outcomes] - Controls: [confounds and how addressed] ### Procedure 1. [Step-by-step protocol] ### Analysis Plan - Primary: [statistical test] - Secondary: [additional analyses] - Multiple comparison correction: [method] ### Timeline - [Phase 1]: [duration] - [Phase 2]: [duration] ### Ethics - IRB/IACUC requirements: [details] - Consent procedure: [details] ```