research-research-design

name	research-research-design
description	Master research study design including hypothesis formation, validity, sampling strategies, and experimental control

Research Design Skill

When to Use This Skill

Use this skill when you need to:

• Plan a research study from scratch
• Choose appropriate research methodology
• Formulate testable hypotheses
• Design sampling strategies
• Ensure validity and reliability
• Control for confounding variables
• Balance internal and external validity
• Navigate ethical considerations

Core Design Elements

1. Research Questions and Hypotheses

FINER Criteria for Research Questions:

F - Feasible: Can you actually do this?
I - Interesting: Does it matter?
N - Novel: Is it new?
E - Ethical: Is it responsible?
R - Relevant: Will it impact the field?

Question to Hypothesis Framework:

from dataclasses import dataclass
from typing import List, Optional
from enum import Enum

class HypothesisType(Enum):
    DIRECTIONAL = "directional"
    NON_DIRECTIONAL = "non-directional"
    NULL = "null"

@dataclass
class ResearchQuestion:
    """Structure a research question"""
    question: str
    population: str
    variables: List[str]
    relationship_type: str  # 'difference', 'relationship', 'prediction'

    def to_hypothesis(self, hypothesis_type: HypothesisType,
                     expected_direction: Optional[str] = None):
        """Convert research question to testable hypothesis"""

        if hypothesis_type == HypothesisType.NULL:
            return f"There is no {self.relationship_type} between " \
                   f"{' and '.join(self.variables)} in {self.population}."

        elif hypothesis_type == HypothesisType.DIRECTIONAL:
            if not expected_direction:
                raise ValueError("Directional hypothesis requires expected_direction")
            return f"There is a {expected_direction} {self.relationship_type} " \
                   f"between {' and '.join(self.variables)} in {self.population}."

        else:  # NON_DIRECTIONAL
            return f"There is a {self.relationship_type} between " \
                   f"{' and '.join(self.variables)} in {self.population}."

# Example
rq = ResearchQuestion(
    question="Does exercise frequency affect anxiety levels in college students?",
    population="college students",
    variables=["exercise frequency", "anxiety levels"],
    relationship_type="relationship"
)

print("Null:", rq.to_hypothesis(HypothesisType.NULL))
print("Directional:", rq.to_hypothesis(HypothesisType.DIRECTIONAL,
                                       "negative"))
print("Non-directional:", rq.to_hypothesis(HypothesisType.NON_DIRECTIONAL))

2. Validity Considerations

Validity Framework:

from typing import List, Dict
from enum import Enum

class ValidityThreat(Enum):
    # Internal validity
    HISTORY = "history"
    MATURATION = "maturation"
    TESTING = "testing"
    INSTRUMENTATION = "instrumentation"
    REGRESSION = "regression_to_mean"
    SELECTION = "selection_bias"
    ATTRITION = "attrition"

    # External validity
    INTERACTION_SELECTION = "selection_treatment_interaction"
    SETTING = "setting_effects"
    HISTORY_TREATMENT = "history_treatment_interaction"

    # Construct validity
    HYPOTHESIS_GUESSING = "hypothesis_guessing"
    EVALUATION_APPREHENSION = "evaluation_apprehension"
    EXPERIMENTER_EFFECTS = "experimenter_effects"
    MONO_OPERATION = "mono_operation_bias"
    MONO_METHOD = "mono_method_bias"

class ValidityAnalysis:
    """Analyze validity threats and controls"""

    def __init__(self, study_design: str):
        self.design = study_design
        self.threats = []
        self.controls = {}

    def add_threat(self, threat: ValidityThreat, description: str,
                  severity: str):
        """Identify potential validity threat"""
        .threats.append({
            : threat.value,
            : description,
            : severity  
        })

     ():
        
        .controls[threat.value] = control

     ():
        
         pandas  pd

        data = []
         threat_info  .threats:
            threat_name = threat_info[]
            data.append({
                : threat_name,
                : threat_info[],
                : threat_info[],
                : .controls.get(threat_name, )
            })

         pd.DataFrame(data)


validity = ValidityAnalysis()

validity.add_threat(
    ValidityThreat.HISTORY,
    ,
    severity=
)
validity.add_control(
    ValidityThreat.HISTORY,
    
)

validity.add_threat(
    ValidityThreat.ATTRITION,
    ,
    severity=
)
validity.add_control(
    ValidityThreat.ATTRITION,
    
    
)

(validity.generate_validity_table())

3. Sampling Strategies

Sampling Design Framework:

from enum import Enum
import numpy as np
from typing import Optional

class SamplingMethod(Enum):
    # Probability sampling
    SIMPLE_RANDOM = "simple_random"
    SYSTEMATIC = "systematic"
    STRATIFIED = "stratified"
    CLUSTER = "cluster"
    MULTISTAGE = "multistage"

    # Non-probability sampling
    CONVENIENCE = "convenience"
    PURPOSIVE = "purposive"
    QUOTA = "quota"
    SNOWBALL = "snowball"

class SamplingDesign:
    """Design and document sampling strategy"""

    def __init__(self, method: SamplingMethod, population_size: int,
                 target_sample_size: int):
        self.method = method
        self.N = population_size
        self.n = target_sample_size
        self.sampling_frame = None
        self.strata = None

    def simple_random_sample(self, population_ids: list, seed: int = 42):
        """Draw simple random sample"""
        np.random.seed(seed)
        return np.random.choice(population_ids, size=self.n, replace=)

     ():
        
        samples = []

         proportional:
            
             stratum_name, stratum_ids  strata_dict.items():
                stratum_n = (.n * (stratum_ids) / .N)
                stratum_sample = np.random.choice(stratum_ids,
                                                 size=stratum_n,
                                                 replace=)
                samples.extend(stratum_sample)
        :
            
            n_per_stratum = .n // (strata_dict)
             stratum_ids  strata_dict.values():
                stratum_sample = np.random.choice(stratum_ids,
                                                 size=n_per_stratum,
                                                 replace=)
                samples.extend(stratum_sample)

         samples

     ():
        
         finite_correction  .N > :
            fpc = np.sqrt((.N - .n) / (.N - ))
            se = (std_dev / np.sqrt(.n)) * fpc
        :
            se = std_dev / np.sqrt(.n)

         se

     ():
        
         scipy  stats

        
        z = stats.norm.ppf(( + confidence_level) / )

        
        n_0 = (z * std_dev / margin_error) ** 

         finite_correction  .N > :
            
            n = n_0 / ( + (n_0 - ) / .N)
        :
            n = n_0

         (np.ceil(n))

     ():
        
        doc = 
         doc


design = SamplingDesign(
    method=SamplingMethod.STRATIFIED,
    population_size=,
    target_sample_size=
)


required_n = design.required_sample_size(
    std_dev=,
    margin_error=,
    confidence_level=
)
()


strata = {
    : ((, )),
    : ((, ))
}
sample = design.stratified_sample(strata, proportional=)
()

4. Experimental Control

Control Strategies:

from dataclasses import dataclass
from typing import List, Dict, Optional
from enum import Enum

class ControlMethod(Enum):
    RANDOMIZATION = "randomization"
    MATCHING = "matching"
    BLOCKING = "blocking"
    STATISTICAL_CONTROL = "statistical_control"
    STANDARDIZATION = "standardization"

@dataclass
class ConfoundingVariable:
    """Define potential confounding variable"""
    name: str
    relationship_to_iv: str
    relationship_to_dv: str
    control_method: ControlMethod
    control_procedure: str

class ExperimentalControl:
    """Design experimental controls"""

    def __init__(self):
        self.confounds = {}
        self.design_features = []

    def identify_confound(self, confound: ConfoundingVariable):
        """Add confounding variable with control plan"""
        self.confounds[confound.name] = confound

    def add_design_feature(self, feature: str, purpose: str):
        """Document design feature for control"""
        self.design_features.append({
            : feature,
            : purpose
        })

     ():
        
        np.random.seed(seed)
        shuffled = np.random.permutation(participants)
        groups = np.array_split(shuffled, n_groups)
         [(g)  g  groups]

     ():
        
         pandas  pd
         sklearn.preprocessing  StandardScaler

        
        scaler = StandardScaler()
        X = scaler.fit_transform(participants_df[matching_vars])

        
         sklearn.cluster  KMeans
        kmeans = KMeans(n_clusters=n_groups, random_state=)
        participants_df[] = kmeans.fit_predict(X)

        
        groups = [[]  _  (n_groups)]
         cluster  (n_groups):
            cluster_members = participants_df[
                participants_df[] == cluster
            ].index.tolist()

            np.random.shuffle(cluster_members)
             i, member  (cluster_members):
                groups[i % n_groups].append(member)

         groups

     ():
        
        plan = 
        plan += 

         name, confound  .confounds.items():
            plan += 
            plan += 
            plan += 
            plan += 
            plan += 

        plan += 
         feature  .design_features:
            plan += 

         plan


control = ExperimentalControl()


control.identify_confound(ConfoundingVariable(
    name=,
    relationship_to_iv=,
    relationship_to_dv=,
    control_method=ControlMethod.RANDOMIZATION,
    control_procedure=
))

control.identify_confound(ConfoundingVariable(
    name=,
    relationship_to_iv=,
    relationship_to_dv=,
    control_method=ControlMethod.STANDARDIZATION,
    control_procedure=
))


control.add_design_feature(
    ,
    
)
control.add_design_feature(
    ,
    
)

(control.generate_control_plan())

Design Patterns

Strong Research Design

✓ Clear, testable hypotheses
✓ Appropriate methodology for question
✓ Threats to validity identified and controlled
✓ Adequate sample size (power analysis)
✓ Random sampling or assignment when possible
✓ Multiple measures/methods (triangulation)
✓ Pilot testing conducted
✓ Pre-registration of hypotheses and methods

Weak Research Design

✗ Vague or non-testable hypotheses
✗ Method-question mismatch
✗ Uncontrolled confounds
✗ Convenience sample assumed representative
✗ Underpowered study
✗ Single method/measure
✗ Post-hoc hypothesizing (HARKing)
✗ P-hacking through multiple analyses

Research Design Types

1. Experimental Designs

True Experiment:

- Random assignment to conditions
- Manipulation of IV
- Control group
- Maximum internal validity

Quasi-Experiment:

- No random assignment
- Manipulation of IV or natural variation
- Comparison group
- Moderate internal validity

Single-Case Design:

- Individual as own control
- Repeated measures over time
- Experimental control through replication
- Good for clinical intervention research

2. Non-Experimental Designs

Correlational:

- Examine relationships between variables
- No manipulation
- Cannot infer causation
- Useful for prediction

Survey:

- Describe population characteristics
- No manipulation
- Generalization to population
- Good for attitudes, beliefs, behaviors

Observational:

- Observe naturally occurring behavior
- No manipulation
- High ecological validity
- Good for exploratory research

Best Practices

1. Planning Phase

• Start with clear research question
• Review literature thoroughly
• Choose design matching question and resources
• Conduct power analysis
• Create detailed protocol
• Pre-register study
• Get IRB approval

2. Design Phase

• Map causal model (DAG)
• Identify all confounds
• Choose appropriate controls
• Balance internal and external validity
• Plan for attrition
• Build in manipulation checks
• Design pilot study

3. Ethical Considerations

• Assess risk-benefit ratio
• Ensure informed consent
• Protect participant privacy
• Plan for adverse events
• Consider vulnerable populations
• Ensure equitable participant selection
• Plan data security

4. Documentation

• Write detailed protocol
• Create decision tree for procedures
• Document all changes from protocol
• Maintain audit trail
• Archive all materials
• Enable reproducibility

Common Design Mistakes

1. Confusing Correlation and Causation

   • Problem: Inferring causation from correlational design
   • Solution: Use causal language only with experimental designs

2. Insufficient Power

   • Problem: Sample too small to detect real effects
   • Solution: Conduct a priori power analysis

3. Unmeasured Confounds

   • Problem: Alternative explanations not ruled out
   • Solution: Create comprehensive causal diagram

4. Convenience Sampling Generalization

   • Problem: Assuming convenience sample represents population
   • Solution: Acknowledge limitations; use probability sampling

5. Demand Characteristics

   • Problem: Participants guess hypothesis and act accordingly
   • Solution: Use blind procedures; cover story; implicit measures

6. Experimenter Bias

   • Problem: Researcher expectations influence results
   • Solution: Double-blind design; automated procedures

Related Skills

• quantitative-methods: Statistical analysis for designed studies
• qualitative-methods: Alternative research approaches
• data-collection: Implementing research protocols
• research-synthesis: Learning from existing research
• research-writing: Documenting research design

Quick Reference

Design Selection Matrix

Question Type        → Design
-------------------------------------
Causation           → True experiment
Association         → Correlational
Prevalence          → Cross-sectional survey
Change over time    → Longitudinal
Lived experience    → Phenomenology
Process/meaning     → Grounded theory
Bounded system      → Case study

Internal Validity Hierarchy

Highest:  Randomized controlled trial
          Quasi-experiment with matching
          Pre-post with control
          Post-only with control
Lowest:   One-group pre-post
          Cross-sectional correlation

Sample Size Quick Rules

Simple comparison:     50-100 per group
Multiple regression:   104 + k (k=predictors)
Factor analysis:       300+ or 10× variables
Structural equation:   200+ minimum
Qualitative:          Until saturation (varies)