conjoint-analysis

name	conjoint-analysis
description	Design, implement, and analyze conjoint experiments for studying multidimensional preferences and choices using causal inference methods (AMCE, ACIE)

Conjoint Analysis Skill

Use this skill when designing, implementing, or analyzing conjoint experiments for studying multidimensional preferences and choices. Conjoint analysis enables researchers to estimate the causal effects of multiple treatment components simultaneously, making it ideal for understanding voter preferences, policy attitudes, immigration opinions, consumer choices, and any scenario where decision-makers evaluate alternatives that vary across multiple attributes.

When to Use Conjoint Analysis

• Multidimensional choices: When alternatives differ on multiple attributes (e.g., candidates with different education, profession, age)
• Decomposing treatment effects: When you need to identify which specific components of a composite treatment drive effects
• Testing multiple hypotheses: When you want to evaluate several causal hypotheses in a single study
• Reducing social desirability bias: Multiple attributes provide respondents with alternative justifications for choices
• Policy design: When you need to identify popular combinations of policy components

Core Concepts

Key Terms

Term	Definition
Profile	A hypothetical alternative characterized by a set of attribute values (e.g., one immigrant applicant)
Attribute	A dimension on which profiles vary (e.g., education, country of origin)
Level	A specific value an attribute can take (e.g., "college degree", "high school")
Choice task	One instance where respondent evaluates J profiles
AMCE	Average Marginal Component Effect - the main causal estimand
ACIE	Average Component Interaction Effect - interaction between attributes

Notation

• N = number of respondents
• J = number of profiles per choice task (typically 2)
• K = number of choice tasks per respondent
• L = number of attributes per profile
• D_l = number of levels for attribute l
• T_ijkl = treatment value for respondent i, profile j, task k, attribute l
• Y_ijk = outcome (choice or rating) for respondent i, profile j, task k

Experimental Design

Outcome Types

1. Choice-based (forced choice): Respondent chooses preferred profile from J alternatives

   • Binary outcome: Y_ijk ∈ {0, 1} with constraint Σ Y_ijk = 1 per task
   • Closely approximates real-world decision-making

2. Rating-based: Respondent rates each profile on a scale (e.g., 1-7)

   • Continuous outcome: Y_ijk ∈ [y_min, y_max]
   • Provides finer-grained preference information

Design Recommendations

Number of attributes (L):     5-10 recommended (>10 may cause information overload)
Number of profiles per task:  2 is most common (J=2)
Number of tasks per respondent: 5-10 typical (K=5-10)
Attribute randomization:       Fully randomized or with plausibility restrictions
Attribute order:               Randomize across respondents to detect order effects

Handling Implausible Combinations

Use restrictions to exclude logically impossible or implausible attribute combinations:

# Example: High-skill professions require college education
restrictions = {
    'profession': {
        'doctor': {'education': ['college', 'graduate']},
        'research_scientist': {'education': ['college', 'graduate']},
        'computer_programmer': {'education': ['two_year_college', 'college', 'graduate']}
    }
}

Important: Restrictions affect AMCE interpretation - effects are only defined for attribute combinations that are possible under the restriction scheme.

Key Assumptions

Assumption 1: Stability and No Carryover Effects

Potential outcomes remain stable across choice tasks and are unaffected by profiles in other tasks.

Y_ijk(T̄_i) = Y_ijk'(T̄'_i)  if  T_ik = T'_ik'

Implication: Can pool data across tasks for efficiency Diagnostic: Compare AMCEs across task numbers

Assumption 2: No Profile-Order Effects

The ordering of profiles within a task does not affect responses.

Y_ij(T_ik) = Y_ij'(T'_ik)  if profiles are swapped but attributes identical

Implication: Can pool data across profile positions Diagnostic: Compare AMCEs by profile position (1st vs 2nd)

Assumption 3: Randomization of Profiles

Potential outcomes are independent of assigned attribute values.

Y_i(t) ⊥ T_ijkl  for all i, j, k, l, t

Guaranteed by design when using proper randomization Diagnostic: Balance checks on respondent characteristics

Causal Estimands

Average Marginal Component Effect (AMCE)

The AMCE represents the marginal effect of attribute l averaged over the joint distribution of remaining attributes:

π̄_l(t_1, t_0, p(t)) = E[Y_i(t_1, T_ijk[-l], T_i[-j]k) - Y_i(t_0, T_ijk[-l], T_i[-j]k)]

Interpretation: The average change in probability of choosing a profile (or change in rating) when attribute l changes from t_0 to t_1, averaged over all possible values of other attributes.

Key property: AMCE is defined relative to:

1. A baseline attribute value (t_0)
2. A distribution of other attributes p(t)

Average Component Interaction Effect (ACIE)

Measures how the effect of one attribute varies depending on another:

π̄_l,m = AMCE_l(given T_m = t_m1) - AMCE_l(given T_m = t_m0)

Conditional AMCE

AMCE estimated within subgroups defined by respondent characteristics:

E[Y_i(t_1, ...) - Y_i(t_0, ...) | X_i]

Important: Respondent characteristics must be measured before showing profiles to avoid post-treatment bias.

Estimation

Simple Case: Completely Independent Randomization

When all attributes are independently randomized (no restrictions):

Difference-in-Means Estimator:

# Python implementation
import pandas as pd
import numpy as np

def estimate_amce_simple(df, outcome, attribute, baseline, treatment):
    """
    Estimate AMCE via difference in means.

    Parameters:
    -----------
    df : DataFrame with profile-level data
    outcome : str, name of outcome variable (0/1 for choice, continuous for rating)
    attribute : str, name of attribute column
    baseline : value of baseline attribute level
    treatment : value of treatment attribute level

    Returns:
    --------
    amce : float, estimated AMCE
    se : float, standard error (naive, needs clustering correction)
    """
    y1 = df.loc[df[attribute] == treatment, outcome].mean()
    y0 = df.loc[df[attribute] == baseline, outcome].mean()

    n1 = (df[attribute] == treatment).sum()
    n0 = (df[attribute] == baseline).sum()

    amce = y1 - y0

    # Naive SE (must cluster by respondent in practice)
    var1 = df.loc[df[attribute] == treatment, outcome].var() / n1
    var0 = df.loc[df[attribute] == baseline, outcome].var() / n0
    se = np.sqrt(var1 + var0)

    return amce, se

Linear Regression Estimator (Preferred):

import statsmodels.api as sm
from statsmodels.formula.api import ols

def estimate_amce_regression(df, outcome, attributes, respondent_id):
    """
    Estimate all AMCEs via OLS with clustered standard errors.

    Parameters:
    -----------
    df : DataFrame with profile-level data
    outcome : str, outcome variable name
    attributes : list of str, attribute column names
    respondent_id : str, respondent identifier for clustering

    Returns:
    --------
    results : statsmodels results object
    """
    # Create formula with all attribute dummies
    # First level of each attribute is automatically baseline
    formula = f"{outcome} ~ " + " + ".join([f"C({attr})" for attr in attributes])

    model = ols(formula, data=df).fit(
        cov_type='cluster',
        cov_kwds={'groups': df[respondent_id]}
    )

    return model

Stata Implementation:

* Simple regression estimator with clustered SEs
regress chosen i.education i.language i.origin i.profession, vce(cluster respondent_id)

* The coefficients on each dummy represent AMCEs relative to omitted category

R Implementation:

library(estimatr)

# Estimate AMCEs with clustered standard errors
model <- lm_robust(
  chosen ~ factor(education) + factor(language) + factor(origin) + factor(profession),
  data = df,
  clusters = respondent_id,
  se_type = "CR2"
)
summary(model)

Complex Case: Conditionally Independent Randomization

When attributes have restrictions (e.g., high-skill jobs require education):

Subclassification Estimator:

def estimate_amce_restricted(df, outcome, focal_attr, conditioning_attr,
                             baseline, treatment, respondent_id):
    """
    Estimate AMCE when focal_attr depends on conditioning_attr.

    Uses subclassification: estimate effect within each stratum of
    conditioning attribute, then take weighted average.
    """
    import statsmodels.formula.api as smf

    # Include interaction between focal and conditioning attributes
    formula = f"{outcome} ~ C({focal_attr}) * C({conditioning_attr})"

    model = smf.ols(formula, data=df).fit(
        cov_type='cluster',
        cov_kwds={'groups': df[respondent_id]}
    )

    # Get stratum-specific effects and weight by stratum probabilities
    strata = df[conditioning_attr].value_counts(normalize=True)

    # Compute weighted average of stratum-specific effects
    # (Implementation depends on specific restrictions)

    return model

Stata with Interactions:

* For attributes with restrictions, include interactions
regress chosen i.occupation##i.education i.language i.origin, vce(cluster respondent_id)

* Then compute weighted average of occupation effects across education levels
* using lincom or margins
margins, dydx(occupation) over(education)

Variance Estimation

Critical: Must account for within-respondent correlation.

1. Cluster-robust standard errors (recommended for most cases):

   • Cluster by respondent ID
   • Valid when number of respondents is large

2. Block bootstrap (for small samples):

   • Resample respondents with replacement
   • Compute AMCE on each bootstrap sample

def bootstrap_amce(df, outcome, attribute, baseline, treatment,
                   respondent_id, n_bootstrap=1000):
    """
    Bootstrap confidence interval for AMCE.
    """
    respondents = df[respondent_id].unique()
    amces = []

    for _ in range(n_bootstrap):
        # Resample respondents
        sampled_respondents = np.random.choice(respondents, size=len(respondents), replace=True)

        # Get all profiles for sampled respondents
        boot_df = pd.concat([df[df[respondent_id] == r] for r in sampled_respondents])

        # Estimate AMCE
        amce, _ = estimate_amce_simple(boot_df, outcome, attribute, baseline, treatment)
        amces.append(amce)

    return np.percentile(amces, [2.5, 97.5])

Diagnostics

1. Carryover Effects (Test Assumption 1)

Compare AMCEs across task numbers:

def test_carryover(df, outcome, attribute, respondent_id):
    """
    Test for carryover effects by comparing AMCEs across tasks.
    """
    results = {}

    for task in df['task_number'].unique():
        task_df = df[df['task_number'] == task]
        model = estimate_amce_regression(task_df, outcome, [attribute], respondent_id)
        results[task] = model.params

    # Compare coefficients across tasks
    # Use F-test for joint significance of task interactions

    return results

* Test for carryover effects
regress chosen i.language##i.task_number, vce(cluster respondent_id)
testparm i.language#i.task_number

If assumption fails: Use only first task data (loses efficiency).

2. Profile Order Effects (Test Assumption 2)

Compare AMCEs by profile position:

def test_profile_order(df, outcome, attribute, respondent_id):
    """
    Test for profile order effects.
    """
    # Compare AMCEs for profile 1 vs profile 2
    profile1 = df[df['profile_number'] == 1]
    profile2 = df[df['profile_number'] == 2]

    model1 = estimate_amce_regression(profile1, outcome, [attribute], respondent_id)
    model2 = estimate_amce_regression(profile2, outcome, [attribute], respondent_id)

    return model1, model2

3. Randomization Balance Check

Verify attributes are balanced across respondent characteristics:

def balance_check(df, attributes, respondent_chars):
    """
    Check if attributes are balanced across respondent characteristics.
    """
    from scipy import stats

    results = {}
    for char in respondent_chars:
        # Regress respondent characteristic on all attribute dummies
        formula = f"{char} ~ " + " + ".join([f"C({attr})" for attr in attributes])
        model = ols(formula, data=df).fit()

        # F-test for joint significance
        f_stat, p_val = model.fvalue, model.f_pvalue
        results[char] = {'F': f_stat, 'p': p_val}

    return results

4. Attribute Order Effects (External Validity)

Test if AMCEs vary by row position of attribute:

def test_row_order(df, outcome, attribute, respondent_id):
    """
    Test for attribute row order effects.
    Requires randomized attribute ordering across respondents.
    """
    # Interact attribute with its row position
    formula = f"{outcome} ~ C({attribute}) * C(row_position_{attribute})"
    model = ols(formula, data=df).fit(
        cov_type='cluster',
        cov_kwds={'groups': df[respondent_id]}
    )

    # Test joint significance of interaction terms
    return model

5. Atypical Profile Effects

Check if results differ for respondents exposed to more/fewer atypical profiles:

def test_atypical_profiles(df, outcome, attributes, respondent_id, atypical_count):
    """
    Compare AMCEs by number of atypical profiles seen.

    atypical_count : column indicating count of atypical profiles per respondent
    """
    # Split by atypical profile exposure
    low = df[df[atypical_count] <= df[atypical_count].quantile(0.33)]
    high = df[df[atypical_count] >= df[atypical_count].quantile(0.67)]

    model_low = estimate_amce_regression(low, outcome, attributes, respondent_id)
    model_high = estimate_amce_regression(high, outcome, attributes, respondent_id)

    return model_low, model_high

Complete Analysis Example

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt

# Load conjoint data (profile-level)
df = pd.read_csv('conjoint_data.csv')

# 1. Main AMCE estimation
attributes = ['education', 'language', 'origin', 'profession', 'job_plans']

# All AMCEs in one regression with clustered SEs
formula = "chosen ~ C(education) + C(language) + C(origin) + C(profession) + C(job_plans)"
model = smf.ols(formula, data=df).fit(
    cov_type='cluster',
    cov_kwds={'groups': df['respondent_id']}
)
print(model.summary())

# 2. Extract and plot AMCEs
def plot_amces(model, attributes, df):
    """Create coefficient plot for AMCEs."""
    coefs = model.params[1:]  # Exclude intercept
    ses = model.bse[1:]
    names = coefs.index

    fig, ax = plt.subplots(figsize=(10, 12))

    y_pos = np.arange(len(coefs))
    ax.errorbar(coefs, y_pos, xerr=1.96*ses, fmt='o', capsize=3)
    ax.axvline(x=0, color='gray', linestyle='--', alpha=0.5)
    ax.set_yticks(y_pos)
    ax.set_yticklabels(names)
    ax.set_xlabel('AMCE (Change in Pr(Chosen))')
    ax.set_title('Average Marginal Component Effects')

    plt.tight_layout()
    return fig

fig = plot_amces(model, attributes, df)
plt.savefig('amce_plot.png', dpi=150, bbox_inches='tight')

# 3. Conditional AMCEs by respondent subgroup
# Example: By respondent education level
df_college = df[df['respondent_education'] == 'college']
df_no_college = df[df['respondent_education'] != 'college']

model_college = smf.ols(formula, data=df_college).fit(
    cov_type='cluster',
    cov_kwds={'groups': df_college['respondent_id']}
)
model_no_college = smf.ols(formula, data=df_no_college).fit(
    cov_type='cluster',
    cov_kwds={'groups': df_no_college['respondent_id']}
)

# 4. Interaction effects (ACIE)
formula_interact = "chosen ~ C(education) * C(job_plans) + C(language) + C(origin)"
model_interact = smf.ols(formula_interact, data=df).fit(
    cov_type='cluster',
    cov_kwds={'groups': df['respondent_id']}
)

# 5. Diagnostics
# Carryover effects
formula_carryover = "chosen ~ C(language) * C(task_number)"
model_carryover = smf.ols(formula_carryover, data=df).fit(
    cov_type='cluster',
    cov_kwds={'groups': df['respondent_id']}
)

# Profile order effects
formula_order = "chosen ~ C(language) * C(profile_position)"
model_order = smf.ols(formula_order, data=df).fit(
    cov_type='cluster',
    cov_kwds={'groups': df['respondent_id']}
)

Interpretation Guidelines

Interpreting AMCE Values

For choice outcomes (binary):

• AMCE of 0.10 means: "Changing attribute from baseline to treatment increases probability of selection by 10 percentage points"

For rating outcomes (rescaled 0-1):

• AMCE of 0.05 means: "Changing attribute from baseline to treatment increases average rating by 5 percentage points on 0-1 scale"

Comparing Effects

AMCEs are on the same scale and can be directly compared:

• Within attributes: "A college degree (AMCE=0.18) has larger effect than high school (AMCE=0.08)"
• Across attributes: "Education (max AMCE=0.18) matters more than origin (max AMCE=0.05)"

Cautionary Notes

1. Baseline matters: AMCEs are relative to chosen baseline category
2. Distribution matters: AMCE depends on distribution p(t) of other attributes
3. External validity: Results depend on included attributes, levels, and sample
4. Cannot identify preferences: AMCE identifies average effects, not individual utility functions

Best Practices Checklist

Design Phase

• Define clear research questions mapping to attributes
• Choose appropriate number of attributes (5-10)
• Select realistic, substantively meaningful attribute levels
• Implement restrictions for implausible combinations
• Randomize attribute order across respondents
• Pilot test for comprehension and fatigue

Analysis Phase

• Use clustered standard errors (by respondent)
• Report AMCEs with confidence intervals
• Test assumptions (carryover, profile order)
• Check randomization balance
• Examine conditional AMCEs for key subgroups
• Test for interaction effects if theoretically motivated

Reporting

• Describe randomization scheme and any restrictions
• Report sample sizes (respondents, profiles, tasks)
• Present AMCEs visually with coefficient plots
• Discuss baseline categories explicitly
• Report diagnostic test results
• Acknowledge limitations (external validity, stated vs revealed preferences)

References

Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2014). Causal inference in conjoint analysis: Understanding multidimensional choices via stated preference experiments. Political Analysis, 22(1), 1-30.

Software Resources

• R package: cjoint - Implements AMCE estimation with diagnostics
• Stata: conjoint user-written package
• Python: Custom implementation using statsmodels (see examples above)
• Survey tools: Conjoint Survey Design Tool (Strezhnev et al.) for Qualtrics integration