Implements multiplicity control for confirmatory clinical trials using graphical procedures (Bretz-Maurer-Hommel), gatekeeping (parallel, serial, mixed), Hochberg/Hommel/Holm with PRDS, and the closed-testing principle (Marcus-Peritz-Gabriel; Goeman 2021 admissibility). Covers FDA Multiple Endpoints Final Guidance (October 2022), graphical procedures via R gMCP, primary + key-secondary + subgroup hierarchies, and FWER vs FDR distinction. Use when designing the multiplicity strategy for confirmatory trials with multiple primary or key secondary endpoints.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Implements multiplicity control for confirmatory clinical trials using graphical procedures (Bretz-Maurer-Hommel), gatekeeping (parallel, serial, mixed), Hochberg/Hommel/Holm with PRDS, and the closed-testing principle (Marcus-Peritz-Gabriel; Goeman 2021 admissibility). Covers FDA Multiple Endpoints Final Guidance (October 2022), graphical procedures via R gMCP, primary + key-secondary + subgroup hierarchies, and FWER vs FDR distinction. Use when designing the multiplicity strategy for confirmatory trials with multiple primary or key secondary endpoints.
tool_type
r
primary_tool
gMCP
goal_approach_exempt
true
Version Compatibility
Reference examples tested with: R gMCP 0.8.16+, graphicalMCP 0.2+, gatekeeping, multcomp, multxpert; Python statsmodels 0.14+ for basic FDR/FWER methods.
Before using code patterns, verify installed versions match. If versions differ:
R: packageVersion('<pkg>') then ?function_name
Python: pip show <package> then
help(module.function)
If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.
Multiplicity Control for Confirmatory Trials
"Design the multiplicity strategy for my trial" -> Specify a closed-testing procedure (graphical, gatekeeping, hierarchical, or step-down Bonferroni-Holm) that controls family-wise error rate at the trial-wide level across primary endpoints, key secondary endpoints, and subgroup analyses, with provable strong FWER control.
The Foundational Theorem -- Closed Testing Is Necessary
Marcus, Peritz & Gabriel 1976 Biometrika 63:655: a hypothesis H_I (I ⊆ {1,...,m}) is rejected iff every intersection hypothesis ∩_{J⊇I} H_J is rejected by a valid α-level local test. Strong FWER control holds for ANY choice of local tests.
Goeman, Hemerik & Solari 2021 Ann Stat 49:1218 tightens this: closed testing is not merely sufficient — it is necessary for admissibility under FDP/FWER/k-FWER. Every admissible multiplicity procedure is equivalent to some closed test. Graphical procedures, gatekeepers, Hommel, fixed-sequence, fallback — all are closed tests in disguise.
FWER vs FDR philosophical divide:
FWER: P(any false positive among m tests) — regulatory standard for confirmatory inference (agency wants to bound per-trial false-positive rate)
FDR: Expected proportion of false discoveries among rejections — exploratory standard (genomics, fMRI, biomarker screens) where many true positives expected
Confirmatory clinical trials use FWER essentially universally.
Algorithmic Taxonomy
Procedure
Type
FWER control
Power profile
Use case
Bonferroni
Single-step
Yes, any dependence
Conservative; loses 30-50% power vs Hommel under positive dependence
Very small m; worst-case dependence
Holm 1979
Step-down
Yes, any dependence
Better than Bonferroni; uniformly dominates
Default for any dependence pattern
Hochberg 1988
Step-up
Yes under PRDS (Sarkar 1998)
Better than Holm under PRDS
Positive correlation; verify PRDS
Hommel 1988
Step-up via closed tests
Yes under PRDS
Uniformly dominates Hochberg by 1-3%
Whenever Hochberg is valid
Fixed-sequence (hierarchical)
Sequential
Yes, any dependence
Full alpha for first; subsequent zero if any fail
When clear priority ordering; "key secondary" labelling
Parallel gatekeeping (Dmitrienko 2003)
Multi-family
Yes
Family-by-family; secondary tested if any primary rejects
Primary family + secondary family
Serial gatekeeping
Sequential families
Yes
Strict: family k tested only if ALL of family k-1 reject
Co-primary + secondary tiers
Mixed gatekeeping (Dmitrienko-Tamhane 2008)
Combination
Yes
Combines closed-testing local procedures across families
Complex hierarchies
Graphical procedures (Bretz-Maurer 2009)
Closed-test as directed graph
Yes by construction
Flexible; allocate alpha to hypotheses via graph weights
gsDesign or rpact with multivariate alpha spending
Hierarchical alpha across both time and endpoints
Bretz-Maurer Graphical Procedures -- The Modern Standard
The Bretz-Maurer-Brannath-Posch 2009 Stat Med 28:586 framework recast weighted Bonferroni-Holm closed tests as directed weighted graphs:
Vertices = elementary null hypotheses with local weights summing to 1
Directed edges = alpha-propagation rule (when a hypothesis is rejected, its weight redistributes to descendants per edge weights)
The graph IS the procedure: a single visual fully specifies a closed-test procedure across primary, key secondary, and subgroup hierarchies
gMCP R package
library(gMCP)# Construct a graph for primary + 2 key secondary endpoints# Primary endpoint at full alpha; if rejected, alpha propagates equally to secondaries
hypotheses <-c('Primary','Sec1','Sec2')
weights <-c(1,0,0)# initial alpha all on primary# Transition matrix: rows = source, columns = target# When Primary rejects, weight 0.5 goes to each secondary; when Sec1/Sec2 rejects, alpha returns
transitions <- matrix(c(0,0.5,0.5,0,0,1,0,1,0), nrow =3, byrow =TRUE,dimnames=list(hypotheses, hypotheses))
graph <- graphMCP(m = transitions, weights = weights, hnames = hypotheses)# Note: in current gMCP, the graph constructor is `graphMCP(m=, weights=, hnames=)`;# `matrix2graph()` appeared in older tutorials and is not the canonical exported API# -- verify with `?graphMCP` / `?gMCP` in the installed gMCP release before scripting.# Set p-values from the trial
p_vals <-c(Primary =0.018, Sec1 =0.042, Sec2 =0.038)# Run the graphical procedure at alpha = 0.025
result <- gMCP(graph, pvalues = p_vals, alpha =0.025)
print(result)# Hierarchical rejection: Primary rejects -> alpha propagates to secondaries -> ...
Bretz et al 2011 -- Simes and parametric extensions
When endpoints are positively correlated, replace the Bonferroni-based intersection test with Simes (for positive dependence) or parametric (using known correlation):
library(gMCP)# Use Simes-based local tests at each intersection
result_simes <- gMCP(graph, pvalues = p_vals, alpha =0.025, test ='Simes')# Or parametric with estimated correlation matrix
result_param <- gMCP(graph, pvalues = p_vals, alpha =0.025, corr = correlation_matrix)
Maurer-Bretz 2013 entangled graphs
Entangled graphs add memory: the alpha propagation can depend on the origin of the alpha. This allows parent-descendant constraints that a single non-entangled graph cannot express. Example: secondary endpoint Sec1 receives alpha only from Primary, never from Sec2.
Postdoc argument: purists argue memory makes the procedure non-coherent in Gabriel's sense; Glimm/Maurer/Bretz argue it matches real-world inferential intent.
Gabriel coherence in plain terms: a coherent procedure rejects a hypothesis H consistently regardless of which superset of H is being tested. Non-entangled graphs are coherent: if H1 is rejected via path A, it would also be rejected via path B. Entangled (memory-bearing) graphs sacrifice coherence: the same H may be rejected when alpha arrives from one parent but not from another, because the propagation history changes the available alpha. The trade-off is operational power -- entangled graphs can encode "secondary X is meaningful only if primary Y rejects, not if primary Z rejects" inferential intent that flat coherent procedures cannot express. Choose based on whether the SAP needs path-dependent priority.
Gatekeeping Procedures
Serial gatekeeping (hierarchical)
Test H1 at full alpha; only if it rejects, test H2 at full alpha; etc. Maximises power for H1 but H_k becomes inferentially worthless once any H_j (j<k) fails.
# Hierarchical / serial: just a chain graph in gMCP
hyp <-c('H1','H2','H3','H4')
weights <-c(1,0,0,0)
trans <- matrix(c(0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0),
nrow=4,dimnames=list(hyp, hyp))
graph <- graphMCP(m = trans, weights = weights, hnames = hyp)
Pre-specification of order is critical — based on clinical importance, NOT expected effect size. Ordering by expected effect is data-driven and inflates Type-I.
Parallel gatekeeping (Dmitrienko 2003)
Secondary family is tested only if at least one primary rejects. Bonferroni-based parallel gatekeeper has stepwise representation (Guilbaud 2007 Biom J 49:917).
Mixed / multistage (Dmitrienko-Tamhane 2008)
Permits using any closed-testing local procedure (e.g., Holm in family 1, Hommel in family 2) and combining via closure principle. R gMCP::generalMixGatekeeping or Mediana/MultXpert packages.
Postdoc tradeoff: parallel gatekeeping power loss vs collapsing endpoints into a composite (which avoids multiplicity but dilutes effect if components move in opposite directions); whether tree gatekeeping (Dmitrienko et al 2008 Stat Med 27:3446) over-engineers vs equivalent graphical procedure.
Hochberg vs Hommel vs Holm
Holm 1979 — step-down rejective Bonferroni; FWER controlled under any joint dependence. Conservative but robust.
Hochberg 1988 — step-up using ordered Simes critical values; needs Simes inequality which requires PRDS (Sarkar 1998, 2008). Under PRDS, Hochberg uniformly dominates Holm.
Hommel 1988 — also Simes-based but uses closed-testing tableau directly (not step-up shortcut). Uniformly more powerful than Hochberg (typically 1-3% gain).
When Hochberg fails (anti-conservative)
Hochberg becomes Type-I-inflated under negative dependence — relevant when comparing endpoints mathematically constrained to move in opposite directions (LDL-C and HDL-C; complementary efficacy and safety endpoints).
Sarkar critique: when PRDS cannot be proven, fall back to Holm. The lost power is the price of robustness.
FDA Multiple Endpoints Final Guidance (October 2022)
Federal Register 2022-22882 finalises 2017 draft. Key changes vs draft:
Explicit recognition of newer methods including win-ratio (Pocock 2012 Eur Heart J) and weighted composites
Clearer language that "key secondary" endpoints are those for which sponsor wishes to make label claims and which must be in a Type-I-error-controlled hierarchy
Appendix with worked graphical-procedure examples
Categories
Category
Approach
Note
Composite
Single test; no multiplicity
Win-ratio, DOOR/RADAR, time-to-first-event
Co-primary (all-win)
Each at full alpha; n inflated for joint power
Power = product of marginals
Multiple primary (any-wins)
Alpha must be split (Bonferroni or graphical)
More n required than co-primary if effects similar
Primary + key secondary
Hierarchical or graphical
Modern preference: graphical for flexibility
Winner's bias warning: when post-hoc-selected endpoints are emphasised, bias-corrected effect estimates are recommended (same selection-bias issue as adaptive design).
The "Almighty Primary Endpoint" Critique
Dmitrienko-D'Agostino 2017 Stat Med 36:4423 editorial surveys progress in trial-multiplicity methodology. A recurring theme motivating that work: insisting on a single primary endpoint can lose power when a therapy has broad multi-domain benefit (heart failure drugs with effects on mortality, hospitalisation, symptoms, biomarkers) -- motivating composite endpoints, the win ratio, or multiple primary endpoints with explicit alpha allocation.
Win-ratio (Pocock-Ariti-Collier-Wang 2012) and hierarchical composite (DOOR/RADAR, Evans 2015) are responses — they preserve a single inferential test while letting multiple endpoints contribute.
FDA counter-position (Hung, O'Neill, Wang): without a designated primary, sponsors and regulators negotiate over secondary endpoints post hoc, destroying inferential meaning. Hence the FDA 2022 guidance reaffirms key-secondary hierarchies.
Per-Method Failure Modes
Hochberg under negative dependence
Trigger: Endpoints constrained to move in opposite directions (LDL vs HDL; efficacy vs harm).
Sensitivity analyses provided: Bonferroni, Holm, Hochberg, Hommel results all in CSR appendix; concordant
"Where is alpha for the subgroup analysis?"
Pre-specified 20% of primary alpha allocated (a common convention); cite Dane 2019 for subgroup discipline
"Why not just composite endpoint?"
Composite would dilute differential effect on mortality vs hospitalisation; key-secondary hierarchy preserves component-level claims
"PRDS check for Hochberg?"
Endpoints positively correlated via simulation under null; PRDS holds; Hochberg/Hommel valid
"Sensitivity in the hierarchy?"
No — sensitivity is "what if" and does not require alpha. Listed as supportive not key secondary.
References
Bretz F, Maurer W, Brannath W, Posch M. 2009. A graphical approach to sequentially rejective multiple test procedures. Stat Med 28:586-604.
Bretz F, Posch M, Glimm E, Klinglmueller F, Maurer W, Rohmeyer K. 2011. Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biom J 53:894-913.
Burman CF, Sonesson C, Guilbaud O. 2009. A recycling framework for the construction of Bonferroni-based multiple tests. Stat Med 28:739-761.
Dmitrienko A, Offen WW, Westfall PH. 2003. Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat Med 22:2387-2400.
Dmitrienko A, Tamhane AC, Wiens BL. 2008. General multistage gatekeeping procedures. Biom J 50:667-677.
FDA. 2022. Multiple Endpoints in Clinical Trials. Final Guidance.
Goeman JJ, Hemerik J, Solari A. 2021. Only closed testing procedures are admissible for controlling false discovery proportions. Ann Stat 49:1218-1238.
Guilbaud O. 2007. Bonferroni parallel gatekeeping -- transparent generalizations, adjusted p-values, and short proofs. Biom J 49:917-927.
Hochberg Y. 1988. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75:800-802.
Holm S. 1979. A simple sequentially rejective multiple test procedure. Scand J Stat 6:65-70.
Hommel G. 1988. A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75:383-386.
Marcus R, Peritz E, Gabriel KR. 1976. On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63:655-660.
Maurer W, Bretz F. 2013. Memory and other properties of multiple test procedures generated by entangled graphs. Stat Med 32:1739-1753.
Pocock SJ, Ariti CA, Collier TJ, Wang D. 2012. The win ratio: a new approach to the analysis of composite endpoints in clinical trials. Eur Heart J 33:176-182.
Sarkar SK. 2008. Generalizing Simes' test and Hochberg's stepup procedure. Ann Stat 36:337-363.
Related Skills
clinical-biostatistics/trial-reporting - Multiplicity strategy reporting per CONSORT 2025
