Add p-value brackets, significance asterisks, and effect-size annotations to distribution plots using ggpubr, ggsignif, and statannotations with correct test selection (parametric vs non-parametric vs paired), multiple-testing adjustment, and rendering of negative results. Use when a boxplot/violin/raincloud needs in-figure statistical comparisons between groups.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Add p-value brackets, significance asterisks, and effect-size annotations to distribution plots using ggpubr, ggsignif, and statannotations with correct test selection (parametric vs non-parametric vs paired), multiple-testing adjustment, and rendering of negative results. Use when a boxplot/violin/raincloud needs in-figure statistical comparisons between groups.
Before using code patterns, verify installed versions match. If versions differ:
R: packageVersion('<pkg>') then ?function_name
Python: pip show <package> then help(module.function) to check signatures
If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.
Statistical Annotation
"Add p-values to my plot" -> Render pairwise group comparisons as brackets with the correct statistical test (parametric vs non-parametric, paired vs unpaired, independent vs nested), adjusted for multiple testing, with rendering of significance as either numerical p OR asterisks. The choices that matter: which test is appropriate for the data, what multiple-testing adjustment applies, and whether to show n.s. (non-significant) results.
The Single Most Important Modern Insight -- The Test Must Match the Data
Tool defaults are NOT data-appropriate. ggpubr::stat_compare_means(method='t.test') uses Welch's two-sample t-test assuming approximate normality and unequal variances. This is wrong when:
Data are non-normal and N is small (<30): use Mann-Whitney U (method='wilcox.test').
Data are paired: use paired t-test or paired Wilcoxon (paired = TRUE).
Comparing >2 groups: ANOVA / Kruskal-Wallis with post-hoc, not all-pairs t-test (multiple-testing penalty).
Data are nested (cells within patients, replicates within samples): linear mixed model, NOT pairwise test.
The bracket-and-asterisk visual is the same; the underlying statistics are not. Choose the test deliberately.
'holm' — stepwise Bonferroni; uniformly more powerful than Bonferroni
'BH' (Benjamini-Hochberg) — FDR; less strict than FWER; standard for genomics
'fdr' — alias for BH
For figure annotations, holm is the modern default — controls FWER and is more powerful than bonferroni. For a small number of pre-planned comparisons (≤3), Bonferroni is fine.
ggpubr -- Standard ggplot2 Workflow
Goal: Add per-comparison p-value brackets between groups on a distribution plot, using a test appropriate to data shape and adjusting for multiple comparisons.
Approach: Build the base plot with ggboxplot(); add stat_compare_means() with explicit method, comparisons, p.adjust.method, and label arguments; render as asterisks (p.signif) for terse display or numeric (p.format) for precise display.
library(ggpubr)# Default boxplot + p-value bracket(s)
ggboxplot(df, x ='group', y ='value', color ='group',
add ='jitter', palette ='npg')+
stat_compare_means(method ='wilcox.test',# explicit; default is t-test
comparisons =list(c('Control','Treatment'),c('Control','Vehicle'),c('Treatment','Vehicle')),
label ='p.signif',# 'p.signif' for asterisks; 'p.format' for numeric
p.adjust.method ='holm',
method.args =list(alternative ='two.sided'))
For an overall test plus pairwise:
# Overall + per-comparison
ggboxplot(df, x ='group', y ='value', color ='group')+
stat_compare_means(method ='kruskal.test',# overall test
label.y =1.05*max(df$value))+
stat_compare_means(comparisons = pairs,
method ='wilcox.test',
p.adjust.method ='holm',
label ='p.signif')
ggsignif -- Lighter Alternative
library(ggsignif)
ggplot(df, aes(group, value, fill = group))+
geom_boxplot()+
geom_signif(comparisons =list(c('Control','Treatment')),
test ='wilcox.test',
map_signif_level =TRUE,# asterisks vs numeric p
step_increase =0.1)+
scale_fill_manual(values =c('#0072B2','#D55E00'))
map_signif_level = TRUE converts p-values to asterisks per Wasserstein-Lazar 2016 convention:
Symptom: Reader assumes untested pairs were significant.
Fix: Either annotate all comparisons (with n.s. for non-significant) OR pre-specify which pairs are tested in the legend/caption.
Reading effect size from p-value
Trigger: Conclusion "highly significant difference" from p = 1e-10 on a tiny effect.
Mechanism: Large N inflates significance for trivial differences.
Symptom: Effect size negligible despite extreme p.
Fix: Always report effect size (Cohen's d, Cliff's delta, median difference with CI) alongside p. The bracket should convey direction AND magnitude, not just significance.
Reconciliation: When Tests Disagree
Pattern
Cause
Action
t-test significant; Wilcoxon n.s.
Outliers driving t-test; rank test robust
Trust Wilcoxon for non-normal data
Unpaired n.s.; paired significant
Within-subject correlation matters
Use paired if subjects are matched
Pairwise all-significant; ANOVA n.s.
Multiple-testing inflation in pairwise
ANOVA / Kruskal-Wallis is the overall test; pairwise post-hoc only after omnibus significant
Pseudo-replicated p < 1e-50; LMM p = 0.1
Pseudoreplication
LMM is correct; pseudo-replicated p is meaningless
Bonferroni-adjusted n.s.; raw p < 0.05
Adjustment correctly identified borderline
Trust adjusted; document the test family
Quantitative Thresholds
Threshold
Value
Source
α for FWER control
0.05 family-wise
Standard
α for FDR control
0.05 expected FDR (BH)
Benjamini-Hochberg 1995
Asterisk convention
* <0.05, ** <0.01, *** <0.001
Common practice
Bonferroni cutoff
0.05 / K(K-1)/2
Standard
Holm step-down
better than Bonferroni for all K
Holm 1979
FDR (BH)
less strict than FWER
Genomics standard
Common Errors
Error / symptom
Cause
Solution
Reviewer asks "why t-test?"
Default not justified
Pre-justify test choice
Many pairwise-significant; doesn't replicate
No multiple-testing adjustment
Holm or BH
Effect "highly significant" but tiny
Large N inflates p
Report effect size
Asterisks only; no p-values
label = 'p.signif' exclusively
Show p.format OR provide table
Pseudoreplication inflated p
Cells treated as independent
LMM or pseudobulk
n.s. comparisons hidden
Selective reporting
Annotate all pre-specified pairs
Numeric p truncated to '<2.22e-16'
R default precision
Manual formatting or report as < 2e-16
References
Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57:289-300.
Dunn OJ. 1964. Multiple comparisons using rank sums. Technometrics 6(3):241-252.
Holm S. 1979. A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65-70.
Kassambara A. 2020. Practical Statistics in R for Comparing Groups: Numerical Variables. (ggpubr / rstatix tutorial).
Wasserstein RL, Lazar NA. 2016. The ASA's statement on p-values: context, process, and purpose. Am Stat 70(2):129-133.
