| name | foundations-r-hypothesis-testing-and-nonparametrics |
| description | Run exact/nonparametric hypothesis tests in R: binom.test, sign test, wilcox.test/wilcox.exact (Wilcoxon signed-rank and Mann-Whitney U), kruskal.test with Dunn post-hoc, Hodges-Lehmann CIs, and binomial power/sample-size functions. Use when a user asks to test proportions, compare paired or independent samples that are non-normal or small-n, run Mann-Whitney/Wilcoxon/Kruskal-Wallis in R, or compute power/sample size for a binomial test. |
| tool_type | r |
| primary_tool | r |
R Hypothesis Testing and Nonparametric Methods
When to Use
- Testing a proportion against a fixed value (e.g., "42 of 120 patients responded — does that differ from a 40% historical rate?").
- Comparing paired measurements (before/after, treated/control on the same subject) with
n too small or too skewed to trust a paired t-test.
- Comparing two independent groups (e.g., biological replicates, treatment arms) that are non-normal — Mann-Whitney U / Wilcoxon rank-sum instead of
t.test.
- Comparing 3+ independent groups nonparametrically (Kruskal-Wallis) and finding which pairs differ (Dunn post-hoc).
- Computing power or required sample size for a binomial test before running an experiment.
Version Compatibility
- R ≥ 4.0, base
stats package (ships with R: binom.test, wilcox.test, kruskal.test, p.adjust).
exactRankTests (CRAN, last updated for R ≥ 3.0) for exact Wilcoxon p-values (wilcox.exact) — needed when ties are present or n is small enough that wilcox.test's exact method refuses.PMCMRplus (actively maintained; the older PMCMR is archived on CRAN) for Dunn's post-hoc test after Kruskal-Wallis.
Prerequisites
install.packages(c("exactRankTests", "PMCMRplus"))- Familiarity with R's
d/p/q/r distribution-function convention (below) and with reading a data.frame.
- Concept prerequisite: understand p-values, null/alternative hypotheses, and one- vs two-sided tests.
R Distribution Function Convention
| Prefix | Returns | Example |
|---|
d | Density/mass at x | dnorm(x, mean, sd) |
p | CDF: P(X ≤ x) | pnorm(q, mean, sd) |
q | Quantile (inverse CDF) | qnorm(p, mean, sd) |
r | Random sample | rnorm(n, mean, sd) |
Suffixes: norm, binom, t, chisq, f, pois, exp, unif, nbinom.
lower.tail=FALSE gives P(X > x).
pbinom(54, size = 100, prob = 0.25, lower.tail = FALSE)
pnorm(252, mean = 262.5, sd = 12, lower.tail = FALSE)
ceiling((12 / 0.5 * qnorm(0.995))^2)
Exact Binomial Test + Power
Goal: decide whether an observed proportion differs from a fixed p0, and (before running the study) determine the sample size needed to detect a given effect.
Approach: binom.test gives the exact p-value from the binomial CDF; a hand-rolled power function built on qbinom/pbinom lets you scan power across a range of true proportions without extra packages.
binomial_test_report <- function(successes, n, p0, alternative = "two.sided") {
exact <- binom.test(successes, n, p0, alternative = alternative)
z <- (successes - n * p0) / sqrt(n * p0 * (1 - p0))
approx_p <- switch(alternative,
"greater" = pnorm(z, lower.tail = FALSE),
"less" = pnorm(z, lower.tail = TRUE),
pnormz
exact_p exactp.value approx_p approx_p statistic_z z
binomial_test_report alternative
n 50; p_0 0.3
qb qbinom n p_0 lower.tail
power p pbinomqb n p lower.tail
power
cat power
required_n p1 p0
qnorm p1 p1 p0 p0 qnorm p0 p1
required_n
Sign Test and Wilcoxon Signed-Rank (Paired Data)
Goal: test whether the median of paired differences is zero (e.g., weight before vs. after a diet).
Approach: the sign test only counts which member of each pair is larger (equivalent to binom.test(p=0.5)); the Wilcoxon signed-rank test additionally ranks the magnitude of each difference, so it is strictly more powerful when differences are symmetric.
weight_before <- c(89.4, 92.1, 78.3, 101.5, 85.0, 95.2, 88.9, 90.1)
weight_after <- c(87.1, 90.0, 79.0, 98.2, 83.5, 93.0, 86.2, 88.5)
b <- sum(weight_after > weight_before)
n <- length(weight_before)
binom.test(b, n, p = 0.5, alternative = "less")
libraryexactRankTests
wilcox.exactweight_after weight_before paired alternative conf.int
manual_signed_rank x y
d x y
rk rankd
signed_rk rk d
W_plus signed_rksigned_rk W_minus signed_rksigned_rk
manual_signed_rankweight_after weight_before
Wilcoxon Rank-Sum / Mann-Whitney U (Independent Samples)
Goal: compare two independent, non-normal (or small-n) samples without assuming equal variance/normality.
Approach: wilcox.test(paired=FALSE) (or wilcox.exact for exact p-values with ties); add conf.int=TRUE to get the robust Hodges-Lehmann location estimate alongside the test.
female <- c(118, 122, 130, 115, 128, 121, 119, 124)
male <- c(135, 128, 140, 132, 138, 130, 142, 136)
wilcox.test(female, male, paired = FALSE, alternative = "two.sided")
library(exactRankTests)
wilcox.exact(female, male, paired = FALSE, alternative conf.int
Kruskal-Wallis Test + Dunn Post-Hoc (3+ Groups)
Goal: nonparametric analogue of one-way ANOVA — test whether ≥3 independent groups share the same distribution, then locate which pairs differ.
Approach: kruskal.test on a list of numeric vectors (or formula + data.frame); if significant, run PMCMRplus::kwAllPairsDunnTest with a multiple-testing correction.
soil_type_1 <- c(23.1, 25.4, 22.8, 24.0, 26.1)
soil_type_2 <- c(28.5, 30.2, 27.9, 29.4, 31.0)
soil_type_3 <- c(21.0, 20.5, 22.2, 19.8, 21.5)
kruskal.test(list(soil_type_1, soil_type_2, soil_type_3))
library(PMCMRplus)
crop_df <- data.frame(
yield = c(soil_type_1, soil_type_2, soil_type_3)
soil factor each
kwAllPairsDunnTestyield soil data crop_df p.adjust.method
Test Selection Guide
| Data | Groups | Parametric | Nonparametric |
|---|
| Proportions | 1 | binom.test | — |
| Paired continuous | 2 | t.test(paired=TRUE) | wilcox.test(paired=TRUE) |
| Independent continuous | 2 | t.test | wilcox.test |
| Independent continuous | ≥3 | aov + TukeyHSD | kruskal.test + Dunn |
Pitfalls
wilcox.test naming: With two unpaired samples it performs Mann-Whitney U. With paired=TRUE it performs Wilcoxon signed-rank. The name is ambiguous — always specify paired=.p.adjust method names: Use "BH" for Benjamini-Hochberg FDR. Passing "fdr" fails silently (not a valid method name).t.test/wilcox.test default is two-sided: Use alternative="greater" or alternative="less" only when you have a pre-specified directional hypothesis.- Exact vs asymptotic Wilcoxon:
wilcox.test's built-in exact=TRUE breaks down with ties (falls back to a warning + normal approximation); use exactRankTests::wilcox.exact when ties are present and you still need an exact p-value.
- Kruskal-Wallis ≠ pairwise differences: A significant
kruskal.test only means at least one group differs. Follow up with Dunn's test (or pairwise.wilcox.test) plus a correction to find which pairs.
- Hodges-Lehmann is a location estimate, not a mean: It's the median of pairwise averages/differences — robust to outliers (a 10x outlier barely moves it, unlike
mean()), but report it, don't confuse it with the sample mean.
- Assumption violations: Check normality (
shapiro.test, Q-Q plot) and homoscedasticity before parametric tests; switch to the nonparametric analogue when assumptions fail.
See Also
foundations-r-regression-correlation-and-diagnostics — parametric follow-up (t-tests, ANOVA, regression assumptions).foundations-probability — underlying distribution theory behind d/p/q/r functions.bio-experimental-design-power-analysis — general power/sample-size calculations beyond the binomial case.bio-experimental-design-multiple-testing — correction methods (p.adjust) for many simultaneous tests.
