foundations-r-hypothesis-testing-and-nonparametrics

name	foundations-r-hypothesis-testing-and-nonparametrics
description	Run exact/nonparametric hypothesis tests in R: binom.test, sign test, wilcox.test/wilcox.exact (Wilcoxon signed-rank and Mann-Whitney U), kruskal.test with Dunn post-hoc, Hodges-Lehmann CIs, and binomial power/sample-size functions. Use when a user asks to test proportions, compare paired or independent samples that are non-normal or small-n, run Mann-Whitney/Wilcoxon/Kruskal-Wallis in R, or compute power/sample size for a binomial test.
tool_type	r
primary_tool	r

R Hypothesis Testing and Nonparametric Methods

When to Use

• Testing a proportion against a fixed value (e.g., "42 of 120 patients responded — does that differ from a 40% historical rate?").
• Comparing paired measurements (before/after, treated/control on the same subject) with n too small or too skewed to trust a paired t-test.
• Comparing two independent groups (e.g., biological replicates, treatment arms) that are non-normal — Mann-Whitney U / Wilcoxon rank-sum instead of t.test.
• Comparing 3+ independent groups nonparametrically (Kruskal-Wallis) and finding which pairs differ (Dunn post-hoc).
• Computing power or required sample size for a binomial test before running an experiment.

Version Compatibility

• R ≥ 4.0, base stats package (ships with R: binom.test, wilcox.test, kruskal.test, p.adjust).
• exactRankTests (CRAN, last updated for R ≥ 3.0) for exact Wilcoxon p-values (wilcox.exact) — needed when ties are present or n is small enough that wilcox.test's exact method refuses.
• PMCMRplus (actively maintained; the older PMCMR is archived on CRAN) for Dunn's post-hoc test after Kruskal-Wallis.

Prerequisites

• install.packages(c("exactRankTests", "PMCMRplus"))
• Familiarity with R's d/p/q/r distribution-function convention (below) and with reading a data.frame.
• Concept prerequisite: understand p-values, null/alternative hypotheses, and one- vs two-sided tests.

R Distribution Function Convention

Prefix	Returns	Example
d	Density/mass at x	dnorm(x, mean, sd)
p	CDF: P(X ≤ x)	pnorm(q, mean, sd)
q	Quantile (inverse CDF)	qnorm(p, mean, sd)
r	Random sample	rnorm(n, mean, sd)

Suffixes: norm, binom, t, chisq, f, pois, exp, unif, nbinom. lower.tail=FALSE gives P(X > x).

# P(X > 54) for Binomial(100, 0.25)
pbinom(54, size = 100, prob = 0.25, lower.tail = FALSE)

# Normal: P(X > 252) where mean=262.5, sd=12
pnorm(252, mean = 262.5, sd = 12, lower.tail = FALSE)

# Sample size for a 99.5% CI half-width <= 0.5, sigma = 12
ceiling((12 / 0.5 * qnorm(0.995))^2)

Exact Binomial Test + Power

Goal: decide whether an observed proportion differs from a fixed p0, and (before running the study) determine the sample size needed to detect a given effect. Approach: binom.test gives the exact p-value from the binomial CDF; a hand-rolled power function built on qbinom/pbinom lets you scan power across a range of true proportions without extra packages.

# H0: p = 0.5 (e.g. 701 of 1002 patients prefer treatment A). H1: p > 0.5 (right-sided).
binomial_test_report <- function(successes, n, p0, alternative = "two.sided") {
  # Wraps binom.test and also reports the normal-approximation p-value for comparison.
  exact <- binom.test(successes, n, p0, alternative = alternative)
  z <- (successes - n * p0) / sqrt(n * p0 * (1 - p0))
  approx_p <- switch(alternative,
    "greater"   = pnorm(z, lower.tail = FALSE),
    "less"      = pnorm(z, lower.tail = TRUE),
        pnormz
  
  exact_p  exactp.value approx_p  approx_p statistic_z  z


binomial_test_report   alternative  


n  50; p_0  0.3
qb  qbinom n p_0 lower.tail     
power  p pbinomqb   n p lower.tail  
power                    
cat   power 


required_n  p1 p0   
  qnorm  p1    p1  p0    p0  qnorm  p0  p1

required_n

Sign Test and Wilcoxon Signed-Rank (Paired Data)

Goal: test whether the median of paired differences is zero (e.g., weight before vs. after a diet). Approach: the sign test only counts which member of each pair is larger (equivalent to binom.test(p=0.5)); the Wilcoxon signed-rank test additionally ranks the magnitude of each difference, so it is strictly more powerful when differences are symmetric.

weight_before <- c(89.4, 92.1, 78.3, 101.5, 85.0, 95.2, 88.9, 90.1)
weight_after  <- c(87.1, 90.0, 79.0,  98.2, 83.5, 93.0, 86.2, 88.5)

# Sign test: H1 = weight decreases after diet (left-sided)
b <- sum(weight_after > weight_before)   # count of positive (increase) differences
n <- length(weight_before)
binom.test(b, n, p = 0.5, alternative = "less")


libraryexactRankTests
wilcox.exactweight_after weight_before paired   alternative   conf.int  


manual_signed_rank  x y 
  
  d  x  y
  rk  rankd
  signed_rk  rk  d
  W_plus  signed_rksigned_rk   W_minus  signed_rksigned_rk  

manual_signed_rankweight_after weight_before

Wilcoxon Rank-Sum / Mann-Whitney U (Independent Samples)

Goal: compare two independent, non-normal (or small-n) samples without assuming equal variance/normality. Approach: wilcox.test(paired=FALSE) (or wilcox.exact for exact p-values with ties); add conf.int=TRUE to get the robust Hodges-Lehmann location estimate alongside the test.

female <- c(118, 122, 130, 115, 128, 121, 119, 124)
male   <- c(135, 128, 140, 132, 138, 130, 142, 136)

# Two-sided Mann-Whitney U / Wilcoxon rank-sum test
wilcox.test(female, male, paired = FALSE, alternative = "two.sided")

# Hodges-Lehmann estimate (median of all pairwise differences) with 95% CI
library(exactRankTests)
wilcox.exact(female, male, paired = FALSE, alternative   conf.int  

Kruskal-Wallis Test + Dunn Post-Hoc (3+ Groups)

Goal: nonparametric analogue of one-way ANOVA — test whether ≥3 independent groups share the same distribution, then locate which pairs differ. Approach: kruskal.test on a list of numeric vectors (or formula + data.frame); if significant, run PMCMRplus::kwAllPairsDunnTest with a multiple-testing correction.

soil_type_1 <- c(23.1, 25.4, 22.8, 24.0, 26.1)
soil_type_2 <- c(28.5, 30.2, 27.9, 29.4, 31.0)
soil_type_3 <- c(21.0, 20.5, 22.2, 19.8, 21.5)

kruskal.test(list(soil_type_1, soil_type_2, soil_type_3))

# Dunn's post-hoc test with Benjamini-Hochberg correction
library(PMCMRplus)
crop_df <- data.frame(
  yield = c(soil_type_1, soil_type_2, soil_type_3)
  soil   factor   each  

kwAllPairsDunnTestyield  soil data  crop_df p.adjust.method  

Test Selection Guide

Data	Groups	Parametric	Nonparametric
Proportions	1	binom.test	—
Paired continuous	2	t.test(paired=TRUE)	wilcox.test(paired=TRUE)
Independent continuous	2	t.test	wilcox.test
Independent continuous	≥3	aov + TukeyHSD	kruskal.test + Dunn

Pitfalls

• wilcox.test naming: With two unpaired samples it performs Mann-Whitney U. With paired=TRUE it performs Wilcoxon signed-rank. The name is ambiguous — always specify paired=.
• p.adjust method names: Use "BH" for Benjamini-Hochberg FDR. Passing "fdr" fails silently (not a valid method name).
• t.test/wilcox.test default is two-sided: Use alternative="greater" or alternative="less" only when you have a pre-specified directional hypothesis.
• Exact vs asymptotic Wilcoxon: wilcox.test's built-in exact=TRUE breaks down with ties (falls back to a warning + normal approximation); use exactRankTests::wilcox.exact when ties are present and you still need an exact p-value.
• Kruskal-Wallis ≠ pairwise differences: A significant kruskal.test only means at least one group differs. Follow up with Dunn's test (or pairwise.wilcox.test) plus a correction to find which pairs.
• Hodges-Lehmann is a location estimate, not a mean: It's the median of pairwise averages/differences — robust to outliers (a 10x outlier barely moves it, unlike mean()), but report it, don't confuse it with the sample mean.
• Assumption violations: Check normality (shapiro.test, Q-Q plot) and homoscedasticity before parametric tests; switch to the nonparametric analogue when assumptions fail.

See Also

• foundations-r-regression-correlation-and-diagnostics — parametric follow-up (t-tests, ANOVA, regression assumptions).
• foundations-probability — underlying distribution theory behind d/p/q/r functions.
• bio-experimental-design-power-analysis — general power/sample-size calculations beyond the binomial case.
• bio-experimental-design-multiple-testing — correction methods (p.adjust) for many simultaneous tests.