statistics-2

name

statistics-2

description

Comprehensive statistical testing library with 37+ methods for normality tests, location tests, correlation tests, time series tests, and model diagnostics. Use when performing hypothesis testing, A/B testing, data quality checks, time series analysis, or regression model validation. All methods return unified TestResult objects with consistent interface including p-value, statistic, confidence interval, and effect size.

Pywayne Statistics

Comprehensive statistical testing library for hypothesis testing, A/B testing, and data analysis.

Quick Start

from pywayne.statistics import NormalityTests, LocationTests
import numpy as np

# Test data normality
nt = NormalityTests()
data = np.random.normal(0, 1, 100)
result = nt.shapiro_wilk(data)
print(f"p-value: {result.p_value:.4f}, is_normal: {not result.reject_null}")

# Compare two groups
lt = LocationTests()
group_a = np.random.normal(100, 15, 50)
group_b = np.random.normal(105, 15, 50)
result = lt.two_sample_ttest(group_a, group_b)
print(f"Significant difference: {result.reject_null}")

Test Categories

NormalityTests (NormalityTests)

Test if data follows a normal distribution or other specified distributions.

Method	Description	Use Case
shapiro_wilk	Shapiro-Wilk test	Small-medium samples (n ≤ 5000)
ks_test_normal	K-S normality test	Medium-large samples
ks_test_two_sample	Two-sample K-S test	Compare two sample distributions
anderson_darling	Anderson-Darling test	Tail-sensitive normality test
dagostino_pearson	D'Agostino-Pearson K²	Based on skewness and kurtosis
jarque_bera	Jarque-Bera test	Large samples, regression residuals
chi_square_goodness_of_fit	Chi-square goodness-of-fit	Categorical data
lilliefors_test	Lilliefors test	Unknown parameters K-S test

Example:

from pywayne.statistics import NormalityTests

nt = NormalityTests()
result = nt.shapiro_wilk(data)
if result.p_value < 0.05:
    print("Data is NOT normally distributed")
else:
    print("Data follows normal distribution")

LocationTests (LocationTests)

Compare means or medians across groups (parametric and non-parametric).

Method	Description	Use Case
one_sample_ttest	One-sample t-test	Compare sample mean to a value
two_sample_ttest	Two-sample t-test	Compare two independent group means
paired_ttest	Paired t-test	Compare before/after measurements
one_way_anova	One-way ANOVA	Compare 3+ group means
mann_whitney_u	Mann-Whitney U test	Non-parametric two-sample test
wilcoxon_signed_rank	Wilcoxon signed-rank	Non-parametric paired test
kruskal_wallis	Kruskal-Wallis H test	Non-parametric multi-group test

Example (A/B Testing):

from pywayne.statistics import LocationTests, NormalityTests

lt = LocationTests()
nt = NormalityTests()

# Check normality first
if nt.shapiro_wilk(control).p_value > 0.05:
    result = lt.two_sample_ttest(control, treatment)
else:
    result = lt.mann_whitney_u(control, treatment)

print(f"Effect significant: {result.reject_null}")

CorrelationTests (CorrelationTests)

Test correlation between variables and independence of categorical variables.

Method	Description	Use Case
pearson_correlation	Pearson correlation	Linear relationship
spearman_correlation	Spearman's rank	Monotonic relationship
kendall_tau	Kendall's tau	Rank correlation, small samples
chi_square_independence	Chi-square independence	Categorical variables
fisher_exact_test	Fisher's exact test	2×2 contingency table
mcnemar_test	McNemar's test	Paired categorical data

Example:

from pywayne.statistics import CorrelationTests

ct = CorrelationTests()
result = ct.pearson_correlation(x, y)
print(f"Correlation: {result.statistic:.3f}, p-value: {result.p_value:.4f}")

TimeSeriesTests (TimeSeriesTests)

Test time series properties: stationarity, autocorrelation, cointegration.

Method	Description	Use Case
adf_test	Augmented Dickey-Fuller	Unit root test for stationarity
kpss_test	KPSS test	Stationarity test (complements ADF)
ljung_box_test	Ljung-Box Q test	Overall autocorrelation
runs_test	Runs test	Randomness testing
arch_test	ARCH effect test	Heteroscedasticity
granger_causality	Granger causality	Causal relationship
engle_granger_cointegration	Engle-Granger cointegration	Long-term equilibrium
breusch_godfrey_test	Breusch-Godfrey	Higher-order autocorrelation

Example:

from pywayne.statistics import TimeSeriesTests

tst = TimeSeriesTests()
adf_result = tst.adf_test(time_series_data)
kpss_result = tst.kpss_test(time_series_data)

if adf_result.reject_null:
    print("Series is stationary")
else:
    print("Series has unit root (non-stationary)")

ModelDiagnostics (ModelDiagnostics)

Regression model diagnostics: heteroscedasticity, autocorrelation, multicollinearity.

Method	Description	Use Case
breusch_pagan_test	Breusch-Pagan	Heteroscedasticity test
white_test	White's test	General heteroscedasticity
goldfeld_quandt_test	Goldfeld-Quandt	Structural break heteroscedasticity
durbin_watson_test	Durbin-Watson	First-order autocorrelation
variance_inflation_factor	VIF	Multicollinearity diagnosis
levene_test	Levene's test	Homogeneity of variance
bartlett_test	Bartlett's test	Homogeneity (normal assumption)
residual_normality_test	Residual normality	Regression assumption check

Example:

from pywayne.statistics import ModelDiagnostics

md = ModelDiagnostics()
residuals = y - model.predict(X)

# Check assumptions
bp_result = md.breusch_pagan_test(residuals, X)
dw_result = md.durbin_watson_test(residuals)

if bp_result.reject_null:
    print("Warning: Heteroscedasticity detected")

TestResult Object

All test methods return a unified TestResult object:

result = nt.shapiro_wilk(data)

# Access results
result.test_name        # Test method name
result.statistic        # Test statistic value
result.p_value          # P-value
result.reject_null      # True if null hypothesis is rejected
result.critical_value   # Critical value (if applicable)
result.confidence_interval # Tuple (lower, upper) if applicable
result.effect_size      # Effect size if applicable
result.additional_info  # Dict with additional information

Utility Functions

list_all_tests()

List all available test methods across all modules.

from pywayne.statistics import list_all_tests
print(list_all_tests())

show_test_usage(method_name)

Display usage and documentation for a specific test.

from pywayne.statistics import show_test_usage
show_test_usage('shapiro_wilk')

Method Selection Guide

Normality Tests

Sample Size	Recommended Method
n < 30	Shapiro-Wilk
30 ≤ n ≤ 300	Shapiro-Wilk, D'Agostino-Pearson
n > 300	Jarque-Bera, Kolmogorov-Smirnov

Location Tests

Condition	Parametric	Non-parametric
Normal data	t-test, ANOVA	-
Non-normal data	-	Mann-Whitney U, Kruskal-Wallis
Paired data	Paired t-test	Wilcoxon signed-rank

Multiple Testing Correction

When performing multiple tests, apply p-value correction:

from statsmodels.stats.multitest import multipletests

p_values = [r.p_value for r in results]
rejected, p_corrected, _, _ = multipletests(
    p_values, alpha=0.05, method='fdr_bh'
)

Common Applications

Data Quality Check

def data_quality_check(data):
    nt = NormalityTests()
    lt = LocationTests()

    normality = nt.shapiro_wilk(data)

    # Outlier detection (IQR)
    Q1, Q3 = np.percentile(data, [25, 75])
    IQR = Q3 - Q1
    outliers = data[(data < Q1 - 1.5*IQR) | (data > Q3 + 1.5*IQR)]

    return {
        'size': len(data),
        'is_normal': not normality.reject_null,
        'p_value': normality.p_value,
        'outliers': len(outliers)
    }

A/B Testing Workflow

def ab_test_analysis(control, treatment):
    nt = NormalityTests()
    lt = LocationTests()

    # Check normality
    norm_c = nt.shapiro_wilk(control[:100])
    norm_t = nt.shapiro_wilk(treatment[:100])

    # Choose appropriate test
    if norm_c.p_value > 0.05 and norm_t.p_value > 0.05:
        result = lt.two_sample_ttest(control, treatment)
    else:
        result = lt.mann_whitney_u(control, treatment)

    return {
        'test_used': result.test_name,
        'p_value': result.p_value,
        'significant': result.reject_null,
        'effect_size': result.effect_size
    }

Regression Model Diagnostics

def diagnose_model(y, X, model):
    md = ModelDiagnostics()
    residuals = y - model.predict(X)

    return {
        'heteroscedasticity_bp': md.breusch_pagan_test(residuals, X).reject_null,
        'autocorrelation_dw': md.durbin_watson_test(residuals).statistic,
        'residuals_normal': md.residual_normality_test(residuals).p_value,
        'vif_max': max(md.variance_inflation_factor(X))
    }

Notes

• All methods accept np.ndarray or list as input
• All methods return TestResult with consistent interface
• Always validate test assumptions before applying parametric tests
• Apply multiple testing correction when performing several tests
• Report effect sizes alongside p-values for complete interpretation