Perform statistical tests, hypothesis testing, correlation analysis, and multiple testing corrections using scipy and statsmodels. Works with ANY LLM provider (GPT, Gemini, Claude, etc.).
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
Perform statistical tests, hypothesis testing, correlation analysis, and multiple testing corrections using scipy and statsmodels. Works with ANY LLM provider (GPT, Gemini, Claude, etc.).
Statistical Analysis (Universal)
Overview
This skill enables you to perform rigorous statistical analyses including t-tests, ANOVA, correlation analysis, hypothesis testing, and multiple testing corrections. Unlike cloud-hosted solutions, this skill uses standard Python statistical libraries (scipy, statsmodels, numpy) and executes locally in your environment, making it compatible with ALL LLM providers including GPT, Gemini, Claude, DeepSeek, and Qwen.
When to Use This Skill
Compare means between groups (t-tests, ANOVA)
Test for correlations between variables
Perform hypothesis testing with p-value calculation
# Use when data is not normally distributed# Mann-Whitney U test (alternative to t-test)
u_statistic, p_value_mw = mannwhitneyu(group1, group2, alternative='two-sided')
print(f"Mann-Whitney U test:")
print(f"U-statistic: {u_statistic:.4f}")
print(f"p-value: {p_value_mw:.4e}")
# Kruskal-Wallis H test (alternative to ANOVA)
h_statistic, p_value_kw = kruskal(*groups)
print(f"\nKruskal-Wallis H test:")
print(f"H-statistic: {h_statistic:.4f}")
print(f"p-value: {p_value_kw:.4e}")
Advanced Features
Normality Testing
from scipy.stats import shapiro, normaltest, kstest
# Test if data follows normal distribution# Shapiro-Wilk test (best for n < 5000)
stat_sw, p_sw = shapiro(data)
print(f"Shapiro-Wilk test: W={stat_sw:.4f}, p={p_sw:.4e}")
# D'Agostino-Pearson test
stat_dp, p_dp = normaltest(data)
print(f"D'Agostino-Pearson test: stat={stat_dp:.4f}, p={p_dp:.4e}")
# Interpretationif p_sw < 0.05:
print("❌ Data does NOT follow normal distribution (p < 0.05)")
print("→ Recommendation: Use non-parametric tests (Mann-Whitney, Kruskal-Wallis)")
else:
print("✅ Data appears normally distributed (p >= 0.05)")
print("→ OK to use parametric tests (t-test, ANOVA)")
Chi-Square Test for Contingency Tables
# Test independence between categorical variables# contingency_table: 2D array (rows=categories1, columns=categories2)# Example: Cell type distribution across conditions
contingency_table = np.array([
[50, 30, 20], # Condition A: T cells, B cells, NK cells
[40, 45, 15], # Condition B
[35, 25, 40] # Condition C
])
chi2, p_value, dof, expected = chi2_contingency(contingency_table)
print(f"Chi-square statistic: {chi2:.4f}")
print(f"p-value: {p_value:.4e}")
print(f"Degrees of freedom: {dof}")
print(f"\nExpected frequencies:\n{expected}")
if p_value < 0.05:
print("✅ Significant association between variables (p < 0.05)")
else:
print("❌ No significant association")
Confidence Intervals
from scipy.stats import t as t_dist
defcalculate_confidence_interval(data, confidence=0.95):
"""Calculate confidence interval for mean"""
n = len(data)
mean = np.mean(data)
std_err = stats.sem(data) # Standard error of mean# t-distribution critical value
t_crit = t_dist.ppf((1 + confidence) / 2, df=n-1)
margin_error = t_crit * std_err
ci_lower = mean - margin_error
ci_upper = mean + margin_error
return mean, ci_lower, ci_upper
# Usage
mean, ci_low, ci_high = calculate_confidence_interval(data, confidence=0.95)
print(f"Mean: {mean:.4f}")
print(f"95% CI: [{ci_low:.4f}, {ci_high:.4f}]")
# Test if a cell type is enriched in a specific cluster# total_cells: total number of cells# cluster_cells: number of cells in cluster# celltype_total: total cells of this type# celltype_in_cluster: cells of this type in clusterfrom scipy.stats import fisher_exact
# Create contingency table
contingency = [
[celltype_in_cluster, cluster_cells - celltype_in_cluster], # In cluster
[celltype_total - celltype_in_cluster, total_cells - cluster_cells - (celltype_total - celltype_in_cluster)] # Not in cluster
]
odds_ratio, p_value = fisher_exact(contingency, alternative='greater')
print(f"Odds ratio: {odds_ratio:.4f}")
print(f"p-value: {p_value:.4e}")
if p_value < 0.05and odds_ratio > 1:
print(f"✅ Cell type is significantly ENRICHED in cluster (p < 0.05)")
elif p_value < 0.05and odds_ratio < 1:
print(f"⚠️ Cell type is significantly DEPLETED in cluster (p < 0.05)")
else:
print("❌ No significant enrichment/depletion")
Batch Effect Detection
# Test if there's a batch effect using ANOVA# gene_expression: DataFrame with genes as rows, samples as columns# batch_labels: array indicating batch for each sample
batch_effect_results = []
for gene in gene_expression.index:
# Get expression values for each batch
batches = [
gene_expression.loc[gene, batch_labels == batch]
for batch in np.unique(batch_labels)
]
# ANOVA test
f_stat, p_val = f_oneway(*batches)
batch_effect_results.append({
'gene': gene,
'f_statistic': f_stat,
'p_value': p_val
})
batch_df = pd.DataFrame(batch_effect_results)
# Apply FDR correction
_, batch_df['q_value'], _, _ = multipletests(batch_df['p_value'], alpha=0.05, method='fdr_bh')
# Count genes with batch effects
genes_with_batch_effect = (batch_df['q_value'] < 0.05).sum()
print(f"Genes with significant batch effect: {genes_with_batch_effect} ({genes_with_batch_effect/len(batch_df)*100:.1f}%)")
if genes_with_batch_effect > len(batch_df) * 0.1:
print("⚠️ WARNING: Strong batch effect detected (>10% genes affected)")
print("→ Recommendation: Apply batch correction (ComBat, Harmony, etc.)")
else:
print("✅ Minimal batch effect")
Best Practices
Check Assumptions: Always test normality before using parametric tests (t-test, ANOVA)
Multiple Testing: Apply FDR or Bonferroni correction when testing many hypotheses
Issue: "Division by zero in effect size calculation"
Solution: Check for zero variance (all values identical)
if np.std(group1) == 0or np.std(group2) == 0:
print("Cannot calculate effect size: zero variance in one or both groups")
else:
d = cohens_d(group1, group2)
Issue: "Test fails with NaN values"
Solution: Remove or impute NaN values before testing
# Remove NaN
group1_clean = group1[~np.isnan(group1)]
group2_clean = group2[~np.isnan(group2)]
# Or filter in DataFrame
df_clean = df.dropna(subset=['column_name'])
Issue: "Insufficient sample size warning"
Solution: Minimum sample sizes for reliable tests:
t-test: n ≥ 30 per group (or ≥ 5 if normally distributed)
ANOVA: n ≥ 20 per group
Correlation: n ≥ 30 total
iflen(group1) < 30orlen(group2) < 30:
print("⚠️ Warning: Small sample size. Results may not be reliable.")
print("Consider using non-parametric tests or collecting more data.")
Technical Notes
Libraries: Uses scipy.stats and statsmodels (widely supported, stable)
Execution: Runs locally in the agent's sandbox
Compatibility: Works with ALL LLM providers (GPT, Gemini, Claude, DeepSeek, Qwen, etc.)
Performance: Most tests complete in milliseconds; large-scale testing (>10K genes) takes 1-5 seconds
Precision: Uses double-precision floating point (numpy default)
Corrections: FDR (Benjamini-Hochberg) recommended for genomics; Bonferroni for small numbers of tests
