lookahead-bias-checklist

name	lookahead-bias-checklist
description	Financial research look-ahead bias verification for Volatio ML pipelines, mandatory baseline comparisons, timezone handling, feature leakage detection. Use when model accuracy seems too good or validating prediction experiments.

lookahead-bias-checklist

When to Use

• Model accuracy > 51% on financial predictions
• Validating any prediction experiment
• Reviewing research pipeline outputs
• Checking for feature leakage

Golden Rule

Any accuracy above 51% on financial prediction should trigger IMMEDIATE skepticism. Markets are efficient; consistent edge is extremely rare.

Mandatory Verification Checklist

1. Verify Target Variable Timing

# Print actual dates to verify alignment
print(df[['trade_date', 'features_date', 'target_date']].head(10))
# Features from day T should predict target from day T+1

2. Compare to Trivial Baselines

# ALWAYS compute these
baseline_always_up = y_test.mean()  # SPY is up ~53% of days
baseline_always_down = 1 - y_test.mean()
print(f"Model: {accuracy:.1%}, Baseline (always up): {baseline_always_up:.1%}")

# If model accuracy ≈ baseline, there's NO signal

3. Check Prediction Distribution

print(f"Predictions: {pd.Series(preds).value_counts().to_dict()}")
# If model predicts all 1s or all 0s, it learned majority class, not a signal

4. Verify Rolling Features Don't Leak

# ❌ BAD: Rolling computed on full dataset before split
df['ma_10'] = df['price'].rolling(10).mean()
X_train, X_test = df[:split], df[split:]

# ✅ GOOD: Rolling computed only on available data
# (pandas rolling default is backward-looking, but verify!)

Required Baseline Report

Every experiment MUST report:

results = {
    "model_accuracy": accuracy,
    "baseline_always_up": y_test.mean(),
    "baseline_always_down": 1 - y_test.mean(),
    "baseline_random": 0.5,
    "lift_over_baseline": accuracy - max(y_test.mean(), 1 - y_test.mean()),
    "prediction_distribution": pd.Series(preds).value_counts().to_dict(),
}

# FAIL if model just predicts majority class
if len(set(preds)) == 1:
    print("⚠️ WARNING: Model predicts single class - NO SIGNAL")

Common Sources of Look-Ahead Bias

Source	Example	Fix
Wrong timestamp column	Using trade timestamp instead of data-available	Verify with actual datetime values
Timezone bugs	AT TIME ZONE returning wrong hours	Use explicit UTC hour ranges
Shift direction	shift(-1) vs shift(1) confusion	Print and verify manually
Rolling on full data	Z-scores before train/test split	Compute only on training data
Future joins	Joining on date without availability check	Add explicit date filters
Entry at CLOSE	Using same-day close instead of next-day open	Entry at T+1 OPEN
Exit signal today	Using current-day signal for exit decision	Use PRIOR DAY signal
Exit at CLOSE	Using same-day close for exit	Exit at OPEN price

Market Data Timezone Handling

PostgreSQL AT TIME ZONE is confusing:

-- ❌ WRONG: AT TIME ZONE can give unexpected results
SELECT * FROM minute_bars
WHERE EXTRACT(HOUR FROM time AT TIME ZONE 'America/New_York') = 15

-- ✅ CORRECT: Use explicit UTC hour ranges
-- Market close 4pm ET = 21:00 UTC (winter) or 20:00 UTC (summer)
SELECT DISTINCT ON (time::date)
    time::date AS trade_date,
    close
FROM minute_bars
WHERE symbol = 'SPY'
  AND EXTRACT(HOUR FROM time) BETWEEN 20 AND 21
ORDER BY time::date, time DESC

When to Be Extra Suspicious

• Accuracy > 55% on liquid assets (SPY, QQQ)
• Accuracy > 60% on any next-day prediction
• Small test set (< 100 samples)
• Rolling/lagged features in feature set
• Model trained on post-2020 data (regime change)

Real-World Expectations

For legitimate trading signals on liquid markets:

• 51-53% accuracy is potentially tradeable with good risk management
• 54-56% accuracy is excellent (hedge fund level)
• >60% accuracy almost certainly has a bug

Don'ts

• Don't celebrate high accuracy without verification
• Don't skip baseline comparisons
• Don't trust small test sets
• Don't use AT TIME ZONE for market data without explicit verification