drl-pair-trading-crypto

name	drl-pair-trading-crypto
description	Deep Reinforcement Learning for dynamic multi-pair trading in cryptocurrency markets. Filter-then-Rank pair selection with PPO+LSTM execution agent within deterministic risk shielding. Evaluated on Binance USD-M Futures with bootstrap robustness validation. Introduces Fixed Risk Adaptive Mean execution model for safe RL in high-variance digital asset environments. (arXiv: 2606.04574)
tags	["economics","trading","deep-learning","reinforcement-learning","cryptocurrency","pair-trading"]
category	economics

Context

This methodology comes from arXiv:2606.04574 "Dynamic Multi-Pair Trading Strategy in Cryptocurrency Markets with Deep Reinforcement Learning" by Damian Lebiedź and Robert Ślepaczuk (June 2026, 61 pages). Addresses the challenge of applying statistical arbitrage (pair trading) to high-variance cryptocurrency markets using Deep Reinforcement Learning as an execution overlay.

Core Problem

Classical pair trading works well in traditional equities but fails in crypto due to:

• Extreme volatility: High-variance environments cause frequent divergence
• Rigid execution rules: Traditional cointegration-based entry/exit rules cannot adapt to regime changes
• Severe divergence risks: Crypto pairs break down more frequently and with larger drawdowns

Core Methodology

Architecture: Hybrid Statistical Arbitrage + DRL

Filter-then-Rank Pair Selection → PPO+LSTM Execution Agent → Deterministic Risk Shielding

Step 1: Filter-then-Rank Pair Selection

Filter Phase (Statistical Criteria):

1. Compute rolling cointegration tests (Engle-Granger or Johansen)
2. Filter pairs with stable cointegration over lookback window
3. Apply spread stationarity tests (ADF test on spread)
4. Filter out pairs with structural breaks in recent window

Rank Phase (Quality Scoring):

1. Score filtered pairs by:
   • Half-life of mean reversion (shorter = better)
   • Spread volatility (lower = better)
   • Cointegration coefficient stability
   • Recent correlation strength
2. Select top-k pairs for trading

Step 2: Fixed Risk, Adaptive Mean (FRAM) Execution Model

Fixed Risk Component:

• Set deterministic risk bounds BEFORE RL agent acts:
  • Maximum position size per pair
  • Maximum portfolio drawdown limit
  • Maximum spread deviation before forced exit
  • Stop-loss levels based on historical spread distribution

Adaptive Mean Component:

• RL agent adapts the mean-reversion target dynamically:
  • Learns when spread "mean" has shifted (regime change)
  • Adjusts entry/exit thresholds based on current market state
  • Overrides rigid z-score rules when market conditions warrant

Step 3: PPO + LSTM Agent Design

State Space:

• Spread z-score (current deviation from mean)
• Spread half-life (estimated mean reversion speed)
• Volatility regime indicator (GARCH or rolling volatility)
• Market momentum indicators (price trends for both legs)
• Position state (current holdings per pair)
• Time features (intraday patterns, day-of-week effects)

Action Space:

• Discrete or continuous actions:
  • Open long spread / Open short spread / Close position / Hold
  • Position sizing (fraction of allowed risk budget)

LSTM Layer:

• Captures temporal dependencies in spread dynamics
• Remembers past regime transitions
• Learns persistence patterns in divergence

Reward Function:

• Risk-adjusted returns (Sharpe ratio or Sortino ratio)
• Penalize excessive trading (transaction costs)
• Penalize large drawdowns
• Bonus for maintaining market-neutral exposure

Step 4: Deterministic Risk Shielding

Critical safety layer that prevents RL agent from catastrophic decisions:

1. Pre-action filter: Before executing RL action, verify it satisfies deterministic constraints:

   • Position size ≤ maximum allowed
   • Portfolio exposure within limits
   • No action if spread exceeds critical divergence threshold

2. Override rules: If RL action violates constraints:

   • Replace with safest valid action (usually "hold" or "reduce position")
   • Log the override for analysis

3. Post-action monitoring: After execution:

   • Monitor spread for abnormal behavior
   • Auto-exit if spread breaks cointegration relationship
   • Emergency flatten if drawdown exceeds threshold

Step 5: Training Protocol

1. Data: 1-hour interval data from Binance USD-M Futures market
2. Train/Validation/Test split: Chronological split (no look-ahead bias)
3. Curriculum learning: Start with simple pairs, gradually add complexity
4. Ensemble training: Train multiple agents with different seeds
5. Select best agent: Based on out-of-sample Sharpe ratio

Implementation Steps

Step 1: Data Preparation

- Fetch 1-hour OHLCV data for top crypto pairs (BTC, ETH, SOL, etc.)
- Compute spread series for all candidate pairs
- Apply cointegration filter (rolling window)
- Create feature matrix with spread statistics, volatility, momentum

Step 2: Pair Selection Pipeline

def filter_then_rank(pairs_data, lookback=90):
    # Filter
    cointegrated = [p for p in pairs_data if is_cointegrated(p, lookback)]
    stationary = [p for p in cointegrated if spread_is_stationary(p)]
    
    # Rank
    scored = [(p, compute_quality_score(p)) for p in stationary]
    scored.sort(key=lambda x: x[1], reverse=True)
    return [p for p, _ in scored[:top_k]]

Step 3: PPO Agent Setup

- State: [z_score, half_life, vol_regime, momentum_1, momentum_2, position_state, time_features]
- Action: [direction (-1, 0, 1), size (0.0 to 1.0)]
- Network: LSTM(64) → Dense(128) → Dense(action_dim)
- PPO hyperparameters: clip=0.2, gamma=0.99, lr=3e-4, entropy_coef=0.01

Step 4: Risk Shield Implementation

def apply_risk_shield(rl_action, portfolio_state, spread_state):
    max_position = portfolio_state.max_position_size
    if rl_action.size > max_position:
        rl_action.size = max_position  # Clip to max
    
    if spread_state.z_score > critical_threshold:
        return Action.CLOSE_ALL  # Forced exit on extreme divergence
    
    if portfolio_state.drawdown > max_drawdown:
        return Action.REDUCE_RISK  # Risk reduction mode
    
    return rl_action  # Pass through if within bounds

Step 5: Validation

- Out-of-sample backtest on held-out period
- Stationary circular block bootstrap (1000 resamples)
- Check if RL outperformance is statistically significant
- Compare against heuristic baseline (traditional pair trading rules)

Key Results

• Out-of-sample performance: RL policy substantially outperformed heuristic baseline
• Bootstrap validation: Risk-adjusted outperformance significant at 10% level
• Marginal 5% significance: Falling short of 5% threshold highlights extreme idiosyncratic variance in crypto
• Safe RL proven: Deterministic shielding successfully mitigates severe divergence risks

Pitfalls

• No risk shielding = catastrophic losses: RL agents can learn to take extreme positions in high-variance environments. Always implement deterministic safety bounds.
• Cointegration breaks frequently in crypto: Rolling windows must be short enough to detect regime changes but long enough for statistical power.
• Transaction costs matter: Crypto futures have funding rates and trading fees that erode pair trading profits. Include in reward calculation.
• Overfitting to specific regime: Crypto markets shift regimes frequently. Use walk-forward validation, not random splits.
• Bootstrap at 10% not 5%: The marginal statistical significance at 10% (not 5%) is expected given crypto's extreme variance. Don't over-interpret; focus on consistent directional outperformance.
• LSTM state management: Ensure LSTM hidden states are reset appropriately at episode boundaries to prevent information leakage.

Verification

1. Cointegration filter: Verify ADF test p-value < 0.05 on spread series
2. Risk shielding: Simulate extreme spread scenarios → verify agent actions are clipped
3. Bootstrap test: Run 1000 block bootstrap resamples → compute p-value of Sharpe difference
4. Out-of-sample test: Evaluate on completely unseen data period → verify performance holds
5. Baseline comparison: Compare RL Sharpe ratio against traditional pair trading Sharpe ratio

Activation

pair trading, cryptocurrency trading, deep reinforcement learning, PPO, LSTM, statistical arbitrage, cointegration, risk management, Binance futures, algorithmic trading, safe reinforcement learning, deterministic shielding, Filter-then-Rank, FRAM, bootstrap validation, crypto volatility, mean reversion, spread trading