backtest-capital-accounting

name	backtest-capital-accounting
description	Critical fix for backtest capital accounting when equity curves are inflated or drawdown metrics are meaningless
author	Claude Code
date	"2025-12-16T00:00:00.000Z"

Backtest Capital Accounting - Research Notes

Experiment Overview

Item	Details
Date	2025-12-16
Goal	Fix inflated equity curves and meaningless drawdown in backtests
Environment	Python 3.10, pandas, custom BacktestEngine
Status	Success

Context

Backtest engines often fail to properly track cash flows, leading to:

• Capital not deducted when positions open (inflated buying power)
• Sale proceeds not properly credited on close
• Equity calculation ignoring open position market values
• Meaningless max_drawdown metrics that can't trigger guardrails

The bug is subtle because the backtest "runs" without errors but produces unrealistic results.

Verified Workflow

1. Correct _open_position() Implementation

def _open_position(self, symbol, side, price, time, bar_idx, confidence):
    """Open position with proper capital accounting."""
    # Apply slippage first
    slippage = price * self.config.slippage_pct
    entry_price = price + (slippage if side == 1 else -slippage)

    # Calculate position size from AVAILABLE capital
    position_value = self.capital * self.config.position_size_pct
    shares = position_value / entry_price

    # CRITICAL: Deduct full position value + commission from capital
    total_cost = (shares * entry_price) + self.config.commission_per_trade
    self.capital -= total_cost  # <-- This line is often missing!

    # Store position
    self.positions[symbol] = Position(...)

2. Correct _close_position() Implementation

def _close_position(self, symbol, price, time, bar_idx, reason):
    """Close position with proper capital accounting."""
    position = self.positions[symbol]

    # Apply slippage
    slippage = price * self.config.slippage_pct
    exit_price = price - (slippage if position.side == 1 else -slippage)

    commission = self.config.commission_per_trade

    # CRITICAL: Add sale proceeds (exit value - commission) to capital
    if position.side == 1:  # Long
        sale_proceeds = position.shares * exit_price - commission
    else:  # Short
        # Short P&L: entry - exit (profit when price falls)
        sale_proceeds = (2 * position.shares * position.entry_price) - \
                       (position.shares * exit_price) - commission

    self.capital += sale_proceeds  # <-- This line is often wrong!

    del self.positions[symbol]

3. Correct _calculate_equity() Implementation

def _calculate_equity(self, current_price: float) -> float:
    """Calculate total equity = cash + market value of positions."""
    equity = self.capital  # Cash on hand

    for position in self.positions.values():
        if position.side == 1:  # Long
            market_value = position.shares * current_price
        else:  # Short
            # Short market value: 2*entry - current (profit when price falls)
            market_value = (2 * position.shares * position.entry_price) - \
                          (position.shares * current_price)
        equity += market_value

    return equity

Failed Attempts (Critical)

Attempt	Why it Failed	Lesson Learned
Only tracking P&L on close	Capital shows full value even with open positions	Must deduct on entry, add on exit
Using capital = initial + sum(pnl)	Ignores unrealized gains/losses	Equity must include position market value
Not applying slippage to entry/exit	Overstated returns	Apply slippage before calculating shares/proceeds
Adding position value to capital on close	Double-counts - position value already in equity	Add sale PROCEEDS, not position value
Short position P&L = (entry - exit) * shares	Wrong sign when price rises	Use 2*entry - exit formula for short market value

Final Parameters

# BacktestConfig with proper defaults
@dataclass
class BacktestConfig:
    initial_capital: float = 100_000.0
    position_size_pct: float = 0.10      # 10% per position
    commission_per_trade: float = 1.0    # $1 per trade
    slippage_pct: float = 0.0005         # 0.05% slippage
    stop_loss_pct: float = 0.02          # 2% stop
    take_profit_pct: float = 0.04        # 4% TP

Key Insights

• The capital accounting bug is silent - backtests run but produce garbage metrics
• Equity must equal cash + market value - at all times, not just on close
• Short positions need special handling - profit when price falls, loss when rises
• Commission affects both entry AND exit - double the impact of what you might expect
• Slippage compounds with position size - large positions have more slippage impact
• Test with round trips - open then close same position, capital should reflect P&L exactly

Validation Check

After fixing, verify with this test:

# After a round trip (buy then sell same shares):
# capital_after = capital_before + realized_pnl - 2*commission - slippage_cost
# If capital_after > capital_before + realistic_pnl, accounting is broken

References

• CLAUDE.md guardrails: "Realistic backtests: Do not alter cash-flow logic without full audit"
• Standard brokerage margin accounting rules
• Walk-forward validation best practices