feat: v9 Differential Sharpe Ratio reward — optimize for consistency, not PnL
Fundamental shift: reward optimizes Sharpe ratio contribution per trade
instead of directional PnL magnitude. Targets Sharpe 5+ via many small
consistent gains (spread capture, execution quality) instead of few
large directional bets.
DSR implementation (Moody & Saffell, 2001):
- Per-episode EMA statistics in portfolio state slots [3]-[5]
- DSR = (B*R - 0.5*A*R²) / (B - A²)^1.5 at trade completion
- 10-trade warmup, clamped [-5, +5]
- w_dsr=5.0 (primary reward signal)
Reward hierarchy restructured:
- DSR: 5.0 (NEW — primary, rewards Sharpe consistency)
- Directional PnL: 2.0 (was 10.0 — demoted to secondary)
- Order credit: 1.0 (was 0.1 — spread capture amplified 10×)
- Urgency credit: 0.5 (was 0.1 — fill quality amplified 5×)
- Inventory penalty: -0.005×|pos|/max_pos per bar (NEW)
- Dense micro-reward: 0.0 (removed — noisy direction signal)
Expected: WinRate increases (many small spread captures), per-trade
variance decreases, Sharpe rises from consistency not prediction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1f6a8436d3