From d0b70b6c8a1ff536a2f0f73874fc06119df84ff6 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Wed, 15 Apr 2026 18:44:18 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20OOS=20Performance=20Enhancement=20?= =?UTF-8?q?=E2=80=94=208=20components=20for=20closing=20val/OOS=20gap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. CVaR objective (risk-sensitive Bellman target, adaptive α) 2. Adaptive DSR (training Sharpe EMA drives w_dsr weight) 3. Ensemble heads (3 heads, epistemic uncertainty for exploration) 4. Mamba2 temporal scan (K=8 bar context, O(K) selective scan) 5. Multi-horizon prediction (1/5/20 bar value heads, regime-blended) 6. Adaptive atom positions (learned C51 spacing via softmax — NOVEL) 7. Regime-conditioned branch gating (ADX/CUSUM → branch importance — NOVEL) Designed from live H100 analysis: val_Sharpe=45 but training_Sharpe=-1..+1. Co-Authored-By: Claude Opus 4.6 (1M context) --- ...4-15-oos-performance-enhancement-design.md | 245 ++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-15-oos-performance-enhancement-design.md diff --git a/docs/superpowers/specs/2026-04-15-oos-performance-enhancement-design.md b/docs/superpowers/specs/2026-04-15-oos-performance-enhancement-design.md new file mode 100644 index 000000000..378c1616c --- /dev/null +++ b/docs/superpowers/specs/2026-04-15-oos-performance-enhancement-design.md @@ -0,0 +1,245 @@ +# OOS Performance Enhancement — Design Spec + +**Goal:** Close the val_Sharpe/OOS gap (val=45 but training Sharpe oscillates -1 to +1) by adding risk-sensitive objectives, temporal context, ensemble diversity, multi-horizon prediction, adaptive reward shaping, and two novel techniques (adaptive atom positions + regime-conditioned branch gating). + +**Problem:** The model achieves val_Sharpe=45+ on a fixed validation window but training Sharpe (OOS proxy) oscillates -1.2 to +0.9. The model finds good strategies but can't consistently profit on fresh data. Root causes: (1) optimizes E[Q] not worst-case, (2) no temporal context — sees one bar at a time, (3) single Q-estimate — can't distinguish epistemic from aleatoric uncertainty, (4) fixed reward shaping — can't adapt to market regime, (5) fixed atom spacing — wastes resolution on unlikely returns. + +**Architecture:** 8 independent components composing through the existing pipeline. + +--- + +## Component 1: CVaR Objective for Bellman Target + +### Problem +Expected SARSA target optimizes E[Q] — tolerates high-variance strategies if the mean is good. Training Sharpe oscillates because the model takes risky bets. + +### Solution +Replace E[Q] with CVaR_α[Q] in the Bellman target action sampling: + +``` +CVaR_α[Q_a] = mean of atoms in the bottom α-percentile of action a's distribution +target_action ~ softmax(CVaR_α[Q_target] / tau) +``` + +α is adaptive: starts at 0.5 (balanced), decreases toward 0.1 (risk-averse) as iqn_readiness increases. + +### Implementation +- Modify `c51_loss_kernel.cu`: the Expected SARSA target sampling computes softmax weights from Q-values. Change to compute CVaR from the CDF (same infrastructure as quantile_q_select) and use CVaR for the softmax weights. +- `α = 0.5 - 0.4 * iqn_readiness` — adaptive schedule. +- Zero extra parameters. + +--- + +## Component 2: Adaptive DSR Reward Shaping + +### Problem +`w_dsr = 1.0` (fixed). Can't adapt to market regime — trending markets need low DSR, ranging needs high. + +### Solution +``` +sharpe_ema = EMA(training_sharpe, adaptive_alpha) +w_dsr_adaptive = w_dsr_base * clamp(1.0 - sharpe_ema, 0.5, 3.0) +``` + +Negative Sharpe → high DSR (force risk reduction). Positive Sharpe → low DSR (exploit). + +### Implementation +- `training_sharpe_ema: f32` field, updated at epoch end from experience collection Sharpe. +- `w_dsr_adaptive` written to pinned device-mapped scalar, read by env_step kernel. +- Zero kernel changes — replaces the fixed `w_dsr` config value. + +--- + +## Component 3: Ensemble Heads (3 heads) + +### Problem +`ensemble_heads=1`. Single Q-estimate. Uncertainty from C51 is aleatoric (market randomness), not epistemic (model ignorance). Can't distinguish "I don't know" from "the market is random." + +### Solution +Increase to `ensemble_heads=3`. Each shares the trunk but has independent branch weights. + +``` +Q_mean = mean(Q_1, Q_2, Q_3) +epistemic_var = var(Q_1, Q_2, Q_3) // model disagreement +exploration_bonus = sqrt(epistemic_var) +Q_select = Q_mean + exploration_bonus // OFU on epistemic uncertainty +``` + +### Implementation +- Config change: `ensemble_heads: 3`. Infrastructure already exists (`ensemble_kernels.cu`). +- ~2× branch parameter increase (~174K extra params). +- Ensemble variance replaces quantile Q-select for exploration (more principled). + +--- + +## Component 4: Mamba2 Selective Scan for Temporal Context + +### Problem +Model processes one bar at a time. No memory of recent market behavior within an episode. + +### Solution +Rolling buffer of last K=8 trunk activations. Mamba2 selective scan compresses into temporal context: + +``` +A_t = sigmoid(W_A @ h_t) // forget gate (input-dependent) +B_t = W_B @ h_t // input projection +x_t = A_t * x_{t-1} + B_t * h_t // recurrent scan (O(K) time) +temporal_context = (W_C @ h_t) * x_t + +h_s2_enriched = h_s2 + temporal_context // residual +``` + +### Implementation +- `h_history[B, K, SH2]` rolling buffer (~128MB for K=8, acceptable on H100). +- State_dim=16. Parameters: 3 × 256 × 16 = 12,288. +- New CUDA kernel `mamba2_temporal_scan`. Runs outside CUDA graph. +- From `crates/ml-supervised/src/mamba/scan_algorithms.rs`. + +--- + +## Component 5: Multi-Horizon Prediction + +### Problem +Single-step return prediction. Urgency branch has no temporal signal — can't distinguish scalping from swing opportunities. + +### Solution +3 value heads (shared trunk, shared branches): 1-bar, 5-bar, 20-bar horizons. + +``` +Q_1bar = compute_expected_q(v_logits_1, adv_logits) +Q_5bar = compute_expected_q(v_logits_5, adv_logits) +Q_20bar = compute_expected_q(v_logits_20, adv_logits) + +// Regime-weighted blend: +trend_weight = sigmoid((ADX - 25) / 5) +Q_blended = (1-trend_weight) * Q_1bar + trend_weight * Q_20bar +``` + +### Implementation +- 2 extra value heads: W_v1_5[VH, SH2], b_v1_5, W_v2_5[NA, VH], b_v2_5 (× 2 horizons). +- ~144K extra params. +- 3 passes of n-step kernel with step counts {1, 5, 20}. +- Urgency branch learns: Q_1bar >> Q_20bar → act now. Q_20bar >> Q_1bar → wait. + +--- + +## Component 6: Adaptive Atom Positions (Novel) + +### Problem +C51 atoms at fixed uniform positions z[j] = v_min + j * delta_z. 90% of atoms may cover returns that never happen, while the critical region near zero has too few atoms. + +### Solution +Learnable atom positions via softmax-normalized spacing: + +``` +spacing_raw[51] // learnable parameter vector +spacing = softmax(spacing_raw) // sum-to-1 normalized +positions[j] = v_min + (v_max - v_min) * cumsum(spacing)[j] +``` + +The model concentrates atoms where returns have mass. Near-zero returns get higher resolution. + +### Implementation +- 51 floats per branch × 4 branches = 204 extra params (in params_buf). +- New CUDA kernel `adaptive_atom_positions`: computes positions from spacing_raw. +- Replaces fixed `v_min + j * delta_z` in compute_expected_q, c51_loss, c51_grad. +- Gradient flows through atom positions via the C51 loss. + +--- + +## Component 7: Regime-Conditioned Branch Gating (Novel) + +### Problem +All 4 branches contribute equally to action selection. But branch importance is regime-dependent: direction matters in trends, magnitude matters in ranges. + +### Solution +Learned regime gate produces per-branch importance weights: + +``` +regime_input = [ADX, CUSUM, Q_gap, atom_utilization] // 4 features +branch_importance[4] = softmax(W_regime @ regime_input + b_regime) + +// Weight Q-values by branch importance: +Q_select[a] = Σ_d branch_importance[d] * Q_d[a_d] +``` + +### Implementation +- `W_regime[4, 4] + b_regime[4]` = 20 params. +- New CUDA kernel `regime_branch_gate`: computes softmax importance, scales Q-values. +- Runs after Q-mean centering, before Q-attention. +- Features already available in the state vector (ADX=40, CUSUM=41) and Q-stats (Q-gap, utilization). + +--- + +## Full Pipeline (after all 8 components) + +``` +── Trunk ── +h_s2 = trunk_forward(states) +h_s2 = attention(h_s2) +h_s2_enriched = h_s2 + mamba2_temporal_scan(h_history) [Component 4] + +── Branches (3 ensemble heads × 4 branches) ── +For each head e, branch d: + h_bd_e = KAN_gate(VSN(h_s2_enriched)) @ W_bdf_e_d + +── Value Heads (3 horizons) ── +v_logits_1/5/20 = value_head_1/5/20(h_s2_enriched) [Component 5] + +── Q-Values ── +Q_raw = compute_expected_q(v_logits_blended, adv_logits) [Component 5] +Q_raw = adaptive_atom_positions(Q_raw) [Component 6] +Q_centered = Q_raw - mean(Q_raw) [existing] +Q_regime = regime_branch_gate(Q_centered, ADX, CUSUM) [Component 7] +Q_coord = cross_branch_attention(Q_regime) [existing] +Q_graph = graph_message_pass(Q_coord) [existing] + +── Action Selection ── +Q_ensemble_mean = mean(Q across 3 heads) [Component 3] +exploration = sqrt(var(Q across 3 heads)) [Component 3] +Q_select = Q_ensemble_mean + exploration + +── Training ── +CVaR target: softmax(CVaR_α[Q_target] / tau) [Component 1] +Adaptive DSR: w_dsr *= clamp(1 - sharpe_ema, 0.5, 3.0) [Component 2] +``` + +--- + +## Implementation Order + +1. Component 2 (Adaptive DSR) — zero kernel changes, CPU-side, immediate OOS impact +2. Component 1 (CVaR objective) — modify existing c51_loss kernel, zero params +3. Component 7 (Regime branch gating) — 20 params, one kernel +4. Component 6 (Adaptive atom positions) — 204 params, modifies existing kernels +5. Component 3 (Ensemble heads) — config change, existing infrastructure +6. Component 4 (Mamba2 temporal scan) — new trunk layer, 12K params +7. Component 5 (Multi-horizon prediction) — 2 new value heads, 144K params + +--- + +## Success Criteria + +| Metric | Current (train-w2z55 ep 21) | Target | Source | +|--------|---------------------------|--------|--------| +| Training Sharpe | -1.2 to +0.9 oscillating | >0.5 sustained | Components 1, 2 | +| val_Sharpe | 45.81 (climbing) | >50 sustained | Components 4, 5, 6 | +| val/OOS gap | val=45, training=0 | val≈training | Components 1, 2, 3 | +| Exploration quality | Aleatoric-based (C51 variance) | Epistemic-based (ensemble disagreement) | Component 3 | +| Temporal awareness | None (single bar) | 8-bar Mamba2 context | Component 4 | +| Branch importance | Uniform (25% each) | Regime-adaptive | Component 7 | +| Atom resolution | Uniform (wasted on unlikely returns) | Concentrated where returns have mass | Component 6 | + +--- + +## Risks and Mitigations + +| Risk | Mitigation | +|------|-----------| +| CVaR too conservative → stops trading | α starts at 0.5 (balanced), not 0.1 (extreme) | +| Adaptive DSR oscillates | EMA smoothing, clamped to [0.5, 3.0] | +| 3 ensemble heads → 2× params → overfitting | Shared trunk, only branches triplicated | +| Mamba2 history buffer 128MB | K=4 for 64MB if needed; H100 has 80GB | +| Multi-horizon 3× C51 loss compute | Value heads are small (VH=256); branch GEMMs shared | +| Adaptive atoms destabilize C51 loss | Softmax spacing → always valid positions; small learning rate for spacing params | +| Regime gate collapses to uniform | Softmax with temperature; initialized to uniform |