# OOS Performance Enhancement — Design Spec

**Goal:** Close the val_Sharpe/OOS gap (val=45 but training Sharpe oscillates -1 to +1) by adding risk-sensitive objectives, temporal context, ensemble diversity, multi-horizon prediction, adaptive reward shaping, and two novel techniques (adaptive atom positions + regime-conditioned branch gating).

**Problem:** The model achieves val_Sharpe=45+ on a fixed validation window but training Sharpe (OOS proxy) oscillates -1.2 to +0.9. The model finds good strategies but can't consistently profit on fresh data. Root causes: (1) optimizes E[Q] not worst-case, (2) no temporal context — sees one bar at a time, (3) single Q-estimate — can't distinguish epistemic from aleatoric uncertainty, (4) fixed reward shaping — can't adapt to market regime, (5) fixed atom spacing — wastes resolution on unlikely returns.

**Architecture:** 8 independent components composing through the existing pipeline.

---

## Component 1: CVaR Objective for Bellman Target

### Problem
Expected SARSA target optimizes E[Q] — tolerates high-variance strategies if the mean is good. Training Sharpe oscillates because the model takes risky bets.

### Solution
Replace E[Q] with CVaR_α[Q] in the Bellman target action sampling:

```
CVaR_α[Q_a] = mean of atoms in the bottom α-percentile of action a's distribution
target_action ~ softmax(CVaR_α[Q_target] / tau)
```

α is adaptive: starts at 0.5 (balanced), decreases toward 0.1 (risk-averse) as iqn_readiness increases.

### Implementation
- Modify `c51_loss_kernel.cu`: the Expected SARSA target sampling computes softmax weights from Q-values. Change to compute CVaR from the CDF (same infrastructure as quantile_q_select) and use CVaR for the softmax weights.
- `α = 0.5 - 0.4 * iqn_readiness` — adaptive schedule.
- Zero extra parameters.

---

## Component 2: Adaptive DSR Reward Shaping

### Problem
`w_dsr = 1.0` (fixed). Can't adapt to market regime — trending markets need low DSR, ranging needs high.

### Solution
```
sharpe_ema = EMA(training_sharpe, adaptive_alpha)
w_dsr_adaptive = w_dsr_base * clamp(1.0 - sharpe_ema, 0.5, 3.0)
```

Negative Sharpe → high DSR (force risk reduction). Positive Sharpe → low DSR (exploit).

### Implementation
- `training_sharpe_ema: f32` field, updated at epoch end from experience collection Sharpe.
- `w_dsr_adaptive` written to pinned device-mapped scalar, read by env_step kernel.
- Zero kernel changes — replaces the fixed `w_dsr` config value.

---

## Component 3: Ensemble Heads (3 heads)

### Problem
`ensemble_heads=1`. Single Q-estimate. Uncertainty from C51 is aleatoric (market randomness), not epistemic (model ignorance). Can't distinguish "I don't know" from "the market is random."

### Solution
Increase to `ensemble_heads=3`. Each shares the trunk but has independent branch weights.

```
Q_mean = mean(Q_1, Q_2, Q_3)
epistemic_var = var(Q_1, Q_2, Q_3)       // model disagreement
exploration_bonus = sqrt(epistemic_var)
Q_select = Q_mean + exploration_bonus     // OFU on epistemic uncertainty
```

### Implementation
- Config change: `ensemble_heads: 3`. Infrastructure already exists (`ensemble_kernels.cu`).
- ~2× branch parameter increase (~174K extra params).
- Ensemble variance replaces quantile Q-select for exploration (more principled).

---

## Component 4: Mamba2 Selective Scan for Temporal Context

### Problem
Model processes one bar at a time. No memory of recent market behavior within an episode.

### Solution
Rolling buffer of last K=8 trunk activations. Mamba2 selective scan compresses into temporal context:

```
A_t = sigmoid(W_A @ h_t)           // forget gate (input-dependent)
B_t = W_B @ h_t                    // input projection
x_t = A_t * x_{t-1} + B_t * h_t   // recurrent scan (O(K) time)
temporal_context = (W_C @ h_t) * x_t

h_s2_enriched = h_s2 + temporal_context  // residual
```

### Implementation
- `h_history[B, K, SH2]` rolling buffer (~128MB for K=8, acceptable on H100).
- State_dim=16. Parameters: 3 × 256 × 16 = 12,288.
- New CUDA kernel `mamba2_temporal_scan`. Runs outside CUDA graph.
- From `crates/ml-supervised/src/mamba/scan_algorithms.rs`.

---

## Component 5: Multi-Horizon Prediction

### Problem
Single-step return prediction. Urgency branch has no temporal signal — can't distinguish scalping from swing opportunities.

### Solution
3 value heads (shared trunk, shared branches): 1-bar, 5-bar, 20-bar horizons.

```
Q_1bar  = compute_expected_q(v_logits_1, adv_logits)
Q_5bar  = compute_expected_q(v_logits_5, adv_logits)
Q_20bar = compute_expected_q(v_logits_20, adv_logits)

// Regime-weighted blend:
trend_weight = sigmoid((ADX - 25) / 5)
Q_blended = (1-trend_weight) * Q_1bar + trend_weight * Q_20bar
```

### Implementation
- 2 extra value heads: W_v1_5[VH, SH2], b_v1_5, W_v2_5[NA, VH], b_v2_5 (× 2 horizons).
- ~144K extra params.
- 3 passes of n-step kernel with step counts {1, 5, 20}.
- Urgency branch learns: Q_1bar >> Q_20bar → act now. Q_20bar >> Q_1bar → wait.

---

## Component 6: Adaptive Atom Positions (Novel)

### Problem
C51 atoms at fixed uniform positions z[j] = v_min + j * delta_z. 90% of atoms may cover returns that never happen, while the critical region near zero has too few atoms.

### Solution
Learnable atom positions via softmax-normalized spacing:

```
spacing_raw[51]  // learnable parameter vector
spacing = softmax(spacing_raw)  // sum-to-1 normalized
positions[j] = v_min + (v_max - v_min) * cumsum(spacing)[j]
```

The model concentrates atoms where returns have mass. Near-zero returns get higher resolution.

### Implementation
- 51 floats per branch × 4 branches = 204 extra params (in params_buf).
- New CUDA kernel `adaptive_atom_positions`: computes positions from spacing_raw.
- Replaces fixed `v_min + j * delta_z` in compute_expected_q, c51_loss, c51_grad.
- Gradient flows through atom positions via the C51 loss.

---

## Component 7: Regime-Conditioned Branch Gating (Novel)

### Problem
All 4 branches contribute equally to action selection. But branch importance is regime-dependent: direction matters in trends, magnitude matters in ranges.

### Solution
Learned regime gate produces per-branch importance weights:

```
regime_input = [ADX, CUSUM, Q_gap, atom_utilization]  // 4 features
branch_importance[4] = softmax(W_regime @ regime_input + b_regime)

// Weight Q-values by branch importance:
Q_select[a] = Σ_d branch_importance[d] * Q_d[a_d]
```

### Implementation
- `W_regime[4, 4] + b_regime[4]` = 20 params.
- New CUDA kernel `regime_branch_gate`: computes softmax importance, scales Q-values.
- Runs after Q-mean centering, before Q-attention.
- Features already available in the state vector (ADX=40, CUSUM=41) and Q-stats (Q-gap, utilization).

---

## Full Pipeline (after all 8 components)

```
── Trunk ──
h_s2 = trunk_forward(states)
h_s2 = attention(h_s2)
h_s2_enriched = h_s2 + mamba2_temporal_scan(h_history)     [Component 4]

── Branches (3 ensemble heads × 4 branches) ──
For each head e, branch d:
  h_bd_e = KAN_gate(VSN(h_s2_enriched)) @ W_bdf_e_d

── Value Heads (3 horizons) ──
v_logits_1/5/20 = value_head_1/5/20(h_s2_enriched)        [Component 5]

── Q-Values ──
Q_raw = compute_expected_q(v_logits_blended, adv_logits)    [Component 5]
Q_raw = adaptive_atom_positions(Q_raw)                      [Component 6]
Q_centered = Q_raw - mean(Q_raw)                            [existing]
Q_regime = regime_branch_gate(Q_centered, ADX, CUSUM)       [Component 7]
Q_coord = cross_branch_attention(Q_regime)                   [existing]
Q_graph = graph_message_pass(Q_coord)                        [existing]

── Action Selection ──
Q_ensemble_mean = mean(Q across 3 heads)                    [Component 3]
exploration = sqrt(var(Q across 3 heads))                   [Component 3]
Q_select = Q_ensemble_mean + exploration

── Training ──
CVaR target: softmax(CVaR_α[Q_target] / tau)               [Component 1]
Adaptive DSR: w_dsr *= clamp(1 - sharpe_ema, 0.5, 3.0)     [Component 2]
```

---

## Implementation Order

1. Component 2 (Adaptive DSR) — zero kernel changes, CPU-side, immediate OOS impact
2. Component 1 (CVaR objective) — modify existing c51_loss kernel, zero params
3. Component 7 (Regime branch gating) — 20 params, one kernel
4. Component 6 (Adaptive atom positions) — 204 params, modifies existing kernels
5. Component 3 (Ensemble heads) — config change, existing infrastructure
6. Component 4 (Mamba2 temporal scan) — new trunk layer, 12K params
7. Component 5 (Multi-horizon prediction) — 2 new value heads, 144K params

---

## Success Criteria

| Metric | Current (train-w2z55 ep 21) | Target | Source |
|--------|---------------------------|--------|--------|
| Training Sharpe | -1.2 to +0.9 oscillating | >0.5 sustained | Components 1, 2 |
| val_Sharpe | 45.81 (climbing) | >50 sustained | Components 4, 5, 6 |
| val/OOS gap | val=45, training=0 | val≈training | Components 1, 2, 3 |
| Exploration quality | Aleatoric-based (C51 variance) | Epistemic-based (ensemble disagreement) | Component 3 |
| Temporal awareness | None (single bar) | 8-bar Mamba2 context | Component 4 |
| Branch importance | Uniform (25% each) | Regime-adaptive | Component 7 |
| Atom resolution | Uniform (wasted on unlikely returns) | Concentrated where returns have mass | Component 6 |

---

## Risks and Mitigations

| Risk | Mitigation |
|------|-----------|
| CVaR too conservative → stops trading | α starts at 0.5 (balanced), not 0.1 (extreme) |
| Adaptive DSR oscillates | EMA smoothing, clamped to [0.5, 3.0] |
| 3 ensemble heads → 2× params → overfitting | Shared trunk, only branches triplicated |
| Mamba2 history buffer 128MB | K=4 for 64MB if needed; H100 has 80GB |
| Multi-horizon 3× C51 loss compute | Value heads are small (VH=256); branch GEMMs shared |
| Adaptive atoms destabilize C51 loss | Softmax spacing → always valid positions; small learning rate for spacing params |
| Regime gate collapses to uniform | Softmax with temperature; initialized to uniform |
