# Dual Distributional Architecture: C51 + IQN — Design Spec

## Problem

The system has both C51 (101 atoms) and IQN (32 quantiles) implemented, but:
1. IQN trains in isolation — its gradients don't flow back to the shared trunk
2. IQN's quantile output is never used for action selection or risk management
3. `use_cvar_action_selection` exists in config but is hardcoded to `false`
4. The trunk learns only from C51's cross-entropy loss, missing the risk signal from IQN

## Solution

Wire the existing IQN dual-head for two purposes:
1. **Risk-sensitive position sizing** via CVaR at 5th percentile
2. **Uncertainty gating** via C51/IQN expected-Q disagreement

### Architecture

```
                    Shared Trunk (256→256)
                         │
                    save_h_s2 [B, 256]
                         │
              ┌──────────┴──────────┐
              │                     │
         C51 Head              IQN Head
        (101 atoms)          (32 quantiles)
              │                     │
     Expected Q = Σ(p·z)    Q(s,a,τ) per τ
    → argmax → action a*    → CVaR₅% = mean(Q|τ<0.05)
              │              → E_iqn = mean(Q|all τ)
              │                     │
              └────────┬────────────┘
                       │
              Risk-Gated Dispatch:
              1. C51 picks direction (a*)
              2. IQN sizes position: target_pos *= cvar_scale
              3. Uncertainty = |E_c51 - E_iqn| → reduce if high
```

### Why This Works

- **C51 excels at expected Q** — fixed support, cross-entropy loss, dense gradients, CUDA Graph compatible
- **IQN excels at tail risk** — exact CVaR/VaR without interpolation, no v_min/v_max needed
- **Disagreement = uncertainty** — when C51 and IQN disagree on expected Q, the model is uncertain about this state

### Literature Support

- Dabney et al. 2018 (IQN): IQN subsumes both C51 and QR-DQN, outperforms both on Atari
- arXiv:2501.04421 (2025): C51 and IQN produce the most controllable risk-averse policies for futures trading
- arXiv:2305.16877 (2023): Dual distributional objectives prevent distributional collapse
- arXiv:2306.07124 (2023): Combining different distributional representations yields better uncertainty

### What Changes

| Component | Today | After | Effort |
|-----------|-------|-------|--------|
| C51 training | Working | No change | 0 |
| IQN training | Working, disconnected | Gradient flows to trunk | LOW |
| IQN inference | Kernel compiled, never called | Wired for CVaR | MEDIUM |
| CVaR computation | Not implemented | New trivial CUDA kernel | LOW |
| C51/IQN disagreement | Not implemented | |E_c51 - E_iqn| | LOW |
| Position sizing | Fixed by C51 action | CVaR-modulated | MEDIUM |
| Hyperopt | iqn_lambda searched | Add cvar_alpha | LOW |

### VRAM: +3.3 MB (<1.2%). Compute: <5% training, ~2.5x inference (not bottleneck).

## Implementation Order

1. Wire IQN gradient to shared trunk (Phase 1)
2. Wire IQN inference + CVaR computation (Phase 2)
3. Add disagreement uncertainty signal (Phase 3)
4. Hyperopt integration for cvar_alpha (Phase 4)
5. **THEN** expand exposure 5→9 (separate spec)

IQN wiring before 9-action expansion because CVaR gating becomes essential with finer exposure granularity.
