# Cross-Pollinated Branch Intelligence — Design Spec

**Goal:** Combine Variable Selection (TFT), Gated Linear Units (TFT/Llama), Cross-Branch Q Attention, Liquid adaptive tau, and Mamba selective replay into a unified branch architecture that selects, gates, conditions, coordinates, and adapts.

**Problem:** The 4-branch DQN treats each branch as an independent FC layer reading the same h_s2 trunk. Every branch sees every feature equally (no selection), uses all-or-nothing ReLU gating (no soft suppression), produces Q-values in isolation (no coordination), and learns at a uniform rate (no per-branch adaptation). The replay buffer prioritizes transitions by TD-error magnitude regardless of current model state (no curriculum).

**Architecture:** Five additive layers on top of the existing 4-branch DQN + Layer 3 direction conditioning:

1. **Variable Selection** — per-branch softmax mask on h_s2 features
2. **GLU Gating** — sigmoid gate replaces ReLU in branch FC
3. **Cross-Branch Q Attention** — 2-head attention on the 12 Q-values
4. **Liquid Tau** — per-branch adaptive learning rate via continuous-time ODE
5. **Mamba-2 Selective Replay** — learned state-conditioned priority gate (from Mamba2's SelectiveStateSpace/StateImportance pattern in `crates/ml-supervised/src/mamba/selective_state.rs`)

Plus dead code cleanup (11 items across 6 files).

---

## Component 1: Variable Selection (from TFT)

### Problem
All 4 branches read the same h_s2[B, SH2] with equal weight on all features. Direction should attend to trend features (MACD, RSI), magnitude to volatility (ATR), order to microstructure (OFI), urgency to regime (CUSUM, ADX). Currently the branch FC must learn feature relevance implicitly through its weight matrix — wasting capacity.

### Solution
Input-dependent variable selection (true TFT VSN): per-sample feature masks conditioned on the current state via a bottleneck projection. Unlike a static learned vector, the mask adapts to market regime — trending states emphasize momentum features, ranging states emphasize mean-reversion.

```
// Two-layer bottleneck per branch: SH2 → R → SH2 (R=16 bottleneck rank)
h_proj_d[B, R]   = h_s2[B, SH2] @ W_vsn1_d[SH2, R]       // compress
mask_d[B, SH2]   = softmax(h_proj_d[B, R] @ W_vsn2_d[R, SH2])  // per-sample mask
h_masked[B, SH2] = h_s2[B, SH2] * mask_d[B, SH2]          // element-wise gate
```

### Properties
- 4 × (SH2×R + R×SH2) = 4 × 2 × 256 × 16 = 32,768 extra parameters
- Two small GEMMs per branch: [SH2, R] then [R, SH2] (R=16 bottleneck)
- Input-dependent: mask changes per sample based on current market state
- Interpretable: softmax(mask_d) shows which features each branch attends to for each sample
- Gradient flows through both projection layers to h_s2 (additional trunk gradient)
- Bottleneck initialized to produce uniform masks (W_vsn2 near zero → softmax(0) = uniform)

### Implementation
- New CUDA kernel `variable_select_bottleneck` in experience_kernels.cu: two matmuls + softmax + element-wise multiply, one thread-block per sample
- 8 weight matrices stored in params_buf: W_vsn1_d[SH2, R] and W_vsn2_d[R, SH2] for d=0..3 (indices 26-33)
- Biases omitted (bottleneck projections don't need bias — the softmax normalizes)

---

## Component 2: Gated Linear Units (from TFT GRN / Llama)

### Problem
ReLU activation in branch FC is all-or-nothing: negative activations are zeroed, positive pass unchanged. This creates dead neurons (permanently zero for some input distributions) and can't suppress positive-but-irrelevant activations.

### Solution
Replace ReLU with GLU (Gated Linear Unit):

```
Current:  h_bd = ReLU(W_bdf @ input + b_bdf)
Proposed: gate   = sigmoid(W_gate_d @ input + b_gate_d)   [B, AH]
          value  = W_bdf_d @ input + b_bdf_d               [B, AH]
          h_bd   = gate * value                             [B, AH]
```

The sigmoid gate learns a smooth [0,1] suppression per activation. Unlike ReLU, it can suppress positive values ("this feature is active but irrelevant") and partially pass negative values ("this feature is slightly negative but informative").

### Properties
- Doubles the branch FC parameters: 4 extra weight matrices [AH, K_d] + 4 bias vectors [AH]
- Total: ~262,144 + 1,024 = 263,168 extra parameters
- One extra GEMM per branch (4 total, same shape as existing branch FC)
- Proven in TFT, Llama, PaLM — consistently outperforms ReLU in gated architectures
- No dead neuron problem — sigmoid gate is always differentiable

### Implementation
- Gate weights stored after variable selection bottleneck (indices 34-41: w_gate0, b_gate0, ..., w_gate3, b_gate3)
- Forward: two parallel GEMMs per branch (gate path + value path), then element-wise multiply
- Backward: standard GLU backward — dgate = dout * value * sigmoid'(gate_pre), dvalue = dout * gate
- cuBLASLt RELU_BIAS epilogue is no longer used for branch FC — replaced by separate sigmoid kernel after gate GEMM
- New CUDA kernel `glu_combine`: `output[i] = sigmoid(gate[i]) * value[i]`
- New CUDA kernel `glu_backward`: computes dgate and dvalue from doutput
- **Initialization**: W_gate zeros, b_gate zeros → sigmoid(0)=0.5 → GLU starts as 0.5 × value (halved linear). Model gradually learns which neurons to suppress/amplify. Avoids random gate patterns disrupting training.

### Forward pass detail
```
// Variable selection (Component 1) — input-dependent bottleneck
h_proj = h_s2 @ W_vsn1_d                         [B, R]    — bottleneck down
mask   = softmax(h_proj @ W_vsn2_d)              [B, SH2]  — per-sample mask
h_masked = h_s2 * mask                           [B, SH2]  — element-wise gate

// For d==1: h_masked = [h_masked; Q_dir/dz]     (Layer 3, already done)
input_d = h_masked  (or concat for d==1)          [B, K_d]

// GLU: two parallel GEMMs
gate_pre = W_gate_d @ input_d + b_gate_d          [B, AH]  — GEMM 1 (init: zeros → sigmoid=0.5)
value    = W_bdf_d  @ input_d + b_bdf_d            [B, AH]  — GEMM 2 (existing, xavier init)
h_bd     = sigmoid(gate_pre) * value              [B, AH]  — fused kernel
```

---

## Component 3: Cross-Branch Q Attention

### Problem
After computing 12 Q-values (4 branches × 3 actions), action selection treats them independently. But action quality is interdependent: Long+Full is aggressive (direction × magnitude correlation), Long+Small is cautious (different risk profile). The Q-values should coordinate.

Layer 3 provides direction→magnitude conditioning at the FC level. Cross-branch attention provides ALL-to-ALL coordination at the Q-value level.

### Solution
Apply 2-head self-attention on the 12 Q-values:

```
Q_raw[B, 12] = compute_expected_q(v_logits, adv_logits)

// Attention on Q-values
Q_proj = Q_raw @ W_QKV[3, 12, 12]  + b_QKV[3, 12]     // Q, K, V projections
// 2 heads, dim 6 each
For each head h:
  attn_h = softmax(Q_h @ K_h^T / sqrt(6))              // [B, 6, 6] (tiny)
  head_h = attn_h @ V_h                                 // [B, 6]
concat = [head_0; head_1]                                // [B, 12]
Q_coord = concat @ W_O[12, 12] + b_O[12]               // output projection
Q_coord = layernorm(Q_coord + Q_raw)                    // residual + LN
```

### Properties
- 624 parameters total (3×144 QKV + 144 W_O + 24 biases + 24 LN)
- 12-dim vectors — the attention computation is trivially cheap (no GEMM needed, just register-level dot products)
- Single CUDA kernel, one thread per sample, all computation in registers
- Residual connection ensures Q_coord ≈ Q_raw initially (W_O, W_QKV initialized near zero)
- Generalizes Layer 3: direction↔magnitude interaction emerges naturally, plus order↔urgency and all other pairs

### Gradient Signal
The attention needs direct gradient flow, not just delayed reward feedback. Two integration points:

1. **Experience collection**: Q_coord used for action selection (Boltzmann). Better actions → better rewards → indirect signal.
2. **Training forward**: Q_coord used for the **Expected SARSA target** (stochastic action sampling in c51_loss_kernel). The target action is sampled from softmax(Q_coord/tau) instead of softmax(Q_raw/tau). This gives DIRECT gradient flow: C51 target quality depends on Q_coord → gradient flows through attention backward → updates W_QKV.

The second point is critical. Without it, 624 parameters learning from delayed reward alone would take hundreds of epochs.

### Implementation
- New CUDA kernel `cross_branch_q_attention` in experience_kernels.cu
- 624-param weight buffer, separate from DQN params (own Adam state with LR=1e-4)
- Experience collection: runs AFTER compute_expected_q, BEFORE action selection
- Training: runs inside c51_loss_kernel or as a pre-processing step that transforms Q-values used for Expected SARSA target sampling
- Backward: standard attention backward on 12-dim vectors (trivial), applied after C51 backward produces d_Q_coord
- Near-zero init for W_O and W_QKV: Q_coord ≈ Q_raw at start (residual dominates)

---

## Component 4: Liquid Tau — Per-Branch Adaptive Learning Rate

### Problem
All 4 branches learn at the same rate via a shared Adam optimizer. But branches have different convergence speeds: direction (3 high-impact actions) converges faster than urgency (3 low-impact actions). The single Q-gap momentum (spread velocity) is a global modulator — it can't help a stuck branch while slowing a converging one.

### Solution
Per-branch continuous-time dynamics inspired by Liquid Time-Constant (LTC) neurons:

```
For each branch d:
  q_gap_d = max(Q_d) - mean(Q_d)                    // per-branch Q-gap
  velocity_d = q_gap_d - q_gap_ema_d                 // per-branch velocity
  
  // Liquid ODE: tau controls adaptation speed
  tau_d = tau_min + (tau_max - tau_min) * sigmoid(velocity_d / delta_z)
  
  // Branch scale evolves toward 1.0 at rate 1/tau_d
  branch_scale_d += (1.0/tau_d) * (1.0 - branch_scale_d) * dt
```

- Fast learning (positive velocity) → large tau → slow adaptation → don't overshoot
- Stuck (zero velocity) → small tau → fast adaptation → push harder
- Shrinking Q-gap → very small tau → emergency push → prevent collapse

### Properties
- 4 per-branch Q-gap EMAs (replace single q_gap_ema from spread momentum)
- tau_min = 0.01, tau_max = 1.0 (from existing Liquid hyperparameters)
- dt = 1.0 per training step (discrete approximation of continuous ODE)
- Zero extra parameters — only CPU-side state (4 floats EMA + 4 floats liquid_mod)
- Liquid tau MODULATES IQL branch_scales, does NOT overwrite them. IQL computes per-sample per-branch scales `[B, 4]`. Liquid tau produces 4 per-branch scalars `liquid_mod[4]`. The C51 grad kernel reads both: `effective_scale = branch_scales[b, d] * liquid_mod[d]`
- 4 pinned device-mapped floats `liquid_mod[4]` (same pattern as `spread_velocity[1]`)
- Subsumes the global spread_velocity modulator (spread_velocity becomes `liquid_mod[0]` — direction branch tau)

### Implementation
- In `reduce_current_q_stats`: compute per-branch Q-gaps from the 12 Q-values
- In `update_eval_v_range` (or new method `update_liquid_tau`): run liquid ODE for each branch, write to `liquid_mod_pinned[4]`
- c51_grad_kernel: add `liquid_mod` parameter `[4]`, multiply `branch_scale * liquid_mod[d]` for final scale
- spread_velocity parameter removed from c51_grad_kernel (subsumed by liquid_mod[0])

---

## Component 5: Mamba-2 Selective Replay

### Problem
PER (Prioritized Experience Replay) uses `|TD_error| + epsilon` as priority. This is state-agnostic: a transition with TD_error=0.5 gets the same priority whether the model is currently struggling with that state region or has already mastered it. The model wastes training on transitions it's already learned.

### Solution
Learned selectivity gate conditioned on the current model state, inspired by Mamba2's selective state space mechanism:

```
// During training, after forward pass produces save_h_s2:
selectivity[B] = sigmoid(W_sel @ save_h_s2[B, SH2] + b_sel)   // [B, 1]

// Update PER priorities with state-conditioned relevance
priority[i] = |TD_error[i]| * (1 + selectivity[i]) + epsilon

// Train selectivity gate via BCE on per-sample loss feedback
// Per-sample gradient norms are INFEASIBLE ([B, total_params] = 32GB).
// Per-sample C51 loss is already computed in per_sample_loss_buf[B] — cheap, accurate proxy.
loss_target[i] = per_sample_loss[i] / mean(per_sample_loss)   // normalize to ~1.0
L_sel = BCE(selectivity, clamp(loss_target, 0, 1))             // binary cross-entropy
// Separate Adam update for W_sel, b_sel
```

### Properties
- 257 extra parameters: W_sel[1, SH2] + b_sel[1]
- Separate Adam state: 2 × 257 = 514 floats (negligible)
- Forward: one dot-product [SH2] per sample → sigmoid → scale priorities
- The gate learns which trunk activations predict high-loss (informative) transitions
- Early training: selectivity ≈ 0.5 (uniform) — behaves like standard PER
- Late training: selectivity differentiates — focuses on frontier states where model is uncertain
- Uses per_sample_loss_buf (already computed by C51 loss kernel) as training target — zero extra GPU compute

### Implementation
- W_sel and b_sel stored in a separate small buffer (not in main params_buf)
- New CUDA kernel `selectivity_forward`: `sel[i] = sigmoid(dot(W_sel, h_s2[i]) + b_sel)` — one thread per sample
- New CUDA kernel `selectivity_backward`: BCE gradient on (sel, normalized per_sample_loss) → dW_sel, db_sel
- Priority update in PER: multiply existing priority by `(1 + sel)` after each training step
- Separate Adam update for W_sel/b_sel (2 momentum buffers, tiny, LR=1e-4, clip=1.0)

### Training loop integration
```
1. Sample batch from PER (existing)
2. Forward pass → save_h_s2
3. Selectivity forward: sel = sigmoid(W_sel @ save_h_s2 + b_sel)
4. C51/MSE loss + backward → per_sample_loss_buf, grad_buf
5. Selectivity backward: BCE(sel, per_sample_loss / mean(per_sample_loss)) → update W_sel
6. Adam step (main params)
7. Update PER priorities: priority *= (1 + sel)
```

---

## Component 6: Trajectory Backtracking (single GPU)

### Problem
Every training run hits the Q-gap plateau at epoch ~22. The current approach (spread gradient, Q-gap momentum, liquid tau) tries to PREVENT the plateau. But prevention can fail. There is no RECOVERY mechanism — once stuck, the model trains for 178 more frozen epochs, wasting compute.

Shrink-and-perturb perturbs FORWARD from the stuck state. But the stuck state's optimizer momentum (Adam m/v buffers) points in a dead-end direction. Perturbing weights while keeping bad momentum rarely escapes.

### Solution
Lazy trajectory backtracking: save checkpoints ranked by improvement rate, detect plateau, REWIND to the best pre-plateau checkpoint, apply informed perturbation, retry. Single GPU, sequential exploration.

```
Normal: epoch 1→18 (improving, checkpoint saved) → 19→22 (plateau detected)
Rewind: restore epoch 18 checkpoint
Route 1: reset Adam + LR×2 → train 10 epochs → still stuck?
Route 2: restore epoch 18 + shrink-perturb(σ=0.1) + temp boost → train 10 epochs → improving? KEEP
Total: 42 epochs instead of 200 frozen epochs
```

### Checkpoint Ring Buffer
```rust
struct TrajectoryCheckpoint {
    epoch: usize,
    improvement_rate: f32,      // d(val_Sharpe)/d(epoch) at save time
    params_host: Vec<f32>,      // DtoH copy of params_buf (~2MB)
    target_host: Vec<f32>,      // DtoH copy of target_params_buf (~2MB)
    adam_step: u32,             // optimizer step counter
    ema_state: EmaSnapshot,    // q_gap_ema, eval_q_mean_ema, etc.
}
// Top-3 by improvement rate, minimum parameter distance between entries
```

### Plateau Detection
Uses existing liquid tau per-branch velocity. Plateau = all 4 branch velocities ≈ 0 for N consecutive Q-stats updates (N=50, ~5 epochs at 10 updates/epoch).

### Informed Perturbation Cycle
Each rewind attempt uses a different strategy, cycling through:

| Route | Strategy | Rationale |
|-------|----------|-----------|
| 1 | Reset Adam (m=0, v=0) | Fresh gradient estimates from good weights |
| 2 | Shrink-perturb (σ=0.1) on branch heads only | Diversify action evaluation, keep trunk features |
| 3 | Temperature boost (2× Boltzmann + SARSA tau for 5 epochs, then cool) | Force re-exploration from good weights |
| 4 | LR×2 for 10 epochs then restore | Stronger gradient signal to break saddle point |

### Properties
- ~8MB per checkpoint (params + target + scalars), 3 checkpoints = 24MB CPU RAM
- Zero CUDA kernel changes — all Rust in training_loop.rs
- Max 3 rewinds per run, min 10 epochs between rewinds
- Max 4 routes per rewind (cycle through perturbation strategies)
- Depth limit prevents infinite loops
- Replay buffer NOT restored (transitions are still valid data)
- Liquid tau EMAs reset on rewind (stale velocity from pre-plateau)
- Compatible with CUDA graph (params_buf pointer unchanged, only data changes via HtoD)

### Early Termination (PLATEAU_EXHAUSTED)
When all rewinds × all routes are exhausted without breaking through:

```
rewind 1 (best checkpoint) → 4 routes → all failed (no improvement > threshold)
rewind 2 (2nd best)        → 4 routes → all failed
rewind 3 (3rd best)        → 4 routes → all failed
→ STOP. Save best model. Exit with status PLATEAU_EXHAUSTED.
```

**Exit criteria:** `rewind_count >= max_rewinds (3) && route_idx >= max_routes (4)` for the current rewind.

**"Failed" route definition:** improvement rate over M=10 epochs is below `min_improvement_rate` threshold (e.g. 0.1 val_Sharpe/epoch). A route that improves but slowly still counts as success.

**On termination:**
1. Save the best checkpoint across ALL attempts (highest absolute val_Sharpe)
2. Log: `"PLATEAU_EXHAUSTED: {rewinds}×{routes}={total} attempts. Best val_Sharpe={best} at epoch {ep}. Structural change needed."`
3. Exit training loop cleanly (not a crash — the Argo workflow sees success with the best model saved)

**Benefit:** train-w6qfd ran 200 epochs frozen from epoch 22 = 178 wasted epochs (~3 hours H100). With backtracking: worst case ~142 epochs with a definitive answer. Best case: breaks through and continues improving.

### Implementation
- `BacktrackingState` struct in `crates/ml/src/trainers/dqn/trainer/mod.rs`
- `TrajectoryCheckpoint` struct for checkpoint data
- Checkpoint save/restore methods in `training_loop.rs`
- Plateau detection reads liquid tau velocity (already computed)
- Early termination returns `Ok(TrainingResult::PlateauExhausted { best_epoch, best_sharpe })`
- ~150 lines total, no new files

---

## Dead Code Cleanup

Remove before building new system:

| Item | File | Action |
|------|------|--------|
| `_reward_norm_kernel` field + load | gpu_experience_collector.rs | Delete field, delete load_function call |
| `reward_normalize_kernel` function | nstep_kernel.cu | Keep in .cu (other kernels in same file), delete Rust load |
| `cea_weight` parameter | experience_kernels.cu env_step | Remove param, remove from all Rust launch sites |
| `mixup_alpha`, `mixup_seed`, `mixup_barrier` params | c51_loss_kernel.cu | Remove params, remove from all Rust launch sites |
| `clipped_saxpy_kernel` field | gpu_dqn_trainer.rs | Delete field, delete load |
| `apply_cql_clipped_saxpy` method | gpu_dqn_trainer.rs | Delete entire method |
| `clip_grad_buf_inplace` method | gpu_dqn_trainer.rs | Delete entire method |
| `debug_forward_no_graph` method | gpu_dqn_trainer.rs | Delete entire method |
| `GpuTrainingGuard::new()` alias | gpu_training_guard.rs | Replace callers with direct constructor |
| Decision transformer `#[allow(dead_code)]` block | decision_transformer.rs | Delete dead block |
| GPU curiosity `#[allow(dead_code)]` block | gpu_curiosity_trainer.rs | Delete dead block |

---

## Weight Layout

Existing 26 tensors (indices 0-25) unchanged. New tensors appended:

| Index | Tensor | Size | Purpose |
|-------|--------|------|---------|
| 26 | w_vsn1_0 | SH2 × R | Direction VSN bottleneck down (R=16) |
| 27 | w_vsn2_0 | R × SH2 | Direction VSN bottleneck up |
| 28 | w_vsn1_1 | SH2 × R | Magnitude VSN bottleneck down |
| 29 | w_vsn2_1 | R × SH2 | Magnitude VSN bottleneck up |
| 30 | w_vsn1_2 | SH2 × R | Order VSN bottleneck down |
| 31 | w_vsn2_2 | R × SH2 | Order VSN bottleneck up |
| 32 | w_vsn1_3 | SH2 × R | Urgency VSN bottleneck down |
| 33 | w_vsn2_3 | R × SH2 | Urgency VSN bottleneck up |
| 34 | w_gate_0 | AH × SH2 | Direction GLU gate weight (init: zeros) |
| 35 | b_gate_0 | AH | Direction GLU gate bias (init: zeros) |
| 36 | w_gate_1 | AH × (SH2+3) | Magnitude GLU gate weight (init: zeros, sees Q_dir) |
| 37 | b_gate_1 | AH | Magnitude GLU gate bias (init: zeros) |
| 38 | w_gate_2 | AH × SH2 | Order GLU gate weight (init: zeros) |
| 39 | b_gate_2 | AH | Order GLU gate bias (init: zeros) |
| 40 | w_gate_3 | AH × SH2 | Urgency GLU gate weight (init: zeros) |
| 41 | b_gate_3 | AH | Urgency GLU gate bias (init: zeros) |

Separate buffers (not in params_buf, own Adam state):
- Q attention: 624 floats
- Selectivity: 257 floats
- Liquid tau: 4 pinned device-mapped floats (CPU-written, GPU-read)

NUM_WEIGHT_TENSORS: 26 → 42

### Parameter count summary
| Component | Parameters |
|-----------|-----------|
| VSN bottleneck (8 matrices, R=16) | 32,768 |
| GLU gate weights (4 × AH × K_d) | ~263,168 |
| GLU gate biases (4 × AH) | 1,024 |
| Q attention (separate) | 624 |
| Selectivity (separate) | 257 |
| Liquid tau (CPU state) | 4 |
| **Total new** | **~297,845** |

---

## CUDA Graph Impact

- Variable selection bottleneck: two small GEMMs + softmax per branch, inside graph (input-dependent, must run per-step)
- GLU gate GEMM: inside graph (replaces RELU_BIAS fused epilogue)
- Cross-branch Q attention: AFTER graph (runs on Q-values, not on the training forward)
- Liquid tau: CPU-side only (writes to pinned device-mapped branch_scales)
- Selectivity gate: AFTER graph (runs on save_h_s2 from graph forward)

Only the GLU gate GEMM is inside the CUDA graph. This requires graph recapture (one-time cost). The existing RELU_BIAS epilogue is removed for branch FC — replaced by separate gate GEMM + sigmoid_multiply kernel.

---

## Success Criteria

| Metric | Current (train-w6qfd) | Target | Measurement |
|--------|----------------------|--------|-------------|
| val_Sharpe sustained | 22.18 frozen at epoch 22 | >25 through epoch 50+ | No freeze |
| Q-gap growth | 0.133 locked | >0.2 by epoch 30, still growing | Per-branch Q-gaps |
| OOS Sharpe | ~0 oscillating | >0 sustained for 10+ epochs | OOS from trade stats |
| Feature selectivity | N/A | Branches show different feature masks | softmax(v_select) divergence |
| Branch learning rates | Uniform | Direction converges 2-3× faster than urgency | Per-branch tau values |
| Replay selectivity | Uniform PER | Selectivity varies 0.2-0.8 across states | selectivity distribution |

---

## Risks and Mitigations

| Risk | Mitigation |
|------|-----------|
| GLU doubles branch FC parameters → overfitting | Q-gap momentum + Liquid tau prevent overfitting by modulating gradient intensity |
| Cross-branch attention creates circular dependency in action selection | Residual connection + near-zero init ensures Q_coord ≈ Q_raw early → gradual coordination |
| Selectivity gate collapse (all 0 or all 1) | BCE loss normalizes targets to ~1.0; sigmoid saturates slowly at ±5 |
| Variable selection mask collapse (one feature dominates) | Softmax temperature: mask = softmax(v_select / temperature), temperature=1.0 initially |
| Graph recapture from GLU change | One-time cost at epoch 0, well-understood pattern |
| Checkpoint incompatibility (38 vs 26 tensors) | New training runs only, no migration from old checkpoints |
| Selectivity gate training instability | Separate Adam with low LR (1e-4), gradient clipping to 1.0 |

---

## Implementation Order

1. **Dead code cleanup** — remove 11 items, clean build
2. **Variable selection + GLU** — branch FC upgrade (Components 1+2, tightly coupled)
3. **Cross-branch Q attention** — post-Q coordination (Component 3, independent)
4. **Liquid tau** — per-branch dynamics (Component 4, independent)
5. **Mamba selective replay** — learned curriculum (Component 5, independent)

Components 3, 4, 5 are independent of each other and can be implemented in any order after 1+2.
