# Gems & Pearls — 32+ Generalization Techniques for DQN Trading Agent

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Close the in-sample/OOS generalization gap (Sharpe +0.57 IS → -1.30 OOS). The model memorizes price sequences instead of learning regime-invariant market structure. These 28 techniques attack every axis of memorization.

**Architecture:** Branching DQN with 81 factored actions (9 exposure x 3 order x 3 urgency), 14 active features, fused CUDA training pipeline, C51+MSE blended loss, CQL regularization, curiosity model, ensemble heads.

**Tech Stack:** Rust, CUDA (cudarc), existing fused CUDA kernels, existing curiosity forward model

**Status Key:** DONE = committed, WIP = in progress, PLAN = not started

---

## Previously Planned (Tasks 1-8 from original plan)

### Task 1: Domain Randomization — Per-Epoch Simulation Parameter Jitter [DONE]

**Commit:** `c3ea0e57`

**Files:**
- Modified: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modified: `crates/ml/src/trainers/dqn/config.rs`

Per-epoch randomization of spread, tx_cost, fill probabilities, initial capital, episode start positions, and episode lengths. Prevents memorizing fixed simulation parameters.

- [x] Config flag `enable_domain_randomization`
- [x] Randomize ExperienceCollectorConfig per epoch
- [x] Episode start jitter +-25% stride
- [x] Variable episode length
- [x] TOML configs updated
- [x] Compiled + tested

---

### Task 2: CQL Alpha Boost + Gradient Budget Rebalance [DONE]

**Commit:** `ae995673`

**Files:**
- Modified: `crates/ml/src/trainers/dqn/config.rs`
- Modified: `crates/ml/src/trainers/dqn/fused_training.rs`

CQL alpha 0.1 -> 1.0, gradient budget 15% -> 25%, C51 budget 70% -> 60%.

- [x] CQL alpha increased
- [x] Gradient budget rebalanced
- [x] Compiled + tested

---

### Task 3: Periodic Shrink-and-Perturb [DONE]

**Commit:** `37cb034e`

**Files:**
- Modified: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modified: `crates/ml/src/trainers/dqn/config.rs`

Shrink-and-perturb every 20 epochs: keep 85% of weights, reinitialize 15% with noise sigma=0.01. Kills memorized weight patterns periodically.

- [x] Config fields `shrink_perturb_interval`, `_alpha`, `_sigma`
- [x] Periodic S&P in training loop
- [x] Compiled + tested

---

### Task 4: Adversarial Regime Injection [DONE]

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/mod.rs`

When the model achieves low drawdown (<10%) for 3+ consecutive epochs, inject adversarial conditions: 3x spread, 2x tx_cost, 0.5x fill probability. Forces the model to handle worst-case scenarios.

- [ ] Add config `adversarial_injection_threshold` (0.10), `adversarial_injection_consecutive` (3)
- [ ] Add tracking field `consecutive_low_dd_epochs` in trainer state
- [ ] Implement detection + injection in training loop
- [ ] Apply adversarial multipliers to ExperienceCollectorConfig
- [ ] Log when adversarial mode activates
- [ ] Compile + test + commit

---

### Task 5: Anti-Correlation Reward Penalty [DONE] (via beta_penalty in env_step kernel)

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu`

At epoch end, compare action distributions between training and validation. If cosine similarity > 0.8, the model does the same thing everywhere — it hasn't learned to differentiate. Penalize next epoch's rewards by 0.8x.

- [ ] Compute cosine similarity of train vs val action distributions
- [ ] Add `reward_scale_override: f32` to ExperienceCollectorConfig
- [ ] Multiply reward by scale factor in experience kernel
- [ ] Compile + test + commit

---

### Task 6: Curiosity Prediction Error as Q-Penalty [DONE] (via curiosity_q_penalty_lambda in loss kernels)

**Files:**
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`
- Modify: `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` or `c51_loss_kernel.cu`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

The curiosity forward model's prediction error measures how "novel" a state-action pair is. Wire it as an additive penalty to the Q-target in the Bellman equation:
```
target_Q = reward + gamma * (1-done) * target_Q_next - curiosity_penalty * prediction_error
```
High prediction error = novel state = conservative Q-values. Same principle as CQL but data-driven.

- [ ] Add config `curiosity_q_penalty_weight` (default 0.5)
- [ ] Compute per-sample curiosity error in fused training step
- [ ] Pass `curiosity_errors` buffer to loss kernel
- [ ] Subtract penalty from Q-target in kernel
- [ ] Wire through Rust launch code
- [ ] TOML configs
- [ ] Compile + full smoke test

---

### Task 7: Counterfactual Experience Augmentation [DONE]

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`

For every trade taken, also compute the counterfactual: what would have happened with the mirror action (Long<->Short). Store both with inverted reward signs. Doubles effective data diversity.

- [ ] In experience kernel, compute reward for mirror action
- [ ] Write both to output buffers (2x output size)
- [ ] Counterfactual reward = -1 x actual reward
- [ ] Compile + test + commit

---

### Task 8: Hindsight Regime Labeling — Feature Importance Filtering [DONE]

**Files:**
- Create: `crates/ml/src/trainers/dqn/feature_importance.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

After each epoch, compute gradient x activation for each feature to identify which features predicted profitable trades. Mask the bottom 50% (set to 0) next epoch. Iterative: the model discovers which features carry signal and ignores noise.

- [ ] Compute per-feature importance from gradient x activation
- [ ] Rank features, compute binary mask (top-K survive)
- [ ] Apply mask in experience kernel (zero out masked features)
- [ ] Iterate — recompute importance each epoch
- [ ] Compile + test + commit

---

### Task 9: Regime-Adversarial Training (Gradient Reversal/DANN) [DONE]

**Files:**
- Create: `crates/ml/src/cuda_pipeline/regime_discriminator_kernel.cu`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`

Train a small classifier head on DQN hidden features that predicts which time window (fold) data comes from. Apply GRADIENT REVERSAL — the DQN learns features that the discriminator CANNOT use to identify the regime. Domain Adversarial Neural Networks (DANN) adapted for temporal domains.

- [ ] Add discriminator kernel: small MLP [hidden_dim -> 64 -> num_folds]
- [ ] Wire gradient reversal: negate discriminator gradient before shared trunk
- [ ] Add fold ID to experience buffer
- [ ] Compile + test + commit

---

## Gems & Pearls — Category A: Symmetry & Invariance Exploits

### Task 10: Mirror Universe Training [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Easy | **Impact:** High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (return sign flip)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (mirror flag)
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (epoch-level toggle)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Flip ALL returns (multiply by -1) and invert the action mapping (L100<->S100, L75<->S75, etc.). A model that truly understands market structure should produce identical P&L on mirror data with mirror actions. If IS Sharpe on mirrored data differs from normal data, the model has learned a directional bias, not structure. Train on both simultaneously — doubles data diversity with zero extra market data.

- [ ] Add config `enable_mirror_universe` (default true)
- [ ] Add `mirror_active: bool` to ExperienceCollectorConfig
- [ ] In experience kernel: when mirror=true, negate all return-based features and invert action index (8-action_idx for exposure)
- [ ] Alternate: epoch N normal, epoch N+1 mirrored (or 50/50 within epoch)
- [ ] Verify P&L symmetry on mirrored vs normal epochs
- [ ] Compile + test + commit

---

### Task 11: Time-Reversal Augmentation [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (reversed indexing)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (reverse flag)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Play episodes backwards. Market microstructure has statistical time-reversal symmetry at short horizons (bid-ask bounce). If the model can't trade profitably on reversed sequences, it's fitting to temporal artifacts (trends, momentum) rather than structural features. Mix 20% reversed episodes into each epoch — forces reliance on instantaneous features (spread, order flow) rather than trajectory.

- [ ] Add config `time_reversal_fraction` (default 0.2)
- [ ] In experience kernel: reverse the bar indexing for selected episodes (bar[T-t] instead of bar[t])
- [ ] Adjust next_close accordingly (now previous close)
- [ ] Mark reversed episodes so reward calculation handles inverted time
- [ ] Compile + test + commit

---

### Task 12: Price-Level Invariance Enforcement [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Medium | **Impact:** Very High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (dual forward pass)
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (invariance loss term)
- Modify: `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` (extra loss component)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Add a regularization loss: run the same features through the network twice — once at actual levels, once with all price-derived features shifted by a random constant. The Q-values MUST be identical. If they differ, the model has learned absolute price levels (which never generalize).

```
L_invariance = MSE(Q(s), Q(s + delta))
```

Added to the Bellman loss with configurable weight.

- [ ] Add config `price_invariance_weight` (default 0.1)
- [ ] During training step, create shifted copy of feature batch (add random delta to price-derived feature indices)
- [ ] Forward pass both original and shifted through Q-network
- [ ] Compute MSE between Q-value outputs
- [ ] Add weighted invariance loss to total loss
- [ ] Compile + test + commit

---

### Task 13: Volatility Regime Normalization [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** Very High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (feature preprocessing)
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (normalization in kernel)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Before feeding features to the network, divide ALL return-based features by a 20-bar realized vol estimate. This makes the feature vector scale-invariant across regimes. A 1% move in a 5% vol regime should look identical to a 0.2% move in a 1% vol regime. This is preprocessing, not model change — but eliminates the #1 source of regime-specific memorization.

- [ ] Add config `enable_vol_normalization` (default true), `vol_lookback` (default 20)
- [ ] Compute running realized vol from close-to-close returns in experience kernel
- [ ] Divide return-based features (indices 0-7 approximately) by max(realized_vol, 1e-8)
- [ ] Leave non-return features (ADX, CUSUM, volume ratios) unnormalized
- [ ] Compile + test + commit

---

## Gems & Pearls — Category B: Adversarial & Game-Theoretic

### Task 14: Market Maker Adversary (GAN for Trading) [DONE] (superseded by GPU-native saboteur #33)

**Phase:** 3 (Architecture) | **Difficulty:** Hard | **Impact:** Very High

**Files:**
- Create: `crates/ml/src/cuda_pipeline/adversary_kernel.cu`
- Create: `crates/ml/src/trainers/dqn/market_maker_adversary.rs`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Train a SECOND tiny network as the "adversarial market maker." It controls spread, fill probability, and slippage, and is trained to MAXIMIZE the DQN's losses. The DQN trains against this adversary. Cycle: 5 epochs adversarial training, 5 epochs recovery with frozen adversary. The DQN learns to survive worst-case execution — the ultimate generalization pressure.

- [ ] Add config `enable_market_maker_adversary` (default false), `adversary_cycle_epochs` (5)
- [ ] Create adversary network: small MLP [state_dim -> 32 -> 3] outputting (spread_mult, fill_prob, slippage_mult)
- [ ] Adversary loss = -1 * DQN reward (maximize DQN losses)
- [ ] In training loop, alternate: adversary trains for N epochs, then freeze and let DQN recover for N epochs
- [ ] Pass adversary outputs to ExperienceCollectorConfig
- [ ] Compile + test + commit

---

### Task 15: Gradient Vaccine [DONE] (implemented as Task 32)

**Phase:** 2 (CUDA changes) | **Difficulty:** Medium | **Impact:** Very High

**Files:**
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (gradient projection)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (dual gradient computation)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

At each training step, compute gradients on BOTH the training batch and a held-out validation batch. Project out the component of the training gradient that CONTRADICTS the validation gradient:

```
g_safe = g_train - (g_train . g_val / |g_val|^2) * g_val   [when dot product < 0]
```

Only update in directions that both sets agree on. The model literally cannot learn anything that doesn't generalize — overfitting gradients are surgically removed.

- [ ] Add config `enable_gradient_vaccine` (default true), `vaccine_val_fraction` (0.2)
- [ ] Split each training batch: 80% train, 20% held-out validation
- [ ] Compute gradients on both subsets separately
- [ ] Compute dot product of gradient vectors
- [ ] When dot product < 0, project out conflicting component
- [ ] Apply safe gradient to optimizer
- [ ] Compile + test + commit

---

### Task 16: Phantom Liquidity Episodes [DONE]

**Phase:** 3 (Architecture) | **Difficulty:** Medium | **Impact:** High

**Files:**
- Create: `crates/ml/src/trainers/dqn/synthetic_data.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Every 5th epoch, replace 30% of episodes with SYNTHETIC data — GBM with matched vol and drift, or shuffled 50-bar blocks of real data. The model must trade these profitably too. If it can only trade on real sequences but fails on statistically-identical synthetic ones, it has memorized specific price patterns. Ultimate memorization detector AND training regularizer.

- [ ] Add config `phantom_fraction` (0.3), `phantom_epoch_interval` (5)
- [ ] Create GBM price generator: match vol, drift, and autocorrelation from real data
- [ ] Alternative: block-shuffle real data (50-bar blocks, random permutation)
- [ ] Replace selected episodes' price data with synthetic prices
- [ ] Recompute features from synthetic prices
- [ ] Track Sharpe on phantom vs real episodes separately (diagnostic)
- [ ] Compile + test + commit

---

## Gems & Pearls — Category C: Loss & Reward Reshaping

### Task 17: Counterfactual Regret Minimization Reward [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Medium | **Impact:** High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (regret computation)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Replace raw PnL reward with **regret**: `reward = PnL(action_taken) - max(PnL(all_actions))`. From game theory (CFR). The model learns to minimize regret rather than maximize absolute P&L. Regret is naturally normalized across regimes — a -$100 trade when the best was -$50 has regret -$50, same as a +$200 trade when the best was +$250. Eliminates regime-dependent reward scaling.

- [ ] Add config `enable_regret_reward` (default false), can coexist with normal reward
- [ ] In experience kernel, after computing reward for taken action, compute reward for all 9 exposure levels
- [ ] Regret = reward(taken) - max(reward(all_exposures))
- [ ] Regret is always <= 0 (perfect play = 0 regret)
- [ ] Option: blend regret with raw reward (0.5 * reward + 0.5 * regret)
- [ ] Compile + test + commit

---

### Task 18: Asymmetric Drawdown Loss [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` (asymmetric penalty)
- Modify: `crates/ml/src/cuda_pipeline/c51_loss_kernel.cu` (asymmetric penalty)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Add a term to the Bellman loss that specifically penalizes Q-value OVERESTIMATION on states where the portfolio is in drawdown:

```
L_dd = max(0, Q_predicted - Q_target) * drawdown_depth^2
```

When the model is in drawdown, overestimating Q-values is catastrophic (it holds losing positions expecting recovery). Makes the model pessimistic during drawdowns — the #1 OOS failure mode.

- [ ] Add config `asymmetric_dd_weight` (default 0.5)
- [ ] Pass current drawdown depth to loss kernel as per-sample float
- [ ] In kernel: when Q_pred > Q_target AND drawdown > 0, add penalty = overestimation * dd^2 * weight
- [ ] Penalty only activates during drawdown (zero cost when equity is at highs)
- [ ] Compile + test + commit

---

### Task 19: Entropy-Regulated Position Trajectory [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (position histogram + entropy bonus)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Add an entropy bonus not to the ACTION distribution (standard) but to the POSITION TRAJECTORY over an episode. The model should visit multiple exposure levels, not get stuck at one. Compute the entropy of [time_at_S100, time_at_S75, ..., time_at_L100] and add to reward:

```
reward += position_entropy_weight * H(position_histogram)
```

Prevents degenerate strategies of "always flat" or "always long" that score well IS but collapse OOS.

- [ ] Add config `position_entropy_weight` (default 0.01)
- [ ] Track per-episode position histogram in experience kernel (shared memory, 9 bins)
- [ ] At episode end, compute entropy of histogram
- [ ] Add scaled entropy bonus to final episode reward
- [ ] Compile + test + commit

---

## Gems & Pearls — Category D: Network Architecture Tricks

### Task 20: Weight Lottery Ticket Pruning [DONE]

**Phase:** 3 (Architecture) | **Difficulty:** Medium | **Impact:** High

**Files:**
- Create: `crates/ml/src/trainers/dqn/pruning.rs`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (apply mask)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (mask buffers)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

After 50 epochs of training, prune 70% of weights (smallest magnitude -> zero), then RETRAIN from scratch keeping only the surviving mask. The "lottery ticket hypothesis" says the sparse subnetwork generalizes better because it physically cannot memorize as much. Run twice: 100% -> 30% -> 9% of original parameters.

- [ ] Add config `pruning_epoch` (50), `pruning_fraction` (0.7), `pruning_rounds` (2)
- [ ] After pruning_epoch, compute weight magnitude across all layers
- [ ] Create binary mask: top 30% survive, bottom 70% = 0
- [ ] Store mask as GPU buffer, apply element-wise after each optimizer step (w = w * mask)
- [ ] Reset optimizer state for pruned weights
- [ ] Optional: second round at epoch 100 (prune 70% of surviving 30% = 9% total)
- [ ] Compile + test + commit

---

### Task 21: Stochastic Depth (Layer Dropout) [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (layer skip logic)
- Modify: `crates/ml-dqn/src/dqn.rs` (forward pass with skip)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

During training, randomly skip entire layers of the DQN trunk with 20% probability (replace with identity/passthrough). Each forward pass uses a random subset of the network's depth. Forces EVERY layer to produce useful features independently, not rely on deep compositional memorization. At inference, use all layers with scaled weights. Proven in vision transformers but nobody has applied it to RL trading.

- [ ] Add config `stochastic_depth_prob` (default 0.2)
- [ ] During forward pass, for each hidden layer, generate random float
- [ ] If random < prob, skip layer (output = input, bypass matmul + activation)
- [ ] Scale surviving layers by 1/(1-prob) to maintain expected magnitude
- [ ] At inference (eval mode), disable stochastic depth, use all layers
- [ ] Compile + test + commit

---

### Task 22: Feature-Wise Noise Injection [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (noise injection)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Instead of zeroing features (dropout), add calibrated Gaussian noise to each feature with sigma proportional to that feature's standard deviation:

```
feature_noisy = feature + N(0, noise_scale * std(feature))
```

Strictly better than dropout for continuous features: the model sees signal plus noise, must learn to be robust to feature measurement error. Real market features ARE noisy.

- [ ] Add config `feature_noise_scale` (default 0.1)
- [ ] Compute running std per feature (EMA or batch std)
- [ ] In experience kernel, add N(0, scale * std) to each feature during training
- [ ] Disable noise during evaluation/inference
- [ ] Use curand for GPU-native noise generation
- [ ] Compile + test + commit

---

## Gems & Pearls — Category E: Temporal & Causal Structure

### Task 23: Causal Feature Masking (Random Feature Forests) [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (feature masking)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (per-epoch mask generation)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Each epoch, randomly select 60% of features and mask the other 40% to zero. Different random subset each epoch. Over 100 epochs, the model trains on every possible feature combination. The surviving policy uses NO single feature as a crutch — it has built redundant representations. The "random forest" principle applied to RL feature extraction.

- [ ] Add config `feature_mask_fraction` (default 0.4), `enable_causal_masking` (default true)
- [ ] At epoch start, generate random binary mask of length num_features
- [ ] Upload mask to GPU
- [ ] In experience kernel, multiply features by mask (zero out masked features)
- [ ] Log which features are active each epoch
- [ ] Compile + test + commit

---

### Task 24: Drawdown-Conditional Learning Rate (Anti-Intuitive) [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (LR scheduling)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (adam LR parameter)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

When the model is doing WELL (low drawdown, high Sharpe), INCREASE the learning rate 3x. When doing poorly, decrease it. The OPPOSITE of standard practice. Why: when the model is doing well IS, it's likely settling into an overfit minimum. High LR kicks it out. When struggling, low LR stabilizes. Creates natural oscillation that prevents settling into regime-specific policies.

- [ ] Add config `enable_anti_lr` (default true), `anti_lr_good_mult` (3.0), `anti_lr_bad_mult` (0.3), `anti_lr_sharpe_threshold` (0.3)
- [ ] After epoch metrics, check if epoch Sharpe > threshold
- [ ] If good (Sharpe > threshold): next epoch LR *= good_mult
- [ ] If bad (Sharpe < -threshold): next epoch LR *= bad_mult
- [ ] If neutral: use base LR
- [ ] Clamp LR to [base_lr * 0.1, base_lr * 5.0] to prevent explosion
- [ ] Update adam_lr parameter in GPU trainer
- [ ] Compile + test + commit

---

### Task 25: Trade Clustering Penalty [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (inter-trade timing)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Compute the coefficient of variation of inter-trade intervals within each episode. If trades are temporally clustered (bursts then silence), penalize:

```
penalty = CV(inter_trade_times) * clustering_penalty_weight
```

Real generalizable strategies trade somewhat uniformly. Temporal clustering is a fingerprint of pattern-matching specific price sequences. Forces strategies that work across the ENTIRE episode.

- [ ] Add config `trade_clustering_penalty` (default 0.1)
- [ ] Track trade timestamps in experience kernel (shared memory array)
- [ ] At episode end, compute inter-trade intervals
- [ ] Compute CV = std(intervals) / mean(intervals)
- [ ] Subtract penalty * CV from episode reward
- [ ] Compile + test + commit

---

### Task 26: HER + Regime Tag Oversampling [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Medium | **Impact:** High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_her.rs` (regime-aware relabeling)
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (regime tagging)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

Extend existing HER: when relabeling goals, also tag each experience with a coarse regime label (trending/mean-reverting/volatile, derived from ADX + realized vol). During replay, OVERSAMPLE experiences from underrepresented regimes. The model sees equal amounts of each regime regardless of real data distribution.

- [ ] Add config `enable_regime_oversampling` (default true), `num_regime_bins` (3)
- [ ] In experience kernel, compute regime tag: ADX > 25 = trending, realized_vol > 2*median = volatile, else = mean-reverting
- [ ] Store regime tag in experience buffer alongside existing fields
- [ ] In PER sampling, weight by inverse regime frequency (rare regime = higher priority)
- [ ] Log regime distribution per epoch
- [ ] Compile + test + commit

---

## Gems & Pearls — Category F: Meta-Learning & Selection

### Task 27: Ensemble Disagreement as Q-Penalty [DONE]

**Phase:** 1 (Rust-only) | **Difficulty:** Easy | **Impact:** Very High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (ensemble variance computation)
- Modify: `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` or `c51_loss_kernel.cu`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`

You have an ensemble of Q-networks. When ensemble members DISAGREE on Q-value (high variance across heads), that state-action pair is UNCERTAIN. Add ensemble disagreement as a PENALTY to the Q-target:

```
Q_target -= beta * std(Q_head1, Q_head2, ..., Q_headN)
```

High disagreement -> lower Q-target -> conservative behavior on uncertain states. Novel OOS states will have high disagreement -> conservative behavior -> fewer catastrophic losses. This is epistemic uncertainty penalization.

- [ ] Add config `ensemble_disagreement_weight` (default 0.3)
- [ ] After ensemble forward pass, compute per-sample std across heads
- [ ] Pass std buffer to loss kernel
- [ ] Subtract weighted std from Q-target
- [ ] Monitor average disagreement per epoch (diagnostic: should decrease during training)
- [ ] Compile + test + commit

---

### Task 28: Survival-of-the-Flattest Model Selection [DONE]

**Phase:** 3 (Architecture) | **Difficulty:** Medium | **Impact:** High

**Files:**
- Create: `crates/ml/src/trainers/dqn/sharpness.rs`
- Modify: `crates/ml/src/hyperopt/adapters/dqn.rs` (selection criterion)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

During hyperopt, don't select the model with the best OOS Sharpe. Select the model whose loss landscape is FLATTEST — lowest |nabla^2 L| (Hessian trace). Approximate cheaply: perturb weights by epsilon in 10 random directions, measure loss change. Models in flat minima generalize; models in sharp minima memorize. Sharpness-Aware Minimization (SAM) applied to model SELECTION rather than training.

- [ ] Add config `sharpness_perturbation_epsilon` (0.01), `sharpness_num_directions` (10)
- [ ] After training completes, compute sharpness score:
  - For each of 10 random unit vectors, perturb all weights by +/- epsilon * direction
  - Compute loss on validation set for each perturbation
  - Sharpness = mean(|loss_perturbed - loss_original|)
- [ ] In hyperopt trial scoring: `score = oos_sharpe - sharpness_penalty_weight * sharpness`
- [ ] Models with low sharpness (flat minima) are preferred even if Sharpe is slightly lower
- [ ] Compile + test + commit

---

### Task 29: Cross-Fold Action Consistency Score [DONE]

**Phase:** 2 (CUDA changes) | **Difficulty:** Medium | **Impact:** Medium

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (cross-fold comparison)
- Modify: `crates/ml/src/hyperopt/adapters/dqn.rs` (consistency scoring)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

During walk-forward, for each state that appears in multiple folds' validation sets (overlapping features), compare the actions the model would take. If the model takes DIFFERENT actions on the same market conditions in different folds, it hasn't generalized:

```
consistency = 1 - mean(action_variance_across_folds)
```

Use as SECONDARY selection criterion alongside Sharpe. A model with Sharpe 0.3 and consistency 0.9 beats Sharpe 0.5 and consistency 0.4.

- [ ] Add config `consistency_weight` (default 0.3)
- [ ] During walk-forward, save action predictions for each fold's validation set
- [ ] For overlapping states (same bar indices across folds), compute action agreement
- [ ] Consistency = fraction of states where majority action matches
- [ ] In hyperopt scoring: `score = oos_sharpe * (1 + consistency_weight * consistency)`
- [ ] Compile + test + commit

---

## Complete Execution Order

### Phase 1 — Quick Wins, Rust-only (next session)
| # | Technique | Status |
|---|-----------|--------|
| 1 | Domain Randomization | DONE |
| 2 | CQL Alpha Boost | DONE |
| 3 | Shrink-and-Perturb | DONE |
| 4 | Adversarial Regime Injection | PLAN |
| 13 | Volatility Regime Normalization | PLAN |
| 18 | Asymmetric Drawdown Loss | PLAN |
| 22 | Feature-Wise Noise Injection | PLAN |
| 23 | Causal Feature Masking | PLAN |
| 24 | Anti-Intuitive LR | PLAN |
| 25 | Trade Clustering Penalty | PLAN |
| 27 | Ensemble Disagreement Q-Penalty | PLAN |

### Phase 2 — CUDA Kernel Changes
| # | Technique | Status |
|---|-----------|--------|
| 5 | Anti-Correlation Penalty | PLAN |
| 6 | Curiosity Q-Penalty | PLAN |
| 7 | Counterfactual Augmentation | PLAN |
| 10 | Mirror Universe Training | PLAN |
| 11 | Time-Reversal Augmentation | PLAN |
| 12 | Price-Level Invariance | PLAN |
| 15 | Gradient Vaccine | PLAN |
| 17 | Counterfactual Regret Reward | PLAN |
| 19 | Position Entropy | PLAN |
| 21 | Stochastic Depth | PLAN |
| 26 | HER + Regime Oversampling | PLAN |
| 29 | Cross-Fold Consistency | PLAN |

### Phase 3 — Architecture Changes (after validation)
| # | Technique | Status |
|---|-----------|--------|
| 8 | Feature Importance Filtering | PLAN |
| 9 | Regime-Adversarial (DANN) | PLAN |
| 14 | Market Maker Adversary | PLAN |
| 16 | Phantom Liquidity Episodes | PLAN |
| 20 | Lottery Ticket Pruning | PLAN |
| 28 | Flattest Selection (SAM) | PLAN |

## Expected Combined Impact

| Category | Techniques | Gap Reduction |
|----------|-----------|---------------|
| Symmetry & Invariance (A) | 10-13 | 40-60% |
| Adversarial & Game-Theoretic (B) | 14-16 | 30-50% |
| Loss & Reward Reshaping (C) | 17-19 | 25-40% |
| Network Architecture (D) | 20-22 | 20-35% |
| Temporal & Causal (E) | 23-26 | 30-45% |
| Meta-Learning & Selection (F) | 27-29 | 20-30% |
| Previously Planned (1-9) | 1-9 | 50-80% |

**Combined target:** OOS Sharpe from -1.30 to > 0.0 (positive expectancy OOS).

---

## Critical: FP32 Experience Collector Weights

### Task 30: Experience Collector FP32 Weight Conversion [DONE]

**Phase:** 1 (Critical) | **Difficulty:** Medium | **Impact:** Very High

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (change weight buffers from bf16 to f32)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (sync online weights as f32)
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (read f32 weights instead of bf16)

The experience collector currently receives online Q-network weights in bf16 format for its
forward passes during experience collection. This introduces quantization noise in Q-value
estimates used for action selection, TD target computation, and priority calculation.
Since the master weights in the Adam optimizer are already f32, the experience collector
should use f32 copies to eliminate bf16 quantization artifacts during data collection.

- [ ] Change `online_params_flat` in GpuExperienceCollector from `CudaSlice<half::bf16>` to `CudaSlice<f32>`
- [ ] Update weight sync to copy f32 master weights directly (skip bf16 conversion)
- [ ] Update cuBLAS SGEMM calls to use f32 (already supports it, just change pointer types)
- [ ] Update CUDA kernels to read f32 weights for forward pass during experience collection
- [ ] Verify Q-value precision improvement in smoke tests
- [ ] Compile + test + commit

---

## THE GEM OF GEMS

### Task 31: Temporal Causal Bottleneck — 2D Information Compression [DONE]

**Phase:** 0 (HIGHEST PRIORITY) | **Difficulty:** Medium | **Impact:** TRANSFORMATIVE

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/batched_forward.rs` (add bottleneck GEMM layer)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (bottleneck weights, backward pass)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (bottleneck in experience forward)
- Modify: `crates/ml-dqn/src/branching.rs` (add bottleneck layer to network definition)
- Modify: `crates/ml/src/trainers/dqn/config.rs` (bottleneck_dim config)

**The Core Idea:**

Force ALL information from the 14-feature market state through a 2-dimensional
bottleneck before it reaches the Q-network. Two neurons. The entire market must be
compressed into 2 numbers — call them **opportunity** and **risk**.

**Why this is the gem of all gems:**

1. **Information-theoretic impossibility of memorization.** With only 2 floats, the
   network physically CANNOT encode "which specific price sequence am I looking at."
   A 2D bottleneck has ~100 bits of capacity vs ~450 bits in the full 14-feature vector.
   Memorizing 100K episodes requires far more than 100 bits.

2. **Forced abstraction.** The bottleneck encoder must learn to compress 14 features
   into the 2 most DECISION-RELEVANT dimensions. These are by definition
   regime-invariant — if they weren't, the Q-network couldn't use them to trade
   profitably across regimes.

3. **Interpretability for free.** Plot the 2D bottleneck activations and literally
   see what the model thinks "opportunity" and "risk" look like. Immediate diagnostic
   for whether it learned signal or garbage.

4. **Architecture integration:** Add a single [market_dim → 2] linear + tanh layer
   before h_s1. One extra GEMM in the cuBLAS forward pass. The 2D output concatenates
   with 3 portfolio features → [5 → shared_h1] trunk. Tiny compute cost.

5. **The killer diagnostic:** During hyperopt, compare 2D bottleneck distributions
   between IS and OOS data. If they OVERLAP, the model learned regime-invariant
   features. If they DON'T overlap, the model is overfitting — visible BEFORE
   checking OOS Sharpe.

**Why 2 and not 4 or 8?** 2 is the minimum to express long/short/flat (need ≥2 bits
for 3+ actions). Any higher leaks memorization capacity. Start maximally constrained.

This attacks the ROOT CAUSE of the generalization gap — all 28 other techniques fight
symptoms. The bottleneck makes memorization structurally impossible at the architecture
level.

**Implementation steps:**

- [ ] **Step 1: Add bottleneck weights to network definition**

  In `branching.rs`, add:
  ```rust
  pub bottleneck_w: Tensor,  // [market_dim, bottleneck_dim] (e.g. [42, 2])
  pub bottleneck_b: Tensor,  // [bottleneck_dim]
  ```

- [ ] **Step 2: Add bottleneck GEMM to cuBLAS forward pass**

  In `batched_forward.rs`, before the first shared layer:
  ```
  // Bottleneck: [B, market_dim] → [B, bottleneck_dim] via GemmEx + tanh
  gemmex_bf16(W_bn, states, h_bn, bottleneck_dim, batch, market_dim)
  add_bias_tanh(h_bn, b_bn, bottleneck_dim, batch)

  // Concatenate: [B, bottleneck_dim + portfolio_dim] → input to h_s1
  concat_kernel(h_bn, portfolio_features, h_concat, bottleneck_dim, portfolio_dim, batch)
  ```

  The existing h_s1 GEMM changes from [state_dim → shared_h1] to [bottleneck_dim+3 → shared_h1].

- [ ] **Step 3: Add bottleneck to training backward pass**

  Chain rule through the bottleneck: d_loss/d_W_bn = d_loss/d_h_bn @ states.T.
  The existing backward pass already computes d_loss/d_states. Replace the first
  backward GEMM to route through the bottleneck.

- [ ] **Step 4: Add to experience collector forward pass**

  The experience collector has its own cuBLAS forward. Add the same bottleneck
  GEMM before the existing h_s1 computation.

- [ ] **Step 5: Add config field `bottleneck_dim`**

  ```rust
  pub bottleneck_dim: usize,  // 2 = maximum compression, 0 = disabled (bypass)
  ```
  Default: 2. When 0, skip the bottleneck (backward compatible).

- [ ] **Step 6: Add bottleneck activation logging**

  At epoch boundary, read the 2D bottleneck activations for a sample of training
  data. Log min/max/mean of each dimension. This provides free interpretability.

- [ ] **Step 7: Add IS/OOS distribution overlap metric**

  During walk-forward evaluation, compute the Wasserstein distance between IS and
  OOS bottleneck activation distributions. Low distance = good generalization.
  Log as `bottleneck_w_distance` metric.

- [ ] **Step 8: Compile + smoke test + commit**

  Verify that:
  - Bottleneck forward/backward produces non-zero gradients
  - Q-values are in normal range (not collapsed to constant)
  - 2D activations show non-trivial structure (not all same value)
  - Smoke tests pass with bottleneck_dim=2

---

### Task 32: Gradient Vaccine — Mathematically Impossible to Overfit [DONE]

**Phase:** 0 (HIGHEST PRIORITY) | **Difficulty:** Medium | **Impact:** TRANSFORMATIVE

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (gradient projection between forward and adam)
- Modify: `crates/ml/src/cuda_pipeline/dqn_utility_kernels.cu` (dot product + projection kernels)
- Modify: `crates/ml/src/trainers/dqn/config.rs` (vaccine config)

**The Core Idea:**

At each training step, compute gradients on both the training batch and a held-out
validation batch. Project out the component of the training gradient that
CONTRADICTS the validation gradient:

```
if dot(g_train, g_val) < 0:
    g_safe = g_train - proj(g_train, g_val)
           = g_train - (g_train · g_val / |g_val|²) * g_val
```

The model literally cannot learn anything that doesn't generalize.

**Why this is the pearl of pearls:**

1. **Mathematical guarantee.** Every gradient update that would overfit is surgically
   removed. The optimizer is constrained to move only in directions where BOTH
   training and validation agree.

2. **Complementary to the bottleneck (#31):**
   - Bottleneck constrains what the model can REPRESENT (100 bits, no memorization)
   - Vaccine constrains how the model LEARNS (only generalizing gradients survive)
   - Together they attack both capacity and optimization simultaneously
   - "You can only remember 2 numbers, and those 2 numbers must work on unseen data."

3. **Implementation in existing pipeline:**
   - Split each replay batch 80/20 (train/val)
   - Reuse the existing MSE+C51 dual-path infrastructure (already runs both)
   - Compute dot(g_train, g_val) on grad_buf — one reduction kernel
   - When dot < 0, project: g_safe = g_train - scale * g_val — one SAXPY
   - Two kernel launches between replay_forward() and replay_adam()

**Implementation steps:**

- [ ] **Step 1: Split batch for vaccine validation**

  In `upload_batch_gpu()`, split the batch: first 80% → training, last 20% → validation.
  Upload both to states_buf and a new `vaccine_states_buf`.

- [ ] **Step 2: Dual forward-backward in submit_forward_ops**

  After the normal forward+loss+grad (which fills grad_buf with g_train), launch a
  second pass on the vaccine batch:
  - Forward on vaccine states → vaccine logits
  - Loss on vaccine batch → vaccine_grad_buf (g_val)

- [ ] **Step 3: Add gradient projection kernels**

  New kernels in dqn_utility_kernels.cu:
  ```cuda
  // Reduction: dot = sum(g_train[i] * g_val[i])
  extern "C" __global__ void gradient_dot_product(
      const __nv_bfloat16* g_train, const __nv_bfloat16* g_val,
      float* dot_result, int n);

  // Projection: g_train -= (dot / norm_sq) * g_val  when dot < 0
  extern "C" __global__ void gradient_project(
      __nv_bfloat16* g_train, const __nv_bfloat16* g_val,
      const float* dot_result, const float* norm_sq, int n);
  ```

- [ ] **Step 4: Launch projection between forward and adam**

  In `submit_forward_ops()`, after the backward pass:
  ```rust
  // Compute dot(g_train, g_val)
  self.launch_gradient_dot_product()?;
  // If dot < 0, project out conflicting component
  self.launch_gradient_project()?;
  ```
  These run inside the CUDA Graph — the dot_result buffer contents change
  per step but the addresses are fixed.

- [ ] **Step 5: Add config**

  ```rust
  pub enable_gradient_vaccine: bool,  // default: true
  pub vaccine_val_fraction: f64,      // default: 0.2 (20% of batch is validation)
  ```

- [ ] **Step 6: Compile + test + commit**

---

## THE CROWN JEWELS — Four Layers of Defense

Together these form a complete anti-memorization fortress:

| Layer | Technique | Attacks | Level |
|-------|-----------|---------|-------|
| **Gem** | Temporal Causal Bottleneck (#31) | Can only remember 2 numbers | Architecture |
| **Pearl** | Gradient Vaccine (#32) | Those 2 numbers must work on unseen data | Optimization |
| **King** | Adversarial Self-Play (#33) | Past self exploits any memorized pattern | Strategic |
| **Emperor** | Causal Intervention (#34) | Only features that CAUSE outcomes, not correlate | Epistemic |

---

### Task 33: Adversarial Self-Play with Future Self (THE KING) [DONE]

**Phase:** 3 (Architecture) | **Difficulty:** Hard | **Impact:** TRANSFORMATIVE

**Files:**
- Create: `crates/ml/src/trainers/dqn/adversarial_self_play.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (saboteur controls microstructure)
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (checkpoint management + self-play cycle)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

**The Core Idea:**

Train TWO copies of the DQN against each other. The "trader" picks entries and exits.
The "saboteur" controls market microstructure (spread, slippage, fill probability,
latency) and tries to MAXIMIZE the trader's losses. The twist: the saboteur is a
PAST version of the trader (checkpoint from 50 epochs ago). It knows exactly what the
current trader USED to do — every pattern it exploited. It specifically attacks those.

The trader must evolve AWAY from its own past strategies to survive. It can't settle
into any fixed policy because its past self knows that policy and will destroy it.
This creates an evolutionary arms race where the only stable equilibrium is a policy
that works against ALL possible adversaries — including itself.

**Why this is the king:**

1. **No technique in RL trading literature does this.** Game theory meets self-play
   meets temporal evolution. AlphaGo's self-play but for market microstructure.

2. **Past-self adversary is the perfect opponent.** It has full knowledge of the
   trader's old strategies — correlation patterns, entry signals, regime preferences.
   It generates EXACTLY the market conditions that would have exploited those patterns.

3. **Evolutionary pressure toward robustness.** The only stable policy is one that
   works regardless of market microstructure — which IS generalization.

**Implementation:**

- [ ] **Step 1: Checkpoint manager** — save model weights every 50 epochs
- [ ] **Step 2: Saboteur network** — small MLP [state_dim → 32 → 3] outputting
      (spread_mult, fill_prob, slippage_mult). Initialized from past checkpoint.
- [ ] **Step 3: Self-play cycle** — alternating phases:
      - Epochs 1-50: normal training (build initial policy)
      - Epochs 51-55: saboteur trains against current trader (maximize losses)
      - Epochs 56-100: trader trains against frozen saboteur
      - Epoch 100: save new checkpoint, load as next saboteur
- [ ] **Step 4: Saboteur output → ExperienceCollectorConfig** — saboteur's output
      replaces fixed spread/fill/slippage values per epoch
- [ ] **Step 5: Compile + test + commit**

---

### Task 34: Causal Intervention Training (THE EMPEROR) [DONE]

**Phase:** 3 (Architecture) | **Difficulty:** Hard | **Impact:** TRANSFORMATIVE

**Files:**
- Create: `crates/ml/src/cuda_pipeline/causal_intervention_kernel.cu`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (intervention forward passes)
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs` (causal loss term)
- Modify: `crates/ml/src/trainers/dqn/config.rs`

**The Core Idea:**

Instead of learning correlations (feature X predicts profit), learn CAUSATION
(changing feature X CAUSES different outcomes). At each training step:

1. Take the current state batch [B, state_dim]
2. For each feature k, create an INTERVENED copy: set feature k to a counterfactual
   value (e.g. +1, 0, -1) while keeping everything else fixed
3. Run forward passes on the intervened states
4. Compare Q-values: if changing feature k doesn't change Q, it's noise

Add a causal regularization loss:
```
L_causal = -sum_k variance(Q(do(X_k = x)) for x in {-1, 0, +1})
```

This MAXIMIZES sensitivity to features that matter and MINIMIZES sensitivity
to noise. Pearl's do-calculus applied to RL.

**Why this is the emperor:**

1. **Correlations break OOS. Causation doesn't.** The only features that survive
   the causal intervention test are features that genuinely CAUSE different trading
   outcomes across all regimes. Spurious correlations (which cause overfitting)
   are mathematically eliminated.

2. **Nobody has done this in RL trading.** Causal inference in ML is growing,
   but applying do-calculus to Q-function feature sensitivity is novel.

3. **Synergy with bottleneck (#31).** The bottleneck compresses 14 features into 2
   dimensions. The causal intervention ensures those 2 dimensions capture features
   with genuine causal influence, not spurious correlations.

**Implementation:**

- [ ] **Step 1: Intervention kernel** — for each feature k (0..market_dim), create
      3 copies of the state batch with feature k set to {-1, 0, +1}. Total:
      3 * market_dim forward passes. With market_dim=14 (active features), that's
      42 extra forward passes per training step — ~10% overhead with cuBLAS.

- [ ] **Step 2: Compute per-feature causal sensitivity** — for each feature k,
      var(Q(do(k=-1)), Q(do(k=0)), Q(do(k=+1))). High variance = causal feature.

- [ ] **Step 3: Causal regularization loss** — add to total loss:
      `L_causal = -causal_weight * sum(sensitivities)`
      The negative sign MAXIMIZES sensitivity (model is rewarded for using causal features).

- [ ] **Step 4: Causal feature importance logging** — log per-feature sensitivity
      at epoch boundary. Provides interpretable feature ranking.

- [ ] **Step 5: Config**
      ```rust
      pub enable_causal_intervention: bool,  // default: false (expensive)
      pub causal_weight: f64,                // default: 0.1
      pub causal_intervention_values: Vec<f64>,  // default: [-1.0, 0.0, 1.0]
      ```

- [ ] **Step 6: Compile + test + commit**

---

## Complete Defense Architecture

```
                    ┌──────────────────────────────┐
                    │     EMPEROR: Causal           │
                    │  Only causal features survive │
                    │  (epistemic defense)          │
                    └──────────────┬───────────────┘
                                   │
                    ┌──────────────▼───────────────┐
                    │      KING: Self-Play          │
                    │  Past self exploits memorized │
                    │  patterns (strategic defense)  │
                    └──────────────┬───────────────┘
                                   │
                    ┌──────────────▼───────────────┐
                    │      PEARL: Vaccine           │
                    │  Only generalizing gradients  │
                    │  survive (optimization defense)│
                    └──────────────┬───────────────┘
                                   │
                    ┌──────────────▼───────────────┐
                    │       GEM: Bottleneck         │
                    │  2D information compression   │
                    │  (architecture defense)        │
                    └──────────────┬───────────────┘
                                   │
                    ┌──────────────▼───────────────┐
                    │      DQN Q-Network            │
                    │  Can only learn generalizable │
                    │  trading strategies            │
                    └──────────────────────────────┘
```
