diff --git a/docs/superpowers/plans/2026-03-30-gems-and-pearls-generalization.md b/docs/superpowers/plans/2026-03-30-gems-and-pearls-generalization.md index 314dc40a2..26e01774b 100644 --- a/docs/superpowers/plans/2026-03-30-gems-and-pearls-generalization.md +++ b/docs/superpowers/plans/2026-03-30-gems-and-pearls-generalization.md @@ -726,3 +726,117 @@ should use f32 copies to eliminate bf16 quantization artifacts during data colle - [ ] Update CUDA kernels to read f32 weights for forward pass during experience collection - [ ] Verify Q-value precision improvement in smoke tests - [ ] Compile + test + commit + +--- + +## THE GEM OF GEMS + +### Task 31: Temporal Causal Bottleneck — 2D Information Compression [PLAN] + +**Phase:** 0 (HIGHEST PRIORITY) | **Difficulty:** Medium | **Impact:** TRANSFORMATIVE + +**Files:** +- Modify: `crates/ml/src/cuda_pipeline/batched_forward.rs` (add bottleneck GEMM layer) +- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (bottleneck weights, backward pass) +- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (bottleneck in experience forward) +- Modify: `crates/ml-dqn/src/branching.rs` (add bottleneck layer to network definition) +- Modify: `crates/ml/src/trainers/dqn/config.rs` (bottleneck_dim config) + +**The Core Idea:** + +Force ALL information from the 14-feature market state through a 2-dimensional +bottleneck before it reaches the Q-network. Two neurons. The entire market must be +compressed into 2 numbers — call them **opportunity** and **risk**. + +**Why this is the gem of all gems:** + +1. **Information-theoretic impossibility of memorization.** With only 2 floats, the + network physically CANNOT encode "which specific price sequence am I looking at." + A 2D bottleneck has ~100 bits of capacity vs ~450 bits in the full 14-feature vector. + Memorizing 100K episodes requires far more than 100 bits. + +2. **Forced abstraction.** The bottleneck encoder must learn to compress 14 features + into the 2 most DECISION-RELEVANT dimensions. These are by definition + regime-invariant — if they weren't, the Q-network couldn't use them to trade + profitably across regimes. + +3. **Interpretability for free.** Plot the 2D bottleneck activations and literally + see what the model thinks "opportunity" and "risk" look like. Immediate diagnostic + for whether it learned signal or garbage. + +4. **Architecture integration:** Add a single [market_dim → 2] linear + tanh layer + before h_s1. One extra GEMM in the cuBLAS forward pass. The 2D output concatenates + with 3 portfolio features → [5 → shared_h1] trunk. Tiny compute cost. + +5. **The killer diagnostic:** During hyperopt, compare 2D bottleneck distributions + between IS and OOS data. If they OVERLAP, the model learned regime-invariant + features. If they DON'T overlap, the model is overfitting — visible BEFORE + checking OOS Sharpe. + +**Why 2 and not 4 or 8?** 2 is the minimum to express long/short/flat (need ≥2 bits +for 3+ actions). Any higher leaks memorization capacity. Start maximally constrained. + +This attacks the ROOT CAUSE of the generalization gap — all 28 other techniques fight +symptoms. The bottleneck makes memorization structurally impossible at the architecture +level. + +**Implementation steps:** + +- [ ] **Step 1: Add bottleneck weights to network definition** + + In `branching.rs`, add: + ```rust + pub bottleneck_w: Tensor, // [market_dim, bottleneck_dim] (e.g. [42, 2]) + pub bottleneck_b: Tensor, // [bottleneck_dim] + ``` + +- [ ] **Step 2: Add bottleneck GEMM to cuBLAS forward pass** + + In `batched_forward.rs`, before the first shared layer: + ``` + // Bottleneck: [B, market_dim] → [B, bottleneck_dim] via GemmEx + tanh + gemmex_bf16(W_bn, states, h_bn, bottleneck_dim, batch, market_dim) + add_bias_tanh(h_bn, b_bn, bottleneck_dim, batch) + + // Concatenate: [B, bottleneck_dim + portfolio_dim] → input to h_s1 + concat_kernel(h_bn, portfolio_features, h_concat, bottleneck_dim, portfolio_dim, batch) + ``` + + The existing h_s1 GEMM changes from [state_dim → shared_h1] to [bottleneck_dim+3 → shared_h1]. + +- [ ] **Step 3: Add bottleneck to training backward pass** + + Chain rule through the bottleneck: d_loss/d_W_bn = d_loss/d_h_bn @ states.T. + The existing backward pass already computes d_loss/d_states. Replace the first + backward GEMM to route through the bottleneck. + +- [ ] **Step 4: Add to experience collector forward pass** + + The experience collector has its own cuBLAS forward. Add the same bottleneck + GEMM before the existing h_s1 computation. + +- [ ] **Step 5: Add config field `bottleneck_dim`** + + ```rust + pub bottleneck_dim: usize, // 2 = maximum compression, 0 = disabled (bypass) + ``` + Default: 2. When 0, skip the bottleneck (backward compatible). + +- [ ] **Step 6: Add bottleneck activation logging** + + At epoch boundary, read the 2D bottleneck activations for a sample of training + data. Log min/max/mean of each dimension. This provides free interpretability. + +- [ ] **Step 7: Add IS/OOS distribution overlap metric** + + During walk-forward evaluation, compute the Wasserstein distance between IS and + OOS bottleneck activation distributions. Low distance = good generalization. + Log as `bottleneck_w_distance` metric. + +- [ ] **Step 8: Compile + smoke test + commit** + + Verify that: + - Bottleneck forward/backward produces non-zero gradients + - Q-values are in normal range (not collapsed to constant) + - 2D activations show non-trivial structure (not all same value) + - Smoke tests pass with bottleneck_dim=2