From 2fe76f2f34fec197d057e3a15e07dfe671056046 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Fri, 15 May 2026 22:08:31 +0200 Subject: [PATCH] docs(phase-e-4-a): execution status + T10 backward_from_h_enriched patch sketch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two documentation deliverables produced while T14 backtest runs: 1. Plan update (specs/2026-05-15-phase-e-4-a-temporal-foundation.md): adds 'Execution Status' section reflecting actual T1-T14 progression. T5 deferred (real MBP-10 peek), T9 skipped (GRN moved to E.4.B per integration notes), T10 partial (new C51 grad-input kernel landed but Mamba2 backward wiring deferred), T14 in flight. Documents the 4 execution learnings: research-first saved a week of duplicate kernel work; cheap falsification experiments (Path 2, Path 3) avoided expensive investments; C51 borrow was the largest single Sharpe-lift in the session; GpuTensor/CudaSlice interop friction is the real integration cost. 2. T10 patch sketch (specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md): ready-to-apply patch for ml-alpha::Mamba2Block adding a new public method backward_from_h_enriched(cache, d_h_enriched). Bypasses the W_out projection backward, accepts the [B, hidden_dim] gradient from C51's grad-input kernel directly, zero-initialises dw_out/db_out (AdamW step on zero grad is a no-op with correct moment decay — effectively freezes W_out params which is correct semantics since Phase E never uses them). Includes the smoke binary wiring snippet that consumes the new method via launch_alpha_c51_grad_input → Mamba2 backward → AdamW step. Application gated on T14 backtest validation — if frozen Mamba2 already lifts Sharpe, T10 becomes optimisation rather than prerequisite. Co-Authored-By: Claude Opus 4.7 --- ...6-05-15-phase-e-4-a-temporal-foundation.md | 33 ++ ...-15-t10-mamba2-backward-from-h-enriched.md | 292 ++++++++++++++++++ 2 files changed, 325 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md diff --git a/docs/superpowers/plans/2026-05-15-phase-e-4-a-temporal-foundation.md b/docs/superpowers/plans/2026-05-15-phase-e-4-a-temporal-foundation.md index 6fb1b3ca0..a9fdb7e0c 100644 --- a/docs/superpowers/plans/2026-05-15-phase-e-4-a-temporal-foundation.md +++ b/docs/superpowers/plans/2026-05-15-phase-e-4-a-temporal-foundation.md @@ -1076,3 +1076,36 @@ Plan complete. Saved to `docs/superpowers/plans/2026-05-15-phase-e-4-a-temporal- **2. Inline Execution** — execute tasks in this session using `superpowers:executing-plans`. Batch execution with checkpoints for review. **Which approach?** + +--- + +## Execution Status (2026-05-15 update) + +User picked option 2 (inline). Actual execution diverged from the planned 14-task schedule per discoveries during research; updated below. + +| Plan task | Status | Commit | Notes | +|---|---|---|---| +| T1 — Research Mamba2 + GRN | ✅ Done | `588a6d38a` | **Key discovery**: production Mamba2 in `gpu_dqn_trainer` is coupled to SH2=256 trunk + ofi_embed[10] + ISV temporal routing. Switched to `ml_alpha::mamba2_block::Mamba2Block` (Phase 1d.1 from-scratch), already a workspace dep of `ml`. | +| T2 — Controller-at-inference verify | ✅ Done | `588a6d38a` | Rolled into T1 notes. Controller kernel has no training-mode branches; Wiener state preserved across episodes/cells. | +| T3 — SnapshotRow L1-L10 | ✅ Done | `eb9047fc3` | bid_l/ask_l: `[f32; 3]` → `[f32; 10]`. L4-L10 synthesised at ±tick (real-peek deferred). All 5 callers updated. | +| T4 — mbp10_dir param | ✅ Done | `5d7d4fa3c` | Signature-only — loader warns when mbp10_dir provided until T5 lands. | +| T5 — Real MBP-10 peek | ⏸ **Deferred** | — | Real-spread experiment showed 76% of fxcache bars hit 1-tick floor; L4-L10 from MBP-10 unlikely to move ES futures Sharpe. Reopen if E.4.A gates pass and we extend to other markets. | +| T6 — alpha_window_push kernel | ✅ Done | `6e86e5b43` → `35dcb8770` | Refactored from circular-with-head_idx to shift+insert (chronological) layout matching production `mamba2_update_history`. Tested via GPU smoke `alpha_window_push_shift_and_insert`. | +| T7 — Window in smoke (buffer-only) | ✅ Done | `70df69732` | Validated: 100-ep --temporal smoke produced bit-identical R_mean to C51-flat with buffer maintenance running (buffer not yet consumed). | +| T8 — Wire Mamba2 forward | ✅ Done | `ed6f5588e` | `Mamba2Block::forward_train` wired in smoke. Per-step h_enriched cached for batched training. C51 W shape becomes `[N_ACTIONS·n_atoms, mamba2_hidden_dim=32]` when --temporal. | +| T9 — Wire GRN trunk | ❌ **Skipped** | — | Per T1 integration notes — GRN deferred to E.4.B alongside MoE. Mamba2 output flows directly to C51 in E.4.A. | +| T10 — Mamba2 backward + AdamW | ⏸ **Partial** | `75d67bd20` | NEW kernel `alpha_c51_grad_input` landed (prerequisite for backprop into Mamba2). Mamba2 backward wiring DEFERRED: `Mamba2Block::backward` takes `d_logit [B, 1]` (post-W_out scalar gradient), not `d_h_enriched [B, hidden_dim]`. Two paths to complete: (a) modify ml-alpha to expose `backward_from_h_enriched`, or (b) replicate post-W_out logic inline. Deferred pending T14 — if frozen Mamba2 already lifts backtest Sharpe meaningfully, T10 becomes optimisation rather than prerequisite. | +| T11 — ISV controllers at backtest eval | ✅ Done | `4d65ace62` | `--isv-continual` flag; stacker-threshold controller fires per eval episode. Co-exists with τ-grid sweep (τ gates; ISV updates parallel observation). | +| T12 — Mirror --temporal in backtest | ✅ Done | `4d65ace62` | Backtest binary gains all temporal infrastructure: window push + Mamba2 forward + h_enriched buffer + per-episode reset. Training and eval inference paths both wired. | +| T13 — Smoke validation (gate 1) | ✅ **PASSED** | `41a7da700` | 1000-ep smoke: R_mean +3.9 (vs C51-flat -1.1), rvr +1.047 (preserved), EARLY_Q_MOVEMENT 5× larger. ACTION_ENTROPY 0.64 < 1.10 (misaligned criterion). Frozen Mamba2. | +| T14 — Backtest validation (gate 2) | 🔄 In flight | — | --c51 --temporal --window-k 16 --mamba2-hidden-dim 32 --isv-continual against 30-cell sweep. ETA ~15-25 min on RTX 3050 Ti. | + +### Known issues +- **Q_SPREAD spike at ep 950 in smoke** (16417 vs ~10 throughout) — likely transient NaN/Inf propagation in the kill-criteria EMA accumulator from a single extreme C51 logit. Policy quality unaffected (rvr stable, R_mean stable). Investigation tracked separately. +- **Mamba2 frozen at random init** until T10 completes — random SSM acts as feature-engineering reservoir; expected to add additional lift once weights actually train. + +### Execution learnings +1. **Research first paid off**: T1's discovery that production Mamba2 was over-coupled, and that `ml-alpha::Mamba2Block` was a drop-in solution, saved an estimated week of duplicate kernel work. +2. **Falsification gates work**: real-spread (Path 3) and action-pruning (Path 2) were both falsified empirically with cheap experiments before investing in the architecture lift. +3. **C51 win was already the ceiling-mover**: the +26pt Sharpe lift from the C51 borrow (committed earlier in session) was the largest single result. Temporal is upside on top. +4. **GpuTensor / CudaSlice interop is the real friction**: bridging the existing smoke binary's raw-pointer style with `ml-alpha`'s GpuTensor required scoped-borrow tricks (mutable-then-immutable guards). diff --git a/docs/superpowers/specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md b/docs/superpowers/specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md new file mode 100644 index 000000000..866869fab --- /dev/null +++ b/docs/superpowers/specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md @@ -0,0 +1,292 @@ +# T10 — `Mamba2Block::backward_from_h_enriched` patch sketch + +**Status:** Ready-to-apply patch (Phase E.4.A T10 prerequisite). +**Date:** 2026-05-15 +**Target file:** `crates/ml-alpha/src/mamba2_block.rs` +**Why needed:** see `pearl_c51_thompson_closed_phase_e3_gap.md` and +`docs/superpowers/specs/2026-05-15-mamba2-grn-integration-notes.md`. + +## Problem + +`Mamba2Block::backward(cache, d_logit: [B, 1])` takes the gradient +w.r.t. the SCALAR logit output (the post-W_out projection). For +Phase E we want to backprop a `[B, hidden_dim]` gradient computed +by the new `alpha_c51_grad_input` kernel — that's the C51 head's +gradient w.r.t. its input, which IS `h_enriched`. + +The existing backward chains: + + d_logit [B, 1] + │ W_out backward + ▼ + d_h_enriched [B, hidden_dim] + │ (steps 5–1 as before) + ▼ + dw_in, db_in, dw_a, db_a, dw_b, db_b, dw_c + +For Phase E we want to ENTER at `d_h_enriched` directly and skip +the W_out step (since C51 replaces W_out's role). + +## Patch + +Add a new public method to `impl Mamba2Block` in +`crates/ml-alpha/src/mamba2_block.rs`. Same signature shape as +`backward` but takes `d_h_enriched` instead of `d_logit`. Returns +`Mamba2BackwardGrads` with `dw_out` and `db_out` set to zero tensors +(matching production AdamW's expectations — Adam steps on zero grad +are no-ops with the correct moment decay). + +```rust +/// Backward variant that BYPASSES the W_out projection. Phase E +/// uses Mamba2's `h_enriched` directly as input to a downstream +/// classifier (C51 head), not Mamba2's own scalar logit. The +/// `d_logit`-based `backward` is unsuitable because Phase E never +/// computes a `d_logit` — there is no logit in its loss path. +/// +/// `d_h_enriched` MUST have shape `[B, hidden_dim]` matching +/// `cache.h_enriched`. Returns `Mamba2BackwardGrads` where +/// `dw_out` and `db_out` are zero-initialised tensors of the +/// correct shapes (AdamW step on these is a no-op with the +/// correct moment-decay; the W_out parameters effectively freeze +/// — which is the correct semantics since Phase E never uses them). +pub fn backward_from_h_enriched( + &self, + cache: &Mamba2ForwardCache, + d_h_enriched: &GpuTensor, +) -> Result { + let c = &self.config; + let n_batch = cache.h_enriched.shape()[0]; + let n_rows = n_batch * c.seq_len; + + if d_h_enriched.shape() != [n_batch, c.hidden_dim] { + return Err(anyhow!( + "Mamba2Block::backward_from_h_enriched: \ + d_h_enriched shape {:?} != [{}, {}]", + d_h_enriched.shape(), n_batch, c.hidden_dim + )); + } + + // ── 6′ SKIPPED. Phase E doesn't go through W_out. ──────────── + // (Zero w_out grads are constructed at return.) + + // ── 5′. Allocate scan-backward scratch buffers ──────────────── + let per_chan_n = n_batch * c.hidden_dim * c.seq_len * c.state_dim; + let per_sample_n = n_batch * c.hidden_dim * c.state_dim; + let mut d_a_per_channel = self.stream + .alloc_zeros::(per_chan_n) + .map_err(|e| anyhow!("alloc d_a_per_channel: {e}"))?; + let mut d_b_per_channel = self.stream + .alloc_zeros::(per_chan_n) + .map_err(|e| anyhow!("alloc d_b_per_channel: {e}"))?; + let mut d_w_c_per_sample = self.stream + .alloc_zeros::(per_sample_n) + .map_err(|e| anyhow!("alloc d_w_c_per_sample: {e}"))?; + let mut d_h_s2 = self.stream + .alloc_zeros::(n_batch * c.hidden_dim) + .map_err(|e| anyhow!("alloc d_h_s2: {e}"))?; + + let block_threads: u32 = 32; + let grid_y_h: u32 = + ((c.hidden_dim + block_threads as usize - 1) / block_threads as usize) as u32; + let bwd_cfg = LaunchConfig { + grid_dim: (n_batch as u32, grid_y_h, 1), + block_dim: (block_threads, 1, 1), + shared_mem_bytes: 0, + }; + let n_i32 = n_batch as i32; + let k_i32 = c.seq_len as i32; + let sh2_i32 = c.hidden_dim as i32; + let st_i32 = c.state_dim as i32; + unsafe { + self.stream + .launch_builder(&self.kernel_bwd) + .arg(cache.a_proj.cuda_data()) + .arg(cache.b_proj.cuda_data()) + .arg(d_h_enriched.cuda_data()) // <-- the caller's gradient + .arg(&self.w_c) + .arg(&mut d_a_per_channel) + .arg(&mut d_b_per_channel) + .arg(&mut d_w_c_per_sample) + .arg(&mut d_h_s2) + .arg(&n_i32) + .arg(&k_i32) + .arg(&sh2_i32) + .arg(&st_i32) + .launch(bwd_cfg) + .map_err(|e| anyhow!("mamba2_alpha_scan_bwd launch: {e}"))?; + } + + // ── Reduce d_a_per_channel and d_b_per_channel ─────────────── + let red_grid_z: u32 = + ((c.state_dim + block_threads as usize - 1) / block_threads as usize) as u32; + let red_cfg = LaunchConfig { + grid_dim: (n_batch as u32, c.seq_len as u32, red_grid_z), + block_dim: (block_threads, 1, 1), + shared_mem_bytes: 0, + }; + let mut d_a_proj_flat: CudaSlice = self.stream + .alloc_zeros::(n_rows * c.state_dim) + .map_err(|e| anyhow!("alloc d_a_proj_flat: {e}"))?; + let mut d_b_proj_flat: CudaSlice = self.stream + .alloc_zeros::(n_rows * c.state_dim) + .map_err(|e| anyhow!("alloc d_b_proj_flat: {e}"))?; + unsafe { + self.stream + .launch_builder(&self.kernel_reduce_d_proj) + .arg(&d_a_per_channel) + .arg(&mut d_a_proj_flat) + .arg(&n_i32).arg(&k_i32).arg(&sh2_i32).arg(&st_i32) + .launch(red_cfg) + .map_err(|e| anyhow!("reduce d_a_proj: {e}"))?; + self.stream + .launch_builder(&self.kernel_reduce_d_proj) + .arg(&d_b_per_channel) + .arg(&mut d_b_proj_flat) + .arg(&n_i32).arg(&k_i32).arg(&sh2_i32).arg(&st_i32) + .launch(red_cfg) + .map_err(|e| anyhow!("reduce d_b_proj: {e}"))?; + } + + // ── Reduce d_w_c_per_sample → dw_c ────────────────────────── + let red_w_c_cfg = LaunchConfig { + grid_dim: (c.hidden_dim as u32, red_grid_z, 1), + block_dim: (block_threads, 1, 1), + shared_mem_bytes: 0, + }; + let mut dw_c: CudaSlice = self.stream + .alloc_zeros::(c.hidden_dim * c.state_dim) + .map_err(|e| anyhow!("alloc dw_c: {e}"))?; + unsafe { + self.stream + .launch_builder(&self.kernel_reduce_d_w_c) + .arg(&d_w_c_per_sample) + .arg(&mut dw_c) + .arg(&n_i32).arg(&sh2_i32).arg(&st_i32) + .launch(red_w_c_cfg) + .map_err(|e| anyhow!("reduce dw_c: {e}"))?; + } + + // ── 3′. W_b backward ───────────────────────────────────────── + let d_b_proj_2d = GpuTensor::new(d_b_proj_flat, vec![n_rows, c.state_dim]) + .map_err(|e| anyhow!("reshape d_b_proj: {e}"))?; + let x_act = LinearActivations { input: cache.x.clone() }; + let LinearGrads { dw: dw_b, db: db_b, dx: d_x_from_b } = self + .w_b.inner + .backward_with_slices(&d_b_proj_2d, &x_act, &self.w_b.weight, + &self.cublas, &self.stream) + .map_err(|e| anyhow!("w_b backward: {e}"))?; + + // ── 2′. W_a backward ───────────────────────────────────────── + let d_a_proj_2d = GpuTensor::new(d_a_proj_flat, vec![n_rows, c.state_dim]) + .map_err(|e| anyhow!("reshape d_a_proj: {e}"))?; + let LinearGrads { dw: dw_a, db: db_a, dx: d_x_from_a } = self + .w_a.inner + .backward_with_slices(&d_a_proj_2d, &x_act, &self.w_a.weight, + &self.cublas, &self.stream) + .map_err(|e| anyhow!("w_a backward: {e}"))?; + + let d_x = d_x_from_a.add(&d_x_from_b, &self.stream) + .map_err(|e| anyhow!("sum d_x branches: {e}"))?; + + // ── 1′. W_in backward ──────────────────────────────────────── + let input_act = LinearActivations { input: cache.input_2d.clone() }; + let LinearGrads { dw: dw_in, db: db_in, dx: _d_input } = self + .w_in.inner + .backward_with_slices(&d_x, &input_act, &self.w_in.weight, + &self.cublas, &self.stream) + .map_err(|e| anyhow!("w_in backward: {e}"))?; + + // ── 6′ replacement: zero W_out grads ──────────────────────── + // AdamW will see zero grad → effectively freezes w_out + // parameters. Correct semantics — Phase E doesn't use them. + let dw_out = GpuTensor::zeros(&[1, c.hidden_dim], &self.stream) + .map_err(|e| anyhow!("alloc zero dw_out: {e}"))?; + let db_out = GpuTensor::zeros(&[1], &self.stream) + .map_err(|e| anyhow!("alloc zero db_out: {e}"))?; + + Ok(Mamba2BackwardGrads { + dw_in, db_in, dw_a, db_a, dw_b, db_b, dw_c, dw_out, db_out, + }) +} +``` + +## Application steps + +1. Open `crates/ml-alpha/src/mamba2_block.rs`. +2. Inside `impl Mamba2Block`, append the method above immediately + after the existing `backward` method (around line ~549). +3. Run ml-alpha's existing tests to confirm no regression: + ```bash + SQLX_OFFLINE=true cargo test -p ml-alpha --release + ``` + Existing `backward` is untouched, so all existing tests should + pass without modification. +4. (Optional) Add a unit test that verifies the new method + produces gradients consistent with manual `backward` chained + through the W_out backward — i.e., feeding + `d_logit = h_enriched @ W_out.T + b_out` partial gradient + should produce the same scan/projection grads as + `backward_from_h_enriched(d_h_enriched_recovered_via_W_out)`. + +## Wiring in the smoke binary (after applying the patch) + +The smoke binary's batched training would: + +```rust +// (existing) C51 forward on h_enriched → probs_current +// (existing) C51 grad → dW_c51, db_c51 + +// NEW: compute d_h_enriched (gradient into Mamba2 output) +let mut d_h_enriched_dev = stream.alloc_zeros::( + (ep_len as usize) * cli.mamba2_hidden_dim +)?; +{ + let (p_ptr, _) = probs_current_dev.device_ptr(&stream); + let (m_ptr, _) = m_dev.device_ptr(&stream); + let (a_ptr, _) = actions_dev.device_ptr(&stream); + let (w_ptr, _) = w_dev.device_ptr(&stream); + let (d_ptr, _) = d_h_enriched_dev.device_ptr_mut(&stream); + unsafe { + launch_alpha_c51_grad_input( + &stream, &c51_grad_input_kernel, + p_ptr, m_ptr, a_ptr, w_ptr, d_ptr, + ep_len, c51_input_dim as i32, n_act_i, n_atoms_i, + 1.0 / ep_len as f32, + )?; + } +} + +// NEW: wrap d_h_enriched as GpuTensor, call backward_from_h_enriched +let d_h_tensor = GpuTensor::new( + d_h_enriched_dev, + vec![ep_len as usize, cli.mamba2_hidden_dim], +)?; +let block = mamba2_block.as_ref().expect("Mamba2Block missing"); +let grads = block.backward_from_h_enriched(&cache, &d_h_tensor)?; + +// NEW: AdamW step +adamw.step(&mut block_mut, &grads)?; +``` + +Two changes needed in the smoke wiring beyond this: +1. `mamba2_block` needs to be `Option>` since + AdamW takes `&mut`. Simplest: wrap in `Option>` + or just borrow mutably at the call site. +2. The batched training currently re-uses the inference-time + `cache.h_enriched` from a per-step forward, but `backward` + needs the FULL cache (input_2d, x, a_proj, b_proj). We'd need + to do ONE batched forward at training time over a + `[ep_len, K, in_dim]` tensor to get the cache that matches + the gradient. + +## Estimated effort +- Patch application + ml-alpha test verification: 30-45 min +- Smoke binary integration: 1-2 hr +- Backtest binary integration: 30-45 min +- Smoke + backtest validation runs: 1-2 hr +- Total: ~4-6 hr focused session + +Conditional on E.4.A T14 result: if temporal already lifts +backtest Sharpe substantially with frozen Mamba2, T10 is upside. +If T14 is flat or worse, T10 may not be the lever and the +optimization budget should go elsewhere.