spec: mega-graph CUDA pipeline — single graph launch per step

Target: 6.4 sps → 50+ sps by capturing the entire per-step pipeline
into ONE cuGraphLaunch. Four prerequisite phases: lobsim raw_launch,
LR controller to GPU kernel, PER to main stream, then mega-capture.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

docs/superpowers/specs/2026-05-27-mega-graph-pipeline.md

@@ -0,0 +1,129 @@
+# Mega-Graph CUDA Pipeline
+
+**Date**: 2026-05-27  
+**Status**: Draft  
+**Goal**: Capture the entire per-step training pipeline into one CUDA graph launch, eliminating ~150 individual kernel launches and ~140ms of host-side overhead. Target: 50+ sps (from 6.4).
+
+## Problem
+
+nsys profiling shows GPU utilization at 8.5% (13ms kernel time per 156ms step). The host spends 143ms between kernel launches on: Rust arg assembly, Result unwrapping, conditional logic, 5 graph launches + ~20 individual raw_launch calls. The CUDA APIs themselves are fast (<1ms total); the overhead is Rust code between them.
+
+## Current Per-Step Pipeline
+
+```
+HOST                              GPU (13ms total)
+─────────────────────────────     ────────────────────────────
+build 3 gather args (50μs)   →   sample_and_gather (13ms)
+                                  gather_next (0.04ms)
+                                  gather_current (0.04ms)
+graph_launch prefill (50μs)  →   GRAPH A: encoder + Q/V/π + actions (~2ms)
+build 4 gate args (100μs)    →   conf_gate + frd_gate + action_ent + log_pi (0.1ms)
+lobsim.apply_snapshot (20μs) →   book_update (0.01ms)
+graph_launch postfill (50μs) →   GRAPH A2: risk + trail + actions_to_market (~1ms)
+lobsim.step_fill (20μs)      →   fill + pnl_track (0.05ms)
+graph_launch reward (50μs)   →   GRAPH C: reward + scale + clamp + context (~2ms)
+build 2 PER args (20μs)      →   per_push_ring + per_push_flush (0.05ms)
+perception.step_batched (10μs)→  PERCEPTION GRAPH: encoder train (~2ms)
+graph_launch training (50μs) →   GRAPH D: Q/π/V backward + Adam (~3ms)
+build 4 target args (50μs)   →   soft_update + q_divergence (0.1ms)
+DiagFrame memcpy (100μs)     →   (async, background thread)
+───────────────────────────
+Total host: ~570μs arg work
+But Rust overhead: ~155ms (!)
+```
+
+The 570μs of pure arg work can't explain 155ms. The remaining ~154ms is:
+- `step_synthetic()` → reads ISV from mapped-pinned → launches LR controller → builds training graph args → launches GRAPH D → reads ISV for LR plateau
+- `step_with_lobsim_reward_and_train()` → builds reward graph args → conditional K-loop → event-based cross-stream sync → PER sample on train_stream
+
+Each of these involves deep Rust function call chains (integrated.rs is 8400 lines) with Result unwrapping, debug_asserts, conditional branches, and method dispatch through trait objects.
+
+## Approaches
+
+### Approach A: Mega-Graph (Recommended)
+
+Capture ALL kernels from a single step into ONE graph during warmup. On subsequent steps, launch the mega-graph with a single `cuGraphLaunch` call. Zero host work between kernels.
+
+**Requirements**:
+1. ALL kernel arguments use stable device pointers (pre-allocated at init)
+2. No host-side conditional logic during the captured section
+3. No memory allocation/deallocation during capture
+4. K-loop fixed at K=1 (current config)
+5. cuBLAS workspace pre-allocated (already done)
+
+**What changes per step** (must be handled):
+- `sample_and_gather` uses per-step PRNG state → device-side, graph-safe
+- Perception's `ts_ns` value → write to mapped-pinned staging, kernel reads from stable pointer
+- ISV bus → modified by controller kernels in-place, graph-safe
+- LR controller → reads host-side loss values → **BLOCKER**: needs mapped-pinned reads
+
+**LR controller blocker**: The host reads `last_q_loss`, `last_pi_loss`, `last_v_loss` from mapped-pinned memory between the reward graph and the training graph. These feed the per-head LR scaling. Fix: move LR controller to a GPU kernel that reads from mapped-pinned device pointers directly. The host doesn't need to see the LR values — the Adam kernel reads LR from ISV.
+
+**Lobsim blocker**: `apply_snapshot_from_device` and `step_fill_from_market_targets` are lobsim methods with internal state. These need to be either: (a) converted to raw_launch with cached pointers, or (b) kept outside the mega-graph as a "lobsim mini-graph."
+
+**Estimated result**: 1 graph launch per step. Host does: graph launch (50μs) + DiagFrame memcpy (100μs) = 150μs/step → **~6600 sps** (GPU-bound at 13ms/step → ~77 sps with GPU saturation).
+
+### Approach B: Coalesce Existing Graphs
+
+Merge the 4 existing graphs + loose kernels into 2 larger graphs: "env_graph" (A + gates + A2 + fill + C) and "train_graph" (PER + perception + D + target).
+
+**Simpler**: doesn't require solving the LR controller blocker or lobsim abstraction. Just expand the existing capture boundaries.
+
+**Host work**: 2 graph launches + cross-stream event + DiagFrame = ~300μs. But the Rust overhead between the two launches is still significant (K-loop, LR reads, etc.).
+
+**Estimated result**: ~20-30 sps (2× improvement, not 10×).
+
+### Approach C: Async Host Pipeline
+
+Keep the current graph structure but pipeline: host launches step N+1's gathers while step N's training graph runs. Double-buffer all state.
+
+**Complex**: requires double-buffering 50+ device buffers. Weight updates from step N must be visible to step N+1's forward. Correctness is hard to verify.
+
+**Estimated result**: ~12 sps (2× from pipelining).
+
+## Recommendation: Approach A (Mega-Graph)
+
+The only approach that reaches 50+ sps. The blockers are solvable:
+
+1. **LR controller → GPU kernel**: already ISV-driven; just move the host-side `AdamW.lr` mutation to a kernel that writes the LR value into the Adam kernel's arg buffer. ~30 lines.
+
+2. **Lobsim → raw_launch**: `apply_snapshot_from_device` is 2 DtoD copies + 1 kernel. `step_fill_from_market_targets` is 2 kernels. Convert both to raw_launch with cached pointers. ~100 lines.
+
+3. **K-loop fixed at K=1**: already the case in production config. The mega-graph captures one iteration.
+
+4. **Cross-stream PER**: PER sample runs on train_stream, but at K=1 it's just 1 sample per step. Can be moved to the main stream (the separate stream was for K>1 pipelining).
+
+## Implementation Phases
+
+### Phase 1: Convert lobsim methods to raw_launch (prerequisite)
+- `apply_snapshot_from_device` → 2 DtoD + 1 kernel via raw_launch
+- `step_fill_from_market_targets` → 2 kernels via raw_launch
+- Cache all lobsim internal pointers in LobPtrs
+
+### Phase 2: Move LR controller to GPU kernel
+- New `rl_lr_from_mapped_pinned.cu` kernel reads loss from mapped-pinned device pointers
+- Writes per-head LR to ISV slots
+- Adam kernel reads LR from ISV (already supported)
+
+### Phase 3: Move PER sample to main stream
+- At K=1, PER sample + priority_update + tree_rebuild run once per step
+- Move from train_stream to self.stream
+- Eliminate cross-stream events
+
+### Phase 4: Mega-graph capture
+- Warmup step 0: eager execution (all kernels launch individually)
+- Warmup step 1: `begin_capture` → entire pipeline → `end_capture`
+- Step 2+: single `cuGraphLaunch` per step
+- Host per-step: ts_ns write to mapped-pinned + graph launch + DiagFrame
+
+### Phase 5: Validation
+- nsys: confirm 1 cuGraphLaunch per step, GPU utilization >80%
+- Functional: wr/pnl match baseline within 1%
+- Smoke: 1k steps local, 5k steps L40S
+
+## Anti-patterns to avoid
+
+- No `cuGraphExecUpdate` (overkill — our topology is fixed)
+- No conditional graph nodes (CUDA 12.4 feature, complex, fragile)
+- No multi-stream graphs (correctness nightmare)
+- No host reads inside the captured section (use mapped-pinned reads OUTSIDE or defer)