docs: IQL H100 hardening spec — drift readiness, p5 floor, buffer-relative staleness

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-04-13-iql-h100-hardening-design.md

@@ -0,0 +1,131 @@
+# IQL H100 Hardening Design
+
+## Goal
+
+Fix three production concerns before H100 baseline: readiness activates too fast on large batches, per-sample C51 support drops samples silently, staleness decay is ineffective at scale. All fixes self-calibrating, zero new hyperparameters.
+
+## Fix 1: V(s) Inter-Batch Drift Readiness
+
+Replace CV-based readiness (`(cv_initial - cv_current) / cv_initial`) with two-timescale EMA drift detection on V(s) mean.
+
+### Kernel changes (`iql_adv_sigma_ema_update`)
+
+The kernel currently computes advantage sigma EMA + CV readiness. Replace the readiness computation:
+
+```
+// Compute v_mean from v_out_buf (grid-stride loop in same single-thread kernel)
+v_mean = sum(v_out_buf[0..B]) / B
+
+// Two-timescale EMA
+v_ema_fast = 0.9 * v_ema_fast + 0.1 * v_mean
+v_ema_slow = 0.99 * v_ema_slow + 0.01 * v_mean
+
+// Relative drift
+drift = |v_ema_fast - v_ema_slow| / max(|v_ema_slow|, 1e-4)
+
+// Self-calibrating scale: capture first drift after 10-step warmup
+if (step_counter < 10) { readiness = 0; step_counter++; }
+else if (drift_scale < 1e-8) { drift_scale = drift; readiness = 0; }
+else { readiness = 1 - clamp(drift / drift_scale, 0, 1); }
+```
+
+### Buffer changes
+
+Expand `readiness_buf` from `[2]` to `[6]`:
+- `[0]` = readiness scalar
+- `[1]` = drift_scale (self-calibrated)
+- `[2]` = v_ema_fast
+- `[3]` = v_ema_slow
+- `[4]` = step_counter (as float, cast to int in kernel)
+- `[5]` = reserved
+
+### Kernel signature change
+
+Add `const float* __restrict__ v_out_buf` and `int batch_size` parameters to `iql_adv_sigma_ema_update` so it can compute v_mean internally.
+
+### Dead code
+
+The CV-based readiness in the current kernel (`cv = sigma_ema / |mean|`) is replaced entirely. The `readiness_buf[1]` which stored `initial_cv` is repurposed for `drift_scale`.
+
+## Fix 2: Frugal p5 Quantile Floor for C51 Support
+
+### New kernel: `iql_support_floor`
+
+Single-thread kernel (grid=1, block=1) that runs after `iql_compute_per_sample_support`:
+
+```
+// Step 1: Frugal p5 estimator update
+// Read half_widths from per_sample_support (recoverable as (v_max - v_min) / 2)
+for b in 0..batch_size:
+    hw = (per_sample_support[b*3+1] - per_sample_support[b*3+0]) / 2
+    if hw < p5_estimate:
+        p5_estimate -= p5_estimate * 0.05  // proportional step down
+    else:
+        p5_estimate += p5_estimate * 0.05 * (0.05 / 0.95)  // asymmetric for p5
+
+// Step 2: Floor delta_z for all samples
+floor_hw = p5_estimate * 0.1
+floor_dz = (2 * floor_hw) / (num_atoms - 1)
+for b in 0..batch_size:
+    if per_sample_support[b*3+2] < floor_dz:
+        // Recompute v_min/v_max/delta_z with floored half_width
+        center = (per_sample_support[b*3+0] + per_sample_support[b*3+1]) / 2
+        per_sample_support[b*3+0] = center - floor_hw
+        per_sample_support[b*3+1] = center + floor_hw
+        per_sample_support[b*3+2] = floor_dz
+```
+
+### Buffer changes
+
+- New: `p5_ema_buf: CudaSlice<f32>` size 1, initialized to 1.0 (safe default)
+- New kernel function loaded from cubin: `iql_support_floor`
+
+### Integration
+
+Called from `submit_aux_ops` immediately after `compute_per_sample_support`, before `compute_branch_scales`.
+
+## Fix 3: Buffer-Relative Staleness
+
+### Kernel change (`iql_modulate_td_errors`)
+
+Replace:
+```cuda
+float decay = expf(-staleness_lambda * (float)age / fmaxf(staleness_tau, 1.0f));
+```
+With:
+```cuda
+float age_norm = (float)age / fmaxf((float)capacity, 1.0f);
+float decay = expf(-3.0f * age_norm);
+```
+
+Remove `staleness_lambda` and `staleness_tau` parameters from kernel signature. Remove `float beta` parameter (the advantage temperature is already baked into the advantage weights by `iql_compute_advantage_weights`).
+
+Wait — `beta` is used for computing K (dynamic clamp). Keep `beta`. Only remove `staleness_lambda` and `staleness_tau`.
+
+### Rust changes
+
+- Remove `staleness_lambda` and `staleness_tau` from `GpuIqlConfig`
+- Remove corresponding `.arg()` calls in `modulate_td_errors()` launcher
+- Remove from `GpuIqlConfig::default()`
+- Remove from `FusedTrainingCtx::new` where they're set to `0.001` and `10000.0`
+
+### Dead code removed
+
+- `GpuIqlConfig::staleness_lambda` field
+- `GpuIqlConfig::staleness_tau` field
+- Both default values
+- Both `.arg()` calls in the Rust launcher
+
+## Files Modified
+
+| File | Changes |
+|------|---------|
+| `iql_value_kernel.cu` | Rewrite `iql_adv_sigma_ema_update` (drift readiness), new `iql_support_floor` kernel, simplify `iql_modulate_td_errors` |
+| `gpu_iql_trainer.rs` | Expand `readiness_buf` to [6], add `p5_ema_buf`, load `iql_support_floor` kernel, update launcher signatures, remove staleness config |
+| `fused_training.rs` | Call `support_floor` after `compute_per_sample_support`, remove staleness args from IQL config init |
+
+## Testing
+
+1. `test_no_nan_after_graph_capture` — basic smoke
+2. `test_production_training_stability` — 10 epochs, check Best Sharpe
+3. `test_walk_forward_no_overfitting_50_epochs` — 50 epochs × 3 folds, verify peak epoch moves later and Sharpe holds