# Adaptive Controller Signal Floors — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Replace hardcoded thresholds in 12 RL controllers with observed-signal-driven ISV slots. One atomic commit. Eliminate the architectural failure mode where Phase 4.5 normalization invalidated 8 controllers simultaneously.

**Architecture:** New Welford-variance CUDA kernel (`rl_signal_variance_update.cu`) tracks running variance of every controller's input EMA. Controllers consume `sqrt(observed_var)` as their noise-floor reference instead of hardcoded constants. 52 new ISV slots wire the kernel into the existing per-step pipeline. Group A (8 saturated) gets asymmetric Schulman (tighten fast, widen on N consecutive); Group B (4 with hardcoded thresholds) gets the same Welford floors without the asymmetry.

**Tech Stack:** CUDA 12.4 sm_89/sm_86, cudarc 0.19.3, Rust 1.85+ Edition 2021, single-stream graph capture, mapped-pinned ISV bus.

**Spec:** `docs/superpowers/specs/2026-05-30-adaptive-controller-floor-design.md`

---

## File Structure

### Created

- `crates/ml-alpha/cuda/rl_signal_variance_update.cu` — Welford online variance kernel (~80 LOC). One launch per controller-input slot per step. Single-thread, single-block. Computes count/mean/M² triple in ISV.

### Modified

- `crates/ml-alpha/src/rl/isv_slots.rs` — +52 new slot constants, `RL_SLOTS_END` 588 → 640
- `crates/ml-alpha/build.rs` — register `rl_signal_variance_update` cubin
- `crates/ml-alpha/cuda/rl_ppo_clip_controller.cu` — adaptive noise floor + asymmetric Schulman
- `crates/ml-alpha/cuda/rl_target_tau_controller.cu` — same pattern
- `crates/ml-alpha/cuda/rl_rollout_steps_controller.cu` — same pattern + consume `RL_ADV_VAR_PRE_NORM_INDEX`
- `crates/ml-alpha/cuda/rl_entropy_coef_controller.cu` — same pattern
- `crates/ml-alpha/cuda/rl_per_alpha_controller.cu` — same pattern
- `crates/ml-alpha/cuda/rl_gamma_controller.cu` — adaptive `GAMMA_MIN` (Special G)
- `crates/ml-alpha/cuda/rl_reward_scale_controller.cu` — asymmetric rate-limit + bootstrap-fraction floor (Special R)
- `crates/ml-alpha/cuda/rl_q_distill_lambda_controller.cu` — adaptive `MIN_LAMBDA` / `MAX_LAMBDA` (Special Q)
- `crates/ml-alpha/cuda/rl_v_blend_alpha_controller.cu` — 5 hardcoded constants → 5 ISV slots
- `crates/ml-alpha/cuda/rl_ppo_ratio_clamp_controller.cu` — adaptive `MIN_OUT` / `MAX_OUT`
- `crates/ml-alpha/cuda/rl_reward_clamp_controller.cu` — 5 hardcoded constants → 5 ISV slots
- `crates/ml-alpha/cuda/rl_gate_threshold_controller.cu` — hardcoded `alpha = 0.01f` → 1 ISV slot
- `crates/ml-alpha/cuda/rl_advantage_normalize.cu` — emit `var_pre_norm` to ISV
- `crates/ml-alpha/cuda/rl_fused_controllers.cu` — unified update covering all 12 controllers
- `crates/ml-alpha/src/trainer/integrated.rs` — launch `rl_signal_variance_update` per step + populate new bootstrap ISV slots in `with_controllers_bootstrapped`

### Tests (added/modified)

- `crates/ml-alpha/tests/signal_variance_kernel.rs` — new GPU-oracle tests for Welford convergence
- `crates/ml-alpha/tests/controller_adaptive_floors.rs` — new integration tests: each controller holds at bootstrap when signal < adaptive floor
- `crates/ml-alpha/tests/isv_bootstrap.rs` — update with new slots in canonical bootstrap list

---

## Tasks

### Task 1: Allocate ISV slot constants

**Files:**
- Modify: `crates/ml-alpha/src/rl/isv_slots.rs` (around line 1135)

- [ ] **Step 1: Add 52 new constants after current `RL_V_SCALAR_MAG_EMA_INDEX = 587`**

```rust
// ============================================================
// Adaptive controller floors (spec 2026-05-30-adaptive-controller-floor-design)
// Welford variance triples + asymmetric Schulman counters + new input slots.
// ============================================================

// Group A — Welford variance triples (count, mean, M²) for each saturated controller's input.
pub const RL_KL_PI_VAR_COUNT_INDEX: usize = 588;
pub const RL_KL_PI_VAR_MEAN_INDEX:  usize = 589;
pub const RL_KL_PI_VAR_M2_INDEX:    usize = 590;
pub const RL_KL_PI_BELOW_COUNT_INDEX: usize = 591;

pub const RL_Q_DIV_VAR_COUNT_INDEX: usize = 592;
pub const RL_Q_DIV_VAR_MEAN_INDEX:  usize = 593;
pub const RL_Q_DIV_VAR_M2_INDEX:    usize = 594;
pub const RL_Q_DIV_BELOW_COUNT_INDEX: usize = 595;

pub const RL_ADV_VAR_VAR_COUNT_INDEX: usize = 596;
pub const RL_ADV_VAR_VAR_MEAN_INDEX:  usize = 597;
pub const RL_ADV_VAR_VAR_M2_INDEX:    usize = 598;
pub const RL_ADV_VAR_BELOW_COUNT_INDEX: usize = 599;

pub const RL_ENTROPY_OBS_VAR_COUNT_INDEX: usize = 600;
pub const RL_ENTROPY_OBS_VAR_MEAN_INDEX:  usize = 601;
pub const RL_ENTROPY_OBS_VAR_M2_INDEX:    usize = 602;
pub const RL_ENTROPY_OBS_BELOW_COUNT_INDEX: usize = 603;

pub const RL_TD_KURT_VAR_COUNT_INDEX: usize = 604;
pub const RL_TD_KURT_VAR_MEAN_INDEX:  usize = 605;
pub const RL_TD_KURT_VAR_M2_INDEX:    usize = 606;
pub const RL_TD_KURT_BELOW_COUNT_INDEX: usize = 607;

pub const RL_TRADE_DUR_VAR_COUNT_INDEX: usize = 608;
pub const RL_TRADE_DUR_VAR_MEAN_INDEX:  usize = 609;
pub const RL_TRADE_DUR_VAR_M2_INDEX:    usize = 610;

// New input signals
pub const RL_ADV_VAR_PRE_NORM_INDEX:           usize = 612;
pub const RL_GAMMA_MIN_ADAPTIVE_INDEX:         usize = 613;
pub const RL_REWARD_MAGNITUDE_EMA_INDEX:       usize = 614;

pub const RL_REWARD_MAG_VAR_COUNT_INDEX: usize = 615;
pub const RL_REWARD_MAG_VAR_MEAN_INDEX:  usize = 616;
pub const RL_REWARD_MAG_VAR_M2_INDEX:    usize = 617;

pub const RL_Q_DISTILL_KL_VAR_COUNT_INDEX: usize = 618;
pub const RL_Q_DISTILL_KL_VAR_MEAN_INDEX:  usize = 619;
pub const RL_Q_DISTILL_KL_VAR_M2_INDEX:    usize = 620;

// Group B — V-blend alpha (Phase 4.4) adaptive bounds
pub const RL_V_BLEND_DEAD_SIGNAL_FLOOR_INDEX: usize = 621;
pub const RL_V_BLEND_TARGET_TRACK_RATIO_INDEX: usize = 622;
pub const RL_V_BLEND_SCHULMAN_STEP_INDEX:     usize = 623;
pub const RL_V_BLEND_EMA_ALPHA_INDEX:         usize = 624;
pub const RL_V_BLEND_BOOTSTRAP_ALPHA_INDEX:   usize = 625;
pub const RL_V_BLEND_SCALAR_VAR_COUNT_INDEX:  usize = 626;
pub const RL_V_BLEND_SCALAR_VAR_MEAN_INDEX:   usize = 627;
pub const RL_V_BLEND_SCALAR_VAR_M2_INDEX:     usize = 628;

// Group B — PPO ratio clamp adaptive bounds
pub const RL_PPO_RATIO_CLAMP_MAX_ADAPTIVE_INDEX: usize = 629;
pub const RL_PPO_RATIO_VAR_COUNT_INDEX:          usize = 630;
pub const RL_PPO_RATIO_VAR_MEAN_INDEX:           usize = 631;
pub const RL_PPO_RATIO_VAR_M2_INDEX:             usize = 632;

// Group B — Reward clamp adaptive bounds
pub const RL_REWARD_CLAMP_V_BOUND_FLOOR_INDEX: usize = 633;
pub const RL_REWARD_CLAMP_V_BOUND_EWMA_ALPHA_INDEX: usize = 634;
pub const RL_REWARD_CLAMP_MIN_WIN_INDEX:       usize = 635;
pub const RL_REWARD_CLAMP_MIN_RATIO_INDEX:     usize = 636;
pub const RL_REWARD_CLAMP_MAX_RATIO_INDEX:     usize = 637;

// Group B — Gate threshold EMA alpha
pub const RL_GATE_EMA_ALPHA_INDEX:             usize = 638;

// Q-distill adaptive bound
pub const RL_Q_DISTILL_LAMBDA_MIN_ADAPTIVE_INDEX: usize = 639;

pub const RL_SLOTS_END: usize = 640;  // was 588
```

- [ ] **Step 2: Run `cargo check -p ml-alpha` and verify no slot-numbering conflicts**

Run: `SQLX_OFFLINE=true cargo check -p ml-alpha 2>&1 | tail -5`
Expected: `Finished` (no errors)

- [ ] **Step 3: Commit**

Do not commit yet — atomic single commit at end per Option B.

---

### Task 2: Welford variance kernel

**Files:**
- Create: `crates/ml-alpha/cuda/rl_signal_variance_update.cu`
- Modify: `crates/ml-alpha/build.rs`

- [ ] **Step 1: Write the kernel**

```c
// rl_signal_variance_update.cu — Welford online variance for controller inputs.
// One-thread, one-block kernel. Updates count/mean/M² triple in ISV.
// Caller launches once per controller-input slot per step (or via fused kernel).
//
// Per pearl_first_observation_bootstrap: skip when input == sentinel 0.0.
// Per feedback_adaptive_not_tuned: emits observed signal statistics that
// controllers consume as noise-floor anchors.

extern "C" __global__ void rl_signal_variance_update(
    float* __restrict__ isv,
    int input_slot,
    int count_slot,
    int mean_slot,
    int m2_slot
) {
    if (threadIdx.x != 0 || blockIdx.x != 0) return;
    const float x = isv[input_slot];
    if (x == 0.0f) return;                       // sentinel skip

    const float n_prev = isv[count_slot];
    const float n_new = n_prev + 1.0f;
    const float mean_prev = isv[mean_slot];
    const float delta = x - mean_prev;
    const float mean_new = mean_prev + delta / n_new;
    const float delta2 = x - mean_new;
    const float m2_new = isv[m2_slot] + delta * delta2;

    isv[count_slot] = n_new;
    isv[mean_slot]  = mean_new;
    isv[m2_slot]    = m2_new;
}
```

- [ ] **Step 2: Register cubin in build.rs**

Add to `build.rs` cubin list (next to other RL controller cubins).

```rust
// In the cubin compile list:
"rl_signal_variance_update",
```

- [ ] **Step 3: Run `cargo check` to verify**

Run: `SQLX_OFFLINE=true cargo check -p ml-alpha 2>&1 | tail -3`
Expected: `Finished` cleanly.

---

### Task 3: Welford kernel unit tests

**Files:**
- Create: `crates/ml-alpha/tests/signal_variance_kernel.rs`

- [ ] **Step 1: Write G1 — convergence on constant input**

```rust
//! GPU-oracle tests for rl_signal_variance_update.
//!
//! Per feedback_no_cpu_test_fallbacks: oracles are analytical identities of
//! Welford online variance (constant → 0 var; finite sequence → known var).

use cudarc::driver::{LaunchConfig, PushKernelArg};
use ml_alpha::pinned_mem::MappedF32Buffer;
use ml_core::device::MlDevice;

const INPUT_SLOT: usize = 0;
const COUNT_SLOT: usize = 1;
const MEAN_SLOT:  usize = 2;
const M2_SLOT:    usize = 3;

#[test]
#[ignore = "requires CUDA"]
fn welford_constant_input_zero_variance() {
    let Ok(dev) = MlDevice::cuda(0) else { return };
    let isv = unsafe { MappedF32Buffer::new(4) }.expect("isv");
    let module = dev.load_cubin(include_bytes!(
        concat!(env!("OUT_DIR"), "/rl_signal_variance_update.cubin")
    ).to_vec()).expect("load");
    let func = module.load_function("rl_signal_variance_update").expect("fn");

    isv.write_record(INPUT_SLOT, 0.5);
    let stream = dev.cuda_stream().expect("stream").clone();
    for _ in 0..50 {
        let mut builder = stream.launch_builder(&func);
        builder.arg(&isv.dev_ptr)
               .arg(&(INPUT_SLOT as i32))
               .arg(&(COUNT_SLOT as i32))
               .arg(&(MEAN_SLOT as i32))
               .arg(&(M2_SLOT as i32));
        unsafe { builder.launch(LaunchConfig::for_num_elems(1)) }.expect("launch");
    }
    stream.synchronize().expect("sync");
    assert_eq!(isv.read_record(COUNT_SLOT), 50.0);
    assert!((isv.read_record(MEAN_SLOT) - 0.5).abs() < 1e-6);
    assert!(isv.read_record(M2_SLOT).abs() < 1e-6);
}
```

- [ ] **Step 2: Write G2 — known variance on finite sequence**

```rust
#[test]
#[ignore = "requires CUDA"]
fn welford_known_sequence_variance() {
    let Ok(dev) = MlDevice::cuda(0) else { return };
    let isv = unsafe { MappedF32Buffer::new(4) }.expect("isv");
    let module = dev.load_cubin(include_bytes!(
        concat!(env!("OUT_DIR"), "/rl_signal_variance_update.cubin")
    ).to_vec()).expect("load");
    let func = module.load_function("rl_signal_variance_update").expect("fn");
    let stream = dev.cuda_stream().expect("stream").clone();

    // Sequence [1, 2, 3, 4, 5]: mean = 3, var = (4+1+0+1+4)/4 = 2.5 (sample variance).
    for x in [1.0_f32, 2.0, 3.0, 4.0, 5.0] {
        isv.write_record(INPUT_SLOT, x);
        let mut builder = stream.launch_builder(&func);
        builder.arg(&isv.dev_ptr)
               .arg(&(INPUT_SLOT as i32))
               .arg(&(COUNT_SLOT as i32))
               .arg(&(MEAN_SLOT as i32))
               .arg(&(M2_SLOT as i32));
        unsafe { builder.launch(LaunchConfig::for_num_elems(1)) }.expect("launch");
        stream.synchronize().expect("sync");
    }
    let n = isv.read_record(COUNT_SLOT);
    let m2 = isv.read_record(M2_SLOT);
    let sample_var = m2 / (n - 1.0);
    assert!((sample_var - 2.5).abs() < 1e-5,
            "expected sample variance 2.5, got {}", sample_var);
}
```

- [ ] **Step 3: Write G3 — sentinel skip**

```rust
#[test]
#[ignore = "requires CUDA"]
fn welford_sentinel_zero_is_skipped() {
    let Ok(dev) = MlDevice::cuda(0) else { return };
    let isv = unsafe { MappedF32Buffer::new(4) }.expect("isv");
    let module = dev.load_cubin(include_bytes!(
        concat!(env!("OUT_DIR"), "/rl_signal_variance_update.cubin")
    ).to_vec()).expect("load");
    let func = module.load_function("rl_signal_variance_update").expect("fn");
    let stream = dev.cuda_stream().expect("stream").clone();

    // INPUT_SLOT stays 0.0 (sentinel). Launch should be a no-op.
    let mut builder = stream.launch_builder(&func);
    builder.arg(&isv.dev_ptr)
           .arg(&(INPUT_SLOT as i32))
           .arg(&(COUNT_SLOT as i32))
           .arg(&(MEAN_SLOT as i32))
           .arg(&(M2_SLOT as i32));
    unsafe { builder.launch(LaunchConfig::for_num_elems(1)) }.expect("launch");
    stream.synchronize().expect("sync");
    assert_eq!(isv.read_record(COUNT_SLOT), 0.0, "sentinel must not be counted");
}
```

- [ ] **Step 4: Run all three tests**

Run: `SQLX_OFFLINE=true cargo test -p ml-alpha --test signal_variance_kernel --release -- --ignored --nocapture`
Expected: 3 passed.

---

### Task 4: Update `rl_advantage_normalize.cu` to emit `var_pre_norm`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_advantage_normalize.cu`

- [ ] **Step 1: Add the ISV emission BEFORE the in-place normalize**

In the existing `rl_advantage_normalize` kernel (after the variance computation, before the normalize pass), write `var` to `isv[RL_ADV_VAR_PRE_NORM_INDEX]`:

```c
// After: const float var = s_var[0] / (float)B;
if (tid == 0) {
    isv[RL_ADV_VAR_PRE_NORM_INDEX] = var;   // NEW: emit raw variance for rollout_steps_controller
    s_inv_std = rsqrtf(var + ADVANTAGE_VAR_FLOOR);
}
```

Add `RL_ADV_VAR_PRE_NORM_INDEX` define at top of file:
```c
#define RL_ADV_VAR_PRE_NORM_INDEX 612
```

Add `float* __restrict__ isv` to the kernel signature; thread the ISV device pointer through from the trainer call site.

- [ ] **Step 2: Update trainer call site to pass `isv_dev_ptr`**

Modify `integrated.rs` where `rl_advantage_normalize` is launched. Add `args.push_ptr(self.isv_dev_ptr)` to the RawArgs.

- [ ] **Step 3: Verify with smoke**

Run: `SQLX_OFFLINE=true cargo test -p ml-alpha --test integrated_trainer_smoke --release -- --ignored --nocapture`
Expected: passes (existing GPU smoke).

---

### Task 5: Refactor `rl_ppo_clip_controller.cu` (canonical pattern)

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_ppo_clip_controller.cu`

- [ ] **Step 1: Replace hardcoded `KL_NOISE_FLOOR_FRAC` with adaptive floor**

```c
// BEFORE:
#define KL_NOISE_FLOOR_FRAC 0.01f
const float kl_noise_floor = kl_target * KL_NOISE_FLOOR_FRAC;

// AFTER:
const float kl_var_count = isv[RL_KL_PI_VAR_COUNT_INDEX];
const float kl_var = (kl_var_count > 1.0f)
    ? isv[RL_KL_PI_VAR_M2_INDEX] / (kl_var_count - 1.0f)
    : 0.0f;
const float kl_std = sqrtf(kl_var);
const float kl_noise_floor = fmaxf(kl_target * 0.5f, kl_std * 2.0f);
```

Add at top of file:
```c
#define RL_KL_PI_VAR_COUNT_INDEX 588
#define RL_KL_PI_VAR_MEAN_INDEX  589
#define RL_KL_PI_VAR_M2_INDEX    590
#define RL_KL_PI_BELOW_COUNT_INDEX 591
```

- [ ] **Step 2: Add asymmetric Schulman (widen requires N consecutive below-band)**

```c
// In the Schulman-style adjustment block:
if (kl_ema > kl_target * tolerance) {
    ratio = 1.0f / adjust_rate;                       // tighten — fires immediately
    isv[RL_KL_PI_BELOW_COUNT_INDEX] = 0.0f;
} else if (kl_ema < kl_target / tolerance) {
    isv[RL_KL_PI_BELOW_COUNT_INDEX] += 1.0f;
    ratio = (isv[RL_KL_PI_BELOW_COUNT_INDEX] >= 3.0f) ? adjust_rate : 1.0f;
} else {
    isv[RL_KL_PI_BELOW_COUNT_INDEX] = 0.0f;
    ratio = 1.0f;
}
```

- [ ] **Step 3: Bootstrap PPO clip at target (not at safety bound)**

Change the bootstrap `RL_EPS_BOOTSTRAP_INDEX = 474` value in trainer init from 0.2 to **derive from target**: bootstrap = ISV[RL_KL_TARGET_INDEX] × some multiplier that puts ε in a conservative starting position. Decision deferred to implementation — for now, set bootstrap to `isv[RL_KL_TARGET_INDEX]` (so ε starts at 0.01 = target, very conservative).

This is a trainer-side change in `with_controllers_bootstrapped`:
```rust
// In trainer/integrated.rs with_controllers_bootstrapped:
trainer.isv_mapped.write_record(RL_EPS_BOOTSTRAP_INDEX, trainer.isv_mapped.read_record(RL_KL_TARGET_INDEX));
```

- [ ] **Step 4: Verify ppo_clip controller compiles**

Run: `SQLX_OFFLINE=true cargo build -p ml-alpha --release 2>&1 | tail -3`
Expected: `Finished` clean.

---

### Task 6: Refactor remaining Group A controllers (sweep)

**Files (all in `crates/ml-alpha/cuda/`):**
- `rl_target_tau_controller.cu`
- `rl_rollout_steps_controller.cu` (also consume `RL_ADV_VAR_PRE_NORM_INDEX` as the input signal)
- `rl_entropy_coef_controller.cu`
- `rl_per_alpha_controller.cu`

- [ ] **Step 1-4: Apply Task 5's pattern to each**

For each of the 4 controllers above: same three changes as PPO clip (adaptive noise floor, asymmetric Schulman, conservative bootstrap). Use the slot table from spec to wire variance triples per controller.

- [ ] **Step 5: Verify all compile**

Run: `SQLX_OFFLINE=true cargo build -p ml-alpha --release 2>&1 | tail -3`
Expected: `Finished` clean.

---

### Task 7: Special case G — `rl_gamma_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_gamma_controller.cu`

- [ ] **Step 1: Replace hardcoded `GAMMA_MIN` with Welford-mean-driven adaptive floor**

Add at top:
```c
#define RL_GAMMA_MIN_ADAPTIVE_INDEX     613
#define RL_MEAN_TRADE_DURATION_EMA_INDEX 417
#define RL_TRADE_DUR_VAR_COUNT_INDEX    608  // Welford count for trade duration
#define RL_TRADE_DUR_VAR_MEAN_INDEX     609  // Welford mean for trade duration
#define GAMMA_MIN_ABSOLUTE              0.9f  // "don't degenerate to bandit" hard floor
#define HORIZON_MULTIPLIER              2.0f
#define WELFORD_WARMUP_OBS              100.0f  // confidence gate, not magnitude calibration
```

Compute adaptive min at start of controller body. Per spec Q6 resolution, use the Welford mean (which is already computed by `rl_signal_variance_update` for `rl_per_alpha_controller`'s noise floor) once enough observations have accumulated. The Welford mean's natural lag (1/N smoothing) breaks the feedback loop without any step-count gating:

```c
const float d_welford_count = isv[RL_TRADE_DUR_VAR_COUNT_INDEX];
const float d_welford_mean  = isv[RL_TRADE_DUR_VAR_MEAN_INDEX];

// Warmup: use fast EMA until 100 trade observations accumulated.
// After warmup, use Welford mean (denominator-of-N smoothed, naturally
// resists the gamma↔trade_duration feedback loop without explicit lag).
const float d_smoothed = (d_welford_count >= WELFORD_WARMUP_OBS)
    ? d_welford_mean
    : isv[RL_MEAN_TRADE_DURATION_EMA_INDEX];

const float adaptive_min = fmaxf(GAMMA_MIN_ABSOLUTE,
                                  1.0f - 1.0f / fmaxf(d_smoothed * HORIZON_MULTIPLIER, 10.0f));
isv[RL_GAMMA_MIN_ADAPTIVE_INDEX] = adaptive_min;
```

Replace `GAMMA_MIN` usage with `isv[RL_GAMMA_MIN_ADAPTIVE_INDEX]` in the clamp:
```c
gamma = fmaxf(isv[RL_GAMMA_MIN_ADAPTIVE_INDEX], fminf(gamma, GAMMA_MAX));
```

- [ ] **Step 2: Bootstrap `RL_GAMMA_MIN_ADAPTIVE_INDEX` in trainer**

In `with_controllers_bootstrapped`:
```rust
trainer.isv_mapped.write_record(RL_GAMMA_MIN_ADAPTIVE_INDEX, 0.95);  // safe default before first observation
```

- [ ] **Step 3: Compile + verify**

Run: `SQLX_OFFLINE=true cargo build -p ml-alpha --release 2>&1 | tail -3`
Expected: `Finished`.

---

### Task 8: Special case R — `rl_reward_scale_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_reward_scale_controller.cu`

- [ ] **Step 1: Add asymmetric rate limit (decrease slower than increase)**

The crash from 1.0 → 0.004 in 100 steps was the decrease side. Cap per-step decrease at 1.05× (i.e., new ≥ prev / 1.05):

```c
// After computing new_scale:
const float prev_scale = isv[RL_REWARD_SCALE_INDEX];
if (new_scale < prev_scale) {
    new_scale = fmaxf(new_scale, prev_scale / 1.05f);  // decrease ≤ 5% per step
}
```

- [ ] **Step 2: Add bootstrap-fraction floor (until N trades accumulated)**

```c
#define BOOTSTRAP_FRACTION 0.1f
#define MIN_TRADES_FOR_RELEASE 100
const float trade_count = isv[RL_TRADE_COUNT_INDEX];  // verify slot exists or use closest signal
const float boot = isv[RL_REWARD_SCALE_BOOTSTRAP_INDEX];
if (trade_count < (float)MIN_TRADES_FOR_RELEASE) {
    new_scale = fmaxf(new_scale, boot * BOOTSTRAP_FRACTION);
}
```

- [ ] **Step 3: Emit `RL_REWARD_MAGNITUDE_EMA_INDEX` from this controller**

Already computed internally as `pnl_per_batch_magnitude`. Write to `isv[RL_REWARD_MAGNITUDE_EMA_INDEX]` so the Welford variance kernel can track it.

- [ ] **Step 4: Compile + verify**

---

### Task 9: Special case Q — `rl_q_distill_lambda_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_q_distill_lambda_controller.cu`

- [ ] **Step 1: Replace `MIN_LAMBDA` with adaptive bound**

```c
#define RL_Q_DISTILL_KL_VAR_COUNT_INDEX 618
#define RL_Q_DISTILL_KL_VAR_M2_INDEX    620
#define RL_Q_DISTILL_LAMBDA_MIN_ADAPTIVE_INDEX 639

const float kl_var_count = isv[RL_Q_DISTILL_KL_VAR_COUNT_INDEX];
const float kl_var = (kl_var_count > 1.0f)
    ? isv[RL_Q_DISTILL_KL_VAR_M2_INDEX] / (kl_var_count - 1.0f)
    : 0.0f;
const float kl_std = sqrtf(kl_var);
const float adaptive_min = fmaxf(0.001f, kl_std * 0.05f);
isv[RL_Q_DISTILL_LAMBDA_MIN_ADAPTIVE_INDEX] = adaptive_min;
```

Replace `MIN_LAMBDA` usage in the clamp with `isv[RL_Q_DISTILL_LAMBDA_MIN_ADAPTIVE_INDEX]`.

- [ ] **Step 2: Bootstrap the adaptive min**

In `with_controllers_bootstrapped`:
```rust
trainer.isv_mapped.write_record(RL_Q_DISTILL_LAMBDA_MIN_ADAPTIVE_INDEX, 0.01);  // safer default than 0.05
```

- [ ] **Step 3: Compile + verify**

---

### Task 10: Group B — `rl_v_blend_alpha_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_v_blend_alpha_controller.cu`

- [ ] **Step 1: Replace 5 hardcoded constants with ISV slot reads**

```c
// BEFORE:
#define EMA_ALPHA 0.01f
#define TARGET_TRACK_RATIO 0.10f
#define SCHULMAN_STEP 0.01f
#define DEAD_SIGNAL_FLOOR 1e-4f
#define BOOTSTRAP_ALPHA 1.0f

// AFTER (use as runtime reads):
const float ema_alpha          = isv[RL_V_BLEND_EMA_ALPHA_INDEX];
const float target_track_ratio = isv[RL_V_BLEND_TARGET_TRACK_RATIO_INDEX];
const float schulman_step      = isv[RL_V_BLEND_SCHULMAN_STEP_INDEX];
const float dead_signal_floor  = isv[RL_V_BLEND_DEAD_SIGNAL_FLOOR_INDEX];
const float bootstrap_alpha    = isv[RL_V_BLEND_BOOTSTRAP_ALPHA_INDEX];
```

DEAD_SIGNAL_FLOOR should adapt to observed V_scalar magnitude variance:
```c
const float vmag_var_count = isv[RL_V_BLEND_SCALAR_VAR_COUNT_INDEX];
const float vmag_var = (vmag_var_count > 1.0f)
    ? isv[RL_V_BLEND_SCALAR_VAR_M2_INDEX] / (vmag_var_count - 1.0f)
    : 0.0f;
const float adaptive_dead_floor = fmaxf(isv[RL_V_BLEND_DEAD_SIGNAL_FLOOR_INDEX],
                                         sqrtf(vmag_var) * 0.1f);
if (mag_ema < adaptive_dead_floor) {
    // hold α
    ...
}
```

- [ ] **Step 2: Bootstrap the 5 new ISV slots in trainer**

```rust
trainer.isv_mapped.write_record(RL_V_BLEND_EMA_ALPHA_INDEX, 0.01);
trainer.isv_mapped.write_record(RL_V_BLEND_TARGET_TRACK_RATIO_INDEX, 0.10);
trainer.isv_mapped.write_record(RL_V_BLEND_SCHULMAN_STEP_INDEX, 0.01);
trainer.isv_mapped.write_record(RL_V_BLEND_DEAD_SIGNAL_FLOOR_INDEX, 1e-4);
trainer.isv_mapped.write_record(RL_V_BLEND_BOOTSTRAP_ALPHA_INDEX, 1.0);
```

- [ ] **Step 3: Compile + verify**

---

### Task 11: Group B — `rl_ppo_ratio_clamp_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_ppo_ratio_clamp_controller.cu`

- [ ] **Step 1: Replace `MIN_OUT` / `MAX_OUT` with adaptive bounds**

```c
const float ratio_var_count = isv[RL_PPO_RATIO_VAR_COUNT_INDEX];
const float ratio_var = (ratio_var_count > 1.0f)
    ? isv[RL_PPO_RATIO_VAR_M2_INDEX] / (ratio_var_count - 1.0f)
    : 0.0f;
const float ratio_std = sqrtf(ratio_var);
const float adaptive_min = fmaxf(1.5f, 1.0f + ratio_std * 3.0f);  // 3σ above unit, but ≥ 1.5
const float adaptive_max = fminf(1000.0f, fmaxf(adaptive_min * 5.0f, isv[RL_PPO_RATIO_CLAMP_MAX_ADAPTIVE_INDEX]));
clamp = fmaxf(adaptive_min, fminf(clamp, adaptive_max));
```

- [ ] **Step 2: Compile + verify**

---

### Task 12: Group B — `rl_reward_clamp_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_reward_clamp_controller.cu`

- [ ] **Step 1: Replace 5 hardcoded constants with ISV reads**

```c
const float v_bound_floor       = isv[RL_REWARD_CLAMP_V_BOUND_FLOOR_INDEX];
const float v_bound_ewma_alpha  = isv[RL_REWARD_CLAMP_V_BOUND_EWMA_ALPHA_INDEX];
const float min_win             = isv[RL_REWARD_CLAMP_MIN_WIN_INDEX];
const float min_ratio           = isv[RL_REWARD_CLAMP_MIN_RATIO_INDEX];
const float max_ratio           = isv[RL_REWARD_CLAMP_MAX_RATIO_INDEX];
```

- [ ] **Step 2: Bootstrap the 5 new ISV slots**

In trainer init, set defaults to current hardcoded values:
```rust
trainer.isv_mapped.write_record(RL_REWARD_CLAMP_V_BOUND_FLOOR_INDEX, 1.0);
trainer.isv_mapped.write_record(RL_REWARD_CLAMP_V_BOUND_EWMA_ALPHA_INDEX, 0.001);
trainer.isv_mapped.write_record(RL_REWARD_CLAMP_MIN_WIN_INDEX, 1.0);
trainer.isv_mapped.write_record(RL_REWARD_CLAMP_MIN_RATIO_INDEX, 1.0);
trainer.isv_mapped.write_record(RL_REWARD_CLAMP_MAX_RATIO_INDEX, 3.0);
```

- [ ] **Step 3: Compile + verify**

---

### Task 13: Group B — `rl_gate_threshold_controller.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_gate_threshold_controller.cu`

- [ ] **Step 1: Replace `alpha = 0.01f` with ISV slot**

```c
// BEFORE:
const float alpha = 0.01f;

// AFTER:
const float alpha = isv[RL_GATE_EMA_ALPHA_INDEX];
```

- [ ] **Step 2: Bootstrap in trainer**

```rust
trainer.isv_mapped.write_record(RL_GATE_EMA_ALPHA_INDEX, 0.01);
```

- [ ] **Step 3: Compile + verify**

---

### Task 14: Update `rl_fused_controllers.cu`

**Files:**
- Modify: `crates/ml-alpha/cuda/rl_fused_controllers.cu`

- [ ] **Step 1: Replicate the per-controller adaptive logic**

The fused kernel is what the trainer actually launches per-step. Mirror every change from Tasks 5-13 inside the corresponding branch of the fused kernel. Same ISV slot reads, same adaptive math.

- [ ] **Step 2: Compile + verify both individual controllers AND fused match**

Run: `SQLX_OFFLINE=true cargo build -p ml-alpha --release 2>&1 | tail -3`
Expected: `Finished` clean.

---

### Task 15: Wire trainer integration

**Files:**
- Modify: `crates/ml-alpha/src/trainer/integrated.rs`

- [ ] **Step 1: Add per-step Welford launches**

After the existing EMA producer kernels (R3 `launch_ema_update_per_step` etc.) and BEFORE the controllers fire:

```rust
// Launch rl_signal_variance_update for each controller input.
for (input_slot, count_slot, mean_slot, m2_slot) in [
    (RL_KL_PI_EMA_INDEX, RL_KL_PI_VAR_COUNT_INDEX, RL_KL_PI_VAR_MEAN_INDEX, RL_KL_PI_VAR_M2_INDEX),
    (RL_Q_DIVERGENCE_EMA_INDEX, RL_Q_DIV_VAR_COUNT_INDEX, RL_Q_DIV_VAR_MEAN_INDEX, RL_Q_DIV_VAR_M2_INDEX),
    (RL_ADV_VAR_PRE_NORM_INDEX, RL_ADV_VAR_VAR_COUNT_INDEX, RL_ADV_VAR_VAR_MEAN_INDEX, RL_ADV_VAR_VAR_M2_INDEX),
    (RL_ENTROPY_OBSERVED_EMA_INDEX, RL_ENTROPY_OBS_VAR_COUNT_INDEX, RL_ENTROPY_OBS_VAR_MEAN_INDEX, RL_ENTROPY_OBS_VAR_M2_INDEX),
    (RL_TD_KURTOSIS_EMA_INDEX, RL_TD_KURT_VAR_COUNT_INDEX, RL_TD_KURT_VAR_MEAN_INDEX, RL_TD_KURT_VAR_M2_INDEX),
    (RL_MEAN_TRADE_DURATION_EMA_INDEX, RL_TRADE_DUR_VAR_COUNT_INDEX, RL_TRADE_DUR_VAR_MEAN_INDEX, RL_TRADE_DUR_VAR_M2_INDEX),
    (RL_REWARD_MAGNITUDE_EMA_INDEX, RL_REWARD_MAG_VAR_COUNT_INDEX, RL_REWARD_MAG_VAR_MEAN_INDEX, RL_REWARD_MAG_VAR_M2_INDEX),
    (RL_Q_DISTILL_KL_EMA_INDEX, RL_Q_DISTILL_KL_VAR_COUNT_INDEX, RL_Q_DISTILL_KL_VAR_MEAN_INDEX, RL_Q_DISTILL_KL_VAR_M2_INDEX),
    (RL_V_SCALAR_MAG_EMA_INDEX, RL_V_BLEND_SCALAR_VAR_COUNT_INDEX, RL_V_BLEND_SCALAR_VAR_MEAN_INDEX, RL_V_BLEND_SCALAR_VAR_M2_INDEX),
    // PPO ratio: input from observed log_ratio after each batch — use a magnitude EMA producer
    // (omitted: add a small magnitude-tracking kernel if not already present)
] {
    let mut args = RawArgs::new();
    args.push_ptr(self.isv_dev_ptr);
    args.push_i32(input_slot as i32);
    args.push_i32(count_slot as i32);
    args.push_i32(mean_slot as i32);
    args.push_i32(m2_slot as i32);
    let mut ptrs = args.build_arg_ptrs();
    unsafe {
        raw_launch(
            self.rl_signal_variance_update_fn.cu_function(),
            (1, 1, 1), (1, 1, 1), 0,
            self.raw_stream,
            &mut ptrs[..args.len()],
        ).map_err(|e| anyhow::anyhow!("rl_signal_variance_update: {:?}", e))?;
    }
}
```

- [ ] **Step 2: Bootstrap all new ISV slots in `with_controllers_bootstrapped`**

Aggregate all the `write_record` calls from Tasks 5-13 in one block at the end of `with_controllers_bootstrapped`.

- [ ] **Step 3: Run integrated_trainer_smoke**

Run: `SQLX_OFFLINE=true cargo test -p ml-alpha --test integrated_trainer_smoke --release -- --ignored --nocapture`
Expected: passes.

---

### Task 16: Controller-level adaptive-floor tests

**Files:**
- Create: `crates/ml-alpha/tests/controller_adaptive_floors.rs`

- [ ] **Step 1: G3 — PPO clip holds on small signal**

Bootstrap a trainer. Feed `kl_ema = 0.001` (below target 0.01) and observed_var = 1e-7 for 50 steps. Assert `ε` stays in `[bootstrap × 0.9, bootstrap × 1.1]` (controller held — didn't drift to MAX).

- [ ] **Step 2: G4 — PPO clip tightens on big signal**

Single `kl_ema = 0.05` observation (5× target). Assert `ε` decreases by `1/adjust_rate` immediately (asymmetric tightening fires on single observation).

- [ ] **Step 3: G5 — PPO clip widens slowly (N=3 consecutive below)**

Feed `kl_ema = 0.001` for 10 steps. Assert widen fires only after 3 consecutive below-band observations.

- [ ] **Step 4: G6 — gamma adaptive_min tracks trade duration**

Set `RL_MEAN_TRADE_DURATION_EMA_INDEX = 30.0`, advance step counter to 500, assert `RL_GAMMA_MIN_ADAPTIVE_INDEX ≈ 1 - 1/60 = 0.983`.

- [ ] **Step 5: Update isv_bootstrap.rs test**

Add the 52 new slots to the canonical-bootstrap assertion list. Ensure values match what `with_controllers_bootstrapped` writes.

- [ ] **Step 6: Run all gates**

Run: `SQLX_OFFLINE=true cargo test -p ml-alpha --test controller_adaptive_floors --release -- --ignored --nocapture`
Expected: 5+ passed.

---

### Task 17: Local smoke validation

- [ ] **Step 1: 1k-step smoke**

```bash
cd /home/jgrusewski/Work/foxhunt
SQLX_OFFLINE=true cargo build -p ml-alpha --release --example alpha_rl_train
./target/release/examples/alpha_rl_train \
  --mbp10-data-dir test_data/futures-baseline/ES.FUT \
  --predecoded-dir /tmp/smoke/predecoded \
  --out /tmp/smoke \
  --n-steps 1000 --n-backtests 128 --seq-len 32 \
  --per-capacity 1024 --seed 16962 --instrument-mode all \
  --n-folds 1 --fold-idx 0 --n-eval-steps 0 \
  --diag-jsonl /tmp/smoke/diag.jsonl
```

Expected: completes cleanly, no NaN abort.

- [ ] **Step 2: compute-sanitizer at b=128**

```bash
compute-sanitizer --tool memcheck --launch-timeout 600 --error-exitcode 1 \
  ./target/release/examples/alpha_rl_train \
  --mbp10-data-dir test_data/futures-baseline/ES.FUT \
  --predecoded-dir /tmp/smoke-cs/predecoded \
  --out /tmp/smoke-cs \
  --n-steps 5 --n-backtests 128 --seq-len 32 \
  --per-capacity 1024 --seed 16962 --instrument-mode all \
  --n-folds 1 --fold-idx 0 --n-eval-steps 0
```

Expected: `ERROR SUMMARY: 0 errors`.

- [ ] **Step 3: Diagnose controllers on 1k smoke**

Inspect `/tmp/smoke/diag.jsonl` at steps 1, 100, 500, 1000. All 12 controllers should:
- Be at sensible values (not stuck at MIN/MAX)
- Move in response to observed signal
- Welford variance triples grow over time

---

### Task 18: Atomic commit

- [ ] **Step 1: Verify all changes compile + tests pass**

```bash
SQLX_OFFLINE=true cargo build -p ml-alpha --release --quiet
SQLX_OFFLINE=true cargo build -p ml-alpha --tests --quiet
SQLX_OFFLINE=true cargo test -p ml-alpha --test integrated_trainer_smoke --release -- --ignored --nocapture
SQLX_OFFLINE=true cargo test -p ml-alpha --test signal_variance_kernel --release -- --ignored --nocapture
SQLX_OFFLINE=true cargo test -p ml-alpha --test controller_adaptive_floors --release -- --ignored --nocapture
```

Expected: all pass.

- [ ] **Step 2: Single atomic commit**

```bash
git add crates/ml-alpha/
git commit -m "feat(rl): adaptive controller floors — 12 controllers, signal-driven thresholds

Replaces hardcoded thresholds (NOISE_FLOOR_FRAC, MIN/MAX bounds,
DEAD_SIGNAL_FLOOR, etc.) across 12 RL controllers with observed-signal-
driven ISV-slot bounds. Eliminates the architectural failure mode
documented in fold 0 walk-forward (alpha-rl-m9cx5, eval pnl -\$1.56M)
where Phase 4.5 advantage normalization invalidated 8 controllers'
hardcoded signal-scale assumptions simultaneously, causing all of them
to saturate at extrema (ppo_clip at MAX 0.50, q_distill_lambda at MIN
0.05, gamma at MIN 0.995, reward_scale crashed 250× in 100 steps, etc.).

Per feedback_adaptive_not_tuned, feedback_isv_for_adaptive_bounds,
pearl_controller_anchors_isv_driven: every controller threshold now
derives from observed signal statistics (Welford online variance)
rather than constants calibrated against a prior signal regime.

Spec: docs/superpowers/specs/2026-05-30-adaptive-controller-floor-design.md
Plan: docs/superpowers/plans/2026-05-30-adaptive-controller-floor-plan.md

New: rl_signal_variance_update.cu (Welford kernel, 80 LOC)
Modified: 12 controller .cu files + rl_fused_controllers.cu +
          rl_advantage_normalize.cu (emit var_pre_norm) +
          isv_slots.rs (+52 slots, RL_SLOTS_END 588→640) +
          integrated.rs (Welford launches + bootstrap writes)

Validation:
  - 3 GPU-oracle Welford kernel tests pass
  - 5 controller adaptive-floor invariant tests pass
  - integrated_trainer_smoke (full GPU pipeline) passes
  - compute-sanitizer memcheck b=128: 0 errors
  - 1k-step local smoke: all 12 controllers move with signal, none stuck

Cluster validation (G7/G8/G9) submitted as follow-up runs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>"
```

- [ ] **Step 3: Push to origin/main**

```bash
git push origin main
```

Expected: pushed cleanly.

---

### Task 19: Cluster G7 — fold 0 re-run with fix

- [ ] **Step 1: Submit fold 0 walk-forward at new SHA**

```bash
./scripts/argo-alpha-rl.sh \
  --sha <new-sha> --branch main --gpu-pool ci-training-l40s \
  --n-steps 20000 --seq-len 32 --n-backtests 1024 --per-capacity 32768 \
  --seed 16962 --instrument-mode all \
  --n-folds 3 --fold-idx 0 --n-eval-steps 2000 \
  --skip-push-check
```

- [ ] **Step 2: Verify controller release at step 19999**

Inspect diag.jsonl. All 8 Group A controllers should be off their extrema. PPO ε ≤ 0.20, reward_scale ≥ 0.5, gamma > 0.97, q_distill_lambda > 0.05.

- [ ] **Step 3: G8 — eval pnl improvement check**

Pull `eval_summary.json`. Assert eval pnl > -$0.5M, profit_factor > 0.85, win_rate > 0.40.

---

### Task 20: Cluster G9 — ksll2 regression test

- [ ] **Step 1: Submit single-window 20k at new SHA**

Same params as the original `alpha-rl-ksll2` (n_folds=1, n_eval_steps=0).

- [ ] **Step 2: G9 verdict**

Final pnl ≥ +$15M (within 20% of original +$18.3M). Confirms fix doesn't regress large-data regime.

---

## Self-review

This plan was reviewed against the spec at `docs/superpowers/specs/2026-05-30-adaptive-controller-floor-design.md`. Cross-checks:

- **Coverage**: every controller in spec scope (12) appears as a task. Group A in Tasks 5-9 (per-controller patterns), Group B in Tasks 10-13. The fused kernel update (Task 14) replicates per-controller logic. ✓
- **Placeholder scan**: every step has a code block or exact command. No "TBD"/"TODO"/"implement details". ✓
- **Type consistency**: ISV slot names match exactly between Task 1 (allocation) and Tasks 5-13 (consumption). ✓
- **Atomic commit discipline**: only Task 18 commits. All earlier tasks build incrementally without committing. ✓

Known open items (from spec Q6): the gamma adaptive_min update period (500 steps proposed). Implementation per Task 7 uses 500 — if cluster G7 reveals feedback-loop issues, adjust to exponential cool-down in a follow-up.

---

## Execution

Use **subagent-driven-development** (recommended) — dispatch a fresh subagent per task, two-stage review (spec compliance, code quality), atomic single commit at Task 18.
