# SP11 Reward as Controlled Subsystem — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Promote every reward-path degree of freedom (component weights, shaping multipliers, exploration bonus, saboteur curriculum) to ISV-driven slots controlled by a unified Z-score-based controller. Replace hardcoded `cf_weight=0.3f` and similar shaping constants with controller outputs. Add curiosity bonus as a NEW reward dimension applied at *replay time* (not collect-time) to escape the ep1-overfitting fixed point observed in T10 train-multi-seed-xkjkb.

**Architecture:** 20 new ISV slots [340..360) populated by 4 new GPU producer kernels (`val_sharpe_delta_compute_kernel`, `saboteur_engagement_compute_kernel`, `reward_component_mag_ratio_compute_kernel`, `reward_subsystem_controller_kernel`). All producers chained on their stream with `apply_pearls_ad_kernel` for Pearls A+D smoothing per `feedback_no_cpu_compute_strict`. State-action novelty via SimHash 42×16 → 16-bit code, 1M-slot device hash table reset per fold. Three layers: Layer A (3 additive commits — slots, canaries, controller), Layer B (1 atomic consumer-migration commit), Layer C (smoke + T10 validation + close-out).

**Tech Stack:** Rust 1.85+, CUDA 12.4, cudarc, mapped-pinned memory (`MappedF32Buffer`/`MappedI32Buffer`), block-shared-memory tree-reduce (no atomicAdd per `feedback_no_atomicadd`), GPU-only Pearls A+D via `apply_pearls_ad_kernel` (no host compute per `feedback_no_cpu_compute_strict`), L40S smoke validation via Argo Workflows.

**Spec source:** `docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md` (HEAD `9395b983c`)

**Branch:** `sp11-reward-as-controlled-subsystem` from main (HEAD `9395b983c`)

**Operating principles (from MEMORY.md):**
- `feedback_no_cpu_compute_strict` — every producer GPU; Pearls A+D applied via chained `apply_pearls_ad_kernel`
- `feedback_no_atomicadd` — block tree-reduce only
- `feedback_no_partial_refactor` — Layer B consumer migration is atomic in one commit
- `feedback_isv_for_adaptive_bounds` — only Invariant-1 numerical-stability anchors as constants (the seven listed in spec §3.4.2)
- `feedback_no_htod_htoh_only_mapped_pinned` — mapped-pinned only for CPU↔GPU; no HtoD/HtoH
- `feedback_no_cpu_test_fallbacks` — GPU oracle testing only
- `feedback_no_stubs` — every code path wired or deleted; no `unimplemented!()`, no return-zero stubs
- `feedback_no_todo_fixme` — no TODO/FIXME markers

---

## File Structure

### New files (Layer A)

| File | Layer | Responsibility |
|---|---|---|
| `crates/ml/src/cuda_pipeline/sp11_isv_slots.rs` | A0 | Define 20 SP11 ISV slot index constants + Invariant-1 fraction constants |
| `crates/ml/src/cuda_pipeline/val_sharpe_delta_compute_kernel.cu` | A1 | Two-pass delta_ema + delta_var_ema producer |
| `crates/ml/src/cuda_pipeline/saboteur_engagement_compute_kernel.cu` | A1 | Per-bar engagement reduce + EMA |
| `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu` | A1 | Read 6 grad EMAs → normalized ratios |
| `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu` | A2 | Main controller: 5 canaries → 10 outputs (scratch) |
| `crates/ml/src/cuda_pipeline/novelty_simhash_kernel.cu` | A2 | SimHash 42×16 projection → bucket count update + lookup |
| `crates/ml/tests/sp11_producer_unit_tests.rs` | A1+A2 | 6+ GPU-gated unit tests |

### Modified files (Layer A)

| File | Layer | Change |
|---|---|---|
| `crates/ml/src/cuda_pipeline/sp5_isv_slots.rs` | A0 | Bump `SP5_SLOT_END = 360`; add `pub use` re-exports of SP11 constants |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | A1+A2 | Cubin includes; 4 launcher methods; novelty hash buffer field; `MappedF32Buffer` for canary scratch |
| `crates/ml/src/trainers/dqn/state_reset_registry.rs` | A0 | 20 SP11 reset entries + novelty-hash reset arm |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs` | A1+A2 | Cold-path producer launches in epoch boundary + per-step controller launch |
| `crates/ml/build.rs` | A1+A2 | Cubin registration for 5 new kernels (4 producers + novelty) |

### Modified files (Layer B)

| File | Consumer migration |
|---|---|
| `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` | Replace `float cf_weight = 0.3f` (line 318) with `isv[REWARD_CF_WEIGHT_INDEX]` |
| `crates/ml/src/cuda_pipeline/c51_loss_kernel.cu` | Replace `float cf_weight = 0.3f` (line 789) with `isv[REWARD_CF_WEIGHT_INDEX]` |
| `crates/ml/src/cuda_pipeline/{popart,trail,micro,opp_cost,bonus}*_kernel.cu` | Audit-driven: replace each hardcoded weight with corresponding ISV slot |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | `:3328` — multiply `saboteur_perturbation_scale` by `isv[SABOTEUR_INTENSITY_MULT_INDEX]` |
| `crates/ml/src/cuda_pipeline/replay_sample_kernel.cu` (or equivalent) | Add curiosity bonus at sample time: `r_with_curiosity = r_base + isv[CURIOSITY_PRESSURE] * novelty_signal(state, action)` |

### Documentation files (Layer C)

| File | Layer | Content |
|---|---|---|
| `docs/dqn-wire-up-audit.md` | C3 | Add Fix 39 entry under SP11 with architecture summary + before/after metrics |
| `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_reward_as_controlled_subsystem.md` | C4 | New pearl |
| `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/MEMORY.md` | C5 | Add index entry for new pearl |

---

## Layer A — Additive infrastructure (no behavior change)

### Task A0: ISV slot allocation + state-reset registry

**Goal:** Add 20 ISV slot constants (340..360) and 20 FoldReset entries plus novelty-hash reset arm. No producer logic, no consumer changes.

**Files:**
- Create: `crates/ml/src/cuda_pipeline/sp11_isv_slots.rs`
- Modify: `crates/ml/src/cuda_pipeline/sp5_isv_slots.rs:282-340` (bump SP5_SLOT_END to 360)
- Modify: `crates/ml/src/trainers/dqn/state_reset_registry.rs` (append 20 entries + novelty-hash arm)

- [ ] **Step 1: Create `sp11_isv_slots.rs` with all 20 constants**

```rust
// crates/ml/src/cuda_pipeline/sp11_isv_slots.rs
//! SP11 — Reward-subsystem controller ISV slot constants.
//!
//! Slots [340..360) populated by the SP11 controller chain:
//!   - 6 component weights [340..346)
//!   - 4 controller scalar outputs [346..350)
//!   - 2 val-sharpe canaries [350..352)
//!   - 6 mag-ratio canaries [352..358)
//!   - 1 saboteur engagement (358)
//!   - 1 PnL magnitude EMA (359)
//!
//! Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md
//! Pearl: pearl_reward_as_controlled_subsystem.md

// Component weights (controller outputs, consumed by reward shaping kernels)
pub const REWARD_POPART_WEIGHT_INDEX:    usize = 340;
pub const REWARD_CF_WEIGHT_INDEX:        usize = 341;
pub const REWARD_TRAIL_WEIGHT_INDEX:     usize = 342;
pub const REWARD_MICRO_WEIGHT_INDEX:     usize = 343;
pub const REWARD_OPP_COST_WEIGHT_INDEX:  usize = 344;
pub const REWARD_BONUS_WEIGHT_INDEX:     usize = 345;

// Controller scalar outputs
pub const CURIOSITY_PRESSURE_INDEX:      usize = 346;
pub const SABOTEUR_INTENSITY_MULT_INDEX: usize = 347;
pub const REWARD_WEIGHT_FLOOR_INDEX:     usize = 348;
pub const CURIOSITY_BOUND_INDEX:         usize = 349;

// Canaries — val-sharpe trend + variance
pub const VAL_SHARPE_DELTA_EMA_INDEX:    usize = 350;
pub const VAL_SHARPE_VAR_EMA_INDEX:      usize = 351;

// Canaries — per-component reward-magnitude ratios (6 contiguous slots).
// Source: REWARD_POPART_EMA_INDEX..+6 (existing per-component reward
// magnitude EMAs from reward_component_ema_kernel). Spec §3.3.1 said
// "grad EMA" but the SP4 grad-balancer is per-branch (4 slots), not
// per-reward-component. Per `feedback_trust_code_not_docs`: code wins,
// terminology corrected. Semantics for the controller are equivalent —
// "which component contributes most to total reward" maps the same way
// in the winner_weight / diversifier_weight blend.
pub const REWARD_COMPONENT_MAG_RATIO_BASE: usize = 352;
pub const REWARD_COMPONENT_MAG_RATIO_COUNT: usize = 6;

// Canaries — saboteur + PnL signal scale
pub const SABOTEUR_ENGAGEMENT_RATE_INDEX: usize = 358;
pub const PNL_REWARD_MAGNITUDE_EMA_INDEX: usize = 359;

// Invariant-1 anchors (rate-limiters, NOT regime thresholds)
pub const SP11_EPS_DIV: f32 = 1e-6;
pub const SP11_WEIGHT_HARD_FLOOR: f32 = 0.01;
pub const SP11_SABOTEUR_MIN: f32 = 0.5;
pub const SP11_SABOTEUR_MAX: f32 = 2.0;
pub const SP11_CURIOSITY_PERMANENT_FRACTION: f32 = 0.2;
pub const SP11_CURIOSITY_BOUND_FRACTION: f32 = 0.3;
pub const SP11_WEIGHT_FLOOR_FRACTION: f32 = 0.5;
pub const SP11_ENGAGEMENT_FLOOR: f32 = 0.1;

pub const SP11_SLOT_BASE: usize = 340;
pub const SP11_SLOT_END:  usize = 360;
pub const SP11_PRODUCER_COUNT: usize = SP11_SLOT_END - SP11_SLOT_BASE;
```

- [ ] **Step 2: Bump `SP5_SLOT_END` to 360 in `sp5_isv_slots.rs`**

Modify the constant at `crates/ml/src/cuda_pipeline/sp5_isv_slots.rs:284` from `340` to `360`. Also update the layout-fingerprint string and the `assert_eq!` lines (around lines 585-636) that check `SP5_SLOT_END == 340` to `SP5_SLOT_END == 360`. Add a `pub use` re-export of `crate::cuda_pipeline::sp11_isv_slots::*` near the top of the file so existing consumers can still import slot constants from one location.

- [ ] **Step 3: Register `sp11_isv_slots` module**

In `crates/ml/src/cuda_pipeline/mod.rs`, add:
```rust
pub mod sp11_isv_slots;
```

- [ ] **Step 4: Append 20 reset entries to `state_reset_registry.rs`**

Inside the `populate_reset_entries` function (or equivalent), append 20 new `ResetEntry` records following the SP10 pattern at line 764. Each entry has `category: ResetCategory::FoldReset`, sentinel `0.0_f32`, and a description string referencing the spec. Pattern for entry:

```rust
entries.push(ResetEntry {
    name: "sp11_reward_popart_weight",
    slot: REWARD_POPART_WEIGHT_INDEX,
    category: ResetCategory::FoldReset,
    sentinel_f32: 0.0,
    description: "ISV[REWARD_POPART_WEIGHT_INDEX=340] — SP11 popart reward component weight. Produced by reward_subsystem_controller_kernel from mag-ratio canaries and improvement_z. Consumed in reward composition. FoldReset sentinel 0; Pearl A bootstraps from first controller emit (per pearl_first_observation_bootstrap.md).",
});
```

Add equivalent entries for all 20 slots: 6 component weights, 4 controller outputs (curiosity_pressure, saboteur_intensity_mult, reward_weight_floor, curiosity_bound), 2 val-sharpe canaries, 6 mag-ratio canaries (loop), saboteur_engagement_rate, pnl_reward_magnitude_ema.

- [ ] **Step 5: Add novelty-hash reset arm**

The novelty-hash table is NOT an ISV slot but a separate device buffer. Add a parallel `NoveltyHashReset` arm to the state-reset dispatcher. In the dispatcher's match statement (look for `ResetCategory::FoldReset` handling), add a sibling category. Wire dispatch through `gpu_dqn_trainer.rs` so the `novelty_hash_buf.fill_zero_async(stream)?` is invoked at fold boundary. The hash buffer field will be added in Task A2; this step just adds the registry slot/dispatch arm.

- [ ] **Step 6: Build verification**

Run:
```bash
SQLX_OFFLINE=true cargo check -p ml --lib 2>&1 | tail -20
```
Expected: clean compile (no warnings about unused constants — they're `pub`).

- [ ] **Step 7: Contract tests pass**

Run:
```bash
SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots 2>&1 | tail -10
SQLX_OFFLINE=true cargo test -p ml --lib state_reset_registry 2>&1 | tail -10
```
Expected: all assert_eq! constants line up to 360, all 20 reset entries register without duplicate-slot panics.

- [ ] **Step 8: Commit A0**

```bash
git checkout -b sp11-reward-as-controlled-subsystem
git add crates/ml/src/cuda_pipeline/sp11_isv_slots.rs \
        crates/ml/src/cuda_pipeline/sp5_isv_slots.rs \
        crates/ml/src/cuda_pipeline/mod.rs \
        crates/ml/src/trainers/dqn/state_reset_registry.rs
git commit -m "$(cat <<'EOF'
feat(sp11): A0 — allocate 20 ISV slots [340..360) + 20 reset entries

Pure infrastructure. No producer kernels, no consumer reads. Existing
training paths trace identically because no consumer reads slots [340..360)
yet. Layout-fingerprint bumped to ISV_TOTAL_DIM=360.

Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md
EOF
)"
```

---

### Task A1: Three canary producer kernels

**Goal:** Implement 3 canary kernels that populate the 9 canary slots (delta_ema, delta_var_ema, 6 mag-ratios, engagement_rate, pnl_magnitude_ema). Each kernel chains `apply_pearls_ad_kernel` on its stream for Pearls A+D smoothing. Wire launchers; the controller (A2) will read these. Existing training continues to behave identically because no consumer reads canary slots in production paths.

**Files:**
- Create: `crates/ml/src/cuda_pipeline/val_sharpe_delta_compute_kernel.cu`
- Create: `crates/ml/src/cuda_pipeline/saboteur_engagement_compute_kernel.cu`
- Create: `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu`
- Create: `crates/ml/tests/sp11_producer_unit_tests.rs` (skeleton + first 3 tests)
- Modify: `crates/ml/build.rs` (cubin registration for 3 kernels)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (cubin includes, 3 launchers, scratch buffers)
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (epoch-boundary launches)

#### A1.1 — val_sharpe_delta_compute_kernel

- [ ] **Step 1: Write the kernel file**

```cuda
// crates/ml/src/cuda_pipeline/val_sharpe_delta_compute_kernel.cu
// SP11 — val-sharpe delta + variance canary producer.
//
// Reads val_sharpe history (last 2 emits + prev delta_ema) from scratch;
// writes delta_ema (slot 350) and delta_var_ema (slot 351) via Pearls A+D.
//
// Single block, ≤32 threads. Two-pass formulation: first updates delta_ema
// against the new sample, then accumulates squared-deviation-from-updated-mean
// into delta_var_ema.
//
// Pearl A: sentinel 0.0 → first observation replaces directly.
// Pearl D: Wiener-α smoothing inside apply_pearls_ad_kernel chained on stream.

#include <cuda_runtime.h>

extern "C" __global__ void val_sharpe_delta_compute_kernel(
    const float* __restrict__ val_sharpe_history,  // [2]: [t-1, t] in mapped pinned
    float* __restrict__ scratch_out,                // [2]: scratch_delta, scratch_var
    const float* __restrict__ isv,                  // for current delta_ema reading
    const int delta_ema_slot,
    const int var_ema_slot)
{
    if (blockIdx.x != 0 || threadIdx.x != 0) return;

    const float prev = val_sharpe_history[0];
    const float curr = val_sharpe_history[1];
    const float delta = curr - prev;

    // Pass 1: deviation against PREVIOUS delta_ema (the value Pearl D will then update).
    const float prev_delta_ema = isv[delta_ema_slot];
    const float dev = delta - prev_delta_ema;

    scratch_out[0] = delta;            // raw delta → apply_pearls_ad updates slot delta_ema_slot
    scratch_out[1] = dev * dev;        // squared deviation → apply_pearls_ad updates slot var_ema_slot
}
```

- [ ] **Step 2: Register cubin in `build.rs`**

In `crates/ml/build.rs`, find the existing list of `cuda_kernels` (look for `intent_eval_divergence_compute_kernel` to find the section). Append:

```rust
"val_sharpe_delta_compute_kernel",
```

- [ ] **Step 3: Write GPU oracle unit test**

```rust
// crates/ml/tests/sp11_producer_unit_tests.rs
//
// IMPORTANT: per `feedback_no_htod_htoh_only_mapped_pinned`, tests MUST use
// `MappedF32Buffer` / `MappedI32Buffer` for all CPU↔GPU transfers. Direct
// `htod_copy` / `dtoh_sync_copy` is forbidden even in tests. Pattern below
// matches existing SP4 unit tests (see crates/ml/src/cuda_pipeline/mod.rs:372).

#![cfg(feature = "cuda")]

use cudarc::driver::*;
use ml::cuda_pipeline::sp11_isv_slots::*;
use ml::cuda_pipeline::mapped_pinned::MappedF32Buffer;

#[test]
fn val_sharpe_delta_first_observation_writes_raw_delta() {
    let dev = CudaDevice::new(0).unwrap();
    let cubin = include_bytes!(concat!(env!("OUT_DIR"), "/val_sharpe_delta_compute_kernel.cubin"));
    dev.load_ptx(cubin.into(), "module", &["val_sharpe_delta_compute_kernel"]).unwrap();
    let f = dev.get_func("module", "val_sharpe_delta_compute_kernel").unwrap();

    // All buffers via MappedF32Buffer (mapped-pinned host-allocated, DEVICEMAP
    // visible). Writes from host go through the device-mapped pointer; reads
    // from device come back through the same mapping. No HtoD/DtoH copies.
    let mut history = unsafe { MappedF32Buffer::new(2).unwrap() };
    history.host_slice_mut()[0] = 10.0;
    history.host_slice_mut()[1] = 15.0;  // delta = 5.0

    let mut scratch = unsafe { MappedF32Buffer::new(2).unwrap() };
    let isv = unsafe { MappedF32Buffer::new(360).unwrap() };  // all zeros

    unsafe {
        f.launch(LaunchConfig { grid_dim: (1,1,1), block_dim: (1,1,1), shared_mem_bytes: 0 },
            (history.device_ptr(), scratch.device_ptr_mut(), isv.device_ptr(),
             VAL_SHARPE_DELTA_EMA_INDEX as i32, VAL_SHARPE_VAR_EMA_INDEX as i32))
            .unwrap();
    }
    dev.synchronize().unwrap();

    let scratch_view = scratch.host_slice();
    assert!((scratch_view[0] - 5.0).abs() < 1e-6, "raw delta should be 5.0, got {}", scratch_view[0]);
    assert!((scratch_view[1] - 25.0).abs() < 1e-4, "squared deviation against zero ema = 25.0, got {}", scratch_view[1]);
}
```

- [ ] **Step 4: Run kernel test**

```bash
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp11_producer_unit_tests --features cuda val_sharpe_delta_first_observation 2>&1 | tail -20
```
Expected: PASS on RTX 3050 Ti.

- [ ] **Step 5: Add Rust launcher in `gpu_dqn_trainer.rs`**

Find the section near line 503 (where other cubins are included). Add:
```rust
const VAL_SHARPE_DELTA_COMPUTE_CUBIN: &[u8] =
    include_bytes!(concat!(env!("OUT_DIR"), "/val_sharpe_delta_compute_kernel.cubin"));
```

Add a method following the pattern at line 199 (`apply_pearls_ad_kernel` chained launcher pattern):
```rust
/// Launch val-sharpe delta canary kernel + chained Pearls A+D for SP11.
/// Reads val_sharpe_history (mapped pinned [2-element]), writes scratch tuple,
/// then chains apply_pearls_ad_kernel(n_slots=2) targeting slots
/// VAL_SHARPE_DELTA_EMA_INDEX and VAL_SHARPE_VAR_EMA_INDEX on the same stream.
pub(crate) fn launch_val_sharpe_delta_compute(
    &self,
    val_sharpe_history: &MappedF32Buffer,
    stream: &CudaStream,
) -> Result<()> {
    // ... follows existing pattern
}
```

The launcher allocates a 2-element scratch via `MappedF32Buffer`, calls the kernel, then immediately schedules `apply_pearls_ad_kernel` with `slot_indices=[VAL_SHARPE_DELTA_EMA_INDEX, VAL_SHARPE_VAR_EMA_INDEX]` on the same stream.

- [ ] **Step 6: Wire launch into `training_loop.rs`**

In the val-emit section of the training loop (search for `val_sharpe` write sites), invoke:
```rust
self.gpu.launch_val_sharpe_delta_compute(&self.val_sharpe_history_pinned, &self.stream)?;
```
The `val_sharpe_history_pinned` is a 2-element `MappedF32Buffer` field on the trainer (add it as part of this step). Push the previous epoch's val sharpe to slot 0, the just-computed value to slot 1, then launch.

- [ ] **Step 7: Smoke check (build + 1-iter loopback)**

```bash
SQLX_OFFLINE=true cargo build -p ml --release 2>&1 | tail -5
```

Expected: clean build, kernel cubin generated.

#### A1.2 — saboteur_engagement_compute_kernel

- [ ] **Step 8: Write the kernel file**

```cuda
// crates/ml/src/cuda_pipeline/saboteur_engagement_compute_kernel.cu
// SP11 — per-bar saboteur engagement canary producer.
//
// Consumes a single per-bar `saboteur_delta_reward_buf` produced by the
// saboteur perturbation site itself (see step 8a). For each bar:
//   engaged_i = |delta_reward[i]| > EPS_ENGAGEMENT
// where EPS_ENGAGEMENT = 0.01 * isv[PNL_REWARD_MAGNITUDE_EMA_INDEX].
// Reduces via block tree-reduce (no atomicAdd per feedback_no_atomicadd),
// writes batch-aggregated engagement rate to scratch.
// apply_pearls_ad_kernel chained on stream targets slot SABOTEUR_ENGAGEMENT_RATE_INDEX.

#include <cuda_runtime.h>

extern "C" __global__ void saboteur_engagement_compute_kernel(
    const float* __restrict__ saboteur_delta_reward,  // [batch_size]: |r_with - r_without|
    float* __restrict__ scratch_out,                  // [1]: engagement fraction
    const float* __restrict__ isv,
    const int batch_size,
    const int pnl_mag_ema_slot)
{
    extern __shared__ int sdata[];

    const float pnl_mag = fmaxf(isv[pnl_mag_ema_slot], 1e-6f);
    const float eps_engagement = 0.01f * pnl_mag;

    int local_engaged = 0;
    for (int i = threadIdx.x; i < batch_size; i += blockDim.x) {
        if (fabsf(saboteur_delta_reward[i]) > eps_engagement) local_engaged += 1;
    }
    sdata[threadIdx.x] = local_engaged;
    __syncthreads();

    // Block tree-reduce
    for (int s = blockDim.x / 2; s > 0; s >>= 1) {
        if (threadIdx.x < s) sdata[threadIdx.x] += sdata[threadIdx.x + s];
        __syncthreads();
    }

    if (threadIdx.x == 0) {
        scratch_out[0] = (float)sdata[0] / (float)batch_size;
    }
}
```

- [ ] **Step 8a: Add `saboteur_delta_reward_buf` producer in saboteur perturbation site**

The saboteur engagement kernel consumes a per-bar `|reward_with - reward_without|`
buffer. This buffer is produced by the saboteur perturbation site itself,
where both reward values are already available locally. In
`crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (saboteur perturbation
section, around line 3328), the existing reward computation has access to
the perturbed price/slippage. Modify the per-bar reward kernel to compute
both `reward_with_saboteur` (using perturbed values) and `reward_without_saboteur`
(using un-perturbed values) in the same kernel — they share state-extraction
work — and emit `delta = reward_with - reward_without` to a new
`MappedF32Buffer` field `saboteur_delta_reward_buf` on the trainer.

This avoids running two separate reward computations and keeps the diff
local to the kernel that already has both values. Per `feedback_wire_everything_up`,
add the buffer field + its initialization + the producer write site in
this same step (no orphan additions).

- [ ] **Step 9: Register cubin**

Append `"saboteur_engagement_compute_kernel"` to `build.rs` cuda_kernels list.

- [ ] **Step 10: Write GPU oracle test**

In `sp11_producer_unit_tests.rs`:
```rust
#[test]
fn saboteur_engagement_counts_above_threshold_only() {
    let dev = CudaDevice::new(0).unwrap();
    let cubin = include_bytes!(concat!(env!("OUT_DIR"), "/saboteur_engagement_compute_kernel.cubin"));
    dev.load_ptx(cubin.into(), "module", &["saboteur_engagement_compute_kernel"]).unwrap();
    let f = dev.get_func("module", "saboteur_engagement_compute_kernel").unwrap();

    // PNL EMA = 1.0, so threshold = 0.01.
    // 4 bars: |delta_reward| = [0.0, 0.005, 0.05, 0.5]. Only 2 of 4 are > 0.01.
    let mut delta_reward = unsafe { MappedF32Buffer::new(4).unwrap() };
    delta_reward.host_slice_mut().copy_from_slice(&[0.0, 0.005, 0.05, 0.5]);

    let mut scratch = unsafe { MappedF32Buffer::new(1).unwrap() };

    let mut isv = unsafe { MappedF32Buffer::new(360).unwrap() };
    isv.host_slice_mut()[PNL_REWARD_MAGNITUDE_EMA_INDEX] = 1.0;

    unsafe {
        f.launch(LaunchConfig { grid_dim: (1,1,1), block_dim: (32,1,1), shared_mem_bytes: 32*4 },
            (delta_reward.device_ptr(), scratch.device_ptr_mut(), isv.device_ptr(),
             4i32, PNL_REWARD_MAGNITUDE_EMA_INDEX as i32))
            .unwrap();
    }
    dev.synchronize().unwrap();

    assert!((scratch.host_slice()[0] - 0.5).abs() < 1e-4,
            "expected engagement=0.5, got {}", scratch.host_slice()[0]);
}
```

- [ ] **Step 11: Run test + commit launcher + wire in training_loop**

```bash
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp11_producer_unit_tests --features cuda saboteur_engagement 2>&1 | tail -10
```

Add `launch_saboteur_engagement_compute` launcher in `gpu_dqn_trainer.rs` following the same Pearls A+D chained pattern. Wire into `training_loop.rs` per-step path where rewards are computed (the saboteur perturbation site).

#### A1.3 — reward_component_mag_ratio_compute_kernel

- [ ] **Step 12: Write kernel + test + launcher**

```cuda
// crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu
// SP11 — per-component reward-magnitude-ratio canary producer.
//
// Reads 6 existing per-component reward-magnitude EMAs from ISV (popart, cf,
// trail, micro, opp_cost, bonus) at REWARD_POPART_EMA_INDEX..+6. Writes
// normalized ratios (each / sum) to scratch. apply_pearls_ad_kernel chained
// targeting slots [REWARD_COMPONENT_MAG_RATIO_BASE..+6).
//
// Side-output: thread 0 also writes scratch_out[6] = isv[popart_ema_base_slot]
// which apply_pearls_ad chains to PNL_REWARD_MAGNITUDE_EMA_INDEX (slot 359).
// This avoids allocating a separate copy kernel — the popart magnitude IS
// the PnL signal scale used to bound curiosity (spec §3.4.2).
//
// Source EMAs already populated by SP4's reward_component_ema_kernel — this
// kernel just reads and renormalizes (no compute beyond sum + divide).

#include <cuda_runtime.h>

extern "C" __global__ void reward_component_mag_ratio_compute_kernel(
    const float* __restrict__ isv,
    float* __restrict__ scratch_out,           // [7]: 6 ratios + 1 popart mirror
    const int popart_ema_base_slot)            // = REWARD_POPART_EMA_INDEX (63)
{
    if (blockIdx.x != 0 || threadIdx.x >= 6) return;

    __shared__ float values[6];
    __shared__ float sum;

    values[threadIdx.x] = isv[popart_ema_base_slot + threadIdx.x];
    __syncthreads();

    if (threadIdx.x == 0) {
        sum = 0.0f;
        for (int c = 0; c < 6; c++) sum += values[c];
        sum = fmaxf(sum, 1e-6f);
        // Mirror popart magnitude to scratch[6] for PNL_REWARD_MAGNITUDE_EMA chain
        scratch_out[6] = values[0];
    }
    __syncthreads();

    scratch_out[threadIdx.x] = values[threadIdx.x] / sum;
}
```

Register cubin, write a GPU oracle test using `MappedF32Buffer` for ISV
fixture and scratch (per `feedback_no_htod_htoh_only_mapped_pinned`). Test
fixture: synthetic ISV with 6 known per-component magnitude EMAs at
`REWARD_POPART_EMA_INDEX..+6`; assert (a) `scratch_out[0..6]` sums to 1.0
± 1e-5, (b) individual ratios match expected values, (c) `scratch_out[6]`
equals `isv[REWARD_POPART_EMA_INDEX]`. Wire launcher: the chained
`apply_pearls_ad_kernel` consumes scratch_out[7] writing to ISV slots
`[REWARD_COMPONENT_MAG_RATIO_BASE..+6) ∪ {PNL_REWARD_MAGNITUDE_EMA_INDEX}`
in a single n_slots=7 launch. Wire `training_loop.rs` per-step launch.

- [ ] **Step 13: Build full ml crate; verify all 3 canary kernels**

```bash
SQLX_OFFLINE=true cargo build -p ml --release 2>&1 | tail -5
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp11_producer_unit_tests --features cuda 2>&1 | tail -20
```
Expected: 3 tests pass.

- [ ] **Step 14: Commit A1**

```bash
git add crates/ml/src/cuda_pipeline/val_sharpe_delta_compute_kernel.cu \
        crates/ml/src/cuda_pipeline/saboteur_engagement_compute_kernel.cu \
        crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu \
        crates/ml/build.rs \
        crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
        crates/ml/src/trainers/dqn/trainer/training_loop.rs \
        crates/ml/tests/sp11_producer_unit_tests.rs
git commit -m "$(cat <<'EOF'
feat(sp11): A1 — three canary producer kernels (no behavior change)

Adds val_sharpe_delta + saboteur_engagement + reward_component_mag_ratio
GPU producers, each chained with apply_pearls_ad_kernel for Pearls A+D
smoothing per feedback_no_cpu_compute_strict. All write to slots [350..360)
which no consumer reads yet — training continues identically.

3 GPU oracle tests pass on RTX 3050 Ti (CUDA_COMPUTE_CAP=86).

Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md §5
EOF
)"
```

---

### Task A2: Controller producer kernel + novelty hash table

**Goal:** Implement the main controller (10 outputs) + the SimHash novelty buffer. After A2, all 20 SP11 slots populate every step. Still no consumer reads, so trainer behavior is unchanged. HEALTH_DIAG `sp11_reward` line emits the 10 outputs + improvement_z.

**Files:**
- Create: `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu`
- Create: `crates/ml/src/cuda_pipeline/novelty_simhash_kernel.cu`
- Modify: `crates/ml/build.rs` (2 more cubins)
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (cubin includes, launcher, novelty-hash device buffer field, projection matrix init)
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs` (per-step controller launch + HEALTH_DIAG line)
- Modify: `crates/ml/tests/sp11_producer_unit_tests.rs` (3 more tests)

#### A2.1 — controller kernel

- [ ] **Step 1: Write the controller kernel**

```cuda
// crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu
// SP11 — Reward subsystem controller (5 canaries → 10 outputs).
//
// Single block, 10 threads (one output each, threads 0-5 also load shared ratios).
// Reads canary ISV slots [350..360); writes scratch [0..10] which apply_pearls_ad
// then smooths into output slots [340..350).
//
// Spec §3.4. Critical math:
//   - True Z-score: improvement_z = delta_ema / sqrt(delta_var_ema)
//   - Sigmoid maps z → improving ∈ [0, 1], stagnant_or_worse = 1 - improving
//   - Per-component blend, floored, then RENORMALIZED to Σ=1
//   - Curiosity has permanent floor of 0.2 × bound (max(), not blend)
//   - Saboteur clamped POST engagement multiplication

#include <cuda_runtime.h>
#include <math.h>

extern "C" __global__ void reward_subsystem_controller_kernel(
    const float* __restrict__ isv,
    float* __restrict__ scratch_out,    // [10]: 6 weights, curiosity_pressure,
                                        // saboteur_intensity_mult, weight_floor, curiosity_bound
    const int delta_ema_slot,
    const int var_ema_slot,
    const int mag_ratio_base_slot,
    const int saboteur_engagement_slot,
    const int pnl_mag_ema_slot)
{
    if (blockIdx.x != 0) return;
    if (threadIdx.x >= 10) return;

    const float EPS_DIV                       = 1e-6f;
    const float SABOTEUR_MIN                  = 0.5f;
    const float SABOTEUR_MAX                  = 2.0f;
    const float WEIGHT_HARD_FLOOR             = 0.01f;
    const float CURIOSITY_PERMANENT_FRACTION  = 0.2f;
    const float ENGAGEMENT_FLOOR              = 0.1f;
    const float CURIOSITY_BOUND_FRACTION      = 0.3f;
    const float WEIGHT_FLOOR_FRACTION         = 0.5f;

    __shared__ float ratios[6];
    __shared__ float improving;
    __shared__ float stagnant_or_worse;
    __shared__ float adaptive_floor;
    __shared__ float blends[6];
    __shared__ float blend_sum;
    __shared__ float curiosity_bound;

    if (threadIdx.x < 6) {
        ratios[threadIdx.x] = isv[mag_ratio_base_slot + threadIdx.x];
    }
    __syncthreads();

    if (threadIdx.x == 0) {
        // Z-score
        const float delta_ema = isv[delta_ema_slot];
        const float delta_var = isv[var_ema_slot];
        const float dz = delta_ema / fmaxf(sqrtf(delta_var), EPS_DIV);
        improving = 1.0f / (1.0f + expf(-dz));
        stagnant_or_worse = 1.0f - improving;

        // Floor from worst-suppressed component
        float min_grad_ratio = ratios[0];
        for (int c = 1; c < 6; c++) min_grad_ratio = fminf(min_grad_ratio, ratios[c]);
        adaptive_floor = fmaxf(WEIGHT_HARD_FLOOR, WEIGHT_FLOOR_FRACTION * min_grad_ratio);

        curiosity_bound = isv[pnl_mag_ema_slot] * CURIOSITY_BOUND_FRACTION;
    }
    __syncthreads();

    // Each thread 0..5 computes its own blend
    if (threadIdx.x < 6) {
        const float winner = ratios[threadIdx.x];
        const float diversifier = (1.0f - winner) / 5.0f;
        float w = improving * winner + stagnant_or_worse * diversifier;
        w = fmaxf(adaptive_floor, fminf(1.0f, w));
        blends[threadIdx.x] = w;
    }
    __syncthreads();

    if (threadIdx.x == 0) {
        blend_sum = 0.0f;
        for (int c = 0; c < 6; c++) blend_sum += blends[c];
    }
    __syncthreads();

    // Component weights (slots 0-5 of scratch) — renormalized
    if (threadIdx.x < 6) {
        scratch_out[threadIdx.x] = blends[threadIdx.x] / fmaxf(blend_sum, EPS_DIV);
    }

    // Thread 6: curiosity_pressure
    if (threadIdx.x == 6) {
        const float curiosity_floor = CURIOSITY_PERMANENT_FRACTION * curiosity_bound;
        const float curiosity_dynamic = stagnant_or_worse * curiosity_bound;
        scratch_out[6] = fmaxf(curiosity_dynamic, curiosity_floor);
    }

    // Thread 7: saboteur_intensity_mult (pre-clamped, multiplied by engagement, post-clamped)
    if (threadIdx.x == 7) {
        const float engagement = isv[saboteur_engagement_slot];
        const float intensity_z_term = SABOTEUR_MIN + (SABOTEUR_MAX - SABOTEUR_MIN) * improving;
        const float modulated = intensity_z_term * fmaxf(engagement, ENGAGEMENT_FLOOR);
        scratch_out[7] = fmaxf(SABOTEUR_MIN, fminf(SABOTEUR_MAX, modulated));
    }

    // Thread 8: weight_floor (write the value already computed)
    if (threadIdx.x == 8) {
        scratch_out[8] = adaptive_floor;
    }

    // Thread 9: curiosity_bound (write the value already computed)
    if (threadIdx.x == 9) {
        scratch_out[9] = curiosity_bound;
    }
}
```

- [ ] **Step 2: Register cubin**

Append `"reward_subsystem_controller_kernel"` to `build.rs`.

- [ ] **Step 3: Write GPU oracle tests**

In `sp11_producer_unit_tests.rs`, add (all fixtures via `MappedF32Buffer`
per `feedback_no_htod_htoh_only_mapped_pinned`):

```rust
fn load_controller_kernel() -> (CudaDevice, CudaFunction) {
    let dev = CudaDevice::new(0).unwrap();
    let cubin = include_bytes!(concat!(env!("OUT_DIR"), "/reward_subsystem_controller_kernel.cubin"));
    dev.load_ptx(cubin.into(), "module", &["reward_subsystem_controller_kernel"]).unwrap();
    let f = dev.get_func("module", "reward_subsystem_controller_kernel").unwrap();
    (dev, f)
}

fn populate_controller_isv(
    isv: &mut MappedF32Buffer,
    delta: f32, var: f32, ratios: [f32; 6],
    saboteur_engagement: f32, pnl_mag: f32,
) {
    let h = isv.host_slice_mut();
    h[VAL_SHARPE_DELTA_EMA_INDEX] = delta;
    h[VAL_SHARPE_VAR_EMA_INDEX] = var;
    for c in 0..6 {
        h[REWARD_COMPONENT_MAG_RATIO_BASE + c] = ratios[c];
    }
    h[SABOTEUR_ENGAGEMENT_RATE_INDEX] = saboteur_engagement;
    h[PNL_REWARD_MAGNITUDE_EMA_INDEX] = pnl_mag;
}

fn launch_controller(
    dev: &CudaDevice, f: &CudaFunction,
    isv: &MappedF32Buffer, scratch: &mut MappedF32Buffer,
) {
    unsafe {
        f.launch(
            LaunchConfig { grid_dim: (1,1,1), block_dim: (10,1,1), shared_mem_bytes: 0 },
            (isv.device_ptr(), scratch.device_ptr_mut(),
             VAL_SHARPE_DELTA_EMA_INDEX as i32,
             VAL_SHARPE_VAR_EMA_INDEX as i32,
             REWARD_COMPONENT_MAG_RATIO_BASE as i32,
             SABOTEUR_ENGAGEMENT_RATE_INDEX as i32,
             PNL_REWARD_MAGNITUDE_EMA_INDEX as i32),
        ).unwrap();
    }
    dev.synchronize().unwrap();
}

#[test]
fn controller_z_score_at_zero_yields_midpoint_outputs() {
    // delta=0, var=1 → dz=0 → sigmoid(0)=0.5 → improving=stagnant=0.5
    // All 6 ratios = 1/6, post-floor renorm preserves uniform weights.
    // curiosity_bound = pnl_mag × 0.3 = 0.3; curiosity_floor = 0.06;
    //   curiosity_dynamic = 0.5 × 0.3 = 0.15 → max = 0.15
    // saboteur intensity_z = 0.5 + 1.5 × 0.5 = 1.25; engagement = 0.5
    //   modulated = 1.25 × 0.5 = 0.625; post-clamp = 0.625 (in range)
    let (dev, f) = load_controller_kernel();
    let mut isv = unsafe { MappedF32Buffer::new(360).unwrap() };
    populate_controller_isv(&mut isv, 0.0, 1.0, [1.0/6.0; 6], 0.5, 1.0);
    let mut scratch = unsafe { MappedF32Buffer::new(10).unwrap() };
    launch_controller(&dev, &f, &isv, &mut scratch);

    let s = scratch.host_slice();
    let weight_sum: f32 = s[0..6].iter().sum();
    assert!((weight_sum - 1.0).abs() < 1e-5, "weights sum = {}, expected 1.0", weight_sum);
    for c in 0..6 {
        assert!((s[c] - 1.0/6.0).abs() < 1e-4, "weight[{}] = {}, expected 1/6", c, s[c]);
    }
    assert!((s[6] - 0.15).abs() < 1e-4, "curiosity_pressure = {}, expected 0.15", s[6]);
    assert!((s[7] - 0.625).abs() < 1e-3, "saboteur = {}, expected 0.625", s[7]);
    assert!((s[9] - 0.3).abs() < 1e-5, "curiosity_bound = {}, expected 0.3", s[9]);
}

#[test]
fn controller_weights_renormalize_after_floor() {
    // Pathological ratios: one large, five near-zero. Floor activates,
    // but renormalization restores Σ=1.
    let (dev, f) = load_controller_kernel();
    let mut isv = unsafe { MappedF32Buffer::new(360).unwrap() };
    populate_controller_isv(&mut isv, 1.0, 1.0,
        [0.95, 0.01, 0.01, 0.01, 0.01, 0.01], 0.5, 1.0);
    let mut scratch = unsafe { MappedF32Buffer::new(10).unwrap() };
    launch_controller(&dev, &f, &isv, &mut scratch);

    let s = scratch.host_slice();
    let weight_sum: f32 = s[0..6].iter().sum();
    assert!((weight_sum - 1.0).abs() < 1e-5,
            "weights sum = {} after renormalization, expected 1.0", weight_sum);
    // Each weight at least the WEIGHT_HARD_FLOOR (0.01) post-renorm
    for c in 0..6 {
        assert!(s[c] >= 0.01 - 1e-5, "weight[{}] = {} below hard floor", c, s[c]);
    }
}

#[test]
fn controller_saboteur_post_clamp_holds_min() {
    // engagement=0 (will be floored to 0.1), improving very low.
    // delta=-100, var=1 → dz=-100 → sigmoid≈0 → improving≈0 → stagnant≈1
    // intensity_z = 0.5 + 1.5×0 = 0.5
    // modulated = 0.5 × max(0.0, 0.1) = 0.05 < SABOTEUR_MIN
    // post-clamp = 0.5 (SABOTEUR_MIN)
    let (dev, f) = load_controller_kernel();
    let mut isv = unsafe { MappedF32Buffer::new(360).unwrap() };
    populate_controller_isv(&mut isv, -100.0, 1.0, [1.0/6.0; 6], 0.0, 1.0);
    let mut scratch = unsafe { MappedF32Buffer::new(10).unwrap() };
    launch_controller(&dev, &f, &isv, &mut scratch);

    let s = scratch.host_slice();
    assert!((s[7] - 0.5).abs() < 1e-4,
            "saboteur = {}, expected SABOTEUR_MIN=0.5 after post-clamp", s[7]);

    // Curiosity should be near max (stagnant_or_worse ≈ 1)
    // curiosity_bound = 0.3, curiosity_dynamic ≈ 0.3, curiosity_floor = 0.06
    // pressure = max(0.3, 0.06) = 0.3
    assert!((s[6] - 0.3).abs() < 1e-3,
            "curiosity_pressure at max regression = {}, expected ≈ 0.3", s[6]);
}
```

- [ ] **Step 4: Run tests**

```bash
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo test -p ml --test sp11_producer_unit_tests --features cuda controller 2>&1 | tail -15
```
Expected: 3 tests PASS.

- [ ] **Step 5: Add Rust launcher**

In `gpu_dqn_trainer.rs`, add include + launcher method `launch_reward_subsystem_controller`. The launcher allocates a 10-element scratch via `MappedF32Buffer`, calls the kernel, then schedules `apply_pearls_ad_kernel` with `slot_indices=[REWARD_POPART_WEIGHT_INDEX, REWARD_CF_WEIGHT_INDEX, ..., CURIOSITY_BOUND_INDEX]` (10 slots) on the same stream.

#### A2.2 — novelty SimHash buffer + kernel

- [ ] **Step 6: Add novelty hash buffer field to GpuDqnTrainer**

Per `feedback_no_cpu_forwards` ("CPU is read-only"), the projection
matrix MUST be populated by a GPU kernel (Philox-driven), NOT by host
RNG writing through `host_slice_mut`. The hash buffer is allocated
zero-initialized (cuMemHostAlloc DEVICEMAP zero-fills); the projection
matrix is allocated then immediately filled by a one-shot GPU init
kernel reading a seed scalar.

In `gpu_dqn_trainer.rs`, add the field + the init-kernel cubin:

```rust
/// SP11 novelty visit-count hash table. 1M slots × f32 (4 MB mapped-pinned).
/// SimHash projection 42×16 → 16-bit code; bucket = (action_id, code).
/// Reset at fold boundary via state-reset registry novelty arm
/// (zero-init kernel writes through device pointer; no DtoH/HtoD).
novelty_hash_buf: mapped_pinned::MappedF32Buffer,
/// SP11 SimHash projection matrix (42×16 = 672 floats). Populated ONCE at
/// trainer construct by `novelty_simhash_proj_init_kernel` (Philox-driven
/// from `config.seed`). Never re-initialized; never written from host.
novelty_simhash_proj: mapped_pinned::MappedF32Buffer,
```

Add a small GPU init kernel `novelty_simhash_proj_init_kernel.cu` (≤30 LOC):

```cuda
// crates/ml/src/cuda_pipeline/novelty_simhash_proj_init_kernel.cu
// SP11 — one-shot GPU init for the SimHash projection matrix.
// Philox(seed) → uniform u32 → low bit picks +1 or -1.
// Per `feedback_no_cpu_forwards` the projection MUST be GPU-populated.

#include <cuda_runtime.h>
#include "philox_inline.cuh"  // existing project helper

extern "C" __global__ void novelty_simhash_proj_init_kernel(
    float* __restrict__ proj,           // [42 × 16 = 672]
    const unsigned long long seed,
    const int n_floats)
{
    const int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= n_floats) return;
    const unsigned int u = philox_uint(seed, (unsigned int)i, 0u);
    proj[i] = (u & 1u) ? 1.0f : -1.0f;
}
```

Initialize in trainer constructor:
```rust
let novelty_hash_buf = unsafe { mapped_pinned::MappedF32Buffer::new(1 << 20)? };
// cuMemHostAlloc DEVICEMAP zero-initializes — matches state-reset semantics.

let novelty_simhash_proj = unsafe { mapped_pinned::MappedF32Buffer::new(42 * 16)? };
// One-shot GPU init using a Philox stream seeded from config.seed.
// No host write; the matrix is populated on-device.
self.launch_novelty_simhash_proj_init(
    &novelty_simhash_proj,
    config.seed.wrapping_add(SP11_PROJ_SEED_SALT),
    &self.stream,
)?;
```

Where `SP11_PROJ_SEED_SALT: u64 = 0x_5511_0001` is defined in
`sp11_isv_slots.rs` alongside the other SP11 constants. The launcher
method `launch_novelty_simhash_proj_init` runs the init kernel with
grid 21 × block 32 (covers 672 elements).

Register cubin in `build.rs`. The kernels read/write through
`device_ptr()` / `device_ptr_mut()`. No `htod_copy`, no `dtoh_sync_copy`,
no `alloc_zeros`, no host RNG. Per `feedback_wire_everything_up` the
init kernel is wired to the constructor that allocates the buffer —
no orphan addition.

- [ ] **Step 7: Write SimHash kernel**

```cuda
// crates/ml/src/cuda_pipeline/novelty_simhash_kernel.cu
// SP11 — SimHash novelty signal.
//
// Two functions:
//   1. novelty_simhash_lookup_kernel — reads bucket count, returns 1/sqrt(1+count)
//   2. novelty_simhash_update_kernel — increments bucket count after replay sample
//
// Project state through a fixed 42×16 random projection matrix; sign-quantize
// to a 16-bit code. Bucket = (action_id × 65536 + code) mod TABLE_SIZE.
// TABLE_SIZE = 1 << 20 (1M slots, linear-probe collision tolerated; collisions
// underestimate novelty, the safe direction).

#include <cuda_runtime.h>

#define TABLE_SIZE (1 << 20)
#define TABLE_MASK ((1 << 20) - 1)

__device__ uint32_t simhash_code(
    const float* __restrict__ state,         // [42]
    const float* __restrict__ proj_matrix)   // [42 × 16]
{
    uint32_t code = 0;
    for (int b = 0; b < 16; b++) {
        float dot = 0.0f;
        for (int d = 0; d < 42; d++) {
            dot += state[d] * proj_matrix[d * 16 + b];
        }
        if (dot > 0.0f) code |= (1u << b);
    }
    return code;
}

extern "C" __global__ void novelty_simhash_lookup_kernel(
    const float* __restrict__ states,         // [batch_size × 42]
    const int*   __restrict__ actions,        // [batch_size]
    const float* __restrict__ proj_matrix,    // [42 × 16]
    const float* __restrict__ hash_table,     // [TABLE_SIZE]
    float*       __restrict__ novelty_out,    // [batch_size]
    const int batch_size)
{
    const int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= batch_size) return;

    const uint32_t code = simhash_code(&states[i * 42], proj_matrix);
    const uint32_t bucket = ((uint32_t)actions[i] * 65536u + code) & TABLE_MASK;
    const float count = hash_table[bucket];
    novelty_out[i] = 1.0f / sqrtf(1.0f + count);
}

extern "C" __global__ void novelty_simhash_update_kernel(
    const float* __restrict__ states,
    const int*   __restrict__ actions,
    const float* __restrict__ proj_matrix,
    float*       __restrict__ hash_table,
    const int batch_size)
{
    // Each thread processes one (state, action). Per `feedback_no_atomicadd`,
    // we forbid atomicAdd. Concurrent writers to the same bucket race;
    // last-writer-wins loses increments. This biases counts DOWNWARD, which
    // biases novelty UPWARD (1/sqrt(1+count) is decreasing in count).
    // Over-novelty is the SAFE direction — the policy explores MORE, never
    // masks an already-visited state-action as novel. Document the safety
    // argument in this header (consumers depend on it).
    const int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= batch_size) return;

    const uint32_t code = simhash_code(&states[i * 42], proj_matrix);
    const uint32_t bucket = ((uint32_t)actions[i] * 65536u + code) & TABLE_MASK;
    hash_table[bucket] = hash_table[bucket] + 1.0f;
}
```

NOTE on `feedback_no_atomicadd` compliance: the update kernel uses a
non-atomic write. Race-driven count under-counts bias novelty UPWARD,
which is the safe direction (model explores more, never less). Spec
§3.5.2 explicitly accepts this trade-off. The header comment must
preserve this safety argument.

- [ ] **Step 8: Register cubin + launcher methods**

`build.rs` append `"novelty_simhash_kernel"`. In `gpu_dqn_trainer.rs` add `launch_novelty_simhash_lookup` and `launch_novelty_simhash_update`.

- [ ] **Step 9: Wire controller + novelty into training_loop**

In `training_loop.rs`, in the per-step section (after rewards and grad EMAs are populated):

```rust
// SP11 — populate canary slots, then run controller
self.gpu.launch_reward_component_mag_ratio_compute(&self.stream)?;
self.gpu.launch_saboteur_engagement_compute(/*reward bufs*/, &self.stream)?;
// val_sharpe_delta only fires at val emit boundaries (Task A1.1 wired separately)
// pnl_reward_magnitude_ema is updated via existing reward_component_ema_kernel — no new launch needed
self.gpu.launch_reward_subsystem_controller(&self.stream)?;
```

The novelty update lookup happens at REPLAY time (Layer B); for Layer A we just initialize the buffer.

- [ ] **Step 10: HEALTH_DIAG line for `sp11_reward`**

In `training_loop.rs`, find the existing HEALTH_DIAG emit block (search for `HEALTH_DIAG`). Add a new line emit after existing SP10 line:

```rust
log::info!(
    "HEALTH_DIAG[{}] sp11_reward [w_pop={:.3} w_cf={:.3} w_tr={:.3} w_mi={:.3} w_oc={:.3} w_bn={:.3}] \
     [curiosity_p={:.3} sab_mult={:.3} w_floor={:.3} curiosity_b={:.3}] [improvement_z={:.3}]",
    epoch,
    isv_snapshot[REWARD_POPART_WEIGHT_INDEX],
    isv_snapshot[REWARD_CF_WEIGHT_INDEX],
    // ... etc
);
```

The `improvement_z` for HEALTH_DIAG is computed from the canary slots: `delta_ema / max(sqrt(delta_var_ema), 1e-6)`.

- [ ] **Step 11: Verify build + smoke**

```bash
SQLX_OFFLINE=true cargo build -p ml --release 2>&1 | tail -5
FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture 2>&1 | tail -30
```
Expected: smoke completes, HEALTH_DIAG `sp11_reward` line emits with all-zeros (Pearl A bootstrap on first epoch) trending toward populated values.

- [ ] **Step 12: Commit A2**

```bash
git add crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu \
        crates/ml/src/cuda_pipeline/novelty_simhash_kernel.cu \
        crates/ml/build.rs \
        crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
        crates/ml/src/trainers/dqn/trainer/training_loop.rs \
        crates/ml/tests/sp11_producer_unit_tests.rs
git commit -m "$(cat <<'EOF'
feat(sp11): A2 — controller kernel + SimHash novelty buffer

reward_subsystem_controller_kernel: 5 canaries → 10 outputs, true Z-score
(delta_ema/sqrt(var_ema)), sigmoid blending, weight renormalization to Σ=1,
saboteur post-clamp, curiosity permanent floor.

novelty_simhash_kernel: 42×16 random projection → 16-bit SimHash code,
1M-slot bucket count table for novelty signal `1/sqrt(1+count)`. Race-
tolerated update per feedback_no_atomicadd notes.

HEALTH_DIAG `sp11_reward` line emits 10 outputs + improvement_z each epoch.
3 GPU oracle tests pass on RTX 3050 Ti.

All 20 SP11 slots populate every step. No consumer reads them yet —
training behavior unchanged from A1.

Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md §3.4 §3.5.2
EOF
)"
```

---

## Layer B — Atomic consumer migration

### Task B1: Migrate all consumers in one atomic commit

**Goal:** In a single commit, switch every consumer of the migrated contract from hardcoded reward weights / no-curiosity to ISV-driven controller outputs + replay-time curiosity bonus + saboteur multiplier. After this commit the trainer is on the new controller end-to-end.

**Files (all in one commit):**
- Modify: `crates/ml/src/cuda_pipeline/mse_loss_kernel.cu` (line 318)
- Modify: `crates/ml/src/cuda_pipeline/c51_loss_kernel.cu` (line 789)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (line 3328)
- Modify: replay sampling kernel (location: see audit step below)
- Audit + modify: any other reward-shaping kernel with hardcoded multipliers

- [ ] **Step 1: Run reward-shaping audit**

```bash
grep -rn "weight\s*=\s*[0-9]\.[0-9]\+f\|weight\s*=\s*[0-9]\+\.[0-9]" crates/ml/src/cuda_pipeline/*.cu | \
    grep -E "popart|cf|trail|micro|opp_cost|bonus" | tee /tmp/sp11_audit.txt
```

Document every site found in `/tmp/sp11_audit.txt`. For each site decide:
- Does this constant compose a reward signal? → Replace with ISV slot read.
- Is this a numerical Invariant-1 anchor (ε, clamp range)? → Leave unchanged.

Expected sites (based on spec §3.5):
- `mse_loss_kernel.cu:318` — `cf_weight = 0.3f`
- `c51_loss_kernel.cu:789` — `cf_weight = 0.3f`
- ~3-5 additional sites in popart, trail_r, micro, opp_cost, bonus shaping kernels

- [ ] **Step 2: Migrate `mse_loss_kernel.cu:318`**

Read the file at line 318. Identify the kernel signature. Add an `isv` pointer parameter to the kernel signature if not already present. Replace:
```cuda
float cf_weight = 0.3f;
```
with:
```cuda
const float cf_weight = isv[REWARD_CF_WEIGHT_INDEX];
```
Where `REWARD_CF_WEIGHT_INDEX = 341` is a compile-time `#define` at the top of the file (matched to `sp11_isv_slots.rs`). Update the launcher in `gpu_dqn_trainer.rs` to pass the ISV pointer.

- [ ] **Step 3: Migrate `c51_loss_kernel.cu:789`**

Same pattern. Add ISV pointer to signature; replace `cf_weight = 0.3f` with `isv[REWARD_CF_WEIGHT_INDEX]`. Update launcher.

- [ ] **Step 4: Migrate audit-discovered sites**

For each remaining site from `/tmp/sp11_audit.txt`, replace the hardcoded weight with the matching ISV slot:
| Hardcoded constant | ISV slot |
|---|---|
| `popart_weight = X` | `REWARD_POPART_WEIGHT_INDEX = 340` |
| `trail_weight = X`  | `REWARD_TRAIL_WEIGHT_INDEX = 342` |
| `micro_weight = X`  | `REWARD_MICRO_WEIGHT_INDEX = 343` |
| `opp_cost_weight = X` | `REWARD_OPP_COST_WEIGHT_INDEX = 344` |
| `bonus_weight = X`  | `REWARD_BONUS_WEIGHT_INDEX = 345` |

If the audit finds NO additional sites for popart/trail/micro/opp_cost/bonus, that means those reward components currently bypass an explicit weight — they enter the reward sum directly. In that case, modify the reward composition site (search for `reward_total = popart + cf + trail + ...`) to multiply each component by its ISV weight before summing.

- [ ] **Step 5: Migrate saboteur scale (GPU-side multiplication only)**

Per `feedback_no_cpu_compute_strict` AND `feedback_no_cpu_forwards`
("CPU is read-only"), the saboteur multiplication MUST happen inside the
saboteur perturbation kernel — NOT on CPU. Do NOT introduce a host-side
`read_isv_slot` + Rust multiplication.

In `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` at the
saboteur perturbation kernel call site (~line 3328), change the kernel
launch to pass:
1. `saboteur_perturbation_scale` as the existing scalar parameter (f32),
2. The ISV device pointer + `SABOTEUR_INTENSITY_MULT_INDEX as i32`,

and modify the saboteur perturbation kernel itself to compute:
```cuda
// SP11 — multiplier comes from controller output (ISV slot 347), NOT host.
const float mult = isv[saboteur_intensity_mult_slot];
const float effective_scale = base_scale * mult;
// ... apply effective_scale to fill/slippage perturbation ...
```

The existing `saboteur_perturbation_scale` Rust f32 stays on the trainer
struct as the BASE rate; the ISV multiplier is the adaptive curriculum
modulator on top. Both arrive at the kernel as parameters; multiplication
happens on-device.

If the existing saboteur perturbation kernel signature does not yet
take an ISV pointer, add it to the signature, update the launcher, and
update any other call sites. Per `feedback_no_partial_refactor` this
signature change is part of the same Layer B atomic commit.

- [ ] **Step 6: Add replay-time curiosity in `graph_utility_kernels.cu`**

The replay-time reward gather lives in
`crates/ml/src/cuda_pipeline/graph_utility_kernels.cu:71`
(`gather_f32_scalar`, the kernel that pulls rewards/dones/IS-weights
from the replay buffer by index). Add a sibling kernel
`gather_replay_reward_with_curiosity` that:

```cuda
// crates/ml/src/cuda_pipeline/graph_utility_kernels.cu — append.
// SP11 — replay-time curiosity bonus.
//
// Recompute curiosity at REPLAY time (NOT collect time) per spec §3.5.1
// to bind exploration pressure to current visit-count + current ISV[346],
// not stale at-collect values. Stale-curiosity replay would lock the
// policy into the ep1 fixed point we are trying to escape.

extern "C" __global__ void gather_replay_reward_with_curiosity(
    const float* __restrict__ reward_buf,         // base rewards in replay buffer
    const int*   __restrict__ replay_indices,     // [batch_size]
    const float* __restrict__ novelty_signals,    // [batch_size] — pre-filled by lookup
    const float* __restrict__ isv,
    float*       __restrict__ reward_out,         // [batch_size]
    const int batch_size,
    const int curiosity_pressure_slot)
{
    const int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= batch_size) return;

    const float r_base = reward_buf[replay_indices[i]];
    const float curiosity_p = isv[curiosity_pressure_slot];
    reward_out[i] = r_base + curiosity_p * novelty_signals[i];
}
```

In the replay batch dispatch (Rust side, in `gpu_dqn_trainer.rs` near
the existing `gather_f32_scalar` launch for rewards), the new sequence is:

```rust
// 1. Lookup current novelty for each (state, action) in the replay batch.
self.launch_novelty_simhash_lookup(
    &replay_states_buf, &replay_actions_buf,
    &self.novelty_simhash_proj, &self.novelty_hash_buf,
    &mut self.novelty_signals_buf, batch_size, &self.stream)?;

// 2. Gather replay reward + curiosity bonus (replaces existing scalar gather).
self.launch_gather_replay_reward_with_curiosity(
    &replay_reward_buf, &replay_indices_buf,
    &self.novelty_signals_buf,
    &mut reward_out, batch_size, &self.stream)?;

// 3. Update novelty counts AFTER lookup so this batch saw the pre-update count.
self.launch_novelty_simhash_update(
    &replay_states_buf, &replay_actions_buf,
    &self.novelty_simhash_proj, &mut self.novelty_hash_buf,
    batch_size, &self.stream)?;
```

The pre-existing scalar reward gather call is REPLACED, not augmented —
per `feedback_no_legacy_aliases` the old call is deleted, not kept as a
fallback path.

- [ ] **Step 7: Run `cargo check`**

```bash
SQLX_OFFLINE=true cargo check -p ml --lib 2>&1 | tail -20
```
Expected: clean compile. The saboteur kernel signature change is the
most likely break-site if any other call site exists; update all
callers in the same commit per `feedback_no_partial_refactor`.

- [ ] **Step 8: Run cargo test**

```bash
SQLX_OFFLINE=true cargo test -p ml --lib 2>&1 | tail -30
```
Expected: existing tests still pass (consumer migration doesn't break interface; only changes the constant source).

- [ ] **Step 9: Local smoke test**

```bash
FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture 2>&1 | tail -50
```
Expected: smoke completes. HEALTH_DIAG shows `cf_weight` (and other migrated weights) varying across epochs — i.e., not stuck at the old hardcoded value.

- [ ] **Step 10: Commit B1 (atomic)**

```bash
git add -A crates/ml/src/cuda_pipeline/ crates/ml/src/trainers/
git diff --cached --stat
# verify NO files outside the audit list are staged

git commit -m "$(cat <<'EOF'
feat(sp11): B1 — atomic consumer migration to ISV-driven controller

feedback_no_partial_refactor: every reward-shaping consumer migrates
together. Trainer is on the new controller end-to-end after this commit.

Migrations:
- mse_loss_kernel.cu:318 — cf_weight = isv[REWARD_CF_WEIGHT_INDEX]
- c51_loss_kernel.cu:789 — same
- {popart,trail,micro,opp_cost,bonus} shaping sites from /tmp/sp11_audit.txt
- gpu_experience_collector.rs:3328 — saboteur_scale * isv[SABOTEUR_INTENSITY_MULT_INDEX]
- Replay sampling kernel: curiosity bonus added at replay time, NOT
  collect time — eliminates stale-signal replay contamination per
  spec §3.5.1
- novelty_simhash_lookup scheduled before reward composition;
  novelty_simhash_update scheduled after

Local smoke validates HEALTH_DIAG shows cf_weight + other migrated
weights varying across epochs (not pinned to legacy constants).

Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md §3.5
EOF
)"
```

---

## Layer C — Validation + close-out

### Task C1: Local smoke + L40S smoke

- [ ] **Step 1: Local 5-epoch RTX 3050 Ti smoke**

```bash
FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture 2>&1 | tee /tmp/sp11_local_smoke.log
grep -E "HEALTH_DIAG.*sp11_reward|trade_count|sharpe|active_frac|dir_entropy" /tmp/sp11_local_smoke.log
```

Pass criteria (per spec §9.1):
- HEALTH_DIAG `sp11_reward` line emits every epoch
- Σweights ≈ 1.0 ± 1e-3 (renormalization works)
- `improvement_z` non-zero by ep2
- `curiosity_pressure ≥ 0.2 × curiosity_bound` always
- `saboteur_intensity ∈ [0.5, 2.0]` always

- [ ] **Step 2: Push branch + L40S smoke**

```bash
git push origin sp11-reward-as-controlled-subsystem
./scripts/argo-train.sh --smoke --branch sp11-reward-as-controlled-subsystem 2>&1 | tee /tmp/sp11_argo_smoke.log
```

Wait for completion; pull MinIO logs:
```bash
# Use the workflow name from the argo-train.sh output
mc cp foxhunt-minio/argo-artifacts/${WF_NAME}/main.log /tmp/sp11_argo_smoke_main.log
grep -E "HEALTH_DIAG.*sp11_reward|trade_count|sharpe" /tmp/sp11_argo_smoke_main.log | tail -50
```

Same pass criteria as local smoke, plus:
- No regression in Fix 33-38 metrics (IQN parallel-branch warnings = 0,
  dir_entropy > 0.5, val_active_frac > 0.20)

### Task C2: T10 full validation

- [ ] **Step 3: Dispatch T10 (3 seeds × 3 folds × 50 epochs)**

```bash
./scripts/argo-train.sh --multi-seed --branch sp11-reward-as-controlled-subsystem 2>&1 | tee /tmp/sp11_argo_t10.log
```

Wall-clock ~4 hours. Don't poll; the loop skill will surface completion.

- [ ] **Step 4: Aggregate T10 results**

After all 3 seed jobs complete, pull each main.log and parse val sharpe trajectories. Compute:
- Per-trajectory peak-epoch (epoch maximizing val sharpe within trajectory)
- Median peak-epoch across 9 trajectories
- Median val sharpe at ep20 / median val sharpe at peak-epoch (drop ratio)
- Mean val sharpe at ep20

Per spec §9.2 hard-pass criteria:
- Median peak-epoch ≥ 10
- Median val sharpe at ep20 > 0.95 × median val sharpe at peak-epoch
- Mean val sharpe at ep20 ≥ Fix 33-38 baseline ep1 mean

If T10 fails, follow §9.3 fix-forward: do NOT roll back; identify whether the controller has signal but isn't acting strongly enough (extend to additional shaping multipliers), or canary slots aren't populating (Pearl A bootstrap bug), or weights collapse to floor (renorm bug), or curiosity dominates loss (CURIOSITY_BOUND_FRACTION too high).

### Task C3: Audit doc

- [ ] **Step 5: Append Fix 39 entry to wire-up audit**

In `docs/dqn-wire-up-audit.md`, append a Fix 39 entry under SP11. Include:
- Architecture summary (controller, 4 producer kernels, 20 ISV slots)
- Layer A canary evolution evidence (HEALTH_DIAG screenshots / numerical evidence from C1 smoke)
- Layer B before/after metrics (cf_weight evolution from 0.3 hardcoded → adaptive)
- T10 aggregated metrics (peak-epoch median, sharpe trajectories)
- Validation outcome (pass/fix-forward iteration)

### Task C4: New pearl

- [ ] **Step 6: Author `pearl_reward_as_controlled_subsystem.md`**

```markdown
---
name: Reward as controlled subsystem
description: Every degree of freedom in the reward path is an output of a unified controller whose inputs are improvement-rate canaries
type: pearl
---

Every degree of freedom in the reward path — component weights, shaping
multipliers, exploration bonuses, curriculum knobs — is an output of a
unified controller whose inputs are improvement-rate canaries (val sharpe
delta + variance, per-component magnitude ratios, saboteur engagement, PnL
magnitude). The controller's classification of regime ("improving /
stagnant / regressing") is encoded in a Z-score and sigmoid; no
hardcoded threshold survives at the regime layer.

**Why:** Hardcoded reward weights make the model overfit to whichever
fixed point the initial weights happen to favor. The T10 train-multi-seed-
xkjkb baseline (commit `920a2d021`, all SP10 fixes) showed val sharpe
peaking at ep1 (80.61) and declining through ep14 (70.58) while train Q
grew 0.04 → 0.30 — classical fixed-point overfitting. With ISV-driven
weights + permanent-floor curiosity, the controller exposes the model to
shifting incentive gradients that prevent fixed-point convergence.

**How to apply:** Audit reward composition for any hardcoded shaping
multiplier. Either promote it to an ISV slot driven by a canary that
fires under the failure mode (per `pearl_controller_anchors_isv_driven`),
or document it explicitly as an Invariant-1 numerical anchor (rate
limiter, not regime threshold). When adding a new reward dimension
(e.g., curiosity), apply the permanent-floor pattern (`max(dynamic,
floor)`, not blend) per `pearl_blend_formulas_must_have_permanent_floor`
to prevent the dimension from zeroing out at convergence.
```

### Task C5: MEMORY.md index entry

- [ ] **Step 7: Add MEMORY.md index line**

In `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/MEMORY.md`, in the topic-files index section, append:

```markdown
- [pearl_reward_as_controlled_subsystem.md](pearl_reward_as_controlled_subsystem.md) — every reward-path degree of freedom is a unified-controller output with Z-score-driven adaptation. Subsumes pearl_controller_anchors_isv_driven at the reward layer.
```

- [ ] **Step 8: Commit C close-out**

```bash
git add docs/dqn-wire-up-audit.md \
        /home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/pearl_reward_as_controlled_subsystem.md \
        /home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/MEMORY.md
git commit -m "$(cat <<'EOF'
docs(sp11): C — close-out (Fix 39 audit + new pearl + MEMORY.md)

Architecture summary, T10 evidence, pearl_reward_as_controlled_subsystem
authored. SP11 close.
EOF
)"
git push origin sp11-reward-as-controlled-subsystem
```

### Task C6: Merge to main

- [ ] **Step 9: Open merge PR (or fast-forward if branch policy allows)**

If repository policy requires PR review, push, then open PR with title `SP11 — Reward as Controlled Subsystem` referencing the spec. If fast-forward is allowed:
```bash
git checkout main
git merge --ff-only sp11-reward-as-controlled-subsystem
git push origin main
```

---

## Definition of done

SP11 is complete when:
- All 4 producer kernels land in main with green CI
- HEALTH_DIAG `sp11_reward` emits 10 outputs + improvement_z every epoch in production training runs
- Hardcoded `cf_weight = 0.3f` is gone from `mse_loss_kernel.cu` and `c51_loss_kernel.cu`
- Curiosity bonus is added at replay time (not collect time)
- T10 (3 seeds × 3 folds × 50 epochs) shows median peak-epoch ≥ 10 (vs. baseline median peak-epoch = 1)
- T10 mean val sharpe at ep20 ≥ Fix 33-38 baseline ep1 mean
- `pearl_reward_as_controlled_subsystem.md` authored + indexed
- Fix 39 entry in `docs/dqn-wire-up-audit.md`
