# Reward v7.1: Six Critical Fixes — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Fix 6 concerns from the v7 hyperopt run: i%3 exposure degeneracy (auxiliary loss), OFI epoch gate, CEA warmup extension, exit timing bug, Kelly prior seeding, and reward noise scaling. Remove all dead v6 code. Zero test regressions (891+ tests must pass).

**Architecture:** Fixes 2-6 are surgical edits to the experience kernel and training loop. Fix 1 adds a new `exposure_aux_grad_kernel` in `dqn_utility_kernels.cu`, following the IQN/CQL auxiliary gradient pattern. Dead v6 fields (`loss_aversion`, `beta_penalty`, `regret_blend`, `trade_clustering_penalty`) are removed from `ExperienceCollectorConfig` and the kernel launch chain.

**Tech Stack:** CUDA (nvcc → cubin via build.rs), Rust (cudarc), TOML config

---

### Task 1: Fixes 4, 5, 6 — Surgical CUDA Kernel Edits

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu`

These three fixes are localized edits within the reward section of `experience_env_step`. No signature changes, no Rust wiring.

- [ ] **Step 1: Fix 4 — Add `closing_sign` variable for exit timing**

Find where `prev_sign` is computed (search for `int prev_sign`). It's computed early in the kernel from `pre_trade_position`. Add `closing_sign` immediately after:

```cuda
    int prev_sign = (pre_trade_position > 0.001f) ? 1 : ((pre_trade_position < -0.001f) ? -1 : 0);
    int closing_sign = prev_sign;  /* v7.1: capture for exit timing before reversal logic */
```

Then find Layer 7 (exit timing quality, search for `Layer 7: Exit timing`). Replace `prev_sign` with `closing_sign`:

```cuda
        /* ── Layer 7: Exit timing quality ── */
        if (exit_timing_weight > 0.0f && (exiting_trade || reversing_trade)) {
            float post_exit_move = (closing_sign > 0)
                ? (raw_next - raw_close) / fmaxf(raw_close, 1.0f)
                : (raw_close - raw_next) / fmaxf(raw_close, 1.0f);
            float timing_quality = -post_exit_move / vol_proxy;
            reward += exit_timing_weight * asymmetric_soft_clamp(timing_quality * 10.0f) * 0.1f;
        }
```

- [ ] **Step 2: Fix 5 — Kelly prior seeding**

Find Layer 9 (Kelly-optimal, search for `Layer 9: Kelly`). Replace the entire block:

```cuda
        /* ── Layer 9: Kelly-optimal position sizing signal ── */
        if (kelly_sizing_weight > 0.0f) {
            /* Bayesian prior (Jeffrey's): 2 pseudo-wins + 2 pseudo-losses.
             * Regularizes toward no-bet with no data, converges to empirical
             * Kelly as real trades accumulate. Always fires (no threshold). */
            float prior_wins = 2.0f;
            float prior_losses = 2.0f;
            float prior_sum_wins = 0.01f;
            float prior_sum_losses = 0.01f;

            float eff_wins = win_count + prior_wins;
            float eff_losses = loss_count + prior_losses;
            float eff_total = eff_wins + eff_losses;

            float win_rate_k = eff_wins / eff_total;
            float avg_win = (sum_wins + prior_sum_wins) / eff_wins;
            float avg_loss = (sum_losses + prior_sum_losses) / eff_losses;
            float payoff_ratio = avg_win / fmaxf(avg_loss, 0.0001f);
            float kelly_f = (payoff_ratio * win_rate_k - (1.0f - win_rate_k))
                           / fmaxf(payoff_ratio, 0.0001f);
            kelly_f = fmaxf(fminf(kelly_f, 1.0f), 0.0f);

            float actual_fraction = fabsf(position) / fmaxf(max_position, 1.0f);
            float kelly_closeness = 1.0f - fabsf(actual_fraction - kelly_f);
            kelly_closeness = fmaxf(kelly_closeness, 0.0f);

            if (segment_return > 0.0f) {
                reward += kelly_sizing_weight * kelly_closeness * 0.5f;
            }
        }
```

- [ ] **Step 3: Fix 6 — Reward noise magnitude scaling**

Find the reward label smoothing block (search for `Reward label smoothing`). Replace:

```cuda
    /* ── Reward label smoothing (magnitude-relative) ── */
    if (reward_noise_scale > 0.0f && reward != 0.0f) {
        unsigned int hash = (unsigned int)(i * 31337 + bar_idx * 7919 + current_t * 1013);
        hash ^= (hash >> 16); hash *= 0x45d9f3bu; hash ^= (hash >> 16);
        float pseudo_noise = ((float)(hash & 0xFFFF) / 65535.0f - 0.5f) * 2.0f;
        /* Scale noise to reward_noise_scale% of reward magnitude, floor at 0.01 */
        float noise_magnitude = fmaxf(fabsf(reward) * reward_noise_scale, 0.01f);
        reward += noise_magnitude * pseudo_noise;
    }
```

- [ ] **Step 4: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: `Finished`

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/experience_kernels.cu
git commit -m "fix(reward-v7.1): exit timing closing_sign, Kelly prior seeding, noise magnitude scaling"
```

---

### Task 2: Fix 1a — Best Exposure Output in Experience Kernel

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu` (kernel signature + CEA loop)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (buffer alloc + arg wiring)

- [ ] **Step 1: Add `out_best_exposure` parameter to kernel signature**

Find the kernel signature for `experience_env_step`. Add `int* out_best_exposure` parameter BEFORE `saboteur_params` (saboteur must remain last). Insert after `float reward_noise_scale,`:

```cuda
    int*                      out_best_exposure,  /* v7.1: counterfactual best exposure idx per step (-1 = no trade exit) */
    /* #33 Per-episode saboteur params ... */
    const float* __restrict__ saboteur_params
```

- [ ] **Step 2: Track best exposure in the CEA loop and write output**

Find the CEA loop (search for `Layer 3: Counterfactual Exposure Advantage`). Modify to track `best_k`:

```cuda
        /* ── Layer 3: Counterfactual Exposure Advantage (CEA) ── */
        int best_exposure_k = 0;
        float best_exposure_reward = -1e30f;
        if (cea_weight > 0.0f) {
            float sum_alt_rewards = 0.0f;
            float price_delta = raw_next - raw_close;
            for (int k = 0; k < b0_size; k++) {
                float alt_pos = compute_target_position(k, b0_size, max_position);
                float alt_pnl = alt_pos * price_delta;
                float alt_return = alt_pnl / (prev_equity > 1.0f ? prev_equity : 1.0f);
                float alt_vol_norm = alt_return / vol_norm;
                float alt_reward = asymmetric_soft_clamp(10.0f * alt_vol_norm);
                sum_alt_rewards += alt_reward;
                if (alt_reward > best_exposure_reward) {
                    best_exposure_reward = alt_reward;
                    best_exposure_k = k;
                }
            }
            float mean_alt_reward = sum_alt_rewards / (float)b0_size;
            float exposure_advantage = reward - mean_alt_reward;
            reward += cea_weight * exposure_advantage;
        }
```

Then find where `out_rewards[out_off] = reward;` is written (near end of kernel). Add immediately BEFORE it:

```cuda
    /* v7.1: Write counterfactual best exposure for auxiliary loss */
    if (out_best_exposure != NULL) {
        out_best_exposure[out_off] = (segment_complete && segment_hold_time > 0.0f)
            ? best_exposure_k : -1;
    }
```

- [ ] **Step 3: Add buffer and wiring in gpu_experience_collector.rs**

Add a new field to the collector struct. Find `rewards_out: CudaSlice<f32>` (line ~529). Add after `done_out`:

```rust
    best_exposure_out: CudaSlice<i32>,        // [alloc_episodes * alloc_timesteps] v7.1 aux target
```

Allocate in the constructor. Find where `done_out` is allocated (search for `alloc done_out`). Add after:

```rust
        let best_exposure_out = stream
            .alloc_zeros::<i32>(alloc_episodes * alloc_timesteps)
            .map_err(|e| MLError::ModelError(format!("alloc best_exposure_out: {e}")))?;
```

Add to the struct literal where `done_out` is assigned:

```rust
            best_exposure_out,
```

Wire in the kernel launch. Find the `.arg(&config.reward_noise_scale)` line in the env_step launch. Add AFTER it (before saboteur):

```rust
                    .arg(&mut self.best_exposure_out)  // v7.1: counterfactual best exposure
```

Add a public accessor for the training loop to read the buffer:

```rust
    /// v7.1: Best counterfactual exposure per step (for auxiliary loss).
    pub fn best_exposure_buf(&self) -> &CudaSlice<i32> {
        &self.best_exposure_out
    }
```

- [ ] **Step 4: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: `Finished`

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/experience_kernels.cu \
       crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
git commit -m "feat(reward-v7.1): track best_exposure_idx in experience kernel for auxiliary loss"
```

---

### Task 3: Fix 1b — Exposure Auxiliary Gradient Kernel

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/dqn_utility_kernels.cu`

- [ ] **Step 1: Add `exposure_aux_grad_kernel` to the utility kernels file**

Find the END of `dqn_utility_kernels.cu` (after the last kernel). Append:

```cuda
/* ══════════════════════════════════════════════════════════════════════
 * EXPOSURE AUXILIARY GRADIENT KERNEL (v7.1)
 *
 * Breaks the i%3 Q-value degeneracy in branching DQN by giving each
 * exposure output a UNIQUE gradient via cross-entropy against the
 * counterfactual best exposure.
 *
 * Follows IQN/CQL pattern: accumulates auxiliary gradients into grad_buf
 * between C51 backward and Adam. The C51 gradient only differentiates
 * selected vs non-selected (-1/9 for all 8 non-selected). This kernel
 * gives each exposure output a target-specific gradient.
 *
 * Launch: grid=(ceil(batch_size/256)), block=(256).
 * ══════════════════════════════════════════════════════════════════════ */
extern "C" __global__ void exposure_aux_grad_kernel(
    const float* __restrict__ adv_logits,  /* [batch_size, b0_size * num_atoms] exposure branch logits (f32) */
    const int*   __restrict__ targets,     /* [batch_size] best exposure idx (-1 = skip) */
    float*       __restrict__ grad_buf,    /* [total_params] gradient accumulator */
    int          adv_offset,               /* offset into grad_buf for exposure adv weights */
    int          b0_size,                  /* 9 */
    int          num_atoms,                /* 51 or 101 */
    float        aux_weight,               /* scaled weight (decays over epochs) */
    int          batch_size
) {
    int sample = blockIdx.x * blockDim.x + threadIdx.x;
    if (sample >= batch_size) return;

    int target = targets[sample];
    if (target < 0 || target >= b0_size) return;  /* skip non-exit steps */

    const float* logits = adv_logits + (long long)sample * b0_size * num_atoms;
    float inv_batch = aux_weight / (float)batch_size;

    for (int z = 0; z < num_atoms; z++) {
        /* Stable softmax over b0_size for atom z */
        float max_l = -1e30f;
        for (int a = 0; a < b0_size; a++) {
            float l = logits[a * num_atoms + z];
            if (l > max_l) max_l = l;
        }
        float sum_exp = 0.0f;
        for (int a = 0; a < b0_size; a++) {
            sum_exp += expf(logits[a * num_atoms + z] - max_l);
        }
        float inv_sum = 1.0f / fmaxf(sum_exp, 1e-10f);

        /* Cross-entropy gradient: softmax(logit) - one_hot(target) */
        for (int a = 0; a < b0_size; a++) {
            float prob = expf(logits[a * num_atoms + z] - max_l) * inv_sum;
            float grad = (prob - ((a == target) ? 1.0f : 0.0f)) * inv_batch;
            int param_idx = adv_offset + a * num_atoms + z;
            atomicAdd(&grad_buf[param_idx], grad);
        }
    }
}
```

- [ ] **Step 2: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: `Finished` (CUDA compiled by build.rs at deploy time)

- [ ] **Step 3: Commit**

```bash
git add crates/ml/src/cuda_pipeline/dqn_utility_kernels.cu
git commit -m "feat(reward-v7.1): add exposure_aux_grad_kernel for per-branch gradient signal"
```

---

### Task 4: Fix 1c — Wire Auxiliary Kernel in Training Pipeline

**Files:**
- Modify: `crates/ml/src/trainers/dqn/config.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`

- [ ] **Step 1: Add config fields**

In `crates/ml/src/trainers/dqn/config.rs`, find the v7 gem fields (near `urgency_credit_weight`). Add:

```rust
    /// v7.1: Exposure auxiliary loss weight (scheduled decay).
    pub exposure_aux_weight: f64,
    /// v7.1: Epochs over which exposure aux weight decays from base to 10%.
    pub exposure_aux_warmup_epochs: usize,
```

Add defaults in BOTH `Default` impl and `conservative()`:

```rust
            exposure_aux_weight: 0.5,
            exposure_aux_warmup_epochs: 5,
```

In `apply_family_scaling`, add in the loss_shaping block:

```rust
        self.exposure_aux_weight = (self.exposure_aux_weight * li).clamp(0.0, 2.0);
```

- [ ] **Step 2: Load kernel in GpuDqnTrainer**

In `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`, find where utility kernels are loaded (search for `load_function("dqn_grad_norm_kernel")`). The utility module is loaded from cubin. Add near the other kernel loads:

```rust
    let exposure_aux_grad = module.load_function("exposure_aux_grad_kernel")
        .map_err(|e| MLError::ModelError(format!("exposure_aux_grad_kernel load: {e}")))?;
```

Add the function handle to the struct fields (search for `grad_norm_finalize_kernel: CudaFunction`):

```rust
    exposure_aux_grad_kernel: CudaFunction,
```

Store it in the constructor's struct literal:

```rust
            exposure_aux_grad_kernel: exposure_aux_grad,
```

Add a public method to launch the kernel:

```rust
    /// v7.1: Launch exposure auxiliary gradient kernel.
    /// Adds cross-entropy gradient to grad_buf for exposure branch outputs.
    pub fn launch_exposure_aux_grad(
        &self,
        targets: &CudaSlice<i32>,
        aux_weight: f32,
        batch_size: usize,
    ) -> Result<(), MLError> {
        if aux_weight < 1e-6 { return Ok(()); }
        let _evt_guard = EventTrackingGuard::new(self.stream.context());

        let adv_logits_ptr = self.on_adv_logits_buf.raw_ptr();  // exposure branch logits (f32)
        let targets_ptr = targets.raw_ptr();
        let grad_ptr = self.ptrs.grad_buf;
        let adv_offset = self.exposure_adv_grad_offset() as i32;
        let b0 = self.config.branch_0_size as i32;
        let na = self.config.num_atoms as i32;
        let bs = batch_size as i32;

        let blocks = ((batch_size + 255) / 256) as u32;
        unsafe {
            self.stream
                .launch_builder(&self.exposure_aux_grad_kernel)
                .arg(&adv_logits_ptr)
                .arg(&targets_ptr)
                .arg(&grad_ptr)
                .arg(&adv_offset)
                .arg(&b0)
                .arg(&na)
                .arg(&aux_weight)
                .arg(&bs)
                .launch(LaunchConfig {
                    grid_dim: (blocks, 1, 1),
                    block_dim: (256, 1, 1),
                    shared_mem_bytes: 0,
                })
                .map_err(|e| MLError::ModelError(format!("exposure_aux_grad: {e}")))?;
        }
        Ok(())
    }

    /// Offset into the flat grad_buf where the exposure advantage weights begin.
    fn exposure_adv_grad_offset(&self) -> usize {
        // grad_buf layout: [shared_weights..., value_weights..., adv_weights...]
        // The advantage section starts after shared + value weights.
        // For branching DQN: first b0_size * num_atoms elements of adv are the exposure branch.
        self.config.shared_param_count() + self.config.value_param_count()
    }
```

NOTE: The implementer must verify `on_adv_logits_buf` exists as an f32 buffer (it may be bf16 — check and cast if needed), and `exposure_adv_grad_offset()` computes the correct offset by inspecting the existing `compute_total_params` and weight layout functions.

- [ ] **Step 3: Wire in fused training**

In `crates/ml/src/trainers/dqn/fused_training.rs`, find `submit_aux_ops()` where IQN trunk gradient and CQL gradient are applied (search for `apply_iqn_trunk_gradient` or `apply_cql_gradient`). Add the exposure aux gradient call AFTER the CQL block:

```rust
        // v7.1: Exposure auxiliary gradient (breaks i%3 degeneracy)
        if self.exposure_aux_weight > 0.0 {
            if let Some(ref targets_buf) = self.best_exposure_targets {
                self.trainer.launch_exposure_aux_grad(
                    targets_buf,
                    self.exposure_aux_weight as f32,
                    self.batch_size,
                ).map_err(|e| anyhow::anyhow!("Exposure aux grad: {e}"))?;
            }
        }
```

Add the `exposure_aux_weight` field and `best_exposure_targets` buffer to `FusedTrainingCtx`:

```rust
    exposure_aux_weight: f64,
    best_exposure_targets: Option<CudaSlice<i32>>,  // copied from experience collector per epoch
```

Initialize in the constructor from hyperparams:

```rust
            exposure_aux_weight: hyperparams.exposure_aux_weight,
            best_exposure_targets: None,
```

Add a method to upload targets from the experience collector's buffer:

```rust
    pub fn set_exposure_targets(&mut self, targets: &CudaSlice<i32>, batch_size: usize) -> Result<()> {
        if self.best_exposure_targets.is_none() {
            self.best_exposure_targets = Some(
                self.stream.alloc_zeros::<i32>(batch_size)
                    .map_err(|e| anyhow::anyhow!("alloc exposure targets: {e}"))?
            );
        }
        let dst = self.best_exposure_targets.as_mut().unwrap();
        self.stream.memcpy_dtod(dst, targets)
            .map_err(|e| anyhow::anyhow!("copy exposure targets: {e}"))?;
        Ok(())
    }
```

- [ ] **Step 4: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: `Finished`

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/trainers/dqn/config.rs \
       crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs \
       crates/ml/src/trainers/dqn/fused_training.rs
git commit -m "feat(reward-v7.1): wire exposure auxiliary gradient kernel in training pipeline"
```

---

### Task 5: Fixes 2, 3 — Training Loop Schedule Changes

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`

- [ ] **Step 1: Fix 3 — Extended CEA warmup (25% linear decay)**

Find where `cea_weight` is set in the `ExperienceCollectorConfig` constructor (search for `cea_weight:`). Replace:

```rust
            cea_weight: {
                // v7.1: Linear warmup over 25% of training — blend from 1.0 to configured value
                let base = self.hyperparams.cea_weight as f32;
                let warmup_end = (self.hyperparams.epochs as f32 * 0.25).max(3.0);
                if (self.current_epoch as f32) < warmup_end {
                    let progress = self.current_epoch as f32 / warmup_end;
                    1.0_f32 * (1.0 - progress) + base * progress
                } else {
                    base
                }
            },
```

- [ ] **Step 2: Fix 2 — OFI epoch gate**

Find where `ofi_reward_weight` is set (should be near `cea_weight`). Replace:

```rust
            ofi_reward_weight: {
                // v7.1: Disable OFI for first 5 epochs — model learns direction first
                let base = self.hyperparams.ofi_reward_weight as f32;
                if self.current_epoch < 5 { 0.0 } else { base }
            },
```

- [ ] **Step 3: Add exposure aux weight decay schedule**

Find where the `ExperienceCollectorConfig` is constructed. After the config is built, compute the decayed exposure aux weight and pass it to the fused context:

```rust
            // v7.1: Exposure auxiliary loss weight with scheduled decay
            let exposure_aux_w = {
                let base = self.hyperparams.exposure_aux_weight;
                let warmup = self.hyperparams.exposure_aux_warmup_epochs.max(1) as f64;
                let progress = (self.current_epoch as f64 / warmup).min(1.0);
                base * (1.0 - 0.9 * progress)  // decays from base to 10% of base
            };
            if let Some(ref mut fused) = self.fused_ctx {
                fused.exposure_aux_weight = exposure_aux_w;
            }
```

- [ ] **Step 4: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: `Finished`

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/trainers/dqn/trainer/training_loop.rs
git commit -m "fix(reward-v7.1): CEA 25% linear warmup, OFI epoch gate, exposure aux decay schedule"
```

---

### Task 6: Remove Dead v6 Code

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/config.rs`
- Modify: `crates/ml/src/training_profile.rs`

Remove dead v6 config fields that are zeroed and no longer used by the kernel.

- [ ] **Step 1: Remove dead fields from ExperienceCollectorConfig**

In `gpu_experience_collector.rs`, find and REMOVE these struct fields:

```rust
    pub loss_aversion: f32,        // v6 dead: replaced by asymmetric_soft_clamp
    pub beta_penalty: f32,         // v6 dead: was disabled (0.0)
    pub regret_blend: f32,         // v6 dead: replaced by CEA
    pub trade_clustering_penalty: f32,  // v6 dead: removed
```

Remove them from the `Default` impl too:

```rust
            loss_aversion: 1.5,           // REMOVE
            beta_penalty: 0.0,            // REMOVE
            regret_blend: 0.0,            // REMOVE
            trade_clustering_penalty: 0.0, // REMOVE
```

- [ ] **Step 2: Remove dead field assignments from training_loop.rs**

Find where these fields are assigned in the `ExperienceCollectorConfig` constructor:

```rust
            loss_aversion: self.hyperparams.loss_aversion as f32,  // REMOVE
            beta_penalty: self.hyperparams.beta_penalty_strength as f32,  // REMOVE
            regret_blend: self.hyperparams.regret_blend as f32,  // REMOVE
            trade_clustering_penalty: self.hyperparams.trade_clustering_penalty as f32,  // REMOVE
```

- [ ] **Step 3: Fix any compilation errors**

After removing fields, search for any remaining references to the removed fields and fix them. Common locations:
- `training_profile.rs`: may assign these fields — change to no-ops or remove
- Other test files or constructors

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | grep "error"` and fix each one.

- [ ] **Step 4: Verify build + run tests**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Then: `SQLX_OFFLINE=true cargo test -p ml --lib 2>&1 | grep "^test result:"`
Expected: `ok. 891 passed; 0 failed`

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_experience_collector.rs \
       crates/ml/src/trainers/dqn/trainer/training_loop.rs \
       crates/ml/src/trainers/dqn/config.rs \
       crates/ml/src/training_profile.rs
git commit -m "cleanup: remove dead v6 reward fields (loss_aversion, beta_penalty, regret_blend, trade_clustering)"
```

---

### Task 7: Update TOML Configs + Final Test Pass

**Files:**
- Modify: `config/training/dqn-production.toml`
- Modify: `config/training/dqn-hyperopt.toml`
- Modify: `config/training/dqn-smoketest.toml`
- Modify: `config/training/dqn-localdev.toml`

- [ ] **Step 1: Add v7.1 params to all TOMLs**

Add to `[reward]` section of each file:

```toml
exposure_aux_weight = 0.5
exposure_aux_warmup_epochs = 5
```

For `dqn-hyperopt.toml`, also add to `[search_space]`:

```toml
exposure_aux_weight = [0.1, 1.0]
```

And to `[phase_fast]`:

```toml
exposure_aux_weight = 0.5
exposure_aux_warmup_epochs = 5
```

- [ ] **Step 2: Run full test suite**

Run: `SQLX_OFFLINE=true cargo test -p ml --lib 2>&1 | grep "^test result:"`
Expected: `ok. 891 passed; 0 failed` (or more — zero regressions)

Fix any test expectation failures from config changes.

- [ ] **Step 3: Run workspace build**

Run: `SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -5`
Expected: `Finished`

- [ ] **Step 4: Commit**

```bash
git add config/training/ crates/ml/src/
git commit -m "config(reward-v7.1): add exposure_aux_weight to all TOML profiles, final test pass"
```
