# Bug Fixes Implementation Plan (Spec B)

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Fix all 9 CRITICAL bugs and top-priority IMPORTANT issues found by the 9-agent audit, eliminating wrong training results and silent error suppression.

**Architecture:** Config-only fixes first (lowest risk), then Rust refactors, then CUDA kernel changes last. Each task is independently testable and committable. Order follows the spec's Phase 1 → Phase 2 risk-based sequencing.

**Tech Stack:** Rust, CUDA (.cu kernels compiled by build.rs), cudarc 0.19.3, serde

**Spec:** `docs/superpowers/specs/2026-03-27-bug-fixes-design.md`

---

## File Map

| File | Bugs | Action |
|---|---|---|
| `crates/ml/src/trainers/dqn/config.rs` | #4, #5 | Make gradient_clip_norm and entropy_coefficient non-optional |
| `crates/ml/src/trainers/dqn/fused_training.rs` | #4 | Remove unwrap_or on gradient_clip_norm |
| `crates/ml/src/trainers/dqn/trainer/constructor.rs` | #4, #5 | Remove unwrap_or fallbacks |
| `crates/ml/src/trainers/dqn/data_loading.rs` | #3 | Replace unwrap_or(1.0) with direct indexing |
| `crates/ml-dqn/src/network.rs` | #8 | Conditional step_count increment |
| `crates/ml-dqn/src/quantile_regression.rs` | #6, #7 | Single CUDA context, remove silent fallbacks |
| `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu` | #1, #2 | Complete episode reset, update max_equity |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | #10 | Export budget fraction constants |
| `crates/ml-dqn/src/branching.rs` | #16 | Remove MaybeNoisyLinear enum |
| `crates/ml/src/trainers/dqn/trainer/action.rs` | #15 | Replace unwrap_or(2) with expect |
| `crates/ml/src/trainers/dqn/trainer/state.rs` | #15 | Replace unwrap_or(0.0) with expect |
| `crates/ml/src/cuda_pipeline/gpu_action_selector.rs` | #9 | Add NoisyNet architecture comment |

---

### Task 1: Unify gradient_clip_norm — eliminate inconsistent defaults

Bug #4: Main trainer clips at 10.0, IQN/IQL clip at 1.0. The fix: resolve `gradient_clip_norm` once at the top of `FusedTrainingCtx::new()` and pass the resolved value to ALL sub-configs.

**Files:**
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`

- [ ] **Step 1: Resolve gradient_clip_norm once at top of constructor**

At the top of `FusedTrainingCtx::new()` (before any config construction), add:

```rust
let resolved_grad_norm = hyperparams.gradient_clip_norm.unwrap_or(10.0);
```

Then replace all 3 sites:
- Line 194: `max_grad_norm: hyperparams.gradient_clip_norm.unwrap_or(10.0) as f32,` → `max_grad_norm: resolved_grad_norm as f32,`
- Line 263: `max_grad_norm: hyperparams.gradient_clip_norm.unwrap_or(1.0) as f32,` → `max_grad_norm: resolved_grad_norm as f32,`
- Line 298: `max_grad_norm: hyperparams.gradient_clip_norm.unwrap_or(1.0) as f32,` → `max_grad_norm: resolved_grad_norm as f32,`

- [ ] **Step 2: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true cargo test -p ml --lib -- dqn 2>&1 | tail -5`
Expected: Compiles, all tests pass.

- [ ] **Step 3: Commit**

```bash
git add crates/ml/src/trainers/dqn/fused_training.rs
git commit -m "fix: unify gradient_clip_norm — resolve once, pass to all sub-trainers (was 10.0 vs 1.0)"
```

---

### Task 2: Make entropy_coefficient non-optional

Bug #5: `Option<f64>` with 3 different `unwrap_or` defaults (0.001, 0.01, 0.02). Fix: make it non-optional with a single canonical default.

**Files:**
- Modify: `crates/ml/src/trainers/dqn/config.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/constructor.rs`

- [ ] **Step 1: Change type in DQNHyperparameters**

In `config.rs` (~line 960), change:
```rust
pub entropy_coefficient: Option<f64>,
```
To:
```rust
#[serde(default = "default_entropy_coefficient")]
pub entropy_coefficient: f64,
```

Add the default function nearby:
```rust
fn default_entropy_coefficient() -> f64 { 0.001 }
```

In the `conservative()` and `default()` impl, change:
```rust
entropy_coefficient: Some(0.001),
```
To:
```rust
entropy_coefficient: 0.001,
```

- [ ] **Step 2: Remove unwrap_or in constructor.rs**

Line 314: `entropy_coefficient: hyperparams.entropy_coefficient.unwrap_or(0.01),` → `entropy_coefficient: hyperparams.entropy_coefficient,`

Line 399: `let coeff = hyperparams.entropy_coefficient.unwrap_or(0.01);` → `let coeff = hyperparams.entropy_coefficient;`

- [ ] **Step 3: Fix all compilation errors from type change**

Run `SQLX_OFFLINE=true cargo check --workspace 2>&1 | head -40` — fix any sites that still use `Some(...)` or `.unwrap()` on entropy_coefficient. Common patterns:
- `Some(0.001)` → `0.001`
- `.unwrap_or(x)` → remove
- Pattern match on `Some(v)` → use directly

- [ ] **Step 4: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check --workspace && SQLX_OFFLINE=true cargo test -p ml --lib -- dqn && SQLX_OFFLINE=true cargo test -p ml-dqn --lib 2>&1 | tail -5`
Expected: All pass.

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/trainers/dqn/config.rs crates/ml/src/trainers/dqn/trainer/constructor.rs
git commit -m "fix: make entropy_coefficient non-optional — single canonical default 0.001"
```

---

### Task 3: Replace close price unwrap_or(1.0) with direct indexing

Bug #3: Silent wrong rewards when price parse fails. The indices are always in-bounds (loop from 1..n), so direct indexing is safe and surfaces any future bugs.

**Files:**
- Modify: `crates/ml/src/trainers/dqn/data_loading.rs`

- [ ] **Step 1: Find and replace the unwrap_or(1.0)**

At line 306, change:
```rust
let prev = close_prices_f32.get(i - 1).copied().unwrap_or(1.0);
let curr = close_prices_f32.get(i).copied().unwrap_or(prev);
```
To:
```rust
let prev = close_prices_f32[i - 1];
let curr = close_prices_f32[i];
```

- [ ] **Step 2: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true cargo test -p ml --lib -- dqn 2>&1 | tail -5`

- [ ] **Step 3: Commit**

```bash
git add crates/ml/src/trainers/dqn/data_loading.rs
git commit -m "fix: replace close price unwrap_or(1.0) with direct indexing — no silent wrong rewards"
```

---

### Task 4: Fix step_count incrementing during inference

Bug #8: `forward()` unconditionally increments step_count, advancing dropout schedule during eval.

**Files:**
- Modify: `crates/ml-dqn/src/network.rs`

- [ ] **Step 1: Read current forward() to understand full context**

Read `network.rs` around line 245-260 to see how `step_count` and `dropout_scheduler` are used.

- [ ] **Step 2: Add `training: bool` parameter to forward()**

Change the forward method to accept a training flag. Only increment step_count and advance dropout scheduler when training=true. Update the method signature and all call sites.

If there's a trait constraint, add a `forward_eval` convenience method that calls `forward(state, false)`.

- [ ] **Step 3: Update all call sites**

Search for `.forward(` across the codebase. Inference sites (backtest, action selection) pass `false`. Training sites pass `true`.

- [ ] **Step 4: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check --workspace && SQLX_OFFLINE=true cargo test -p ml-dqn --lib 2>&1 | tail -5`

- [ ] **Step 5: Commit**

```bash
git add crates/ml-dqn/src/network.rs
git commit -m "fix: step_count only increments during training — no inference side effects"
```

---

### Task 5: Single CUDA context in quantile loss

Bug #6: Creates CudaContext 3× per call. Refactor to create once and pass through.

**Files:**
- Modify: `crates/ml-dqn/src/quantile_regression.rs`

- [ ] **Step 1: Read the full quantile_huber_loss function**

Understand the 3 context creation sites and the inner `_per_sample` function.

- [ ] **Step 2: Refactor to single context**

Create context + stream once at the top. Pass the stream to all operations that need it. If `to_host()` creates its own context internally, check if there's a `to_host_with_stream()` variant or use manual `memcpy_dtoh`.

- [ ] **Step 3: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check -p ml-dqn && SQLX_OFFLINE=true cargo test -p ml-dqn --lib 2>&1 | tail -5`

- [ ] **Step 4: Commit**

```bash
git add crates/ml-dqn/src/quantile_regression.rs
git commit -m "fix: single CUDA context per quantile loss call — was 3× context creation"
```

---

### Task 6: Remove silent fallbacks in quantile loss

Bug #7: `unwrap_or(0.0)` / `unwrap_or(0.5)` on tensor indexing hides bugs.

**Files:**
- Modify: `crates/ml-dqn/src/quantile_regression.rs`

- [ ] **Step 1: Replace unwrap_or with direct indexing**

At line ~385-387, change:
```rust
let pred_val = pred_host.get(idx).copied().unwrap_or(0.0);
let tgt_val  = tgt_host.get(idx).copied().unwrap_or(0.0);
let tau_val  = tau_host.get(idx).copied().unwrap_or(0.5);
```
To:
```rust
let pred_val = pred_host[idx];
let tgt_val  = tgt_host[idx];
let tau_val  = tau_host[idx];
```

- [ ] **Step 2: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo test -p ml-dqn --lib 2>&1 | tail -5`

- [ ] **Step 3: Commit**

```bash
git add crates/ml-dqn/src/quantile_regression.rs
git commit -m "fix: remove silent 0.0/0.5 fallbacks in quantile loss — panic on OOB"
```

---

### Task 7: Complete backtest episode reset + fix stale max_equity

Bugs #1 + #2: The most impactful CUDA kernel changes. Reset all 8 portfolio fields, update max_equity before floor check.

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu`

- [ ] **Step 1: Fix pre-trade floor reset (lines 105-112)**

Replace:
```cuda
portfolio_state[ps + 0] = liq_value;
portfolio_state[ps + 1] = 0.0f;
portfolio_state[ps + 2] = liq_value;
portfolio_state[ps + 3] = 0.0f;
portfolio_state[ps + 4] = max_equity;
portfolio_state[ps + 5] = 0.0f;
portfolio_state[ps + 6] = cum_return + liq_ret;
portfolio_state[ps + 7] += 1.0f;
```
With:
```cuda
portfolio_state[ps + 0] = liq_value;   // value = new initial capital
portfolio_state[ps + 1] = 0.0f;         // position = flat
portfolio_state[ps + 2] = liq_value;   // cash = new initial capital
portfolio_state[ps + 3] = 0.0f;         // entry_price = none
portfolio_state[ps + 4] = liq_value;   // max_equity = RESET (was stale)
portfolio_state[ps + 5] = 0.0f;         // hold_time = 0
portfolio_state[ps + 6] = 0.0f;         // cum_return = RESET (was accumulating)
portfolio_state[ps + 7] = 0.0f;         // step_count = RESET (was incrementing)
```

- [ ] **Step 2: Fix max_equity update before post-trade floor check (line ~176-184)**

After `float new_value = cash + position * close;` (line 176), insert BEFORE the floor check:
```cuda
// Update max_equity BEFORE floor check — prevents stale peak
max_equity = fmaxf(max_equity, new_value);
```

- [ ] **Step 3: Fix post-trade floor reset (lines 199-206)**

Replace with complete reset (same pattern as pre-trade):
```cuda
portfolio_state[ps + 0] = new_value;
portfolio_state[ps + 1] = 0.0f;
portfolio_state[ps + 2] = new_value;
portfolio_state[ps + 3] = 0.0f;
portfolio_state[ps + 4] = new_value;   // max_equity = RESET
portfolio_state[ps + 5] = 0.0f;
portfolio_state[ps + 6] = 0.0f;         // cum_return = RESET
portfolio_state[ps + 7] = 0.0f;         // step_count = RESET
```

- [ ] **Step 4: Verify build (cubins recompile via build.rs)**

Run: `SQLX_OFFLINE=true cargo check -p ml 2>&1 | tail -5`
Expected: build.rs recompiles the .cu file, check passes.

- [ ] **Step 5: Run tests**

Run: `SQLX_OFFLINE=true cargo test -p ml --lib -- dqn 2>&1 | tail -5`

- [ ] **Step 6: Commit**

```bash
git add crates/ml/src/cuda_pipeline/backtest_env_kernel.cu
git commit -m "fix: complete backtest episode reset (all 8 fields) + update max_equity before floor check"
```

---

### Task 8: Export budget fraction constants + fix hardcoded values

Bug #10: `0.10` and `0.05` hardcoded in gpu_dqn_trainer.rs should reference the constants from fused_training.rs.

**Files:**
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`
- Modify: `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs`

- [ ] **Step 1: Make budget constants public**

In `fused_training.rs`, change:
```rust
const CQL_GRAD_BUDGET: f32 = 0.15;
const IQN_GRAD_BUDGET: f32 = 0.10;
const ENS_GRAD_BUDGET: f32 = 0.05;
```
To:
```rust
pub(crate) const CQL_GRAD_BUDGET: f32 = 0.15;
pub(crate) const IQN_GRAD_BUDGET: f32 = 0.10;
pub(crate) const ENS_GRAD_BUDGET: f32 = 0.05;
```

- [ ] **Step 2: Reference constants in gpu_dqn_trainer.rs**

At line ~831 in `apply_iqn_trunk_gradient`, change:
```rust
let max_component_norm = self.config.max_grad_norm * 0.10; // IQN_GRAD_BUDGET
```
To:
```rust
let max_component_norm = self.config.max_grad_norm * crate::trainers::dqn::fused_training::IQN_GRAD_BUDGET;
```

At line ~1072 in `apply_ensemble_trunk_gradient`, change:
```rust
let max_component_norm = self.config.max_grad_norm * 0.05; // ENS_GRAD_BUDGET
```
To:
```rust
let max_component_norm = self.config.max_grad_norm * crate::trainers::dqn::fused_training::ENS_GRAD_BUDGET;
```

- [ ] **Step 3: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true cargo test -p ml --lib -- dqn 2>&1 | tail -5`

- [ ] **Step 4: Commit**

```bash
git add crates/ml/src/trainers/dqn/fused_training.rs crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
git commit -m "fix: reference budget fraction constants — eliminate hardcoded 0.10/0.05 in gpu_dqn_trainer"
```

---

### Task 9: Remove MaybeNoisyLinear single-variant enum

Bug #16: Enum with only `Noisy` variant — unnecessary indirection.

**Files:**
- Modify: `crates/ml-dqn/src/branching.rs`

- [ ] **Step 1: Read current MaybeNoisyLinear usage**

Search for all `MaybeNoisyLinear` references in branching.rs to understand the scope of changes.

- [ ] **Step 2: Replace enum with direct NoisyLinear**

- Remove the `MaybeNoisyLinear` enum definition
- Change all struct fields from `MaybeNoisyLinear` to `NoisyLinear`
- Remove all `let MaybeNoisyLinear::Noisy(n) = self;` destructuring — use the field directly
- Inline the thin wrapper methods (forward, reset_noise, etc.) or call directly on NoisyLinear

- [ ] **Step 3: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check --workspace && SQLX_OFFLINE=true cargo test -p ml-dqn --lib 2>&1 | tail -5`

- [ ] **Step 4: Commit**

```bash
git add crates/ml-dqn/src/branching.rs
git commit -m "fix: remove MaybeNoisyLinear single-variant enum — use NoisyLinear directly"
```

---

### Task 10: Fix Tier 1 silent unwrap_or fallbacks

Bug #15: Critical silent fallbacks in action selection and state processing.

**Files:**
- Modify: `crates/ml/src/trainers/dqn/trainer/action.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/state.rs`

- [ ] **Step 1: Fix action.rs:161 — argmax fallback**

Change `unwrap_or(2)` (silent Flat action) to `.expect("argmax on non-empty Q-values")`.

- [ ] **Step 2: Fix state.rs:73 — price to_f32 fallback**

Change `unwrap_or(0.0)` to `.expect("trading price fits f32")`.

- [ ] **Step 3: Verify compilation + tests**

Run: `SQLX_OFFLINE=true cargo check -p ml && SQLX_OFFLINE=true cargo test -p ml --lib -- dqn 2>&1 | tail -5`

- [ ] **Step 4: Commit**

```bash
git add crates/ml/src/trainers/dqn/trainer/action.rs crates/ml/src/trainers/dqn/trainer/state.rs
git commit -m "fix: replace silent unwrap_or fallbacks in action selection and state processing"
```

---

### Task 11: Add NoisyNet architecture comment to action selector

Bug #9 (NOT A BUG): Document the architectural choice so future audits don't re-flag this.

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_action_selector.rs`

- [ ] **Step 1: Add comment at top of file**

After the module doc comment, add:
```rust
// NoisyNet exploration note: Q-values received by the action selector already
// include factorized Gaussian noise from NoisyLinear layers in the network
// forward pass. Epsilon-greedy acts as a secondary fallback, not the primary
// exploration mechanism when NoisyNet is enabled. Explicit noise injection
// here would double-count exploration.
```

- [ ] **Step 2: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_action_selector.rs
git commit -m "docs: clarify NoisyNet exploration is in network forward, not action selector"
```

---

## Execution Dependencies

```
Task 1 (gradient_clip_norm) — independent
Task 2 (entropy_coefficient) — independent
Task 3 (close price) — independent
Task 4 (step_count) — independent
Task 5 (CUDA context) — independent
Task 6 (quantile fallbacks) — after Task 5 (same file)
Task 7 (backtest reset) — independent
Task 8 (budget constants) — independent
Task 9 (MaybeNoisyLinear) — independent
Task 10 (unwrap_or fixes) — independent
Task 11 (NoisyNet comment) — independent
```

Tasks 5-6 must be sequential (same file). All others are independent and can run in parallel.

## Verification

After ALL tasks:
```bash
SQLX_OFFLINE=true cargo check --workspace
SQLX_OFFLINE=true cargo test -p ml --lib -- dqn
SQLX_OFFLINE=true cargo test -p ml-dqn --lib
```

All must pass with zero failures.