# Experience Collector Zero-Copy Refactor

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Eliminate all DtoD weight copies between trainer and experience collector. Collector reads trainer's `params_buf` via direct u64 pointer. Enable bottleneck forward in collector. Delete all dead weight buffers and sync methods.

**Architecture:** Replace `online_params_f32` (never synced, always zeros) and `online_params_flat` (synced but never read) with a single `trainer_params_ptr: u64` that points into the trainer's `params_buf`. Recompute `param_sizes` from the trainer's real config (bottleneck_dim=16). Collector gets its own `CublasForward` with correct `s1_input_dim=54`. Bottleneck forward (GEMM + tanh + concat) activates using the existing `bn_tanh_concat_fn` kernel.

**Tech Stack:** Rust 1.85, cudarc, CUDA 12.4

---

## File Map

| File | Action | Changes |
|------|--------|---------|
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Modify | Delete 5 dead fields + 3 dead methods, add `trainer_params_ptr`, fix `param_sizes` + `CublasForward` + bottleneck forward, resize bn buffers |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs` | Modify | Delete `sync_gpu_weights`, pass `trainer_params_ptr` + `s1_input_dim` + `bottleneck_dim` + `market_dim` to collector constructor |
| `crates/ml/src/trainers/dqn/fused_training.rs` | Modify | Expose `params_buf_ptr()` + `param_sizes` for collector construction |

---

### Task 1: Delete dead weight fields and methods from collector

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`

- [ ] **Step 1: Delete `online_params_flat` field and its allocation**

Remove field `online_params_flat: CudaSlice<f32>` (line 478), its allocation at lines 796-798, and its entry in the `Self { ... }` constructor return at line 1138.

- [ ] **Step 2: Delete `online_params_f32` field and its allocation**

Remove field `online_params_f32: CudaSlice<f32>` (line 597), its allocation at line 1054, and its entry in the constructor return at line 1192.

- [ ] **Step 3: Delete `online_weights` and `target_weights` (DuelingWeightSet)**

Remove fields `online_weights` and `target_weights` (used only by dead `flatten_online_weights`). Remove the construction at lines 810-837 (the 50-line copy block). Remove constructor params `online_dueling` and `target_dueling` from `fn new()` signature (line 697-698). Remove from `Self { ... }` at lines 1168-1169.

- [ ] **Step 4: Delete `online_branching` and `target_branching` (BranchingWeightSet)**

Remove fields, construction at lines 848-861, constructor params `online_branching` and `target_branching` from `fn new()` (lines 700-701), and from `Self { ... }` at lines 1173-1174.

- [ ] **Step 5: Delete `curiosity_weights`, `online_rmsnorm`, `target_rmsnorm`**

These are all-zero / all-ones stubs never read by the cuBLAS forward. Remove fields, construction at lines 841-845, and from `Self { ... }` at lines 1170-1172.

- [ ] **Step 6: Delete dead sync methods**

Remove `sync_weights_flat()` (lines 2288-2293), `sync_weights_f32()` (lines 2298-2302), `flatten_online_weights()` (lines 2310-2361). Also remove `online_params_flat_ptr()` accessor and `total_param_bytes()` accessor (search for them).

- [ ] **Step 7: Verify compilation**

```bash
SQLX_OFFLINE=true cargo check -p ml --lib 2>&1 | tail -5
```

Expect: compilation errors in `training_loop.rs` (calls deleted methods) and anywhere else that passes the removed constructor params. Those are fixed in Task 2.

- [ ] **Step 8: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
git commit -m "refactor: delete 5 dead weight fields + 3 dead sync methods from exp collector"
```

---

### Task 2: Add `trainer_params_ptr` and fix constructor signature

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs`
- Modify: `crates/ml/src/trainers/dqn/trainer/training_loop.rs`
- Modify: `crates/ml/src/trainers/dqn/fused_training.rs`

- [ ] **Step 1: Add new constructor params and field**

Replace the deleted `online_dueling`, `target_dueling`, `online_branching`, `target_branching` params in `fn new()` with:

```rust
pub fn new(
    stream: Arc<CudaStream>,
    trainer_params_ptr: u64,        // raw pointer into trainer's params_buf (stable)
    bottleneck_dim: usize,          // trainer's bottleneck_dim (16 in production)
    market_dim_cfg: usize,          // trainer's market_dim (42)
    initial_capital: f32,
    _avg_spread: f32,
    _cash_reserve_pct: f32,
    network_dims: (usize, usize, usize, usize),
    kernel_dims: (usize, usize, usize),
    n_episodes: usize,
    timesteps_per_episode: usize,
) -> Result<Self, MLError> {
```

Add field `trainer_params_ptr: u64` to the struct (replacing `online_params_flat`).

- [ ] **Step 2: Fix `param_sizes` computation to use real bottleneck config**

Replace the synthetic `GpuDqnTrainConfig` at lines 775-786:

```rust
let train_cfg = GpuDqnTrainConfig {
    state_dim,
    shared_h1,
    shared_h2,
    value_h,
    adv_h,
    num_atoms,
    branch_0_size: branch_sizes[0],
    branch_1_size: branch_sizes[1],
    branch_2_size: branch_sizes[2],
    branch_3_size: 3,
    bottleneck_dim: bottleneck_dim,   // was: 0 (WRONG — different layout)
    market_dim: market_dim_cfg,        // was: 0
    ..GpuDqnTrainConfig::default()
};
let param_sizes = compute_param_sizes(&train_cfg);
let total_params = compute_total_params(&train_cfg);
```

- [ ] **Step 3: Fix `CublasForward` construction with correct `s1_input_dim`**

The collector's `CublasForward::new` call (around line 744) needs the correct `s1_input_dim`:

```rust
let s1_input_dim = if bottleneck_dim > 0 {
    bottleneck_dim + state_dim.saturating_sub(market_dim_cfg)
} else {
    state_dim
};
let cublas_forward = CublasForward::new(
    &stream,
    alloc_episodes,
    state_dim,
    shared_h1, shared_h2, value_h, adv_h,
    num_atoms,
    branch_sizes[0], branch_sizes[1], branch_sizes[2], branch_sizes[3],
    s1_input_dim,
)?;
```

- [ ] **Step 4: Fix bottleneck buffer allocation**

Replace the `bn_alloc = bn_dim_from_params.max(1)` at line 1114 with:

```rust
let bn_alloc = bottleneck_dim.max(1);
let portfolio_dim_bn = state_dim.saturating_sub(market_dim_cfg);
let exp_bn_hidden = stream.alloc_zeros::<f32>(alloc_episodes * bn_alloc)
    .map_err(|e| MLError::ModelError(format!("alloc exp_bn_hidden: {e}")))?;
let exp_bn_concat = stream.alloc_zeros::<f32>(alloc_episodes * (bn_alloc + portfolio_dim_bn) + 128)
    .map_err(|e| MLError::ModelError(format!("alloc exp_bn_concat: {e}")))?;
```

Set `bottleneck_dim: bottleneck_dim` (not `bn_dim_from_params`) in the `Self { ... }` constructor return. And `market_dim_bn: market_dim_cfg`.

- [ ] **Step 5: Replace `online_params_f32` reads with `trainer_params_ptr`**

In the forward pass (lines 1926-1984), replace:
```rust
self.online_params_f32.raw_ptr()
```
with:
```rust
self.trainer_params_ptr
```

There are 2 occurrences: line 1927 (bottleneck SGEMM) and line 1983 (main forward).

Store `trainer_params_ptr` in `Self { ... }` replacing `online_params_flat`.

- [ ] **Step 6: Delete `sync_gpu_weights` from training_loop.rs**

Remove the method at lines 1475-1495 and the call at line 387.

- [ ] **Step 7: Fix collector construction call site in training_loop.rs**

Find where `GpuExperienceCollector::new(...)` is called. Replace the old params (weight sets) with the new params:

```rust
let collector = GpuExperienceCollector::new(
    stream.clone(),
    fused.params_flat_ptr(),          // trainer_params_ptr
    hyperparams.bottleneck_dim,       // bottleneck_dim
    42,                                // market_dim (always 42)
    initial_capital,
    avg_spread,
    cash_reserve_pct,
    network_dims,
    kernel_dims,
    n_episodes,
    timesteps_per_episode,
)?;
```

- [ ] **Step 8: Verify compilation and tests**

```bash
SQLX_OFFLINE=true cargo check -p ml --lib
SQLX_OFFLINE=true cargo test -p ml-dqn --lib
SQLX_OFFLINE=true cargo test -p ml --lib -- gradient_budget config monitoring
```

- [ ] **Step 9: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_experience_collector.rs crates/ml/src/trainers/dqn/trainer/training_loop.rs crates/ml/src/trainers/dqn/fused_training.rs
git commit -m "refactor: exp collector reads trainer params_buf directly — zero DtoD copies"
```

---

### Task 3: Smoke test + compute-sanitizer verification

**Files:** (no changes — verification only)

- [ ] **Step 1: Run smoke test and check action diversity**

```bash
FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib -- smoke_tests::feature_coverage --ignored --test-threads=1 --nocapture 2>&1 | grep -iE "Action diversity|LOW ORDER|test result"
```

Expected: order diversity should improve from 1/3 (zero weights → no Q-gap) to 2/3 or 3/3 (real Xavier-init weights → Boltzmann distributes across order types).

- [ ] **Step 2: Run compute-sanitizer**

```bash
SQLX_OFFLINE=true cargo test -p ml --lib --no-run 2>&1 | tail -1
FOXHUNT_TEST_DATA=test_data/futures-baseline compute-sanitizer --tool memcheck target/debug/deps/ml-*.test "smoke_tests::feature_coverage" --ignored --test-threads=1
```

Expected: `ERROR SUMMARY: 0 errors`

- [ ] **Step 3: Run full test suite**

```bash
SQLX_OFFLINE=true cargo test -p ml-dqn --lib
SQLX_OFFLINE=true cargo test -p ml --lib
FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib -- smoke_tests --ignored --test-threads=1
```

Expected: All pass.

- [ ] **Step 4: Commit and push**

```bash
git add -A
git commit -m "fix: exp collector zero-copy — direct pointer into trainer params_buf, bottleneck active"
git push origin main
```
