foxhunt/docs/superpowers/cleanup/dqn-cleanup-prompt.md

# DQN Cleanup Loop — Iteration Protocol

You are in a **Ralph Loop** doing code cleanup on the Foxhunt DQN training stack. Your role: run **one bounded cleanup iteration**, leaving the repo strictly better than you found it, then exit. Ralph will re-feed this exact prompt for the next iteration.

---

## 0. Non-negotiable project rules (read before anything)

- Working dir: `/home/jgrusewski/Work/foxhunt`
- Build: `SQLX_OFFLINE=true cargo check --workspace`
- Gate test: `SQLX_OFFLINE=true cargo test -p ml-dqn --lib`
- **NEVER** `git push`, `git reset --hard`, `git commit --no-verify`, `git push --force`, `cargo publish`
- **NEVER** touch: `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, or any `docs/` file other than the scoreboard.
- **NEVER** modify `*.cu` files in a way that alters numerical output. Pure constant-extraction only. Anything else goes to "Needs human review".
- **ALWAYS** Read a file before editing it.
- Respect `CLAUDE.md` at repo root and memory files at `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/`. The `feedback_*.md` files are law — especially:
  - `feedback_no_feature_flags.md` — `enable_*`/`use_*` booleans must be deleted, features unconditional.
  - `feedback_no_hiding.md` — wire up or delete; never suppress.
  - `feedback_gpu_cpu_roundtrip.md` — GPU→CPU roundtrips on the training hot path are never acceptable.
  - `feedback_no_legacy_aliases.md` — no deprecated wrappers; rename call sites directly.
  - `feedback_fix_everything.md` — zero tolerance for failures.

## 1. Scope (in priority order)

1. `crates/ml-dqn/`
2. `crates/ml/src/trainers/dqn/`
3. `crates/ml/src/cuda_pipeline/`
4. `crates/ml/src/hyperopt/adapters/dqn.rs`
5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/`
6. All other `crates/ml*` (ml-features, ml-data, ml-ppo, ml-regime, ml-supervised, ml-ensemble, ml-hyperopt, ml-labeling, ml-risk, ml-checkpoint, ml-core, ml-backtesting, ml-observability, ml-explainability, ml-validation, ml-data-validation, ml-asset-selection, ml-paper-trading, ml-security, ml-stress-testing, ml-universe, ml-regime-detection)
7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/`
8. `bin/fxt/` (DQN-related commands only)

## 2. State file

Durable scoreboard: `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md`

If it doesn't exist, create it with:

```markdown
---
iteration: 0
last_scan: 1970-01-01T00:00:00Z
last_scoreboard_hash: none
consecutive_stall_count: 0
status: active
---

## Open findings
<empty>

## Resolved this run
<empty>

## Needs human review
<empty>

## BLOCKED
<empty>
```

### Finding line format

```
- [ ] [CAT-###] <short description>: `<file>:<line>` — S=<1-5> B=<1-5> E=<1-5> score=<float>
```

Where `CAT` ∈ {PINMEM, CPURO, ROMEM, LOCKHOT, BORROW, GPUSYNC, FFLAG, FALLBACK, ACCOUNT, DIMMIX, DEAD, UNWRAP, MAGIC, PDRIFT, TESTROT, VERIFY}. IDs are stable per category — do not renumber.

## 3. Categories and scan commands

| ID prefix | Category | Severity | Scan |
|---|---|---|---|
| PINMEM | Unpinned `htod` / `dtod` transfers on training hot path — should stage through pinned host memory (`cuMemHostAlloc`) or be eliminated by moving the computation onto GPU. **Top priority.** | 5 | `rg -n '\b(memcpy_htod\|clone_htod\|htod_copy\|memcpy_dtod_sync\|dtod_copy)\b' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**' -g '!**/smoke_tests/**' -g '!**/examples/**'` |
| CPURO | CPU reads GPU-resident data on hot path (sizes, reductions, metrics that could live on device). A CPU-only read of GPU state forces synchronization even when no data is downloaded. | 5 | `rg -n '\.(len\|numel\|shape\|dims\|is_empty\|first\|last)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs` — then classify per call site |
| ROMEM | Write to CPU-read-only memory — `*(ptr as *mut T)` where `ptr` came from a `*const` / read-only pinned allocation / FFI handle marked const. Undefined behavior under CUDA's mapped-memory model. | 5 | `rg -n '(as \*mut\|cast_mut\(\))' crates/ml crates/ml-dqn services --type rust -g '!**/tests/**' -g '!**/benches/**'` — classify by origin of source pointer |
| LOCKHOT | `RwLock::(read\|write)` / `Mutex::lock` called on the training hot path (per-step / per-batch). Serializes the CPU side of the loop; also hides data races when the lock is held across an `await`. | 5 | `rg -n '\.(read\|write\|lock\|try_read\|try_write\|try_lock)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,network,gpu_replay_buffer}.rs` |
| BORROW | Shared-borrow integrity violations — `&T` promoted to `&mut T` via `as *mut` / `UnsafeCell` / `RefCell` / `transmute`, outside of documented FFI shims. | 5 | `rg -n '(UnsafeCell\|RefCell\|Cell<\|&\*.*as \*mut\|transmute.*\*const.*\*mut)' crates/ml crates/ml-dqn services --type rust -g '!**/tests/**'` |
| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 | `rg -n '(dtoh_sync\|sync_reclaim\|\.to_host\(\|htod_sync\|\.synchronize\(\)\|Vec::<f[0-9]+>::from\()' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**'` |
| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 | `rg -n '\b(enable_\|use_\|disable_)\w+\s*:\s*bool' crates/ml crates/ml-dqn services` |
| FALLBACK | Silent fallback paths (`unwrap_or_default`, etc.) | 5 | `rg -n 'unwrap_or_default\|unwrap_or_else\|\.or_else\(' crates/ml crates/ml-dqn services` |
| ACCOUNT | Accounting-only code (computed, logged, not plumbed) | 4 | Heuristic: find `let <x> = …;` where `<x>` appears only inside `info!`/`debug!`/`trace!` spans in the same function |
| DIMMIX | Dimension / layout drift | 4 | `rg -n '\b(62\|81\|42\|27)\b' crates/ml-dqn/src/*.cu crates/ml/src/cuda_pipeline/*.cu` → cross-check against `state_layout` constants |
| UNWRAP | `.unwrap()` on training hot path | 3 | `rg -n '\.unwrap\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs` |
| DEAD | Dead fields / unused functions / unreachable kernels | 3 | `SQLX_OFFLINE=true cargo check --workspace --message-format=json 2>&1 \| rg '"(dead_code\|unused)"'` |
| MAGIC | Magic numbers in kernel launch dims / hyperparams | 2 | `rg -n '\b(threads_per_block\|num_blocks\|shared_mem_bytes\|block_size)\s*[=:]\s*[0-9]' crates/ml crates/ml-dqn` |
| PDRIFT | Parameter name drift (same concept, different names) | 2 | `rg -n '\b(gamma\|discount\|tau\|target_update_rate\|alpha\|cql_alpha\|cql_weight)\b' crates/ml crates/ml-dqn` |
| TESTROT | Test rot (`#[ignore]` without ticket, broken builds) | 2 | `rg -n '#\[ignore' crates/ml crates/ml-dqn` + `SQLX_OFFLINE=true cargo test --workspace --no-run 2>&1 \| rg 'error\[E'` |
| VERIFY | Verification gate failed in Step 4 | 5 | (dynamic — added on verify failure) |

**Score formula:** `score = severity × blast_radius ÷ effort`
- `blast_radius`: 5 if touches a training kernel; 3 if trainer Rust; 1 if config/test.
- `effort`: 1 if fix <20 LOC; 3 if <100 LOC; 5 if <300 LOC.
- Tie-break: alphabetical by ID.

## 4. Iteration protocol (do these in order, do not skip)

### Step 1. Load state
Read `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` with the `Read` tool. If missing, create it per §2. Parse `iteration`, `last_scoreboard_hash`, `consecutive_stall_count`.

### Step 2. Refresh scans (selectively)
Re-run all 10 scanners from §3 **only if**: `last_scan` is older than 1 hour **or** Open findings has fewer than 5 entries.
- Append NEW findings (next available ID in the category).
- REMOVE Open findings whose `file:line` no longer matches the pattern (probably fixed by hand between iterations).
- Do **not** renumber existing IDs.

### Step 3. Stall detection
Compute `sha256` over the exact bytes of the `## Open findings` block (header line excluded).
- If equal to `last_scoreboard_hash` in frontmatter → increment `consecutive_stall_count`.
- If count reaches **2**: set `status: blocked`, write under `## BLOCKED`: "No progress in 2 iterations. Last attempted: `[CAT-###]`. Reason: `<error text or ambiguity>`", commit scoreboard, **emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`** and exit.
- Otherwise reset count to 0.

### Step 4. Completion gate
If `Open findings` is empty:
```bash
SQLX_OFFLINE=true cargo check --workspace
SQLX_OFFLINE=true cargo test -p ml-dqn --lib
```
- Both green → set `status: complete`, commit scoreboard, **emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`** and exit.
- Either red → add a `VERIFY-<YYYYMMDDHHMMSS>` finding with the exact failing output excerpt, S=5 B=5 E=2 score=12.5, continue to Step 5.

### Step 5. Select findings
Pick the top 1-3 findings such that **total estimated LOC ≤ 200** (use the `effort` score as a rough guide: E=1→20, E=3→100, E=5→300). Do not cherry-pick low-score items to pad.

### Step 6. Fix each selected finding
For each, in order:

**6a. Read the code.** Use the `Read` tool on the target file. Read 30 lines above and below the reported line. Never rely on memory of the file.

**6b. Classify.** Is this actually a bug / smell, or a false positive?
- False positive → move to `Resolved this run` with commit sha `none`, note `false-positive: <reason>`. No code change. Skip to next finding.

**6c. No deferrals (user directive, iter 7).** The `Needs human review` escape hatch is **closed**. Every finding must be solved in-iteration unless it's genuinely a false positive. If a finding looks hard:
- >200 LOC delta → split it into sub-findings (A/B/C suffix, e.g. FFLAG-018a / FFLAG-018b) and solve the tractable sub-finding this iteration, log the rest for the next. Do NOT defer.
- Touches kernel numerics → only constant-extraction is allowed; if the fix needs a kernel change, write the new kernel in a follow-up iteration, but do the Rust-side prep (rename, const-ify, add scaffolding) NOW.
- Semantic intent unclear → read the call sites, read the git blame if needed, decide. If the flag/field is unreferenced (write-only) it is dead regardless of what the name implies.
- Cross-cuts >3 files → still solve it. The ceiling is "3 commits per iter", not "3 files".

**6d. Apply the smallest correct fix.**
- `Read` → `Edit` only. Use `Write` only when creating a small new file is unavoidable.
- Feature-flag booleans: **delete the field, delete the `if flag {}` branch, keep the true-branch code, run call-site fix-ups**. See `feedback_no_feature_flags.md`.
- Fallback paths: **delete the fallback, propagate the missing-data case to the caller** (usually `return Err`). See `feedback_no_hiding.md`.
- GPU→CPU sync on hot path: move to a diagnostics-only path behind `#[cfg(debug_assertions)]` or a `diagnostics` feature, OR delete if purely informational. See `feedback_gpu_cpu_roundtrip.md`.
- `.unwrap()` on hot path: replace with `expect("<invariant that must hold>")` only if truly infallible, else propagate `Result`.
- Magic numbers in kernel launches: extract to `const` at module top with a comment explaining the value.
- Dead fields: `rg` to confirm zero readers in workspace. If confirmed, delete field + its write sites.

**6e. Verify.**
```bash
SQLX_OFFLINE=true cargo check -p <owning-crate>
```
If red → `git checkout -- <touched files>`, read the error, and retry the fix correctly. Do NOT defer. If the error is in a file you did not touch (pre-existing user WIP), note it in scoreboard's "Known external state" section and continue with your fix.

**6f. Commit (one finding per commit).**
```bash
git add <specific file paths — no -A, no .>
git commit -m "cleanup(<cat-lowercase>): <one-line fix> — [CAT-###]"
```
Use a HEREDOC body with 2-4 lines explaining: what was wrong, what changed, why it's safe.
Never `--no-verify`. Never `git add -A`. Never `git add .`.

Record the commit sha in `## Resolved this run`.

### Step 7. Update scoreboard frontmatter
- `iteration += 1`
- `last_scan` = current ISO-8601 UTC (only if Step 2 re-scanned)
- `last_scoreboard_hash` = sha256 of the new Open findings block
- `consecutive_stall_count` updated per Step 3
- Commit scoreboard separately: `git commit -m "cleanup: update scoreboard — iter <N>"`

### Step 8. Exit normally
Do **not** emit the completion promise unless Step 3 or Step 4 said so.

Finish with a 3-line summary to stdout:
```
Iteration <N>: fixed <k>, false-positive <f>.
Open findings: <count>. Next iteration will target [CAT-###], [CAT-###].
```

Ralph re-feeds this exact prompt.

---

## 5. Hard rules (recap — most important)

1. **Per-iteration ceiling**: 3 commits, 200 total LOC changed, 10 files touched. Stop early if reached.
2. **Never** touch `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, `docs/` except the scoreboard.
3. **Never** delete code unless `rg` this iteration confirms zero callers workspace-wide.
4. **Never** alter `*.cu` numerical behavior. Constant extraction only.
5. **Never** bypass pre-commit hooks.
6. **One finding per commit.** Specific files in `git add`, never `-A` / `.`.
7. **Cargo check breaks** → revert, diagnose the actual error, fix it properly. Never defer via Needs-human.
8. **Emit the completion promise ONLY** when Step 4 passes both gates, or Step 3 hit a 2-iteration stall. Never as an escape hatch.

---

## 6. Learned patterns (iters 1-8)

Past iterations surfaced recurring shapes. Recognize them on sight:

### Dead-flag chains (most common)
A config field plumbed across multiple structs but never actually read:
- Pattern: `SomeConfig.enable_foo: bool` → local `let foo_enabled = config.enable_foo` → assigned into another struct's `.enabled: bool` → zero workspace readers.
- Signal: comment on the default setter says `// always on` or `// always enabled`.
- Examples fixed: FFLAG-009, FFLAG-011, FFLAG-012, FFLAG-013a, FFLAG-013b.
- Fix: delete every link in the chain (field + default + local + downstream struct field). One commit.

### Aspirational config (knobs with zero implementation)
Config struct has `use_xyz: bool` but no code branches on it anywhere — the feature was never actually built. Examples: `FFLAG-016` (4 transformer flags), `FFLAG-017` (3 feature-set flags).
- Fix: delete the fields. Do NOT try to implement the feature — if the user wants it, they'll say so.

### Dishonest fallback functions
A function labeled "fallback" or "legacy" that actually calls the primary path internally and discards its own work. Example: `standard_attention` in flash_attention (did Q@K^T, scale, mask, then threw it all away and called `io_aware.compute_attention`).
- Fix: delete the fallback + its caller's if/else gate. Often removes 40-80 LOC.

### Inference-side flag flips
A flag defaults `true` everywhere in training configs but a service (`services/trading_service/...`) flips it `false` at inference. Example: FFLAG-007 `use_count_bonus` (UCB exploration bonus — on for training, off for live trading).
- Fix: split the function into `_training()` and `_inference()` variants. Do NOT delete the flag without replacing its semantics.

### Accounting-only values
A variable is computed, logged, and never multiplied into gradients/outputs. Example (from user history): F8/G5 `c51_budget` — was accounting-only until explicitly wired as a real grad scale.
- Signal: `let x = …; info!("…{x}…")` where `x` never reappears outside that span.
- Fix: either wire it into the real computation or delete it.

### Orphan files
A `.rs` file exists but no `mod foo;` declaration anywhere. Compiler never sees it. Example: `transformers/features.rs` (126 LOC of dangling test code).
- Find via: `rg -l '^' crates/*/src/**/*.rs` then cross-check against every `mod.rs` and `lib.rs`.
- Fix: delete the file.

### Hidden GPU→CPU sync in cold paths that hit the hot path
A function advertises "cold path" or "Zero CPU roundtrip" in its docstring but the actual implementation does `to_host()` per batch. Hot-path callers sometimes bypass this, but validation paths still hit it. Examples: GPUSYNC-006 (curiosity MSE), GPUSYNC-008 (regime classification).
- Fix: write a real GPU kernel for the primitive. Do NOT leave the lying docstring in place.

### Unpinned htod/dtod transfers on the training hot path
Pageable-host → device memcpies bounce through an OS staging buffer (one extra copy per transfer, no DMA overlap with compute). Pinned host memory (allocated via `cuMemHostAlloc`) enables direct DMA and async overlap with streams. This codebase already uses pinned memory for critical scalars (`size_pinned`, `rng_step_pinned` in `gpu_replay_buffer.rs`) — extend the pattern to every per-step transfer.

- **Signal**: `stream.memcpy_htod(&host_vec, &mut device_buf)` or `stream.clone_htod(&host_vec)` where `host_vec` is a plain `Vec<T>` / `&[T]`, invoked once per training step / per batch / per epoch.
- **Benign**: transfers in constructors (one-time init), `examples/`, `smoke_tests/`, `tests/`, or explicit "cold path" validation code. Scan excludes those directories.
- **Fix hierarchy (in order of preference)**:
  1. **Eliminate the transfer** — if the host value can be produced on GPU (e.g. `cuRAND` instead of `thread_rng()`, or an on-device counter instead of a Rust-side index), do that. See `dqn_utility_kernels.cu` comment: "Replaces CPU `rand::thread_rng()` + `memcpy_htod` that ran every training step."
  2. **Stage through pinned memory** — alloc a pinned host buffer once at init, write into it per step, then `memcpy_htod_async` from pinned → device. Follow the existing `unsafe { cuMemHostAlloc(...) }` pattern in `gpu_replay_buffer.rs:282`.
  3. **For `dtod_copy` / `memcpy_dtod_sync`**: switch to `memcpy_dtod_async` on the compute stream. Sync variants serialize the GPU.
- **Commit-per-site**: each hot-path htod gets its own `cleanup(pinmem):` commit so regressions can be bisected.

### CPU reads of GPU-resident data on hot path (CPURO)
Even when no data crosses the PCIe bus, a CPU-side `.len()` / `.shape()` / `.numel()` / `.is_empty()` call on a GPU buffer forces the host thread to synchronize on the cudarc handle, which stalls the kernel launch pipeline. Symptoms are subtler than GPUSYNC (no `to_host`) but the stall is real.

- **Signal**: `gpu_tensor.len()`, `slice.numel()`, `buf.shape()[0]` inside a per-step function.
- **Fix hierarchy**:
  1. **Cache at construction** — store `batch_size` / `n_actions` / `state_dim` as plain `usize` fields on the trainer, populated once in `new()`. Stop re-reading them on each step.
  2. **Keep running statistics on GPU** — reductions (mean/max/sum) via `ReductionKernels::stats()` (already wired for Q-value monitoring) instead of `.iter().sum()`.
  3. **Use event polling** — if the CPU genuinely needs to observe a GPU counter, use `CudaEvent::record()` + `event.query()` (non-blocking) or pinned device-mapped memory (already used for `size_pinned` in gpu_replay_buffer).

### Writes to CPU-read-only memory (ROMEM)
CUDA's pinned / mapped memory model designates some host buffers as read-only from the CPU side (GPU writes, CPU reads). Casting a `*const` to `*mut` and writing through it is undefined behavior, even if tests happen to pass. FFI shims that wrap `cublas_sys` / `cudnn_sys` APIs legitimately take `*mut` void ptrs for workspaces — those are fine, but flag them so they stay documented.

- **Signal**: `*(ptr as *mut T) = value` where `ptr` is declared `*const`, comes from `cuMemHostAllocReadOnly`, or was given to `stream.memcpy_dtoh_sync()` as destination.
- **Benign exceptions**: cuBLAS / cuDNN / cutlass workspace pointers (the C APIs take `*mut void` purely for calling convention, not because they write).
- **Fix**: either change the allocation flag to include `cuMemHostAllocMapped` / `cuMemHostAllocWriteCombined`, or split into two buffers (one RO for CPU reads, one WO for GPU writes). Never silently upgrade `*const` to `*mut`.

### Lock contention on the training hot path (LOCKHOT)
`RwLock::read()` and `Mutex::lock()` on the per-step code path serialize the CPU side of training and can hide deadlocks behind `.await` points when using `tokio::sync::RwLock`.

- **Signal**: `self.thing.lock()` or `self.thing.read().await` inside the training inner loop, forward pass, or replay-buffer insert path.
- **High-value targets** in this codebase: `std::sync::Mutex<GpuDropout>` and `Mutex<Option<DropoutScheduler>>` in `network.rs` (fired every forward pass), `Arc<Mutex<NStepBuffer>>` in `dqn.rs` (fired per step), `Arc<RwLock<TrainingMetrics>>` in `constructor.rs` (read frequency unclear — verify).
- **Fix hierarchy**:
  1. **Remove the lock** — if the wrapped state is only touched from one thread, the `Mutex` is cargo-culted. Move to plain `&mut self`.
  2. **Move the lock outside the hot loop** — acquire once per epoch, not per step.
  3. **Use lock-free primitives** — `AtomicU64`, `AtomicF32` (via bit-cast), or single-producer-single-consumer channels.
  4. **Never hold `tokio::sync::RwLock` across `.await`** — take a snapshot and drop the guard before yielding.

### Shared-borrow integrity violations (BORROW)
Rust's `&T` is a shared read; promoting it to `&mut T` via `as *mut`, `transmute`, or interior mutability (`UnsafeCell`, `RefCell`, `Cell<*mut ...>`) breaks the aliasing contract the optimizer relies on. Miri / ThreadSanitizer will catch it; clippy may not.

- **Signal (regex)**: `let dst = &mut *(&mut self.field as *mut X as *mut Y);` — transmute-via-ptr-cast to change the element type of a `&mut` reference.
- **Real example (already shipped)**: `gpu_replay_buffer.rs:690` — `&mut *(&mut self.sample_episode_ids as *mut CudaSlice<i32> as *mut CudaSlice<u32>)`. This changes the element type from i32 to u32 through a `&mut`, which is UB under strict aliasing even though the underlying bytes are identical.
- **Fix hierarchy**:
  1. **Use a type-erased buffer** — store as `CudaSlice<u8>` and re-interpret via a typed view function that returns `&[i32]` or `&[u32]` explicitly.
  2. **Transmute the slice reference, not through `&mut *`** — `std::slice::from_raw_parts_mut(ptr as *mut u32, n)` is more explicit (still unsafe but at least the reader knows).
  3. **Change the declared type** — if the buffer is always read as u32, declare it as `CudaSlice<u32>`; convert at write time, not read time.

Begin with Step 1.