foxhunt/docs/superpowers/cleanup/dqn-cleanup-prompt.md

DQN Cleanup Loop — Iteration Protocol

You are in a Ralph Loop doing code cleanup on the Foxhunt DQN training stack. Your role: run one bounded cleanup iteration, leaving the repo strictly better than you found it, then exit. Ralph will re-feed this exact prompt for the next iteration.

---

0. Non-negotiable project rules (read before anything)

• Working dir: /home/jgrusewski/Work/foxhunt
• Build: SQLX_OFFLINE=true cargo check --workspace
• Gate test: SQLX_OFFLINE=true cargo test -p ml-dqn --lib
• NEVER git push, git reset --hard, git commit --no-verify, git push --force, cargo publish
• NEVER touch: infra/, scripts/argo-*, .gitlab-ci.yml, .claude/, or any docs/ file other than the scoreboard.
• NEVER modify *.cu files in a way that alters numerical output. Pure constant-extraction only. Anything else goes to "Needs human review".
• ALWAYS Read a file before editing it.
• Respect CLAUDE.md at repo root and memory files at /home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/. The feedback_*.md files are law — especially:
  • feedback_no_feature_flags.md — enable_*/use_* booleans must be deleted, features unconditional.
  • feedback_no_hiding.md — wire up or delete; never suppress.
  • feedback_gpu_cpu_roundtrip.md — GPU→CPU roundtrips on the training hot path are never acceptable.
  • feedback_no_legacy_aliases.md — no deprecated wrappers; rename call sites directly.
  • feedback_fix_everything.md — zero tolerance for failures.

1. Scope (in priority order)

1. crates/ml-dqn/
2. crates/ml/src/trainers/dqn/
3. crates/ml/src/cuda_pipeline/
4. crates/ml/src/hyperopt/adapters/dqn.rs
5. crates/ml/src/data_loaders/, crates/ml/src/data_pipeline/
6. All other crates/ml* (ml-features, ml-data, ml-ppo, ml-regime, ml-supervised, ml-ensemble, ml-hyperopt, ml-labeling, ml-risk, ml-checkpoint, ml-core, ml-backtesting, ml-observability, ml-explainability, ml-validation, ml-data-validation, ml-asset-selection, ml-paper-trading, ml-security, ml-stress-testing, ml-universe, ml-regime-detection)
7. services/ml_training_service/, services/data_acquisition_service/, services/backtesting_service/
8. bin/fxt/ (DQN-related commands only)

2. State file

Durable scoreboard: docs/superpowers/cleanup/dqn-cleanup-scoreboard.md

If it doesn't exist, create it with:

---
iteration: 0
last_scan: 1970-01-01T00:00:00Z
last_scoreboard_hash: none
consecutive_stall_count: 0
status: active
---

## Open findings
<empty>

## Resolved this run
<empty>

## Needs human review
<empty>

## BLOCKED
<empty>

Finding line format

- [ ] [CAT-###] <short description>: `<file>:<line>` — S=<1-5> B=<1-5> E=<1-5> score=<float>

Where CAT ∈ {PINMEM, CPURO, ROMEM, LOCKHOT, BORROW, GPUSYNC, FFLAG, FALLBACK, ACCOUNT, DIMMIX, DEAD, UNWRAP, MAGIC, PDRIFT, TESTROT, VERIFY}. IDs are stable per category — do not renumber.

3. Categories and scan commands

ID prefix	Category	Severity	Scan
PINMEM	Unpinned htod / dtod transfers on training hot path — should stage through pinned host memory (cuMemHostAlloc) or be eliminated by moving the computation onto GPU. Top priority.	5	rg -n '\b(memcpy_htod|clone_htod|htod_copy|memcpy_dtod_sync|dtod_copy)\b' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**' -g '!**/smoke_tests/**' -g '!**/examples/**'
CPURO	CPU reads GPU-resident data on hot path (sizes, reductions, metrics that could live on device). A CPU-only read of GPU state forces synchronization even when no data is downloaded.	5	rg -n '\.(len|numel|shape|dims|is_empty|first|last)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs — then classify per call site
ROMEM	Write to CPU-read-only memory — *(ptr as *mut T) where ptr came from a *const / read-only pinned allocation / FFI handle marked const. Undefined behavior under CUDA's mapped-memory model.	5	rg -n '(as \*mut|cast_mut\(\))' crates/ml crates/ml-dqn services --type rust -g '!**/tests/**' -g '!**/benches/**' — classify by origin of source pointer
LOCKHOT	RwLock::(read|write) / Mutex::lock called on the training hot path (per-step / per-batch). Serializes the CPU side of the loop; also hides data races when the lock is held across an await.	5	rg -n '\.(read|write|lock|try_read|try_write|try_lock)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,network,gpu_replay_buffer}.rs
BORROW	Shared-borrow integrity violations — &T promoted to &mut T via as *mut / UnsafeCell / RefCell / transmute, outside of documented FFI shims.	5	rg -n '(UnsafeCell|RefCell|Cell<|&\*.*as \*mut|transmute.*\*const.*\*mut)' crates/ml crates/ml-dqn services --type rust -g '!**/tests/**'
GPUSYNC	Hidden GPU→CPU sync on hot path	5	rg -n '(dtoh_sync|sync_reclaim|\.to_host\(|htod_sync|\.synchronize\(\)|Vec::<f[0-9]+>::from\()' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**'
FFLAG	enable_* / use_* / disable_* booleans	5	rg -n '\b(enable_|use_|disable_)\w+\s*:\s*bool' crates/ml crates/ml-dqn services
FALLBACK	Silent fallback paths (unwrap_or_default, etc.)	5	rg -n 'unwrap_or_default|unwrap_or_else|\.or_else\(' crates/ml crates/ml-dqn services
ACCOUNT	Accounting-only code (computed, logged, not plumbed)	4	Heuristic: find let <x> = …; where <x> appears only inside info!/debug!/trace! spans in the same function
DIMMIX	Dimension / layout drift	4	rg -n '\b(62|81|42|27)\b' crates/ml-dqn/src/*.cu crates/ml/src/cuda_pipeline/*.cu → cross-check against state_layout constants
UNWRAP	.unwrap() on training hot path	3	rg -n '\.unwrap\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs
DEAD	Dead fields / unused functions / unreachable kernels	3	SQLX_OFFLINE=true cargo check --workspace --message-format=json 2>&1 | rg '"(dead_code|unused)"'
MAGIC	Magic numbers in kernel launch dims / hyperparams	2	rg -n '\b(threads_per_block|num_blocks|shared_mem_bytes|block_size)\s*[=:]\s*[0-9]' crates/ml crates/ml-dqn
PDRIFT	Parameter name drift (same concept, different names)	2	rg -n '\b(gamma|discount|tau|target_update_rate|alpha|cql_alpha|cql_weight)\b' crates/ml crates/ml-dqn
TESTROT	Test rot (#[ignore] without ticket, broken builds)	2	rg -n '#\[ignore' crates/ml crates/ml-dqn + SQLX_OFFLINE=true cargo test --workspace --no-run 2>&1 | rg 'error\[E'
VERIFY	Verification gate failed in Step 4	5	(dynamic — added on verify failure)

Score formula: score = severity × blast_radius ÷ effort

• blast_radius: 5 if touches a training kernel; 3 if trainer Rust; 1 if config/test.
• effort: 1 if fix <20 LOC; 3 if <100 LOC; 5 if <300 LOC.
• Tie-break: alphabetical by ID.

4. Iteration protocol (do these in order, do not skip)

Step 1. Load state

Read docs/superpowers/cleanup/dqn-cleanup-scoreboard.md with the Read tool. If missing, create it per §2. Parse iteration, last_scoreboard_hash, consecutive_stall_count.

Step 2. Refresh scans (selectively)

Re-run all 10 scanners from §3 only if: last_scan is older than 1 hour or Open findings has fewer than 5 entries.

• Append NEW findings (next available ID in the category).
• REMOVE Open findings whose file:line no longer matches the pattern (probably fixed by hand between iterations).
• Do not renumber existing IDs.

Step 3. Stall detection

Compute sha256 over the exact bytes of the ## Open findings block (header line excluded).

• If equal to last_scoreboard_hash in frontmatter → increment consecutive_stall_count.
• If count reaches 2: set status: blocked, write under ## BLOCKED: "No progress in 2 iterations. Last attempted: [CAT-###]. Reason: <error text or ambiguity>", commit scoreboard, emit <promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise> and exit.
• Otherwise reset count to 0.

Step 4. Completion gate

If Open findings is empty:

SQLX_OFFLINE=true cargo check --workspace
SQLX_OFFLINE=true cargo test -p ml-dqn --lib

• Both green → set status: complete, commit scoreboard, emit <promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise> and exit.
• Either red → add a VERIFY-<YYYYMMDDHHMMSS> finding with the exact failing output excerpt, S=5 B=5 E=2 score=12.5, continue to Step 5.

Step 5. Select findings

Pick the top 1-3 findings such that total estimated LOC ≤ 200 (use the effort score as a rough guide: E=1→20, E=3→100, E=5→300). Do not cherry-pick low-score items to pad.

Step 6. Fix each selected finding

For each, in order:

6a. Read the code. Use the Read tool on the target file. Read 30 lines above and below the reported line. Never rely on memory of the file.

6b. Classify. Is this actually a bug / smell, or a false positive?

• False positive → move to Resolved this run with commit sha none, note false-positive: <reason>. No code change. Skip to next finding.

6c. No deferrals (user directive, iter 7). The Needs human review escape hatch is closed. Every finding must be solved in-iteration unless it's genuinely a false positive. If a finding looks hard:

• 200 LOC delta → split it into sub-findings (A/B/C suffix, e.g. FFLAG-018a / FFLAG-018b) and solve the tractable sub-finding this iteration, log the rest for the next. Do NOT defer.

• Touches kernel numerics → only constant-extraction is allowed; if the fix needs a kernel change, write the new kernel in a follow-up iteration, but do the Rust-side prep (rename, const-ify, add scaffolding) NOW.
• Semantic intent unclear → read the call sites, read the git blame if needed, decide. If the flag/field is unreferenced (write-only) it is dead regardless of what the name implies.
• Cross-cuts >3 files → still solve it. The ceiling is "3 commits per iter", not "3 files".

6d. Apply the smallest correct fix.

• Read → Edit only. Use Write only when creating a small new file is unavoidable.
• Feature-flag booleans: delete the field, delete the if flag {} branch, keep the true-branch code, run call-site fix-ups. See feedback_no_feature_flags.md.
• Fallback paths: delete the fallback, propagate the missing-data case to the caller (usually return Err). See feedback_no_hiding.md.
• GPU→CPU sync on hot path: move to a diagnostics-only path behind #[cfg(debug_assertions)] or a diagnostics feature, OR delete if purely informational. See feedback_gpu_cpu_roundtrip.md.
• .unwrap() on hot path: replace with expect("<invariant that must hold>") only if truly infallible, else propagate Result.
• Magic numbers in kernel launches: extract to const at module top with a comment explaining the value.
• Dead fields: rg to confirm zero readers in workspace. If confirmed, delete field + its write sites.

6e. Verify.

SQLX_OFFLINE=true cargo check -p <owning-crate>

If red → git checkout -- <touched files>, read the error, and retry the fix correctly. Do NOT defer. If the error is in a file you did not touch (pre-existing user WIP), note it in scoreboard's "Known external state" section and continue with your fix.

6f. Commit (one finding per commit).

git add <specific file paths — no -A, no .>
git commit -m "cleanup(<cat-lowercase>): <one-line fix> — [CAT-###]"

Use a HEREDOC body with 2-4 lines explaining: what was wrong, what changed, why it's safe. Never --no-verify. Never git add -A. Never git add ..

Record the commit sha in ## Resolved this run.

Step 7. Update scoreboard frontmatter

• iteration += 1
• last_scan = current ISO-8601 UTC (only if Step 2 re-scanned)
• last_scoreboard_hash = sha256 of the new Open findings block
• consecutive_stall_count updated per Step 3
• Commit scoreboard separately: git commit -m "cleanup: update scoreboard — iter <N>"

Step 8. Exit normally

Do not emit the completion promise unless Step 3 or Step 4 said so.

Finish with a 3-line summary to stdout:

Iteration <N>: fixed <k>, false-positive <f>.
Open findings: <count>. Next iteration will target [CAT-###], [CAT-###].

Ralph re-feeds this exact prompt.

---

5. Hard rules (recap — most important)

1. Per-iteration ceiling: 3 commits, 200 total LOC changed, 10 files touched. Stop early if reached.
2. Never touch infra/, scripts/argo-*, .gitlab-ci.yml, .claude/, docs/ except the scoreboard.
3. Never delete code unless rg this iteration confirms zero callers workspace-wide.
4. Never alter *.cu numerical behavior. Constant extraction only.
5. Never bypass pre-commit hooks.
6. One finding per commit. Specific files in git add, never -A / ..
7. Cargo check breaks → revert, diagnose the actual error, fix it properly. Never defer via Needs-human.
8. Emit the completion promise ONLY when Step 4 passes both gates, or Step 3 hit a 2-iteration stall. Never as an escape hatch.

---

6. Learned patterns (iters 1-8)

Past iterations surfaced recurring shapes. Recognize them on sight:

Dead-flag chains (most common)

A config field plumbed across multiple structs but never actually read:

• Pattern: SomeConfig.enable_foo: bool → local let foo_enabled = config.enable_foo → assigned into another struct's .enabled: bool → zero workspace readers.
• Signal: comment on the default setter says // always on or // always enabled.
• Examples fixed: FFLAG-009, FFLAG-011, FFLAG-012, FFLAG-013a, FFLAG-013b.
• Fix: delete every link in the chain (field + default + local + downstream struct field). One commit.

Aspirational config (knobs with zero implementation)

Config struct has use_xyz: bool but no code branches on it anywhere — the feature was never actually built. Examples: FFLAG-016 (4 transformer flags), FFLAG-017 (3 feature-set flags).

• Fix: delete the fields. Do NOT try to implement the feature — if the user wants it, they'll say so.

Dishonest fallback functions

A function labeled "fallback" or "legacy" that actually calls the primary path internally and discards its own work. Example: standard_attention in flash_attention (did Q@K^T, scale, mask, then threw it all away and called io_aware.compute_attention).

• Fix: delete the fallback + its caller's if/else gate. Often removes 40-80 LOC.

Inference-side flag flips

A flag defaults true everywhere in training configs but a service (services/trading_service/...) flips it false at inference. Example: FFLAG-007 use_count_bonus (UCB exploration bonus — on for training, off for live trading).

• Fix: split the function into _training() and _inference() variants. Do NOT delete the flag without replacing its semantics.

Accounting-only values

A variable is computed, logged, and never multiplied into gradients/outputs. Example (from user history): F8/G5 c51_budget — was accounting-only until explicitly wired as a real grad scale.

• Signal: let x = …; info!("…{x}…") where x never reappears outside that span.
• Fix: either wire it into the real computation or delete it.

Orphan files

A .rs file exists but no mod foo; declaration anywhere. Compiler never sees it. Example: transformers/features.rs (126 LOC of dangling test code).

• Find via: rg -l '^' crates/*/src/**/*.rs then cross-check against every mod.rs and lib.rs.
• Fix: delete the file.

Hidden GPU→CPU sync in cold paths that hit the hot path

A function advertises "cold path" or "Zero CPU roundtrip" in its docstring but the actual implementation does to_host() per batch. Hot-path callers sometimes bypass this, but validation paths still hit it. Examples: GPUSYNC-006 (curiosity MSE), GPUSYNC-008 (regime classification).

• Fix: write a real GPU kernel for the primitive. Do NOT leave the lying docstring in place.

Unpinned htod/dtod transfers on the training hot path

Pageable-host → device memcpies bounce through an OS staging buffer (one extra copy per transfer, no DMA overlap with compute). Pinned host memory (allocated via cuMemHostAlloc) enables direct DMA and async overlap with streams. This codebase already uses pinned memory for critical scalars (size_pinned, rng_step_pinned in gpu_replay_buffer.rs) — extend the pattern to every per-step transfer.

• Signal: stream.memcpy_htod(&host_vec, &mut device_buf) or stream.clone_htod(&host_vec) where host_vec is a plain Vec<T> / &[T], invoked once per training step / per batch / per epoch.
• Benign: transfers in constructors (one-time init), examples/, smoke_tests/, tests/, or explicit "cold path" validation code. Scan excludes those directories.
• Fix hierarchy (in order of preference):
  1. Eliminate the transfer — if the host value can be produced on GPU (e.g. cuRAND instead of thread_rng(), or an on-device counter instead of a Rust-side index), do that. See dqn_utility_kernels.cu comment: "Replaces CPU rand::thread_rng() + memcpy_htod that ran every training step."
  2. Stage through pinned memory — alloc a pinned host buffer once at init, write into it per step, then memcpy_htod_async from pinned → device. Follow the existing unsafe { cuMemHostAlloc(...) } pattern in gpu_replay_buffer.rs:282.
  3. For dtod_copy / memcpy_dtod_sync: switch to memcpy_dtod_async on the compute stream. Sync variants serialize the GPU.
• Commit-per-site: each hot-path htod gets its own cleanup(pinmem): commit so regressions can be bisected.

CPU reads of GPU-resident data on hot path (CPURO)

Even when no data crosses the PCIe bus, a CPU-side .len() / .shape() / .numel() / .is_empty() call on a GPU buffer forces the host thread to synchronize on the cudarc handle, which stalls the kernel launch pipeline. Symptoms are subtler than GPUSYNC (no to_host) but the stall is real.

• Signal: gpu_tensor.len(), slice.numel(), buf.shape()[0] inside a per-step function.
• Fix hierarchy:
  1. Cache at construction — store batch_size / n_actions / state_dim as plain usize fields on the trainer, populated once in new(). Stop re-reading them on each step.
  2. Keep running statistics on GPU — reductions (mean/max/sum) via ReductionKernels::stats() (already wired for Q-value monitoring) instead of .iter().sum().
  3. Use event polling — if the CPU genuinely needs to observe a GPU counter, use CudaEvent::record() + event.query() (non-blocking) or pinned device-mapped memory (already used for size_pinned in gpu_replay_buffer).

Writes to CPU-read-only memory (ROMEM)

CUDA's pinned / mapped memory model designates some host buffers as read-only from the CPU side (GPU writes, CPU reads). Casting a *const to *mut and writing through it is undefined behavior, even if tests happen to pass. FFI shims that wrap cublas_sys / cudnn_sys APIs legitimately take *mut void ptrs for workspaces — those are fine, but flag them so they stay documented.

• Signal: *(ptr as *mut T) = value where ptr is declared *const, comes from cuMemHostAllocReadOnly, or was given to stream.memcpy_dtoh_sync() as destination.
• Benign exceptions: cuBLAS / cuDNN / cutlass workspace pointers (the C APIs take *mut void purely for calling convention, not because they write).
• Fix: either change the allocation flag to include cuMemHostAllocMapped / cuMemHostAllocWriteCombined, or split into two buffers (one RO for CPU reads, one WO for GPU writes). Never silently upgrade *const to *mut.

Lock contention on the training hot path (LOCKHOT)

RwLock::read() and Mutex::lock() on the per-step code path serialize the CPU side of training and can hide deadlocks behind .await points when using tokio::sync::RwLock.

• Signal: self.thing.lock() or self.thing.read().await inside the training inner loop, forward pass, or replay-buffer insert path.
• High-value targets in this codebase: std::sync::Mutex<GpuDropout> and Mutex<Option<DropoutScheduler>> in network.rs (fired every forward pass), Arc<Mutex<NStepBuffer>> in dqn.rs (fired per step), Arc<RwLock<TrainingMetrics>> in constructor.rs (read frequency unclear — verify).
• Fix hierarchy:
  1. Remove the lock — if the wrapped state is only touched from one thread, the Mutex is cargo-culted. Move to plain &mut self.
  2. Move the lock outside the hot loop — acquire once per epoch, not per step.
  3. Use lock-free primitives — AtomicU64, AtomicF32 (via bit-cast), or single-producer-single-consumer channels.
  4. Never hold tokio::sync::RwLock across .await — take a snapshot and drop the guard before yielding.

Shared-borrow integrity violations (BORROW)

Rust's &T is a shared read; promoting it to &mut T via as *mut, transmute, or interior mutability (UnsafeCell, RefCell, Cell<*mut ...>) breaks the aliasing contract the optimizer relies on. Miri / ThreadSanitizer will catch it; clippy may not.

• Signal (regex): let dst = &mut *(&mut self.field as *mut X as *mut Y); — transmute-via-ptr-cast to change the element type of a &mut reference.
• Real example (already shipped): gpu_replay_buffer.rs:690 — &mut *(&mut self.sample_episode_ids as *mut CudaSlice<i32> as *mut CudaSlice<u32>). This changes the element type from i32 to u32 through a &mut, which is UB under strict aliasing even though the underlying bytes are identical.
• Fix hierarchy:
  1. Use a type-erased buffer — store as CudaSlice<u8> and re-interpret via a typed view function that returns &[i32] or &[u32] explicitly.
  2. Transmute the slice reference, not through &mut * — std::slice::from_raw_parts_mut(ptr as *mut u32, n) is more explicit (still unsafe but at least the reader knows).
  3. Change the declared type — if the buffer is always read as u32, declare it as CudaSlice<u32>; convert at write time, not read time.

Begin with Step 1.