# DQN Cleanup Ralph Loop — Design

**Date:** 2026-04-20
**Status:** Approved, ready for execution
**Artifact:** `docs/superpowers/cleanup/dqn-cleanup-prompt.md`
**State file:** `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md`

## Problem

The DQN training stack has accumulated technical debt that a one-shot session cannot surface: 21K Rust LOC + 1050 CUDA LOC in `ml-dqn`; 127K Rust LOC + 30 kernels in `ml`; the largest single file (`gpu_dqn_trainer.rs`) is 13,399 lines. Recent post-mortems (PER 1D-for-2D kernel, OFI all-zero, F8/G5 accounting-only c51_budget) show the class of bug that hides at this scale: plumbing that looks wired but isn't, fallback paths that mask missing data, feature flags that were never flipped back off.

## Goals

1. Sweep the DQN-surface codebase for 10 defined smell categories.
2. Fix each as an isolated, reviewable commit.
3. Be resumable: a single invocation can be interrupted and re-run without losing progress.
4. Respect the zero-tolerance rules in `~/.claude/projects/.../memory/feedback_*.md`.
5. Auto-terminate when scoreboard is empty AND verification gates pass.

## Non-goals

- Refactors that don't serve a specific finding.
- Kernel numerical changes (kernels are touched only for constant-extraction).
- Anything in `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`.

## Approach (chosen: C — Scoreboard-driven bounded-batch)

A durable scoreboard at `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` carries state between iterations. Each Ralph iteration does exactly one bounded pass: refresh scans → select 1-3 top findings → fix each as its own commit → update scoreboard → exit. Ralph re-feeds the prompt until the scoreboard is empty and `cargo check --workspace` + `cargo test -p ml-dqn --lib` are green, at which point the loop emits the completion promise.

### Why scoreboard-driven

- **Idempotent prompt.** Ralph's model is re-feeding the same text; state must live on disk, not in the prompt.
- **Resumable.** User can Ctrl-C, inspect commits, resume.
- **Bounded blast radius.** A 200-LOC-per-iteration ceiling prevents runaway refactors.
- **Humans can audit between iterations.** Each iteration leaves clean commits + an updated scoreboard.

### Rejected alternatives

- **A. Single-finding-per-iteration.** Too slow (200+ iterations). No meaningful advantage over C's bounded batch.
- **B. Category-per-iteration.** Mass rewrites are too risky on a production trading codebase. One category can easily exceed 200 LOC.

## Scope

1. `crates/ml-dqn/`
2. `crates/ml/src/trainers/dqn/`
3. `crates/ml/src/cuda_pipeline/`
4. `crates/ml/src/hyperopt/adapters/dqn.rs`
5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/`
6. All other `crates/ml*` (23 crates)
7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/`
8. `bin/fxt/` (DQN commands only)

## Categories

Ten scanner-backed categories with severity weights tied to memory-file priorities:

| ID | Category | Severity |
|---|---|---|
| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 |
| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 |
| FALLBACK | Silent fallback paths | 5 |
| ACCOUNT | Accounting-only (computed, never used) | 4 |
| DIMMIX | Dimension / layout drift | 4 |
| UNWRAP | `.unwrap()` on training hot path | 3 |
| DEAD | Dead fields / unused functions | 3 |
| MAGIC | Magic numbers in launch dims / hyperparams | 2 |
| PDRIFT | Parameter name drift | 2 |
| TESTROT | Test rot (#[ignore], broken builds) | 2 |

Plus a dynamic `VERIFY` category for verification-gate failures.

Score formula: `severity × blast_radius ÷ effort`. Tie-break alphabetical.

## Scoreboard schema

YAML frontmatter + four markdown sections (`Open findings`, `Resolved this run`, `Needs human review`, `BLOCKED`). Finding IDs are stable per category (`GPUSYNC-001` … `GPUSYNC-999`). Hashes detect stalls.

## Iteration protocol

8 steps in the prompt: (1) Load state (2) Refresh scans selectively (3) Stall detection (4) Completion gate (5) Select 1-3 findings (6) Fix each with Read→Classify→Reject-hard→Edit→Verify→Commit (7) Update scoreboard (8) Exit.

Per-iteration ceiling: 3 commits, 200 LOC, 10 files.

## Completion gate

Scoreboard Open findings empty AND:
- `SQLX_OFFLINE=true cargo check --workspace` exits 0
- `SQLX_OFFLINE=true cargo test -p ml-dqn --lib` exits 0

On pass, emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`.

## Safety rails

1. Never push, force-push, reset --hard, or publish.
2. Never touch infra/scripts/.claude/docs (except scoreboard).
3. Never delete code without `rg`-confirmed zero callers.
4. Never alter `*.cu` numerical behavior.
5. Never use `--no-verify` or `git add -A`.
6. Cargo check breaks → revert, defer to human.
7. Emit completion promise only at Step 4 pass or Step 3 stall — never as escape hatch.

## Invocation

```
/ralph-loop \
  --completion-promise 'CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN' \
  --max-iterations 80 \
  "$(cat docs/superpowers/cleanup/dqn-cleanup-prompt.md)"
```

80-iteration ceiling is deliberate: at ~2-3 fixes/iteration, that's a 160-240-finding budget.

## Risks and mitigations

| Risk | Mitigation |
|---|---|
| False-positive fix breaks training silently | Per-iteration `cargo check` gate + optional smoke test before commit |
| Ralph over-deletes "dead" code | `rg` zero-caller check inside fix step; kernel numerics carved out entirely |
| Scoreboard drifts out of sync with reality | Scanner refresh at start of each iteration reconciles stale entries |
| Loop runs forever on a genuinely hard finding | 2-iteration stall detection + `--max-iterations 80` hard ceiling |
| Commits accumulate unreviewed | `cleanup(<cat>):` prefix makes `git log --grep='^cleanup('` trivial; user audits between iterations |

## Success criteria

- All 10 scanner categories return zero open findings.
- `cargo check --workspace` green.
- `cargo test -p ml-dqn --lib` green.
- Every commit on the branch has a single `[CAT-###]` finding reference and passes pre-commit hooks.
- `Needs human review` section lists anything rejected as too-large/too-ambiguous, with precise reasons.
