diff --git a/docs/superpowers/cleanup/dqn-cleanup-prompt.md b/docs/superpowers/cleanup/dqn-cleanup-prompt.md new file mode 100644 index 000000000..ce4e8acd3 --- /dev/null +++ b/docs/superpowers/cleanup/dqn-cleanup-prompt.md @@ -0,0 +1,190 @@ +# DQN Cleanup Loop — Iteration Protocol + +You are in a **Ralph Loop** doing code cleanup on the Foxhunt DQN training stack. Your role: run **one bounded cleanup iteration**, leaving the repo strictly better than you found it, then exit. Ralph will re-feed this exact prompt for the next iteration. + +--- + +## 0. Non-negotiable project rules (read before anything) + +- Working dir: `/home/jgrusewski/Work/foxhunt` +- Build: `SQLX_OFFLINE=true cargo check --workspace` +- Gate test: `SQLX_OFFLINE=true cargo test -p ml-dqn --lib` +- **NEVER** `git push`, `git reset --hard`, `git commit --no-verify`, `git push --force`, `cargo publish` +- **NEVER** touch: `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, or any `docs/` file other than the scoreboard. +- **NEVER** modify `*.cu` files in a way that alters numerical output. Pure constant-extraction only. Anything else goes to "Needs human review". +- **ALWAYS** Read a file before editing it. +- Respect `CLAUDE.md` at repo root and memory files at `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/`. The `feedback_*.md` files are law — especially: + - `feedback_no_feature_flags.md` — `enable_*`/`use_*` booleans must be deleted, features unconditional. + - `feedback_no_hiding.md` — wire up or delete; never suppress. + - `feedback_gpu_cpu_roundtrip.md` — GPU→CPU roundtrips on the training hot path are never acceptable. + - `feedback_no_legacy_aliases.md` — no deprecated wrappers; rename call sites directly. + - `feedback_fix_everything.md` — zero tolerance for failures. + +## 1. Scope (in priority order) + +1. `crates/ml-dqn/` +2. `crates/ml/src/trainers/dqn/` +3. `crates/ml/src/cuda_pipeline/` +4. `crates/ml/src/hyperopt/adapters/dqn.rs` +5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/` +6. All other `crates/ml*` (ml-features, ml-data, ml-ppo, ml-regime, ml-supervised, ml-ensemble, ml-hyperopt, ml-labeling, ml-risk, ml-checkpoint, ml-core, ml-backtesting, ml-observability, ml-explainability, ml-validation, ml-data-validation, ml-asset-selection, ml-paper-trading, ml-security, ml-stress-testing, ml-universe, ml-regime-detection) +7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/` +8. `bin/fxt/` (DQN-related commands only) + +## 2. State file + +Durable scoreboard: `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` + +If it doesn't exist, create it with: + +```markdown +--- +iteration: 0 +last_scan: 1970-01-01T00:00:00Z +last_scoreboard_hash: none +consecutive_stall_count: 0 +status: active +--- + +## Open findings + + +## Resolved this run + + +## Needs human review + + +## BLOCKED + +``` + +### Finding line format + +``` +- [ ] [CAT-###] : `:` — S=<1-5> B=<1-5> E=<1-5> score= +``` + +Where `CAT` ∈ {GPUSYNC, FFLAG, FALLBACK, ACCOUNT, DIMMIX, DEAD, UNWRAP, MAGIC, PDRIFT, TESTROT, VERIFY}. IDs are stable per category — do not renumber. + +## 3. Categories and scan commands + +| ID prefix | Category | Severity | Scan | +|---|---|---|---| +| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 | `rg -n '(dtoh_sync\|sync_reclaim\|\.to_host\(\|htod_sync\|\.synchronize\(\)\|Vec::::from\()' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**'` | +| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 | `rg -n '\b(enable_\|use_\|disable_)\w+\s*:\s*bool' crates/ml crates/ml-dqn services` | +| FALLBACK | Silent fallback paths (`unwrap_or_default`, etc.) | 5 | `rg -n 'unwrap_or_default\|unwrap_or_else\|\.or_else\(' crates/ml crates/ml-dqn services` | +| ACCOUNT | Accounting-only code (computed, logged, not plumbed) | 4 | Heuristic: find `let = …;` where `` appears only inside `info!`/`debug!`/`trace!` spans in the same function | +| DIMMIX | Dimension / layout drift | 4 | `rg -n '\b(62\|81\|42\|27)\b' crates/ml-dqn/src/*.cu crates/ml/src/cuda_pipeline/*.cu` → cross-check against `state_layout` constants | +| UNWRAP | `.unwrap()` on training hot path | 3 | `rg -n '\.unwrap\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs` | +| DEAD | Dead fields / unused functions / unreachable kernels | 3 | `SQLX_OFFLINE=true cargo check --workspace --message-format=json 2>&1 \| rg '"(dead_code\|unused)"'` | +| MAGIC | Magic numbers in kernel launch dims / hyperparams | 2 | `rg -n '\b(threads_per_block\|num_blocks\|shared_mem_bytes\|block_size)\s*[=:]\s*[0-9]' crates/ml crates/ml-dqn` | +| PDRIFT | Parameter name drift (same concept, different names) | 2 | `rg -n '\b(gamma\|discount\|tau\|target_update_rate\|alpha\|cql_alpha\|cql_weight)\b' crates/ml crates/ml-dqn` | +| TESTROT | Test rot (`#[ignore]` without ticket, broken builds) | 2 | `rg -n '#\[ignore' crates/ml crates/ml-dqn` + `SQLX_OFFLINE=true cargo test --workspace --no-run 2>&1 \| rg 'error\[E'` | +| VERIFY | Verification gate failed in Step 4 | 5 | (dynamic — added on verify failure) | + +**Score formula:** `score = severity × blast_radius ÷ effort` +- `blast_radius`: 5 if touches a training kernel; 3 if trainer Rust; 1 if config/test. +- `effort`: 1 if fix <20 LOC; 3 if <100 LOC; 5 if <300 LOC. +- Tie-break: alphabetical by ID. + +## 4. Iteration protocol (do these in order, do not skip) + +### Step 1. Load state +Read `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` with the `Read` tool. If missing, create it per §2. Parse `iteration`, `last_scoreboard_hash`, `consecutive_stall_count`. + +### Step 2. Refresh scans (selectively) +Re-run all 10 scanners from §3 **only if**: `last_scan` is older than 1 hour **or** Open findings has fewer than 5 entries. +- Append NEW findings (next available ID in the category). +- REMOVE Open findings whose `file:line` no longer matches the pattern (probably fixed by hand between iterations). +- Do **not** renumber existing IDs. + +### Step 3. Stall detection +Compute `sha256` over the exact bytes of the `## Open findings` block (header line excluded). +- If equal to `last_scoreboard_hash` in frontmatter → increment `consecutive_stall_count`. +- If count reaches **2**: set `status: blocked`, write under `## BLOCKED`: "No progress in 2 iterations. Last attempted: `[CAT-###]`. Reason: ``", commit scoreboard, **emit `CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN`** and exit. +- Otherwise reset count to 0. + +### Step 4. Completion gate +If `Open findings` is empty: +```bash +SQLX_OFFLINE=true cargo check --workspace +SQLX_OFFLINE=true cargo test -p ml-dqn --lib +``` +- Both green → set `status: complete`, commit scoreboard, **emit `CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN`** and exit. +- Either red → add a `VERIFY-` finding with the exact failing output excerpt, S=5 B=5 E=2 score=12.5, continue to Step 5. + +### Step 5. Select findings +Pick the top 1-3 findings such that **total estimated LOC ≤ 200** (use the `effort` score as a rough guide: E=1→20, E=3→100, E=5→300). Do not cherry-pick low-score items to pad. + +### Step 6. Fix each selected finding +For each, in order: + +**6a. Read the code.** Use the `Read` tool on the target file. Read 30 lines above and below the reported line. Never rely on memory of the file. + +**6b. Classify.** Is this actually a bug / smell, or a false positive? +- False positive → move to `Resolved this run` with commit sha `none`, note `false-positive: `. No code change. Skip to next finding. + +**6c. Reject hard cases.** If the fix requires **any** of the following, move to `Needs human review` with a precise reason and skip: +- >200 LOC delta +- Touches kernel numerics / determinism +- Semantic intent unclear (e.g. `unwrap_or_default` on a `Option` read from user config could be intentional UX) +- Cross-cuts >3 files + +**6d. Apply the smallest correct fix.** +- `Read` → `Edit` only. Use `Write` only when creating a small new file is unavoidable. +- Feature-flag booleans: **delete the field, delete the `if flag {}` branch, keep the true-branch code, run call-site fix-ups**. See `feedback_no_feature_flags.md`. +- Fallback paths: **delete the fallback, propagate the missing-data case to the caller** (usually `return Err`). See `feedback_no_hiding.md`. +- GPU→CPU sync on hot path: move to a diagnostics-only path behind `#[cfg(debug_assertions)]` or a `diagnostics` feature, OR delete if purely informational. See `feedback_gpu_cpu_roundtrip.md`. +- `.unwrap()` on hot path: replace with `expect("")` only if truly infallible, else propagate `Result`. +- Magic numbers in kernel launches: extract to `const` at module top with a comment explaining the value. +- Dead fields: `rg` to confirm zero readers in workspace. If confirmed, delete field + its write sites. + +**6e. Verify.** +```bash +SQLX_OFFLINE=true cargo check -p +``` +If red → `git checkout -- `, move finding to `Needs human review` with the exact error. Continue to next finding. + +**6f. Commit (one finding per commit).** +```bash +git add +git commit -m "cleanup(): — [CAT-###]" +``` +Use a HEREDOC body with 2-4 lines explaining: what was wrong, what changed, why it's safe. +Never `--no-verify`. Never `git add -A`. Never `git add .`. + +Record the commit sha in `## Resolved this run`. + +### Step 7. Update scoreboard frontmatter +- `iteration += 1` +- `last_scan` = current ISO-8601 UTC (only if Step 2 re-scanned) +- `last_scoreboard_hash` = sha256 of the new Open findings block +- `consecutive_stall_count` updated per Step 3 +- Commit scoreboard separately: `git commit -m "cleanup: update scoreboard — iter "` + +### Step 8. Exit normally +Do **not** emit the completion promise unless Step 3 or Step 4 said so. + +Finish with a 3-line summary to stdout: +``` +Iteration : fixed , deferred to Needs-human, false-positive . +Open findings: . Next iteration will target [CAT-###], [CAT-###]. +``` + +Ralph re-feeds this exact prompt. + +--- + +## 5. Hard rules (recap — most important) + +1. **Per-iteration ceiling**: 3 commits, 200 total LOC changed, 10 files touched. Stop early if reached. +2. **Never** touch `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, `docs/` except the scoreboard. +3. **Never** delete code unless `rg` this iteration confirms zero callers workspace-wide. +4. **Never** alter `*.cu` numerical behavior. Constant extraction only. +5. **Never** bypass pre-commit hooks. +6. **One finding per commit.** Specific files in `git add`, never `-A` / `.`. +7. **Cargo check breaks** → revert, move to Needs-human. No exceptions. +8. **Emit the completion promise ONLY** when Step 4 passes both gates, or Step 3 hit a 2-iteration stall. Never as an escape hatch. + +Begin with Step 1. diff --git a/docs/superpowers/cleanup/dqn-cleanup-scoreboard.md b/docs/superpowers/cleanup/dqn-cleanup-scoreboard.md new file mode 100644 index 000000000..ceff024d3 --- /dev/null +++ b/docs/superpowers/cleanup/dqn-cleanup-scoreboard.md @@ -0,0 +1,19 @@ +--- +iteration: 0 +last_scan: 1970-01-01T00:00:00Z +last_scoreboard_hash: none +consecutive_stall_count: 0 +status: active +--- + +## Open findings + + +## Resolved this run + + +## Needs human review + + +## BLOCKED + diff --git a/docs/superpowers/specs/2026-04-20-dqn-cleanup-ralph-loop-design.md b/docs/superpowers/specs/2026-04-20-dqn-cleanup-ralph-loop-design.md new file mode 100644 index 000000000..16325665f --- /dev/null +++ b/docs/superpowers/specs/2026-04-20-dqn-cleanup-ralph-loop-design.md @@ -0,0 +1,129 @@ +# DQN Cleanup Ralph Loop — Design + +**Date:** 2026-04-20 +**Status:** Approved, ready for execution +**Artifact:** `docs/superpowers/cleanup/dqn-cleanup-prompt.md` +**State file:** `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` + +## Problem + +The DQN training stack has accumulated technical debt that a one-shot session cannot surface: 21K Rust LOC + 1050 CUDA LOC in `ml-dqn`; 127K Rust LOC + 30 kernels in `ml`; the largest single file (`gpu_dqn_trainer.rs`) is 13,399 lines. Recent post-mortems (PER 1D-for-2D kernel, OFI all-zero, F8/G5 accounting-only c51_budget) show the class of bug that hides at this scale: plumbing that looks wired but isn't, fallback paths that mask missing data, feature flags that were never flipped back off. + +## Goals + +1. Sweep the DQN-surface codebase for 10 defined smell categories. +2. Fix each as an isolated, reviewable commit. +3. Be resumable: a single invocation can be interrupted and re-run without losing progress. +4. Respect the zero-tolerance rules in `~/.claude/projects/.../memory/feedback_*.md`. +5. Auto-terminate when scoreboard is empty AND verification gates pass. + +## Non-goals + +- Refactors that don't serve a specific finding. +- Kernel numerical changes (kernels are touched only for constant-extraction). +- Anything in `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`. + +## Approach (chosen: C — Scoreboard-driven bounded-batch) + +A durable scoreboard at `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` carries state between iterations. Each Ralph iteration does exactly one bounded pass: refresh scans → select 1-3 top findings → fix each as its own commit → update scoreboard → exit. Ralph re-feeds the prompt until the scoreboard is empty and `cargo check --workspace` + `cargo test -p ml-dqn --lib` are green, at which point the loop emits the completion promise. + +### Why scoreboard-driven + +- **Idempotent prompt.** Ralph's model is re-feeding the same text; state must live on disk, not in the prompt. +- **Resumable.** User can Ctrl-C, inspect commits, resume. +- **Bounded blast radius.** A 200-LOC-per-iteration ceiling prevents runaway refactors. +- **Humans can audit between iterations.** Each iteration leaves clean commits + an updated scoreboard. + +### Rejected alternatives + +- **A. Single-finding-per-iteration.** Too slow (200+ iterations). No meaningful advantage over C's bounded batch. +- **B. Category-per-iteration.** Mass rewrites are too risky on a production trading codebase. One category can easily exceed 200 LOC. + +## Scope + +1. `crates/ml-dqn/` +2. `crates/ml/src/trainers/dqn/` +3. `crates/ml/src/cuda_pipeline/` +4. `crates/ml/src/hyperopt/adapters/dqn.rs` +5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/` +6. All other `crates/ml*` (23 crates) +7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/` +8. `bin/fxt/` (DQN commands only) + +## Categories + +Ten scanner-backed categories with severity weights tied to memory-file priorities: + +| ID | Category | Severity | +|---|---|---| +| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 | +| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 | +| FALLBACK | Silent fallback paths | 5 | +| ACCOUNT | Accounting-only (computed, never used) | 4 | +| DIMMIX | Dimension / layout drift | 4 | +| UNWRAP | `.unwrap()` on training hot path | 3 | +| DEAD | Dead fields / unused functions | 3 | +| MAGIC | Magic numbers in launch dims / hyperparams | 2 | +| PDRIFT | Parameter name drift | 2 | +| TESTROT | Test rot (#[ignore], broken builds) | 2 | + +Plus a dynamic `VERIFY` category for verification-gate failures. + +Score formula: `severity × blast_radius ÷ effort`. Tie-break alphabetical. + +## Scoreboard schema + +YAML frontmatter + four markdown sections (`Open findings`, `Resolved this run`, `Needs human review`, `BLOCKED`). Finding IDs are stable per category (`GPUSYNC-001` … `GPUSYNC-999`). Hashes detect stalls. + +## Iteration protocol + +8 steps in the prompt: (1) Load state (2) Refresh scans selectively (3) Stall detection (4) Completion gate (5) Select 1-3 findings (6) Fix each with Read→Classify→Reject-hard→Edit→Verify→Commit (7) Update scoreboard (8) Exit. + +Per-iteration ceiling: 3 commits, 200 LOC, 10 files. + +## Completion gate + +Scoreboard Open findings empty AND: +- `SQLX_OFFLINE=true cargo check --workspace` exits 0 +- `SQLX_OFFLINE=true cargo test -p ml-dqn --lib` exits 0 + +On pass, emit `CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN`. + +## Safety rails + +1. Never push, force-push, reset --hard, or publish. +2. Never touch infra/scripts/.claude/docs (except scoreboard). +3. Never delete code without `rg`-confirmed zero callers. +4. Never alter `*.cu` numerical behavior. +5. Never use `--no-verify` or `git add -A`. +6. Cargo check breaks → revert, defer to human. +7. Emit completion promise only at Step 4 pass or Step 3 stall — never as escape hatch. + +## Invocation + +``` +/ralph-loop \ + --completion-promise 'CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN' \ + --max-iterations 80 \ + "$(cat docs/superpowers/cleanup/dqn-cleanup-prompt.md)" +``` + +80-iteration ceiling is deliberate: at ~2-3 fixes/iteration, that's a 160-240-finding budget. + +## Risks and mitigations + +| Risk | Mitigation | +|---|---| +| False-positive fix breaks training silently | Per-iteration `cargo check` gate + optional smoke test before commit | +| Ralph over-deletes "dead" code | `rg` zero-caller check inside fix step; kernel numerics carved out entirely | +| Scoreboard drifts out of sync with reality | Scanner refresh at start of each iteration reconciles stale entries | +| Loop runs forever on a genuinely hard finding | 2-iteration stall detection + `--max-iterations 80` hard ceiling | +| Commits accumulate unreviewed | `cleanup():` prefix makes `git log --grep='^cleanup('` trivial; user audits between iterations | + +## Success criteria + +- All 10 scanner categories return zero open findings. +- `cargo check --workspace` green. +- `cargo test -p ml-dqn --lib` green. +- Every commit on the branch has a single `[CAT-###]` finding reference and passes pre-commit hooks. +- `Needs human review` section lists anything rejected as too-large/too-ambiguous, with precise reasons.