docs(cleanup): ralph-loop prompt + scoreboard for DQN code cleanup sweep
Adds scoreboard-driven cleanup infrastructure for bounded-batch Ralph iteration across crates/ml*, services/ml_training_service, backtesting, data_acquisition, and bin/fxt. Ten smell categories (GPUSYNC, FFLAG, FALLBACK, ACCOUNT, DIMMIX, UNWRAP, DEAD, MAGIC, PDRIFT, TESTROT) with per-iter ceiling of 3 commits / 200 LOC / 10 files. Completion gate: empty scoreboard + cargo check --workspace + cargo test -p ml-dqn --lib.
This commit is contained in:
jgrusewski
190
docs/superpowers/cleanup/dqn-cleanup-prompt.md
Normal file
190
docs/superpowers/cleanup/dqn-cleanup-prompt.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# DQN Cleanup Loop — Iteration Protocol
|
||||
|
||||
You are in a **Ralph Loop** doing code cleanup on the Foxhunt DQN training stack. Your role: run **one bounded cleanup iteration**, leaving the repo strictly better than you found it, then exit. Ralph will re-feed this exact prompt for the next iteration.
|
||||
|
||||
---
|
||||
|
||||
## 0. Non-negotiable project rules (read before anything)
|
||||
|
||||
- Working dir: `/home/jgrusewski/Work/foxhunt`
|
||||
- Build: `SQLX_OFFLINE=true cargo check --workspace`
|
||||
- Gate test: `SQLX_OFFLINE=true cargo test -p ml-dqn --lib`
|
||||
- **NEVER** `git push`, `git reset --hard`, `git commit --no-verify`, `git push --force`, `cargo publish`
|
||||
- **NEVER** touch: `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, or any `docs/` file other than the scoreboard.
|
||||
- **NEVER** modify `*.cu` files in a way that alters numerical output. Pure constant-extraction only. Anything else goes to "Needs human review".
|
||||
- **ALWAYS** Read a file before editing it.
|
||||
- Respect `CLAUDE.md` at repo root and memory files at `/home/jgrusewski/.claude/projects/-home-jgrusewski-Work-foxhunt/memory/`. The `feedback_*.md` files are law — especially:
|
||||
- `feedback_no_feature_flags.md` — `enable_*`/`use_*` booleans must be deleted, features unconditional.
|
||||
- `feedback_no_hiding.md` — wire up or delete; never suppress.
|
||||
- `feedback_gpu_cpu_roundtrip.md` — GPU→CPU roundtrips on the training hot path are never acceptable.
|
||||
- `feedback_no_legacy_aliases.md` — no deprecated wrappers; rename call sites directly.
|
||||
- `feedback_fix_everything.md` — zero tolerance for failures.
|
||||
|
||||
## 1. Scope (in priority order)
|
||||
|
||||
1. `crates/ml-dqn/`
|
||||
2. `crates/ml/src/trainers/dqn/`
|
||||
3. `crates/ml/src/cuda_pipeline/`
|
||||
4. `crates/ml/src/hyperopt/adapters/dqn.rs`
|
||||
5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/`
|
||||
6. All other `crates/ml*` (ml-features, ml-data, ml-ppo, ml-regime, ml-supervised, ml-ensemble, ml-hyperopt, ml-labeling, ml-risk, ml-checkpoint, ml-core, ml-backtesting, ml-observability, ml-explainability, ml-validation, ml-data-validation, ml-asset-selection, ml-paper-trading, ml-security, ml-stress-testing, ml-universe, ml-regime-detection)
|
||||
7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/`
|
||||
8. `bin/fxt/` (DQN-related commands only)
|
||||
|
||||
## 2. State file
|
||||
|
||||
Durable scoreboard: `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md`
|
||||
|
||||
If it doesn't exist, create it with:
|
||||
|
||||
```markdown
|
||||
---
|
||||
iteration: 0
|
||||
last_scan: 1970-01-01T00:00:00Z
|
||||
last_scoreboard_hash: none
|
||||
consecutive_stall_count: 0
|
||||
status: active
|
||||
---
|
||||
|
||||
## Open findings
|
||||
<empty>
|
||||
|
||||
## Resolved this run
|
||||
<empty>
|
||||
|
||||
## Needs human review
|
||||
<empty>
|
||||
|
||||
## BLOCKED
|
||||
<empty>
|
||||
```
|
||||
|
||||
### Finding line format
|
||||
|
||||
```
|
||||
- [ ] [CAT-###] <short description>: `<file>:<line>` — S=<1-5> B=<1-5> E=<1-5> score=<float>
|
||||
```
|
||||
|
||||
Where `CAT` ∈ {GPUSYNC, FFLAG, FALLBACK, ACCOUNT, DIMMIX, DEAD, UNWRAP, MAGIC, PDRIFT, TESTROT, VERIFY}. IDs are stable per category — do not renumber.
|
||||
|
||||
## 3. Categories and scan commands
|
||||
|
||||
| ID prefix | Category | Severity | Scan |
|
||||
|---|---|---|---|
|
||||
| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 | `rg -n '(dtoh_sync\|sync_reclaim\|\.to_host\(\|htod_sync\|\.synchronize\(\)\|Vec::<f[0-9]+>::from\()' crates/ml crates/ml-dqn services -g '!**/benches/**' -g '!**/tests/**'` |
|
||||
| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 | `rg -n '\b(enable_\|use_\|disable_)\w+\s*:\s*bool' crates/ml crates/ml-dqn services` |
|
||||
| FALLBACK | Silent fallback paths (`unwrap_or_default`, etc.) | 5 | `rg -n 'unwrap_or_default\|unwrap_or_else\|\.or_else\(' crates/ml crates/ml-dqn services` |
|
||||
| ACCOUNT | Accounting-only code (computed, logged, not plumbed) | 4 | Heuristic: find `let <x> = …;` where `<x>` appears only inside `info!`/`debug!`/`trace!` spans in the same function |
|
||||
| DIMMIX | Dimension / layout drift | 4 | `rg -n '\b(62\|81\|42\|27)\b' crates/ml-dqn/src/*.cu crates/ml/src/cuda_pipeline/*.cu` → cross-check against `state_layout` constants |
|
||||
| UNWRAP | `.unwrap()` on training hot path | 3 | `rg -n '\.unwrap\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,gpu_replay_buffer}.rs` |
|
||||
| DEAD | Dead fields / unused functions / unreachable kernels | 3 | `SQLX_OFFLINE=true cargo check --workspace --message-format=json 2>&1 \| rg '"(dead_code\|unused)"'` |
|
||||
| MAGIC | Magic numbers in kernel launch dims / hyperparams | 2 | `rg -n '\b(threads_per_block\|num_blocks\|shared_mem_bytes\|block_size)\s*[=:]\s*[0-9]' crates/ml crates/ml-dqn` |
|
||||
| PDRIFT | Parameter name drift (same concept, different names) | 2 | `rg -n '\b(gamma\|discount\|tau\|target_update_rate\|alpha\|cql_alpha\|cql_weight)\b' crates/ml crates/ml-dqn` |
|
||||
| TESTROT | Test rot (`#[ignore]` without ticket, broken builds) | 2 | `rg -n '#\[ignore' crates/ml crates/ml-dqn` + `SQLX_OFFLINE=true cargo test --workspace --no-run 2>&1 \| rg 'error\[E'` |
|
||||
| VERIFY | Verification gate failed in Step 4 | 5 | (dynamic — added on verify failure) |
|
||||
|
||||
**Score formula:** `score = severity × blast_radius ÷ effort`
|
||||
- `blast_radius`: 5 if touches a training kernel; 3 if trainer Rust; 1 if config/test.
|
||||
- `effort`: 1 if fix <20 LOC; 3 if <100 LOC; 5 if <300 LOC.
|
||||
- Tie-break: alphabetical by ID.
|
||||
|
||||
## 4. Iteration protocol (do these in order, do not skip)
|
||||
|
||||
### Step 1. Load state
|
||||
Read `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` with the `Read` tool. If missing, create it per §2. Parse `iteration`, `last_scoreboard_hash`, `consecutive_stall_count`.
|
||||
|
||||
### Step 2. Refresh scans (selectively)
|
||||
Re-run all 10 scanners from §3 **only if**: `last_scan` is older than 1 hour **or** Open findings has fewer than 5 entries.
|
||||
- Append NEW findings (next available ID in the category).
|
||||
- REMOVE Open findings whose `file:line` no longer matches the pattern (probably fixed by hand between iterations).
|
||||
- Do **not** renumber existing IDs.
|
||||
|
||||
### Step 3. Stall detection
|
||||
Compute `sha256` over the exact bytes of the `## Open findings` block (header line excluded).
|
||||
- If equal to `last_scoreboard_hash` in frontmatter → increment `consecutive_stall_count`.
|
||||
- If count reaches **2**: set `status: blocked`, write under `## BLOCKED`: "No progress in 2 iterations. Last attempted: `[CAT-###]`. Reason: `<error text or ambiguity>`", commit scoreboard, **emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`** and exit.
|
||||
- Otherwise reset count to 0.
|
||||
|
||||
### Step 4. Completion gate
|
||||
If `Open findings` is empty:
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo check --workspace
|
||||
SQLX_OFFLINE=true cargo test -p ml-dqn --lib
|
||||
```
|
||||
- Both green → set `status: complete`, commit scoreboard, **emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`** and exit.
|
||||
- Either red → add a `VERIFY-<YYYYMMDDHHMMSS>` finding with the exact failing output excerpt, S=5 B=5 E=2 score=12.5, continue to Step 5.
|
||||
|
||||
### Step 5. Select findings
|
||||
Pick the top 1-3 findings such that **total estimated LOC ≤ 200** (use the `effort` score as a rough guide: E=1→20, E=3→100, E=5→300). Do not cherry-pick low-score items to pad.
|
||||
|
||||
### Step 6. Fix each selected finding
|
||||
For each, in order:
|
||||
|
||||
**6a. Read the code.** Use the `Read` tool on the target file. Read 30 lines above and below the reported line. Never rely on memory of the file.
|
||||
|
||||
**6b. Classify.** Is this actually a bug / smell, or a false positive?
|
||||
- False positive → move to `Resolved this run` with commit sha `none`, note `false-positive: <reason>`. No code change. Skip to next finding.
|
||||
|
||||
**6c. Reject hard cases.** If the fix requires **any** of the following, move to `Needs human review` with a precise reason and skip:
|
||||
- >200 LOC delta
|
||||
- Touches kernel numerics / determinism
|
||||
- Semantic intent unclear (e.g. `unwrap_or_default` on a `Option<String>` read from user config could be intentional UX)
|
||||
- Cross-cuts >3 files
|
||||
|
||||
**6d. Apply the smallest correct fix.**
|
||||
- `Read` → `Edit` only. Use `Write` only when creating a small new file is unavoidable.
|
||||
- Feature-flag booleans: **delete the field, delete the `if flag {}` branch, keep the true-branch code, run call-site fix-ups**. See `feedback_no_feature_flags.md`.
|
||||
- Fallback paths: **delete the fallback, propagate the missing-data case to the caller** (usually `return Err`). See `feedback_no_hiding.md`.
|
||||
- GPU→CPU sync on hot path: move to a diagnostics-only path behind `#[cfg(debug_assertions)]` or a `diagnostics` feature, OR delete if purely informational. See `feedback_gpu_cpu_roundtrip.md`.
|
||||
- `.unwrap()` on hot path: replace with `expect("<invariant that must hold>")` only if truly infallible, else propagate `Result`.
|
||||
- Magic numbers in kernel launches: extract to `const` at module top with a comment explaining the value.
|
||||
- Dead fields: `rg` to confirm zero readers in workspace. If confirmed, delete field + its write sites.
|
||||
|
||||
**6e. Verify.**
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo check -p <owning-crate>
|
||||
```
|
||||
If red → `git checkout -- <touched files>`, move finding to `Needs human review` with the exact error. Continue to next finding.
|
||||
|
||||
**6f. Commit (one finding per commit).**
|
||||
```bash
|
||||
git add <specific file paths — no -A, no .>
|
||||
git commit -m "cleanup(<cat-lowercase>): <one-line fix> — [CAT-###]"
|
||||
```
|
||||
Use a HEREDOC body with 2-4 lines explaining: what was wrong, what changed, why it's safe.
|
||||
Never `--no-verify`. Never `git add -A`. Never `git add .`.
|
||||
|
||||
Record the commit sha in `## Resolved this run`.
|
||||
|
||||
### Step 7. Update scoreboard frontmatter
|
||||
- `iteration += 1`
|
||||
- `last_scan` = current ISO-8601 UTC (only if Step 2 re-scanned)
|
||||
- `last_scoreboard_hash` = sha256 of the new Open findings block
|
||||
- `consecutive_stall_count` updated per Step 3
|
||||
- Commit scoreboard separately: `git commit -m "cleanup: update scoreboard — iter <N>"`
|
||||
|
||||
### Step 8. Exit normally
|
||||
Do **not** emit the completion promise unless Step 3 or Step 4 said so.
|
||||
|
||||
Finish with a 3-line summary to stdout:
|
||||
```
|
||||
Iteration <N>: fixed <k>, deferred <j> to Needs-human, false-positive <f>.
|
||||
Open findings: <count>. Next iteration will target [CAT-###], [CAT-###].
|
||||
```
|
||||
|
||||
Ralph re-feeds this exact prompt.
|
||||
|
||||
---
|
||||
|
||||
## 5. Hard rules (recap — most important)
|
||||
|
||||
1. **Per-iteration ceiling**: 3 commits, 200 total LOC changed, 10 files touched. Stop early if reached.
|
||||
2. **Never** touch `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`, `docs/` except the scoreboard.
|
||||
3. **Never** delete code unless `rg` this iteration confirms zero callers workspace-wide.
|
||||
4. **Never** alter `*.cu` numerical behavior. Constant extraction only.
|
||||
5. **Never** bypass pre-commit hooks.
|
||||
6. **One finding per commit.** Specific files in `git add`, never `-A` / `.`.
|
||||
7. **Cargo check breaks** → revert, move to Needs-human. No exceptions.
|
||||
8. **Emit the completion promise ONLY** when Step 4 passes both gates, or Step 3 hit a 2-iteration stall. Never as an escape hatch.
|
||||
|
||||
Begin with Step 1.
|
||||
19
docs/superpowers/cleanup/dqn-cleanup-scoreboard.md
Normal file
19
docs/superpowers/cleanup/dqn-cleanup-scoreboard.md
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
iteration: 0
|
||||
last_scan: 1970-01-01T00:00:00Z
|
||||
last_scoreboard_hash: none
|
||||
consecutive_stall_count: 0
|
||||
status: active
|
||||
---
|
||||
|
||||
## Open findings
|
||||
<empty — populated on first iteration>
|
||||
|
||||
## Resolved this run
|
||||
<empty>
|
||||
|
||||
## Needs human review
|
||||
<empty>
|
||||
|
||||
## BLOCKED
|
||||
<empty unless stall detected>
|
||||
@@ -0,0 +1,129 @@
|
||||
# DQN Cleanup Ralph Loop — Design
|
||||
|
||||
**Date:** 2026-04-20
|
||||
**Status:** Approved, ready for execution
|
||||
**Artifact:** `docs/superpowers/cleanup/dqn-cleanup-prompt.md`
|
||||
**State file:** `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md`
|
||||
|
||||
## Problem
|
||||
|
||||
The DQN training stack has accumulated technical debt that a one-shot session cannot surface: 21K Rust LOC + 1050 CUDA LOC in `ml-dqn`; 127K Rust LOC + 30 kernels in `ml`; the largest single file (`gpu_dqn_trainer.rs`) is 13,399 lines. Recent post-mortems (PER 1D-for-2D kernel, OFI all-zero, F8/G5 accounting-only c51_budget) show the class of bug that hides at this scale: plumbing that looks wired but isn't, fallback paths that mask missing data, feature flags that were never flipped back off.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Sweep the DQN-surface codebase for 10 defined smell categories.
|
||||
2. Fix each as an isolated, reviewable commit.
|
||||
3. Be resumable: a single invocation can be interrupted and re-run without losing progress.
|
||||
4. Respect the zero-tolerance rules in `~/.claude/projects/.../memory/feedback_*.md`.
|
||||
5. Auto-terminate when scoreboard is empty AND verification gates pass.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Refactors that don't serve a specific finding.
|
||||
- Kernel numerical changes (kernels are touched only for constant-extraction).
|
||||
- Anything in `infra/`, `scripts/argo-*`, `.gitlab-ci.yml`, `.claude/`.
|
||||
|
||||
## Approach (chosen: C — Scoreboard-driven bounded-batch)
|
||||
|
||||
A durable scoreboard at `docs/superpowers/cleanup/dqn-cleanup-scoreboard.md` carries state between iterations. Each Ralph iteration does exactly one bounded pass: refresh scans → select 1-3 top findings → fix each as its own commit → update scoreboard → exit. Ralph re-feeds the prompt until the scoreboard is empty and `cargo check --workspace` + `cargo test -p ml-dqn --lib` are green, at which point the loop emits the completion promise.
|
||||
|
||||
### Why scoreboard-driven
|
||||
|
||||
- **Idempotent prompt.** Ralph's model is re-feeding the same text; state must live on disk, not in the prompt.
|
||||
- **Resumable.** User can Ctrl-C, inspect commits, resume.
|
||||
- **Bounded blast radius.** A 200-LOC-per-iteration ceiling prevents runaway refactors.
|
||||
- **Humans can audit between iterations.** Each iteration leaves clean commits + an updated scoreboard.
|
||||
|
||||
### Rejected alternatives
|
||||
|
||||
- **A. Single-finding-per-iteration.** Too slow (200+ iterations). No meaningful advantage over C's bounded batch.
|
||||
- **B. Category-per-iteration.** Mass rewrites are too risky on a production trading codebase. One category can easily exceed 200 LOC.
|
||||
|
||||
## Scope
|
||||
|
||||
1. `crates/ml-dqn/`
|
||||
2. `crates/ml/src/trainers/dqn/`
|
||||
3. `crates/ml/src/cuda_pipeline/`
|
||||
4. `crates/ml/src/hyperopt/adapters/dqn.rs`
|
||||
5. `crates/ml/src/data_loaders/`, `crates/ml/src/data_pipeline/`
|
||||
6. All other `crates/ml*` (23 crates)
|
||||
7. `services/ml_training_service/`, `services/data_acquisition_service/`, `services/backtesting_service/`
|
||||
8. `bin/fxt/` (DQN commands only)
|
||||
|
||||
## Categories
|
||||
|
||||
Ten scanner-backed categories with severity weights tied to memory-file priorities:
|
||||
|
||||
| ID | Category | Severity |
|
||||
|---|---|---|
|
||||
| GPUSYNC | Hidden GPU→CPU sync on hot path | 5 |
|
||||
| FFLAG | `enable_*` / `use_*` / `disable_*` booleans | 5 |
|
||||
| FALLBACK | Silent fallback paths | 5 |
|
||||
| ACCOUNT | Accounting-only (computed, never used) | 4 |
|
||||
| DIMMIX | Dimension / layout drift | 4 |
|
||||
| UNWRAP | `.unwrap()` on training hot path | 3 |
|
||||
| DEAD | Dead fields / unused functions | 3 |
|
||||
| MAGIC | Magic numbers in launch dims / hyperparams | 2 |
|
||||
| PDRIFT | Parameter name drift | 2 |
|
||||
| TESTROT | Test rot (#[ignore], broken builds) | 2 |
|
||||
|
||||
Plus a dynamic `VERIFY` category for verification-gate failures.
|
||||
|
||||
Score formula: `severity × blast_radius ÷ effort`. Tie-break alphabetical.
|
||||
|
||||
## Scoreboard schema
|
||||
|
||||
YAML frontmatter + four markdown sections (`Open findings`, `Resolved this run`, `Needs human review`, `BLOCKED`). Finding IDs are stable per category (`GPUSYNC-001` … `GPUSYNC-999`). Hashes detect stalls.
|
||||
|
||||
## Iteration protocol
|
||||
|
||||
8 steps in the prompt: (1) Load state (2) Refresh scans selectively (3) Stall detection (4) Completion gate (5) Select 1-3 findings (6) Fix each with Read→Classify→Reject-hard→Edit→Verify→Commit (7) Update scoreboard (8) Exit.
|
||||
|
||||
Per-iteration ceiling: 3 commits, 200 LOC, 10 files.
|
||||
|
||||
## Completion gate
|
||||
|
||||
Scoreboard Open findings empty AND:
|
||||
- `SQLX_OFFLINE=true cargo check --workspace` exits 0
|
||||
- `SQLX_OFFLINE=true cargo test -p ml-dqn --lib` exits 0
|
||||
|
||||
On pass, emit `<promise>CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN</promise>`.
|
||||
|
||||
## Safety rails
|
||||
|
||||
1. Never push, force-push, reset --hard, or publish.
|
||||
2. Never touch infra/scripts/.claude/docs (except scoreboard).
|
||||
3. Never delete code without `rg`-confirmed zero callers.
|
||||
4. Never alter `*.cu` numerical behavior.
|
||||
5. Never use `--no-verify` or `git add -A`.
|
||||
6. Cargo check breaks → revert, defer to human.
|
||||
7. Emit completion promise only at Step 4 pass or Step 3 stall — never as escape hatch.
|
||||
|
||||
## Invocation
|
||||
|
||||
```
|
||||
/ralph-loop \
|
||||
--completion-promise 'CLEANUP SCOREBOARD EMPTY AND VERIFICATION GATES GREEN' \
|
||||
--max-iterations 80 \
|
||||
"$(cat docs/superpowers/cleanup/dqn-cleanup-prompt.md)"
|
||||
```
|
||||
|
||||
80-iteration ceiling is deliberate: at ~2-3 fixes/iteration, that's a 160-240-finding budget.
|
||||
|
||||
## Risks and mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| False-positive fix breaks training silently | Per-iteration `cargo check` gate + optional smoke test before commit |
|
||||
| Ralph over-deletes "dead" code | `rg` zero-caller check inside fix step; kernel numerics carved out entirely |
|
||||
| Scoreboard drifts out of sync with reality | Scanner refresh at start of each iteration reconciles stale entries |
|
||||
| Loop runs forever on a genuinely hard finding | 2-iteration stall detection + `--max-iterations 80` hard ceiling |
|
||||
| Commits accumulate unreviewed | `cleanup(<cat>):` prefix makes `git log --grep='^cleanup('` trivial; user audits between iterations |
|
||||
|
||||
## Success criteria
|
||||
|
||||
- All 10 scanner categories return zero open findings.
|
||||
- `cargo check --workspace` green.
|
||||
- `cargo test -p ml-dqn --lib` green.
|
||||
- Every commit on the branch has a single `[CAT-###]` finding reference and passes pre-commit hooks.
|
||||
- `Needs human review` section lists anything rejected as too-large/too-ambiguous, with precise reasons.
|
||||
Reference in New Issue
Block a user