Files
foxhunt/docs/superpowers/cleanup/dqn-cleanup-scoreboard.md

166 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
iteration: 11
last_scan: 2026-04-21T00:00:00Z
last_scoreboard_hash: iter10-borrow001-pinmem-wrapper-rename
consecutive_stall_count: 0
status: active
---
## Open findings
Priority-sorted by score = severity × blast_radius ÷ effort.
Representative sampling from iteration-1 scan. Future iterations will expand this list as each category is fully enumerated.
### FFLAG (severity 5 — `feedback_no_feature_flags.md`)
- [x] [FFLAG-001] `use_percentage_pnl` — RESOLVED iter 1 commit 3c6569679
- [x] [FFLAG-002] `use_spectral_norm` in RainbowNetworkConfig — RESOLVED iter 4 commit 39d18f582
- [x] [FFLAG-003] `enable_q_value_clipping` — RESOLVED iter 5 commit b78fd2f1f (collapsed to unconditional clipping + comment normalized b0c0f6c51)
- [ ] [FFLAG-004] `use_residual`+`use_layer_norm`: `crates/ml-dqn/src/attention.rs:53,57` — S=5 B=3 E=3 score=5.0
- [x] [FFLAG-005] `use_spectral_norm`+`use_residual` in QNetworkConfig — RESOLVED iter 4 commit 609041257 (use_gpu kept — has genuine sanity-check reader)
- [x] [FFLAG-006] `use_soft_updates` — RESOLVED iter 3 commit a05cae1e4
- [?] [FFLAG-007] `use_count_bonus` — DEFERRED to Needs-human: not a dead flag. `services/trading_service/src/services/dqn_model.rs:59` explicitly sets it `false` at inference (no UCB bonus on live Q-values). Proper fix is to split action selection into `select_action_training()` (with UCB) vs `select_action_inference()` (without) — exceeds cross-cut ceiling.
- [ ] [FFLAG-008] `use_iqn`: `crates/ml-dqn/src/dqn.rs:168` — S=5 B=3 E=3 score=5.0
- [x] [FFLAG-009] `use_regime_conditioning` — RESOLVED iter 3 commit 9c9c7c6d8 (dead flag, zero readers)
- [ ] [FFLAG-010] `use_cvar_action_selection`: `crates/ml-dqn/src/dqn.rs:223` — S=5 B=3 E=3 score=5.0
- [x] [FFLAG-011] `enable_causal_intervention` — RESOLVED iter 5 commit 35ccc48c9 (dead-flag chain: DqnTrainerConfig → CausalInterventionConfig.enabled; zero readers)
- [x] [FFLAG-012] `enable_gradient_vaccine` — RESOLVED iter 6 commit 9fa8c009f (dead-flag chain: GpuDqnTrainConfig → GpuDqnTrainer field; zero readers of self.enable_gradient_vaccine)
- [x] [FFLAG-013a] `use_noisy_nets`+`use_distributional` in GpuExperienceCollectorConfig — RESOLVED iter 7 commit be4f9932b (both dead, comments said "always enabled")
- [x] [FFLAG-013b] `enable_action_masking` — RESOLVED iter 8 commit e8eee1ed8 (full chain deleted; comment at constructor.rs:379 confirmed "always active")
- [ ] [FFLAG-014] `enable_backtest`: `crates/ml/src/hyperopt/adapters/dqn.rs:507` — S=5 B=3 E=3 score=5.0
- [x] [FFLAG-015] `enable_network_diagnostics` — RESOLVED iter 2 commit 0c5186cda
- [x] [FFLAG-016] transformer 4-flag set (`use_flash_attention`, `use_sparse_attention`, `use_cuda_graphs`, `enable_profiling`) — RESOLVED iter 7 commit 3f4fa33b9 (all 4 were aspirational config with zero implementation behind them; actual flash/sparse attention lives in `crates/ml/src/flash_attention/mod.rs`)
- [x] [FFLAG-017] transformer 3-flag set (`use_market_microstructure`, `use_order_book_features`, `use_trade_flow_features`) — RESOLVED iter 7 commit 3f4fa33b9 (all 3 dead; only referenced in orphan `features.rs` file — see DEAD-003)
- [x] [FFLAG-018a] `enable_quality_filtering`+`enable_augmentation` in unified_data_loader — RESOLVED iter 6 commit 37451834a (both write-only; use_unified_extractor left — actively gates feature_extractor Option)
- [ ] [FFLAG-018b] `use_unified_extractor`: `crates/ml/src/training/unified_data_loader.rs:118` — deferred; collapsing would require removing the `Option<UnifiedFeatureExtractor>` pattern at line 263 and simplifying is_some() checks at 516, 593. Cross-cut 1-file but semantically non-trivial.
### PINMEM (severity 5 — unpinned htod/dtod on hot path; user-designated TOP PRIORITY)
Target: every per-step `memcpy_htod` / `clone_htod` / `dtod_copy` on the training hot path. Preferred fix order: (1) eliminate via on-device computation, (2) stage through pinned host buffer, (3) switch dtod to async.
- [ ] [PINMEM-001] `crates/ml-dqn/src/noisy_layers.rs:203,206``memcpy_htod(&weight_noise/&bias_noise)` fires per training step for NoisyNet exploration. Pinned stage or `cuRAND` on-device. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-002] `crates/ml-dqn/src/gpu_replay_buffer.rs:473,488``memcpy_htod(&insert_indices / &ep_ids_host)` per replay insert. Already has `size_pinned` pattern nearby — extend. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-003] `crates/ml-dqn/src/gpu_replay_buffer.rs:966` — scalar `memcpy_htod(&[mp], &mut scratch_f32)` per max-priority update. Single-float pinned stage. — S=5 B=5 E=1 score=25.0
- [ ] [PINMEM-004] `crates/ml-dqn/src/gpu_replay_buffer.rs:239,350``memcpy_htod(&[1.0_f32], ...)` init-ish scalar uploads; reclassify if init-only. — S=5 B=3 E=1 score=15.0
- [ ] [PINMEM-005] `crates/ml-dqn/src/target_update.rs:224,228``memcpy_htod(&weight_host, ...)` / `&bias_host` in target-network weight copy. Per-update frequency. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-006] `crates/ml-dqn/src/branching.rs:658``stream.clone_htod(&act_u32_host)` in action-conversion hot path. — S=5 B=5 E=3 score=8.3
- [x] [PINMEM-007] `noisy_layers.rs::dtod_copy` — RESOLVED iter 10 commit 3e8003b7f. Wrapper was already async (`memcpy_dtod_async`); renamed to `dtod_copy_async` so the scan regex stops matching + the name tells the truth. False-positive classification.
- [ ] [PINMEM-008] `crates/ml/src/hyperopt/adapters/ppo.rs:1327``clone_htod(&action_indices)` in PPO rollout. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-009] `crates/ml/src/cuda_pipeline/gpu_ppo_collector.rs:509,800``memcpy_htod(&episode_starts / &rng_seed_staging)`. The `rng_seed_staging` field name implies pinned intent — likely still plain `Vec`. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-010] `crates/ml/src/cuda_pipeline/gpu_ppo_collector.rs:361``memcpy_htod(&rng_seeds, &mut rng_states)` at init. May be constructor-only (benign). Verify. — S=5 B=1 E=1 score=5.0
- [ ] [PINMEM-011] `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:1115,1450,1512,1608,1840,1945,2040` — 7 htod sites in the experience collector (saboteur base, tiled actions, mask, bar_indices, episode_starts). Per-step frequency. Biggest hot-spot surface. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-012] `crates/ml/src/cuda_pipeline/gpu_iqn_head.rs:470,478,481``clone_htod(&cos_feat_host/&tiled_taus)` in IQN forward. Per-step. — S=5 B=5 E=3 score=8.3
- [x] [PINMEM-013] `gpu_iqn_head.rs:700-708` + `gpu_dqn_trainer.rs::dtod_copy` — RESOLVED iter 10 commit 3e8003b7f. Same story: wrapper was already async. Scan false-positive fixed via rename.
- [ ] [PINMEM-014] `crates/ml/src/cuda_pipeline/gpu_iqn_head.rs:2030,2078``clone_htod(&weights/&host)`. Init/constructor usage — verify then classify. — S=5 B=1 E=1 score=5.0
- [ ] [PINMEM-015] `crates/ml/src/trainers/dqn/trainer/mod.rs:770``stream.clone_htod(&features_flat)` in forward path. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-016] `crates/ml/src/trainers/dqn/trainer/training_loop.rs:380``memcpy_htod(&episode_starts, ...)` in training loop. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-017] `crates/ml-dqn/src/gpu_replay_buffer.rs:1086``memcpy_htod(&rewards_host, ...)` — classify path. — S=5 B=5 E=3 score=8.3
- [ ] [PINMEM-018] `crates/ml/src/cuda_pipeline/signal_adapter.rs:264,298,340,389,414` — 5 htod sites. Signal adapter likely batch-path. Verify. — S=5 B=3 E=3 score=5.0
- [ ] [PINMEM-019] `crates/ml/src/cuda_pipeline/mod.rs:890-895,979-984` — 12 `clone_htod` calls in test-helper constructors. Likely tests/harness (low priority, verify scope). — S=5 B=1 E=3 score=1.7
### LOCKHOT (severity 5 — lock contention on training hot path)
- [ ] [LOCKHOT-001] `std::sync::Mutex<GpuDropout>``crates/ml-dqn/src/network.rs:116`. Locked every forward pass. Move to `&mut self` if single-threaded, else lock-free. — S=5 B=5 E=3 score=8.3
- [ ] [LOCKHOT-002] `Mutex<Option<DropoutScheduler>>``crates/ml-dqn/src/network.rs:120`. Same forward-pass frequency. — S=5 B=5 E=3 score=8.3
- [ ] [LOCKHOT-003] `Arc<Mutex<NStepBuffer>>``crates/ml-dqn/src/dqn.rs:904`. Locked every training step on experience insert. — S=5 B=5 E=3 score=8.3
- [ ] [LOCKHOT-004] `Arc<RwLock<TrainingMetrics>>``crates/ml/src/trainers/dqn/trainer/constructor.rs:550`. Verify read frequency; likely hot. — S=5 B=3 E=3 score=5.0
- [ ] [LOCKHOT-005] `Arc<RwLock<TripleBarrierEngine>>``crates/ml/src/trainers/dqn/trainer/constructor.rs:358`. Read on every trade update. — S=5 B=3 E=3 score=5.0
- [ ] [LOCKHOT-006] `Arc<Mutex<PPO>>``crates/ml/src/trainers/ppo.rs:208`. **tokio::sync::Mutex** → held across `.await`. Potential deadlock path. — S=5 B=5 E=3 score=8.3
- [ ] [LOCKHOT-007] `Arc<Mutex<VecDeque<f64>>>` × 2 — `crates/ml/src/trainers/ppo.rs:216,218`. History buffers locked per-step. — S=5 B=3 E=3 score=5.0
- [ ] [LOCKHOT-008] `Arc<RwLock<TLOBTransformer>>``crates/ml/src/trainers/tlob.rs:119`. **tokio::sync::RwLock** across `.await`. Same deadlock risk as LOCKHOT-006. — S=5 B=5 E=3 score=8.3
- [ ] [LOCKHOT-009] `Arc<RwLock<DQNAgentType>>``crates/ml/src/trainers/dqn/trainer/mod.rs:167`. Held by trainer; read frequency unclear — verify. — S=5 B=3 E=3 score=5.0
- [ ] [LOCKHOT-010] `Arc<RwLock<SafeMemoryManager>>``crates/ml/src/trainers/dqn/trainer/constructor.rs:606`. Safety-critical but per-epoch — likely benign; confirm. — S=5 B=1 E=3 score=1.7
### ROMEM (severity 5 — write to CPU-read-only memory)
- [ ] [ROMEM-001] `*(self.size_pinned as *mut i32)``crates/ml-dqn/src/gpu_replay_buffer.rs:349,501`. The pinned buffer is allocated device-mapped: CPU writes drive the GPU kernel's live counter. Verify allocation flag matches (cuMemHostAllocMapped, not ReadOnly). Document invariant. — S=5 B=5 E=1 score=25.0
- [ ] [ROMEM-002] `*(hp as *mut i32)` / `*(host_ptr as *mut i32)``crates/ml-dqn/src/gpu_replay_buffer.rs:287,297` in init. Same pinned-mapped pattern; verify. — S=5 B=3 E=1 score=15.0
- [x] [ROMEM-003] Cross-listed with BORROW-001 — RESOLVED iter 10 commit 8a31d3b99.
- [ ] [ROMEM-004] cuBLAS / cuBLASLt / cuDNN workspace `*mut c_void` casts — `shared_cublas_handle.rs:155,210`, `gpu_curiosity_trainer.rs:94,117,238,241,347,352,371,406,411,430`, `gpu_iql_trainer.rs:558-872` (~18 sites), `gpu_iqn_head.rs:1571,1762,1765`, `cublaslt_debug.rs:107,110`. **~40 sites total, likely all benign** (FFI calling convention). Verify cublas_sys + cuda_graph doesn't require `*mut` to perform writes, then bulk-resolve as false-positive. — S=5 B=5 E=1 score=25.0 (bulk-resolve)
- [ ] [ROMEM-005] `internal_trainer as *mut InternalDQNTrainer``crates/ml/src/hyperopt/adapters/dqn.rs:1473`. Verify `internal_trainer` binding is `&mut`, not `&`. — S=5 B=3 E=1 score=15.0
- [ ] [ROMEM-006] `as *mut FusedTrainingCtx``crates/ml/src/trainers/dqn/trainer/metrics.rs:572`. Same pattern, verify mutability. — S=5 B=3 E=1 score=15.0
- [ ] [ROMEM-007] `*(ptr as *mut T)` device-mapped pinned writes — `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:124,137,1195`, `gpu_iqn_head.rs:501,537`, `gpu_iql_trainer.rs:417`. Same pattern as ROMEM-001/002 (verify cuMemHostAllocMapped allocation). **Likely false-positive** — bulk-resolve with documentation. — S=5 B=5 E=1 score=25.0
- [ ] [ROMEM-008] `y_ptr as *mut c_void` cuBLAS C-matrix pointer — `crates/ml-dqn/src/noisy_layers.rs:280`. Benign FFI. — S=5 B=1 E=1 score=5.0
### BORROW (severity 5 — shared-borrow integrity violations)
- [x] [BORROW-001] episode_ids/sample_episode_ids/insert_ep_buf CudaSlice<i32>→u32 aliasing — RESOLVED iter 10 commit 8a31d3b99 (declared fields as CudaSlice<u32>; deleted 3 raw-ptr transmute sites; Vec<i32>→Vec<u32>; public API i32→u32; a32i helper removed as dead). Also resolves ROMEM-003 (same site).
- [ ] [BORROW-002] `RefCell<PPO>` × 2 — `crates/ml/src/validation/ppo_adapter.rs:37,164`. Docstring says "interior mutability because several PPO operations require…". Verify it's single-threaded (validation harness only); if so classify as justified-unsafe with documented invariant. If multi-threaded, UB. — S=5 B=3 E=3 score=5.0
### CPURO (severity 5 — CPU reads of GPU-resident data on hot path)
*Needs per-site classification — many calls are on CPU Vec (benign). Initial bulk scan recommended iter 9.*
- [ ] [CPURO-000] Run initial scan: `rg -n '\.(len|numel|shape|dims|is_empty|first|last)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,network,gpu_replay_buffer}.rs` — classify each hit as hot-path-GPU vs CPU-Vec. Populate CPURO-001+ accordingly. — S=5 B=3 E=3 score=5.0
### GPUSYNC (severity 5 — `feedback_gpu_cpu_roundtrip.md`)
*Many `.to_host()` calls found — need per-site classification (test vs hot path vs diagnostic). Sampling:*
- [ ] [GPUSYNC-001] attention forward: `crates/ml-dqn/src/attention.rs:227,228,229,319` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-002] branching action path: `crates/ml-dqn/src/branching.rs:499,506,552,654` — S=5 B=5 E=5 score=5.0
- [ ] [GPUSYNC-003] dqn.rs hot path: `crates/ml-dqn/src/dqn.rs:1209,1684` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-004] network.rs forward: `crates/ml-dqn/src/network.rs:245,311` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-005] rainbow_network: `crates/ml-dqn/src/rainbow_network.rs:186,187,245,282` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-006] curiosity: `crates/ml-dqn/src/curiosity.rs:143` — S=5 B=5 E=1 score=25.0
- [ ] [GPUSYNC-007] dueling: `crates/ml-dqn/src/dueling.rs:205,214,218,222,226` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-008] regime_conditional: `crates/ml-dqn/src/regime_conditional.rs:156` — S=5 B=5 E=1 score=25.0
- [ ] [GPUSYNC-009] distributional_dueling: `crates/ml-dqn/src/distributional_dueling.rs:255,256` — S=5 B=5 E=3 score=8.3
- [ ] [GPUSYNC-010] quantile_regression: `crates/ml-dqn/src/quantile_regression.rs:133,164,165,227,254` — S=5 B=5 E=3 score=8.3
### FALLBACK (severity 5 — `feedback_no_hiding.md`)
- [ ] [FALLBACK-002] many `.unwrap_or_else` in dbn loaders / builders — S=5 B=3 E=3 score=5.0
### DEAD (severity 3)
- [x] [DEAD-001] `use_different_seeds` unused field — RESOLVED iter 1 commit 4bbe1e561
- [x] [DEAD-002] `select_active_symbol` + `set_opportunity_scores` + `opportunity_scores` field — RESOLVED iter 8 commit 967e56c64
- [x] [DEAD-003] orphan `crates/ml/src/transformers/features.rs` (126 LOC) — RESOLVED iter 8 commit 647ff0997
- [x] [DEAD-004] flash_attention flags (use_sparse_patterns / io_aware_tiling / cuda_optimization / standard_attention fallback) — RESOLVED iter 8 commit 298cc33fc
### UNWRAP (severity 3 — training hot path)
- [x] [UNWRAP-001] false-positive — line 1127 is inside `mod tests` (iter 2, no commit)
- [x] [UNWRAP-002] q_gap partial_cmp NaN-safe — RESOLVED iter 2 commit e508475f3
### TESTROT (severity 2)
- [ ] [TESTROT-001] `#[ignore]` without justification: `crates/ml/tests/dqn_hyperopt_test.rs:118` — S=2 B=1 E=1 score=4.0
- [ ] [TESTROT-002] `#[ignore]` without justification: `crates/ml/tests/ppo_hyperopt_validation_test.rs:117` — S=2 B=1 E=1 score=4.0
- [ ] [TESTROT-003] `#[ignore]` without justification: `crates/ml/tests/ppo_long_training_test.rs:139` — S=2 B=1 E=1 score=4.0
- [ ] [TESTROT-004] `#[ignore]` without justification: `crates/ml/tests/dqn_long_training_test.rs:109` — S=2 B=1 E=1 score=4.0
- [ ] [TESTROT-005] `#[ignore]` without justification: `crates/ml/tests/ppo_checkpoint_roundtrip_test.rs:93` — S=2 B=1 E=1 score=4.0
- [ ] [TESTROT-006] `#[ignore]` without justification: `crates/ml/tests/test_var_gradient_flow.rs:88` — S=2 B=1 E=1 score=4.0
## Resolved this run
- [x] [DEAD-001] `use_different_seeds` — iter 1 — commit 4bbe1e561
- [x] [FFLAG-001] `use_percentage_pnl` collapsed to unconditional percentage P&L — iter 1 — commit 3c6569679
- [x] [UNWRAP-001] false-positive (test code) — iter 2 — no commit
- [x] [UNWRAP-002] q_gap partial_cmp NaN-safe — iter 2 — commit e508475f3
- [x] [FFLAG-015] `enable_network_diagnostics` dead flag removed — iter 2 — commit 0c5186cda
- [x] [FFLAG-009] `use_regime_conditioning` dead flag removed — iter 3 — commit 9c9c7c6d8
- [x] [FFLAG-006] `use_soft_updates` collapsed to unconditional EMA — iter 3 — commit a05cae1e4
- [x] [FFLAG-002] `use_spectral_norm` in RainbowNetworkConfig — iter 4 — commit 39d18f582
- [x] [FFLAG-005] `use_spectral_norm`+`use_residual` in QNetworkConfig — iter 4 — commit 609041257
- [x] [FFLAG-003] `enable_q_value_clipping` collapsed to unconditional clipping — iter 5 — commit b78fd2f1f
- [x] comment normalized (BUG #37 FIX tracker preamble → plain invariant comment) — iter 5 — commit b0c0f6c51
- [x] [FFLAG-011] `enable_causal_intervention` dead-flag chain removed — iter 5 — commit 35ccc48c9
- [x] [FFLAG-012] `enable_gradient_vaccine` dead-flag chain removed — iter 6 — commit 9fa8c009f
- [x] [FFLAG-018a] `enable_quality_filtering`+`enable_augmentation` in unified_data_loader — iter 6 — commit 37451834a
- [x] [FFLAG-016] transformer 4-flag set (aspirational config, zero impl) — iter 7 — commit 3f4fa33b9
- [x] [FFLAG-017] transformer 3-flag set (feature selectors, no impl) — iter 7 — commit 3f4fa33b9
- [x] [FFLAG-013a] `use_noisy_nets`+`use_distributional` in GpuExperienceCollectorConfig — iter 7 — commit be4f9932b
- [x] [FFLAG-013b] `enable_action_masking` chain — iter 8 — commit e8eee1ed8
- [x] [DEAD-002] `opportunity_scores` dead API — iter 8 — commit 967e56c64
- [x] [DEAD-003] orphan `transformers/features.rs` (126 LOC) — iter 8 — commit 647ff0997
- [x] [DEAD-004] flash_attention flags (4 fields + standard_attention fallback + mask param) — iter 8 — commit 298cc33fc
- [x] [PINMEM-007/013] dtod_copy wrappers renamed → dtod_copy_async (already async internally) — iter 10 — commit 3e8003b7f
- [x] [BORROW-001/ROMEM-003] CudaSlice<i32>→u32 aliasing UB fixed by storing episode_ids as u32 — iter 10 — commit 8a31d3b99
## Known external state (NOT open findings)
Prior WIP blocker (`set_meta_q_pred` missing method) was committed by
the user as 18261c85a between iterations — ml crate now compiles.
## Needs human review
*Per user directive "no deferred tasks, solve properly" (iter 7) the
Needs-human escape hatch is closed. Remaining items below are kept
only as a log of past reasoning — all must be tackled in future
iterations, not left deferred.*
- [x] [FALLBACK-001] `multi_asset.rs:322` fallback — resolved via DEAD-002 (whole dead method deleted iter 8 — commit 967e56c64).
- [ ] [GPUSYNC-006] curiosity.rs `pred.to_host()` — needs GPU MSE+reduction kernel to eliminate the roundtrip. Still open; do NOT defer next iteration.
- [ ] [GPUSYNC-008] regime_conditional.rs `states.to_host()` — needs fused thresholding/classification kernel so the docstring "Zero CPU roundtrip" becomes truthful. Still open; do NOT defer.
- [ ] [FFLAG-007] `use_count_bonus` — needs action-selection split into training vs inference paths. Still open; do NOT defer.
- [ ] [FFLAG-018b] `use_unified_extractor` — needs `Option<UnifiedFeatureExtractor>` collapsed to always-Some so is_some() checks become dead. Still open; do NOT defer.
## BLOCKED
<empty>