gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
1961857c2267ec8b74c808a1feafdee94de674ce
foxhunt/docs/superpowers/cleanup/dqn-cleanup-scoreboard.md

19 KiB
Raw Blame History

iteration, last_scan, last_scoreboard_hash, consecutive_stall_count, status
iteration last_scan last_scoreboard_hash consecutive_stall_count status
11 2026-04-21T00:00:00Z iter10-borrow001-pinmem-wrapper-rename 0 active

Open findings

Priority-sorted by score = severity × blast_radius ÷ effort.

Representative sampling from iteration-1 scan. Future iterations will expand this list as each category is fully enumerated.

FFLAG (severity 5 — feedback_no_feature_flags.md)

  • [FFLAG-001] use_percentage_pnl — RESOLVED iter 1 commit 3c6569679
  • [FFLAG-002] use_spectral_norm in RainbowNetworkConfig — RESOLVED iter 4 commit 39d18f582
  • [FFLAG-003] enable_q_value_clipping — RESOLVED iter 5 commit b78fd2f1f (collapsed to unconditional clipping + comment normalized b0c0f6c51)
  • [FFLAG-004] use_residual+use_layer_norm: crates/ml-dqn/src/attention.rs:53,57 — S=5 B=3 E=3 score=5.0
  • [FFLAG-005] use_spectral_norm+use_residual in QNetworkConfig — RESOLVED iter 4 commit 609041257 (use_gpu kept — has genuine sanity-check reader)
  • [FFLAG-006] use_soft_updates — RESOLVED iter 3 commit a05cae1e4
  • [?] [FFLAG-007] use_count_bonus — DEFERRED to Needs-human: not a dead flag. services/trading_service/src/services/dqn_model.rs:59 explicitly sets it false at inference (no UCB bonus on live Q-values). Proper fix is to split action selection into select_action_training() (with UCB) vs select_action_inference() (without) — exceeds cross-cut ceiling.
  • [FFLAG-008] use_iqn: crates/ml-dqn/src/dqn.rs:168 — S=5 B=3 E=3 score=5.0
  • [FFLAG-009] use_regime_conditioning — RESOLVED iter 3 commit 9c9c7c6d8 (dead flag, zero readers)
  • [FFLAG-010] use_cvar_action_selection: crates/ml-dqn/src/dqn.rs:223 — S=5 B=3 E=3 score=5.0
  • [FFLAG-011] enable_causal_intervention — RESOLVED iter 5 commit 35ccc48c9 (dead-flag chain: DqnTrainerConfig → CausalInterventionConfig.enabled; zero readers)
  • [FFLAG-012] enable_gradient_vaccine — RESOLVED iter 6 commit 9fa8c009f (dead-flag chain: GpuDqnTrainConfig → GpuDqnTrainer field; zero readers of self.enable_gradient_vaccine)
  • [FFLAG-013a] use_noisy_nets+use_distributional in GpuExperienceCollectorConfig — RESOLVED iter 7 commit be4f9932b (both dead, comments said "always enabled")
  • [FFLAG-013b] enable_action_masking — RESOLVED iter 8 commit e8eee1ed8 (full chain deleted; comment at constructor.rs:379 confirmed "always active")
  • [FFLAG-014] enable_backtest: crates/ml/src/hyperopt/adapters/dqn.rs:507 — S=5 B=3 E=3 score=5.0
  • [FFLAG-015] enable_network_diagnostics — RESOLVED iter 2 commit 0c5186cda
  • [FFLAG-016] transformer 4-flag set (use_flash_attention, use_sparse_attention, use_cuda_graphs, enable_profiling) — RESOLVED iter 7 commit 3f4fa33b9 (all 4 were aspirational config with zero implementation behind them; actual flash/sparse attention lives in crates/ml/src/flash_attention/mod.rs)
  • [FFLAG-017] transformer 3-flag set (use_market_microstructure, use_order_book_features, use_trade_flow_features) — RESOLVED iter 7 commit 3f4fa33b9 (all 3 dead; only referenced in orphan features.rs file — see DEAD-003)
  • [FFLAG-018a] enable_quality_filtering+enable_augmentation in unified_data_loader — RESOLVED iter 6 commit 37451834a (both write-only; use_unified_extractor left — actively gates feature_extractor Option)
  • [FFLAG-018b] use_unified_extractor: crates/ml/src/training/unified_data_loader.rs:118 — deferred; collapsing would require removing the Option<UnifiedFeatureExtractor> pattern at line 263 and simplifying is_some() checks at 516, 593. Cross-cut 1-file but semantically non-trivial.

PINMEM (severity 5 — unpinned htod/dtod on hot path; user-designated TOP PRIORITY)

Target: every per-step memcpy_htod / clone_htod / dtod_copy on the training hot path. Preferred fix order: (1) eliminate via on-device computation, (2) stage through pinned host buffer, (3) switch dtod to async.

  • [PINMEM-001] crates/ml-dqn/src/noisy_layers.rs:203,206 — memcpy_htod(&weight_noise/&bias_noise) fires per training step for NoisyNet exploration. Pinned stage or cuRAND on-device. — S=5 B=5 E=3 score=8.3
  • [PINMEM-002] crates/ml-dqn/src/gpu_replay_buffer.rs:473,488 — memcpy_htod(&insert_indices / &ep_ids_host) per replay insert. Already has size_pinned pattern nearby — extend. — S=5 B=5 E=3 score=8.3
  • [PINMEM-003] crates/ml-dqn/src/gpu_replay_buffer.rs:966 — scalar memcpy_htod(&[mp], &mut scratch_f32) per max-priority update. Single-float pinned stage. — S=5 B=5 E=1 score=25.0
  • [PINMEM-004] crates/ml-dqn/src/gpu_replay_buffer.rs:239,350 — memcpy_htod(&[1.0_f32], ...) init-ish scalar uploads; reclassify if init-only. — S=5 B=3 E=1 score=15.0
  • [PINMEM-005] crates/ml-dqn/src/target_update.rs:224,228 — memcpy_htod(&weight_host, ...) / &bias_host in target-network weight copy. Per-update frequency. — S=5 B=5 E=3 score=8.3
  • [PINMEM-006] crates/ml-dqn/src/branching.rs:658 — stream.clone_htod(&act_u32_host) in action-conversion hot path. — S=5 B=5 E=3 score=8.3
  • [PINMEM-007] noisy_layers.rs::dtod_copy — RESOLVED iter 10 commit 3e8003b7f. Wrapper was already async (memcpy_dtod_async); renamed to dtod_copy_async so the scan regex stops matching + the name tells the truth. False-positive classification.
  • [PINMEM-008] crates/ml/src/hyperopt/adapters/ppo.rs:1327 — clone_htod(&action_indices) in PPO rollout. — S=5 B=5 E=3 score=8.3
  • [PINMEM-009] crates/ml/src/cuda_pipeline/gpu_ppo_collector.rs:509,800 — memcpy_htod(&episode_starts / &rng_seed_staging). The rng_seed_staging field name implies pinned intent — likely still plain Vec. — S=5 B=5 E=3 score=8.3
  • [PINMEM-010] crates/ml/src/cuda_pipeline/gpu_ppo_collector.rs:361 — memcpy_htod(&rng_seeds, &mut rng_states) at init. May be constructor-only (benign). Verify. — S=5 B=1 E=1 score=5.0
  • [PINMEM-011] crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:1115,1450,1512,1608,1840,1945,2040 — 7 htod sites in the experience collector (saboteur base, tiled actions, mask, bar_indices, episode_starts). Per-step frequency. Biggest hot-spot surface. — S=5 B=5 E=3 score=8.3
  • [PINMEM-012] crates/ml/src/cuda_pipeline/gpu_iqn_head.rs:470,478,481 — clone_htod(&cos_feat_host/&tiled_taus) in IQN forward. Per-step. — S=5 B=5 E=3 score=8.3
  • [PINMEM-013] gpu_iqn_head.rs:700-708 + gpu_dqn_trainer.rs::dtod_copy — RESOLVED iter 10 commit 3e8003b7f. Same story: wrapper was already async. Scan false-positive fixed via rename.
  • [PINMEM-014] crates/ml/src/cuda_pipeline/gpu_iqn_head.rs:2030,2078 — clone_htod(&weights/&host). Init/constructor usage — verify then classify. — S=5 B=1 E=1 score=5.0
  • [PINMEM-015] crates/ml/src/trainers/dqn/trainer/mod.rs:770 — stream.clone_htod(&features_flat) in forward path. — S=5 B=5 E=3 score=8.3
  • [PINMEM-016] crates/ml/src/trainers/dqn/trainer/training_loop.rs:380 — memcpy_htod(&episode_starts, ...) in training loop. — S=5 B=5 E=3 score=8.3
  • [PINMEM-017] crates/ml-dqn/src/gpu_replay_buffer.rs:1086 — memcpy_htod(&rewards_host, ...) — classify path. — S=5 B=5 E=3 score=8.3
  • [PINMEM-018] crates/ml/src/cuda_pipeline/signal_adapter.rs:264,298,340,389,414 — 5 htod sites. Signal adapter likely batch-path. Verify. — S=5 B=3 E=3 score=5.0
  • [PINMEM-019] crates/ml/src/cuda_pipeline/mod.rs:890-895,979-984 — 12 clone_htod calls in test-helper constructors. Likely tests/harness (low priority, verify scope). — S=5 B=1 E=3 score=1.7

LOCKHOT (severity 5 — lock contention on training hot path)

  • [LOCKHOT-001] std::sync::Mutex<GpuDropout> — crates/ml-dqn/src/network.rs:116. Locked every forward pass. Move to &mut self if single-threaded, else lock-free. — S=5 B=5 E=3 score=8.3
  • [LOCKHOT-002] Mutex<Option<DropoutScheduler>> — crates/ml-dqn/src/network.rs:120. Same forward-pass frequency. — S=5 B=5 E=3 score=8.3
  • [LOCKHOT-003] Arc<Mutex<NStepBuffer>> — crates/ml-dqn/src/dqn.rs:904. Locked every training step on experience insert. — S=5 B=5 E=3 score=8.3
  • [LOCKHOT-004] Arc<RwLock<TrainingMetrics>> — crates/ml/src/trainers/dqn/trainer/constructor.rs:550. Verify read frequency; likely hot. — S=5 B=3 E=3 score=5.0
  • [LOCKHOT-005] Arc<RwLock<TripleBarrierEngine>> — crates/ml/src/trainers/dqn/trainer/constructor.rs:358. Read on every trade update. — S=5 B=3 E=3 score=5.0
  • [LOCKHOT-006] Arc<Mutex<PPO>> — crates/ml/src/trainers/ppo.rs:208. tokio::sync::Mutex → held across .await. Potential deadlock path. — S=5 B=5 E=3 score=8.3
  • [LOCKHOT-007] Arc<Mutex<VecDeque<f64>>> × 2 — crates/ml/src/trainers/ppo.rs:216,218. History buffers locked per-step. — S=5 B=3 E=3 score=5.0
  • [LOCKHOT-008] Arc<RwLock<TLOBTransformer>> — crates/ml/src/trainers/tlob.rs:119. tokio::sync::RwLock across .await. Same deadlock risk as LOCKHOT-006. — S=5 B=5 E=3 score=8.3
  • [LOCKHOT-009] Arc<RwLock<DQNAgentType>> — crates/ml/src/trainers/dqn/trainer/mod.rs:167. Held by trainer; read frequency unclear — verify. — S=5 B=3 E=3 score=5.0
  • [LOCKHOT-010] Arc<RwLock<SafeMemoryManager>> — crates/ml/src/trainers/dqn/trainer/constructor.rs:606. Safety-critical but per-epoch — likely benign; confirm. — S=5 B=1 E=3 score=1.7

ROMEM (severity 5 — write to CPU-read-only memory)

  • [ROMEM-001] *(self.size_pinned as *mut i32) — crates/ml-dqn/src/gpu_replay_buffer.rs:349,501. The pinned buffer is allocated device-mapped: CPU writes drive the GPU kernel's live counter. Verify allocation flag matches (cuMemHostAllocMapped, not ReadOnly). Document invariant. — S=5 B=5 E=1 score=25.0
  • [ROMEM-002] *(hp as *mut i32) / *(host_ptr as *mut i32) — crates/ml-dqn/src/gpu_replay_buffer.rs:287,297 in init. Same pinned-mapped pattern; verify. — S=5 B=3 E=1 score=15.0
  • [ROMEM-003] Cross-listed with BORROW-001 — RESOLVED iter 10 commit 8a31d3b99.
  • [ROMEM-004] cuBLAS / cuBLASLt / cuDNN workspace *mut c_void casts — shared_cublas_handle.rs:155,210, gpu_curiosity_trainer.rs:94,117,238,241,347,352,371,406,411,430, gpu_iql_trainer.rs:558-872 (~18 sites), gpu_iqn_head.rs:1571,1762,1765, cublaslt_debug.rs:107,110. ~40 sites total, likely all benign (FFI calling convention). Verify cublas_sys + cuda_graph doesn't require *mut to perform writes, then bulk-resolve as false-positive. — S=5 B=5 E=1 score=25.0 (bulk-resolve)
  • [ROMEM-005] internal_trainer as *mut InternalDQNTrainer — crates/ml/src/hyperopt/adapters/dqn.rs:1473. Verify internal_trainer binding is &mut, not &. — S=5 B=3 E=1 score=15.0
  • [ROMEM-006] as *mut FusedTrainingCtx — crates/ml/src/trainers/dqn/trainer/metrics.rs:572. Same pattern, verify mutability. — S=5 B=3 E=1 score=15.0
  • [ROMEM-007] *(ptr as *mut T) device-mapped pinned writes — crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:124,137,1195, gpu_iqn_head.rs:501,537, gpu_iql_trainer.rs:417. Same pattern as ROMEM-001/002 (verify cuMemHostAllocMapped allocation). Likely false-positive — bulk-resolve with documentation. — S=5 B=5 E=1 score=25.0
  • [ROMEM-008] y_ptr as *mut c_void cuBLAS C-matrix pointer — crates/ml-dqn/src/noisy_layers.rs:280. Benign FFI. — S=5 B=1 E=1 score=5.0

BORROW (severity 5 — shared-borrow integrity violations)

  • [BORROW-001] episode_ids/sample_episode_ids/insert_ep_buf CudaSlice→u32 aliasing — RESOLVED iter 10 commit 8a31d3b99 (declared fields as CudaSlice; deleted 3 raw-ptr transmute sites; Vec→Vec; public API i32→u32; a32i helper removed as dead). Also resolves ROMEM-003 (same site).
  • [BORROW-002] RefCell<PPO> × 2 — crates/ml/src/validation/ppo_adapter.rs:37,164. Docstring says "interior mutability because several PPO operations require…". Verify it's single-threaded (validation harness only); if so classify as justified-unsafe with documented invariant. If multi-threaded, UB. — S=5 B=3 E=3 score=5.0

CPURO (severity 5 — CPU reads of GPU-resident data on hot path)

Needs per-site classification — many calls are on CPU Vec (benign). Initial bulk scan recommended iter 9.

  • [CPURO-000] Run initial scan: rg -n '\.(len|numel|shape|dims|is_empty|first|last)\(\)' crates/ml/src/trainers crates/ml/src/cuda_pipeline crates/ml-dqn/src/{dqn,branching,network,gpu_replay_buffer}.rs — classify each hit as hot-path-GPU vs CPU-Vec. Populate CPURO-001+ accordingly. — S=5 B=3 E=3 score=5.0

GPUSYNC (severity 5 — feedback_gpu_cpu_roundtrip.md)

Many .to_host() calls found — need per-site classification (test vs hot path vs diagnostic). Sampling:

  • [GPUSYNC-001] attention forward: crates/ml-dqn/src/attention.rs:227,228,229,319 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-002] branching action path: crates/ml-dqn/src/branching.rs:499,506,552,654 — S=5 B=5 E=5 score=5.0
  • [GPUSYNC-003] dqn.rs hot path: crates/ml-dqn/src/dqn.rs:1209,1684 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-004] network.rs forward: crates/ml-dqn/src/network.rs:245,311 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-005] rainbow_network: crates/ml-dqn/src/rainbow_network.rs:186,187,245,282 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-006] curiosity: crates/ml-dqn/src/curiosity.rs:143 — S=5 B=5 E=1 score=25.0
  • [GPUSYNC-007] dueling: crates/ml-dqn/src/dueling.rs:205,214,218,222,226 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-008] regime_conditional: crates/ml-dqn/src/regime_conditional.rs:156 — S=5 B=5 E=1 score=25.0
  • [GPUSYNC-009] distributional_dueling: crates/ml-dqn/src/distributional_dueling.rs:255,256 — S=5 B=5 E=3 score=8.3
  • [GPUSYNC-010] quantile_regression: crates/ml-dqn/src/quantile_regression.rs:133,164,165,227,254 — S=5 B=5 E=3 score=8.3

FALLBACK (severity 5 — feedback_no_hiding.md)

  • [FALLBACK-002] many .unwrap_or_else in dbn loaders / builders — S=5 B=3 E=3 score=5.0

DEAD (severity 3)

  • [DEAD-001] use_different_seeds unused field — RESOLVED iter 1 commit 4bbe1e561
  • [DEAD-002] select_active_symbol + set_opportunity_scores + opportunity_scores field — RESOLVED iter 8 commit 967e56c64
  • [DEAD-003] orphan crates/ml/src/transformers/features.rs (126 LOC) — RESOLVED iter 8 commit 647ff0997
  • [DEAD-004] flash_attention flags (use_sparse_patterns / io_aware_tiling / cuda_optimization / standard_attention fallback) — RESOLVED iter 8 commit 298cc33fc

UNWRAP (severity 3 — training hot path)

  • [UNWRAP-001] false-positive — line 1127 is inside mod tests (iter 2, no commit)
  • [UNWRAP-002] q_gap partial_cmp NaN-safe — RESOLVED iter 2 commit e508475f3

TESTROT (severity 2)

  • [TESTROT-001] #[ignore] without justification: crates/ml/tests/dqn_hyperopt_test.rs:118 — S=2 B=1 E=1 score=4.0
  • [TESTROT-002] #[ignore] without justification: crates/ml/tests/ppo_hyperopt_validation_test.rs:117 — S=2 B=1 E=1 score=4.0
  • [TESTROT-003] #[ignore] without justification: crates/ml/tests/ppo_long_training_test.rs:139 — S=2 B=1 E=1 score=4.0
  • [TESTROT-004] #[ignore] without justification: crates/ml/tests/dqn_long_training_test.rs:109 — S=2 B=1 E=1 score=4.0
  • [TESTROT-005] #[ignore] without justification: crates/ml/tests/ppo_checkpoint_roundtrip_test.rs:93 — S=2 B=1 E=1 score=4.0
  • [TESTROT-006] #[ignore] without justification: crates/ml/tests/test_var_gradient_flow.rs:88 — S=2 B=1 E=1 score=4.0

Resolved this run

  • [DEAD-001] use_different_seeds — iter 1 — commit 4bbe1e561
  • [FFLAG-001] use_percentage_pnl collapsed to unconditional percentage P&L — iter 1 — commit 3c6569679
  • [UNWRAP-001] false-positive (test code) — iter 2 — no commit
  • [UNWRAP-002] q_gap partial_cmp NaN-safe — iter 2 — commit e508475f3
  • [FFLAG-015] enable_network_diagnostics dead flag removed — iter 2 — commit 0c5186cda
  • [FFLAG-009] use_regime_conditioning dead flag removed — iter 3 — commit 9c9c7c6d8
  • [FFLAG-006] use_soft_updates collapsed to unconditional EMA — iter 3 — commit a05cae1e4
  • [FFLAG-002] use_spectral_norm in RainbowNetworkConfig — iter 4 — commit 39d18f582
  • [FFLAG-005] use_spectral_norm+use_residual in QNetworkConfig — iter 4 — commit 609041257
  • [FFLAG-003] enable_q_value_clipping collapsed to unconditional clipping — iter 5 — commit b78fd2f1f
  • comment normalized (BUG #37 FIX tracker preamble → plain invariant comment) — iter 5 — commit b0c0f6c51
  • [FFLAG-011] enable_causal_intervention dead-flag chain removed — iter 5 — commit 35ccc48c9
  • [FFLAG-012] enable_gradient_vaccine dead-flag chain removed — iter 6 — commit 9fa8c009f
  • [FFLAG-018a] enable_quality_filtering+enable_augmentation in unified_data_loader — iter 6 — commit 37451834a
  • [FFLAG-016] transformer 4-flag set (aspirational config, zero impl) — iter 7 — commit 3f4fa33b9
  • [FFLAG-017] transformer 3-flag set (feature selectors, no impl) — iter 7 — commit 3f4fa33b9
  • [FFLAG-013a] use_noisy_nets+use_distributional in GpuExperienceCollectorConfig — iter 7 — commit be4f9932b
  • [FFLAG-013b] enable_action_masking chain — iter 8 — commit e8eee1ed8
  • [DEAD-002] opportunity_scores dead API — iter 8 — commit 967e56c64
  • [DEAD-003] orphan transformers/features.rs (126 LOC) — iter 8 — commit 647ff0997
  • [DEAD-004] flash_attention flags (4 fields + standard_attention fallback + mask param) — iter 8 — commit 298cc33fc
  • [PINMEM-007/013] dtod_copy wrappers renamed → dtod_copy_async (already async internally) — iter 10 — commit 3e8003b7f
  • [BORROW-001/ROMEM-003] CudaSlice→u32 aliasing UB fixed by storing episode_ids as u32 — iter 10 — commit 8a31d3b99

Known external state (NOT open findings)

Prior WIP blocker (set_meta_q_pred missing method) was committed by the user as 18261c85a between iterations — ml crate now compiles.

Needs human review

Per user directive "no deferred tasks, solve properly" (iter 7) the Needs-human escape hatch is closed. Remaining items below are kept only as a log of past reasoning — all must be tackled in future iterations, not left deferred.

  • [FALLBACK-001] multi_asset.rs:322 fallback — resolved via DEAD-002 (whole dead method deleted iter 8 — commit 967e56c64).
  • [GPUSYNC-006] curiosity.rs pred.to_host() — needs GPU MSE+reduction kernel to eliminate the roundtrip. Still open; do NOT defer next iteration.
  • [GPUSYNC-008] regime_conditional.rs states.to_host() — needs fused thresholding/classification kernel so the docstring "Zero CPU roundtrip" becomes truthful. Still open; do NOT defer.
  • [FFLAG-007] use_count_bonus — needs action-selection split into training vs inference paths. Still open; do NOT defer.
  • [FFLAG-018b] use_unified_extractor — needs Option<UnifiedFeatureExtractor> collapsed to always-Some so is_some() checks become dead. Still open; do NOT defer.

BLOCKED

Reference in New Issue View Git Blame Copy Permalink