Per user directive, ran rg scans for PINMEM, ROMEM, LOCKHOT, BORROW,
CPURO and populated the scoreboard with concrete file:line findings.
PINMEM: 10 new findings (PINMEM-011..019, plus rescan confirmed 001-010)
- Biggest surface: gpu_experience_collector.rs (7 htod sites)
- High priority: gpu_iqn_head.rs dtod_copy for iqn_rewards/iqn_dones
(PINMEM-013, score 25.0, switch to async-dtod, E=1)
ROMEM: 5 new findings (004 expanded, 005-008 added)
- ROMEM-004 now covers ~40 cuBLAS/cuBLASLt/cuDNN workspace casts across
shared_cublas_handle.rs, gpu_iql_trainer.rs, gpu_iqn_head.rs,
gpu_curiosity_trainer.rs, cublaslt_debug.rs — bulk false-positive
candidates (FFI convention, not actual RO writes)
- ROMEM-007 adds 6 more device-mapped pinned write sites (same pattern
as ROMEM-001/002 — benign cuMemHostAllocMapped)
LOCKHOT: 5 new findings (006-010)
- LOCKHOT-006/008: tokio::sync::Mutex<PPO> and RwLock<TLOBTransformer>
held across .await — deadlock risk. High priority.
- LOCKHOT-007: Arc<Mutex<VecDeque<f64>>> history locks per step
BORROW: 1 new finding (002)
- BORROW-002: RefCell<PPO> × 2 in validation/ppo_adapter.rs —
documented single-threaded, needs invariant verified
CPURO: deferred — .len()/.shape() scan returns hundreds of mostly-Vec
matches; left CPURO-000 task for next iter to classify per-site.
Scoreboard now has ~40 open findings across 12 active categories.
Expanding the scanner surface per user directive "all should be
addressed". Four distinct root causes for CPU-side violations around
GPU-owned data — each with its own scan command, fix hierarchy, and
learned-patterns entry.
- CPURO: CPU reads of GPU-resident data (sizes, reductions) that
force implicit sync. Cache at construction, keep stats on device.
- ROMEM: `*(ptr as *mut T)` writes where ptr came from a const /
read-only source. CUDA mapped-memory flags matter; cuBLAS/cuDNN
workspace casts are benign FFI.
- LOCKHOT: Mutex/RwLock on per-step path. High-value hits already
visible: Mutex<GpuDropout>, Mutex<Option<DropoutScheduler>> in
network.rs (every forward pass), Arc<Mutex<NStepBuffer>> in dqn.rs.
Never hold tokio::sync::RwLock across .await.
- BORROW: shared-&T promoted to &mut T via unsafe ptr casts or
UnsafeCell/RefCell. Real example shipped: gpu_replay_buffer.rs:690
changes CudaSlice<i32> → CudaSlice<u32> through raw-ptr cast.
Scoreboard seeded with 10 findings. Top scores:
- ROMEM-001 (25.0) - size_pinned mapped write, verify allocation flag
- ROMEM-004 (25.0) - cuBLAS workspace casts, likely false-positive bulk
- ROMEM-002 (15.0) - init-time pinned mapped writes
- LOCKHOT-001/002/003 (8.3) - dropout + nstep_buffer locks on hot path
- BORROW-001 (8.3) - CudaSlice element-type aliasing
CPURO is seeded with a scan-task (CPURO-000) to populate per-site
findings in iter 9 — too many Vec::len() false positives to list upfront.
Per user directive: htod/dtod transfers on the training hot path that
bounce through pageable host memory are a top-priority cleanup target.
Pinned memory (cuMemHostAlloc) + memcpy_htod_async enables DMA + stream
overlap. The codebase already has the pattern (size_pinned,
rng_step_pinned in gpu_replay_buffer.rs) — extend it to every per-step
transfer.
Prompt changes:
- New PINMEM category in §3 table (severity 5)
- Scan command excludes tests/benches/examples/smoke_tests
- New §6 learned pattern entry with 3-tier fix hierarchy:
(1) eliminate via on-device compute → (2) stage through pinned →
(3) async for dtod
- Added PINMEM to category ID allowlist in §2
Scoreboard seed: 10 initial findings from an iter-9 scan. Top-scoring:
PINMEM-003 (scratch_f32 scalar, E=1, score=25) and PINMEM-004 (init
scalars, score=15). Hot-path candidates include NoisyNet epsilon
refresh, PER replay-buffer inserts, target-net sync, PPO rollout.
Per user directive from iter 7 ("no deferred tasks, solve properly"),
the Needs-human review escape hatch is closed. Updated:
- Step 6c: replaced "move to Needs human review" with split-into-
sub-findings guidance; every finding must be solved.
- Step 6e: cargo-check failure now requires diagnosis + retry, not
deferral. Pre-existing external errors go to Known external state.
- Step 8 summary line: dropped the "deferred j" counter.
- Rule 7: similar rephrasing.
- New §6: learned patterns from iters 1-8 (dead flag chains,
aspirational config, dishonest fallbacks, inference-side flag flips,
accounting-only values, orphan files, hidden GPU sync).
Ralph re-feeds this file verbatim, so the next iteration picks up the
updated rules automatically.
- Layer 2 labeled as G2-G5, G5 explicit for gradient budget
- P2 spectral_gap_norm added as 7th component in composition (rebalanced weights)
- N6 ensemble oracle has explicit threshold (>0.8 triggers N3 immediately)
- N8 meta-Q wiring clarified: logged but not in composition until validated
- Files Changed deduplicated, paths qualified
- Success criteria: WinRate >55% (above random baseline ~25%)
- HEALTH_DIAG log line includes all 7 components
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes Q-value collapse via unified LearningHealth signal that senses
training health (6 components) and continuously adapts:
- CQL regularization (regime + health gated)
- Gradient budget (IQN/CQL/Ens/C51 dynamic allocation)
- Tau target EMA (health-coupled)
- Expected SARSA temperature (continuous, no hardcoded threshold)
Plus 4 pearls (PER priorities, spectral detection, gradient consistency,
adaptive gamma) and 8 novels (self-distillation, barrier loss, plasticity
injection, CF curriculum, information bottleneck, ensemble oracle,
contrarian override, meta-Q network).
Core principle: training hyperparameters are OUTPUTS of the temporal
pipeline, not static schedules. The system meta-learns its own settings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Training and validation use different kernels with different state layouts,
making generalization impossible. Single state_layout.cuh + one assembly
function + STATE_DIM as compile-time constant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed the legacy non-branching Sequential q_network and target_network
from DQNAgent. These were never used (branching+dueling always active)
but allocated VRAM and ran noise resets every step. -190 lines.
Config tuning for dense micro-reward system:
- n_steps: 5→1 (TD(0), micro-rewards cancel over n>1)
- tau: 0.007→0.01 (faster target tracking for TD(0))
- c51_alpha_max: 0.5→1.0 (full C51, PopArt handles normalization)
- curiosity_weight: 0.1→0.0 (dense micro-reward replaces curiosity)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
target_dim expansion: adds raw_open (OHLCV) and mid_price_open
(MBP-10 midpoint at bar formation) to fxcache targets. FXCACHE_VERSION
2→3 for auto-rebuild. Legacy v2 files handled with close-price fallback.
Spec v5 adds 3 pearls:
- Bar duration encoding in Mamba2 (continuous-time SSM awareness)
- Order book center of mass from all 10 MBP-10 levels (aggression signal)
- Retrospective hold quality bonus (teaches exit timing)
Plus: Hold action (4th direction), DSR Sharpe EMA fix, counterfactual
magnitude/order sign fix, MFT mid-price mark-to-market.
OFI embed MLP now 18→10 (was 16→8). Mamba2 width SH2+10 (was SH2+8).
Attention width D+10 (was D+8).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
target_dim 4→6: adds raw_open (OHLCV) and mid_price_open (MBP-10
best bid/ask midpoint at bar formation) to the targets buffer.
Micro-reward now uses real intra-bar move (close - open) / atr
instead of close-to-close approximation. Also adds spread cost
awareness via |open - mid| penalty. Requires fxcache recomputation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Major changes from v1:
- Fix OFI data pipeline (ofi_gpu never wired, indices 42→66)
- Expand PORTFOLIO_STRIDE 30→38 for prev-OFI storage
- Add Mamba2 d_h_history backward (2 new cuBLAS GEMMs)
- Expose attention d_input_scratch for gradient flow
- Pre-compute OFI deltas in experience collection (state[74..82])
- Lower rank normalization threshold to 1e-5 for micro-rewards
- Use close-to-close return instead of unavailable open_price
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dense per-bar reward using order flow momentum × price confirmation.
OFI embedding MLP (16→8 via cuBLAS) feeds into Mamba2 history AND
attention input. Replaces sparse exit-only reward with continuous
temporal signal for the SSM and attention heads to learn from.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Single cuGraphExecLaunch per step. PER sampling, gather, training,
priority update ALL as child graph nodes in one parent. Direct-to-trainer
gather eliminates DtoD copies. GPU-side counters eliminate host writes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: CUfunction sharing between graphed children and ungraphed
outside-graph ops corrupts kernel state on Hopper → 3100ms adam replay.
Fix: capture EVERYTHING unconditionally in child graphs. Selectivity,
denoise, causal intervention, vaccine, Q-stats all become graphed children.
Zero ungraphed launches = zero CUfunction conflicts.
7 children, ~190 kernel nodes, one capture, one replay per step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 results: 4.0s/step GPU compute (graph replay = ungraphed, cuBLAS
replaced manual matmul but feature count increased). CPU pipeline fully
async (0.1ms/step). Total ~718s/epoch — 100% GPU bound.
Updated Phase 2 targets: aux parallelism (2b) is highest priority —
80/177 kernels with 4 independent trainers. nsys profiling should run
first to validate SM vs memory-bound assumptions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Inventory penalty makes flat optimal — model learns to do nothing.
Instead: penalize inaction proportional to model's own predicted edge
(Q-value spread). Creates virtuous cycle: better temporal attention →
higher self-imposed penalty for missed trades → more trading on signal.
No hindsight bias (uses predicted edge, not actual price change).
Micro-reward (already exists) rewards correct positioning.
Churn penalty (new) prevents rapid flips.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
forward_child: 4 branch streams fork after h_s2, join before loss
aux_child: IQL/IQN/attention on parallel streams
Uses CU_STREAM_CAPTURE_MODE_RELAXED with fork-join events as graph edges
Existing branch_streams[4] + events in batched_forward.rs ready to activate
Target: <40s epochs (from <80s Phase 1)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Portfolio slots ps[15]-ps[19] are Kelly accumulators (active).
Plan params moved to ps[23]-ps[29]. PORTFOLIO_STRIDE grows 23→30.
Fixed asymmetry: scales profit_target only (not stop_loss).
Clarified epsilon mid-plan: overridden by direction lock (intended).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
