Replace the GPU->CPU sync barrier (to_vec1 readback) in the DQN training
hot path with the GpuTrainingGuard CUDA kernel that performs NaN detection,
loss clipping, and gradient collapse checks entirely on-device. The guard
is lazy-initialized on first training step and falls back to the original
CPU readback path if CUDA kernel compilation fails.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wraps 4 CUDA kernels (training_guard_check, training_guard_accumulate,
qvalue_stats_reduce, qvalue_divergence_check) with a Rust struct that
uses OnceLock PTX caching, pre-allocated device buffers, and host-side
Vec mirrors for zero-allocation readbacks per training step.
Accumulator (3-float acc_buf) stays on-device for epoch-boundary
averaging without CPU roundtrips.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add use_dsr, dsr_eta, and n_steps fields to ExperienceCollectorConfig
with safe defaults (DSR off, eta=0.01, n_steps=1). Pass them as kernel
args after fill_simulation_enabled, matching the parameter order added
in Tasks 4 and 5. Wire from DqnHyperparams in the trainer hot path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirror the DSR (Differential Sharpe Ratio) and n-step return
accumulation added by Task 4 to the scalar kernel into the warp
kernel (dqn_full_experience_kernel_warp). DSR/n-step state runs
on lane 0 only, matching the existing reward computation pattern.
All accumulators are reset on episode boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Differential Sharpe Ratio reward shaping and n-step return
accumulation to dqn_full_experience_kernel (scalar path). DSR replaces
EMA normalization when use_dsr=1; n-step ring buffer accumulates
discounted returns for effective_n>1. Both are reset on episode
boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Append third CUDA kernel to epsilon_greedy_kernel.cu for Branching DQN:
3 independent epsilon-greedy heads (exposure/5, order/3, urgency/3)
compose to factored action index 0-44 (exposure*9 + order*3 + urgency).
Add Rust composition tests verifying full 45-action coverage and
round-trip decomposition correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
is_gpu_prioritized() and grad_norm squeeze are only available with CUDA.
Without the gate, compile-services (CPU-only) fails with E0599.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-step to_scalar()/to_vec1() readbacks with GPU-resident
tensor accumulation and single epoch-boundary sync. On H100 this
removes ~12-15 pipeline flushes per training step (~5μs each),
enabling full GPU saturation with zero CPU sync in the inner loop.
Key changes:
- GpuTrainResult: train_step() returns GPU scalar tensors (loss_gpu,
grad_norm_gpu) instead of f32 — zero readback per step
- Regime conditional: train all 3 heads unconditionally with
zero-masked weights (mathematical no-op) instead of 3-6
to_scalar() mask count checks per step
- GPU PER: max_priority as GPU tensor with flush_max_priority(),
delta-based priority update via index_add (no CPU dedup)
- Deferred diagnostics: NaN detection, CQL logging, Q-value
estimation all moved to epoch boundary where pipeline is
already synced
- Legacy CPU-readback wrappers removed entirely
9 files changed, +393/-295 lines. 520 DQN tests passing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate GPU→CPU→GPU roundtrip in the experience collection hot path.
Kernel outputs (states, rewards, actions, dones) now stay on GPU via
device-to-device copy into candle Tensors. Only rewards + actions are
downloaded for monitoring metrics (~0.5MB vs ~3.14GB at 32K episodes).
- Extract launch_kernel() helper from collect_experiences() (DRY)
- Add collect_experiences_gpu() — DtoD path for GPU PER
- Add cuda_slice_to_tensor_f32/i32_to_u32 DtoD copy utilities
- GPU next_states via tensor narrow/cat/reshape (no CPU loop)
- Trainer selects GPU vs CPU path based on is_gpu_prioritized()
- stream.synchronize() barrier ensures cross-stream data visibility
0 warnings, 1266 tests pass (874 ml + 392 ml-dqn)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Forward-port from worktree-gpu-hotpath-audit (2 commits):
1. Zero-roundtrip DQN experience collection via cuMemcpyDtoDAsync:
- GpuExperienceCollector uses device-to-device copies for state tensors
- Shared memory weight caching in GPU replay buffer
- NaN priority clamping on GPU (no CPU readback)
2. Eliminate all GPU→CPU roundtrips from training loop:
- GpuTrainResult: loss + grad_norm stay as GPU scalar tensors
- Single to_scalar() readback at epoch boundary (not per step)
- GPU-resident loss accumulation across training steps
- RegimeConditional head selection via GPU tensor ops
- Removed per-step NaN diagnostic checks (now epoch-level)
Also removes dead `states_tensor` field from ComputeLossResult and
fixes redundant field name clippy warning.
Expected: ~15-20% training throughput improvement on H100 by
eliminating synchronous GPU→CPU transfers in the inner loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes temporary benchmark comment. Docker changes from previous commit
will trigger image rebuilds. This run serves as warm-cache benchmark
after cold-cache run completes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Temporary comment to trigger detect-changes for both services and
training compile steps. Will be removed after benchmarking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. PER duplicate index accumulation (HIGH): update_priorities_gpu()
used index_add delta trick which accumulates deltas for duplicate
indices, overshooting target priority. Now deduplicates the small
indices tensor (batch_size=256, ~1KB) via HashMap before delta
computation. Fast path (no dupes) reuses original tensors.
2. RegimeConditional silent experience drop (MEDIUM):
insert_batch_tensors() returned Ok(()) for CPU replay buffers,
silently discarding all GPU-collected experiences. Now converts
tensors→Vec<Experience> via extracted helper (lazy, only on first
CPU head encountered) and inserts into CPU buffer.
3. Unsafe direct indexing (LOW): gpu_experience_collector.rs used
&states[s..e] in fallback paths, violating deny(indexing_slicing).
Replaced with .get() safe bounds checking.
392/392 ml-dqn + 874/874 ml tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add warp-cooperative CUDA kernel for DQN experience collection that
exploits H100's improved warp shuffle throughput. On sm_90+, 32 threads
(1 warp) cooperate on each dot product via __shfl_xor_sync butterfly
reduction, cutting per-thread register pressure from ~5.5KB to ~200B
and enabling full occupancy on 132 SMs.
Key additions:
- 6 warp-cooperative device functions in common_device_functions.cuh:
distributed/broadcast matvec (clean + NoisyNet), warp RMSNorm,
warp_reduce_sum_all
- 4 TILE_LAYER_WARP_* macros using __syncwarp() for warp-level sync
- 3 warp forward passes: standard dueling, NoisyNet, C51 distributional
(distributed heavy layers + broadcast atom layers for softmax)
- Full warp kernel: dqn_full_experience_kernel_warp with lane-0
simulation, strided state scatter, warp-shuffle curiosity gather
- Host-side compute capability detection (sm_major >= 9) with
automatic fallback to standard 256-thread kernel on older GPUs
- cuCtxSetLimit(STACK_SIZE, 16KB) for standard kernel safety
874/874 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three zen-validated optimizations for H100 GPU experience collection:
1. Dynamic NOISY_MAX_DIM: inject via NVRTC as max(state_dim, shared_h1,
shared_h2). On H100 with SHARED_H1=512, noise now covers all 512 input
dims instead of truncating at 256. Fixes exploration correctness bug.
2. MAX_EPISODES_LIMIT raised 4096→32768: H100 (132 SMs) can now run up to
32768 episodes per kernel launch = 128 blocks = ~97% SM utilization
(was 16 blocks = 12%). All 3 trainer.rs caps updated to match.
3. Scratch array aliasing: scratch_v and scratch_a now alias scratch1
memory after shared layers complete, saving 1024 bytes per thread.
Per-thread local memory reduced from ~6.5KB to ~5.5KB.
Zen validation confirms: kernel is safe and correct for H100 deployment.
Remaining architectural improvements (warp-cooperative matvec, batched
GEMM) deferred to future optimization phase.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: SHMEM_MAX_IN_DIM was hardcoded to 256, but on H100 with
hidden_dim_base=2048, SHARED_H1=SHARED_H2=512. The cooperative tile
loader wrote 64×512=32768 floats into shmem allocated for 64×256=16384,
causing CUDA_ERROR_INVALID_VALUE on kernel launch.
Fixes:
- Make SHMEM_MAX_IN_DIM injectable via NVRTC #define (was hardcoded)
- Compute max(state_dim, shared_h1, shared_h2) and inject at compile time
- Use dynamic value for host-side shmem_bytes allocation
- Shrink eps_out[NOISY_MAX_DIM] → eps_out[SHMEM_TILE_ROWS] (saves 768B/thread)
- Force smoke tests to Device::Cpu (CUDA driver sensitivity to binary layout)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
optimal_n_episodes() returns 8192 on H100 (132 SMs) but the GPU
experience collector pre-allocates buffers for MAX_EPISODES_LIMIT=4096.
Exceeding that caused "Episode count exceeds allocated buffer size"
and silent fallback to CPU. Now all 3 auto-scaling sites clamp at 4096.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU experience collector:
- Add GpuHardwareInfo with SM count detection from device name lookup
(H100=132, A100=108, L40S=142, RTX 4090=128, etc.)
- optimal_n_episodes() scales to GPU: sm_count × 2 warps × 32 threads,
capped by 15% free VRAM budget, 256-aligned for block scheduling
- Remove hardcoded .min(256) cap in trainer — auto-scales from 128 to 8192
- MAX_EPISODES_LIMIT raised from 256 to 4096
Numerical stability:
- BF16 cross-entropy epsilon: 1e-8 → 1e-4 (below BF16 precision floor
1e-8 rounds to zero, making log-stabilization a no-op → -inf → NaN)
- Add log_probs.clamp(-20, 0) guard against -inf × 0 = NaN in loss
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BF16's 7-bit mantissa overflows on exp(50+) in softmax → Inf → NaN.
Cast distributional logits to F32 before softmax in both dueling and
rainbow network heads, then cast back to original dtype.
Increase default batch_size 128→1024 (standard), 128→256 (conservative),
512→2048 (aggressive) to saturate H100 tensor cores. AutoBatchSizer
already caps to VRAM ceiling for smaller GPUs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three root causes for "no metrics" on the training dashboard:
1. Dashboard template variables ($model, $fold) sourced from
foxhunt_training_current_epoch which isn't emitted until the first
epoch completes. Switch to foxhunt_training_step which fires from
step 500 onward.
2. train.sh pod template missing Prometheus annotations
(prometheus.io/scrape, port, path). Also add the
app.kubernetes.io/component label to the eval manifest so
evaluation pods are discoverable too.
3. DQN and PPO trainers only called set_epoch() at the END of each
epoch. Move the call to the TOP of the epoch loop so the gauge
exists from the first training iteration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes validated by 20/20 hyperopt trials on H100 (zero OOM):
1. Workspace default-features: ml-core, ml-dqn, ml-ppo, ml-supervised
workspace deps now have default-features=false. Prevents cudarc
(which requires nvcc) from leaking into CPU service builds via
Cargo feature unification. CI compile-services was failing with
"Failed to execute nvcc: No such file or directory" (exit 101).
2. BF16 comparison fix: Candle's gt()/le() don't support BF16 operands.
Cast ADX/CUSUM features to F32 before threshold comparison in
regime classification. Previous approach (cast threshold to BF16)
failed due to Candle broadcast_as reverting dtype.
3. CI pipeline: expand ML change detection to all 14 sub-crates,
add component:compile labels for sccache network policy matching,
bump training runtime to CUDA 12.6 + Ubuntu 24.04 (glibc 2.39).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three root causes fixed:
1. GPU PER + experience collector (cudarc) created dual CUDA allocator
fragmentation on GPUs ≤8 GB, making training impossible even at
batch_size=1. Added `use_gpu_replay_buffer` config flag; both GPU PER
and experience collector now disabled when VRAM ≤8192 MB.
2. Search space bounds were WIDENED instead of capped — max_batch_size
returning 4096 replaced the original 512 upper bound, and
max_hidden_dim_base_full returning 3072 replaced the original 1024.
Fixed with min() to only narrow, never widen.
3. VRAM estimator assumed GPU features always active, overcharging when
they're disabled on small GPUs. Now conditional: when
replay_buffer_capacity=0 (proxy for GPU PER disabled), collector/cudarc
costs are zero and fragmentation multiplier drops from 3× to 1.5×.
Additional small-GPU guard: GPUs ≤8 GB get clamped search space
(batch≤128, hidden≤512, atoms≤51, buffer≤50K) to fit 3 regime heads
+ C51 + noisy nets + dueling in limited VRAM.
Validated: 7/7 trials complete on RTX 3050 Ti 4 GB, zero OOM, best trial
Sharpe 9.8 with 50.6% win rate. Previous runs had 100% OOM failure rate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bring Branching Dueling Q-Network (Tavakoli 2018) to full Rainbow parity
with the existing GPU hotpath. 3 independent advantage heads (exposure=5,
order=3, urgency=3) decompose the 45-action space into learnable branches.
H1 - CUDA fallback: gate GpuExperienceCollector when use_branching=true
(fused kernel hardcodes NUM_ACTIONS=5, incompatible with 45 factored)
H2 - Per-branch C51 distributional: each branch outputs [batch, n_d, atoms]
log-softmax, loss = avg of D cross-entropies vs projected Bellman target
M1 - NoisyNet: MaybeNoisyLinear enum in branch heads, reset_noise/disable_noise
wired through select_action, compute_loss, and set_eval_mode
M2 - Regime-conditional IS weights: Trending=1.2, Ranging=0.8, Volatile=0.6
applied to branching loss via ADX/CUSUM features at state[40:41]
M3 - State dim alignment: align_dim_for_tensor_cores() in from_dqn_params()
for H100 HMMA dispatch (8-byte alignment)
L1 - Fill simulator: splitmix64 replaces golden ratio hash (chi-squared tested)
L2 - Hyperopt 29D: branch_hidden_dim [64,256] added to PSO search space
Config plumbing: branch_hidden_dim, v_min/v_max/num_atoms, use_distributional,
use_noisy, noisy_sigma_init all flow from DQNConfig → BranchingConfig.
10 files, +3207/-125 lines, 33 branching tests + 387 ml-dqn + 284 ml-core pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
training_dtype() now returns BF16 on all CUDA devices, enabling full
tensor-core utilization. F32 is used only at boundaries (scalar
extraction, loss computation, softmax). VRAM estimator updated to
account for BF16 byte sizes, C51/QR atoms, dueling streams, NoisyNet
param doubling, and GPU PER buffer pre-allocation.
Changes:
- mixed_precision.rs: training_dtype() returns BF16 on CUDA
- curiosity.rs: F32 cast before scalar extraction
- network.rs: F32 output at NetworkLayers forward boundary
- traits.rs: estimate_trial_vram_mb_full() with BF16-aware sizing
- dqn.rs adapter: uses full estimator with worst-case architecture
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand FeatureVector from 40 to 42 dimensions by including ADX(14) at
index 40 and CUSUM direction at index 41 from the existing CPU feature
extraction pipeline. This eliminates proxy-based regime classification
and enables GPU-native regime detection via tensor narrow/comparison ops.
Key changes:
- extraction.rs: wire RegimeADXFeatures + RegimeCUSUMFeatures into
extract_current_features_v2(), output 42 features per bar
- regime_conditional.rs: classify_regime_masks_gpu() creates per-regime
mask tensors entirely on GPU (ADX > 0.25 = trending, |CUSUM| > 0.7 =
volatile, else ranging). Zero CPU roundtrip in training hot path.
- trainer.rs/config.rs: state_dim 43→45 (no OFI), 51→53 (with OFI),
aligned dims unchanged (48/56). GPU batch insertion for all 3 heads.
- CUDA header: MARKET_DIM 40→42
- walk_forward.rs: FEATURE_DIM 40→42
- 42 files updated, all [f64;40]→[f64;42] propagated across workspace
Test results: ml=874/0, ml-dqn=354/0, ml-features=282/0, ml-core=274/0
Real data GPU smoke tests: 7/7 passed (OHLCV + OFI + trade enrichment)
Hyperopt baseline RL: 2 trials completed on local RTX 3050 Ti
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Four monitoring functions (estimate_avg_q_value_with_early_stopping,
collect_qvalue_statistics, compute_q_gap_for_epoch, compute_per_action_q_values)
iterated over batch_sample.experiences to build CPU tensors for forward passes.
When GPU PER (GpuPrioritized) is active, experiences is always vec![] — all data
lives on GPU tensors in gpu_batch. This created zero-element tensors with non-zero
shapes, triggering CUBLAS_STATUS_INVALID_VALUE on the next forward pass.
Fix: all four functions now check for gpu_batch.states and use it directly,
falling back to CPU experiences only for non-GPU buffers. Also removes debug
eprintln probes, wires new_on_device for DQN/RegimeConditionalDQN construction,
and adds GPU-aware smoke tests (33 pass, 874 total, 0 failures).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace expensive CPU↔GPU data transfers with GPU-native operations:
- check_gradients_finite: sum_all() scalar readback (4B per tensor)
replaces flatten_all()+to_vec1() that copied entire gradients to CPU
(up to 200MB per step across 9 call sites in DQN+PPO)
- Huber loss: affine() replaces 3× Tensor::from_vec(vec![const; N])
CPU Vec allocations in the inner loss computation loop
- update_priorities_gpu: per-index slice_scatter (~1KB DMA) replaces
full-buffer to_vec1()+from_vec() roundtrip (1.2MB DMA per step)
- PER sampling: single from_vec + GPU to_dtype replaces duplicate
from_vec calls and CPU type conversion roundtrips
- RegimeConditional: log warning on silent GPU batch drops
822 tests pass (350 ml-dqn, 274 ml-core, 198 ml-ppo), 0 clippy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes for GPU PER hot path:
1. IQN quantile loss used empty CPU weights Vec instead of GPU-resident
weights_tensor_cached — caused CUDA_ERROR_ILLEGAL_ADDRESS from
uninitialized GPU memory. Now uses cached GPU tensor matching C51
and standard DQN paths.
2. GpuPrioritized add()/add_batch() replaced with StagedGpuBuffer:
add() stages on CPU (Vec::push, zero GPU ops), sample() batch-flushes
staging→GPU in one DMA before sampling. Production path (insert_batch_tensors)
bypasses staging entirely — GPU→GPU with zero CPU.
3. All 9 ML sub-crates default to cuda feature so `cargo test -p ml-dqn`
exercises GPU code paths on CUDA workstations. CI service crates use
default-features=false, unaffected.
Test results: 350 passed (was 343+7 failed), 0 failed, 1 ignored.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Wire GpuReplayBuffer into DQN constructor with OOM fallback to CPU PER
- Fix P0: GPU PER priorities never updated in single-batch train_step()
(result.indices was empty CPU Vec; GPU tensors td_errors_gpu/indices_gpu
were silently dropped)
- Fix IS-weight ordering: apply weights AFTER Huber loss, not before
(weighting before nonlinear Huber shifts quadratic/linear regime boundary)
- Fix IQN CVaR: add .contiguous() before sort_last_dim (341/341 tests pass)
- Defer loss scalar readback to after backward pass (piggyback on grad flush)
- Add next_states to ExperienceBatch with episode-aware shift computation
- Add insert_batch_tensors() for direct GPU tensor insertion into replay buffer
- Make Q-value estimation periodic (every 50 steps) to reduce forward passes
- Fix CPU fallback path types (u8 action, i32 fixed-point reward, timestamp)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ml-supervised, ml-ensemble, ml-labeling, ml-explainability, ml-hyperopt
all had default = ["cuda"] which pulled cudarc into the CPU services
build, causing compile-services to fail with "nvcc not found".
Changed all to default = [] and wired ml/Cargo.toml cuda feature to
propagate to all 8 sub-crates (was only 3: ml-core, ml-dqn, ml-ppo).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove cuda from default features in ml-core, ml-dqn, ml-ppo, ml
- Propagate cuda feature from ml → ml-core/ml-dqn/ml-ppo
- CI compile-training already uses --features ml/cuda explicitly
- Fix MaxDD log format: {:.1}% → {:.3}% (was rounding 0.033% to 0.0%)
- Suppress unused_labels/unused_variables warnings for cfg(cuda) code
- Add CALLBACK_ENDPOINT env to ml-training-service deployment
- Fix Grafana active_workers query to use sum() with fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU kernel produced raw percentage PnL rewards (~3.5e-4) which were
~10,000x smaller than Q-values (~7.0), making rewards invisible in the
Bellman backup. Q-values froze at initialization.
Port the CPU RewardNormalizer algorithm (EMA mean/variance, z-score
normalization, [-3,3] clamp) directly into the CUDA kernel hot path
as per-thread register state. Each thread maintains its own running
mean/variance with configurable decay rate (reward_norm_alpha, default
0.01 = ~100-step window). EMA resets on episode boundaries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
