The GPU PER replay buffer had a hardcoded 4 GB MAX_BYTES limit that
rejected the auto-sizer's 10M-entry proposal on H100 (80 GB VRAM).
Now per_max_buffer_bytes() computes 20% of total VRAM (min 1 GB) and
flows through OptimalReplayConfig → DQNConfig → GpuReplayBufferConfig
so both subsystems agree on the budget.
Also fixes misleading regime detection log (indices 211/219 → 40/41)
and renames dqn_config_2025 → dqn_default_config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminates cudaStreamSynchronize between epochs by keeping vol EMA,
portfolio state, and DSR normalizer in persistent CudaSlice buffers.
Kernel reads initial state at launch, writes final state at exit.
- Add epoch_state CudaSlice<f32>[8] field and reset_flags u32 bitfield
to GpuExperienceCollector struct
- Allocate epoch_state in new() with sensible defaults (vol_ema=0.01,
initial_capital for portfolio, dsr_var=1.0 to avoid div-by-zero)
- Pass epoch_state and reset_flags as final args to both kernel variants
(standard per-thread and warp-cooperative)
- Kernel: thread/lane 0 of block 0 applies reset flags atomically with
__threadfence, all threads read 8-float epoch state from L1-cached
global memory, last block writes back updated values at exit
- Auto-clear reset_flags after each launch (one-shot semantics)
- Add set_reset_flags(), clear_reset_flags(), epoch_state_gpu(),
rewards_gpu(), actions_gpu() public methods
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: `option_env!("FOXHUNT_BUILD_VERSION")` in common/build_info.rs
was a compile-time macro tracked by cargo fingerprints (Rust 1.80+). Every
pipeline run with a new tag invalidated `common` → cascading rebuild of all
38 dependent workspace crates, even when zero source files changed.
Fix: replace `option_env!()` with `std::env::var()` (runtime LazyLock). Cargo
no longer tracks the version env var, so `common` only recompiles when its
source actually changes.
Also: skip git checkout when HEAD already matches target SHA (zero mtime
changes), and drop `-x` from git clean to preserve gitignored files.
Expected: ~3.7min → <1min for unchanged-crate rebuilds on warm PVC.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes for GPU-accelerated Branching DQN training:
1. **GPU experience collector**: NoisyLinear creates standalone Vars via
Var::from_tensor(), bypassing VarMap registration. The GPU collector
looks up weights by name ("value_fc.weight") from VarMap and falls
back to CPU (~5x slower) when missing. Fix: register mu vars in
VarMap at construction, keep sigma vars standalone.
2. **Optimizer device mismatch**: Using only vars().all_vars() left
NoisyLinear head params frozen. backward() produces gradients the
optimizer doesn't know about → device mismatch in clip_grad_norm.
Fix: all_trainable_vars() = VarMap (shared+mu) + sigma.
3. **Single-threaded CPU bottleneck**: Runtime::new() creates a
current-thread scheduler → 1 OS thread → all async work serialized.
Fix: multi-thread runtime (4 workers) created once in DQNTrainer::new(),
shared across preload/training/backtest phases. Eliminates 3 fallback
Runtime::new() callsites.
Also: polyak_update_var_pairs with debug_assert_eq, two-phase target
network sync (VarMap Polyak + sigma var_pairs Polyak), copy_weights_from
handles NoisyLinear heads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three optimizations targeting GPU allocation churn and lock contention:
1. In-place polyak update: Var::set() reuses existing GPU buffer instead
of Var::from_tensor() which allocates a new one per call. Eliminates
~10,000 cudaMalloc/cudaFree per epoch (20 params × 500 steps).
2. Fused affine ops: Replace 13 Tensor::full()/Tensor::ones() constant
tensor allocations per step with tensor.affine(mul, add) — a single
fused kernel. Patterns: 1-x → x.affine(-1,1), γ*x → x.affine(γ,0),
0.5*x² → (x*x).affine(0.5,0). Applied across all 4 loss paths
(branching Bellman/Huber, standard Bellman/Huber, IQN). Eliminates
~6,500 GPU allocs/epoch.
3. Batch pre-sampling (K=8): Sample 8 batches under one READ lock, train
all 8 under one WRITE lock. Reduces async lock acquisitions from
2×N to 2×ceil(N/8). Priority staleness across 8 steps is negligible.
Combined estimated impact: 20-35% H100 throughput improvement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Candle's Tensor::cumsum(0) internally allocates an [n,n] upper-triangular
matrix (9.3 GB for n=50K) causing OOM on GPUs ≤48 GB. Replace with a
block-parallel Hillis-Steele scan kernel that is O(n) in time and memory.
Additional fixes in this commit:
- Break autograd chain leak in loss/grad accumulation via .detach()
(was leaking ~32 MB/step across entire training run)
- Release features_raw_cuda/targets_raw_cuda after GPU experience
collection (~164 MB VRAM reclaimed)
- Use softmax eval (temp=0.3) in walk-forward backtest to prevent
action collapse causing trades=0 on early-stage models
Validated: 1286 tests pass (408 ml-dqn + 878 ml), 0 failures.
VRAM stable at 754 MB across 45K+ training steps on RTX 3050 Ti.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents experience collector output tensors from starving backward
graph and GPU PER on VRAM-constrained GPUs like RTX 3050 Ti 4GB.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
free_memory_mb is sampled after model/optimizer load, so subtract only
backward-graph + fragmentation + data upload headroom (400 MB), not the
full model/PER/data stack. Increase collector share to 50%.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Staging/GPU replay buffer: truncate batch when it exceeds ring buffer
capacity (CPU fallback can flush 100K+ experiences at once)
- GpuExperienceCollector: compute shmem tile rows dynamically to stay
under 48 KB default limit instead of hardcoded 64-row constant
- Auto batch size: subtract concurrent VRAM consumers (~530 MB model +
optimizer + PER + data) before budgeting experience collector at 40%
- Hyperopt DQN adapter: GPU PER always on, include replay buffer in
VRAM estimate for small GPUs (was excluded assuming CPU-only PER)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the cfg gate inside the method body instead of gating the entire
function. Without cuda, the GpuPrioritized variant doesn't exist so
matches! returns false — no need for separate cfg blocks at call sites.
Fixes compile-services CI failure (services build without --features cuda).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN hyperopt adapter: remove static VRAM gate for GPU experience
collector and GPU PER — dynamic scaling handles constraints at
runtime with graceful CPU fallback on init failure
- Remove is_parquet_file branching from hyperopt eval path (all data
loads via DBN pipeline now)
- Update training example CLI args for consistency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate all per-step GPU→CPU synchronization barriers from the
training guard. Replace device+host buffer pairs and memcpy_dtoh
with cuMemHostAlloc(DEVICEMAP) mapped pinned memory.
Key changes:
- MappedBuffer struct: cuMemHostAlloc + cuMemHostGetDevicePointer_v2
allocates memory visible to both CPU and GPU simultaneously
- Double-buffering: kernel writes to buffer[N%2], CPU reads buffer
[(N-1)%2] — one-step delayed halt detection, zero sync
- __threadfence_system() in CUDA kernels ensures writes visible to
CPU across PCIe without explicit memcpy
- read_volatile on host pointer prevents CPU-side caching
Eliminated:
- check_and_accumulate: 28-byte memcpy_dtoh (every training step)
- qvalue_stats: 16-byte memcpy_dtoh (every 50 steps)
- qvalue_divergence: 20-byte memcpy_dtoh (every 50 steps)
Kept: read_accumulators memcpy_dtoh (12 bytes, epoch boundary only —
accumulator buffer stays in device memory for kernel read-modify-write).
1286 tests pass (878 ml + 408 ml-dqn), 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two remaining GPU→CPU synchronization barriers in the CUDA-active
training loop:
1. detect_dead_neurons() → to_vec0() called every training step from
log_diagnostics() in the guard path. Added check_gradient_collapse()
method that performs the same collapse detection logic but skips the
expensive per-parameter weight scan. Dead neuron detection now only
runs at epoch boundary via log_diagnostics().
2. forward() Q-value clipping monitoring → to_vec2() called every 1000
steps during compute_loss_internal() and Q-value estimation forward
passes. Added training_forward_active flag that gates the monitoring
block; set to true during all training-path forward() calls
(compute_loss_internal + Q-value estimation), false during
inference/evaluation.
All 1577 tests pass (878 ml + 408 ml-dqn + 291 ml-core), 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire route_exposure_to_factored() into both select_actions_batch_gpu
and select_actions_batch GPU paths, eliminating per-item CPU routing.
Previously, exposure indices (0-4) were downloaded from GPU and routed
to factored indices (0-44) one-by-one on CPU via route_action(). Now
the exposure→factored mapping runs entirely on GPU via the routing
kernel, with a single batch readback of the final factored indices.
Branching DQN path unchanged (already produces factored indices 0-44).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace CPU-bound Q-value monitoring with GPU-resident qvalue_stats and
qvalue_divergence kernels from GpuTrainingGuard. The two readback sites
(estimate_avg_q_value_with_early_stopping's mean_all().to_scalar() and
log_q_values' to_vec2()) are now bypassed when the GPU training guard is
active. CPU fallback path preserved for non-CUDA builds and guard-absent
scenarios.
Changes:
- Add DQN::log_q_values_from_stats() accepting pre-computed GPU stats
- Add DQNAgentType::log_q_values_from_stats() delegate
- Replace Q-estimation in train_step_single_batch with GPU kernel path
- Replace Q-estimation in train_step_with_accumulation with GPU kernel path
- Import IndexOp trait for Tensor::i() in trainer.rs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a standalone CUDA kernel that converts DQN exposure indices (0-4)
to factored action indices (0-44) entirely on GPU, reusing the existing
route_order() device function from common_device_functions.cuh. Wired
into GpuActionSelector as route_exposure_to_factored() method, following
the same DtoD copy pattern as the existing select_actions methods. This
eliminates a GPU->CPU->GPU roundtrip when post-hoc routing is needed
after epsilon_greedy_select.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the GPU->CPU sync barrier (to_vec1 readback) in the DQN training
hot path with the GpuTrainingGuard CUDA kernel that performs NaN detection,
loss clipping, and gradient collapse checks entirely on-device. The guard
is lazy-initialized on first training step and falls back to the original
CPU readback path if CUDA kernel compilation fails.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Seven pure-Rust reference tests covering all four training_guard_kernel.cu
kernels: PTX NVRTC smoke-test (skips on CPU-only), NaN/Inf detection,
loss clip, gradient collapse, accumulator averaging, GuardResult boolean
threshold construction, and qvalue_stats_reduce per-sample max-Q reduction.
Zero clippy warnings in the new file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wraps 4 CUDA kernels (training_guard_check, training_guard_accumulate,
qvalue_stats_reduce, qvalue_divergence_check) with a Rust struct that
uses OnceLock PTX caching, pre-allocated device buffers, and host-side
Vec mirrors for zero-allocation readbacks per training step.
Accumulator (3-float acc_buf) stays on-device for epoch-boundary
averaging without CPU roundtrips.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add use_dsr, dsr_eta, and n_steps fields to ExperienceCollectorConfig
with safe defaults (DSR off, eta=0.01, n_steps=1). Pass them as kernel
args after fill_simulation_enabled, matching the parameter order added
in Tasks 4 and 5. Wire from DqnHyperparams in the trainer hot path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirror the DSR (Differential Sharpe Ratio) and n-step return
accumulation added by Task 4 to the scalar kernel into the warp
kernel (dqn_full_experience_kernel_warp). DSR/n-step state runs
on lane 0 only, matching the existing reward computation pattern.
All accumulators are reset on episode boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Differential Sharpe Ratio reward shaping and n-step return
accumulation to dqn_full_experience_kernel (scalar path). DSR replaces
EMA normalization when use_dsr=1; n-step ring buffer accumulates
discounted returns for effective_n>1. Both are reset on episode
boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Append third CUDA kernel to epsilon_greedy_kernel.cu for Branching DQN:
3 independent epsilon-greedy heads (exposure/5, order/3, urgency/3)
compose to factored action index 0-44 (exposure*9 + order*3 + urgency).
Add Rust composition tests verifying full 45-action coverage and
round-trip decomposition correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add backticks to type names in doc comments (doc_markdown)
- Mark eligible functions as const fn (missing_const_for_fn)
No behavior changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
module_name_repetitions: PpoConfig in ppo module is conventional ML naming.
Renaming breaks every import across the workspace.
integer_division: Basis point calculations, batch size math, combinatorial
formulas — truncation is intentional. Float conversion would introduce bugs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
is_gpu_prioritized() and grad_norm squeeze are only available with CUDA.
Without the gate, compile-services (CPU-only) fails with E0599.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-step to_scalar()/to_vec1() readbacks with GPU-resident
tensor accumulation and single epoch-boundary sync. On H100 this
removes ~12-15 pipeline flushes per training step (~5μs each),
enabling full GPU saturation with zero CPU sync in the inner loop.
Key changes:
- GpuTrainResult: train_step() returns GPU scalar tensors (loss_gpu,
grad_norm_gpu) instead of f32 — zero readback per step
- Regime conditional: train all 3 heads unconditionally with
zero-masked weights (mathematical no-op) instead of 3-6
to_scalar() mask count checks per step
- GPU PER: max_priority as GPU tensor with flush_max_priority(),
delta-based priority update via index_add (no CPU dedup)
- Deferred diagnostics: NaN detection, CQL logging, Q-value
estimation all moved to epoch boundary where pipeline is
already synced
- Legacy CPU-readback wrappers removed entirely
9 files changed, +393/-295 lines. 520 DQN tests passing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate GPU→CPU→GPU roundtrip in the experience collection hot path.
Kernel outputs (states, rewards, actions, dones) now stay on GPU via
device-to-device copy into candle Tensors. Only rewards + actions are
downloaded for monitoring metrics (~0.5MB vs ~3.14GB at 32K episodes).
- Extract launch_kernel() helper from collect_experiences() (DRY)
- Add collect_experiences_gpu() — DtoD path for GPU PER
- Add cuda_slice_to_tensor_f32/i32_to_u32 DtoD copy utilities
- GPU next_states via tensor narrow/cat/reshape (no CPU loop)
- Trainer selects GPU vs CPU path based on is_gpu_prioritized()
- stream.synchronize() barrier ensures cross-stream data visibility
0 warnings, 1266 tests pass (874 ml + 392 ml-dqn)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Forward-port from worktree-gpu-hotpath-audit (2 commits):
1. Zero-roundtrip DQN experience collection via cuMemcpyDtoDAsync:
- GpuExperienceCollector uses device-to-device copies for state tensors
- Shared memory weight caching in GPU replay buffer
- NaN priority clamping on GPU (no CPU readback)
2. Eliminate all GPU→CPU roundtrips from training loop:
- GpuTrainResult: loss + grad_norm stay as GPU scalar tensors
- Single to_scalar() readback at epoch boundary (not per step)
- GPU-resident loss accumulation across training steps
- RegimeConditional head selection via GPU tensor ops
- Removed per-step NaN diagnostic checks (now epoch-level)
Also removes dead `states_tensor` field from ComputeLossResult and
fixes redundant field name clippy warning.
Expected: ~15-20% training throughput improvement on H100 by
eliminating synchronous GPU→CPU transfers in the inner loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes temporary benchmark comment. Docker changes from previous commit
will trigger image rebuilds. This run serves as warm-cache benchmark
after cold-cache run completes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Temporary comment to trigger detect-changes for both services and
training compile steps. Will be removed after benchmarking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
