Remove use_double_dqn, use_dueling, use_per, use_branching,
use_distributional, use_noisy_nets, use_huber_loss, and use_cql
from DQNConfig, DQNHyperparameters, and DqnParams structs.
These features are always enabled (Rainbow DQN standard). The boolean
flags were dead code — every constructor set them to true, and the
only code paths that set them to false were in tests that disabled
features for simplicity. With the fields removed, the features are
unconditionally active, eliminating ~490 lines of dead configuration.
Key changes:
- Struct field declarations removed from 3 core config structs
- Conditional branches (if use_X { ... } else { ... }) simplified:
dueling/branching/PER network creation is now unconditional
- Checkpoint metadata hardcodes "true" for backward compatibility
- Hyperopt search space index 11 (use_branching) fixed at 1.0
- TOML/YAML config files cleaned of removed fields
- Tests that toggled these flags updated or rewritten
45 files changed, -487 net lines. Zero new test failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split CUDA Graph (forward + adam phases with gradient injection point):
- IQN trunk gradient flows through single Adam (no dual optimizer conflict)
- Spectral norm runs BEFORE forward (not after Adam — no tug-of-war)
- σ_max in 40D hyperopt search space [1.0, 10.0]
Attention Phase B backward:
- Full gradient flow through 4-head self-attention weights
- Separate Adam optimizer for attention params
- Backward kernel recomputes forward from saved_input (memory-efficient)
Ensemble multi-head:
- Real cuBLAS value head forward per ensemble head (was copying head 0 logits)
- KL diversity gradient kernel with hierarchical reduction
- forward_value_head() on CublasForward for per-head SGEMM
Regime PER scaling:
- Kernel reads target ADX/CUSUM from states_buf directly (zero CPU readback)
- Removed 2x memcpy_dtoh per training step
Decision Transformer:
- 14 CUDA kernels (embed, causal attention, FFN, CE loss + backward + trajectory building)
- GPU-native trajectory builder (return-to-go reverse cumsum, momentum expert actions)
- Wired into training loop with dt_pretrain_epochs config
HER Future/Final:
- episode_ids flow through PER buffer (GpuBatch, GpuReplayBuffer, GpuBatchSlices)
- GPU-native donor sampling (binary search on episode boundaries)
- Strategy dispatch in fused_training.rs
Backtest SEGV fix:
- Missing q_gaps_buf argument in action_select kernel launch
- Dynamic branch_sizes from agent (not hardcoded)
Local test: objective=10.48, Sharpe=0.0419, 175K trades, zero errors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reward normalization and DSR computation have been moved to the GPU
experience-collection kernel (experience_kernels.cu). This removes the
now-dead CPU-side code:
- Remove RewardNormalizer struct (150 lines) — GPU pnl_ema/pnl_var replaces it
- Remove hold_reward, hold_penalty_weight, enable_normalization from RewardConfig
- Remove use_dsr toggle from RewardConfig and RewardConfigBuilder — DSR is always on
- Remove calculate_hold_reward() — inlined as Decimal::ZERO (GPU w_idle replaces it)
- Make RewardFunction.dsr non-optional (always enabled)
- Make training_loop.rs DSR sync + epoch reset unconditional
- Clean up constructor.rs RewardConfig construction (6 fewer fields)
- Mark DQNHyperparameters.use_dsr and hold_penalty_weight as deprecated
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GpuReplayBuffer, DQNTrainer
GpuExperienceCollector + GpuReplayBuffer: added Drop with cuStreamSync
to prevent cuMemFree racing with pending GPU work.
DQNTrainer: explicit Drop that releases GPU resources (fused_ctx,
experience collector) BEFORE cuda_stream and CudaContext drop. Without
this, the Drop order follows declaration order — GPU resources that sync
on the stream would sync on an already-destroyed stream.
hidden_dim_base from_continuous: clamp floor 256→64 to allow smoketest
and Phase Fast (hidden_dim=128) networks.
VRAM is properly released (nvidia-smi shows 0 MiB after trial).
Trial 2 CUDA_ERROR_INVALID_VALUE on bind_to_thread is a cudarc 0.19
primary context reuse issue — not VRAM related.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause of H100 training time growth (12ms→23ms/step across epochs):
pfx_sum() fell back to single-block O(n) scan when num_blocks > 1024.
With block_dim=256 and buffer_size=500K: num_blocks = 1953 > 1024 → fallback.
Fix: increase block_dim to 1024 (capped at device max_threads_per_block).
Now: num_blocks = 500000/1024 = 489 < 1024 → multi-block 3-phase scan works.
Expected impact: PER sampling cost stays constant as buffer fills instead
of growing linearly. Training step time should be stable across epochs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace single-block prefix sum (1 SM, max 1024 threads) with a 3-phase
multi-block parallel scan that distributes work across all available SMs:
Phase 1: Per-block Hillis-Steele scan (256 elements/block, all SMs active)
Phase 2: Scan block_sums in a single block (num_blocks < 1024)
Phase 3: Propagate block offsets to make results globally correct
For N=100K: 391 blocks x 256 threads vs old 1 block x 1024 threads.
Fast path preserved for N <= 256 (single-block kernel, lower overhead).
Fallback to chunked single-block for N > 256K (num_blocks > max_threads).
block_sums buffer pre-allocated in GpuReplayBuffer::new() — zero
cuMemAlloc in the hot path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminate the 3 CPU roundtrips per sample_proportional() call (called
300x/epoch), which was the #1 performance bottleneck:
1. Replace memcpy_dtoh(cs_total) with DtoD copy to pre-allocated
total_sum_buf — total_sum never leaves GPU
2. Replace CPU rand() + memcpy_htod(thresholds) with GPU-resident
Philox PRNG kernel — thresholds generated directly on device
3. Replace memcpy_htod([0.0], max_weight) with memset_zeros —
async GPU memset, zero host staging
4. is_weights_f32 kernel now reads total_sum from GPU pointer
instead of scalar argument
Pre-allocate 13 PER sampling buffers on the GpuReplayBuffer struct
(thresholds, indices, gathered data, weights, max_weight, total_sum_buf,
rng_step counter). All intermediate computation uses these pre-allocated
buffers. Output GpuBatchSlices are DtoD-cloned for ownership transfer.
Add max_batch_size field to GpuReplayBufferConfig (defaults to 1024).
Delete the cs_total() method entirely.
Net result: zero memcpy_dtoh, zero memcpy_htod, zero CPU synchronization
points in the PER sampling hot path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Change the is_weights_f32 CUDA kernel signature from taking total_sum as
a scalar f32 argument (which requires a CPU readback) to reading it from
a const float* GPU-resident pointer. This eliminates the last memcpy_dtoh
in the PER sampling hot path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminates the CPU roundtrip of memcpy_dtoh(total_sum) -> CPU rand() ->
memcpy_htod(thresholds) by generating all random thresholds directly on
GPU using Philox 4x32-10 counter-based PRNG (Salmon et al., SC 2011).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GpuTrainResult::gpu_tensor_to_cuda_slice: implement via CudaSlice::clone()
(was TODO returning hard error, blocking training guard loss readback)
- DQNAgentType::primary_dqn_mut(): new method returning &mut DQN for both
Standard and RegimeConditional variants
- FusedTrainingCtx: replace as_standard_mut() with primary_dqn_mut() at
all 3 call sites (EMA target update, IQN EMA, VarStore sync)
- fused_post_step/fused_post_step_no_ema/fused_post_step_gpu_bookkeeping:
delegate to primary_dqn_mut() instead of returning error for RegimeConditional
- train_baseline_rl: add error chain logging for fold failures
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Old implementation computed per-row stats().max independently per branch,
which gave wrong results (max of centered advantages != Q at max action).
New: max_aggregate_q = greedy_branch_actions_batch + aggregate_q_for_actions.
Semantically correct: max Q = Q at greedy actions, by definition.
359/359 ml-dqn tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gpu_portfolio.rs: deleted CPU simulate_batch() path (GPU path exists),
get_portfolio_state() returns &CudaSlice (was: download to [f32;8])
branching.rs: GPU-native argmax per branch, GPU gather+mean for Q
aggregation, GPU affine+floor for action decomposition, GPU sub→abs→
max for weight comparison in tests
gpu_tensor.rs: added pub cuda_data() accessor
stream_ops.rs: fixed CudaView type mismatch in dtod_copy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rmsnorm.rs: dedicated CUDA kernels for RMSNorm + LayerNorm forward
(was: download input+weights, CPU norm, re-upload)
residual.rs: ActivationKernels::gelu_fwd() + GPU LayerNorm kernel
(was: 6 DtoH/HtoD per forward call)
target_update.rs: ElementwiseKernels::affine(tau) + binary(add) + DtoD
(was: download ALL params to CPU for blending)
dqn.rs: ActivationKernels::leaky_relu_fwd() in Sequential::forward()
(was: download tensor, CPU branch, re-upload)
Zero memcpy_dtoh in any forward pass or weight update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
elementwise.rs: added broadcast_col_binary CUDA kernel for [N,M]*[N,1]
gpu_tensor.rs: broadcast_mul/broadcast_div now handle column broadcast
stream_ops.rs: gpu_cat_dim1 extended for 3D tensors
branching.rs: NoisyLinear weights registered in GpuVarStore
noisy_layers.rs: register_in_store() method for weight registration
distributional_dueling.rs: NoisyLinear weight registration
dqn.rs: checkpoint load wires weights into VarStore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed #[ignore] from tests that have local infrastructure:
- 3 data_loader tests: auto-detect test_data/real/databento/ via workspace
- 3 memory_profiler tests: nvidia-smi at /usr/bin/nvidia-smi
- 4 benchmark tests (TFT, Mamba2, DQN, PPO): GPU + DBN data available
- 1 inference test: model loading (slow but should run)
- 3 DQN performance smoke tests: GPU available
PPO benchmark: fixed data_path to test_data/real/databento/6E.FUT
Sequential: added vars_mut() accessor
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
elementwise.rs: added abs, neg, exp, log, le, affine CUDA kernel ops
(cases 5-10 in elementwise_unary). All GPU-native, zero host downloads.
gpu_tensor.rs: 7 new methods — abs(), neg(), exp(), log(), le(),
affine(), broadcast_sub(). All dispatch to CUDA kernels.
dqn.rs: DELETED 30+ host-side helper functions (~455 lines) that
downloaded to CPU, computed, re-uploaded. ALL replaced with direct
GpuTensor methods: affine(), clamp(), abs(), neg(), exp(), log(),
broadcast_mul(), broadcast_sub(), broadcast_as(), index_select(),
dim(), sqr(), flatten_all(), gpu_clone(), ActivationKernels.
Zero host-side math remaining in dqn.rs hot paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
branching.rs: MlDevice→Arc<CudaStream>, Candle gather/unsqueeze→host-side,
MaybeNoisyLinear::noisy_sigma_slices(), copy_weights via memcpy pattern.
regime_conditional.rs: device→stream field, all GpuTensor ops get &stream,
batch_softmax_actions via host-side Gumbel-max, gradient merge via BTreeMap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Experience collector: PinnedHostBuf<T> via cuMemHostAlloc(PORTABLE) for
6 DtoH buffers — ~25-30% faster transfers vs pageable memory.
NVRTC caching: 4 OnceLock additions in ml-ppo cuda_nn:
- softmax.rs: was re-compiling NVRTC on EVERY forward pass batch (!)
- linear.rs, lstm.rs: cached for multi-layer construction
Device attribute: OnceLock<u32> for max_threads in gpu_replay_buffer
pfx_sum — eliminates ~5µs driver query per call.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Final cleanup:
- 61 test files + 5 example files: candle imports replaced
- 8 testing/integration files: migrated to cudarc/ml-core types
- 3 services/trading_service test files: migrated
- Root Cargo.toml: candle-core, candle-nn removed from [workspace.dependencies]
- crates/ml/Cargo.toml: candle-nn dependency removed
- testing/e2e/Cargo.toml: candle-core dependency removed
Zero active candle_core/candle_nn/candle_optimisers code references remain.
Zero candle dependency declarations in any Cargo.toml.
Remaining "candle" strings are exclusively in doc comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add CudaSlice<f32> accessor methods to GpuTrainResult and GradientResult
(loss_cuda_slice, grad_norm_cuda_slice) in ml-dqn, eliminating redundant
tensor_to_cuda_slice_f32 calls at every training guard check site.
- GpuTrainResult: add from_fused_scalars() constructor and CudaSlice extractors
- GradientResult: add CudaSlice extractors that delegate to GpuTrainResult
- fused_training.rs: use from_fused_scalars() instead of Tensor::new() wrapping
- train_step.rs: use result.loss_cuda_slice() instead of tensor_to_cuda_slice_f32
- training_loop.rs: same CudaSlice accessor pattern for both fused and accum paths
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All VarBuilder/NoisyLinear init changed from BF16 to F32.
The fused CUDA trainer's Adam kernel operates on F32 master weights
with separate BF16 mirrors for forward kernels. BF16 VarBuilder
caused dtype mismatches in Candle forward paths and broke ensure_f32.
381 ml-dqn tests pass, 0 errors, 0 warnings workspace-wide.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace VarMap/VarBuilder weight storage with GpuVarStore (native CUDA)
for RMSNorm, LayerNorm, and ResidualBlock. Linear layers in ResidualBlock
now use GpuLinear with cuBLAS sgemm. Cold-path forwards convert between
GpuTensor and Candle Tensor at the boundary since downstream ops (GELU,
LayerNorm, dropout, broadcast) still use Candle.
Changes:
- ml-core cuda_autograd: add GpuTensor::to_candle/from_candle interop,
export GpuParam/AdamWConfig/ActivationKernels/LossKernels/LossResult,
make init::upload_to_gpu public
- rmsnorm.rs: RMSNorm/LayerNorm now take (Arc<CudaStream>, Device, dim)
instead of (VarBuilder, dim). Weights stored in GpuVarStore.
- residual.rs: ResidualBlock now takes (Arc<CudaStream>, Device, config, name).
fc1/fc2 are GpuLinear with cuBLAS sgemm forward. LayerNorm params in GpuVarStore.
- distributional_dueling.rs: updated RMSNorm constructor calls to new API
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VarBuilder and NoisyLinear were creating BF16 weights, then ensure_f32
tried Var::set() which rejects dtype changes. Fix: create F32 from the
start. BF16 mirrors are managed separately by GpuDqnTrainer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add dqn_forward_only_kernel to dqn_training_kernel.cu — runs a single
branching forward pass (shared layers -> value head -> 3 branch heads ->
C51 distributional -> expected Q) and writes per-branch Q-values to
q_out[B, 11]. No loss computation, no activation saves, no backward.
~3x faster than forward_loss for pure inference.
Wire the kernel into GpuDqnTrainer via forward_only_q() which takes
CudaSlice<f32> states directly and returns &CudaSlice<f32> Q-values,
replacing the Candle BranchingDuelingQNetwork::forward_branches() call
chain (~160 Candle kernel dispatches -> 1 fused CUDA launch).
Mark all 6 ml-dqn Candle forward methods as #[cold] with documentation
that hot-path compute is handled by fused CUDA kernels:
- noisy_layers.rs: NoisyLinear::forward() -> dqn_experience_kernel.cu
- quantile_regression.rs: QuantileNetwork::forward() -> iqn_dual_head_kernel.cu
- distributional.rs: CategoricalDistribution::to_scalar() -> dqn_forward_only_kernel
- curiosity.rs: ForwardDynamicsModel::predict() -> curiosity_training_kernel.cu
- residual.rs: ResidualBlock::forward() -> dqn_forward_only_kernel
- rmsnorm.rs: RMSNorm::forward() / LayerNorm::forward() -> dqn_experience_kernel.cu
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
BranchingDuelingQNetwork now converts all weight tensors to F32 contiguous
at construction via ensure_f32_contiguous(). This guarantees the invariant
that extract_one/sync_one/reverse_sync_one in gpu_weights.rs can bypass
the flatten_all().to_dtype(F32).contiguous() Candle pipeline and do a
single direct DtoD memcpy from the Var's CUDA storage.
Key changes:
- branching.rs: add ensure_f32_contiguous() + ensure_f32() for NoisyLinear
sigma/epsilon, forward_branches casts to F32 instead of BF16
- noisy_layers.rs: sample_noise/reset_noise/disable_noise use weight_mu
dtype (F32 after ensure_f32) instead of hardcoded BF16, add ensure_f32()
to convert sigma vars and epsilon buffers
- gpu_weights.rs: extract_one/sync_one fast path skips Candle ops when
tensor is already F32 contiguous, reverse_sync_one fixed to access Var's
own CUDA storage directly (was writing to a temporary F32 tensor before)
- fused_training.rs: updated doc comments for CudaSlice-as-source-of-truth
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: shmem_max_in_dim only included trunk dims (state_dim,
shared_h1, shared_h2) but not head dims (value_h, adv_h). When
hidden_dim_base=32 made the trunk narrow while heads stayed at 128,
the BF16 weight tile for branch output (255×128=32640 BF16 elements)
overflowed the shared memory region (12288 BF16 elements). On H100
the overflow landed in unused-but-mapped hardware shmem (silent
corruption). On RTX 3050 (48KB physical shmem) it hit unmapped
memory → CUDA_ERROR_ILLEGAL_ADDRESS.
Changes:
- gpu_dqn_trainer.rs: shmem_max_in_dim includes value_h/adv_h
- Remove all #[ignore] from smoke tests (feature_coverage,
training_stability, gpu_residency)
- Smoke tests use real .dbn data from test_data/ (hard error if missing)
- Remove synthetic_data() fallback — no fake data in tests
- GPU-direct DtoD training path (train_step_gpu, FusedTrainScalars)
- GPU-native PER priority update kernel (zero CPU readback)
- IQN dual-head integration (gpu_iqn_head.rs)
- BF16 dtype fixes across 6 model adapters
- Hyperopt 30D→31D (iqn_lambda)
- portfolio_transformer: unconditional BF16 (remove dead CPU branches)
- liquid/adapter: all tests use Cuda(0) directly
- Fix pre-existing gpu_kernel_parity_test.rs (stale args)
- Fix pre-existing evaluate_baseline.rs (removed fields)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After removing ensure_training_dtype(), forward methods that receive F32
inputs from tests/callers now fail with dtype mismatch against BF16 weights.
Add to_dtype(BF16) at forward entry of xLSTM (slstm, mlstm, block, network),
CfC cell, and diffusion time embedding. Fix quantization to accept BF16
tensors by casting to F32 before INT8 conversion. Update guard to catch
#[cfg(not(feature = "cuda"))] dead code. Bump DQN emergency_safe_defaults
replay_buffer_capacity from 1000 to 2048 (GPU PER floor = 1024).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Delete every CPU fallback block across 14 files (-286 lines):
- dqn.rs: CPU replay buffer, tensor construction, PER fallback, gradient paths
- hyperopt/adapters/ppo.rs: CPU curiosity modules, trajectory generation
- hyperopt/adapters/dqn.rs: CPU backtest fallback, sync no-op
- trainers/ppo.rs: CPU training error stub
- l2_cache.rs: CPU stub functions (gate callers behind cuda too)
- replay_buffer_type.rs: CPU is_gpu_prioritized fallback
- validation/harness.rs: CPU regime breakdown fallback
- build.rs: cpu_only_build cfg (never referenced)
- evaluate_baseline.rs: CPU gpu_handled fallback
- testing/integration/gpu: CPU test stubs
Zero #[cfg(not(feature = "cuda"))] remains in the codebase.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
