Pre-allocated sampling buffers are max_batch_size (1024) elements large, but kernels
only write the first batch_size elements. sample_weights_ref() and sample_indices_ref()
returned the full 1024-element CudaSlice, causing:
- weight[16] == 0 (uninitialized elements beyond batch_size)
- assert_eq!(indices_host.len(), 10) failing with 1024
Fix: track last_batch_size in GpuReplayBuffer and return CudaView<'_, T> sliced to
last_batch_size from the two ref methods. Update all callers to pass &view to
memcpy_dtoh (CudaView implements DevicePtr so no other API changes needed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New test coverage for 3 previously untested GPU training paths:
- hyperopt.rs: test_hyperopt_preloaded_data, test_hyperopt_shared_data,
test_hyperopt_paths_consistent — exercises train_with_preloaded_data and
train_with_shared_data which bypass walk-forward (direct train_with_data_full_loop)
- walk_forward.rs: test_walk_forward_multi_fold — tight fractions (0.3/0.1/0.1/0.1)
to guarantee 2+ folds, validating reset_for_fold, graph_aux invalidation, and
CUDA graph survival across fold boundaries
- regression.rs: test_no_hang_single_epoch (VRAM oversubscription guard),
test_counterfactual_experiences_in_buffer (silent data loss guard),
test_gpu_n_episodes_config_honored (auto-scaling removal guard)
Also fixes: unclosed for-loop brace in gpu_per_integration_test.rs (pre-existing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GpuBatch now stores raw u64 device pointers to pre-allocated replay
buffer memory. Eliminates per-step:
- 14 cuMemAlloc calls (7 in sample_proportional + 7 in into_gpu_batch)
- 14 DtoD copies (clone into owned CudaSlice)
- CPU staging Vec and flush() dead code path
Also removed: GpuBatchSlices, into_gpu_batch, dtod_clone_* helpers,
StagedGpuBuffer.staging field, CPU add/add_batch for GpuPrioritized.
update_priorities_cuda now takes u64 raw pointer.
HER relabel_batch_with_strategy takes u64 episode_ids_ptr.
Cold-path Q-value estimation uses compute_q_stats_internal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 test files had stale types from the bf16→f32 conversion:
- gpu_smoketest: missing adam_epsilon in DQNConfig
- gpu_backtest_validation: closure params bf16→f32
- gpu_kernel_parity_test: market data, weight readback bf16→f32
- gpu_per_integration_test: weights readback bf16→f32
- target_update_tests: varstore register bf16→f32
- smoke_test_real_data: market buffers bf16→f32
- activation_tests, dropout_scheduler_tests: forward() signature change
These tests only compile with --features cuda (CI path), which is why
they passed locally with cargo test --lib.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminate the 3 CPU roundtrips per sample_proportional() call (called
300x/epoch), which was the #1 performance bottleneck:
1. Replace memcpy_dtoh(cs_total) with DtoD copy to pre-allocated
total_sum_buf — total_sum never leaves GPU
2. Replace CPU rand() + memcpy_htod(thresholds) with GPU-resident
Philox PRNG kernel — thresholds generated directly on device
3. Replace memcpy_htod([0.0], max_weight) with memset_zeros —
async GPU memset, zero host staging
4. is_weights_f32 kernel now reads total_sum from GPU pointer
instead of scalar argument
Pre-allocate 13 PER sampling buffers on the GpuReplayBuffer struct
(thresholds, indices, gathered data, weights, max_weight, total_sum_buf,
rng_step counter). All intermediate computation uses these pre-allocated
buffers. Output GpuBatchSlices are DtoD-cloned for ownership transfer.
Add max_batch_size field to GpuReplayBufferConfig (defaults to 1024).
Delete the cs_total() method entirely.
Net result: zero memcpy_dtoh, zero memcpy_htod, zero CPU synchronization
points in the PER sampling hot path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Final cleanup:
- 61 test files + 5 example files: candle imports replaced
- 8 testing/integration files: migrated to cudarc/ml-core types
- 3 services/trading_service test files: migrated
- Root Cargo.toml: candle-core, candle-nn removed from [workspace.dependencies]
- crates/ml/Cargo.toml: candle-nn dependency removed
- testing/e2e/Cargo.toml: candle-core dependency removed
Zero active candle_core/candle_nn/candle_optimisers code references remain.
Zero candle dependency declarations in any Cargo.toml.
Remaining "candle" strings are exclusively in doc comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update all callers to match the new pure-cudarc APIs introduced by the
hive agent CudaSlice migration. Key changes:
- GpuTrainingGuard::new() now takes Arc<CudaStream>; callers use from_device()
- check_and_accumulate/qvalue_stats/qvalue_divergence take &CudaSlice<f32>
instead of &Tensor; callers convert via tensor_to_cuda_slice_f32()
- accumulate_q_value takes f32 scalar, returns () (no Result)
- GpuReplayBuffer::insert_batch gains batch_size arg, takes CudaSlice params
- signal_adapter functions take &Arc<CudaStream> (cudarc 0.17 Arc requirement)
- Add tensor_to_cuda_slice_u32() and cuda_f32_to_tensor() utility functions
- Replace CudaView usage with owned CudaSlice via tensor_to_cuda_slice_f32()
- Fix CudaStorage.device field access (was method call in older API)
- Fix borrow-after-move in copy_actions_out via scoped DtoD copy
Zero errors, zero warnings across lib + tests + examples + full workspace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU PER replay buffer had a hardcoded 4 GB MAX_BYTES limit that
rejected the auto-sizer's 10M-entry proposal on H100 (80 GB VRAM).
Now per_max_buffer_bytes() computes 20% of total VRAM (min 1 GB) and
flows through OptimalReplayConfig → DQNConfig → GpuReplayBufferConfig
so both subsystems agree on the budget.
Also fixes misleading regime detection log (indices 211/219 → 40/41)
and renames dqn_config_2025 → dqn_default_config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand FeatureVector from 40 to 42 dimensions by including ADX(14) at
index 40 and CUSUM direction at index 41 from the existing CPU feature
extraction pipeline. This eliminates proxy-based regime classification
and enables GPU-native regime detection via tensor narrow/comparison ops.
Key changes:
- extraction.rs: wire RegimeADXFeatures + RegimeCUSUMFeatures into
extract_current_features_v2(), output 42 features per bar
- regime_conditional.rs: classify_regime_masks_gpu() creates per-regime
mask tensors entirely on GPU (ADX > 0.25 = trending, |CUSUM| > 0.7 =
volatile, else ranging). Zero CPU roundtrip in training hot path.
- trainer.rs/config.rs: state_dim 43→45 (no OFI), 51→53 (with OFI),
aligned dims unchanged (48/56). GPU batch insertion for all 3 heads.
- CUDA header: MARKET_DIM 40→42
- walk_forward.rs: FEATURE_DIM 40→42
- 42 files updated, all [f64;40]→[f64;42] propagated across workspace
Test results: ml=874/0, ml-dqn=354/0, ml-features=282/0, ml-core=274/0
Real data GPU smoke tests: 7/7 passed (OHLCV + OFI + trade enrichment)
Hyperopt baseline RL: 2 trials completed on local RTX 3050 Ti
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5 tests: training loop cycle, priority-weighted sampling, KS distribution
test, IS weight range validation, ring buffer overwrite correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
