The B0 audit (commit 62ab8ed85) under-counted insert_batch test
callers as 2 (1 production + 1 in-file unit test). Surfaced during
B1.0 implementation when cargo check --workspace --tests failed
with 5 arity-mismatch errors after B0's signature change.
Root cause: B0 audit's grep filter was `grep -v test` and didn't
enumerate crates/ml/src/trainers/dqn/smoke_tests/ (compiled as
part of the lib's test binary, not behind #[cfg(test)]) nor
crates/ml/tests/.
Sites fixed (zero-init i32 alloc, threaded through):
- crates/ml/src/trainers/dqn/smoke_tests/training_stability.rs:152, 196
- crates/ml/src/trainers/dqn/smoke_tests/performance.rs:142
- crates/ml/src/trainers/dqn/smoke_tests/gpu_residency.rs:75
- crates/ml/tests/gpu_per_integration_test.rs:125
No behavior change — the column carries zero data and no consumer
reads it pre-B1.1. B1.1 lands the producer kernel that fills with
-1/0/1 from the 30-bar price trajectory.
Process correction documented in docs/dqn-wire-up-audit.md
"B0.1 cascade-gap fix-up" subsection: future B-series audits must
run cargo check --workspace --tests before claiming cardinality
completeness.
Build: cargo check --workspace --tests clean.
Tests: cargo test -p ml --lib compiles + passes (GPU tests
#[ignore]-gated).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add to_varstore() compatibility shims on DuelingQNetwork and
DistributionalDuelingQNetwork so test/example code can rebuild a
GpuVarStore snapshot when needed. Delete dead tests that referenced
removed DQNAgent, PrioritizedReplayBuffer, and ReplayBufferType.
Fix action index references (action_19/21 -> action_28/30) and
type annotation issues (sin ambiguity, remainder operator).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete bf16 elimination across all crates (ml, ml-core, ml-dqn, ml-ppo,
ml-supervised). Zero half::bf16, __nv_bfloat16, or CudaSlice<half::bf16>
references remain (verified by grep).
CUDA: All 60+ .cu kernels and .cuh headers converted to native float.
- Half-precision intrinsics (__hmul, __hadd, __hdiv) → float operators
- atomicAddBF16 → native atomicAdd(float)
- bf16 wrapper functions → f32 identity passthroughs
Rust: All CudaSlice<half::bf16> → CudaSlice<f32> across 90+ files.
- htod_f32_to_bf16/dtoh_bf16_to_f32 → htod_f32/dtoh_f32 (direct, no conversion)
- Deleted bf16 mirror infrastructure (DuelingWeightSetBf16, alloc_bf16_mirror, etc.)
- Renamed params_bf16→params_flat, d_value_logits_bf16→d_value_logits, etc.
- Fixed .to_f32() sed damage on Decimal::to_f32() and rng.f32()
FxCache: Single f32 disk format (was bf16/f64 dual-version).
- Deleted --bf16 CLI flag from precompute_features
- PVC cache files need regeneration via precompute_features
TF32 tensor cores activated via cublasLtMatmul CUBLAS_COMPUTE_32F — no
explicit TF32 types needed. Storage is pure f32 everywhere.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The replay buffer used its own CUDA stream for seg_tree_update, but
the td_errors buffer was computed on the trainer's stream. Without
synchronization, the replay stream could read incomplete data — causing
a hang on H100 where streams execute truly concurrently.
Fix: update_priorities_gpu now accepts an optional ext_stream parameter.
The fused training path passes the trainer's main stream, ensuring all
GPU work (td_errors computation + seg_tree_update) runs on the same
stream with implicit ordering.
Also pre-allocate update_td_f32, update_batch_max, update_max_merge
scratch buffers at construction — eliminates 3 cuMemAlloc calls per
training step that could trigger implicit device synchronization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pre-allocated sampling buffers are max_batch_size (1024) elements large, but kernels
only write the first batch_size elements. sample_weights_ref() and sample_indices_ref()
returned the full 1024-element CudaSlice, causing:
- weight[16] == 0 (uninitialized elements beyond batch_size)
- assert_eq!(indices_host.len(), 10) failing with 1024
Fix: track last_batch_size in GpuReplayBuffer and return CudaView<'_, T> sliced to
last_batch_size from the two ref methods. Update all callers to pass &view to
memcpy_dtoh (CudaView implements DevicePtr so no other API changes needed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New test coverage for 3 previously untested GPU training paths:
- hyperopt.rs: test_hyperopt_preloaded_data, test_hyperopt_shared_data,
test_hyperopt_paths_consistent — exercises train_with_preloaded_data and
train_with_shared_data which bypass walk-forward (direct train_with_data_full_loop)
- walk_forward.rs: test_walk_forward_multi_fold — tight fractions (0.3/0.1/0.1/0.1)
to guarantee 2+ folds, validating reset_for_fold, graph_aux invalidation, and
CUDA graph survival across fold boundaries
- regression.rs: test_no_hang_single_epoch (VRAM oversubscription guard),
test_counterfactual_experiences_in_buffer (silent data loss guard),
test_gpu_n_episodes_config_honored (auto-scaling removal guard)
Also fixes: unclosed for-loop brace in gpu_per_integration_test.rs (pre-existing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GpuBatch now stores raw u64 device pointers to pre-allocated replay
buffer memory. Eliminates per-step:
- 14 cuMemAlloc calls (7 in sample_proportional + 7 in into_gpu_batch)
- 14 DtoD copies (clone into owned CudaSlice)
- CPU staging Vec and flush() dead code path
Also removed: GpuBatchSlices, into_gpu_batch, dtod_clone_* helpers,
StagedGpuBuffer.staging field, CPU add/add_batch for GpuPrioritized.
update_priorities_cuda now takes u64 raw pointer.
HER relabel_batch_with_strategy takes u64 episode_ids_ptr.
Cold-path Q-value estimation uses compute_q_stats_internal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 test files had stale types from the bf16→f32 conversion:
- gpu_smoketest: missing adam_epsilon in DQNConfig
- gpu_backtest_validation: closure params bf16→f32
- gpu_kernel_parity_test: market data, weight readback bf16→f32
- gpu_per_integration_test: weights readback bf16→f32
- target_update_tests: varstore register bf16→f32
- smoke_test_real_data: market buffers bf16→f32
- activation_tests, dropout_scheduler_tests: forward() signature change
These tests only compile with --features cuda (CI path), which is why
they passed locally with cargo test --lib.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminate the 3 CPU roundtrips per sample_proportional() call (called
300x/epoch), which was the #1 performance bottleneck:
1. Replace memcpy_dtoh(cs_total) with DtoD copy to pre-allocated
total_sum_buf — total_sum never leaves GPU
2. Replace CPU rand() + memcpy_htod(thresholds) with GPU-resident
Philox PRNG kernel — thresholds generated directly on device
3. Replace memcpy_htod([0.0], max_weight) with memset_zeros —
async GPU memset, zero host staging
4. is_weights_f32 kernel now reads total_sum from GPU pointer
instead of scalar argument
Pre-allocate 13 PER sampling buffers on the GpuReplayBuffer struct
(thresholds, indices, gathered data, weights, max_weight, total_sum_buf,
rng_step counter). All intermediate computation uses these pre-allocated
buffers. Output GpuBatchSlices are DtoD-cloned for ownership transfer.
Add max_batch_size field to GpuReplayBufferConfig (defaults to 1024).
Delete the cs_total() method entirely.
Net result: zero memcpy_dtoh, zero memcpy_htod, zero CPU synchronization
points in the PER sampling hot path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Final cleanup:
- 61 test files + 5 example files: candle imports replaced
- 8 testing/integration files: migrated to cudarc/ml-core types
- 3 services/trading_service test files: migrated
- Root Cargo.toml: candle-core, candle-nn removed from [workspace.dependencies]
- crates/ml/Cargo.toml: candle-nn dependency removed
- testing/e2e/Cargo.toml: candle-core dependency removed
Zero active candle_core/candle_nn/candle_optimisers code references remain.
Zero candle dependency declarations in any Cargo.toml.
Remaining "candle" strings are exclusively in doc comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update all callers to match the new pure-cudarc APIs introduced by the
hive agent CudaSlice migration. Key changes:
- GpuTrainingGuard::new() now takes Arc<CudaStream>; callers use from_device()
- check_and_accumulate/qvalue_stats/qvalue_divergence take &CudaSlice<f32>
instead of &Tensor; callers convert via tensor_to_cuda_slice_f32()
- accumulate_q_value takes f32 scalar, returns () (no Result)
- GpuReplayBuffer::insert_batch gains batch_size arg, takes CudaSlice params
- signal_adapter functions take &Arc<CudaStream> (cudarc 0.17 Arc requirement)
- Add tensor_to_cuda_slice_u32() and cuda_f32_to_tensor() utility functions
- Replace CudaView usage with owned CudaSlice via tensor_to_cuda_slice_f32()
- Fix CudaStorage.device field access (was method call in older API)
- Fix borrow-after-move in copy_actions_out via scoped DtoD copy
Zero errors, zero warnings across lib + tests + examples + full workspace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU PER replay buffer had a hardcoded 4 GB MAX_BYTES limit that
rejected the auto-sizer's 10M-entry proposal on H100 (80 GB VRAM).
Now per_max_buffer_bytes() computes 20% of total VRAM (min 1 GB) and
flows through OptimalReplayConfig → DQNConfig → GpuReplayBufferConfig
so both subsystems agree on the budget.
Also fixes misleading regime detection log (indices 211/219 → 40/41)
and renames dqn_config_2025 → dqn_default_config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand FeatureVector from 40 to 42 dimensions by including ADX(14) at
index 40 and CUSUM direction at index 41 from the existing CPU feature
extraction pipeline. This eliminates proxy-based regime classification
and enables GPU-native regime detection via tensor narrow/comparison ops.
Key changes:
- extraction.rs: wire RegimeADXFeatures + RegimeCUSUMFeatures into
extract_current_features_v2(), output 42 features per bar
- regime_conditional.rs: classify_regime_masks_gpu() creates per-regime
mask tensors entirely on GPU (ADX > 0.25 = trending, |CUSUM| > 0.7 =
volatile, else ranging). Zero CPU roundtrip in training hot path.
- trainer.rs/config.rs: state_dim 43→45 (no OFI), 51→53 (with OFI),
aligned dims unchanged (48/56). GPU batch insertion for all 3 heads.
- CUDA header: MARKET_DIM 40→42
- walk_forward.rs: FEATURE_DIM 40→42
- 42 files updated, all [f64;40]→[f64;42] propagated across workspace
Test results: ml=874/0, ml-dqn=354/0, ml-features=282/0, ml-core=274/0
Real data GPU smoke tests: 7/7 passed (OHLCV + OFI + trade enrichment)
Hyperopt baseline RL: 2 trials completed on local RTX 3050 Ti
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5 tests: training loop cycle, priority-weighted sampling, KS distribution
test, IS weight range validation, ring buffer overwrite correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
