Replace PolicyNetwork and ValueNetwork with CudaPolicyNetwork/CudaValueNetwork
backed implementations. Each struct now stores a cuda_nn GPU-native network
(cuBLAS sgemm + CUDA kernels) alongside shadow Candle layers for autograd
training compatibility. Adds forward_cuda() inference paths that bypass Candle
entirely. Wire CudaTrajectoryTensors into TrajectoryBatch with to_cuda_tensors().
Re-export CudaLSTM from lstm_networks and all cuda_nn types from crate root.
Import GpuContext into continuous_policy, continuous_action_masking, flow_policy,
coupling_layer, and adaptive_entropy for future GPU migration. Add CudaVec::to_tensor()
bridge for cuda_nn→Candle interop. Fix upload_host_to_gpu→upload_to_gpu rename
in ml-core cuda_autograd init.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cast all Tensor::full() / Tensor::from_vec() call sites to training_dtype
instead of defaulting to F32. Fixes dtype mismatch errors (BF16 vs F32)
in PPO training on CUDA:
- tensor_ops: scalar_mul, clamp, normalize match operand dtype
- trajectories: TrajectoryBatch/MiniBatch to_tensors cast to training dtype
- continuous_ppo: ContinuousTrajectoryBatch/MiniBatch to_tensors cast
- adaptive_entropy: cast entropy to F32 for alpha multiplication boundary
- continuous_policy: forward() input cast, Tensor::full scalars match dtype
- flow_policy: sample_base_noise cast to training dtype
- hidden_state_manager: reset tensors use training_dtype
- ensemble/ppo adapter: predict input cast to training dtype
- trainable_adapter: test uses training_dtype instead of hardcoded F32
Verified: 198/198 ml-ppo tests pass, 63/63 ml PPO tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Cast input to weight dtype in DQN residual, rmsnorm, noisy_layers
- Set use_gpu=true in QNetworkConfig defaults and all config sites
- Resolve BF16 boundary mismatches in attention, curiosity, branching,
distributional_dueling across ml-dqn
- GPU-resident regime ops with BF16 boundary casts, eliminate .expect() in CUDA paths
- Eliminate all Device::Cpu fallbacks — GPU-only across 10 ML crates
- PPO: cast logits to F32 before softmax, cast batch tensors to training dtype
- Gradient collapse detection for RegimeConditionalDQN
- Wire halt_grad_collapse from CUDA guard kernel to halt training
- Dead neuron detection uses active network VarMap + squeeze factored readback
- Increment gradient_logging_step in GPU PER path
- Gradient collapse warmup guards use original buffer_size
- Cap training steps per epoch + tracing migration
- Replace Tensor::all() with sum_all() for pinned Candle compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace closure-based evaluate() with evaluate_dqn_graphed() for non-OFI
walk-forward backtest path. Extracts DuelingWeightSet from VarMap (branching
or standard dueling) and runs hand-written warp-cooperative CUDA forward
kernel with CUDA Graph capture — zero Candle dispatch overhead per step.
Key changes:
- GpuBacktestEvaluator::stream() getter for weight extraction on eval stream
- DQNAgentType::is_using_branching() / network_dims() for CUDA kernel config
- Hyperopt evaluate_gpu() non-OFI path: extract_dueling_weights_branching()
→ evaluate_dqn_graphed() (CUDA Graph accelerated)
- OFI path: retains Candle closure for state permutation (gather kernel
layout mismatch — future CUDA permutation kernel)
- 66+ GPU hot-path violations hardened to hard errors across DQN/PPO/supervised
- Stripped all gpu-ok suppression comments
- Proper #[cfg(feature = "cuda")] gating for CUDA-only code paths
77 files, 0 errors, 0 warnings across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminate all CPU fallback paths in the PPO trainer, PPO hyperopt
adapter, and DQN hyperopt adapter. On H100, every GPU operation must
succeed or abort — silent CPU fallback runs at 1/10th throughput and
produces stale/inconsistent results.
PPO trainer (5 sites):
- GPU unavailable → hard error (was warn + CPU fallback)
- Collector init, episode reset, collection, weight sync → hard errors
- Test updated: GPU batch limit test expects error without CUDA
PPO hyperopt adapter (8 sites):
- ensure_gpu_data() now returns Result<(), MLError>
- Features/targets upload, collector init, weight sync → hard errors
- gpu_collect_trajectories() now returns Result<TrajectoryBatch>
- Experience collection caller uses ? instead of Option fallback
- GPU backtest failure → hard error
DQN hyperopt adapter (3 sites):
- CUDA synchronize between trials → hard error
- GPU backtest None on CUDA → hard error (upstream of extract_objective)
- extract_objective: panic! on CUDA build if backtest_metrics is None
- CPU backtest path guarded by #[cfg(not(feature = "cuda"))]
continuous_ppo.rs (1 site):
- clip_grads passed &Device::Cpu instead of actor's actual device
- Forces CPU roundtrip for gradient norm computation on every mini-batch
- Fixed: passes `device` (from self.actor.device()) — stays GPU-resident
gpu_experience_collector.rs (1 site):
- cuCtxSetLimit(STACK_SIZE) failure: warn → hard error
- 16KB stack is required — kernel segfaults without it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restore 45-action factored space via Branching DQN (Tavakoli 2018),
outputting 11 Q-values (5+3+3) instead of 45. This was reduced to 5
exposure-only actions during debugging and was never intended as permanent.
- Enable use_branching: true by default in DQNConfig and DQNHyperparameters
- Add branching paths to select_action_with_confidence and select_action_inference
- Update agent.rs select_action_factored for branching-aware selection
- Expand CountBonus to per-branch tracking with bonuses_branched()
- Add order_type + urgency distribution tracking in monitoring
- Add DQN_ORDER_ACTIONS=3, DQN_URGENCY_ACTIONS=3, DQN_TOTAL_ACTIONS=45 to CUDA header
- Fix 7 pre-existing clippy doc_markdown errors in regime_conditional.rs
- Fix pre-existing cognitive_complexity in replay_buffer_type.rs (extract helpers)
- Fix flaky GPU test OOM under parallel execution (CPU fallback + test VRAM safety)
- Delete unused flash_attention submodules (block_sparse, causal_masking, etc.)
- Add GPU hot-path guard scripts and ensemble/hyperopt adapter improvements
Tests: ml-dqn 416/0, ml 905/0, clippy 0 errors on both crates
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On H100 with BF16 mixed precision enabled, Candle's comparison ops (gt,
lt, le, ge) and arithmetic ops require matching dtypes. F32 threshold
constants compared against BF16 state tensors caused every training step
to fail with "dtype mismatch in cmp, lhs: BF16, rhs: F32".
Fixes:
- regime_conditional.rs: ADX/CUSUM thresholds cast to adx.dtype()
- quantile_regression.rs: Huber kappa, zero, and half tensors match
input dtype (both quantile_huber_loss variants)
- rainbow_network.rs: LeakyReLU slope and ELU alpha/one tensors cast
to x.dtype()
- continuous_ppo.rs: Huber delta/half tensors cast to abs_diff.dtype()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused struct fields, dead methods, unreachable code across
ml-dqn, ml-supervised, ml-features, ml-ppo, ml-checkpoint, ml-labeling,
ml-observability, and ml-universe. Gate test-only infra behind #[cfg(test)].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 24 PPO source files + flow_policy/ from ml into standalone
ml-ppo crate. The ml crate's ppo module is now a thin re-export layer
(`pub use ml_ppo::*`) plus two bridge files (trainable_adapter.rs,
stress_testing.rs) that depend on ml-internal types.
Key changes:
- ml-ppo: 25 modules (incl flow_policy subdir), 198 tests, standalone
- ml: depends on ml-ppo, re-exports via ppo/mod.rs
- Import rewrites: crate::common::action → ml_core::action_space,
crate::dqn::{mixed_precision,xavier_init} → ml_core::*,
crate::gradient_accumulation → ml_core::gradient_accumulation
- ml tests: 1929 pass (down from 2127 — 198 moved to ml-ppo)
- Workspace: 0 errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
