Commit Graph

4015 Commits

Author SHA1 Message Date
jgrusewski
da2aff98c6 feat(cuda): add CudaPolicyNetwork and CudaValueNetwork to ml-ppo cuda_nn
GPU-native MLP networks that use CudaLinear layers + cuBLAS sgemm
internally. These are drop-in replacements for the Candle-based
PolicyNetwork and ValueNetwork, providing the same forward/softmax/
log_softmax API surface but with zero Candle overhead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:35:52 +01:00
jgrusewski
4ef9cdb169 refactor(cuda): replace Candle re-export paths with direct cudarc in ml-core
CUDA infrastructure in ml-core (cuda_compile, gpu/capabilities, gpu/l2_cache)
previously accessed cudarc types through candle_core::cuda_backend::cudarc --
a transitive re-export that coupled low-level CUDA driver calls to Candle.

Changes:
- Add cudarc 0.17 as direct optional dep (gated behind cuda feature)
- Replace all candle_core::cuda_backend::cudarc paths with direct cudarc::
- Generalize OOM detection to accept &dyn Debug (was &CandleError)
- Add generic computation_error_to_common_error() alongside legacy alias

Candle references: 163 -> 72 (remaining are autograd-dependent: Tensor,
Var, GradStore, VarBuilder -- cannot be replaced with cudarc raw ops)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:34:58 +01:00
jgrusewski
39feb10f13 feat(cuda): add cuda_nn module to ml-ppo — raw cudarc/cuBLAS neural network primitives
Introduces the cuda_nn module as the foundation for eliminating Candle from
the ml-ppo crate. All primitives store weights as CudaSlice<f32> and use
cuBLAS sgemm for matrix multiplication, custom CUDA kernels for activations.

Components:
- GpuContext: shared cuBLAS handle + CUDA stream wrapper
- CudaLinear: fully connected layer via cuBLAS sgemm + bias-add kernel
- CudaLSTM: fused gate LSTM cell via cuBLAS + gate nonlinearity kernel
- CudaAdam: GPU-resident Adam optimizer (m/v state on GPU, single kernel per step)
- cuda_softmax/cuda_log_softmax: warp-shuffle numerically stable kernels
- cuda_relu/cuda_tanh/cuda_sigmoid: element-wise activation kernels
- CudaVec: GPU buffer wrapper with known length

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:34:07 +01:00
jgrusewski
a29f109b67 fix(cuda): resolve 35 caller-boundary type mismatches after CudaSlice migration
Update all callers to match the new pure-cudarc APIs introduced by the
hive agent CudaSlice migration. Key changes:

- GpuTrainingGuard::new() now takes Arc<CudaStream>; callers use from_device()
- check_and_accumulate/qvalue_stats/qvalue_divergence take &CudaSlice<f32>
  instead of &Tensor; callers convert via tensor_to_cuda_slice_f32()
- accumulate_q_value takes f32 scalar, returns () (no Result)
- GpuReplayBuffer::insert_batch gains batch_size arg, takes CudaSlice params
- signal_adapter functions take &Arc<CudaStream> (cudarc 0.17 Arc requirement)
- Add tensor_to_cuda_slice_u32() and cuda_f32_to_tensor() utility functions
- Replace CudaView usage with owned CudaSlice via tensor_to_cuda_slice_f32()
- Fix CudaStorage.device field access (was method call in older API)
- Fix borrow-after-move in copy_actions_out via scoped DtoD copy

Zero errors, zero warnings across lib + tests + examples + full workspace.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:05:31 +01:00
jgrusewski
bc75cdd7c9 refactor(cuda): enforce F32 contiguous weights, eliminate Candle intermediaries from VarMap extraction
BranchingDuelingQNetwork now converts all weight tensors to F32 contiguous
at construction via ensure_f32_contiguous(). This guarantees the invariant
that extract_one/sync_one/reverse_sync_one in gpu_weights.rs can bypass
the flatten_all().to_dtype(F32).contiguous() Candle pipeline and do a
single direct DtoD memcpy from the Var's CUDA storage.

Key changes:
- branching.rs: add ensure_f32_contiguous() + ensure_f32() for NoisyLinear
  sigma/epsilon, forward_branches casts to F32 instead of BF16
- noisy_layers.rs: sample_noise/reset_noise/disable_noise use weight_mu
  dtype (F32 after ensure_f32) instead of hardcoded BF16, add ensure_f32()
  to convert sigma vars and epsilon buffers
- gpu_weights.rs: extract_one/sync_one fast path skips Candle ops when
  tensor is already F32 contiguous, reverse_sync_one fixed to access Var's
  own CUDA storage directly (was writing to a temporary F32 tensor before)
- fused_training.rs: updated doc comments for CudaSlice-as-source-of-truth

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:36:13 +01:00
jgrusewski
4047ee1961 refactor(cuda): replace Candle Tensor internals with cudarc CudaSlice in GpuReplayBuffer
Eliminate all ~103 Candle Tensor references from GpuReplayBuffer internals.
Ring buffer storage, insert, cumsum, searchsorted, gather, IS-weight
computation, and priority update now run as 16 custom CUDA kernels via
pure cudarc CudaSlice operations -- zero Candle dispatch overhead.

Key changes:
- GpuReplayBuffer stores CudaSlice<u16/u32/f32> + Arc<CudaStream>
- New replay_buffer_kernels.cu with 14 entry points (scatter insert,
  gather, pow_alpha, IS weights, priority update, reduce_max, etc.)
- Removed SearchSorted CustomOp2 and CumsumOp CustomOp1 wrappers
  (direct kernel launch via compiled CudaFunction handles)
- StagedGpuBuffer::flush uses CudaSlice HtoD + insert_batch
- Output GpuBatch preserves Candle Tensor fields for downstream
  neural network compatibility (DtoD wrap at output boundary)
- Added half crate dependency for BF16 CudaSlice<->Tensor wrapping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:32:33 +01:00
jgrusewski
f68c324c30 refactor(cuda): replace Candle Tensor with cudarc CudaSlice in gpu_experience_collector
Eliminate ALL Candle Tensor usage from gpu_experience_collector.rs to fix
cross-stream deadlocks. The old code created Candle Tensors on the forked
CudaStream, which conflicted with Candle's default stream (stream 0) used
by GpuReplayBuffer::insert_batch.

Changes to gpu_experience_collector.rs:
- GpuExperienceBatch now holds CudaSlice<f32>/CudaSlice<i32> instead of Tensor
- Remove cuda_slice_to_tensor_f32() and cuda_slice_i32_to_tensor_u32() helpers
- Add dtod_clone_f32(), dtod_clone_i32() pure-cudarc DtoD copy helpers
- Add build_next_states_dtod() for episode-aware state shift via pointer math
- Remove device: &Device parameter from collect_experiences_gpu()
- Remove candle_core::{DType, Device, Tensor} import entirely

Changes to training_loop.rs:
- Move CudaSlice->Tensor conversion to training_loop boundary (post stream-sync)
- Add cuda_slice_to_tensor_f32() and cuda_slice_i32_to_tensor_u32() at boundary
- Conversions happen AFTER stream.synchronize(), on the synced stream -- no
  cross-stream hazard

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:27:03 +01:00
jgrusewski
9f0bf84443 refactor(cuda): update DQN action callers to use pure CudaSlice API
action.rs:
- Replace Tensor-based selector calls with CudaSlice extraction via
  extract_cuda_f32! macro + ensure_contiguous_f32() helper
- Constructor now passes Arc<CudaStream> via stream_from_device()
- Replace per-element narrow+squeeze+to_scalar readback with bulk
  readback_actions() DtoH memcpy (single transfer vs N scalar reads)
- select_actions_branching() now takes explicit batch_size parameter

metrics.rs:
- Same CudaSlice extraction pattern for validation action selection
- Wrap CudaSlice<u32> result back to Candle Tensor via cuda_u32_to_tensor()
  for downstream GPU PnL/Sharpe computation that needs Tensor ops

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:19:43 +01:00
jgrusewski
81513983d8 refactor(cuda): eliminate Candle Tensor from gpu_iql_trainer and gpu_training_guard
gpu_iql_trainer.rs:
- Remove Tensor-based train_value_step(), compute_advantages(),
  advantage_weights(), forward_only(), and tensor_to_cuda_slice_f32()
- Rename train_value_step_raw() to train_value_step() (now the only version)
- Remove candle_core::{DType, Tensor} import (only cudarc re-export remains)

gpu_training_guard.rs:
- Replace Device field with Arc<CudaStream> — no Candle Device stored
- Constructor takes Arc<CudaStream>; from_device() adapter for callers
- check_and_accumulate() takes &CudaSlice<f32> instead of &Tensor
- qvalue_stats() and qvalue_divergence() take &CudaSlice<f32>
- accumulate_q_value() takes f32 scalar (CPU-side Welford, no GPU tensor)
- read_q_accumulator() and reset_q_accumulator() simplified (no Tensor)

cuda_pipeline/mod.rs:
- Add tensor_to_cuda_slice_f32() shared utility for callers at the boundary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:17:53 +01:00
jgrusewski
4127828d65 refactor(cuda): replace Candle Tensor with cudarc CudaSlice in signal_adapter
Eliminate all 26 Candle Tensor references from signal_adapter.rs.
Three functions (ppo_to_exposure_scores, signal_to_action_scores,
tft_quantile_to_signal) now take CudaSlice<f32> + CudaStream and
return CudaSlice<f32>, backed by three fused CUDA kernels in
signal_adapter_kernel.cu. Dead code evaluate_supervised_gpu_backtest
removed (zero callers). Added cuda_f32_to_tensor helper to
gpu_action_selector for DtoD copy back to Candle Tensor at API
boundaries (PPO hyperopt adapter, evaluate_baseline example).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 13:16:58 +01:00
jgrusewski
3db5e86db9 fix(cuda): synchronize forked stream before cross-stream replay buffer ops
GpuReplayBuffer uses Candle Tensor ops (slice_scatter, cumsum, narrow)
which execute on Candle's default stream (stream 0). The experience
collector outputs data on a forked CudaStream. Without an explicit
synchronization barrier, the default stream may read partially-written
data from the forked stream, causing cross-stream deadlocks or data
corruption.

Add stream.synchronize() in three places:
- training_loop.rs: after experience collection, before insert_batch_tensors
- gpu_experience_collector.rs: cuda_slice_to_tensor_f32 after DtoD copy
- gpu_experience_collector.rs: cuda_slice_i32_to_tensor_u32 after DtoD copy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:54:25 +01:00
jgrusewski
f79739fa51 fix(cuda): remove disable_event_tracking entirely
With unified forked stream, cudarc event tracking is self-referencing
(no cross-stream conflicts). disable_event_tracking caused more
problems than it solved — stale events, Drop failures, hangs.
Let cudarc manage events naturally on the single stream.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:37:09 +01:00
jgrusewski
2196cdc60c fix(cuda): disable event tracking at stream fork, not at trainer init
Moving disable_event_tracking() from GpuDqnTrainer::new() to the
stream fork in constructor.rs ensures ALL CudaSlice allocations
(experience collector, replay buffer, curiosity trainer) are created
with tracking already disabled. Previous placement caused hangs because
earlier allocations had stale events.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:23:34 +01:00
jgrusewski
b39d10c968 fix(cuda): remove Candle from GPU training hot path
- upload_batch_gpu: BF16→F32 via CUDA kernel, no Candle to_dtype()
- IQL: reuses DQN trainer's F32 buffers via train_value_step_raw()
- Re-enable disable_event_tracking() — safe with unified stream
- Reduce smoke test epochs to 1 (debug mode is too slow for 3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:07:59 +01:00
jgrusewski
3b4ba67fc5 perf(cuda): eliminate Candle VarMap/Tensor from DQN GPU training hot path
Replace all Candle tensor operations in the fused training loop with
pure cudarc CudaSlice operations to prevent Candle tensor Drops from
recording events on the default stream (which conflicts with our forked
training stream via cudarc event tracking).

Key changes:
- Re-enable disable_event_tracking() in GpuDqnTrainer::new() — safe now
  that no Candle tensors are created during training
- Rewrite upload_batch_gpu(): BF16 states/next_states use DtoD + bf16→f32
  kernel instead of Candle to_dtype(F32); F32 rewards/dones/weights use
  direct DtoD copy with layout offset handling
- Load bf16_to_f32_kernel from training module (was defined in CUH but
  not loaded)
- Add train_value_step_raw() to GpuIqlTrainer that takes CudaSlice<f32>
  directly, bypassing Candle tensor manipulation
- IQL in fused training now reuses DQN trainer's already-converted F32
  states_buf/rewards_buf instead of creating Candle temporaries
- Fix dtod_from_candle_f32/u32 to respect Candle layout start_offset

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 11:33:14 +01:00
jgrusewski
44ddd38112 fix(cuda): unify all GPU ops onto single forked CudaStream
Fork a dedicated CudaStream once in DQNTrainer constructor and share it
via Arc::clone with all GPU components (experience collector, fused
trainer, portfolio sim, monitoring). This eliminates:

1. Candle's default stream (stream 0) from the training hot path
2. Dual-stream cudarc event tracking conflicts during CUDA Graph capture
3. Implicit serialization points between default and forked streams

The fused training context no longer forks its own stream — it receives
the trainer's unified stream. With all GPU work on a single stream,
cudarc event tracking is safely disabled in GpuDqnTrainer::new(),
removing the cuStreamWaitEvent/cuEventRecord overhead that broke CUDA
Graph capture.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 10:42:56 +01:00
jgrusewski
fa739ba40c wip(cuda): identify dual-stream root cause, TODO for stream unification
The fused DQN trainer uses a forked stream (for CUDA Graph capture)
while the experience collector + curiosity trainer use Candle's
default stream (stream 0). cudarc's context-wide event tracking
creates conflicts between the two streams.

disable_event_tracking() is context-wide — it fixes the forked
stream but breaks the default stream (curiosity trainer hangs).

Proper fix: create ONE forked stream at DQNTrainer construction,
pass it to ALL GPU components (experience collector, curiosity
trainer, portfolio sim, fused trainer). No Candle default stream
in the hot path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 10:17:59 +01:00
jgrusewski
475a22b2d6 refactor(test): single production-config smoke test, all features always on
Replace 13 per-feature toggle tests with 1 production-config test.
There is no code path without curiosity, IQN, branching, CQL, Kelly,
action masking, circuit breaker — these are always on in production.

smoke_params() now enables everything: Rainbow DQN + branching + IQN +
curiosity + CQL + Kelly + action masking + circuit breaker.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 10:02:55 +01:00
jgrusewski
d792abdfbe fix(test): switch smoke_test_real_data to BranchingDuelingQNetwork
DuelingQNetwork uses advantage_fc.* VarMap keys, but
GpuExperienceCollector expects branch_0_fc.* (branching always on).
Also remove stale use_noisy_nets/use_distributional fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 09:41:49 +01:00
jgrusewski
e0dced887f fix(test): resolve test_data_dir from CI PVC layout (ohlcv/ES.FUT)
CI PVC structure is /data/test-data/ohlcv/ES.FUT, local is
test_data/ES.FUT. test_data_dir() now searches both layouts
and checks both FOXHUNT_TEST_DATA and TEST_DATA_DIR env vars.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 09:36:07 +01:00
jgrusewski
472dffd0f3 fix(cuda): shmem tile overflow + cudarc event tracking → 2 CUDA errors
Root cause 1 — CUDA_ERROR_ILLEGAL_ADDRESS:
shmem_max_in_dim only included trunk dims (state_dim, shared_h1,
shared_h2) but not head dims (value_h, adv_h). BF16 weight tile
for branch output overflowed shared memory on RTX 3050 (48KB).

Root cause 2 — CUDA_ERROR_INVALID_VALUE on EMA kernel:
cudarc 0.17's automatic event tracking records read/write events on
CudaSlice buffers. During CUDA Graph capture (events disabled) then
replay (events re-enabled), stale write events from CudaSlice Drops
poison the context error_state. Next bind_to_thread() propagates it.
Fix: disable_event_tracking() at GpuDqnTrainer construction —
single-owner forked stream, all sync points are explicit.

Also:
- Remove all #[ignore] from smoke tests, use real ES.FUT .dbn data
- Validate DBN schema at file level (skip non-OHLCV)
- Organize test_data/ into per-symbol subdirectories
- Fix pre-existing gpu_kernel_parity_test + evaluate_baseline errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 09:18:49 +01:00
jgrusewski
492fdc025e fix(data): skip non-OHLCV record types in DBN loader instead of hard error
MBP-10 .dbn files mixed into test_data/ caused OHLCV loader to crash
on unknown RType 0x42. Now silently skips non-OHLCV records.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 08:38:38 +01:00
jgrusewski
ad28482a93 fix(cuda): shmem tile overflow → CUDA_ERROR_ILLEGAL_ADDRESS on RTX 3050
Root cause: shmem_max_in_dim only included trunk dims (state_dim,
shared_h1, shared_h2) but not head dims (value_h, adv_h). When
hidden_dim_base=32 made the trunk narrow while heads stayed at 128,
the BF16 weight tile for branch output (255×128=32640 BF16 elements)
overflowed the shared memory region (12288 BF16 elements). On H100
the overflow landed in unused-but-mapped hardware shmem (silent
corruption). On RTX 3050 (48KB physical shmem) it hit unmapped
memory → CUDA_ERROR_ILLEGAL_ADDRESS.

Changes:
- gpu_dqn_trainer.rs: shmem_max_in_dim includes value_h/adv_h
- Remove all #[ignore] from smoke tests (feature_coverage,
  training_stability, gpu_residency)
- Smoke tests use real .dbn data from test_data/ (hard error if missing)
- Remove synthetic_data() fallback — no fake data in tests
- GPU-direct DtoD training path (train_step_gpu, FusedTrainScalars)
- GPU-native PER priority update kernel (zero CPU readback)
- IQN dual-head integration (gpu_iqn_head.rs)
- BF16 dtype fixes across 6 model adapters
- Hyperopt 30D→31D (iqn_lambda)
- portfolio_transformer: unconditional BF16 (remove dead CPU branches)
- liquid/adapter: all tests use Cuda(0) directly
- Fix pre-existing gpu_kernel_parity_test.rs (stale args)
- Fix pre-existing evaluate_baseline.rs (removed fields)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 08:34:51 +01:00
jgrusewski
12b9eb2436 refactor(cuda): remove dead non-branching code paths
- Delete BranchingWeightSet::zeros() — branching always enabled,
  placeholder buffers never needed
- gpu_experience_collector: remove use_branching field/param, always
  extract branching weights, always use branching-aware sync
- training_loop: remove use_branching guard from network selection
- gpu_weights test: fix dtype assertion F32→BF16

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:08:55 +01:00
jgrusewski
2ea96c4c00 fix(cuda): restore graph capture, fix SCRATCH1_DIM OOB, fix BF16 grad norm
- CUDA training kernel: SCRATCH1_DIM = max(SHARED_H1, VALUE_H + ADV_H)
  fixes register array OOB in dqn_forward_loss_kernel distributional path
- gpu_dqn_trainer: restore CUDA graph capture/replay (was bypassed for
  debug), remove 4 debug stream.synchronize() that serialized GPU pipeline
- gpu_dqn_trainer: disable/enable cudarc event tracking around graph
  capture to prevent cross-stream event references from invalidating capture
- gpu_dqn_trainer: opt-in to >48KB dynamic shared memory when needed
- fused_training + gpu_backtest_evaluator: fork() dedicated stream for
  graph capture (legacy stream 0 does not support begin_capture)
- gpu_weights: update struct comments from scalar to distributional C51
  shapes, keep BranchingWeightSet::zeros() scalar-sized for experience
  collector placeholder buffers
- 5 supervised adapters (tgnn/tlob/kan/xlstm/diffusion): cast BF16 grad
  norm tensors to F32 before to_scalar::<f32>() extraction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 22:45:08 +01:00
jgrusewski
450c23a6d0 refactor(cuda): eliminate all CPU fallbacks — CUDA mandatory across ML stack
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
  training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator

Zero warnings, zero errors across entire workspace.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 21:01:28 +01:00
jgrusewski
979f135271 fix(cuda): add BF16 input casts to forward methods after mixed_precision removal
After removing ensure_training_dtype(), forward methods that receive F32
inputs from tests/callers now fail with dtype mismatch against BF16 weights.
Add to_dtype(BF16) at forward entry of xLSTM (slstm, mlstm, block, network),
CfC cell, and diffusion time embedding. Fix quantization to accept BF16
tensors by casting to F32 before INT8 conversion. Update guard to catch
#[cfg(not(feature = "cuda"))] dead code. Bump DQN emergency_safe_defaults
replay_buffer_capacity from 1000 to 2048 (GPU PER floor = 1024).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:46:43 +01:00
jgrusewski
c5961cb766 refactor(cuda): remove all #[cfg(not(feature = "cuda"))] dead CPU paths
Delete every CPU fallback block across 14 files (-286 lines):
- dqn.rs: CPU replay buffer, tensor construction, PER fallback, gradient paths
- hyperopt/adapters/ppo.rs: CPU curiosity modules, trajectory generation
- hyperopt/adapters/dqn.rs: CPU backtest fallback, sync no-op
- trainers/ppo.rs: CPU training error stub
- l2_cache.rs: CPU stub functions (gate callers behind cuda too)
- replay_buffer_type.rs: CPU is_gpu_prioritized fallback
- validation/harness.rs: CPU regime breakdown fallback
- build.rs: cpu_only_build cfg (never referenced)
- evaluate_baseline.rs: CPU gpu_handled fallback
- testing/integration/gpu: CPU test stubs

Zero #[cfg(not(feature = "cuda"))] remains in the codebase.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:26:41 +01:00
jgrusewski
d95e205d4b refactor(ml): delete mixed_precision module — BF16 unconditional on CUDA
Eliminate the entire mixed_precision runtime indirection layer:
- Delete crates/ml-core/src/mixed_precision.rs (training_dtype, ensure_training_dtype, align_dim_for_tensor_cores)
- Inline ~100 call sites across 130 files to constants:
  training_dtype(&device) → candle_core::DType::BF16
  ensure_training_dtype(x) → x.to_dtype(candle_core::DType::BF16)
  align_dim_for_tensor_cores(x, &device) → (x + 7) & !7
- Remove re-exports from ml-dqn, ml-supervised, ml lib.rs
- Clean config/toml/json/shell references

No CPU/Metal training path exists — BF16 is the only dtype.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:11:48 +01:00
jgrusewski
cd54a6f27d fix(cuda): resolve 2h H100 test timeout — AutoReplaySizer inflation + CPU PER fallback chain
AutoReplaySizer inflated smoke test buffer_size=500 to 10M on H100 (75GB VRAM),
causing GPU PER empty-sample failures and multi-hour hangs. Root cause: the sizer's
100K minimum was always above the test buffer size, so the guard never triggered.

Fixes:
- constructor.rs: skip auto-sizing when buffer_size < 100K (sizer's own minimum)
- config.rs: insert_batch_tensors falls back to CPU PER (download + add_batch)
  when GPU PER unavailable, instead of hard error
- train_step.rs: skip fused CUDA Graph init when GPU PER not active (stream
  capture can't include CPU→GPU transfers from CPU PER sampling)
- dqn.rs: allow CPU tensor construction path on CUDA when GPU PER falls back;
  remove hard errors in PER priority update (2 locations)
- metrics.rs: fix portfolio tensor shape (broadcast_left→expand for 2D concat);
  add CPU PER fallback for Q-value statistics sampling
- agent.rs, entropy_regularization.rs: GPU-native Gumbel-max via Tensor::rand
  (eliminates per-call CPU Vec allocation + GPU upload)
- smoke test helpers: defense-in-depth replay_buffer_vram_fraction = 0.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 14:11:13 +01:00
jgrusewski
1fef9c401c fix(cuda): force decimal point in V_MIN/V_MAX defines to avoid UDL parse error
nvcc (C++11 mode) parses `25f` as an attempt to use user-defined literal
suffix `f` instead of the standard float suffix. When V_MIN/V_MAX are
injected as integer-formatted values (e.g., `-25f` from Rust's Display
trait on f32), nvcc fails with "user-defined literal operator not found".

Fix: use `{v_min:.1}f` format to guarantee a decimal point (`-25.0f`),
which is unambiguously a float literal in all C++ standards.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 10:39:42 +01:00
jgrusewski
a2ef8f0871 fix(cuda): guard DQN-specific q_forward_dueling_warp_shmem behind #if defined()
Non-DQN kernels (curiosity, PPO, epsilon-greedy, backtest-PPO) include
common_device_functions.cuh but don't define SHARED_H1/SHARED_H2/VALUE_H/ADV_H.
The unguarded q_forward_dueling_warp_shmem() references these constants,
causing nvcc compilation failures (undefined identifiers + cascading
"user-defined literal operator not found" errors).

Wraps the function in #if defined(SHARED_H1) && defined(SHARED_H2) &&
defined(VALUE_H) && defined(ADV_H) ... #endif so it's only compiled
when the DQN architecture constants are injected.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 09:59:32 +01:00
jgrusewski
8e0bcbcad3 fix(cuda): forward-declare warp_reduce_sum_all before BF16 matvec
The BF16 matvec helpers (warp_matvec_bf16_shmem, warp_matvec_bf16_broadcast_shmem)
call warp_reduce_sum_all which is defined later in the header. nvcc on H100
rejects this without a forward declaration — broke experience kernel compilation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 09:38:53 +01:00
jgrusewski
2357337bbd fix(infra): move Argo logs to dedicated argo-logs bucket
Argo artifact logs were stored in foxhunt-training-results which is
meant for model checkpoints. Separate bucket makes logs findable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 09:33:11 +01:00
jgrusewski
c84c51434e fix(tests): add missing TradingState import to DQN trainer tests
The test module used `super::*` but TradingState lives in
crate::dqn, not in the trainer module — broke `cargo test --lib`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 09:25:52 +01:00
jgrusewski
216db0301d fix(gpu): eliminate all GPU→CPU roundtrip violations — zero guard findings
Replace .to_vec1()/.to_vec2() bulk downloads with GPU-resident ops:
- PPO/DQN action selection: Gumbel-max trick (categorical on GPU)
- Scalar readbacks: .to_scalar() instead of .to_vec1()[0]
- GPU stats: abs().max(), sqr().sum_all() — single scalar out
- NaN/Inf check: sum_all().to_scalar().is_finite()
- Guard exclusions: inference output boundaries + CPU fallback with GPU path

26 files across ml-ppo, ml-dqn, ml-supervised, ml (ensemble adapters, metrics, data_loading)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 00:19:09 +01:00
jgrusewski
77715209f6 chore: delete dead demo_dqn.rs, update GPU hot-path guard
- Remove ml-dqn/src/demo_dqn.rs (134 lines, unused)
- Guard: add new hot-path patterns, tighten leak detection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:50:54 +01:00
jgrusewski
3ac51679a1 feat(cuda): fused DQN training kernel + trainer split
Replace 7k-line monolithic trainer.rs with modular trainer/ directory:
  action.rs, constructor.rs, metrics.rs, mod.rs, state.rs,
  tests.rs, training_loop.rs, train_step.rs (6048 lines total)

New fused CUDA training pipeline:
  - dqn_training_kernel.cu: single-kernel forward+loss+backward
  - gpu_dqn_trainer.rs: host-side fused training orchestration
  - fused_training.rs: Rust-side fused training integration

Eliminates per-step CPU↔GPU synchronization in DQN training loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:50:29 +01:00
jgrusewski
6b3a00a0ac feat(cuda): pipeline improvements — double buffer, weights, collectors
- common_device_functions.cuh: extended shared header for fused ops
- double_buffer.rs: async double-buffered GPU memory transfers
- gpu_weights.rs: unified weight management for fused training
- gpu_experience_collector.rs: streamlined GPU experience collection
- gpu_ppo_collector.rs: PPO collector GPU path improvements
- gpu_curiosity_trainer.rs + kernel: curiosity training on GPU
- dqn_experience_kernel.cu: experience kernel updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:49:54 +01:00
jgrusewski
0274f63a2f fix(gpu): delete dead CPU fallback code from GPU hot paths
Remove ~850 lines of unreachable CPU fallback code from 3 hot-path files:

- hyperopt/dqn.rs: delete 250-line CPU backtest path (GPU eval mandatory)
- evaluate_baseline.rs: delete CPU PPO eval + non-CUDA fallback (147 lines)
- trainers/ppo.rs: delete 6 dead CPU rollout methods (~370 lines),
  split train() into dispatcher + #[cfg(feature = "cuda")] train_gpu()

All .to_vec1() GPU hot-path guard violations eliminated.
GPU failures are now hard errors, not silent CPU fallbacks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:48:12 +01:00
jgrusewski
9d564a829e fix(guard): restore #[cfg(test)] exclusion, keep cfg(not(cuda)) filter
Unit tests need scalar readbacks for assertions — .to_scalar() is not
flagged but .to_vec1() in test modules would generate false positives.
Guard now correctly excludes inline #[cfg(test)] modules while checking
all production code including ensemble adapters.

Full --all scan reveals 31 pre-existing production violations across
ml-dqn, ml-ppo, ml-supervised, and ensemble adapters — separate cleanup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:11:06 +01:00
jgrusewski
39e5dafc62 fix(cuda): harden GPU hot-path guard — exclude tests, remove false-positive patterns
- Guard: exclude #[cfg(test)] modules (tests need scalar readbacks for assertions)
- Guard: exclude #[cfg(not(feature = "cuda"))] guarded expressions (dead code with CUDA)
- Guard: remove Tensor::from_vec/from_slice from leak patterns (CPU→GPU is correct direction)
- Guard: remove .to_scalar from leak patterns (single 4-byte readback, not bulk transfer)
- dqn.rs: rewrite log_q_values() to use GPU tensor ops (min/max/mean/var), eliminate to_vec1
- dqn.rs: rewrite clip monitoring to use GPU tensor ops, individual .to_scalar() readbacks
- ppo.rs: replace stacked .to_vec1() metrics readback with individual .to_scalar() calls
- evaluate_baseline.rs: single-bar DQN action from .to_vec1::<u32>() to .to_scalar::<u32>()
- mod.rs: remove gpu_upload_vec/gpu_upload_slice wrappers (guard no longer flags from_vec)
- Delete dead demo_dqn.rs (zero callers, stub returning mock results)

Remaining: 4 .to_vec1() violations across 3 files — porting to existing CUDA implementations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 23:04:46 +01:00
jgrusewski
fee3c858ae fix(cuda): move q_forward_dueling_warp_shmem to common header
The backtest_forward_kernel.cu calls q_forward_dueling_warp_shmem but
it was defined in dqn_experience_kernel.cu — a different compilation unit.
Move the function, TILE_LAYER_WARP_CLEAN macro, SHMEM_MIN and DIST_SIZE
to common_device_functions.cuh so both kernels can use them.

Add #ifndef guards to all macros to prevent redefinition warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 15:56:23 +01:00
jgrusewski
8ea400e599 fix(cuda): fix TMA inline asm syntax for CUDA 12.9 PTX ISA 8.7
The CI toolkit (CUDA 12.9, PTX ISA 8.7) requires:
1. cp.async.bulk needs .mbarrier::complete_tx::bytes completion mechanism
   (mandatory since PTX ISA 8.3 / CUDA 12.3)
2. mbarrier.try_wait.parity.acquire needs .cta scope qualifier between
   .acquire and .shared::cta

Reverts the DISABLE_TMA workaround — TMA now compiles natively to cubin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 15:10:10 +01:00
jgrusewski
8c4861fb70 fix(cuda): always define DISABLE_TMA — CI toolkit cannot assemble cp.async.bulk
The CI CUDA toolkit's ptxas fails on TMA instructions (cp.async.bulk)
with "completion_mechanism modifier required". Prepend #define DISABLE_TMA 1
in compile_ptx_for_device() so ALL kernel compilations (experience collector,
backtest evaluator, PPO collector, curiosity trainer, action selector) use
the float4 cooperative load fallback instead of TMA.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 15:03:52 +01:00
jgrusewski
8c4657a3a4 fix(infra): resolve commit-sha=HEAD to remote branch tip, not local PVC HEAD
When commit-sha defaults to "HEAD", git checkout HEAD is a no-op — it keeps
the PVC's stale checkout. Now resolves HEAD to origin/{branch} after fetch,
ensuring the latest pushed code is always compiled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 14:49:55 +01:00
jgrusewski
76fdd155a1 fix(cuda): compile to native cubin instead of PTX — eliminate driver JIT entirely
nvcc now produces cubin (native SASS) with -cubin -arch=sm_XX instead of
PTX with -ptx -arch=compute_XX. cuModuleLoad() loads SASS directly — zero
driver JIT. TMA instructions (cp.async.bulk on sm_90+) work natively because
nvcc handles them during offline compilation. Removed TMA fallback from
experience collector (double-compile + 15s retry overhead eliminated).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 14:32:53 +01:00
jgrusewski
e807558cd8 fix(infra): remove --initial-capital from train-best step
train_baseline_rl doesn't accept --initial-capital (only hyperopt and
evaluate binaries do). Removing it fixes the exit code 2 in DQN
hyperopt v3 train-best step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 13:50:46 +01:00
jgrusewski
686f180d7b fix(ppo): pure BF16 dtype alignment across all PPO networks and tensor ops
Cast all Tensor::full() / Tensor::from_vec() call sites to training_dtype
instead of defaulting to F32. Fixes dtype mismatch errors (BF16 vs F32)
in PPO training on CUDA:

- tensor_ops: scalar_mul, clamp, normalize match operand dtype
- trajectories: TrajectoryBatch/MiniBatch to_tensors cast to training dtype
- continuous_ppo: ContinuousTrajectoryBatch/MiniBatch to_tensors cast
- adaptive_entropy: cast entropy to F32 for alpha multiplication boundary
- continuous_policy: forward() input cast, Tensor::full scalars match dtype
- flow_policy: sample_base_noise cast to training dtype
- hidden_state_manager: reset tensors use training_dtype
- ensemble/ppo adapter: predict input cast to training dtype
- trainable_adapter: test uses training_dtype instead of hardcoded F32

Verified: 198/198 ml-ppo tests pass, 63/63 ml PPO tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 13:45:22 +01:00
jgrusewski
910f4bdee3 fix(cuda): dim_overrides before common header in backtest kernel, cap n_episodes at 4096
- backtest_forward_kernel: dim_overrides must precede common_device_functions.cuh
  which has #error guards requiring STATE_DIM/MARKET_DIM/PORTFOLIO_DIM to be
  defined before inclusion. Experience collector already had correct ordering.
- Cap MAX_EPISODES from 8192→4096 (diminishing returns above 4096, wastes walltime)
- Cap trainer .min() from 0x8000 (32768) → 4096 to match

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 13:05:43 +01:00