Added raw_ptr() method to vendor/cudarc CudaSlice — returns device
address without event tracking (no SyncOnDrop guard leak).
Replaced ALL 119 raw_device_ptr() wrapper calls with .raw_ptr().
Deleted raw_device_ptr, raw_device_ptr_i32, raw_device_ptr_u32 wrappers.
Added forward_online_raw / forward_target_raw for graph-safe forward.
All CachedPtrs used in graph-captured code paths.
Smoke test: model trains (Sharpe improves -24→+1), but Q-stats
readback still returns 1e30. The issue is NOT device_ptr event
tracking — deeper investigation needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Expert analysis (zen debug): cudarc device_ptr() records stale events
after CUDA graph replay, causing race conditions. Added guards to:
compute_q_values, apply_iqn_trunk_gradient, apply_ensemble_diversity,
apply_cql_gradient, apply_cql_clipped_saxpy, apply_spectral_norm,
target_ema_update.
Smoke test: model trains correctly (Sharpe improves -25→-1),
but Q-value/loss/grad readback still returns garbage due to
cudarc device_ptr corruption in forward_online's raw_bf16_ptr.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- compute_q_stats needs its own forward pass (evaluates UPDATED model)
- Graph_forward doesn't include compute_expected_q for q_out_buf
- Training guard reads loss/grad from GPU (not per-step placeholders)
Smoke test: training completes 3 epochs, Sharpe improves, but
loss/grad_norm readback returns 0 and Q-stats reads 1e30.
Root cause: training_guard kernel reads from total_loss_buf which
graph_forward writes, but the readback pipeline may not sync properly
after graph replay.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause of Q-value NaN divergence: beta2=0.999 can't be represented
in BF16 (7-bit mantissa). bf16(0.999) = bf16(1.0). Then:
1 - beta2^t = 1 - 1^t = 0
v_hat = v / 0 = Inf → NaN → params NaN → forward NaN
Fix: compute bias correction (1 - beta^t) in f32 via powf(), then
convert result to bf16. This is the ONE justified f32 computation
in the kernel — bf16 can't represent 0.999 or 0.001.
Clamp bias correction to ≥1e-4 to prevent div-by-zero.
Smoke test: Q-values stable at -17.125, Sharpe improves
-23→-7→+9.78 across 3 epochs. Training completes without divergence.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause identified: bf16_weight_ptrs called device_ptr() inside
CUDA graph capture, which records cudarc events — not allowed during
capture. Created bf16_weight_ptrs_from_base() that takes pre-resolved
u64 base pointer from CachedPtrs (no device_ptr calls, graph-safe).
Replaced ALL 12 bf16_weight_ptrs calls in gpu_dqn_trainer with
bf16_weight_ptrs_from_base using self.ptrs.params_buf/target_params_buf.
Debug forward (no graph) confirms cuBLAS GemmEx produces valid logits.
Q-value divergence persists in graph mode — Adam or readback issue TBD.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename alloc_f32 → alloc_bf16 (59 occurrences) — function allocates
CudaSlice<half::bf16>, name should match
- C51 Bellman projection: convert float index arithmetic to native bf16
(bf16_floor for bin placement, bf16 frac computation)
- Clean up debug prints from smoke test investigation
Smoke test still shows NaN params from first graph capture. The
forward pass produces NaN logits → NaN gradients → Adam NaN → params
NaN. Root cause TBD: possibly bias pointer offset or cuBLAS forward
configuration issue specific to production-sized networks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All weight pointer computation uses size_of::<half::bf16>() — the
function name should reflect the actual type. Renamed across
batched_forward.rs, gpu_dqn_trainer.rs, gpu_experience_collector.rs.
Smoke test debug: params_buf valid after flatten, NaN after first
graph_adam replay. Root cause: either Adam produces NaN from the
first backward's gradients, or the C51/MSE loss kernel produces
NaN from valid logits. Need to check first-iteration loss output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: cudarc event tracking left stale state after CUDA graph
replay, causing INVALID_VALUE on subsequent kernel launches.
Fix: add EventTrackingGuard to clip_grad_buf_inplace (same pattern
as read_grad_norm_sync, apply_spectral_norm, etc.).
Smoke test now runs 3 epochs / 600 steps successfully.
Q-values diverge (bf16 precision) — needs hyperparameter tuning
for bf16 (lower lr, smaller Huber delta, tighter gradient clip).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Load separate dqn_grad_norm_kernel instance (grad_norm_standalone)
from fresh cubin for clip_grad_buf_inplace and read_grad_norm_sync.
CUDA graph captures function handles — same CUfunction can't be
used both inside captured graph and outside on same stream.
Smoke test status: experience collection + CUDA graph capture +
forward + loss + backward all pass. Remaining blocker:
non-graph kernel launch after graph replay (needs investigation
of cudarc event tracking interaction with CUDA graphs).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix last 2 tests:
- action_masking: test used 81-action FactoredAction decoder for
45-action mask. Fixed to use mask's own 5×9 exposure encoding.
- hyperopt bounds: test both Full (all ranges) and Fast (architecture
fixed) phases. Implemented phase_fast override in continuous_bounds
to pin hidden_dim_base, num_atoms, dueling_hidden_dim from TOML.
Final scorecard:
ml-core: 300/300 (100%)
ml-dqn: 359/359 (100%)
ml-ppo: 168/168 (100%)
ml: 895/895 (100%)
Total: 1722/1722 (100%)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The bulk nvrtc removal sed replaced ALL load_cubin(ptx) with
CQL_GRAD_CUBIN, but kernels live in 12 different cubin files.
Fixed: DQN_UTILITY_CUBIN for utility kernels, C51_LOSS_CUBIN for
c51 loss, MSE_LOSS_CUBIN/MSE_GRAD_CUBIN for MSE, ENSEMBLE_CUBIN
for ensemble, etc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix f32→bf16 type mismatches in 7 ml test files:
- gpu_action_selector, mod.rs, signal_adapter, gpu_residency,
gradient_budget, performance, training_stability
- Convert Vec<f32> → Vec<half::bf16> for memcpy_htod
- Convert readback to bf16 then to f32
- Relax tolerances for BF16 precision
Remaining 57 failures: nvrtc stubs in ml-ppo cuda_nn + kernel name
mismatches + replay buffer type issues.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Agent fixes: kernel name mismatches, sgemm→GemmEx BF16 in linear.rs
and stream_ops.rs, shared memory sizes for BF16 kernels, BF16-safe
optimizer betas (0.999 rounds to 1.0 in BF16), argmax index readback,
test tolerances relaxed for BF16 precision (~3 decimal digits).
ml-core: 300 passed, 0 failed.
ml-dqn: 304 passed, 55 failed (4 dead nvrtc stubs — separate task).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace all 20 dangling `ptx` variable references with correct cubin
static names after nvrtc removal sed. Fix ml-dqn dead code stubs
(residual.rs, rmsnorm.rs, noisy_layers.rs, gpu_replay_buffer.rs).
Clean up unused `let ptx` variables.
Zero compilation errors across full workspace.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GELU forward and backward now use bf16_tanh from common header
instead of float tanhf. tanh(x) = (exp(2x)-1)/(exp(2x)+1) via
bf16_exp — all native __nv_bfloat16 arithmetic. No exceptions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 COMPLETE. Every CUDA kernel in the pipeline compiles with
__nv_bfloat16 parameters and native BF16 arithmetic.
Final 4 kernels fixed:
- iql_value_kernel.cu: removed duplicate bf16_exp/bf16_sqrt, fixed dot-product accumulation
- experience_kernels.cu: fixed mixed type arithmetic, removed duplicate function definitions
- attention_backward_kernel.cu: fixed warp shuffles (float for __shfl_xor_sync), math functions
- ppo_experience_kernel.cu: BF16 overloads for matvec, explicit casts for barrier config
Remaining: 23 Rust errors in ml-core (BF16 boundary conversions).
Phase 2-4 of the plan: delete GpuTensor/nvrtc, fix Rust, wire cuBLAS, test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ema, relu_mask, per_update, iqn_cvar, trade_stats, monitoring,
backward, backtest_fwd_supervised, backtest_fwd_ppo, statistics,
curiosity, her_relabel — all converted to native __nv_bfloat16
with bf16_* wrappers. Zero __bfloat162float/__float2bfloat16 casts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
17 tasks across 4 phases:
- Phase 1: 10 tasks converting 35 CUDA kernel internals to native BF16
(bf16_* wrappers, native arithmetic, no __bfloat162float casts)
- Phase 2: Delete GpuTensor + nvrtc dependency (replace with raw CudaSlice)
- Phase 3: Fix Rust compilation errors at host boundaries
- Phase 4: Wire cublasGemmEx + test
Includes lessons learned from failed bulk-sed approach and explicit
conversion rules for every agent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- bf16_shfl_xor() — warp shuffle XOR for __nv_bfloat16
- bf16_shfl_down() — warp shuffle down
- bf16_warp_sum() — full warp-level sum reduction
- bf16_warp_max() — full warp-level max reduction
These hide the mandatory float cast (no native BF16 shuffle HW)
behind a clean BF16 API. Kernels use bf16_warp_sum(val) instead
of manual __bfloat162float → __shfl_xor_sync → __float2bfloat16.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CUDA kernels: 36 of 37 compiled with __nv_bfloat16* (dt_kernels.cu remaining)
Rust types: ALL CudaSlice<f32> → CudaSlice<half::bf16> across ml + ml-core
Build: all kernels now compiled with common_device_functions.cuh (BF16 helpers)
BF16 math wrappers: bf16_sqrt, bf16_log, bf16_exp, bf16_pow, bf16_fabs, bf16_fmax,
bf16_fmin, bf16_cos, bf16_zero, bf16_one, bf16(), atomicAddBF16
NOT YET COMPILING — ml-core boundary errors (Vec<f32> → Vec<half::bf16>)
and dt_kernels.cu float*__nv_bfloat16 ambiguity remain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- forward_target_bf16() for target network inference path
- BF16 activation pointer accessors for wiring into training path
- States F32→BF16 conversion at system boundary (experience collector output)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- cudarc features: removed "nvrtc" (no longer used, all kernels precompiled)
- cudarc features: added "f16" (enables DeviceRepr + ValidAsZeroBits for half::bf16)
- CudaSlice<half::bf16> now fully functional with all cudarc operations
- Foundation for full BF16 tensor core conversion (next session)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CachedPtrs:
- 35-field struct caching all GPU buffer u64 device pointers
- Computed once at construction, replaces 110+ raw_device_ptr() per step
- Eliminates cudarc event tracking machinery from hot path
Sync removal:
- apply_iqn_trunk_gradient: removed cuStreamSynchronize (same-stream ordering)
- apply_ensemble_trunk_gradient: removed cuStreamSynchronize
- run_ensemble_step: removed cuStreamSynchronize + local EvtGuard struct
- replay_adam_and_readback: zero per-step DtoH — returns 0.0, epoch boundary
uses GPU training guard's accumulator buffer for actual metrics
Logging cleanup:
- Removed per-step tracing::info diagnostic with cuStreamSync (was every 1000 steps)
- Removed per-step tracing::debug for IQL/IQN/CQL (format overhead in debug builds)
- Single tracing::debug at end of run_full_step (zero cost in release)
EventTrackingGuard made pub(crate) for fused_training.rs access.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ROOT CAUSE of fake 92% MaxDD: when capital floor fires (done=1), the
training kernel wrote done but did NOT reset the portfolio state. The
next step read the blown portfolio, hit the pre-trade floor check again,
returned done=1 with reward=-10, and the episode got STUCK in a done
loop for the rest of the epoch. Every step produced done=1 + -10 reward
from dead capital.
Fix: reset all 20 portfolio state fields + current_timesteps to fresh
capital at BOTH:
1. Pre-trade floor check (early return path, line 607)
2. Post-trade done detection (end of kernel, after outputs written)
This ensures the next step starts a clean episode with initial_capital,
flat position, and zero Kelly/realized_pnl accumulators.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The done bar's return (liquidation loss) must be counted in the current
episode's DD before resetting. Previous code reset BEFORE compounding,
which applied the liquidation return to fresh initial_capital.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The training financials computed MaxDD by compounding returns across
episode boundaries. When the capital floor circuit breaker fired (done=1)
and the episode reset with fresh capital, the equity curve kept falling.
This produced fake 92% MaxDD from concatenated episodes.
Fix: added done_flags to TradeStats (downloaded from GPU done_out buffer
at epoch end). MaxDD computation now resets equity/peak at done=1 events,
matching the backtest evaluator's per-window isolation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Research finding: NoisyNet provides undirected parametric noise, count-bonus
provides directed UCB action-level exploration. They don't conflict —
they address different failure modes (NoisyNet: policy lock-in,
count-bonus: action collapse onto Flat).
The CUDA kernel was already fully wired. Only the coefficient changed
from 0.0 to 0.05. Especially important for:
- 81 factored actions (high combinatorial space)
- Sparse trade-completion rewards (long reward-drought periods)
- Non-stationary markets (exploration must remain active)
References: Osband et al. 2019, Bellemare et al. 2016, Ostrovski et al. 2017
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
