At batch_size=16384, per-sample gradient buffer was B*P*4 = 1.7GB
per IQL trainer (3.4GB for dual tau). Caused OOM on H100 for attention.
Fix: tile backward+reduce into chunks of 256 samples. Same kernels,
launched multiple times with offset pointers. Per-sample buffer
shrinks to min(B,256)*P*4 = 27MB. 64x smaller.
Also fixed: weight_grad_reduce uses += (accumulate across tiles)
instead of = (overwrite). Backward kernel takes full_batch_size
param for correct 1/N gradient scaling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: eval compute_expected_q used [-100,100] atom range. With 51
atoms that's delta_z=4.0 — all Q-values collapsed to ~0. Model weights
changed but eval actions didn't → identical val_Sharpe=-9.16 every epoch.
Fix: eval_v_range_buf initialized from config v_min/v_max, then updated
per-epoch from observed Q-stats (q_mean ± 3σ). Eval atoms now track
the actual Q-value distribution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GPU read-modify-write (acc_buf[0] += loss) requires device memory.
Mapped host memory doesn't support atomic-free += from GPU kernels
on all architectures — H100 showed avg_grad=0.000000.
Reverted to CudaSlice<f32> with DtoH at epoch boundary (gated by
FOXHUNT_GPU_SYNC_DIAG). Removed __threadfence_system from accumulator
writes (not needed for device memory).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Default: enabled (syncs ON for debugging).
Production: set FOXHUNT_GPU_SYNC_DIAG=0 to disable.
Gated:
- Experience collector progress logging sync (3x per collection)
- Training guard read_accumulators epoch-boundary sync
Not gated (required for correctness):
- Drop sync in experience collector (prevents VRAM leak)
- compute_q_stats DtoH (returns data to caller — needs mapped memory refactor)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
evaluate_dqn_graphed now takes Option<&mut dyn QValueProvider>.
Training eval passes Some(fused_ctx), standalone eval uses closure path.
Removed dead evaluate_dqn non-graphed branch from evaluate_baseline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Every constant now derives from data:
- cv_max decays at 0.999/step (1000-step half-life) — stale spikes
don't permanently anchor readiness
- p5 estimator inits from first batch's mean spread (not hardcoded 1.0)
- Frugal step uses 1/sqrt(step_count) — convergence guarantee
- Support floor fraction = 1/num_atoms (guarantees ≥1 atom resolution)
- Staleness decay_k = ln(20) — derived from "oldest = 5% weight" invariant
Zero manual tuning. Zero hardcoded constants. All self-calibrating.
50-epoch: fold 2 Sharpe 16.20, fold 3 Sharpe 6.42 at epoch 42.
No regression from making constants adaptive.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Single GPU-resident scalar: CV = sigma_adv / |mean_adv|.
When CV > 1 (noisy advantages), readiness < 1 → features suppressed.
When CV ≤ 1 (stable advantages), readiness = 1 → features fully active.
All 5 IQL downstream kernels blend toward neutral defaults at readiness=0:
- Per-sample C51 support: blends to [-1, 1] default
- PER modulation: blends to td_errors unmodified
- Branch scales: blends to uniform 0.25
- Advantage weights: blends to neutral 1.0
- Expectile gap epsilon: gap scaled by readiness
Zero new hyperparameters. Computed from existing adv_stats_buf.
Best Sharpe improved 11.39 → 14.23 on 10-epoch stability test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wired:
- modulate_td_errors: advantage-weighted PER priorities with staleness decay
- per_sample_epsilon: IQL expectile gap drives state-dependent exploration
in experience_action_select (NULL fallback for backtest evaluator)
- replay_write_cursor/capacity accessors for staleness computation
Removed:
- use_iql config flag (IQL is mandatory)
- rng_states from ALL 7 experience kernel signatures + Rust launchers
+ backtest evaluator (pure stateless Philox, zero LCG)
- Dead v_range comments cleaned
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
training_guard accumulator (loss_sum, grad_sum, step_count) now uses
MappedBuffer — GPU writes via device pointer, CPU reads via
read_volatile. Eliminates synchronous memcpy_dtoh at epoch boundary.
Also added __threadfence_system() after accumulator writes for
CPU visibility guarantee.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Removed rng_states read from experience_action_select (dead after Philox)
- Replaced lcg_random in expert_action_override with philox_uniform
- Replaced rng_states hash in add_advantage_noise with pure philox_uniform
- Removed rng_states write-back from action_select (was writing same value)
Result: rng_states buffer is now fully dead in the collection loop.
26/30 epochs bit-identical across processes (was 1/30 at session start).
Remaining 4/30 differ by 1e-6 (single ULP) — TF32 tensor core rounding.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Feature noise injection: philox_uniform(episode, timestep, feature)
instead of advancing rng_states by 2*market_dim per timestep
- Episode start jitter: philox_uniform(episode, 0, 9999)
- Domain randomization params: philox_uniform(episode, 0, 2000+k)
- Saboteur params: philox_uniform(episode, 0, 3000+k)
All experience collection randomness is now stateless — same inputs
always produce same outputs regardless of execution history.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaced global memory atomicAdd(&dx[j], grad_j) with:
1. Save d_q/d_k/d_v per-feature into shared d_qkv_save[3*Dh]
2. Each thread accumulates dx[j] for its assigned j values
by looping over all features — zero write contention
3. Flush dx_scratch to global dx with single write per j
Shared memory: 2*D+8 → 6*D+8 (dx_scratch[D] + d_qkv_save[3*D])
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- backward_kernels.cu: atomicAdd(&db[j], sum) → db[j] += sum
(single writer per j since out_dim < 256 = one block)
- c51_loss_kernel.cu: removed q_divergence atomicAdd from hot path
(was monitoring-only but ran inside CUDA Graph)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
experience_action_select kernel now uses philox_uniform(episode_id,
timestep, call_counter) instead of stateful lcg_random(&rng).
Eliminates branching-dependent RNG state divergence across processes.
Also passed timestep to backtest evaluator's action_select launch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaced all bf16()/bf16_zero()/bf16_one()/bf16_exp/bf16_sqrt/bf16_fmax/
bf16_fmin/bf16_fabs/bf16_log/bf16_shfl_xor/bf16_shfl_down/atomicAddBF16/
f32_to_bf16 wrapper calls with their native f32 equivalents across all 25
.cu kernel files. Updated stale comments. Fixed monitoring_kernel.cu atomic
CAS helpers to use 32-bit int CAS instead of broken 16-bit. Build verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two root causes of cross-process non-determinism eliminated:
1. PER sampling used Philox RNG for within-segment jitter.
Replaced with deterministic stratified midpoint sampling:
threshold = (i + 0.5) * total_sum / B. Zero randomness.
2. update_adv_sigma did synchronous memcpy_dtoh (GPU sync point).
Replaced with iql_adv_sigma_ema_update kernel — EMA computed
entirely on GPU, modulate_td_errors reads sigma from device buffer.
Result: 28/30 epochs bit-identical across processes (was 1/30).
Remaining ~1e-6 difference is f32 accumulator noise.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Missing set_per_sample_support_ptr/set_branch_scales_ptr calls caused
null pointer dereference in C51 loss kernel at step 0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compute_expected_q reads v_range_buf[0..2] for argmax — cannot use
per_sample_support [B,3]. Added fixed [-100, 100] eval_v_range_buf
for all non-C51-loss kernel launches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 3 scalar per-branch epsilons (epsilon_exp/ord/urg) in
branching_action_select with a single per-sample buffer [B] so that
V_high(s)-V_low(s) from IQL expectile regression drives exploration
uniformly across all 3 heads, superseding manual per-branch heuristics.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
6 new CUDA kernels, C51/grad/epsilon kernel mods, dual-tau IQL,
v_range dead code deletion, per-sample everything.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers: PER advantage-weighted staleness, per-sample C51 atom
support centered on V(s), per-branch advantage decomposition,
dual-tau expectile gap exploration, and v_range dead code removal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two root causes of cross-process non-determinism fixed:
1. MSE→C51 graph switch: graph_forward_mse and graph_forward were
captured in different sessions with different cublasLtMatmul configs.
Fix: always use blended graph_forward (c51_alpha controls blend).
Deleted LossMode enum, graph_forward_mse, and all switching code.
2. Multiple capture sessions at steps 0-2: graph_forward, graph_ddqn,
and graph_adam were captured individually before graph_mega at step 2.
Each capture contaminated cuBLAS internal state.
Fix: capture graph_mega at step 0 (single capture session).
Result: runs with same initial cublasLtMatmul capture are fully
bit-identical across all 30 epochs. Remaining cross-process variation
is from the single initial cublasLtMatmul call during graph capture.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Always use blended graph_forward (MSE+C51). c51_alpha controls
the blend weight. Separate MSE graph capture was the root cause
of cross-process non-determinism — different capture sessions
produce different cublasLtMatmul kernel configs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create SharedCublasHandle once in constructor and pass Arc clones to
CublasGemmSet (forward) and CublasBackwardSet (backward). Remove the
double_dqn_stream, pass1_event, pass3_event infrastructure — DDQN
forward now runs sequentially on the main stream via the shared handle.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace per-instance cuBLAS/cublasLt handle ownership in CublasForward
with Arc<SharedCublasHandle>. The struct is renamed to CublasGemmSet
and a type alias preserves backward compatibility for external callers.
Removed from CublasGemmSet: handle, workspace, lt_handle, lt_workspace,
bias kernels (all now in SharedCublasHandle). Kept: branch streams,
branch workspaces, GEMM caches, events, network dimensions.
External callers (gpu_dqn_trainer, gpu_experience_collector) will be
updated in Tasks 4-5 to pass Arc<SharedCublasHandle> to the constructor.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Capture the experience collector's bottleneck + Q-network forward pass
into a CUDA Graph on first timestep. Prevents ungraphed cublasLtMatmul
calls from contaminating cuBLAS internal state between epochs.
Root cause identified: multiple cuBLAS handles on the same CUDA stream.
NVIDIA docs: "maintain a separate cuBLAS handle for each stream" —
implying one handle per stream. We have 3 handles (experience collector,
training, DDQN) on one stream.
Proper fix (next session): refactor CublasForward to share a single
cuBLAS handle across all components. Separate the handle (shared)
from per-component cached descriptors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
graph_forward replay during eval writes td_errors and per_sample_loss
which PER reads to update priorities. Without save/restore, eval
corrupts PER state → different sample selection → training diverges.
DtoD save before replay, restore after. Cost: 4 DtoD copies of [B]
floats per eval call (~0.01ms).
Combined fixes in this determinism effort:
- IQN/attention backward: zero atomicAdd (restructured kernels)
- cublasLt: AlgoGetIds (verified: same algo_id=16 every run)
- Single CUDA stream (DDQN moved from double_dqn_stream)
- graph_forward persists across folds (never invalidated)
- QValueProvider routes eval through trainer's graph_forward
- PER save/restore prevents eval→training state corruption
Remaining: epochs 1-3 identical across runs, epoch 4 varies.
The cublasLtMatmul kernel execution is non-deterministic between
process invocations despite same algo + single stream. All NVIDIA
documented preconditions are met.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move DDQN forward (Pass 3) from double_dqn_stream to main stream.
NVIDIA docs: "bit-wise reproducibility is valid only when a single
CUDA stream is active." The double_dqn_stream fork is kept but
no longer used for cuBLAS operations.
Combined with AlgoGetIds (verified: same algo_id=16 every run),
this satisfies all documented NVIDIA preconditions for bit-wise
reproducibility. Epochs 1-3 are identical across runs. Epoch 4+
divergence remains under investigation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace cublasLtMatmulAlgoGetHeuristic (performance-ranked, varies
between process invocations) with cublasLtMatmulAlgoGetIds (static
enumeration) + AlgoInit + AlgoCheck.
AlgoGetIds returns a fixed list for a given GPU + cuBLAS version.
We iterate in order and pick the first that passes AlgoCheck. This
guarantees the same algorithm (ID 16) is selected every run.
Verified: algo_id=16 across all runs. Yet val_Sharpe still varies
at epoch 4. The remaining non-determinism is from algo 16's internal
kernel execution (thread scheduling), not from algorithm selection.
NVIDIA guarantees bit-identical results for same GPU + single stream
— investigating why this doesn't hold.
Applied to: create_cached_fwd_gemm_desc, create_cached_fwd_gemm_desc_relu_bias,
create_cached_bwd_gemm_desc.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
