VarStore elimination completion:
- xavier_init_params_buf(): CPU Xavier init directly into flat params_buf + target_params_buf
using compute_param_sizes() for correct bottleneck dims (s1_input_dim=54, not state_dim=80)
- Pad all cuBLAS-accessed flat buffers (grad/m/v/cql/vaccine + IQN + evaluator)
with cutlass_tile_pad to prevent SGEMM 32-element M-tile overreads
- Fix forward+backward GEMM for layer s1: use s1_input_dim/s1_ldb when bottleneck
active (was state_dim=80, writing 80-wide rows into 54-wide bn_d_concat_buf)
- Pad bn_concat_buf/bn_d_concat_buf for K-tile overreads (concat_dim=54 not multiple of 32)
- Vaccine gradient: eliminate 2× DtoD copies with std::mem::swap on CudaSlice structs
- Fix IQN Xavier init to include branch_3 (was [b0,b1,b2], now [b0,b1,b2,b3])
- Pad backtest evaluator buffers (params_flat, activations, logits)
Verified: compute-sanitizer --tool memcheck → 0 errors, 286+57 tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add to_varstore() compatibility shims on DuelingQNetwork and
DistributionalDuelingQNetwork so test/example code can rebuild a
GpuVarStore snapshot when needed. Delete dead tests that referenced
removed DQNAgent, PrioritizedReplayBuffer, and ReplayBufferType.
Fix action index references (action_19/21 -> action_28/30) and
type annotation issues (sin ambiguity, remainder operator).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DuelingWeightSet/BranchingWeightSet now created via from_flat_buffer()
pointing directly into GpuDqnTrainer's params_buf. No intermediate
VarStore extraction, no D2D copies, no flatten/unflatten cycle for
the primary weight sets.
bottleneck_dim=16 now works — smoke test passes. The dimension mismatch
is resolved because params_buf is allocated at the correct reduced
dimension, and weight sets point directly into it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added OwnedGpuLinear to ml-core (self-contained linear layer with owned
weight/bias CudaSlice, forward_with_slices bypasses GpuVarStore lookup)
- Removed GpuVarStore from all 12 ml-dqn modules: Sequential, branching,
distributional_dueling, dueling, network, rmsnorm, residual, curiosity,
attention, iql, quantile_regression, regime_conditional
- Removed get_q_network_vars/vars/store accessors from DQN, branching,
distributional_dueling, dueling, network, quantile_regression
- Replaced GpuLinear+GpuVarStore with OwnedGpuLinear in all cold-path modules
- Made load_from_safetensors a no-op (flat buffer path handles checkpoints)
- Made sync_to_varmap a no-op (flat buffer is source of truth)
- Moved branching->VarStore conversion to gpu_weights::branching_to_varstore
- Re-enabled bottleneck_dim=16 in config defaults + all 3 TOML configs
- Removed flatten_online_weights bottleneck skip workaround
- Zero GpuVarStore references in crates/ml-dqn/src + crates/ml/src/trainers/dqn
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DuelingWeightSet and BranchingWeightSet fields changed from owned
CudaSlice<f32> to raw u64 device pointers + usize element counts.
This enables zero-copy views directly into the flat params_buf for
the training hot path (no D2D copies, no shadow allocations).
Key changes:
- Weight sets store u64 + usize pairs instead of CudaSlice<f32>
- from_flat_buffer() creates zero-copy views into params_buf
- from_slices() creates pointer views from owned CudaSlice allocations
- DuelingWeightBacking/BranchingWeightBacking hold owned CudaSlice
arrays for callers that need independent allocations (experience
collector, ensemble heads, tests)
- flatten/unflatten become no-ops when weight sets point into params_buf
- extract functions return (Backing, WeightSet) tuples
- KernelWeightPack::build() uses u64 fields directly (no raw_device_ptr)
- All sync functions updated for pointer-based signatures
- Ensemble clone functions return backing + pointer view pairs
- gradient_budget tests updated (7/7 pass)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bottleneck_dim=16 causes w_s1 size mismatch between VarStore (full state_dim)
and flat params_buf (reduced bottleneck_concat_dim). EMA target sync crashes
with CUDA_ERROR_ILLEGAL_ADDRESS.
Root cause: GpuVarStore allocates weights at full state_dim but GpuDqnTrainer's
flat buffer uses bottleneck-reduced dimensions. These two weight sources are
fundamentally incompatible when bottleneck is active.
Fix: eliminate GpuVarStore entirely (next task), then re-enable bottleneck.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Launch ensemble_aggregate_kernel per training step to populate
ensemble_var_q_buf with per-atom C51 logit variance across K heads.
Expose via ensemble_var_ptr() on FusedTrainingContext. Add
ensemble_exploration_beta config (default 0.2). Wire into
set_ensemble_variance_bonus() on GpuExperienceCollector which computes
beta * sqrt(mean_var) and tiles it as a constant SAXPY bonus over all
Q-values before Boltzmann action selection. No-op when ensemble_count <= 1.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IQR (Q75-Q25) from IQN's 32 quantile estimates is computed per-action
after each training step and added to Q-values before Boltzmann action
selection. High-IQR actions (uncertain outcomes, typically large-magnitude
positions) get an exploration boost, countering C51's structural bias
toward low-variance Small actions that causes magnitude collapse.
Pipeline: IQN train -> compute_iqr() -> set_iqr_bonus() -> SAXPY on
q_values [N, q_stride] each experience timestep. Alpha default 0.3.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Even timesteps use the existing directional mirror (Short<->Long, negated
reward). Odd timesteps substitute a different magnitude and linearly scale
the reward by alt_mag_frac / actual_mag_frac, giving the magnitude branch
direct gradient signal about what bigger or smaller positions would have
earned. Falls back to directional mirror when reward is near-zero (scaling
uninformative). No extra buffer needed -- reuses the existing N*L
counterfactual slot.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Set c51_warmup_epochs default to 0. The ramp logic in training_loop.rs
already returns alpha_max immediately when c51_warmup_epochs==0, so all
epochs now see c51_alpha=0.5 from the first step. Stable loss landscape,
no CUDA graph recaptures from alpha transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ShortSmall=0, ShortHalf=1, ShortFull=2, Flat=3, LongSmall=4, LongHalf=5,
LongFull=6. Updates all array sizes, bounds checks, names, and distribution
logic across gpu_monitoring, monitoring, training_loop, metrics, and financials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Task 3: Enable PopArt reward normalization (default true, all 3 TOML configs,
wiring confirmed in fused_training.rs submit_forward_ops_main()).
Task 4: Couple PopArt variance with tau reset — add prev_popart_var to
FusedTrainingCtx, read_popart_var() to GpuDqnTrainer, read_popart_variance()
+ should_reset_tau() to FusedTrainingCtx, tau reset injected at epoch boundary
in training_loop.rs after log_phase_timing().
Task 5: Add best_sharpe/best_epoch/best_val_loss reset to reset_for_fold() so
each walk-forward fold competes independently. Also wire v8 reward fields
(popart_enabled, micro_reward_scale, td_lambda, max_trace_length,
hindsight_fraction, hindsight_lookahead) through training_profile.rs apply_to().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 4-branch DQN (direction x magnitude) had 3 degenerate variants
(Short25, Flat, Long25) that all mapped to 0.0 target exposure when
direction=Flat, causing 82% Flat collapse. Collapse these into a
single Flat variant, giving 7 levels (ShortSmall/Half/Full, Flat,
LongSmall/Half/Full) and 63 total factored actions (7x3x3).
- ExposureLevel enum: 9 variants -> 7 (add direction/magnitude/from_dir_mag)
- FactoredAction: 81 -> 63 total actions, from_index/to_index updated
- DQN epsilon-greedy: use from_dir_mag() instead of dir*3+mag indexing
- DQN config: num_actions default 9 -> 7
- PPO action space: 45 -> 63 actions, action masking updated
- Signal adapter CUDA kernel: 5-bin -> 7-bin exposure aggregation
- All tests updated for new variant names and index ranges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fuses GEMM + bias-add + ReLU into a single cublasLtMatmul kernel via
CUBLASLT_EPILOGUE_RELU_BIAS. Eliminates 7 separate add_bias_relu
kernel launches per forward pass (3 trunk + 4 branch hidden layers).
Creates separate cached descriptors with epilogue enabled at init time.
Falls back to separate kernels if the epilogue heuristic isn't available.
Bias pointer set dynamically per-call via set_matmul_desc_attribute.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Performance:
- Cache cuBLAS descriptors: pre-create matmul_desc + layouts + algo at init.
requestedAlgoCount=3 for better algorithm selection. Zero per-GEMM overhead.
- Per-branch workspace: 4 × 32MB separate workspace buffers for multi-stream
branch dispatch. Eliminates workspace contention on parallel execution.
Training stability:
- Remove hardcoded shrink-perturb that fired every epoch on short runs (3-5 epochs).
With epochs=5, interval = epochs/4 = 1 → fired every epoch, destroying epoch 1
learned weights. This caused Sharpe to collapse from +0.60 to -0.29 after epoch 1.
- Phase 3 shrink-perturb now uses config values (was hardcoded alpha=0.9, sigma=0.01).
- The config-defined shrink_perturb_interval=20 now controls all shrink-perturb timing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The backward GEMM heuristic on H100 with CUDA 13.0 requires 32MB
workspace for TF32 algorithm selection. 16MB causes
CUBLAS_STATUS_NOT_SUPPORTED on dW_only (m=256,n=128,k=16384).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IQN was recomputing target h_s2 via iqn_trunk_forward_kernel using the
same target weights on the same next_states that DQN Pass 2 already
computed into tg_h_s2_buf. Now IQN reads from the pre-computed buffer
via a DtoD copy (negligible) instead of re-running 2 cuBLAS GEMMs.
Expected: -15ms GPU time on H100 (batch=16384, shared_h2=256).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Events recorded during CUDA graph capture cannot be queried with
cuEventElapsedTime after graph replay — the graph creates internal
copies. Reverted to original mega-graph capture structure.
Sub-graph timing fields kept in PhaseEvents for future use with
split-graph diagnostic mode or nsys profiling.
Also kept submit_loss_and_grad_ops() extraction and pub(crate) visibility
on launch_cublas_forward/backward for future per-phase graph splitting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split fwd_bwd timing into 5 sub-phases: spectral, forward, loss,
backward, aux. Events recorded during graph capture replay with the
graph on every step.
Also extract submit_loss_and_grad_ops() from submit_forward_ops_main()
and make launch_cublas_forward/backward pub(crate) for sub-graph timing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CI builder uses CUDA 13.0 cudarc which exports _v2 suffix.
Local CUDA 12.x has both names but CI only resolves _v2.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: cuStreamSynchronize blocked the thread, preventing tokio
single-threaded runtime from progressing RwLock .write().await calls.
- Remove cuStreamSynchronize from training loop (stalls pipeline)
- Move log_phase_timing() after process_epoch_boundary (DtoH syncs stream)
- compute_epoch_q_diagnostics takes &mut DQNAgentType param (no re-lock)
- Disable Q-value gap diagnostics (deadlock in single-threaded runtime)
- smoke_trainer helpers: always set fxcache dir
- test_no_hang_single_epoch: use fxcache direct loading
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Delete 7 dead files: prioritized_replay.rs, prioritized_replay_staleness.rs,
replay_buffer.rs, hindsight_replay.rs, rainbow_agent.rs, checkpoint.rs,
strategy_dqn_bridge.rs (-4,760 lines)
- Rewrite replay_buffer_type.rs: ReplayBufferType enum → StagedGpuBuffer struct
(GPU PER is the only replay path, no CPU fallbacks)
- Strip agent.rs to data types only (TradingState + AgentMetrics)
- Convert DQNAgentType from enum to struct wrapping RegimeConditionalDQN
(Standard variant was never constructed, 43 dead match arms removed)
- Remove dead CPU methods: store_experience, fused_post_step,
fused_post_step_no_ema, adaptive_buffer_resize, refresh_stale_per_priorities
- Add always-on CUDA event per-phase profiling (8 events, 4 phases:
upload/fwd_bwd/adam/per_update) with Drop cleanup and error checking
to identify 329ms/batch H100 bottleneck
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: weight tensors packed sequentially in the flat params buffer
had non-aligned start offsets when preceding tensors had odd element counts
(e.g. bias of 51 atoms = 204 bytes, 204 % 16 = 12). cublasLtMatmul with
CUBLAS_COMPUTE_32F_FAST_TF32 requires 16-byte aligned buffer pointers.
Fix: pad each tensor to 4-element boundary (16 bytes) in both
f32_weight_ptrs_from_base (pointer computation) and compute_total_params
(buffer allocation). Added align4() and padded_byte_offset() helpers,
fixed shrink_perturb skip range and bottleneck gradient offset.
Switched compute type: CUBLAS_COMPUTE_32F → CUBLAS_COMPUTE_32F_FAST_TF32
(forward + backward). Explicit TF32 tensor core path, required by cuBLAS
13.0 on H100 SM90.
Deleted dead bf16_weight_ptrs function.
19/19 smoke tests pass on RTX 3050.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add last_epoch_loss metric. All assertions now compare the run's own metrics
against each other: convergence = loss stability (not divergence), overfitting =
best_oos_sharpe >= first_sharpe, OOS collapse = val_loss swing/magnitude ratio.
Walk-forward fold resets make epoch-1 loss unreliable for convergence comparison
(pre-trained weights), so we check stability instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add final_sharpe and final_val_loss to TrainingMetrics (last-epoch values)
- Rewrite test_walk_forward_no_overfitting_50_epochs: all checks are relative
to the run's own metrics — no hardcoded thresholds
- Checks: finiteness, gradient health, OOS collapse ratio, IS→OOS gap ratio
- 19/19 smoke tests pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract shared resolve_data_dir() function — resolves relative data paths
against workspace root via CARGO_MANIFEST_DIR. Used for mbp10_data_dir
and trades_data_dir (was only mbp10 before, causing smoke test failures).
All 19/19 smoke tests pass with pure f32 pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete bf16 elimination across all crates (ml, ml-core, ml-dqn, ml-ppo,
ml-supervised). Zero half::bf16, __nv_bfloat16, or CudaSlice<half::bf16>
references remain (verified by grep).
CUDA: All 60+ .cu kernels and .cuh headers converted to native float.
- Half-precision intrinsics (__hmul, __hadd, __hdiv) → float operators
- atomicAddBF16 → native atomicAdd(float)
- bf16 wrapper functions → f32 identity passthroughs
Rust: All CudaSlice<half::bf16> → CudaSlice<f32> across 90+ files.
- htod_f32_to_bf16/dtoh_bf16_to_f32 → htod_f32/dtoh_f32 (direct, no conversion)
- Deleted bf16 mirror infrastructure (DuelingWeightSetBf16, alloc_bf16_mirror, etc.)
- Renamed params_bf16→params_flat, d_value_logits_bf16→d_value_logits, etc.
- Fixed .to_f32() sed damage on Decimal::to_f32() and rng.f32()
FxCache: Single f32 disk format (was bf16/f64 dual-version).
- Deleted --bf16 CLI flag from precompute_features
- PVC cache files need regeneration via precompute_features
TF32 tensor cores activated via cublasLtMatmul CUBLAS_COMPUTE_32F — no
explicit TF32 types needed. Storage is pure f32 everywhere.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
