Wire the gradient vaccine's validation batch through the training loop:
- Pre-sample a SECOND batch from PER alongside each training batch
- Pass via FusedTrainingCtx::pending_vaccine_batch (consumed per step)
- Separate PER indices ensure train and val batches are independent
The vaccine's non-graph forward+backward runs on this held-out batch
to produce g_val. Gradient projection (dot + SAXPY) ensures g_train
only moves in directions where both batches agree.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Surgical gradient projection that makes overfitting mathematically
impossible. At each training step:
1. Save g_train (from CUDA Graph forward+backward)
2. Run SEPARATE non-graph forward+backward on vaccine batch → g_val
3. Compute dot(g_train, g_val) and |g_val|² via GPU reduction kernel
4. If dot < 0 (gradients DISAGREE), project out conflicting component:
g_train -= (dot/|g_val|²) * g_val
5. Adam only sees gradient directions where train AND val agree
Two new CUDA kernels:
- gradient_dot_and_norm: parallel reduction with warp+block+atomicAdd
Computes both dot product and norm² in single pass (bandwidth optimal)
- gradient_project: conditional SAXPY (skips when dot >= 0, no branch divergence)
The vaccine runs OUTSIDE the CUDA Graph (between replay_forward and
replay_adam) because it needs conditional logic. The non-graph vaccine
forward+backward reuses the same cuBLAS/loss/backward infrastructure.
Vaccine batch is sampled from the replay buffer alongside the training
batch and passed via FusedTrainingCtx::pending_vaccine_batch.
Config: enable_gradient_vaccine=false (default). Enable for mathematical
guarantee that no gradient update makes the model worse on held-out data.
Together with bottleneck (#31): "you can only remember 2 numbers, and
those 2 numbers must work on data you haven't trained on."
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full backward pass for the 2D bottleneck via cuBLAS chain rule:
1. bn_tanh_backward_kernel: d_bn = d_concat[:,:bn_dim] * (1 - tanh^2)
Reads tanh values from forward pass (bn_hidden_buf), applies derivative
2. cast_dx_to_staging: f32 d_bn → bf16 for cuBLAS dW GEMM
3. launch_dw_only: dW_bn[bn_dim, market_dim] += d_bn^T @ states
Uses same cuBLAS infrastructure as all other weight gradient GEMMs
4. bn_bias_grad_kernel: db_bn = sum(d_bn, dim=0)
backward_full() modified: new s1_dx_output parameter computes
d_loss/d_bn_concat when bottleneck is active (was 0 = skip).
Gradients flow through entire bottleneck → tanh → GEMM chain.
Adam optimizer trains bottleneck weights (tensors 20-21) alongside
all other parameters — same flat grad_buf, same spectral norm,
same weight decay. No special handling needed.
3 new CUDA kernels: bn_tanh_backward, bn_bias_grad, bn_tanh_concat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2D information bottleneck: [market_dim → bottleneck_dim] GEMM + tanh
before the Q-network trunk. Market features are compressed through 2
neurons — the model can only encode abstract "opportunity" and "risk",
not specific price sequences. Makes memorization structurally impossible.
Forward pass implementation:
- Weight layout expanded [20] → [22] tensors (w_bn, b_bn at positions 20-21)
- NUM_WEIGHT_TENSORS constant replaces all hardcoded [; 20] array sizes
- cuBLAS GemmEx: states[B, market_dim] @ w_bn^T → h_bn[B, 2]
- bn_tanh_concat_kernel: tanh(h_bn) ++ portfolio_features → input to h_s1
- h_s1 GEMM input changed from [B, 48] to [B, 5] (2 bottleneck + 3 portfolio)
- Backward pass + experience collector bottleneck: next commit
Config: bottleneck_dim=0 (disabled, backward compatible). Set to 2 to enable.
When disabled, zero-sized tensors 20-21 produce unchanged weight layout.
Also adds King (#33 Adversarial Self-Play) and Emperor (#34 Causal
Intervention) to the plan doc — completing the four-layer defense fortress.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Weight system expansion: param layout [20] → [22] tensors with
NUM_WEIGHT_TENSORS constant. Tensors 20-21 (w_bn, b_bn) hold temporal
causal bottleneck weights. Size 0 when bottleneck_dim=0 (backward
compatible — existing models load without changes).
Config: bottleneck_dim field added to DQNHyperparameters, GpuDqnTrainConfig,
TOML [generalization] section, and training profile. Default: 0 (disabled).
Set to 2 for maximum information compression.
Crown Jewels plan (Tasks 31-34):
- Gem (#31): 2D Temporal Causal Bottleneck (architecture defense)
- Pearl (#32): Gradient Vaccine (optimization defense)
- King (#33): Adversarial Self-Play with Past Self (strategic defense)
- Emperor (#34): Causal Intervention Training (epistemic defense)
Four layers of defense making memorization impossible at every level.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mathematical guarantee against overfitting: at each training step,
project out gradient components that contradict a held-out validation
batch. The optimizer can only move in directions where both train and
val agree — overfitting gradients are surgically removed.
Complementary to the Temporal Causal Bottleneck (#31):
- Bottleneck constrains REPRESENTATION (100 bits, no memorization room)
- Vaccine constrains OPTIMIZATION (only generalizing gradients survive)
- Together: "you can only remember 2 numbers, and those 2 numbers must
work on data you haven't trained on"
Implementation: dot(g_train, g_val) reduction + conditional SAXPY
projection between replay_forward() and replay_adam(). Two kernel
launches inside the CUDA Graph.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2D information bottleneck that makes memorization structurally impossible.
All 14 market features compressed through [market_dim → 2] linear + tanh
before reaching the Q-network. With only 2 floats (~100 bits), the network
physically cannot encode which specific price sequence it's looking at —
it can only encode abstract market CONDITIONS.
Attacks root cause of IS/OOS gap at the architecture level. All other 28
techniques fight symptoms; this one eliminates the disease. Provides free
interpretability (plot 2D activations) and a built-in generalization
diagnostic (IS/OOS distribution overlap in bottleneck space).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per-layer drop with 20% probability during training. Each hidden layer
(h_s1, h_s2, h_v) is independently scaled by 0 (dropped) or 1/(1-p)
(kept, expected-value correction). Only applied to online forward pass
on current states — target and Double-DQN passes use full network.
Implementation:
- New stochastic_depth_scale kernel in dqn_utility_kernels.cu
- Per-layer scale buffer [3] f32 — written by host before each graph
replay (CUDA Graphs capture addresses, not contents)
- Scale kernel inserted in launch_cublas_forward after online Pass 1
- update_stochastic_depth_mask() generates random 0/keep scales per step
- At inference (experience collection), all layers active (no drop)
Forces every layer to produce useful features independently — prevents
deep compositional memorization where removal of any single layer
would collapse the output. First RL trading application of stochastic
depth (proven in Vision Transformers, novel in DQN).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Time-reversal (#11): 20% of episodes (index % 5 == 0) played backwards
in state_gather kernel. Reversed bar indexing forces the model to rely
on instantaneous features rather than temporal trajectory patterns.
If it can't trade backwards data, it memorized sequence artifacts.
Counterfactual regret (#17): at trade completion, compute PnL for all
9 exposure levels and blend reward with regret (taken - best_possible).
Default 30% regret / 70% raw PnL. Normalizes rewards across regimes —
a bad trade in a bad market has low regret, a bad trade in a good
market has high regret. From game theory (CFR).
Both fully in CUDA env_step/state_gather kernels. No CPU paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mirror universe (#10): alternate odd/even epochs with negated return
features + inverted exposure actions in CUDA experience kernels.
Doubles effective data diversity without extra market data. The model
must learn direction-invariant structure — if it only works on normal
data but fails on mirrored, it learned directional bias.
Position entropy (#19): GPU-resident position visit histogram [N, 9]
incremented at each timestep in env_step kernel. At episode end,
computes H(histogram) / log(9) and adds scaled bonus to reward.
Forces the model to explore all 9 exposure levels rather than
degenerating to "always flat" or "always long" strategies.
Both fully in CUDA — zero CPU-side computation. Wired through
ExperienceCollectorConfig → kernel args → TOML [generalization].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 generalization techniques to close IS/OOS Sharpe gap (+0.57→-1.30).
All techniques fully wired from DQNHyperparameters → TOML [generalization]
section → ExperienceCollectorConfig → CUDA kernel arguments.
New techniques implemented:
- #13 Vol normalization: divide return features by realized vol (state_gather)
- #18 Asymmetric DD loss: extra penalty on Q-overestimation in drawdown
(mse_loss_batched + c51_loss_batched)
- #22 Feature noise: N(0, scale) per feature via LCG RNG (state_gather)
- #23 Causal feature masking: random 30% feature subset zeroed per epoch
(state_gather, mask uploaded from Rust)
- #24 Anti-intuitive LR: 3x LR when Sharpe good, 0.3x when bad (Rust-only)
- #25 Trade clustering: CV(inter-trade intervals) penalty via ps[3:6]
(env_step, uses reserved portfolio state slots)
- #27 Ensemble disagreement: Q_target -= weight * ensemble_std
(mse_loss + c51_loss, buffer allocated for future ensemble wiring)
Also includes 30-task plan doc with all 28+ techniques across 3 phases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CQL was effectively disabled (0.1 alpha × 0.15 budget = 1.5% of gradient).
Now: alpha=1.0 × 0.25 budget = 25% of gradient enforces conservatism.
C51 reduced from 70% to 60% to accommodate.
CQL penalizes Q-values for actions not in the data — directly prevents
the model from being "confident but wrong" on OOS state-action pairs.
Hyperopt search range updated: [0.0, 1.0] → [0.5, 5.0].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Was referencing training-pipeline which needs binary-tag for pre-built
binaries from GitLab packages. Now references compile-and-train which
compiles from source then trains. No stale binary risk.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New ensemble_reduce_kernel.cu: fused sigmoid→mean→weighted-sum in
one kernel, single block, thread-per-model. Pure f32, no bf16.
- build.rs: nvcc compilation without --use_fast_math
- aggregate_logits_gpu: loads cubin, uploads raw f32 buffers, launches
kernel, reads back single scalar. No GpuTensor/ActivationKernels.
- Removed aggregate_logits_cpu (dead code, GPU-only system)
- ml-explainability: removed unreachable dead code after stub return,
prefixed unused vars. Zero warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
argo-training-workflow was missing DNS egress (port 53), causing
MinIO hostname resolution to timeout. Also missing GitLab SSH (2222),
GitLab API (8181), Pushgateway (9091), and Registry (5000).
Now matches the gpu-test-workflow netpol which works correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6 test assertions hardcoded 22D search space, now 24D (added w_dd,
dd_threshold). Production profile test expected epochs=100, now 200.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- metrics.rs: fallback action_space was hardcoded 5 (old non-branching).
Changed to 9 (exposure levels). Branching is always active.
- smoke_test_real_data: assertion updated to accept >=9 action space
(81 with factored counts, 9 for short runs with empty factored counts)
- gpu_smoketest: added missing adam_epsilon field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 test files had stale types from the bf16→f32 conversion:
- gpu_smoketest: missing adam_epsilon in DQNConfig
- gpu_backtest_validation: closure params bf16→f32
- gpu_kernel_parity_test: market data, weight readback bf16→f32
- gpu_per_integration_test: weights readback bf16→f32
- target_update_tests: varstore register bf16→f32
- smoke_test_real_data: market buffers bf16→f32
- activation_tests, dropout_scheduler_tests: forward() signature change
These tests only compile with --features cuda (CI path), which is why
they passed locally with cargo test --lib.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The script only hashed .cu/.cuh in crates/ml/. Missed:
- build.rs changes (removing --use_fast_math → stale cubins cached)
- crates/ml-dqn/src/*.cu (replay buffer, seg tree kernels)
- crates/ml-core/src/cuda_autograd/*.cu
- crates/ml-ppo/src/cuda_nn/*.cu
Now hashes ALL kernel sources + ALL build.rs files. Any change to
nvcc flags or kernel source invalidates the entire PTX cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- train-dqn: hyperopt-trials 30→50, hyperopt-epochs 8→20,
train-epochs 80→200 (matches dqn-production.toml, 24D search space)
- training-workflow: fixed --max-steps-per-epoch bug (was passing
train-epochs as max-steps, limiting training to 80 steps/epoch
instead of 80 epochs). Changed to --epochs.
- Added FOXHUNT_TRAINING_PROFILE env var pointing to
dqn-production.toml so the training binary uses the correct config
(adam_epsilon=1e-8, cosine LR, drawdown penalty, etc.)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously fixed at w_dd=1.0, dd_threshold=0.02. Now PSO can search:
- w_dd: [0.0, 5.0] — drawdown penalty weight (0=disabled, 5=aggressive)
- dd_threshold: [0.01, 0.15] — drawdown % before penalty starts
Search space expanded from 22D to 24D. Backward compatible: vectors
shorter than 24 elements use defaults via bounds check.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The model had Omega>1 (profitable trade selection) but MaxDD 41-50%
(catastrophic drawdown timing), causing negative total returns despite
winning trades. Root cause: zero reward gradient between 0% and 25% DD.
The only drawdown consequence was the hard capital floor at 25% which
terminates the episode with reward=-10.
Fix: compute_drawdown_penalty() in trade_physics.cuh — smooth linear
ramp from 0 at dd_threshold (2%) to -5.0 at the capital floor (25%).
Applied every step, not just at trade exit, so the model learns to
reduce position size DURING drawdowns.
- Added compute_drawdown() and compute_drawdown_penalty() to trade_physics.cuh
- Wired dd_threshold and w_dd from config through to CUDA kernel
- Added to all 3 TOML profiles (smoketest, localdev, production)
Early results: MaxDD dropped from 88.9% → 36.6% by epoch 3.
Q-values went negative in drawdown states — the model is learning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
total_return_pct used simple pnl/initial_capital (arithmetic).
Omega/Sharpe/Sortino used pnl/current_equity (compounding per-trade).
These diverge after drawdown: Omega>1 but negative total_return was
mathematically impossible yet appeared in every run.
Fix: total_return_pct = ∏(1+r_i)-1 using the same per-trade returns
that feed Omega. Now Omega>1 ⟺ positive total_return. Always.
Single-pass equity curve + returns computation. No separate loops.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Returns were computed as pnl/initial_capital for ALL trades. After a
49% drawdown (equity=51K), a $500 loss was recorded as 500/100K=0.5%
instead of 500/51K=0.98%. This deflated losses after drawdown,
inflating Omega (5.5x) while total return was -9.5% — contradictory.
Fix: track running equity and compute each trade's return relative to
equity AT TIME OF TRADE. Now Omega, Sharpe, Sortino all reflect the
actual impact of each trade on the current portfolio.
This was the 'sneaky bug' causing Omega>5 with negative returns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: each trial created a fresh MlDevice (new CudaStream) but
the primary CudaContext was shared. cudaFreeAsync returns memory to
the allocating STREAM's pool, not the context's global heap. New
trial's new stream couldn't reclaim the old stream's freed blocks.
Evidence: Trial 0 leaked 2063MB, Trial 2 leaked 3212MB. Available
RAM dropped 10.4GB→5.2GB over 3 trials. Later trials ran on a
memory-starved system, explaining systematic performance degradation.
Fix: fork a new stream from the shared device instead of creating
a fresh MlDevice. Forked streams share the same context and async
memory pool — free_async blocks are immediately available to the
next trial's allocations.
Also fixed: TRIAL_SUMMARY best_epoch now shows actual best, not
epochs_completed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
best_epoch always showed 20 (total epochs) instead of the epoch where
the best val_loss checkpoint was saved. Misleading — looked like the
model never peaked. Now correctly reports actual_best_epoch from the
trainer's checkpoint tracker.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
With 10 trials and n_initial=5, PSO got 5 remaining evals for a
20-particle swarm. Only 5 of 20 particles were evaluated before the
budget observer killed the run — 75% of the swarm had no cost value.
PSO can't compute a proper gbest from partial data.
Fix: when remaining_trials < n_particles × 2, skip PSO entirely and
use additional LHS samples instead. LHS gives better space coverage
than an incomplete swarm iteration.
For 10 trials: 5 initial LHS + 5 additional LHS = 10 independent
space-filling samples. Much better than 5 LHS + 5 broken PSO particles.
PSO still runs when budget is sufficient (≥40 remaining for 20 particles).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Argmin defaults: inertia=0.72, cognitive=1.19, social=1.19
Problem: social >> inertia causes particles to collapse toward the
global best immediately. With only 1 LHS trial as the initial best,
the entire swarm clusters around that point and can't explore.
Every PSO trial was worse than the random LHS trial.
Fix: inertia=0.9 (high momentum, maintains exploration),
cognitive=1.5 (strong personal best memory),
social=0.8 (weak global pull, prevents premature convergence).
Applied to both sequential and parallel optimizer paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sharpe/Sortino were annualized with √252 (daily trading assumption).
For intraday strategies with hundreds of trades per eval window, this
inflated magnitudes ~10x, causing Sharpe=-1.4 while Omega=10.9 on
the same return series — mathematically contradictory.
Fix: scale by √N where N = actual number of returns in the series.
This gives the Sharpe of the evaluation window, not a synthetic annual.
- evaluation/metrics.rs: Sharpe, Sortino, Calmar all fixed
- trainer/metrics.rs: val_loss Sharpe (compute_validation_loss)
- ppo.rs: epoch Sharpe proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The docstring claimed 25% HFT activity score. The actual code uses 5%
(-0.05 coefficient). The real objective is 60% risk-adjusted composite
(Sharpe+Sortino+Calmar+Omega). Quality over quantity.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
H100 has the compute budget — 200 epochs with cosine decay gives the
model full room to learn through C51 transition and converge.
Early stopping can't fire before epoch 100.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added lr_decay_type (0=constant, 1=linear, 2=cosine) and lr_min to
the TOML training profile system. Wired through apply_to() with
total_steps derived from epochs.
- dqn-localdev.toml: lr_decay_type=2 (cosine 1e-4→1e-5 over 200 epochs),
min_epochs_before_stopping=80 (was 50, gives model more room to recover
after C51 transition).
- Smoketest config unchanged (lr_decay_type not set → default Constant).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Localdev config: gpu_n_episodes 32→64, gpu_timesteps 100→200
(12,800 exp/epoch, 4x more → stable Sharpe estimation)
2. Localdev config: buffer_size 10K→50K, min_replay_size 100→500
(fills over ~4 epochs instead of <1, reduces overfitting)
3. Hyperopt: added CampaignConfig::dqn_localdev() (10 trials × 20 epochs),
updated test_local_hyperopt to use it with env var overrides
(FOXHUNT_HYPEROPT_TRIALS, FOXHUNT_HYPEROPT_EPOCHS)
4. Q-value range: was [X,X] every epoch because:
- train_step.rs Q-stats code was DEAD (training loop uses
fused.run_full_step() directly, not self.train_step())
- Only sampled 10 experiences for Q-stats (tiny batch → tight range)
Fix: reduce_current_q_stats() reads the training batch's q_out_buf
directly (64 samples, no extra forward pass), wired into the training
loop's inner step. q_stats_kernel.cu converted to f32 arithmetic.
Now shows real variation: [-5.25, 5.03] instead of [2.80, 2.80]
Removed dead Q-estimation code from train_step.rs (was never reached
in the fused GPU training path).
11/11 smoke tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Hyperopt adapter now sets max_training_steps_per_epoch:
RTX 3050 (≤40GB) = 200 steps, H100 (≥40GB) = 2000 steps.
Without this, each trial trained the full dataset (2917 steps/epoch)
making hyperopt 11x slower than necessary on local GPU.
- adam_epsilon default 1e-3→1e-8 everywhere (conservative(), DQNConfig).
The old 1e-3 was a BF16 workaround (bf16(1e-8)=0 → div-by-zero).
Adam is now f32, so standard 1e-8 is correct.
- Early stopping enabled in dqn-localdev.toml (patience=20).
Hyperopt: 2 trials × 5 epochs in 132s (was ~20min). Zero NaN.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- acc_buf bf16→f32: epoch gradient/loss accumulators were bf16, causing
avg_grad to be quantized to 0.895 every epoch. Now f32, shows real
variation (0.776→0.783→0.788).
- EPOCH_DIAG warn!→info!: diagnostic logging, not a warning condition.
- Removed legacy training_guard_check + training_guard_accumulate kernels
(dead code, replaced by fused training_guard_check_and_accumulate).
- Added dqn-localdev.toml: 200-epoch RTX 3050 profile for extended runs.
200-epoch run completed successfully:
Best Sharpe: +6.43 at epoch 107
Final: Sharpe=+2.05, PF=1.35, Return=+1052%, 0 NaN
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The action diversity metric was always 9/45 because:
1. GPU monitoring only tracked exposure (9 levels), discarding order+urgency
2. Training loop re-derived order+urgency via deterministic route_action()
3. Action space was hardcoded as 45 (should be 9×3×3=81)
Fix:
- Expanded monitoring_reduce kernel to decode and count all 3 branches
(exposure[9], order[3], urgency[3]) from factored action indices
- Expanded summary buffer 16→24 bf16 for the extra counts
- MonitoringSummary now has order_counts[3] + urgency_counts[3]
- Training loop feeds raw GPU counts into monitor (no OrderRouter)
- Diversity = active_exp × active_ord × active_urg / (b0 × b1 × b2)
- Branch sizes from agent.branch_sizes(), not hardcoded
- Entropy computed over exposure distribution with correct normalization
Before: "Action diversity=9/45 (20.0%)" — every epoch, every run
After: "Action diversity=18/81 (22.2%) — exposure=3/9 order=3/3 urgency=2/3"
varies per epoch, reflects actual model behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes of sporadic NaN during training:
1. --use_fast_math (nvcc) breaks IEEE 754 NaN semantics: fmaxf(NaN,x)
returns NaN instead of x, isnan()/isinf() compile to false.
Replaced with --ftz=true --fmad=true --prec-div=true --prec-sqrt=true
across all 4 build.rs (ml, ml-dqn, ml-ppo, ml-core).
2. Cross-stream race: replay buffer wrote batch data on the device's
original stream while the trainer read it on a forked stream.
Fixed by passing the forked stream to the DQN agent via agent_device,
so all GPU components share a single CUDA stream (zero sync overhead).
3. Rewards/dones stored as bf16 in replay buffer caused done=0xFFFF NaN.
Converted entire rewards/dones pipeline to f32: experience collector,
replay buffer storage, nstep kernel, loss/grad kernels.
Also:
- Removed fast_isnan/fast_isinf/fast_isfinite wrappers — standard
isnan/isinf/isfinite work correctly without --use_fast_math
- Updated dqn-smoketest.toml: lr=1e-4, epsilon=1e-8 (f32 Adam values)
- Removed debug printfs from gather kernels
- Added curiosity_weight to training profile system
- Cleaned up smoke_params() inline overrides
11/11 smoke tests pass, 5/5 stress runs of 50-epoch test pass,
359/359 ml-dqn + 895/895 ml unit tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
