2D information bottleneck that makes memorization structurally impossible.
All 14 market features compressed through [market_dim → 2] linear + tanh
before reaching the Q-network. With only 2 floats (~100 bits), the network
physically cannot encode which specific price sequence it's looking at —
it can only encode abstract market CONDITIONS.
Attacks root cause of IS/OOS gap at the architecture level. All other 28
techniques fight symptoms; this one eliminates the disease. Provides free
interpretability (plot 2D activations) and a built-in generalization
diagnostic (IS/OOS distribution overlap in bottleneck space).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per-layer drop with 20% probability during training. Each hidden layer
(h_s1, h_s2, h_v) is independently scaled by 0 (dropped) or 1/(1-p)
(kept, expected-value correction). Only applied to online forward pass
on current states — target and Double-DQN passes use full network.
Implementation:
- New stochastic_depth_scale kernel in dqn_utility_kernels.cu
- Per-layer scale buffer [3] f32 — written by host before each graph
replay (CUDA Graphs capture addresses, not contents)
- Scale kernel inserted in launch_cublas_forward after online Pass 1
- update_stochastic_depth_mask() generates random 0/keep scales per step
- At inference (experience collection), all layers active (no drop)
Forces every layer to produce useful features independently — prevents
deep compositional memorization where removal of any single layer
would collapse the output. First RL trading application of stochastic
depth (proven in Vision Transformers, novel in DQN).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Time-reversal (#11): 20% of episodes (index % 5 == 0) played backwards
in state_gather kernel. Reversed bar indexing forces the model to rely
on instantaneous features rather than temporal trajectory patterns.
If it can't trade backwards data, it memorized sequence artifacts.
Counterfactual regret (#17): at trade completion, compute PnL for all
9 exposure levels and blend reward with regret (taken - best_possible).
Default 30% regret / 70% raw PnL. Normalizes rewards across regimes —
a bad trade in a bad market has low regret, a bad trade in a good
market has high regret. From game theory (CFR).
Both fully in CUDA env_step/state_gather kernels. No CPU paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mirror universe (#10): alternate odd/even epochs with negated return
features + inverted exposure actions in CUDA experience kernels.
Doubles effective data diversity without extra market data. The model
must learn direction-invariant structure — if it only works on normal
data but fails on mirrored, it learned directional bias.
Position entropy (#19): GPU-resident position visit histogram [N, 9]
incremented at each timestep in env_step kernel. At episode end,
computes H(histogram) / log(9) and adds scaled bonus to reward.
Forces the model to explore all 9 exposure levels rather than
degenerating to "always flat" or "always long" strategies.
Both fully in CUDA — zero CPU-side computation. Wired through
ExperienceCollectorConfig → kernel args → TOML [generalization].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 generalization techniques to close IS/OOS Sharpe gap (+0.57→-1.30).
All techniques fully wired from DQNHyperparameters → TOML [generalization]
section → ExperienceCollectorConfig → CUDA kernel arguments.
New techniques implemented:
- #13 Vol normalization: divide return features by realized vol (state_gather)
- #18 Asymmetric DD loss: extra penalty on Q-overestimation in drawdown
(mse_loss_batched + c51_loss_batched)
- #22 Feature noise: N(0, scale) per feature via LCG RNG (state_gather)
- #23 Causal feature masking: random 30% feature subset zeroed per epoch
(state_gather, mask uploaded from Rust)
- #24 Anti-intuitive LR: 3x LR when Sharpe good, 0.3x when bad (Rust-only)
- #25 Trade clustering: CV(inter-trade intervals) penalty via ps[3:6]
(env_step, uses reserved portfolio state slots)
- #27 Ensemble disagreement: Q_target -= weight * ensemble_std
(mse_loss + c51_loss, buffer allocated for future ensemble wiring)
Also includes 30-task plan doc with all 28+ techniques across 3 phases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CQL was effectively disabled (0.1 alpha × 0.15 budget = 1.5% of gradient).
Now: alpha=1.0 × 0.25 budget = 25% of gradient enforces conservatism.
C51 reduced from 70% to 60% to accommodate.
CQL penalizes Q-values for actions not in the data — directly prevents
the model from being "confident but wrong" on OOS state-action pairs.
Hyperopt search range updated: [0.0, 1.0] → [0.5, 5.0].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Was referencing training-pipeline which needs binary-tag for pre-built
binaries from GitLab packages. Now references compile-and-train which
compiles from source then trains. No stale binary risk.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New ensemble_reduce_kernel.cu: fused sigmoid→mean→weighted-sum in
one kernel, single block, thread-per-model. Pure f32, no bf16.
- build.rs: nvcc compilation without --use_fast_math
- aggregate_logits_gpu: loads cubin, uploads raw f32 buffers, launches
kernel, reads back single scalar. No GpuTensor/ActivationKernels.
- Removed aggregate_logits_cpu (dead code, GPU-only system)
- ml-explainability: removed unreachable dead code after stub return,
prefixed unused vars. Zero warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
argo-training-workflow was missing DNS egress (port 53), causing
MinIO hostname resolution to timeout. Also missing GitLab SSH (2222),
GitLab API (8181), Pushgateway (9091), and Registry (5000).
Now matches the gpu-test-workflow netpol which works correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6 test assertions hardcoded 22D search space, now 24D (added w_dd,
dd_threshold). Production profile test expected epochs=100, now 200.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- metrics.rs: fallback action_space was hardcoded 5 (old non-branching).
Changed to 9 (exposure levels). Branching is always active.
- smoke_test_real_data: assertion updated to accept >=9 action space
(81 with factored counts, 9 for short runs with empty factored counts)
- gpu_smoketest: added missing adam_epsilon field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 test files had stale types from the bf16→f32 conversion:
- gpu_smoketest: missing adam_epsilon in DQNConfig
- gpu_backtest_validation: closure params bf16→f32
- gpu_kernel_parity_test: market data, weight readback bf16→f32
- gpu_per_integration_test: weights readback bf16→f32
- target_update_tests: varstore register bf16→f32
- smoke_test_real_data: market buffers bf16→f32
- activation_tests, dropout_scheduler_tests: forward() signature change
These tests only compile with --features cuda (CI path), which is why
they passed locally with cargo test --lib.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The script only hashed .cu/.cuh in crates/ml/. Missed:
- build.rs changes (removing --use_fast_math → stale cubins cached)
- crates/ml-dqn/src/*.cu (replay buffer, seg tree kernels)
- crates/ml-core/src/cuda_autograd/*.cu
- crates/ml-ppo/src/cuda_nn/*.cu
Now hashes ALL kernel sources + ALL build.rs files. Any change to
nvcc flags or kernel source invalidates the entire PTX cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- train-dqn: hyperopt-trials 30→50, hyperopt-epochs 8→20,
train-epochs 80→200 (matches dqn-production.toml, 24D search space)
- training-workflow: fixed --max-steps-per-epoch bug (was passing
train-epochs as max-steps, limiting training to 80 steps/epoch
instead of 80 epochs). Changed to --epochs.
- Added FOXHUNT_TRAINING_PROFILE env var pointing to
dqn-production.toml so the training binary uses the correct config
(adam_epsilon=1e-8, cosine LR, drawdown penalty, etc.)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously fixed at w_dd=1.0, dd_threshold=0.02. Now PSO can search:
- w_dd: [0.0, 5.0] — drawdown penalty weight (0=disabled, 5=aggressive)
- dd_threshold: [0.01, 0.15] — drawdown % before penalty starts
Search space expanded from 22D to 24D. Backward compatible: vectors
shorter than 24 elements use defaults via bounds check.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The model had Omega>1 (profitable trade selection) but MaxDD 41-50%
(catastrophic drawdown timing), causing negative total returns despite
winning trades. Root cause: zero reward gradient between 0% and 25% DD.
The only drawdown consequence was the hard capital floor at 25% which
terminates the episode with reward=-10.
Fix: compute_drawdown_penalty() in trade_physics.cuh — smooth linear
ramp from 0 at dd_threshold (2%) to -5.0 at the capital floor (25%).
Applied every step, not just at trade exit, so the model learns to
reduce position size DURING drawdowns.
- Added compute_drawdown() and compute_drawdown_penalty() to trade_physics.cuh
- Wired dd_threshold and w_dd from config through to CUDA kernel
- Added to all 3 TOML profiles (smoketest, localdev, production)
Early results: MaxDD dropped from 88.9% → 36.6% by epoch 3.
Q-values went negative in drawdown states — the model is learning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
total_return_pct used simple pnl/initial_capital (arithmetic).
Omega/Sharpe/Sortino used pnl/current_equity (compounding per-trade).
These diverge after drawdown: Omega>1 but negative total_return was
mathematically impossible yet appeared in every run.
Fix: total_return_pct = ∏(1+r_i)-1 using the same per-trade returns
that feed Omega. Now Omega>1 ⟺ positive total_return. Always.
Single-pass equity curve + returns computation. No separate loops.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Returns were computed as pnl/initial_capital for ALL trades. After a
49% drawdown (equity=51K), a $500 loss was recorded as 500/100K=0.5%
instead of 500/51K=0.98%. This deflated losses after drawdown,
inflating Omega (5.5x) while total return was -9.5% — contradictory.
Fix: track running equity and compute each trade's return relative to
equity AT TIME OF TRADE. Now Omega, Sharpe, Sortino all reflect the
actual impact of each trade on the current portfolio.
This was the 'sneaky bug' causing Omega>5 with negative returns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: each trial created a fresh MlDevice (new CudaStream) but
the primary CudaContext was shared. cudaFreeAsync returns memory to
the allocating STREAM's pool, not the context's global heap. New
trial's new stream couldn't reclaim the old stream's freed blocks.
Evidence: Trial 0 leaked 2063MB, Trial 2 leaked 3212MB. Available
RAM dropped 10.4GB→5.2GB over 3 trials. Later trials ran on a
memory-starved system, explaining systematic performance degradation.
Fix: fork a new stream from the shared device instead of creating
a fresh MlDevice. Forked streams share the same context and async
memory pool — free_async blocks are immediately available to the
next trial's allocations.
Also fixed: TRIAL_SUMMARY best_epoch now shows actual best, not
epochs_completed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
best_epoch always showed 20 (total epochs) instead of the epoch where
the best val_loss checkpoint was saved. Misleading — looked like the
model never peaked. Now correctly reports actual_best_epoch from the
trainer's checkpoint tracker.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
With 10 trials and n_initial=5, PSO got 5 remaining evals for a
20-particle swarm. Only 5 of 20 particles were evaluated before the
budget observer killed the run — 75% of the swarm had no cost value.
PSO can't compute a proper gbest from partial data.
Fix: when remaining_trials < n_particles × 2, skip PSO entirely and
use additional LHS samples instead. LHS gives better space coverage
than an incomplete swarm iteration.
For 10 trials: 5 initial LHS + 5 additional LHS = 10 independent
space-filling samples. Much better than 5 LHS + 5 broken PSO particles.
PSO still runs when budget is sufficient (≥40 remaining for 20 particles).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Argmin defaults: inertia=0.72, cognitive=1.19, social=1.19
Problem: social >> inertia causes particles to collapse toward the
global best immediately. With only 1 LHS trial as the initial best,
the entire swarm clusters around that point and can't explore.
Every PSO trial was worse than the random LHS trial.
Fix: inertia=0.9 (high momentum, maintains exploration),
cognitive=1.5 (strong personal best memory),
social=0.8 (weak global pull, prevents premature convergence).
Applied to both sequential and parallel optimizer paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sharpe/Sortino were annualized with √252 (daily trading assumption).
For intraday strategies with hundreds of trades per eval window, this
inflated magnitudes ~10x, causing Sharpe=-1.4 while Omega=10.9 on
the same return series — mathematically contradictory.
Fix: scale by √N where N = actual number of returns in the series.
This gives the Sharpe of the evaluation window, not a synthetic annual.
- evaluation/metrics.rs: Sharpe, Sortino, Calmar all fixed
- trainer/metrics.rs: val_loss Sharpe (compute_validation_loss)
- ppo.rs: epoch Sharpe proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The docstring claimed 25% HFT activity score. The actual code uses 5%
(-0.05 coefficient). The real objective is 60% risk-adjusted composite
(Sharpe+Sortino+Calmar+Omega). Quality over quantity.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
H100 has the compute budget — 200 epochs with cosine decay gives the
model full room to learn through C51 transition and converge.
Early stopping can't fire before epoch 100.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added lr_decay_type (0=constant, 1=linear, 2=cosine) and lr_min to
the TOML training profile system. Wired through apply_to() with
total_steps derived from epochs.
- dqn-localdev.toml: lr_decay_type=2 (cosine 1e-4→1e-5 over 200 epochs),
min_epochs_before_stopping=80 (was 50, gives model more room to recover
after C51 transition).
- Smoketest config unchanged (lr_decay_type not set → default Constant).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Localdev config: gpu_n_episodes 32→64, gpu_timesteps 100→200
(12,800 exp/epoch, 4x more → stable Sharpe estimation)
2. Localdev config: buffer_size 10K→50K, min_replay_size 100→500
(fills over ~4 epochs instead of <1, reduces overfitting)
3. Hyperopt: added CampaignConfig::dqn_localdev() (10 trials × 20 epochs),
updated test_local_hyperopt to use it with env var overrides
(FOXHUNT_HYPEROPT_TRIALS, FOXHUNT_HYPEROPT_EPOCHS)
4. Q-value range: was [X,X] every epoch because:
- train_step.rs Q-stats code was DEAD (training loop uses
fused.run_full_step() directly, not self.train_step())
- Only sampled 10 experiences for Q-stats (tiny batch → tight range)
Fix: reduce_current_q_stats() reads the training batch's q_out_buf
directly (64 samples, no extra forward pass), wired into the training
loop's inner step. q_stats_kernel.cu converted to f32 arithmetic.
Now shows real variation: [-5.25, 5.03] instead of [2.80, 2.80]
Removed dead Q-estimation code from train_step.rs (was never reached
in the fused GPU training path).
11/11 smoke tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Hyperopt adapter now sets max_training_steps_per_epoch:
RTX 3050 (≤40GB) = 200 steps, H100 (≥40GB) = 2000 steps.
Without this, each trial trained the full dataset (2917 steps/epoch)
making hyperopt 11x slower than necessary on local GPU.
- adam_epsilon default 1e-3→1e-8 everywhere (conservative(), DQNConfig).
The old 1e-3 was a BF16 workaround (bf16(1e-8)=0 → div-by-zero).
Adam is now f32, so standard 1e-8 is correct.
- Early stopping enabled in dqn-localdev.toml (patience=20).
Hyperopt: 2 trials × 5 epochs in 132s (was ~20min). Zero NaN.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- acc_buf bf16→f32: epoch gradient/loss accumulators were bf16, causing
avg_grad to be quantized to 0.895 every epoch. Now f32, shows real
variation (0.776→0.783→0.788).
- EPOCH_DIAG warn!→info!: diagnostic logging, not a warning condition.
- Removed legacy training_guard_check + training_guard_accumulate kernels
(dead code, replaced by fused training_guard_check_and_accumulate).
- Added dqn-localdev.toml: 200-epoch RTX 3050 profile for extended runs.
200-epoch run completed successfully:
Best Sharpe: +6.43 at epoch 107
Final: Sharpe=+2.05, PF=1.35, Return=+1052%, 0 NaN
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The action diversity metric was always 9/45 because:
1. GPU monitoring only tracked exposure (9 levels), discarding order+urgency
2. Training loop re-derived order+urgency via deterministic route_action()
3. Action space was hardcoded as 45 (should be 9×3×3=81)
Fix:
- Expanded monitoring_reduce kernel to decode and count all 3 branches
(exposure[9], order[3], urgency[3]) from factored action indices
- Expanded summary buffer 16→24 bf16 for the extra counts
- MonitoringSummary now has order_counts[3] + urgency_counts[3]
- Training loop feeds raw GPU counts into monitor (no OrderRouter)
- Diversity = active_exp × active_ord × active_urg / (b0 × b1 × b2)
- Branch sizes from agent.branch_sizes(), not hardcoded
- Entropy computed over exposure distribution with correct normalization
Before: "Action diversity=9/45 (20.0%)" — every epoch, every run
After: "Action diversity=18/81 (22.2%) — exposure=3/9 order=3/3 urgency=2/3"
varies per epoch, reflects actual model behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes of sporadic NaN during training:
1. --use_fast_math (nvcc) breaks IEEE 754 NaN semantics: fmaxf(NaN,x)
returns NaN instead of x, isnan()/isinf() compile to false.
Replaced with --ftz=true --fmad=true --prec-div=true --prec-sqrt=true
across all 4 build.rs (ml, ml-dqn, ml-ppo, ml-core).
2. Cross-stream race: replay buffer wrote batch data on the device's
original stream while the trainer read it on a forked stream.
Fixed by passing the forked stream to the DQN agent via agent_device,
so all GPU components share a single CUDA stream (zero sync overhead).
3. Rewards/dones stored as bf16 in replay buffer caused done=0xFFFF NaN.
Converted entire rewards/dones pipeline to f32: experience collector,
replay buffer storage, nstep kernel, loss/grad kernels.
Also:
- Removed fast_isnan/fast_isinf/fast_isfinite wrappers — standard
isnan/isinf/isfinite work correctly without --use_fast_math
- Updated dqn-smoketest.toml: lr=1e-4, epsilon=1e-8 (f32 Adam values)
- Removed debug printfs from gather kernels
- Added curiosity_weight to training profile system
- Cleaned up smoke_params() inline overrides
11/11 smoke tests pass, 5/5 stress runs of 50-epoch test pass,
359/359 ml-dqn + 895/895 ml unit tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added DONE_NAN printf: raw bf16 bits are 0xFFFF (all 1s = bf16 NaN) for
intermittent done corruption. This is NOT from normal bf16 writes (0/1).
Key findings:
- Experience kernel writes valid done flags (debug printf confirmed)
- compute-sanitizer initcheck: 0 uninitialized reads
- Gather bounds check: no OOB indices detected
- The 0xFFFF pattern (all bits set) suggests memset(-1) or uninitialized memory
from a PREVIOUS allocation that was freed and reallocated
Next investigation: check if cudarc's alloc_zeros properly zeroes the dones
buffer, or if the GPU memory allocator returns memory from a previous freed
allocation that had 0xFFFF values (e.g., from a debug sentinel).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added bounds checking to gather_bf16 and gather_u32 kernels (capacity param).
Prevents OOB reads from corrupt PER segment tree indices.
Debug printf in MSE loss kernel confirms:
- done=nan (bf16 replay buffer dones contain NaN bits)
- is_weight sometimes negative (reduce_max_f32 atomicMax bug with neg floats)
- avg_mse=nan cascading from done=nan through Bellman target
The experience kernel writes valid done flags (0/1 as bf16). compute-sanitizer
initcheck: 0 errors (no uninitialized reads). racecheck: pending.
The NaN enters between experience write and loss kernel read — possibly from
the segment tree sampling a slot with corrupt priority (NaN priority → NaN
tree sum → traversal lands on wrong leaf → reads wrong data).
Next: convert dones + rewards to f32 throughout (same pattern as IS-weights).
This eliminates ALL remaining bf16 data paths in the training pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Re-applied f32 IS-weights permanently. The f32_to_bf16_cast kernel had a
manual RNE rounding overflow bug producing NaN for edge-case values.
Fixed cast to use __float2bfloat16 intrinsic, but NaN persists from
a DIFFERENT source: done=nan in replay buffer (not IS-weight).
Debug printf in MSE loss kernel shows:
- done=nan (bf16 replay buffer corruption)
- is_weight negative (should be [0,1] — reduce_max atomicMax bug with negative floats)
- avg_mse=nan cascading from done=nan through Bellman target
Root cause: replay buffer done flag corruption — likely a buffer overwrite
from an adjacent allocation. Need compute-sanitizer to find the OOB write.
895/895 unit + 359/359 ml-dqn tests pass.
Smoke tests: intermittent (NaN from corrupted done flags in replay buffer).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The guard cleanup agent incorrectly removed the backward dX overflow clamp
from relu_mask_kernel. This clamp is a LEGITIMATE bf16 overflow prevention
at the type boundary — identical to the forward-pass bias kernel clamping.
The dX GemmEx writes bf16 output. When the f32 accumulated sum exceeds
bf16 max (~65504), it writes Inf. The relu_mask ±500 clamp converts Inf
to a finite value, preventing cascade through the backward chain.
Also reverted the outside-graph params cast (the in-graph capture is correct).
Remaining issue: grad_norm drops to 0 in epoch 2+ with f32 master weights.
The model learns in epoch 1 (grad_norm=0.004) but stagnates in epoch 2+.
Investigating CUDA graph replay of f32→bf16 params cast.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
