Fix 2: Eliminate validation graph re-capture (DtoD into same buffers)
Fix 3: Async validation on separate CUDA stream (overlap with experience)
Fix 4: Aux GEMM in graph_aux (GPU scalar + kernel sig change)
Fix 7: Mega-graph fusion (spectral+forward+aux → 2 launches)
Combined with 5 implemented fixes: 47s → ~16s/epoch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TD(λ): self-bootstrap with rewards as Q(s') approximation, overwrites n-step
Hindsight: relabel fraction of experience rewards with optimal exit PnL
Curriculum: sort bars by difficulty, restrict to easy bars early, expand over training
No stubs, no debug placeholders — all fully wired.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pre-training: 50 batch supervised direction init at epoch 0
- Exposure aux targets: DtoD copy from collector to fused context
- PopArt: wired behind config flag (disabled by default)
- TD(λ)/hindsight/curriculum: stubs with config guards
- Loss threshold 500→100K for v8 reward distribution
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pre-training: 50 batch supervised direction init at epoch 0 (exposure branch)
- Exposure aux targets: DtoD copy from collector to fused context after experience collection
- PopArt: normalize_rewards_popart_inplace in fused training step (disabled by default)
- TD(λ): stub with debug log (kernel loaded, awaiting V(s) estimates for full wiring)
- Hindsight/curriculum: stubs with config guards (disabled by default)
- Added rewards_buf_mut() and normalize_rewards_popart_inplace() to GpuDqnTrainer
- Added run_pretrain_step() and replay_adam_and_readback_pretrain() wrappers to FusedTrainingCtx
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
At 0.001, the dense micro-reward was 1/1000th the magnitude of the
sparse exit reward (±10). Too weak to meaningfully bootstrap Q-values.
At 0.01, it's 1/100th — strong enough to provide directional feedback
while still subordinate to the exit signal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v8 reward changes (soft-clamp, CEA, micro-reward) produce higher
initial losses as C51 atoms calibrate to the new distribution.
Values up to ~50K are normal in early epochs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The action_select kernel signature changed from single `float epsilon`
to 4 args (eps_start, eps_end, current_epoch, total_epochs) for GPU-side
cosine schedule. The backtest evaluator wasn't updated, causing a
segfault from misaligned kernel args.
Fix: pass eps_start=0, eps_end=0 (greedy in backtest, no exploration).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CRITICAL: softmax_confidence created new CUDA context+stream per call.
Now reuses self.stream — eliminates severe inference latency.
HIGH: GpuPrioritized::add() was a silent no-op hiding bugs.
Now returns error + logs to catch invalid single-experience insertion.
MEDIUM: Duplicated epsilon logic across select_action methods.
Now uses get_effective_epsilon() consistently.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update n_steps test expectations from 3→5 to match current dqn-smoketest.toml
and dqn-production.toml configs. Update reward v7→v8 comments in experience_kernels.cu.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Forces exploration by starting value head biases at -0.1 for all atoms,
so the model must discover positive-value states through experience rather
than confidently exploiting random positives from zero-init.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Exposure aux kernel -> f32 scratch -> bf16 cast -> backward_fc_layer GEMM
- Weight + bias gradients accumulated into grad_buf (pure GPU, zero CPU sync)
- CEA warmup: linear decay 1.0->base over 25% of epochs
- OFI epoch gate: disabled for first 5 epochs
- Exposure aux decay: base -> 10% over warmup_epochs
- Removed dead v6 fields: loss_aversion, beta_penalty, regret_blend, trade_clustering_penalty
from ExperienceCollectorConfig (kernel no longer reads them)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 categories, 18 items:
A) v7.1 fixes: aux GEMM, CEA warmup, OFI gate, dead code
B) Bootstrap trap: GPU cosine epsilon, n-step→5, dense micro-reward
C) Initialization: supervised pre-train, pessimistic Q-init, phase schedule
D) Advanced: TD(λ), PopArt normalization, curriculum learning, hindsight relabel
E) Dead code removal
4 new CUDA kernels, 3 modified, 12 new config fields, 4 removed.
Full GPU, no CPU path, no memory copies. Target: 50%+ win rate.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- test_chunked_step_loop: 32000/512=63 chunks (was 32000/64=500)
- test_spectral_norm: disable bottleneck_dim in test to avoid w_s1 size mismatch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Anti-Flat-collapse: rewards exposure diversity via per-episode
position visit histogram entropy bonus.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Log v7 reward params after apply_family_scaling() so their scaled values are
visible in trial logs. Update loss_shaping_intensity doc comment to reference
v7 params (CEA, order credit, risk efficiency) instead of v6.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add cea_weight, order_credit_weight, risk_efficiency_weight to the
training_loop constructor of ExperienceCollectorConfig. Fix missing
field in Default impl.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace loss_aversion/beta_penalty/trade_clustering_penalty with cea_weight/
order_credit_weight/risk_efficiency_weight; remove regret_blend entirely.
Rust launch arg order matches CUDA parameter order exactly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added b1_size/b2_size params to compute (b0_size/2)*b1_size*b2_size
instead of raw b0_size/2. The metrics kernel decodes actions as
act/(order*urgency) — raw 4 decoded as exposure 0 (SELL), but the
correct flat action (4*3*3=36) decodes as exposure 4 (FLAT).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The batched kernel was written from scratch and diverged from the
working original in 10 ways, causing 78-95% max_dd despite the 25%
capital floor:
Bug 4 (CRITICAL): max_equity reset to cash on going flat — disabled
circuit breaker after every position close. REMOVED entirely.
Bug 9 (CRITICAL): pre-trade floor used break → tail write overwrote
episode-reset state. Changed to return.
Bug 5 (HIGH): inline hold_time didn't reset on reversal. Now uses
update_hold_time() from trade_physics.cuh.
Bug 2 (HIGH): pre-trade liq_ret hardcoded to 0.0. Now computed as
(liq_value - value) / value.
Bug 6 (MEDIUM): entry_price not cleared on flat. Added else-if branch.
Bug 1 (MEDIUM): pre-trade guard used value > 0 vs close > 0. Fixed.
Bug 8 (MEDIUM): step_count used chunk_len instead of steps_processed.
Bug 9b: tail write guarded by !done_flags[w] to prevent overwriting
breach handler state.
Verified: MaxDD=1.35% locally (was 78-95% on H100).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The batched kernel used `cash + position * close * contract_multiplier`
but the original uses `cash + position * close`. Position is already a
fractional exposure — the multiplier is handled internally by
execute_trade for P&L. The extra multiplier inflated equity 50×,
breaking capital floor checks, step returns, and drawdown to 89%.
Verified: MaxDD now 1.66% locally (was 89-100% on H100).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The batched backtest_env_step_batch kernel was missing the post-trade
capital floor check that exists in the original backtest_env_step.
With 16-24x leverage, a single 6% bar move could wipe 98.6% of equity
BEFORE the next bar's pre-trade check fired.
Added post-trade check after computing new_value: if equity breaches
75% floor, emergency liquidate + mark done. This matches the original
env_step kernel AND the training experience collector (which has both
pre-trade and post-trade checks).
Result: max_dd now capped at ~25% instead of 100%. Verified locally:
MaxDD=5.0% and 1.6% in smoke tests (was 100% on H100 hyperopt).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The preload's key-based fxcache lookup failed because the cache key
hash depends on file metadata (size+mtime) which differs between
the precompute pod and hyperopt pod (different PVC mount paths).
Fix: load the first .fxcache file from the cache directory directly.
The precompute step generates exactly one fxcache per dataset — no
ambiguity. Eliminates 285s DBN fallback loading.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bf16 has $32 resolution at $5K equity. Transaction costs ($12.50),
tick PnL ($12.50), and cumulative returns are all below resolution.
All previous hyperopt evaluations used garbage metrics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace stream.synchronize() with cuStreamWaitEvent for cross-stream
visibility of restored weights. The trainer writes best params on its
stream, records a CudaEvent, and the evaluator waits on that event
on ITS stream via cuStreamWaitEvent — purely GPU-side dependency,
zero CPU blocking.
Previous fix (stream.synchronize) was a CPU stall that hid the issue.
This is the proper CUDA approach: inter-stream event dependencies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
