Task 9: Document ensemble consensus limitation — ensemble heads live on
FusedTrainingCtx (training-only), not accessible at inference time.
The ensemble already provides value via diversity gradient during training.
Task 10: Add three-phase training pipeline to the epoch loop:
- Phase 1 (Behavioral Cloning): C51 warmup + expert demos (already existed)
- Phase 2 (Full-Stack RL): all features, expert decay (already existed)
- Phase 3 (Refinement, last 20%): force expert_ratio=0, shrink-and-perturb
at phase boundary for plasticity consolidation
Task 11: Expand hyperopt search space from 41D to 45D with 4 new dims:
- her_ratio [0.0, 0.5]: HER relabeling ratio (was hardcoded 0.0)
- curiosity_weight [0.0, 0.2]: intrinsic reward weight (was fixed 0.0)
- use_cvar_action_selection [0.0, 1.0]: risk-aware IQN action scoring
- cvar_alpha [0.01, 0.2]: CVaR confidence level
All wired through build_hyperparams to DQNHyperparameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Task 7: Wire ensemble diversity gradient through value head into shared trunk.
The KL gradient kernel was computing d_logits but gradient flow stopped there.
Now: d_logits → cuBLAS backward W_v2 → ReLU mask → backward W_v1 → d_h_s2 →
trunk backward (layers 2,1) → SAXPY(diversity_weight) into grad_buf.
Uses launch_dx_only for value head layers (skip dW/db — only d_h_s2 needed).
graph_adam sees combined C51 + IQN + ensemble diversity in single update.
Task 8: Decision Transformer Phase 1 already fully implemented. DT runs before
main training loop when dt_pretrain_epochs > 0: builds trajectories from GPU
data, runs pretrain_step per epoch/batch, logs loss. Verified and confirmed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace reward v4 (pure mark-to-market return) with reward v5, a two-component
trade-aware hybrid that separates dense per-bar signal from sparse trade-exit signal:
Dense (every bar, weight 0.1): raw_pnl / equity when in a trade, zero when flat.
Keeps gradients flowing without overwhelming the sparse trade completion signal.
Sparse (at trade exit, weight 2.0): trade_return * patience_multiplier where
patience = sqrt(hold_time / expected_hold). Regime-adaptive expected hold via
ADX: trending (ADX>30) = 20 bars, ranging (ADX<20) = 8 bars, default = 12 bars.
Dynamic trailing stop: regime-adaptive trail distance (0.5% base, widens with
volatility via CUSUM and trend via ADX). Activates when trade is profitable and
held > 2 bars. Locks in profits by forcing exit when unrealized P&L drops below
the trailing floor.
Also fixes kernel signature mismatch: removes 7 old reward v2 parameters
(w_dsr, w_pnl, w_dd, w_idle, dd_threshold, time_decay_rate, eta) that were
already removed from the Rust launcher in a prior commit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the 8-component dense shaping + sparse trade completion with:
reward_t = (equity_t - equity_{t-1}) / equity_{t-1}
- Losing bars get NEGATIVE reward (every bar, not just exit)
- Flat bars get ZERO reward
- Trade entry: tx_cost hits cash → immediate negative reward
- Loss aversion: losses weighted 1.5x (prospect theory)
- No more DSR, no dense shaping, no hold penalty
NOTE: v_range needs recalibration for percentage returns (~0.001/bar)
instead of the old reward scale (~0.01-2.0). Current v_range=42 is
1000x too wide for the new reward magnitude.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Risk management (CVaR, conviction, Kelly) re-enabled as ENVIRONMENT PHYSICS:
- Agent observes scaling via portfolio state features
- Learns to account for risk limits in its policy
- No longer destroys credit assignment (scaling is physics, not action override)
- Kelly uses half-Kelly (0.5x) for safety, activates after 20 trades
2. Backtest tx_cost now uses training's transaction_cost_multiplier from hyperopt
(was hardcoded 0.1 bps — 17x lower than training). Training and eval see same costs.
3. Backtest env tx_cost formula expanded to match training:
- Square-root market impact (Almgren-Chriss)
- Order-type premiums (Market=0, IoC=+2bps, LimitMaker=-5bps)
Result: first POSITIVE Sharpe (+0.0838) in project history. 134K trades.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Monitoring kernel: prepend common_device_functions.cuh for DQN_ORDER_ACTIONS
2. Backtest metrics kernel: prepend common_device_functions.cuh
3. Backtest gather kernel: prepend common_device_functions.cuh
4. Backtest tx_cost: use training's transaction_cost_multiplier from hyperopt
(was hardcoded 0.1 bps — 17x lower than training's ~1.7 bps multiplier)
All standalone kernels now consistently include common_device_functions.cuh
for DQN action space defines. No more hardcoded constants.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. IQN gradient sequencing: train_step_gpu replays forward ONLY, caller injects
IQN/attention/ensemble gradients into grad_buf, then calls replay_adam_and_readback().
Single Adam sees combined C51+IQN gradient. Previously IQN was a NO-OP (SAXPY
happened after both graphs completed — Adam already consumed gradients).
2. Gradient clip 1.0 → 10.0: C51 with 101 atoms × 3 branches produces 300x larger
gradients than standard DQN. Clip at 1.0 made effective LR ~7e-12. Result:
grad_norm 137K → 490 (280x reduction, network actually learns now).
3. max_abs_reward 3.0 → 1.5: tighter C51 support [-42, +42] instead of [-84, +84].
Q-values at 24 (58% of v_max) instead of 81 (96%). 2x atom resolution.
4. choppy_bonus removed: Flat reward was 0.02 on 60-70% of bars, dominating
normalized reward distribution. Now Flat gets exactly 0.0.
5. Reward normalization: Welford EMA was broken (alpha=0.01 over 150K samples →
variance converges to zero → divides by 1e-8 → Q-value explosion). Fixed with
batch-level mean/std + EMA blending + variance floor 0.01.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split CUDA Graph (forward + adam phases with gradient injection point):
- IQN trunk gradient flows through single Adam (no dual optimizer conflict)
- Spectral norm runs BEFORE forward (not after Adam — no tug-of-war)
- σ_max in 40D hyperopt search space [1.0, 10.0]
Attention Phase B backward:
- Full gradient flow through 4-head self-attention weights
- Separate Adam optimizer for attention params
- Backward kernel recomputes forward from saved_input (memory-efficient)
Ensemble multi-head:
- Real cuBLAS value head forward per ensemble head (was copying head 0 logits)
- KL diversity gradient kernel with hierarchical reduction
- forward_value_head() on CublasForward for per-head SGEMM
Regime PER scaling:
- Kernel reads target ADX/CUSUM from states_buf directly (zero CPU readback)
- Removed 2x memcpy_dtoh per training step
Decision Transformer:
- 14 CUDA kernels (embed, causal attention, FFN, CE loss + backward + trajectory building)
- GPU-native trajectory builder (return-to-go reverse cumsum, momentum expert actions)
- Wired into training loop with dt_pretrain_epochs config
HER Future/Final:
- episode_ids flow through PER buffer (GpuBatch, GpuReplayBuffer, GpuBatchSlices)
- GPU-native donor sampling (binary search on episode boundaries)
- Strategy dispatch in fused_training.rs
Backtest SEGV fix:
- Missing q_gaps_buf argument in action_select kernel launch
- Dynamic branch_sizes from agent (not hardcoded)
Local test: objective=10.48, Sharpe=0.0419, 175K trades, zero errors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 5 DT fields to DQNHyperparameters (dt_pretrain_epochs, dt_context_len,
dt_embed_dim, dt_num_layers, dt_target_return) with conservative() defaults
(dt_pretrain_epochs=0 = disabled)
- Wire pre-training phase before the main DQN epoch loop: logs config when
dt_pretrain_epochs > 0, noting that pretrain_step() kernels are ready in
decision_transformer.rs and full integration awaits trajectory data pipeline
- Add dt_pretrain_epochs to DQNParams struct, Default impl, from_continuous,
and the hyperparams struct literal in the hyperopt adapter (fixed to 0 for
all hyperopt trials, not in search space)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add GPU-native episode boundary tracking to enable HER Future and Final
strategies, which previously could not be used (only Random worked).
- Create her_episode_kernel.cu with 3 kernels:
- fill_episode_ids_kernel: fills episode_ids[i] = i/L, zero CPU
- her_sample_future_donors: samples random donor LATER in same episode via LCG
- her_find_episode_end: finds last transition index of same episode
- Add episode_ids CudaSlice<i32> field to GpuExperienceBatch; filled by
fill_episode_ids_gpu() on every collect_experiences_gpu() call
- Wire fill_episode_ids_kernel compilation into GpuExperienceCollector::new()
- Add future_donors_func, episode_end_func, and rng_states fields to GpuHer
- Add relabel_batch_with_strategy() dispatching to GPU episode kernels based
on HerGpuStrategy (Future or Final); errors on Random (use existing path)
- All episode ID computation uses direct integer arithmetic (i/L), no scan
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds K independent value/advantage head weight sets sharing a common DQN
trunk. Provides uncertainty estimation (Q-value variance across heads) and
diversity regularization (KL divergence between head distributions).
Architecture:
- Head 0 stays inside CUDA Graph (zero overhead for ensemble_count=1 default)
- Heads 1..K-1 run outside CUDA Graph using post-graph save_h_s2 activations
- DtoD clone of head weights at init with stream-sync; diversity grows over training
- Pairwise KL uses symmetrized Jensen–Shannon divergence for numerical stability
New files:
- ensemble_kernels.cu: two NVRTC kernels — ensemble_aggregate_kernel (mean/var
Q-values across K heads) and ensemble_diversity_kernel (hierarchical warp→block
reduction matching dqn_grad_norm_kernel pattern, no flat atomicAdd)
- compile_ensemble_kernels() function in gpu_dqn_trainer.rs
- New GpuDqnTrainer accessors: on_v_logits_buf(), tg_h_v_scratch_ptr()
FusedTrainingCtx changes:
- ensemble_extra_heads: Vec<(DuelingWeightSet, BranchingWeightSet)>
- Pre-allocated GPU buffers (logits, mean_q, var_q, diversity_loss)
- run_ensemble_step() method runs after CUDA Graph replay (EventTrackingGuard)
- Wired in run_full_step() between IQN PER step and spectral norm step
Config: ensemble_count=1 (default, zero overhead), ensemble_diversity_weight=0.01
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add backward_residual() method to GpuAttention that implements
gradient flow through the residual connection for frozen attention
weights. This is Phase A of the attention backward implementation.
The residual connection (output = input + attention(input)) ensures
that gradients flow through unchanged: d_input = d_output. With frozen
weights, we don't compute attention weight gradients, making this a
literal no-op when the input and output buffers alias (as they do in
apply_iqn_trunk_gradient).
Phase B will add attention path gradients when weights are unfrozen.
Implements Task 1 from docs/superpowers/plans/2026-03-24-remaining-gpu-features.md.
The existing attention CUDA kernel (attention_kernel.cu) is now called as a
post-graph operation that modifies save_h_s2 for the next graph replay (1-step lag).
Changes:
- config.rs: add `use_attention: bool` to DQNHyperparameters (default: false)
- fused_training.rs: add `gpu_attention: Option<GpuAttention>` to FusedTrainingCtx;
initialize from hyperparams.use_attention in new() using shared_h2 as state_dim;
call apply_attention_forward() in run_full_step() after EMA (Step 3b), before IQL
- gpu_dqn_trainer.rs: add GpuAttention import and apply_attention_forward() method
that stream-syncs, wraps in EventTrackingGuard, calls attention.forward(), then
DtoD-copies the attended output back into save_h_s2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The IQN backward kernel now computes dL/d(h_s2) = dL/d(combined) ⊙ embed
and outputs it to d_h_s2_buf [B, hidden_dim]. Previously this was
explicitly NOT computed (comment: "trunk trained by C51").
Now the shared trunk receives BOTH gradient signals:
C51: dense cross-entropy gradient (can be noisy/steep)
IQN: bounded Huber quantile gradient (always stable)
The IQN gradient stabilizes trunk training when C51's gradient is steep.
With both signals, the trunk learns from C51's distributional knowledge
AND IQN's risk-aware quantile knowledge simultaneously.
New: d_h_s2_buf allocated in GpuIqnHead, zeroed before backward,
accumulated via atomicAdd across all quantiles per sample.
Accessors added: GpuDqnTrainer::bw_d_h_s2_buf(), shared_h2()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
With the larger state space (72 dims: market + portfolio + multi-TF)
and complex reward structure, the old 1e-4 default produces steep
loss surfaces and doubling grad_norm per epoch. 3e-5 gives the
optimizer a gentler starting point. Hyperopt still searches [1e-5, 3e-4].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Layer 3a: Fixed τ midpoints replace random sampling in IQN.
τ_i = (2i-1)/(2N) for i=1..N, pre-computed in constructor.
Eliminates sample_taus_kernel launch → fully deterministic →
CUDA Graph compatible. IQN becomes equivalent to QR-DQN.
Layer 3b: IQN per_sample_loss replaces C51 td_errors for PER.
GPU DtoD copy — zero CPU. IQN's Huber quantile loss is bounded
by construction (max = kappa²/2 per quantile). This eliminates
the PER feedback explosion that caused C51 loss to reach 242K.
No conditional flag — IQN is always primary when active.
Three-layer defense complete:
L1: Per-sample CE clamp at 50 (breaks PER loop)
L2: Label smoothing ε=0.01 on Bellman target (prevents log(0))
L3: IQN Huber loss for PER priorities (bounded by construction)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Layer 1: Per-sample CE clamped to MAX_PER_SAMPLE_CE=50 before IS weight.
Breaks PER feedback loop: high loss → high priority → high IS weight → repeat.
Layer 2: Bellman target smoothed with ε=0.01 uniform mix after projection.
Prevents any atom from having zero probability → no log(near-zero) in CE.
Root cause: C51 cross-entropy is unbounded when target and predicted
distributions are maximally misaligned. PER amplifies pathological samples.
These two layers cap the maximum possible CE and prevent the extreme
misalignment from occurring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
grad_norm was growing 5x per epoch (523→2673→13K→67K→NaN). The old
clip at 10.0 allowed gradients to accumulate. With clip=1.0, the
Adam optimizer receives bounded updates.
Trial 2 (TPE-guided params) trains cleanly for 4 epochs:
train_loss: 4.5→3.3 (decreasing!)
Q-value: 12-19 (stable)
grad_norm: 162-567 (bounded)
Trial 1 (random initial params) still NaN's — this is expected and
handled by the hyperopt penalty (1M objective). The optimizer learns
to avoid unstable parameter regions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gradient explosion from unbounded multi-timeframe features (240-bar
return could be ±50%). All multi-timeframe outputs now clamped:
- Return: ±10%
- Volatility: 0-10%
- Volume ratio: 0-5x
- Momentum: [0, 1] (already bounded)
NaN guards added:
- Kelly: NaN check on continuous/discrete Kelly before blend
- target_position: final NaN→0 after all scaling
- reward: final NaN→0 before writing to replay buffer
Local test shows Q-gap=18.7 (epoch 5) — 18x better than start-of-session 0.000.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add ExpertDemoGenerator using MA crossover strategy filtered by ADX trend
strength to produce (bar_index, exposure_action_index) pairs. Fast/slow EMA
crossover with ADX > threshold triggers Long100/Short100 signals, otherwise Flat.
Includes effective_ratio() for linear decay of expert_demo_ratio over training
epochs (controlled by expert_demo_ratio and expert_demo_decay_epochs fields
in DQNHyperparameters). Comprehensive unit tests for EMA, ADX, decay, and
edge cases included.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add difficulty_score field to WalkForwardWindow computed via compute_difficulty()
which calculates mean ADX(14) over training bars. Add DifficultyPhase enum (Easy/Mixed/Full)
and filter_by_difficulty() for curriculum-based window filtering — Easy phase trains
only on trending markets (ADX>30), Mixed duplicates trending windows for 2x weight,
Full uses all windows equally. The curriculum_phase field in DQNHyperparameters
(committed in previous ensemble commit) controls the active phase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
16 new features computed directly in the state_gather CUDA kernel
from market data already on GPU. No CPU feature engineering.
Lookback windows: 5, 15, 60, 240 bars (≈5min, 15min, 1hr, 4hr)
Features per window:
- Return over N bars (directional bias)
- Volatility (high-low range / close)
- Volume trend (current / N-bar average)
- Momentum (position within N-bar range [0=bottom, 1=top])
State dim: 66 raw (72 aligned) without OFI, 74 raw (80 aligned) with.
The model now sees price action at 4 timescales simultaneously.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire ensemble agent configuration into DQN hyperparameters:
- ensemble_count: number of agents in ensemble (default 1 = no ensemble)
- ensemble_diversity_weight: KL-divergence diversity loss weight (default 0.01)
Both fields default to backward-compatible values (single agent, no
diversity loss) and are plumbed through the conservative() constructor.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ROOT CAUSE 1: Stop-loss 0.3%→1%, take-profit 0.5%→2% (2:1 R:R).
Old 0.3% = 8.3 ticks on ES. Normal 1-min noise is 4-6 ticks.
99.88% of trades were stopped out by NOISE, not by bad entries.
ROOT CAUSE 2: Dense shaping 0.1x→0.01x. Over 50 bars, old dense
signal = 5.0 vs completion ±2.0 — dense dominated. Now dense = 0.5
vs completion ±2.0 — trade completion is the primary signal.
ROOT CAUSE 3: Action aliasing in 5-bar hold override. When model
chose Flat but was forced to Hold, replay stored (state, Flat, Hold's
reward) — corrupting Q-values for Flat. Now overwrites out_actions
with the ACTUAL held exposure action.
ROOT CAUSE 4: Hyperopt HFT activity weight 25%→5%. Old objective
penalized selective trading. MIN_VIABLE_TRADES 100→20.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GPU experience collection is ALWAYS active in CUDA builds. The boolean
toggle was dead code — no CPU fallback exists. Removed from config,
hyperopt adapter, constructor, smoke tests, and training loop guard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IQN train_iqn_step_gpu() was doing a synchronous DtoH readback of the
loss scalar EVERY training step. This serializes the GPU pipeline.
Fix: return 0.0 placeholder from train step. Actual loss available via
read_loss() method — call only at epoch boundaries for logging.
Remaining readbacks (all once-per-epoch, acceptable):
- epoch_state: 32 bytes (DSR monitoring)
- q_stats: 20 bytes (Q-value diagnostics)
- gradient accum: dead code path (fused CUDA always active)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The compute_cvar_scales() was doing GPU→CPU→sort→CPU→GPU for quantile
CVaR computation. With batch_size=307 × 32 quantiles × 15 actions,
this was ~150KB of DtoH + sort + HtoD per epoch — likely the source
of the 10s/epoch overhead vs 3s baseline.
New inline CUDA kernel (iqn_cvar_kernel):
- One thread per sample (256 threads/block)
- Insertion sort of alpha_count smallest quantiles in registers
- Fast path for alpha_count=1 (just find minimum)
- Zero CPU readback, zero CPU allocation
Compiled once via OnceLock, cached for all subsequent calls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full Kelly implementation in CUDA kernel, no CPU path:
Enhanced Kelly = 0.7 × continuous (μ/σ²) + 0.3 × discrete ((bp-q)/b)
× confidence scaling (1 - 1/√n_trades)
× half-Kelly (0.5)
Clamped to [0.05, 0.25], normalized to position scale.
Tracks 6 statistics in portfolio state (PORTFOLIO_STRIDE 18→20):
ps[14] win_count, ps[15] loss_count, ps[16] sum_wins,
ps[17] sum_losses, ps[18] sum_returns, ps[19] sum_sq_returns
Removed CPU kelly_scale parameter — was a shortcut that violated
the zero-CPU-in-hot-path principle. All Kelly computation now
happens per-step in the env_step kernel from accumulated trade
statistics. Activates after ≥20 trade completions.
Also noted: IQN CVaR compute_cvar_scales() still does a CPU readback
for sorting quantiles. Should be a GPU kernel in next iteration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixed TP/SL produced bimodal reward (all trades hit exactly -0.3% or
+0.5%), making the reward distribution too deterministic. Model learned
"all actions produce the same bounded return" → Q-values converged.
Fix: stops scale with Q-gap conviction (0.5x to 2.0x of base levels):
- High conviction (Q-gap=2.0): SL=-0.6%, TP=+1.0% (let it run)
- Low conviction (Q-gap=0.5): SL=-0.15%, TP=+0.25% (quick exit)
- Time stop also scales: 50-200 bars based on conviction
This makes the reward distribution CONTINUOUS, not bimodal. High-conviction
entries with wider stops produce different returns than low-conviction entries
with tight stops → the model can learn that conviction matters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GpuBacktestEvaluator, GpuDqnTrainer, and hyperopt adapter had hardcoded
branch_0_size=5. Training produced 9-action indices (0-8) but backtest
only allocated 5-action buffers → SIGSEGV (exit 139) during walk-forward
evaluation after Trial 0 training.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three critical fixes from H100 Trial 0 analysis:
1. REMOVE IDLE PENALTY — was making Flat the worst Q-value (Q=-19.4),
forcing the model to trade on 95% of bars. Now Flat = reward 0.0.
DSR already penalizes inaction through the Sharpe denominator.
2. TRADE COMPLETION SCALING — ×1000 clamped to ±1 made ALL trades look
the same (can't distinguish 1-tick from 10-tick return). Now ×200
with clamp ±2, plus sqrt(hold_time) bonus for holding longer.
3. DYNAMIC TRADE MANAGEMENT — model learns ENTRY quality, exits are
managed by the system:
- Stop loss: -0.3% × vol_mult (CUSUM-adaptive, up to -0.75%)
- Take profit: +0.5% × vol_mult × trend_mult (up to +2.5% in trends)
- Time stop: 100 bars × vol_mult (up to 250 in volatile markets)
- Min hold: 5 bars before model can exit voluntarily
- Auto-exit forces flat and computes trade_completion_reward
Regime-adaptive via ADX (trend strength) and CUSUM (volatility)
features already in the GPU memory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CRITICAL: The DQN constructor had hardcoded state_dim=45/53 (old PORTFOLIO_DIM=3).
With PORTFOLIO_DIM=8, raw state_dim is 50 (no OFI) or 58 (with OFI).
The network was sized for 56 dimensions but OFI features (8 dims) were uploaded
and ignored because state_dim didn't include them.
Also fixed:
- constructor.rs: num_actions 5→9
- All GPU trainer/head defaults: state_dim 48→56
- metrics.rs: state_dim calculation
- mod.rs test: state_dim assertions
- Argo template reapplied for stale cache clearing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CRITICAL: gpu_monitoring.rs had [usize; 5] for action_counts but kernel
writes 9 exposure levels → memory corruption. Fixed array, buffer (12→16),
readback size, and the CUDA monitoring_reduce kernel (s_counts[5]→[9],
summary offsets updated). Also fixed remaining 45→81 references in
metrics.rs, training_loop.rs, monitoring.rs, config.rs comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete dual distributional pipeline:
1. After training: fused_ctx.compute_cvar_device_ptr() runs IQN forward
2. CVaR at α=5% computed per sample → [0.25, 1.0] position scaling
3. Device pointer set on collector via set_cvar_scales()
4. Next epoch: env_step kernel applies target_position *= cvar_scales[i]
C51 picks direction (argmax expected Q), IQN sizes risk (CVaR tail).
High tail risk → smaller position. Safe distribution → full size.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- experience_env_step kernel: new cvar_scales parameter (NULL = no scaling)
- target_position *= cvar_scales[i] when buffer is non-NULL
- GpuExperienceCollector: cvar_scales_ptr field + set_cvar_scales() setter
- Default: NULL (0) = no scaling until IQN CVaR is wired from training loop
The GpuIqnHead.compute_cvar_scales() produces the buffer, the collector
passes it to the kernel. Full wiring through training_loop.rs is the
remaining step — needs the collector to receive the device pointer from
the fused training context after each IQN training step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New method on GpuIqnHead: samples τ values, runs forward-only kernel,
computes CVaR at α=5% per sample, returns [0.25, 1.0] scaling factors.
CVaR ≥ 0 (safe) → full position. CVaR < 0 (risky) → reduced position.
Next: wire into env_step kernel for position scaling during experience
collection, and into fused_training.rs to call after IQN training step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
use_qr_dqn was a hacky toggle that gated the IQN dual-head behind
a num_atoms threshold. IQN is now always enabled when iqn_lambda > 0
(default 0.25). C51 remains the main loss; IQN is the auxiliary head
for CVaR risk quantification.
Removed use_qr_dqn from: DQNParams, DQNHyperparameters, fused_training,
hyperopt adapter, all tests, all examples.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Updated all fixed-size arrays across monitoring.rs, financials.rs,
metrics.rs, training_loop.rs from [_;5]→[_;9] and [_;45]→[_;81].
DIRECTION_LUT expanded to 9 levels (-1.0 to +1.0 in 0.25 steps).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1-lot fills at bid/ask. 4-lot moves the market.
impact_scale = 1 + (|delta|/max_position)^2, ranges [1.0, 2.0].
Teaches the model that max-size positions are 2x more expensive to enter.
Makes the 9-exposure granularity meaningful — 25% positions are cheaper
to enter/exit than 100% positions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PDT $25K rule protection: if equity drops below 75% of peak_equity,
the episode terminates (done=1) with a -1.0 penalty reward and forced
flat. The model learns that approaching the floor = game over.
Uses peak_equity (not initial_capital) so it adapts as account grows.
With $35K initial, floor triggers at ~$26.25K — above the $25K minimum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
