- Layer 2 labeled as G2-G5, G5 explicit for gradient budget
- P2 spectral_gap_norm added as 7th component in composition (rebalanced weights)
- N6 ensemble oracle has explicit threshold (>0.8 triggers N3 immediately)
- N8 meta-Q wiring clarified: logged but not in composition until validated
- Files Changed deduplicated, paths qualified
- Success criteria: WinRate >55% (above random baseline ~25%)
- HEALTH_DIAG log line includes all 7 components
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes Q-value collapse via unified LearningHealth signal that senses
training health (6 components) and continuously adapts:
- CQL regularization (regime + health gated)
- Gradient budget (IQN/CQL/Ens/C51 dynamic allocation)
- Tau target EMA (health-coupled)
- Expected SARSA temperature (continuous, no hardcoded threshold)
Plus 4 pearls (PER priorities, spectral detection, gradient consistency,
adaptive gamma) and 8 novels (self-distillation, barrier loss, plasticity
injection, CF curriculum, information bottleneck, ensemble oracle,
contrarian override, meta-Q network).
Core principle: training hyperparameters are OUTPUTS of the temporal
pipeline, not static schedules. The system meta-learns its own settings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Task 3 refactored experience_state_gather (the WRITER) to use assemble_state()
with canonical layout (OFI at [42..62)). But env_step and ofi_embed_build_input
(the READERS) still hardcoded OFI at [66..84) — the OLD pre-refactor layout.
This meant 7 locations were reading MTF/portfolio features as if they were OFI:
1. Line 1745: dense micro-reward ofi_cur = state+66 → actually MTF[4]
2. Line 1778: book_aggression = state[82] → actually plan_isv region
3. Line 1983: ps[30..37] OFI delta storage for NEXT bar — storing MTF data
4. Lines 5833/5836/5838/5840: ofi_embed_build_input — feeds 18→10 MLP into
Mamba2 temporal SSM and attention. Entire temporal pipeline was training
on MTF features dressed as OFI.
Symptoms explained:
- WinRate=20.9% on validation (anti-correlated): dense micro-reward computes
quality=sign_pos × garbage_MTF_deltas, systematically rewarding wrong direction
- mean_reward=+0.004 but Sharpe_raw=-0.0004: shaped reward exploits garbage
signal, real portfolio loses money
- grad_norm=23560 at epoch 2: gradients chasing noise
- Q-value explosion to ±10 in one epoch: learning contradictions
Fix: replaced all hardcoded 66/74/82/83 with SL_OFI_START from state_layout.cuh.
Both reader kernels now use the same canonical layout as assemble_state().
Verified locally: smoke test passes, OFI_DIAG shows correct non-zero values
(raw_mean=-0.36, delta_mean=-0.21, log_dur=-0.23).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the canonical state layout at all 5 sections (market/OFI/MTF/portfolio/padding).
If any section drifts to the wrong offset, this test fails. Since training and
backtest share assemble_state() in state_layout.cuh, a passing test guarantees
both paths produce identical layouts.
Result: ✓ market=41 ofi=9 mtf=9 portfolio=5 padding=all-zero
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- OFI_DIAG now reads positions [42..62) using OFI_START constant instead of
hardcoded [66..74). Verified: raw_mean=0.0891, delta_mean=-0.0370,
book_agg=0.4500, log_dur=-0.2303 (was all zeros before).
- Removed 3 stale state_dim field initializers from evaluate_baseline.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 call sites to GpuExperienceCollector::new() were missing the 12th
argument (curiosity_weight). Pre-existing test issue surfaced during
Task 4 verification. Fixed by passing 0.0 (disabled in tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kernel now writes to local arrays (market[], portfolio[], plan_isv[],
mtf[], ofi[]) then calls assemble_state() from state_layout.cuh to produce
the canonical layout. This eliminates the hardcoded ofi_start=66 that
collided with MTF features and ensures training/validation use identical
state vectors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Training and validation use different kernels with different state layouts,
making generalization impossible. Single state_layout.cuh + one assembly
function + STATE_DIM as compile-time constant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diagnostics served their purpose — confirmed OFI flows through full
pipeline on H100 (state_gather → PER → trainer). Remove:
- OFI host verify readback (upload_ofi_features)
- STATE_GATHER_DIAG pinned memory readback (timestep loop)
- Dead bf16 scatter_insert kernel (states are f32, was never called)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
scatter_insert_f32 is a 1D scalar kernel (5 args: dst, src, cursor, cap,
batch_size). insert_batch called it with 6 args for state matrices, passing
state_dim as batch_size. CUDA silently dropped the 6th arg (actual batch_size).
Result: only 96 floats (1 state row) inserted per experience batch into PER.
The model was training on ~99.99% uninitialized GPU memory. This bug affected
ALL state features, not just OFI — market features and portfolio were also
garbage in PER-sampled training batches.
Fix: added scatter_insert_f32_rows kernel (2D-aware, 6 args: dst, src,
cursor, cap, state_dim, batch_size) matching the existing scatter_insert
(bf16) pattern. States and next_states now use the row-aware kernel.
Verified locally: OFI_DIAG shows non-zero values through the full chain
(state_gather → env_step → PER insert → PER sample → trainer states_buf).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses PinnedHostBuf + cuMemcpyDtoHAsync for state_gather diagnostic.
Reads batch_states[0..state_dim] to verify OFI at positions [66..84).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Host-side check (zero-cost) before clone_htod to confirm data isn't
zero before it reaches the GPU. Fixes crash from previous diagnostic
(memcpy_dtoh size mismatch).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Temporary diagnostic: readback ofi_gpu[0..20] after upload and
batch_states[66..84] after first state_gather. OFI works locally on
RTX 3050 but shows zeros on H100 with identical v4 cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The PVC had a v2 cache that passed has_ofi and nonzero checks but contained
stale OFI data computed by an old binary. load_fxcache accepted v2/v3 for
backward compat, keeping the stale cache alive across every deploy.
v4 is the only valid version. Legacy loading code removed (-50 lines).
Argo ensure-fxcache will delete the v2 cache and rebuild from 148GB MBP-10.
Verified locally: v4 cache produces OFI_DIAG raw_mean=-0.2391 (non-zero).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ofi_gpu is unconditional — wrapping in Option allowed silent NULL pointer
fallback to kernel. Now a plain CudaSlice<f32>.
total_bars default was 10,000 (a lie) — must come from actual data length.
Default changed to 0 with hard error if not set before collect.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The v2 fxcache on PVC passed has_ofi=true validation (mbp10_dir was present
when built) but contained all-zero OFI data. The old binary set the flag
based on directory existence, not actual computed values.
Now counts non-zero OFI rows before accepting a cache — forces rebuild
when MBP-10 data is available but OFI content is all zeros.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OFI (Order Flow Imbalance) features are mandatory — the model is worthless
without MBP-10 order book data. Every silent fallback that degraded to zeros
has been converted to a hard error.
Critical fixes:
- build_batch_states used *8 instead of *20 for OFI dimensions — every
walk-forward backtest was reading corrupted OFI features from adjacent memory
- precompute_features early exit skipped cache rebuild when stale v2/v3
cache existed with has_ofi=false — now validates has_ofi before skipping
- 14 silent OFI fallback paths converted to hard errors across data loading,
training loop, experience collector, state construction, metrics, hyperopt
Dead code removed (-751 lines):
- DoubleBufferedLoader (superseded by init_from_fxcache)
- GpuBufferPool (superseded by init_from_fxcache)
- DqnGpuData::upload legacy method (no OFI support)
- CPU training fallback path (CUDA always required)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The experience_state_gather kernel assembled states from market_features
+ portfolio but NEVER included OFI features. The ofi_gpu buffer was
uploaded to GPU but never passed to the gather kernel. Comment at line
693 said "appended after this kernel" but that code was never written.
Fix: add ofi_features + ofi_dim parameters to the kernel. Writes OFI
at state[66..84): raw_ofi(8) + delta_ofi(8) + book_aggression(1) +
log_duration(1). Now verified non-zero via diagnostic:
OFI_DIAG: raw_mean=0.099, delta_mean=2.901, book_agg=-0.001, log_dur=0.526
This was THE fundamental data pipeline bug — the OFI embed MLP, Mamba2
enrichment, attention enrichment, and dense micro-reward all read zeros
for OFI features during training. Now they get real order flow data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs in attention LN backward:
1. attn_layer_norm_bwd_dx: Rust passed 8 args (old monolithic kernel
signature) but the split kernel takes 6. Args 2-3 (saved_input,
projected_buf) shifted gamma/rstd/d_x/D/B → total corruption.
Fixed: removed extra args, kernel now receives correct 6 args.
2. attn_layer_norm_bwd_dgamma_p1: passed saved_input (D+10 wide) as
LN residual, but kernel indexes with D stride → OOB access.
Fixed: save states→ln_residual [D,B] during forward via copy_f32
kernel (graph-safe). Backward reads D-wide ln_residual correctly.
Root cause: Phase 2 split of monolithic LN kernel into 3 parts didn't
update the Rust arg lists. The attention widening (D→D+10) exposed the
mismatch as CUDA_ERROR_ILLEGAL_ADDRESS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Atom warm-start: bitonic sort rewards → quantile positions → write to
atom_positions_buf as initialization. Existing SGD optimizer refines.
Atoms start where reward mass actually is instead of uniform [-50,+50].
Robust PopArt: median/IQR normalization from sorted rewards replaces
Welford mean/var. More robust for bimodal distribution (many ±0.1
micro-rewards + few ±5.0 trade exits). Conditional: popart_robust=true.
Both reuse the same bitonic sort (~14ms per epoch, amortized).
gather_quantiles kernel extracts positions. extract_median_iqr reads
Q25/median/Q75 from sorted buffer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Volatile regime → price confirmation weighted higher (momentum matters)
Stable regime → book aggression + hold quality weighted higher
Regime transition → hold quality drops (don't hold through shifts)
Uses ISV signals[11] (regime_stability) and signals[2] (volatility)
already available in env_step_batch. Zero new parameters — the temporal
attention pipeline drives reward composition through ISV routing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Counterfactual: Hold(1)/Flat(3) direction mirror maps to self — trivial.
Now falls through to magnitude CF for Hold/Flat instead of wasting a
replay buffer slot on same-action same-reward experiences.
reward_noise_scale: 0.05→0.0 (dense micro-rewards are already noisy,
adding 5% label noise destroys the per-bar signal).
Added micro-reward weight config fields (price_confirm_weight=0.5,
book_aggression_weight=0.3, hold_quality_weight=0.2, micro_reward_temp=3.0,
holding_cost_rate=0.0001) — defined in TOML, kernel plumbing next session.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed the legacy non-branching Sequential q_network and target_network
from DQNAgent. These were never used (branching+dueling always active)
but allocated VRAM and ran noise resets every step. -190 lines.
Config tuning for dense micro-reward system:
- n_steps: 5→1 (TD(0), micro-rewards cancel over n>1)
- tau: 0.007→0.01 (faster target tracking for TD(0))
- c51_alpha_max: 0.5→1.0 (full C51, PopArt handles normalization)
- curiosity_weight: 0.1→0.0 (dense micro-reward replaces curiosity)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FXCACHE_VERSION 3→4. Three precomputed features added to OFI region:
- ofi[8..16): temporal deltas (ofi[bar] - ofi[bar-1]) for 8 features
- ofi[16]: book aggression (MBP-10 10-level center-of-mass asymmetry)
- ofi[17]: log bar duration (imbalance bar formation time, normalized)
Previously 10 of 18 OFI embed MLP inputs were zero. Now all 18 have
real data: raw_ofi(8) + delta_ofi(8) + book_aggression(1) + log_duration(1).
Added read_state_sample() diagnostic for GPU state verification.
Legacy v3 fxcache handled by zeroing new slots (v2→v4 graceful upgrade).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three signal-killing issues fixed:
1. Rank normalization DISABLED — was double-normalizing with PopArt,
destroying magnitude difference between micro-rewards (±0.1) and
trade exits (±5.0). PopArt alone preserves relative magnitude.
2. n_steps 5→1 (TD(0)) — dense micro-rewards alternate ±0.1 each bar.
With n=5, they cancel out over 5 bars. TD(0) preserves the per-bar
signal that the temporal pipeline needs to learn from.
3. per_alpha 0.6→0.3 — lower = more uniform PER sampling. Dense micro-
rewards have tiny TD-errors (easy to predict), so high alpha ignores
them. Lower alpha ensures micro-reward experiences get sampled.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Linear scaling rule (2x batch → 2x LR) doesn't hold for DQN+PER with
major architectural changes (Hold action, OFI embed, wider attention).
The model needs stability to learn the new action space, not speed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Attention W_O was Xavier-initialized, injecting ~sqrt(D) magnitude noise
into the residual connection. With the DtoD→copy_f32 fix making attention
active (was a no-op before), random W_O corrupted all branch head inputs
→ MaxDD=100% on validation.
Fix: W_O zero-init → attention starts as identity (output ≈ h_s2).
W_Q/K/V stay Xavier (they project to SDP space, not the residual).
OFI embed weights also zero-init → Mamba2 history starts as [h_s2; 0]
and attention input starts as [h_s2; 0]. No noise injection at init.
Both learn from zero as gradients shape them toward useful patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hardcoded 12 Q-values in pinned readback buffer and per-branch Q-gap
slice caused panic with b0=4 (Hold action: 4+3+3+3=13 total actions).
Now uses dynamic total_actions from config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Widens W_Q/W_K/W_V from [D,D] to [D,D+10]. OFI embed (10-dim) is
concatenated to attention input via attn_concat_input kernel. Cross-
feature attention now sees order flow dynamics alongside trunk features.
Backward: d_input_scratch widened to [D+10,B]. extract_attn_ofi_grad
kernel extracts the OFI gradient portion. d_ofi_embed_attn buffer
exposed via accessor for Task 8 gradient accumulation.
W_O stays [D,D], LayerNorm stays D-wide — only input projection is
affected. SDP operates on Q/K/V which remain [D,B].
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2 new cuBLAS GEMMs: d_h_history = W_A^T @ d_gate + W_B^T @ d_x.
Extracts last 10 dims as d_ofi_embed_mamba2, accumulated across K=8
timesteps. This enables the OFI embed MLP to learn from temporal
patterns discovered by the Mamba2 SSM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
h_history widened from [B, K, SH2] to [B, K, SH2+10]. OFI embed (10-dim)
appended per timestep. W_A/W_B projections grow to [SH2+10, STATE_D].
W_C stays [SH2, STATE_D] — operates on scan output, not history input.
The SSM temporal scan now sees order flow momentum, book aggression, and
bar duration alongside trunk features. Enables learning sequential
patterns in microstructure dynamics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extracts [raw_ofi(8); delta_ofi(8); book_aggression(1); log_duration(1)]
= 18-dim from state vector, compresses to 10-dim via cuBLAS SGEMM +
bias+ReLU. Xavier init. Runs in training forward path before Mamba2
and attention, making the embedding available for temporal enrichment.
190 trainable params (18×10 + 10 bias). ~0.05ms per step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Action space 81→108: direction(4) × magnitude(3) × order(3) × urgency(3).
Short(0), Hold(1), Long(2), Flat(3). Hold keeps current position with
zero transaction cost, giving the model a "do nothing" option.
Changes: trade_physics.cuh (Hold returns current_position), env_step
(Hold skips trade execution), action.rs (ExposureLevel::Hold variant),
config (branch_0_size 3→4), all match arms updated across 11 files.
Counterfactual mirror: Short↔Long, Hold↔Hold, Flat↔Flat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
memcpy_dtod_async is NOT captured by CUDA Graph — silently skipped
during replay. This caused:
- Attention reading stale trunk data (h_s2→scratch copy skipped)
- Attention output never written back (scratch→h_s2 copy skipped)
- Backward gradients using stale logits (d_logits staging skipped)
- Vaccine comparison using stale reference (prev_grad snapshot skipped)
Attention has been effectively a NO-OP since CUDA Graph was introduced.
Fix: new copy_f32 kernel in graph_utility_kernels.cu (same pattern as
existing mamba2_copy_enriched). 11 call sites replaced across
gpu_dqn_trainer.rs and fused_training.rs. graph_safe_copy_f32() and
graph_safe_copy_f32_on() helpers for single-stream and multi-stream.
Non-graph-captured DtoD copies (checkpoint, init, eval) left as-is.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
100 bars = ~1 hour of imbalance bars. ES futures regimes persist 4-8
hours. Val_Sharpe was artificially inflated by regime autocorrelation
bleeding through the too-short purge gap. 2000 bars = ~1 full trading
day, ensuring validation data is from a different regime.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3 pre-existing bugs fixed:
- ofi_gpu uploaded but never passed to env_step — now wired via
full_batch_states param (OFI at state[66..74) accessible in kernel)
- concat_ofi_features read indices 42/45 (portfolio features) instead
of 66/69 (actual OFI location in 96-dim state vector)
- PORTFOLIO_STRIDE expanded 30→38 for prev-OFI delta storage (ps[30..37])
Added PREV_CLOSE_SLOT=6 and PREV_MID_SLOT=22 for MFT mark-to-market.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
target_dim expansion: adds raw_open (OHLCV) and mid_price_open
(MBP-10 midpoint at bar formation) to fxcache targets. FXCACHE_VERSION
2→3 for auto-rebuild. Legacy v2 files handled with close-price fallback.
Spec v5 adds 3 pearls:
- Bar duration encoding in Mamba2 (continuous-time SSM awareness)
- Order book center of mass from all 10 MBP-10 levels (aggression signal)
- Retrospective hold quality bonus (teaches exit timing)
Plus: Hold action (4th direction), DSR Sharpe EMA fix, counterfactual
magnitude/order sign fix, MFT mid-price mark-to-market.
OFI embed MLP now 18→10 (was 16→8). Mamba2 width SH2+10 (was SH2+8).
Attention width D+10 (was D+8).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
