Replace hardcoded 0.5f flip probability in experience_env_step kernel with
dynamic cf_ratio = clamp(0.5 + 0.3*(1-health), 0.0, 1.0). Healthy (h=1)
keeps cf_ratio=0.5; collapsed (h=0) raises to 0.8 for more counterfactual
exposure to break collapse. Propagated via set_learning_health() each epoch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Track consecutive epochs with health < 0.3 in `unhealthy_epoch_count`; after 3
consecutive unhealthy epochs, trigger shrink_and_perturb(alpha, sigma) and reset
the counter. Remove the periodic interval-based trigger (every N epochs). The
Phase 3 boundary trigger is intentionally kept as an independent mechanism.
last_plasticity_ready: Some(true) = accumulating, Some(false) = just triggered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Compute barrier_loss = 0.5 × max(0, 0.05×health − q_gap)² on the host
from cached q_gap_ema. Written to last_barrier_loss (already declared by
A4) and surfaced in HEALTH_DIAG `barrier=...`. Scalar-only / no gradient.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When learning_health < 0.8, the PER priority update switches from the
standard per_update_pa kernel to pow_alpha_diverse_f32, which multiplies
each priority by (1 + 2*(1-health)*|action - mean_action|). This rescues
the replay buffer's diversity signal during Q-collapse by surfacing
experiences whose action deviates from the batch mean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace static cql_alpha with cql_alpha_eff computed from ISV signals:
health (ISV[12]) × (1 − regime_stability (ISV[11])) × base.
Collapse→0 (CQL off); volatile+healthy→full; stable+healthy→0.
Adds last_cql_alpha_eff field to GpuDqnTrainer (f32, init 0.0),
accessor on FusedTrainingCtx, and propagation into DQNTrainer for
HEALTH_DIAG logging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds 15 new last_* fields (all None) to DQNTrainer for B/C/D tasks to populate,
and replaces the A3 HEALTH_DIAG log line with the full format covering components,
effective hyperparams, and novel mechanism states.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add HealthEmaTrackers struct (metrics.rs): EMA for q_gap/q_var/grad_norm with
scalar grad_consistency proxy (successive grad_norm delta ratio)
- Add LearningHealth + HealthEmaTrackers + 5 last_* fields to DQNTrainer (mod.rs)
- Initialize new fields in constructor.rs
- Add write_isv_signal_at, read_atom_utilization, compute_q_spectral_gap to
GpuDqnTrainer (gpu_dqn_trainer.rs) with coarse max/min spectral-gap proxy
- Forward same three methods on FusedTrainingCtx (fused_training.rs)
- Compute LearningHealth in process_epoch_boundary (training_loop.rs): reads
per_branch_q_gap_ema, epoch min/max, atom_util, spectral_gap; emits HEALTH_DIAG log;
broadcasts health_value to ISV[12] via write_isv_signal_at
Known proxy substitutions (documented in training_loop.rs comment):
- grad_consistency: scalar proxy instead of full Adam vector cosine (per-component buffers)
- spectral_gap: coarse max/min ratio on q_readback_pinned instead of SVD
- ens_disagreement: 0.1 placeholder until D6/N6
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Layer 2 labeled as G2-G5, G5 explicit for gradient budget
- P2 spectral_gap_norm added as 7th component in composition (rebalanced weights)
- N6 ensemble oracle has explicit threshold (>0.8 triggers N3 immediately)
- N8 meta-Q wiring clarified: logged but not in composition until validated
- Files Changed deduplicated, paths qualified
- Success criteria: WinRate >55% (above random baseline ~25%)
- HEALTH_DIAG log line includes all 7 components
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes Q-value collapse via unified LearningHealth signal that senses
training health (6 components) and continuously adapts:
- CQL regularization (regime + health gated)
- Gradient budget (IQN/CQL/Ens/C51 dynamic allocation)
- Tau target EMA (health-coupled)
- Expected SARSA temperature (continuous, no hardcoded threshold)
Plus 4 pearls (PER priorities, spectral detection, gradient consistency,
adaptive gamma) and 8 novels (self-distillation, barrier loss, plasticity
injection, CF curriculum, information bottleneck, ensemble oracle,
contrarian override, meta-Q network).
Core principle: training hyperparameters are OUTPUTS of the temporal
pipeline, not static schedules. The system meta-learns its own settings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Task 3 refactored experience_state_gather (the WRITER) to use assemble_state()
with canonical layout (OFI at [42..62)). But env_step and ofi_embed_build_input
(the READERS) still hardcoded OFI at [66..84) — the OLD pre-refactor layout.
This meant 7 locations were reading MTF/portfolio features as if they were OFI:
1. Line 1745: dense micro-reward ofi_cur = state+66 → actually MTF[4]
2. Line 1778: book_aggression = state[82] → actually plan_isv region
3. Line 1983: ps[30..37] OFI delta storage for NEXT bar — storing MTF data
4. Lines 5833/5836/5838/5840: ofi_embed_build_input — feeds 18→10 MLP into
Mamba2 temporal SSM and attention. Entire temporal pipeline was training
on MTF features dressed as OFI.
Symptoms explained:
- WinRate=20.9% on validation (anti-correlated): dense micro-reward computes
quality=sign_pos × garbage_MTF_deltas, systematically rewarding wrong direction
- mean_reward=+0.004 but Sharpe_raw=-0.0004: shaped reward exploits garbage
signal, real portfolio loses money
- grad_norm=23560 at epoch 2: gradients chasing noise
- Q-value explosion to ±10 in one epoch: learning contradictions
Fix: replaced all hardcoded 66/74/82/83 with SL_OFI_START from state_layout.cuh.
Both reader kernels now use the same canonical layout as assemble_state().
Verified locally: smoke test passes, OFI_DIAG shows correct non-zero values
(raw_mean=-0.36, delta_mean=-0.21, log_dur=-0.23).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the canonical state layout at all 5 sections (market/OFI/MTF/portfolio/padding).
If any section drifts to the wrong offset, this test fails. Since training and
backtest share assemble_state() in state_layout.cuh, a passing test guarantees
both paths produce identical layouts.
Result: ✓ market=41 ofi=9 mtf=9 portfolio=5 padding=all-zero
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- OFI_DIAG now reads positions [42..62) using OFI_START constant instead of
hardcoded [66..74). Verified: raw_mean=0.0891, delta_mean=-0.0370,
book_agg=0.4500, log_dur=-0.2303 (was all zeros before).
- Removed 3 stale state_dim field initializers from evaluate_baseline.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 call sites to GpuExperienceCollector::new() were missing the 12th
argument (curiosity_weight). Pre-existing test issue surfaced during
Task 4 verification. Fixed by passing 0.0 (disabled in tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kernel now writes to local arrays (market[], portfolio[], plan_isv[],
mtf[], ofi[]) then calls assemble_state() from state_layout.cuh to produce
the canonical layout. This eliminates the hardcoded ofi_start=66 that
collided with MTF features and ensures training/validation use identical
state vectors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Training and validation use different kernels with different state layouts,
making generalization impossible. Single state_layout.cuh + one assembly
function + STATE_DIM as compile-time constant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diagnostics served their purpose — confirmed OFI flows through full
pipeline on H100 (state_gather → PER → trainer). Remove:
- OFI host verify readback (upload_ofi_features)
- STATE_GATHER_DIAG pinned memory readback (timestep loop)
- Dead bf16 scatter_insert kernel (states are f32, was never called)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
scatter_insert_f32 is a 1D scalar kernel (5 args: dst, src, cursor, cap,
batch_size). insert_batch called it with 6 args for state matrices, passing
state_dim as batch_size. CUDA silently dropped the 6th arg (actual batch_size).
Result: only 96 floats (1 state row) inserted per experience batch into PER.
The model was training on ~99.99% uninitialized GPU memory. This bug affected
ALL state features, not just OFI — market features and portfolio were also
garbage in PER-sampled training batches.
Fix: added scatter_insert_f32_rows kernel (2D-aware, 6 args: dst, src,
cursor, cap, state_dim, batch_size) matching the existing scatter_insert
(bf16) pattern. States and next_states now use the row-aware kernel.
Verified locally: OFI_DIAG shows non-zero values through the full chain
(state_gather → env_step → PER insert → PER sample → trainer states_buf).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses PinnedHostBuf + cuMemcpyDtoHAsync for state_gather diagnostic.
Reads batch_states[0..state_dim] to verify OFI at positions [66..84).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Host-side check (zero-cost) before clone_htod to confirm data isn't
zero before it reaches the GPU. Fixes crash from previous diagnostic
(memcpy_dtoh size mismatch).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Temporary diagnostic: readback ofi_gpu[0..20] after upload and
batch_states[66..84] after first state_gather. OFI works locally on
RTX 3050 but shows zeros on H100 with identical v4 cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The PVC had a v2 cache that passed has_ofi and nonzero checks but contained
stale OFI data computed by an old binary. load_fxcache accepted v2/v3 for
backward compat, keeping the stale cache alive across every deploy.
v4 is the only valid version. Legacy loading code removed (-50 lines).
Argo ensure-fxcache will delete the v2 cache and rebuild from 148GB MBP-10.
Verified locally: v4 cache produces OFI_DIAG raw_mean=-0.2391 (non-zero).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ofi_gpu is unconditional — wrapping in Option allowed silent NULL pointer
fallback to kernel. Now a plain CudaSlice<f32>.
total_bars default was 10,000 (a lie) — must come from actual data length.
Default changed to 0 with hard error if not set before collect.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The v2 fxcache on PVC passed has_ofi=true validation (mbp10_dir was present
when built) but contained all-zero OFI data. The old binary set the flag
based on directory existence, not actual computed values.
Now counts non-zero OFI rows before accepting a cache — forces rebuild
when MBP-10 data is available but OFI content is all zeros.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OFI (Order Flow Imbalance) features are mandatory — the model is worthless
without MBP-10 order book data. Every silent fallback that degraded to zeros
has been converted to a hard error.
Critical fixes:
- build_batch_states used *8 instead of *20 for OFI dimensions — every
walk-forward backtest was reading corrupted OFI features from adjacent memory
- precompute_features early exit skipped cache rebuild when stale v2/v3
cache existed with has_ofi=false — now validates has_ofi before skipping
- 14 silent OFI fallback paths converted to hard errors across data loading,
training loop, experience collector, state construction, metrics, hyperopt
Dead code removed (-751 lines):
- DoubleBufferedLoader (superseded by init_from_fxcache)
- GpuBufferPool (superseded by init_from_fxcache)
- DqnGpuData::upload legacy method (no OFI support)
- CPU training fallback path (CUDA always required)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The experience_state_gather kernel assembled states from market_features
+ portfolio but NEVER included OFI features. The ofi_gpu buffer was
uploaded to GPU but never passed to the gather kernel. Comment at line
693 said "appended after this kernel" but that code was never written.
Fix: add ofi_features + ofi_dim parameters to the kernel. Writes OFI
at state[66..84): raw_ofi(8) + delta_ofi(8) + book_aggression(1) +
log_duration(1). Now verified non-zero via diagnostic:
OFI_DIAG: raw_mean=0.099, delta_mean=2.901, book_agg=-0.001, log_dur=0.526
This was THE fundamental data pipeline bug — the OFI embed MLP, Mamba2
enrichment, attention enrichment, and dense micro-reward all read zeros
for OFI features during training. Now they get real order flow data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs in attention LN backward:
1. attn_layer_norm_bwd_dx: Rust passed 8 args (old monolithic kernel
signature) but the split kernel takes 6. Args 2-3 (saved_input,
projected_buf) shifted gamma/rstd/d_x/D/B → total corruption.
Fixed: removed extra args, kernel now receives correct 6 args.
2. attn_layer_norm_bwd_dgamma_p1: passed saved_input (D+10 wide) as
LN residual, but kernel indexes with D stride → OOB access.
Fixed: save states→ln_residual [D,B] during forward via copy_f32
kernel (graph-safe). Backward reads D-wide ln_residual correctly.
Root cause: Phase 2 split of monolithic LN kernel into 3 parts didn't
update the Rust arg lists. The attention widening (D→D+10) exposed the
mismatch as CUDA_ERROR_ILLEGAL_ADDRESS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Atom warm-start: bitonic sort rewards → quantile positions → write to
atom_positions_buf as initialization. Existing SGD optimizer refines.
Atoms start where reward mass actually is instead of uniform [-50,+50].
Robust PopArt: median/IQR normalization from sorted rewards replaces
Welford mean/var. More robust for bimodal distribution (many ±0.1
micro-rewards + few ±5.0 trade exits). Conditional: popart_robust=true.
Both reuse the same bitonic sort (~14ms per epoch, amortized).
gather_quantiles kernel extracts positions. extract_median_iqr reads
Q25/median/Q75 from sorted buffer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Volatile regime → price confirmation weighted higher (momentum matters)
Stable regime → book aggression + hold quality weighted higher
Regime transition → hold quality drops (don't hold through shifts)
Uses ISV signals[11] (regime_stability) and signals[2] (volatility)
already available in env_step_batch. Zero new parameters — the temporal
attention pipeline drives reward composition through ISV routing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Counterfactual: Hold(1)/Flat(3) direction mirror maps to self — trivial.
Now falls through to magnitude CF for Hold/Flat instead of wasting a
replay buffer slot on same-action same-reward experiences.
reward_noise_scale: 0.05→0.0 (dense micro-rewards are already noisy,
adding 5% label noise destroys the per-bar signal).
Added micro-reward weight config fields (price_confirm_weight=0.5,
book_aggression_weight=0.3, hold_quality_weight=0.2, micro_reward_temp=3.0,
holding_cost_rate=0.0001) — defined in TOML, kernel plumbing next session.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
