Add `barrier_gradient_direction` CUDA kernel to c51_loss_kernel.cu that
computes barrier = max(0, 0.05*health - q_gap) from the direction branch
logits and ISV[12], then injects gradient via atomicAdd into the CQL
d-logit accumulator buffers. When barrier > 0, it raises Q(argmax) and
lowers Q(second_max) to widen the direction Q-gap.
Wire-up: `apply_cql_gradient` now accepts `barrier_weight` and inlines
the kernel launch after the CQL kernel but before `backward_full`, so
both CQL and barrier share one cuBLAS backward pass with no extra SAXPY.
`submit_aux_ops` passes `barrier_weight = 0.05` every step (kernel is
internally a no-op when q_gap >= min_req). D2/N2 epoch-boundary block
updated to reflect that gradient is now live.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace single-sample max/min proxy with mathematically proper sigma_1/sigma_2
ratio. Maintains a host-side VecDeque of up to 64 Q-value samples; once ≥ 8
samples are available computes the n_cols×n_cols Gram matrix X^T X, then
extracts the two largest singular values via power iteration + rank-one
deflation. Falls back to the coarse max/min ratio until the buffer fills.
compute_q_spectral_gap changed to &mut self; callers updated accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. D6 ensemble oracle no longer dormant — replaces hardcoded ens_disagreement=0.1
with range of per-branch Q-gap EMAs (max - min over 4 branches). This gives a
real signal that collapses to 0 when branches agree on uniform Q and expands
to non-trivial values when branches preserve diverse action differentiation.
D6's smoothstep(0.01, 0.1) window now actually moves.
2. HEALTH_DIAG defaults aligned with constructor inits — cql_budget default
0.10→0.00, c51_budget 0.45→0.55. Only visible when fused_ctx is None
(pre-first-step); eliminates the cosmetic log jump on the first real epoch.
3. reward_v8 smoke-test module changed from pub mod → mod, resolving the
pre-existing unreachable_pub warning. No external consumers of the module.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trains a 10-epoch DQN run on fxcache data and asserts q_gap_ema > 0.05 and
learning_health.value > 0.3 — regression guard against the Q-uniform collapse
attractor the entire adaptive-learning-dynamics spec is designed to prevent.
Run via:
FOXHUNT_TEST_DATA=test_data/futures-baseline \\
cargo test -p ml --lib -- test_adaptive_learning_no_collapse --ignored --nocapture
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses reviewer's IMPORTANT flag: the conviction Q-gap was not sign-flipped when
contrarian_active, so downstream env_step would size contrarian (argmin) positions
with maximum conviction, amplifying losses. During contrarian override, set
out_q_gaps=0 so those trades size minimally — the override is deliberate and
temporary; we don't want to compound it with aggressive position sizing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `contrarian_active` int parameter to `experience_action_select` kernel.
When non-zero, a `q_sign = -1.0f` multiplier is applied to Q values across
all 4 branches (direction/magnitude/order/urgency) before Boltzmann softmax,
converting argmax-favoring sampling to argmin-favoring without touching
temperature, epsilon, conviction filter, masking, or sampling logic.
When zero, q_sign = +1.0f — behavior is bit-identical to before.
Wire: GpuExperienceCollector gains `contrarian_active_cache: u8` field plus
`set_contrarian_active()` / `contrarian_active()` accessors. The flag is
appended as the last arg at the action_select launch site. training_loop.rs
propagates `self.contrarian_active` to the collector immediately after the
D7 Part A state machine updates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Part A fully implemented: state machine tracks low_winrate_count across epochs,
activates contrarian_active for 2 epochs when WinRate<40% × 5 consecutive AND
health<0.3, deactivates on WinRate>=45% recovery. last_epoch_win_rate carries
financials.win_rate (f32 cast) from log_epoch_metrics_and_financials into next
process_epoch_boundary. HEALTH_DIAG already consumes last_contrarian_active as
contrarian=on/off. Part B (kernel argmin flip) deferred: direction selection uses
Boltzmann softmax, not simple argmax, making inversion non-trivial.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace hardcoded 0.5f flip probability in experience_env_step kernel with
dynamic cf_ratio = clamp(0.5 + 0.3*(1-health), 0.0, 1.0). Healthy (h=1)
keeps cf_ratio=0.5; collapsed (h=0) raises to 0.8 for more counterfactual
exposure to break collapse. Propagated via set_learning_health() each epoch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Track consecutive epochs with health < 0.3 in `unhealthy_epoch_count`; after 3
consecutive unhealthy epochs, trigger shrink_and_perturb(alpha, sigma) and reset
the counter. Remove the periodic interval-based trigger (every N epochs). The
Phase 3 boundary trigger is intentionally kept as an independent mechanism.
last_plasticity_ready: Some(true) = accumulating, Some(false) = just triggered.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Compute barrier_loss = 0.5 × max(0, 0.05×health − q_gap)² on the host
from cached q_gap_ema. Written to last_barrier_loss (already declared by
A4) and surfaced in HEALTH_DIAG `barrier=...`. Scalar-only / no gradient.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When learning_health < 0.8, the PER priority update switches from the
standard per_update_pa kernel to pow_alpha_diverse_f32, which multiplies
each priority by (1 + 2*(1-health)*|action - mean_action|). This rescues
the replay buffer's diversity signal during Q-collapse by surfacing
experiences whose action deviates from the batch mean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace static cql_alpha with cql_alpha_eff computed from ISV signals:
health (ISV[12]) × (1 − regime_stability (ISV[11])) × base.
Collapse→0 (CQL off); volatile+healthy→full; stable+healthy→0.
Adds last_cql_alpha_eff field to GpuDqnTrainer (f32, init 0.0),
accessor on FusedTrainingCtx, and propagation into DQNTrainer for
HEALTH_DIAG logging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds 15 new last_* fields (all None) to DQNTrainer for B/C/D tasks to populate,
and replaces the A3 HEALTH_DIAG log line with the full format covering components,
effective hyperparams, and novel mechanism states.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add HealthEmaTrackers struct (metrics.rs): EMA for q_gap/q_var/grad_norm with
scalar grad_consistency proxy (successive grad_norm delta ratio)
- Add LearningHealth + HealthEmaTrackers + 5 last_* fields to DQNTrainer (mod.rs)
- Initialize new fields in constructor.rs
- Add write_isv_signal_at, read_atom_utilization, compute_q_spectral_gap to
GpuDqnTrainer (gpu_dqn_trainer.rs) with coarse max/min spectral-gap proxy
- Forward same three methods on FusedTrainingCtx (fused_training.rs)
- Compute LearningHealth in process_epoch_boundary (training_loop.rs): reads
per_branch_q_gap_ema, epoch min/max, atom_util, spectral_gap; emits HEALTH_DIAG log;
broadcasts health_value to ISV[12] via write_isv_signal_at
Known proxy substitutions (documented in training_loop.rs comment):
- grad_consistency: scalar proxy instead of full Adam vector cosine (per-component buffers)
- spectral_gap: coarse max/min ratio on q_readback_pinned instead of SVD
- ens_disagreement: 0.1 placeholder until D6/N6
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Task 3 refactored experience_state_gather (the WRITER) to use assemble_state()
with canonical layout (OFI at [42..62)). But env_step and ofi_embed_build_input
(the READERS) still hardcoded OFI at [66..84) — the OLD pre-refactor layout.
This meant 7 locations were reading MTF/portfolio features as if they were OFI:
1. Line 1745: dense micro-reward ofi_cur = state+66 → actually MTF[4]
2. Line 1778: book_aggression = state[82] → actually plan_isv region
3. Line 1983: ps[30..37] OFI delta storage for NEXT bar — storing MTF data
4. Lines 5833/5836/5838/5840: ofi_embed_build_input — feeds 18→10 MLP into
Mamba2 temporal SSM and attention. Entire temporal pipeline was training
on MTF features dressed as OFI.
Symptoms explained:
- WinRate=20.9% on validation (anti-correlated): dense micro-reward computes
quality=sign_pos × garbage_MTF_deltas, systematically rewarding wrong direction
- mean_reward=+0.004 but Sharpe_raw=-0.0004: shaped reward exploits garbage
signal, real portfolio loses money
- grad_norm=23560 at epoch 2: gradients chasing noise
- Q-value explosion to ±10 in one epoch: learning contradictions
Fix: replaced all hardcoded 66/74/82/83 with SL_OFI_START from state_layout.cuh.
Both reader kernels now use the same canonical layout as assemble_state().
Verified locally: smoke test passes, OFI_DIAG shows correct non-zero values
(raw_mean=-0.36, delta_mean=-0.21, log_dur=-0.23).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the canonical state layout at all 5 sections (market/OFI/MTF/portfolio/padding).
If any section drifts to the wrong offset, this test fails. Since training and
backtest share assemble_state() in state_layout.cuh, a passing test guarantees
both paths produce identical layouts.
Result: ✓ market=41 ofi=9 mtf=9 portfolio=5 padding=all-zero
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- OFI_DIAG now reads positions [42..62) using OFI_START constant instead of
hardcoded [66..74). Verified: raw_mean=0.0891, delta_mean=-0.0370,
book_agg=0.4500, log_dur=-0.2303 (was all zeros before).
- Removed 3 stale state_dim field initializers from evaluate_baseline.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 call sites to GpuExperienceCollector::new() were missing the 12th
argument (curiosity_weight). Pre-existing test issue surfaced during
Task 4 verification. Fixed by passing 0.0 (disabled in tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kernel now writes to local arrays (market[], portfolio[], plan_isv[],
mtf[], ofi[]) then calls assemble_state() from state_layout.cuh to produce
the canonical layout. This eliminates the hardcoded ofi_start=66 that
collided with MTF features and ensures training/validation use identical
state vectors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diagnostics served their purpose — confirmed OFI flows through full
pipeline on H100 (state_gather → PER → trainer). Remove:
- OFI host verify readback (upload_ofi_features)
- STATE_GATHER_DIAG pinned memory readback (timestep loop)
- Dead bf16 scatter_insert kernel (states are f32, was never called)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
scatter_insert_f32 is a 1D scalar kernel (5 args: dst, src, cursor, cap,
batch_size). insert_batch called it with 6 args for state matrices, passing
state_dim as batch_size. CUDA silently dropped the 6th arg (actual batch_size).
Result: only 96 floats (1 state row) inserted per experience batch into PER.
The model was training on ~99.99% uninitialized GPU memory. This bug affected
ALL state features, not just OFI — market features and portfolio were also
garbage in PER-sampled training batches.
Fix: added scatter_insert_f32_rows kernel (2D-aware, 6 args: dst, src,
cursor, cap, state_dim, batch_size) matching the existing scatter_insert
(bf16) pattern. States and next_states now use the row-aware kernel.
Verified locally: OFI_DIAG shows non-zero values through the full chain
(state_gather → env_step → PER insert → PER sample → trainer states_buf).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses PinnedHostBuf + cuMemcpyDtoHAsync for state_gather diagnostic.
Reads batch_states[0..state_dim] to verify OFI at positions [66..84).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Host-side check (zero-cost) before clone_htod to confirm data isn't
zero before it reaches the GPU. Fixes crash from previous diagnostic
(memcpy_dtoh size mismatch).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Temporary diagnostic: readback ofi_gpu[0..20] after upload and
batch_states[66..84] after first state_gather. OFI works locally on
RTX 3050 but shows zeros on H100 with identical v4 cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The PVC had a v2 cache that passed has_ofi and nonzero checks but contained
stale OFI data computed by an old binary. load_fxcache accepted v2/v3 for
backward compat, keeping the stale cache alive across every deploy.
v4 is the only valid version. Legacy loading code removed (-50 lines).
Argo ensure-fxcache will delete the v2 cache and rebuild from 148GB MBP-10.
Verified locally: v4 cache produces OFI_DIAG raw_mean=-0.2391 (non-zero).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ofi_gpu is unconditional — wrapping in Option allowed silent NULL pointer
fallback to kernel. Now a plain CudaSlice<f32>.
total_bars default was 10,000 (a lie) — must come from actual data length.
Default changed to 0 with hard error if not set before collect.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The v2 fxcache on PVC passed has_ofi=true validation (mbp10_dir was present
when built) but contained all-zero OFI data. The old binary set the flag
based on directory existence, not actual computed values.
Now counts non-zero OFI rows before accepting a cache — forces rebuild
when MBP-10 data is available but OFI content is all zeros.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
