Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
Adds module-level `upload_via_mapped_{f32,i32,u32,u64}` helpers and an
in-place `update_via_mapped_f32`. Each stages CPU data through a
transient mapped pinned buffer and DtoD-copies into the destination
`CudaSlice<T>`, then stream-syncs so the staging buffer is safe to
drop. Destination buffer types remain `CudaSlice<T>` so every existing
consumer (raw_ptr / kernel arg) is untouched, satisfying
`feedback_no_partial_refactor.md` for these one-shot init paths.
Migrated COLD sites in `GpuDqnTrainer::new`:
- weight_decay_mask (TOTAL_PARAMS f32)
- branch_slice_starts_dev, branch_slice_lens_dev ([i32; 4])
- branch_grad_scales_dev ([f32; 4])
- per_branch_gamma_base/max_dev ([f32; 4] each)
- q_quantile_branch_offsets/sizes_dev ([i32; 4] each)
- spectral_norm_descriptors_dev ([u64; 78])
- stochastic_depth_scale_buf ([f32; 3])
- stochastic_depth_rng_state ([u32; 1])
- vsn_group_begins_buf, vsn_group_ends_buf ([i32; num_groups] each)
- mamba2_params Xavier init (mamba2_param_count f32)
11 COLD HtoD sites eliminated. cargo check clean (15 warnings; +2 over
baseline 13 are the new MappedU32/U64 struct visibility warnings,
matching the existing MappedF32/I32 pattern).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
`GpuActionSelector::new` previously did one HtoD memcpy to upload
RNG seeds. Now allocates `MappedU32Buffer`, writes seeds via
host_ptr, and passes `dev_ptr` (CUdeviceptr) to the three kernels
that consume rng_states (epsilon_greedy, epsilon_greedy_routed,
branching_action_select).
Adds `MappedU32Buffer` and `MappedU64Buffer` to `mapped_pinned.rs`
mirroring the existing `MappedF32Buffer`/`MappedI32Buffer` API
(new/write_from_slice/read_all/Drop). The U64 variant is staged
for the upcoming spectral-norm host_desc[78] descriptor table
migration in `gpu_dqn_trainer::new`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
`set_count_bonuses` is called per action-selection — converting the
three bonus buffers from CudaSlice<f32> (HtoD memcpy) to
MappedF32Buffer eliminates 3-6 HtoD memcpys per call.
Before: 3 reused-buffer htod_f32 (lines 92/96/100) + 3 first-call
clone_htod_f32 (lines 93/97/101) on every call = 3 HtoD steady-state.
After: zero HtoD — direct host_ptr writes via write_from_slice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The IQN readiness gauge on `GpuDqnTrainer` is a streaming improvement
fraction `(iqn_loss_initial - iqn_loss_ema) / iqn_loss_initial`
(gpu_dqn_trainer.rs:5805-5818). `iqn_loss_initial` is captured once on
the first `update_iqn_readiness` call where it is still ~0 and never
resets — across folds it stays pinned to the fold-0 epoch-1 IQN loss.
The pinned device-mapped readiness slot is read by `c51_loss_kernel`
as the CVaR α and by IQN gradient-weight gating, so a stale anchor
either pins readiness near 1 (over-confident, full IQN gradient
weight on a still-recovering head) or near 0 (under-weighting a
converged head).
Fix:
- New `pub fn reset_iqn_readiness_state` on `GpuDqnTrainer` zeroes the
three coupled scalars (iqn_loss_initial / iqn_loss_ema /
iqn_readiness) and writes 0.0 through the device-mapped pinned host
slot — no HtoD copy, the mapping propagates the host write directly.
- New `pub(crate) fn reset_iqn_readiness_state` wrapper on
`FusedTrainingCtx` mirrors the existing `reset_eval_v_range_state`
pattern.
- Three new `FoldReset` registry entries (`isv_iqn_loss_initial`,
`isv_iqn_loss_ema`, `isv_iqn_readiness`) dispatch through one match
arm in `reset_named_state` (training_loop.rs).
`update_iqn_readiness` already handles `iqn_loss_initial < 1e-12` as
the bootstrap branch, so the new fold's first batch is recaptured as
the anchor on the first call after reset, by design.
Per `feedback_no_partial_refactor.md` — same fold-boundary
state-migration contract, all three coupled scalars migrate together.
This commit completes the 3-fix sequence (Fix 1 IQN target sync,
Fix 2 aux Adam states, Fix 3 IQN readiness) closing the fold-boundary
contract gap surfaced by the audit. Bug signature being resolved:
F0 ep5 Q=+0.79 (healthy)
F1 ep1 Q=+0.82 (boundary fine; c51_alpha warmup blends IQN low)
F1 ep2 Q=+2.23 (drift starts — IQN online↔target gap widens)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66 (geometric ~2.3×/epoch)
F1 ep5 NaN at step 5 (fp32 overflow past atom support)
Fix 1 (IQN target sync) is the load-bearing fix for the geometric
drift itself; Fix 2 prevents first-epoch Adam-step overshoot in
seven aux optimizers; Fix 3 closes the gauge-staleness path.
cargo check -p ml --lib clean at 13 warnings (workspace baseline).
cargo test -p ml --lib --no-run clean. All three existing
`state_reset_registry::tests` pass without modification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend the fold-boundary Adam reset contract to every auxiliary
optimizer added after the main DQN's `reset_adam_state` was wired:
- `GpuDqnTrainer::reset_adam_state` now also memsets q_attn /
sel / denoise / mamba2 / ofi_embed (m, v) buffer pairs and
zeroes the corresponding adam_step counters.
- New `pub(crate) fn reset_adam_state` on `GpuTlob` (memsets
adam_m/v, zeroes adam_step + writes 0 to the device-mapped
pinned host counter so the next adam_step kernel reads t=1
after `increment_adam_step`).
- New `pub fn reset_adam_state` on `GpuIqlTrainer` (same pattern,
m_buf/v_buf/adam_step/t_pinned).
- `FusedTrainingCtx::reset_for_fold` invokes the three new
helpers immediately after the existing IQN Adam reset block.
Both `gpu_iql` and `gpu_iql_low` are reset.
Per `feedback_no_partial_refactor.md` — the fold-boundary
state-migration contract has multiple consumers (main DQN trunk,
IQN head, aux Adam states across q_attn / sel / denoise / mamba2
/ ofi_embed / tlob / iql / iql_low). Adding `sync_target_from_online`
to the main DQN years ago without extending it to subsequently-
added optimizers left a "partial migration strictly worse than
the original" state. This commit + Fix 1 + Fix 3 close all three
remaining gaps surfaced by the fold-boundary audit.
All new resets follow the existing main-DQN pattern: DtoD memset-
zero only, pinned host counter written through the device-mapped
page (no HtoD/HtoH/DtoH paths). cargo check -p ml --lib clean
at 13 warnings (workspace baseline). cargo test -p ml --lib
--no-run clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `GpuIqnHead::sync_target_from_online` (DtoD copy of online_params →
target_params) and call it from `FusedTrainingCtx::reset_for_fold`
alongside the existing IQN Adam reset.
Root cause: at fold boundary the main DQN does shrink-and-perturb +
hard target sync (gpu_dqn_trainer.rs:12495 — the latter explicitly added
to prevent fold-1 grad explosion). The IQN head was added later but
only had Polyak EMA `target_ema_update` — no hard-sync existed. Polyak
at τ=0.005/step closes ~0.5%/step, far too slow to close the
fold-boundary online↔target gap before inflated Bellman TD-error
compounds geometrically through the IQN backward pass.
Bug signature this resolves:
F0 ep5 Q=+0.79 (healthy)
F1 ep1 Q=+0.82 (boundary fine; c51_alpha warmup blends IQN low)
F1 ep2 Q=+2.23 (drift starts — c51_alpha ramp + IQN online↔target gap widens)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66 (geometric ~2.3×/epoch)
F1 ep5 NaN at step 5 (fp32 overflow past atom support)
Per `feedback_no_partial_refactor.md`: when a shared contract
(fold-boundary state migration) changes, every consumer must migrate
together. The main DQN sync was added explicitly — the IQN, TLOB, IQL,
and aux Adam states were added later but never extended to that
contract. This commit closes the IQN gap (primary). Aux Adam + IQN
readiness gaps follow in subsequent commits.
Touched:
- gpu_iqn_head.rs: new `sync_target_from_online` (mirrors gpu_dqn_trainer.rs:12495 exactly)
- fused_training.rs: call the new sync inside the existing IQN reset block
- docs/dqn-wire-up-audit.md: Invariant 7 entry covering Fix 1 of 3
cargo check -p ml --lib clean at 13 warnings (workspace baseline). No
fingerprint change. DtoD memcpy only — no HtoD/HtoH/DtoH paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Best-checkpoint save (val Sharpe improvement, ~30% of epochs in
convergent runs) blocked the epoch loop for 20-40s on each improvement:
serialize_model() chained N (~26) per-tensor DtoH downloads via
GpuTensor::to_host → memcpy_dtoh, each forcing an implicit stream sync
on a busy training stream. The DtoH chain also violates
feedback_no_htod_htoh_only_mapped_pinned.md (only cuMemHostAlloc
DEVICEMAP allowed for CPU↔GPU paths).
Plan B:
- Introduce snapshot_model_to_pinned (mod.rs): allocate one
MappedF32Buffer sized for all named weight slices concatenated,
cuMemcpyDtoDAsync each slice into the buffer's device pointer
(which aliases the host page), single stream sync, copy bytes out
to a Send + 'static Vec<u8>. One sync per snapshot, replaces N.
- serialize_snapshot_bytes (mod.rs): pure-CPU safetensors construction
from CheckpointSnapshot. Static — callable without &self, so the
worker can move the snapshot across thread boundary.
- handle_epoch_checkpoints_and_early_stopping on val-Sharpe
improvement: save_best_gpu_params (DtoD, fast) + snapshot to pinned
+ tokio::task::spawn_blocking the safetensors construction +
checkpoint_callback invocation. JoinHandle parked on
pending_checkpoint_handles. Training loop continues immediately.
- await_pending_checkpoint_handles drains in-flight workers at
training end (success branch + early-stop branches) and before
any synchronous cold-path checkpoint write to keep disk ordering
deterministic.
- F bound on train / train_walk_forward / train_fold_from_slices
gains + 'static so the callback can be moved into the worker.
All public callers already use 'static-compatible move closures
(test fixtures with shared mutable state migrate to Arc<Mutex<T>>).
Internal pipeline uses CheckpointCallbackHandle =
Arc<std::sync::Mutex<Box<dyn FnMut + Send + 'static>>> so the
same callback flows through multi-fold walk-forward into every
fold's worker.
- serialize_model itself rewritten via the snapshot path: the
no-DtoH rule now holds across ALL checkpoint paths (best, periodic,
early-stop, plateau-exhausted). The pre-existing GpuTensor::to_host
path is no longer reachable from the DQN trainer.
The audit's spec called for an mpsc channel(1) drop-old worker, but
the multi-fold + &mut F pre-existing API made the simpler
fire-and-forget spawn_blocking pattern a cleaner fit (Mutex
serialises any concurrent invocations; Vec<JoinHandle> drain at end
guarantees disk writes complete before the trainer returns). Same
overlap benefit (training rolls while serialize+disk run on a
blocking thread); upper bound on in-flight work is one-per-improved-
epoch which approximates the spec's depth=1 in realistic training
runs.
Per feedback_no_partial_refactor: every site that constructs a
checkpoint payload migrated in lockstep — best-improvement uses
the worker; periodic / plateau-exhausted / early-stop call the
shared Arc<Mutex<F>> handle inline. All paths read params via
snapshot_model_to_pinned, so the no-DtoH rule applies uniformly.
Test fixtures (8 .rs files) updated for the + 'static bound (move
closures + cloned PathBufs / Arc<Mutex<T>> for shared mutable state).
Verified: SQLX_OFFLINE=true cargo check --workspace --tests clean
(warnings unchanged from baseline). cargo test -p ml --lib --no-run
clean. No fingerprint change.
Wire-up audit entry extended with Plan B file:line edit sites
(rides under the same Async-validation overlap section started by
the companion Plan A commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The train-multi-seed-p526h repro (with MSE-clamp + PopArt-carry already
landed) reproduced a geometric Q-divergence in fold 1 that train_loss
can no longer hide:
F0 ep5 Q=+0.79 (healthy carryover)
F1 ep1 Q=+0.82 (boundary handoff fine)
F1 ep2 Q=+2.23 (2.7× jump — drift starts)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66
F1 ep5 NaN at step 5
A model with this Q dynamic is catastrophically unsafe for live trading:
Kelly cap floats with Q-confidence, so runaway Q drives oversized
positions and can blow up the strategy before any downstream safety
trips. This commit installs a production safety net — NOT a root-cause
fix — that hard-halts training the moment Q-drift is detected:
if |q_mean| > 2× prev AND |q_mean| > 1.5 production-unsafe floor
→ return Err, training halts, model rejected from deployment
Inserted in training_loop.rs immediately before the existing
`self.prev_epoch_q_mean = q_mean` update so the comparison uses the
same source-of-truth values that already feed downstream diagnostics.
The 2× ratio catches genuine geometric divergence (typical healthy
growth <30%/epoch); the 1.5 absolute floor prevents false positives
in early training where small Q magnitudes oscillate by large ratios
(verified: F0 ep4→ep5 ratio 5.3× would NOT trigger because |0.79|<1.5).
Both thresholds are numerical-stability bounds (Invariant 1 carve-
out), not tuned hyperparameters. Skips first epoch (no prev_q). Does
NOT skip fold-boundary epochs — those are exactly the failures
we're catching.
This is a safety net. The actual root cause is a fold-boundary reset
gap (which reset isn't firing correctly?). A systematic audit is
queued to identify it. Suspects: prev_grad_buf, PopArt GPU Welford
buffers (popart_count huge from fold 0 → new-fold stats track too
slowly), isv_q_abs_ref_* magnitude EMAs, spectral norm σ EMAs, TLOB
Adam state.
Per user's "can't happen at production" philosophy: production
deployments must have this safety net regardless of whether the
root-cause fix lands first.
Validation backtest was serialised on the training stream by passing
self.cuda_stream as parent_stream into GpuBacktestEvaluator::new (the
forked eval-stream still depends on training-stream completion, but the
metrics + plan_diag readbacks went through clone_dtoh / memcpy_dtoh,
each forcing an implicit sync that blocked behind the training stream's
pending work). The prior comment claimed single-stream eval was needed
"for cuBLAS determinism" — incorrect: cuBLAS bit-wise reproducibility
caveats apply per-handle, not across the process, and each stream
already owns its own PerStreamCublasHandles. Costs ~30-40s/epoch on L40S.
Plan A:
- Route eval through self.validation_stream (already forked at
constructor.rs:547 but unused on the eval path) and add a fresh
cuda_stream-recorded train_done_event so validation_stream +
evaluator's internal forked stream both wait on it before launching
eval kernels. GPU-side cuStreamWaitEvent — no CPU sync.
- Replace metrics_buf and plan_diag_buf in GpuBacktestEvaluator from
CudaSlice<f32> to MappedF32Buffer (cuMemHostAlloc DEVICEMAP). Kernels
write through the buffer's dev_ptr; host reads via volatile host_ptr
after a single eval_stream.synchronize() per evaluation. Replaces
the forbidden clone_dtoh / memcpy_dtoh path
per feedback_no_htod_htoh_only_mapped_pinned.md. The single sync
blocks only the eval stream — training rolls on uninterrupted.
- Update the misleading "single-stream cuBLAS determinism" comment
to explain the actual per-stream-handle architecture.
Per feedback_no_partial_refactor: every consumer of the
metrics_buf / plan_diag_buf contract migrated in lockstep (struct
fields, alloc, kernel-arg passing, host readout). Per-stream cuBLAS
handle parity with training preserved (val TLOB still uses
PerStreamCublasHandles::new(&eval_stream); evaluator forks its own
internal cuBLAS state from parent_stream).
Verified: cargo check -p ml --lib clean (13 warnings, workspace
baseline). cargo test -p ml --lib --no-run clean. No fingerprint
change — mapped-pinned is allocation-method orthogonal to kernel
buffer layout.
Companion commit lands Plan B (async best-checkpoint serialize).
Wire-up audit entry covers Plan A here; will be extended with Plan B
edit sites in the companion commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous fixes (PopArt carry-forward + iqr floor in d710f9d50) reduced
fold-1 ep-1 train loss 16× but ep-2/ep-3 still oscillated 10k-200k
while val Sharpe stayed healthy at 76 — confirming the model itself
trains fine and the inflated train_loss is a pure measurement artefact.
Root cause in mse_loss_kernel.cu:457: the Bellman target
target_q = reward + γ·(1-done)·target_eq
is NOT clamped to the C51 atom support [v_min, v_max], whereas
`online_eq` is naturally bounded (it's an expected value of softmax
probabilities over atoms in that support). When PopArt-normalised
reward lands outside support — most acutely at fold boundaries when
cached robust stats lag the new fold's distribution by one epoch —
td = online_eq - target_q
grows to O(10^3–10^4), and
mse = 0.5·td²
inflates to 10^5–10^6 per-sample, dwarfing the (correctly bounded)
C51 distributional loss in the blended warmup total
(1-α)·MSE + α·C51
C51 already handles this in c51_loss_kernel.cu:247
t_z = fminf(fmaxf(t_z, v_min), v_max)
because the categorical projection requires it. MSE was missing the
matching clamp.
Fix: one-line `target_q = fminf(fmaxf(target_q, v_min), v_max)` in
mse_loss_kernel.cu between the ensemble-disagreement adjustment and
the td compute. Mirrors C51's clamp exactly.
No information lost: anything outside [v_min, v_max] is unrepresentable
in the categorical distribution anyway, so MSE warmup tracking targets
that the network categorically CANNOT learn produces a meaningless
reading. With the clamp, train loss stays bounded by support × atoms,
matching healthy fold-0 readings.
Per pearl_blend_formulas_must_have_permanent_floor.md (numerical-
stability bound carve-out under Invariant 1).
L40S 15-epoch repro #2 (train-multi-seed-kdkdv) revealed train_loss
exploded 1000-2000× in fold 1 (3.1 → 318k-694k) while val Sharpe
stayed healthy and CLIMBING (73 → 114). Train/val disconnect = pure
measurement bug, not real instability.
Mechanism: reset_for_fold() zeroed cached_iqr/cached_median at every
fold boundary. The `if cached_iqr > 0.0` guard at fused_training.rs:1220
then forced fold N+1's first epoch to the Welford GPU path. Welford
running stats inherited from fold N, combined with fold N+1's slightly
different reward distribution post-S&P + adversarial regime, produced
normalized rewards far outside C51 atom support [-50, 50] — categorical
loss readings 10^5x inflated. On rare timing-sensitive paths the
near-zero divide overflowed to Inf → NaN (the run-1 flagged=[2=on_b_logits,
3=mse_loss, 6=grad_buf, 7=save_current_lp, 8=save_projected]
diagnostic — what we caught was downstream of THIS root cause).
The diagnostic infrastructure from 756b1ef31 + 32e5375ac worked perfectly:
its precise signal of "on_v_logits clean, on_b_logits NaN, but loss is
also NaN, params clean pre-forward" surfaced the train/val disconnect
that pointed at the reward normalization bug.
Fix A (carry-forward, fused_training.rs:919-922):
Stop resetting cached_iqr / cached_median at fold boundary. Carry
fold N's final-epoch median/IQR forward as fold N+1's epoch-1
default. Same instrument, similar reward distribution between
adjacent walk-forward folds, so the carry is safe and gets replaced
by fresh stats at the end of fold N+1's epoch 1.
prev_popart_var still resets (sole consumer is tau-change detection;
a fresh fold counts as a change point regardless).
Fix B (permanent floor, dqn_utility_kernels.cu:1657):
Raise iqr fmaxf floor 1e-6 → 1e-4. With iqr=1e-6 a trade-exit
reward of 5.0 normalizes to 5e6 (vs post-fix 5e4) — much harder to
hit fp32 overflow. Defensive bound for genuinely-pathological iqr
paths (e.g. genuinely degenerate data quantiles), not a tuned
knob — Invariant 1 carve-out for numerical-stability bounds.
Per pearl_blend_formulas_must_have_permanent_floor.md (the same
recipe that resolved Kelly cap warmup + var_scale collapse before).
Resolves task #84 ("Fold-boundary state reset gap causes fold 1 grad
explosion").
Previous L40S 15-epoch repro fired the wired NaN diagnostic with
flagged=[2=on_b_logits, 3=mse_loss_scalar, 6=grad_buf, 7=save_current_lp,
8=save_projected] while value stream (flag 1) and pre-forward params
(flags 4-5) stayed clean. That isolates the corruption to the branch
advantage GEMM/activation but leaves one open question: is the GEMM
input (h_s2, shared between value and branch heads) finite or already
NaN?
Flag 12 = save_h_s2 covers the post-trunk activation. Outcomes:
- flag 12 clean + flag 2 NaN → branch GEMM overflowed (advantage-side
fp32 overflow under post-S&P + adversarial reward stress).
- flag 12 NaN → trunk encoder corrupted upstream of both heads.
Mechanically: one new check_nan_f32 call against save_h_s2 in
run_nan_checks_post_forward, names array slot 12 updated rsv12 →
save_h_s2 in training_loop.rs error message. Same ~5µs cost as the
other 12 checks. Slots 13-15 still reserved.
Per `feedback_no_partial_refactor.md` (extending the NaN diagnostic
contract atomically) and `feedback_no_stubs.md` (rsv12 was a real
reservation; now consumed).
Migrates the remaining 4 forward sites:
- VSN Linear_1 (x6 groups): adds 6 per-group RELU_BIAS shapes to the
relu_bias cache; migrates sgemm_f32_ldb + launch_add_bias_relu_f32_raw
pairs to sgemm_f32_fused_relu_bias.
- VSN Linear_2 (x6 groups, single shape (1, B, VSN_HIDDEN, VSN_HIDDEN)):
migrates to sgemm_f32_fused_bias.
- GLU value head + gate (launch_vsn_glu_branch Steps 3-4): migrates to
sgemm_f32_fused_bias with per-stream workspace selection.
Also fixes a partial-refactor leak in forward_online_raw — the
sequential branch fallback (distinct_branches=false) was still on the
unfused sgemm_f32 + launch_add_bias_relu_f32_raw pair while its
multi-stream sibling already used the fused epilogue. Now both code
paths take the fused-first contract.
Layout fingerprint unchanged (no params buffer changes).
Migrates 6 branch FC adv_logits sites (decoder_forward_only,
forward_online_raw multi-stream + sequential, forward_online_f32
multi-stream + sequential, forward_target_raw multi-stream) from
sgemm_f32(_branch) + launch_add_bias_f32_raw pairs to
sgemm_f32_fused_bias.
The branch FC output projects [B, adv_h] -> [B, branch_size_d * num_atoms]
across 4 distinct branch shapes, all pre-cached. Multi-stream sites use
the per-branch workspace (branch_workspace_ptrs[d]) to avoid contention
with concurrent branch streams. Drops up to 4 add_bias_f32 launches per
training step (one per branch direction) on each of {online, target,
f32-collector} forwards.
Migrates 4 v_logits sites (forward_online_raw, forward_online_f32,
forward_value_head ensemble heads, forward_target_raw) from
sgemm_f32 + launch_add_bias_f32_raw pairs to the new
sgemm_f32_fused_bias helper. The value-head output projects
[B, value_h] -> [B, num_atoms] (C51 atom logits), no ReLU on output.
Drops 4 standalone add_bias_f32 kernel launches per training step.
Migrates 8 trunk sites (4 online + 4 target) from
sgemm_f32 + launch_add_bias_f32_raw pairs to the new
sgemm_f32_fused_bias helper. Covers all four GRN Linear layers per
encoder: h_s1 Linear_a, h_s1 Linear_b, h_s2 Linear_a, h_s2 Linear_b.
Linear_residual has no bias term and stays a plain sgemm_f32_ldb.
Drops 8 standalone add_bias_f32 kernel launches + 8 memory
round-trips per (online + target) forward step.
Adds gemm_cache_bias field, create_cached_fwd_gemm_desc_bias builder,
and sgemm_f32_fused_bias helper. Pre-creates descriptors for every
BIAS-only output shape used in the forward pass. No call sites
migrated yet — subsequent commits flip individual sites to use the
fused helper.
Mirrors the established RELU_BIAS infrastructure pattern: deterministic
algorithm selection, per-call bias-pointer wiring, Err-on-missing-cache
contract for graceful fall-back.
Fold 1 of train-multi-seed-72fl6 hit NaN at step 5 with `flagged=[]` —
diagnostic told us nothing because:
1. The 8 NaN-check kernels in `run_nan_checks_*_forward` were never
invoked from production training. Allocated-zero flags read back
as zeros; halt-on-NaN reported "no buffer flagged" while loss was
demonstrably NaN via host-side pinned readback.
2. The error message labels (training_loop.rs:1853) referenced
`bf16_params` — dead code from before the bf16→TF32 switch — and
the format string did not match the actual kernel index→buffer
mapping.
Wired the existing NaN check kernels into `submit_post_aux_ops` so
they run every step inside the captured parent graph (post_aux child).
Flags persist across steps within a fold so the FIRST buffer to go
NaN remains visible at host-readback time; reset is host-side at fold
boundary in `reset_for_fold`.
Extended coverage beyond the original 8 buffers (states, on_v_logits,
on_b_logits, mse_loss, params, grad_buf, save_current_lp) with 4
loss-component buffers most likely to break under post-S&P + adversarial
regime stress:
Flag 8 save_projected C51 target distribution
Flag 9 moe_gate_softmax MoE gate over 8 experts
Flag 10 aux_nb_loss_scalar aux next-bar MSE
Flag 11 aux_rg_loss_scalar aux regime CE
Flag buffer grown 8 → 16; slots 12-15 reserved for future expansion
(CQL / IQN / per-branch backward).
Updated training_loop.rs error message: 16 named slots matching actual
kernel layout, per-flag indexed name in flagged list, dropped stale
`bf16_params` / `_bf16` suffixes, explicit hint when `flagged=[]`.
Cost per step: ~12 single-block reductions × ~5µs = ~60µs. <0.1%
overhead at the 258ms/step measured on L40S — pinpoints source
buffer on next NaN halt.
Per `feedback_no_legacy_aliases.md`, `feedback_no_stubs.md`,
`feedback_trust_code_not_docs.md`.
TLOB attention GEMMs (M=TLOB_OUT=16, K=TLOB_IN=32, N=batch) are too
narrow for cuBLAS-Lt's heuristic search; cublasLtMatmulAlgoGetHeuristic
returns "no algo found" → GpuTlob::new fails → graceful-degrade hides
the bug per feedback_no_hiding.md. The previous symbol-rename fix
(7208836d1) merely surfaced this deeper API-choice problem.
Per NVIDIA best practice, cuBLAS-Lt is for tensor-core-optimised GEMMs
at scale (M,K,N ≥ 64-128); classic cublasSgemm_v2 is for general-
purpose any-shape GEMMs. TLOB's tiny attention dims belong in the
second category. Migrating all 8 TLOB GEMM call sites:
- 3× fwd Q/K/V proj (M=16, N=batch, K=32)
- 1× fwd O proj (M=16, N=batch, K=16)
- 3× bwd dW_QKV (M=16, N=32, K=batch)
- 1× bwd dW_O (M=16, N=16, K=batch)
- 1× bwd dX_O (M=16, N=batch, K=16)
Single API, single code path, no try/catch. Removed graceful-degrade
wrap in fused_training.rs and trainer/metrics.rs — TLOB is no longer
optional. Field type changed from Option<GpuTlob> to GpuTlob; all five
consumer sites (forward/backward in fused_training, mean_max in
training_loop, val-side init+sync in metrics) migrated together per
feedback_no_partial_refactor.md.
PerStreamCublasHandles registry gained a classic_handle field +
classic_for(stream) accessor mirroring the existing lt_for(stream) API.
Both handles share the same 32 MB workspace, both bound to their
stream + workspace at creation time. Default-stream classic handle is
created in PerStreamCublasHandles::new; side-stream handles are
provisioned lazily alongside their cuBLAS-Lt sibling inside
classic_for/lt_for/pre_register_stream. Classic handles default to
CUBLAS_TF32_TENSOR_OP_MATH (matching gpu_curiosity_trainer's existing
convention; same TF32 path as the cuBLAS-Lt CUBLAS_COMPUTE_32F_FAST_TF32
compute type).
Verification:
- SQLX_OFFLINE=true cargo check --workspace — clean
- cargo test -p ml --lib --release gpu_tlob — new
tlob_sgemm_parity_with_cpu_reference test passes; verifies all 8
GEMMs (forward Q/K/V/O, backward dX_O, dW_O, dW_Q/K/V) match a CPU
sgemm reference within 2e-3 absolute on batch_size=64 synthetic
OFI input
- dqn-wire-up-audit.md updated; new pearl
pearl_cublas_lt_vs_classic_sgemm.md captures the rule
The cuBLAS-Lt path remains in use for the larger gpu_attention module
(trunk attention, dims 128+) where tensor-core throughput pays off.
This is the standard split: Lt for big, classic for small.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gpu_tlob.rs:232 was loading symbol `attn_adam_update` from the
attention_backward_kernel cubin, but the kernel is defined as
`attn_adam_kernel` (attention_backward_kernel.cu:70); gpu_attention.rs:281
already loads the same symbol with the correct name. Mismatch caused
GpuTlob::new to fail with "named symbol not found" on every workflow
init, suppressed by the surrounding "training continues without TLOB"
warning. Per feedback_no_hiding.md, graceful-degraded modules ARE the
ghost-feature pattern.
With the symbol now loading, a separate cuBLAS-Lt heuristic failure
surfaces ("tlob heuristic (fwd_q): no algo found") — the TLOB GEMM
shape M=TLOB_OUT=16, K=TLOB_IN=32, N=batch is too narrow for cuBLAS-Lt
heuristics to find an algo. That's a real follow-up (pad dims, classic
sgemm fallback, or custom small-M kernel) but is independent of and
strictly upstream of this commit's symbol-name fix. The previous symbol
error was hiding the heuristic error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Val backtest subsamples every Nth bar at metrics.rs:517-518 but
annualization_factor at gpu_backtest_evaluator.rs:674 was still computed
from the full bars_per_day. Per audit
`docs/lookahead-bias-audit-2026-04-28.md` §5, this inflates reported
sharpe_annualised by sqrt(N) — for N=4, factor of 2.
Fix: thread the subsample stride through `GpuBacktestConfig` as a new
`val_subsample_stride: u32` field (default 1 = no subsampling). The
constructor divides `bars_per_day` by the sanitized stride before
sqrt, so reported Sharpe / Sortino / Calmar all reflect the effective
sampling cadence.
The val call site at metrics.rs uses a single `VAL_SUBSAMPLE_STRIDE`
const that drives both the `if i % stride != 0 { continue; }` predicate
AND the config field — single source of truth means future cadence
changes automatically keep the annualization in sync. Per-bar Sharpe
trajectory is unchanged.
Other GpuBacktestConfig consumers (hyperopt adapters,
evaluate_baseline, gpu_backtest_validation tests) all use
`..Default::default()` and inherit `val_subsample_stride: 1`.
Regression test added:
test_annualization_factor_compensates_for_subsample_stride asserts the
ratio f1/f4 = 2.0 for stride=4, plus stride=0 sanitization.
dqn-wire-up-audit.md updated to reflect the new contract:
metrics.rs::VAL_SUBSAMPLE_STRIDE is the single source of truth paired
with GpuBacktestConfig::val_subsample_stride.
NOTE TO OPERATORS: Sharpe values reported in HEALTH_DIAG val [...]
lines post-merge will be ~2× LOWER than pre-merge for the same model.
This is the corrected number, not a regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OFI features at bar t were computed over [close(bar_t), close(bar_{t+1}))
— bar t+1's formation period — so 31 of 32 OFI dims at the policy's
input for bar t carried next-bar microstructure data. Per audit
`docs/lookahead-bias-audit-2026-04-28.md` §3, this is a DEFINITE leak
contaminating both training inputs and validation backtest.
Fix: shift the window backward to (close(bar_{t-1}), close(bar_t)] so
OFI at bar t uses only data accumulated during bar t's own formation.
Mirrors the correct `log_bar_duration` convention at
precompute_features.rs:567-575 (audit's smoking-gun citation).
Per feedback_no_partial_refactor, both write sites migrated in lockstep:
- crates/ml/examples/precompute_features.rs:441-475 (precompute path)
- crates/ml/src/trainers/dqn/data_loading.rs:396-448 (DBN fallback)
FXCACHE_VERSION bumped 6 → 7. PVC auto-regen via schema-hash check on
next deploy. Local regen:
./target/release/examples/precompute_features \
--data-dir test_data/futures-baseline \
--mbp10-data-dir test_data/futures-baseline-mbp10 \
--trades-data-dir test_data/futures-baseline-trades \
--output-dir test_data/feature-cache \
--symbol ES.FUT --data-source mbp10 --yes
Regression test added: tests/ofi_features_test.rs::
test_ofi_window_alignment_uses_formation_interval validates the new
partition_point predicates against a synthetic 5-bar dataset, including
an explicit anti-regression assertion that bar t+1 snapshots are NEVER
picked up for bar t.
dqn-wire-up-audit.md updated to reflect new contract for
`trainers/dqn/data_loading.rs`. Audit report
`docs/lookahead-bias-audit-2026-04-28.md` checked in alongside the fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace static moe_lambda=0.01 with a GPU-driven controller that scales
λ inversely with observed gate-entropy: λ_eff = λ_floor + λ_max_extra ×
max(0, ent_target − ent_ema)/ent_target, where ent_ema is the existing
ISV[126] producer (Phase 2 task 2.4 wire-up).
Motivation: the L40S validation run train-multi-seed-fgg9x exposed
single-expert specialization in fold 1 (expert 4 → 81% util, 7/8
experts < 5%) under static λ=0.01. The load-balance push was too weak
to prevent winner-take-all once the gate had a strong preference.
Adaptive λ rises proportional to the entropy deficit, restoring
diversity pressure when the gate concentrates.
Per pearl_blend_formulas_must_have_permanent_floor: λ_floor remains a
permanent minimum (= old static 0.01) so the controller never weakens
below baseline. Per feedback_isv_for_adaptive_bounds: signal flows
through ISV[128] (new MOE_LAMBDA_EFF_INDEX); consumer kernel reads
the slot at runtime; no DtoH on hot path.
Wiring (feedback_no_partial_refactor):
• new kernel moe_lambda_eff_update (single-block cold-path cadence,
matches kelly_cap_update / cql_alpha_seed_update precedent)
• new ISV slot 128, ISV_TOTAL_DIM 127 → 129
• layout_fingerprint_seed updated (schema_hash bumps; PVC fxcache will
auto-regen on next deploy)
• moe_load_balance_loss kernel takes (isv_signals, isv_lambda_eff_idx)
instead of float lambda — matches mag_concat_qdir's ISV-read pattern
• config: pub moe_lambda field replaced with moe_lambda_floor (0.01),
moe_lambda_max_extra (0.09), moe_entropy_target_frac (0.7)
• state-reset registry entry — ISV[128] resets to floor at fold boundary
• HEALTH_DIAG aux_moe line gains λ_eff field for observability
• hyperopt adapter (DQNHyperparameters) migrated to new knobs
• constructor bootstraps ISV[128]=floor + ISV[118..127)=uniform 1/K +
ISV[126]=ln(K) so cold-start matches legacy byte-for-byte
Smoke validates: magnitude_distribution still passes/fails identically
to HEAD (eq=0.586 same as pre-change eq=0.592 — unrelated Kelly cap
behaviour per project_magnitude_eval_collapse_kelly_capped.md).
HEALTH_DIAG fold 3 confirms controller live: ent decays 1.381 → 0.746
across epochs 7-19, λ_eff rises 0.0146 → 0.0539 monotonically.
Unit test moe_lambda_eff_update_writes_correct_values exercises 5
regimes (above-target / at-target / half-collapse / full-collapse /
deeply-above-target) on the GPU kernel — all pass within 1e-5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compute-sanitizer caught 16 OOB errors in magma_sgemmEx_kernel
<f,f,f,1,0,6,4,6,3,4>+0xe90 surfacing as mamba2_scan_projected_bwd
LAUNCH_FAILED. Pre-existing latent, exposed when q_var_buf_trainer
allocator reshuffle (fa92cb8db) moved subsequent buffer pointers.
Root cause: DuelingWeightSet::from_flat_buffer and
BranchingWeightSet::from_flat_buffer in gpu_weights.rs used hard-coded
layout indices [0..11] / [12..23] from before the GRN trunk expansion
(Plan 4 Task 2c.3a). The expansion inserted 9 GRN tensors at indices
1..12, shifting every value/branch tensor by +9. The 12-index sequential
walk in from_flat_buffer silently slid every dueling-weight pointer
forward into adjacent tensors:
- online_dueling.w_v1 → w_residual_h_s1 (sized SH1*s1_input_dim, not
VALUE_H*SH2)
- online_dueling.b_v1 → gamma_h_s1 (sized SH1=64 floats=256 bytes,
not VALUE_H=128 floats=512 bytes)
- online_dueling.w_v2 → beta_h_s1; b_v2 → w_a_h_s2; w_a1 → b_a_h_s2; …
The pathological alias surfaced via the ensemble multi-head clone path
(forward_value_head_for_ensemble): the cuBLAS RELU_BIAS epilogue tried
to read bias[0..128] from b_v1's 64-float buffer, producing exactly the
observed 16 thread (0..7, 2..3) reads at 1..61 bytes past a 256-byte
allocation.
Production GEMMs never tripped this because they read from
params_buf via f32_weight_ptrs_from_base which has always used the
correct 163-tensor GRN layout. flatten/unflatten paths used
matched-stale self-copies (no-op when online_d.w_s1 == params_buf_ptr).
Only the ensemble clone (`clone_dueling_weights`) made independent
copies of the misaliased pointers and then handed them to the GEMM as
if they were value-head tensors.
Per feedback_no_partial_refactor: every consumer of the weight-set/
flat-buffer contract migrated in lockstep:
- New DUELING_FLAT_INDICES = [0,1,2,3, 13,14,15,16, 17,18,19,20]
and BRANCHING_FLAT_INDICES = [21..32] in gpu_weights.rs encode the
authoritative mapping from DWS/BWS slots to GRN-expanded layout
indices.
- DuelingWeightSet::from_flat_buffer + BranchingWeightSet::from_flat_buffer
rewritten to use these mappings with a full prefix-sum byte-offsets
table (matches f32_weight_ptrs_from_base byte layout).
- flatten_online_weights, unflatten_online_weights,
flatten_target_weights, unflatten_target_weights (gpu_dqn_trainer.rs)
rewritten to keyed [(ptr, layout_idx); 24] pairs and write at
byte_offsets[layout_idx] instead of sequential prefix-sum over
sizes[0..23]. The no-op zero-copy check (online_d.w_s1 == src_base)
is preserved because DUELING_FLAT_INDICES[0] == 0.
Sanitizer (RTX 3050 Ti, magnitude_distribution smoke):
magma_sgemmEx_kernel OOB count: 16 → 0
Non-sanitizer smoke completes all 20 epochs without LAUNCH_FAILED
(was crashing on epoch 1 prior to fix); MAG_DIST/EVAL_DIST results
reflect real model behavior (Q=0.349, H=0.298, F=0.353 train-mode).
The unrelated F_Full eval-cap assertion is the ongoing Kelly cap
issue (project_magnitude_eval_collapse_kelly_capped.md).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke after var_scale floor (73fd49f4b) still showed eval Full = 0.000 with
intent = 0.911. Root cause: trade_physics.cuh::unified_env_step_core applied
Kelly cap using ONLY direction conviction (q_dir_gap / q_dir_abs_ref). At
smoke horizon with direction Q-values close (q_s≈q_h≈q_l≈q_f within ~0.07),
direction conviction was ~0.04 → safety = max(0.75, 0.04) = 0.75,
effective_kelly = max(0, 0.75) = 0.75, cap = 0.75 × max_pos × 0.75 =
0.5625 × max_pos → Half bucket pin (abs_pos < 0.75).
The magnitude branch was simultaneously highly confident (q_f / q_h ≈ 1.9,
mag conviction ≈ 0.5) — the policy strongly preferred Full. That signal
never reached the cap.
Thread per-sample magnitude conviction into the cap formula:
- new buffer magnitude_conviction_buf[N*L] (training) parallel to
conviction_buf, and chunked_magnitude_conviction_buf[chunk_len * n_windows]
(eval) parallel to chunked_conviction_buf
- experience_action_select writes (max(q_mag) − min(q_mag)) /
fmaxf(ISV[16], 1e-6); ISV[16] = Q_ABS_REF (magnitude branch's |Q| EMA,
recycled — no new ISV slot)
- scripted_policy_kernel writes 0.0 (no real magnitude preference; helper
composer falls back to direction signal)
- unified_env_step_core takes a parallel magnitude_conviction parameter
- combined: policy_conviction = max(dir_conv, mag_conv); cap formula
becomes safety = max(health_safety, policy_conviction); warmup_floor
uses policy_conviction
- all three env_step kernel call sites migrated together
(backtest_env_step, backtest_env_step_batch, experience_env_step) per
feedback_no_partial_refactor — no mixed state
Per pearl_blend_formulas_must_have_permanent_floor: max() composition,
not multiplicative blend. Per feedback_isv_for_adaptive_bounds: signal
flows from existing ISV[16] = Q_ABS_REF, no tuned constants.
Smoke verification (20-epoch RTX 3050 Ti, magnitude_distribution test):
- pre-fix: [EVAL_DIST] Q=0.586 H=0.414 F=0.000 (intent F=0.911)
- post-fix: [EVAL_DIST] Q=0.618 H=0.153 F=0.229 (intent F=0.911)
Intent unchanged (network already learned Full preference); the Kelly
cap no longer silences it. Both smoke assertions now pass:
ef >= 0.05 (gate) and eh + ef >= 0.30 (Task 2.2 H10).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compute-sanitizer caught 4 OOB writes in compute_expected_q at threads
60..63 of block 0 (production: batch=64, total_actions=13). Stride
52 bytes/thread (= 13 floats), increasing 45 → 97 → 149 → 201 bytes
past a 4-byte-aligned neighbour allocation. The kernel writes
`q_variance[i*total_actions + a]` with stride total_actions=13 (4+3+3+3
factored layout) into a buffer that was still sized at b*12 from the
legacy 3+3+3+3 exposure layout. Same class of bug as `d625ca28e` (q_readback
12→total_actions): the direction branch grew from 3 → 4 actions but
several legacy 12-stride consumers were not migrated.
Per feedback_no_partial_refactor: q_var_buf_trainer's contract is a
shared one — every consumer migrates in lockstep. Five touch points:
• alloc q_var_buf_trainer: b*12 → b*total_actions
• compute_expected_q (writer): unchanged, already stride total_actions
• q_denoise_step kernel: add `var_stride` param, read var_q at stride
total_actions; FC layer dim D=12 unchanged (denoiser MLP weights stay)
• q_denoise_backward kernel: same — add `var_stride`, two read sites
• denoise_build_input kernel: add `var_stride` between D and schedule
• launch_loss_reduce: b_qvar = b*12 → b*total_actions for the third
c51_loss_reduce launch (ISV slot 3/4 mean reduction)
The denoiser's internal D=12 is preserved — its FC layers operate on the
12-slot exposure layout and only consume the first 12 variance entries
per sample. The 13th variance slot (last urgency action) is computed by
compute_expected_q but currently unused by the denoiser; epistemic_gate
already reads with stride total_actions and benefits from full per-action
variance now that the buffer is correctly sized.
This was a latent bug masked by allocator fortune — earlier mag_concat
corruption (`8bc6f1ccd`) was overwriting buffers first, so this OOB
landed in already-corrupted memory and produced no compute-sanitizer
report. With mag_concat fixed, the q_var stride mismatch became the
dominant remaining corruption source, surfacing as
`bias_grad_reduce_f32_p1: DriverError(CUDA_ERROR_LAUNCH_FAILED)` on the
next launch in production runs.
Sanitizer (RTX 3050 Ti, magnitude_distribution smoke):
compute_expected_q OOB count: 4 → 0
Note: the run still trips 16 OOB reads in `magma_sgemmEx_kernel` (cuBLAS
GEMM) under a different code path; those are pre-existing — they appear
only because q_var growing from b*12 to b*total_actions reshuffled
allocation addresses, exposing a separate latent cuBLAS leading-dim
mismatch that was previously aliasing into q_var's old footprint. That
finding is filed in the wire-up audit and is not in scope for this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke (with mag_concat off-by-one fixed and fxcache regenerated):
- training MAG_DIST: Q=0.287 H=0.301 F=0.412 (healthy diversity)
- INTENT at eval: Q=0.045 H=0.045 F=0.911 (network learned Full)
- realised at eval: Q=0.592 H=0.408 F=0.000 (Full silenced)
Even though q_f >> q_h >> q_q (network strongly prefers Full), eval
realised Full = 0.000. Position-sizing pipeline composes four
multiplicative shrinks at experience_kernels.cu:1615-1672:
effective_max_pos = max_position
× cvar_scale (line 1621)
× q_gap_conviction (line 1628, clamped [0.25,1])
× kelly_f (line 1649, only if >20 trades)
× var_scale (line 1671, = 1/(1+sqrt(var_q)))
Each term well-bounded individually but composing them silences the
policy at validation when var_q persists high (var_q ≈ 30 → var_scale
≈ 0.15; even with conviction = 1.0, kelly_f = 1.0, the compound 0.15
falls in the Quarter bucket [0, 0.375)). Network INTENT reaches the
target_position kernel correctly; var_scale strips it back out.
Same family as val-Flat-collapse (warm-branch Kelly = 0 from balanced
priors) — fix is the same pearl
(`pearl_blend_formulas_must_have_permanent_floor.md`):
var_scale = max(var_scale, q_gap_conviction)
The q_gap-derived conviction is already an adaptive ISV-coupled signal
(line 1628), already clamped to [0.25, 1.0]. Using it as a permanent
floor on var_scale lets the policy's magnitude intent reach the
realised position when conviction is high, regardless of variance.
No tuned constants — feedback_adaptive_not_tuned + feedback_isv_for_adaptive_bounds
both honoured by reusing the existing adaptive bound rather than
introducing a new threshold.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke and localdev profiles previously used data_source = "ohlcv", divergent
from production (mbp10). Mismatch silently produced cache-key collisions in
the SHA256 hash path: smoke fxcache could not be loaded by production-shape
training without an explicit override. Local-dev fxcache regen also required
remembering to pass --data-source ohlcv to match.
Globalize mbp10 as the default everywhere it isn't deliberately overridden:
- config/training/dqn-smoketest.toml: data_source = "mbp10"
- config/training/dqn-localdev.toml: data_source = "mbp10"
- training_profile.rs: doc Default → "mbp10"
- trainers/dqn/config.rs: Default impl → "mbp10"
- hyperopt/adapters/dqn.rs: default → "mbp10"
- examples/precompute_features.rs: doc updated
- fxcache.rs / feature_cache.rs: discover_and_load + cache-key tests
use "mbp10" arguments
- docs/dqn-wire-up-audit.md: new entry per Invariant 7
Documentation strings retained "ohlcv" only where they document the two
available choices (config.rs:946, training_profile.rs:83).
Local fxcache regenerated to v6 mbp10:
test_data/feature-cache/13c0b086a975cc7e2384377a2cd0e97738c9410292fcfecb5807c29bf885cb48.fxcache
(175874 bars, 55 MB, OFI_DIM=32). Stale v5 ohlcv fxcache untracked
from git index (already gitignored post-79578bbaf).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compute-sanitizer pinpointed mag_concat_qdir at experience_kernels.cu:3590
writing 260 floats/state (SH2 + b0_size where b0=4) into a buffer
allocated for 259 floats/state (SH2 + 3, legacy 3-direction layout).
Off-by-one corrupted the next row's first column on every write and
overran past the buffer end on the final row, surfacing as
CUDA_ERROR_ILLEGAL_ADDRESS in downstream kernels (denoise_bias_grad_p1,
cublasLt h_v matmul) on L40S production batch sizes.
The `+3` constant was overloaded:
- direction-conditioned (mag_concat, w_b1fc, w_gate_1) — incorrectly
hardcoded SH2+3 instead of SH2+branch_0_size when the kernel migrated
to 4-direction (S/H/L/F).
- OFI-conditioned (ord_concat, urg_concat, w_b2fc, w_b3fc, w_gate_2,
w_gate_3) — correctly SH2+3 for 3 OFI features per branch
(concat_ofi_features).
Migrated all direction-conditioned consumers in lockstep
(feedback_no_partial_refactor):
- gpu_dqn_trainer.rs: w_b1fc, w_gate_1 use shared_h2+branch_0_size
- gpu_dqn_trainer.rs: split mag_concat_dim (SH2+b0) from
ofi_concat_dim (SH2+3) for buffer alloc
- gpu_dqn_trainer.rs: accumulate_d_h_s2_from_concat takes src_stride
param so mag callers pass SH2+b0, ord/urg callers pass SH2+3
- batched_forward.rs: split mag_concat_dim (SH2+b0) from ofi_concat_dim
(SH2+3); strided_scatter dst_stride and fc_k now diverge between mag
(d==1) and ord/urg (d==2,3); add separate (SH2+3) GEMM cache shape
- batched_backward.rs: d==1 (magnitude) dX/dW dims use SH2+b0;
d==2/3 (order/urgency) keep SH2+3; add separate (SH2+3) GEMM cache
shape for OFI branches
- gradient_budget.rs: smoke test now allocates b1 with SH2+b0
and b2/b3 with SH2+3 (was buggy SH2+3 for all three)
- value_decoder.rs: doc updated
- docs/dqn-named-dims.md: new "Branch FC input strides" section
documenting the direction- vs OFI-conditioning invariant
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
L40S production training crashed in launch_ofi_embed_backward with
CUDA_ERROR_ILLEGAL_ADDRESS at the first denoise_bias_grad_p1 launch in
the adam_grad child graph (workflow train-multi-seed-tdjtr fold 0).
Root cause: denoise_bias_grad_p1/p2 CUfunction handles were shared
across two independently-captured child graphs (post_aux and adam_grad).
On Ada/Hopper, when partials buffers land in different VRAM arenas
(L40S B=16384 post-MoE memory layout), the driver's per-node handle
validation trips. Pre-existing latent bug — masked on smaller GPUs and
pre-MoE allocator states where buffers happened to be address-adjacent.
Fix: load a dedicated CUmodule for the OFI bias-grad reduction kernels,
separate from the denoise variant. Same kernel bytecode (EXPECTED_Q_CUBIN
contains both); each child graph gets its own CUfunction reference.
Mirrors the graph_safe_copy_kernel precedent at line ~11523:
"Hopper CUfunction isolation: each child graph needs its own CUmodule".
Constructor adds ~2ms of cubin-load time; zero runtime cost. Smoke
validates no regression at B=64.
Production validation deferred to follow-up L40S re-deploy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reads the most recent HEALTH_DIAG aux_moe line from the smoke log
and asserts: (1) no expert utilization < 2% (anti-collapse working),
(2) no expert > 80% (no snap-collapse to single expert), (3) gate
entropy < 0.9·ln(8) (peakier than uniform on average).
Failures are findings, not test infrastructure bugs — concrete numbers
reported per feedback_kill_runs_on_anomaly_quickly.md.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §8.3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Atomic cleanup per feedback_no_partial_refactor.md:
- Delete crates/ml-dqn/src/regime_conditional.rs and all
RegimeConditional* exports from lib.rs. regime_classifier.rs
(RegimeType, RegimeClassConfig) is kept — used by validation layer
and ml-regime-detection crate.
- Rewrite DQNAgentType to wrap DQN directly (no RegimeConditionalDQN
field, no 3-head delegation gymnastics).
- DQNAgentType::get_count_bonuses_branched no longer returns hardcoded
None — it delegates to DQN::get_count_bonuses_branched() and UCB
count bonuses reach the GPU action selector for the first time
(ghost feature from Phase 0 deferral). action.rs caller simplified
to direct destructure of fixed arrays, no Option matching.
- Drop 4 regime-threshold fields from DQNConfig (regime_adx_idx,
regime_cusum_idx, regime_adx_threshold, regime_cusum_threshold) +
matching Default and aggressive() builder entries.
- serialize_model rewritten to serialize single DQN branching network
directly (no trending__/ranging__/volatile__ prefix namespace).
- curriculum.rs import fixed: crate::dqn::regime_conditional::RegimeType
→ crate::dqn::RegimeType (re-exported from regime_classifier).
- Constructor drops RegimeConditionalDQN::new_on_device, calls
DQN::new_on_device directly.
- Stale doc comments in dqn.rs, moe.rs, regime_classifier.rs updated.
- Wire-up audit updated.
Phase 3 MoE (commit a52d99613) provides the regime-conditioned behavior
the legacy 3-head architecture pretended to do — gate sees ADX (40)
and CUSUM (41) as part of the full 128-dim state vector and learns its
own decomposition, strictly subsuming the threshold classifier.
Smoke: 3/3 folds passed (405s), gate util=0.119,0.119,0.169,... preserved,
all fold checkpoints written. Workspace: 0 errors across all crates,
tests, and examples.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §7.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 3 of the MoE regime redesign per
docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md.
Atomic forward+backward wire-up — replaces the existing h_s2 producer
in fused_training.rs with the gated expert mixture:
state[B, 128] -> shared GRN trunk -> h_s1[B, 256]
|
+-> 8 expert MLPs (256->64->256) -> expert_outputs[8, B, 256]
+-> gate (128->64->8 softmax) ----> gate[B, 8]
|
moe_mixture_forward -> h_s2[B, 256] -> branching heads + C51 + IQN
Backward: moe_mixture_backward (de_k = g · dh_s2) + moe_dgate_reduce
(dg = Σ_c e_k · dh_s2) + load-balance aux gradient + cuBLAS SGEMM
backward through gate + each expert's 2 linear layers. Adam optimizer
step now updates gate + 8 experts via params_buf.
Loss: λ · K · Σ_k (mean_b g[b,k])² added to total loss with λ from
hyperparams.moe_lambda (default 0.01).
Per-step ISV producer launch (moe_expert_util_ema_update) writes 8
utilization EMA + 1 gate-entropy EMA into ISV[118..127). Per-epoch
HEALTH_DIAG aux_moe line emits utilization vector + entropy live so
operators can see whether experts are differentiating or collapsing.
Smoke test: DONE. 3/3 folds, all checkpoints saved, 728s (12.1 min,
within 25-min budget). Gate differentiated by epoch 1: expert 2 rose
from 0.119 → 0.286 → 0.323 over fold 1-2 while others remained at
0.097-0.113. Gate entropy 1.611 at fold 2 epoch 4 < ln(8)=2.079.
val_loss finite across all 3 folds; average fold metric 22.4.
Per feedback_no_partial_refactor.md: all consumers of the h_s2 contract
(branching heads, IQN aux, attention focus, backward chain) migrate in
this single commit.
Per feedback_no_htod_htoh_only_mapped_pinned.md: no new HtoD/HtoH
introduced; gate softmax + expert outputs + mixture all GPU-resident,
ISV producer GPU-driven. Load-balance scalar uses cuMemAllocHost +
cuMemHostGetDevicePointer_v2 (mapped pinned), matching existing pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single-thread cold-path-cadence kernel writes 8 per-expert utilization
EMAs + 1 gate-entropy EMA into ISV slots [118..127), α=0.05. Same shape
as h_s2_rms_ema_update / aux_heads_loss_ema_update.
GPU-resident, CPU read-only per pearl_cold_path_no_exception_to_gpu_drives.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-step design (per-k contribution then K-element reduce) avoids
atomicAdd per feedback_no_atomicadd.md. Tests verify CPU reference
match and uniform-gate minimum (loss = λ).
Backward gradient lands in Phase 3 (wire-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-stage backward: moe_mixture_backward computes de_k = g · dh_s2 in
one kernel; moe_dgate_reduce computes dg via shmem reduction over c.
FD test verifies analytic backward matches numerical for B=2, K=4, C=32
within 1e-3 tolerance (perturbation 1e-3).
All test data flows via mapped pinned buffers (no HtoD/HtoH).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves MappedF32Buffer + MappedI32Buffer from distributional_q_tests.rs
local definitions to crates/ml/src/cuda_pipeline/mapped_pinned.rs so all
kernel test wrappers (Phase 0.F, upcoming MoE Phase 2 tests) share one
implementation.
Adds write_from_slice helper for direct host_ptr write (no memcpy)
per feedback_no_htod_htoh_only_mapped_pinned.md.
Test 0.F bit-identical post-move: sigma_C51, argmax, Thompson counts
unchanged.
Per spec docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §6.4
and feedback_no_htod_htoh_only_mapped_pinned.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
T1.6 implementer correctly identified that the gate input dim is
ml_core::state_layout::STATE_DIM=128, not the literal 42 the spec/plan
incorrectly stated. The 42-dim figure was the bar-feature subset; the
actual state vector is 128-dim (42 features + portfolio + MTF + OFI
padded to 128 for cuBLAS alignment).
Updated spec §3 architecture diagram, §4.1 gate subnetwork description
+ parameter count (3,272 → 8,776), and plan header architecture line.
Implementation in commit 28c707f6a is correct; this commit just makes
the spec match the implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Gate (4 tensors, [127..131)) + 8 experts × 4 tensors (32 tensors, [131..163))
appended to GpuDqnTrainer params_buf layout. NUM_WEIGHT_TENSORS 127 → 163.
Layout fingerprint hash recomputes — old checkpoints fail to load with
fingerprint-mismatch error per the no-fallback contract in the spec.
Gate tensors: gate_w1[STATE_DIM=128,64], gate_b1[64], gate_w2[64,8],
gate_b2[8]. Zero-init so g(s) = uniform 1/K at cold start.
Expert tensors (per expert k∈[0,8)): w1[SH2,BTN], b1[BTN], w2[BTN,SH2],
b2[SH2] where SH2=cfg.shared_h2=256, BTN=MOE_EXPERT_BOTTLENECK=64.
Xavier init on w1/w2; zero on biases. Total ~268k new params.
Adam state (m_buf/v_buf), gradient scratch, target_params_buf all extend
in lockstep — all sized from compute_total_params() which sums over the
full 163-tensor layout. No static sizes to update.
Tensors allocated but not yet wired into forward/backward — Phase 3
wires gate forward, expert forward, mixture replacement of h_s2, and
the corresponding backward chain.
No test assert updates required — no test hardcodes NUM_WEIGHT_TENSORS
or the layout fingerprint value.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §6.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 slots for per-expert utilization EMA + 1 slot for gate-entropy EMA.
ISV_TOTAL_DIM 118 -> 127. Not yet consumed; producers + consumers land
in subsequent commits per the MoE plan.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §4.6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Anti-collapse load-balancing aux loss weight for the upcoming Mixture-of-
Experts redesign. Configurable via hyperopt; not a kernel constant.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §4.3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Atomic removal of 5 boolean feature flags from DQNConfig per
`feedback_no_feature_flags.md`, all of which gated dead or redundant code.
(use_iqn already removed in da632446c.)
- `use_dueling`, `use_distributional`, `use_noisy_nets`, `use_branching`:
pure-cosmetic always-on flags. Only metadata strings remained.
4 `if true /* always on */` blocks in dqn.rs collapsed to live arms;
dead else arms (legacy non-branching code paths, 5-flat ExposureLevel)
deleted.
- `use_count_bonus`: redundant with `count_bonus_coefficient` (the live
numeric kill-switch). Recording made unconditional; coefficient is the
single dial.
- `use_cvar_action_selection`: dead post-use_iqn cleanup (only consumers
were inside the deleted IQN arms). cvar_alpha retained for cuda_pipeline
CVaR loss usage.
count_bonus.rs API tightened: `bonuses_branched_f32(&self,
exp_out: &mut [f32], ord_out: &mut [f32], urg_out: &mut [f32])` write-into
API alongside the existing Vec<f64> form (kept for tests). Production
DQN::get_count_bonuses_branched() returns fixed `([f32; 4], [f32; 3],
[f32; 3])` arrays — allocation-free, f32 throughout, fed directly to GPU
action selector via stack-allocated buffers.
Test 0.F outputs bit-identical vs pre-cleanup (sigma_C51, argmax,
Thompson counts) — confirms legacy use_* arms were unreachable as
expected.
`docs/dqn-wire-up-audit.md` updated per Invariant 7.
Precondition for the MoE regime redesign per
`docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per feedback_no_htod_htoh_only_mapped_pinned.md (newly recorded): every
CPU<->GPU path in this redesign uses mapped pinned memory exclusively.
No cudaMemcpy HtoD, no Vec-to-Vec defensive copies, including in test
code. CPU is strictly read-only on the production surface.
Plan changes:
- New Task 2.0 promotes MappedF32Buffer / MappedI32Buffer from
distributional_q_tests.rs local definitions to a shared
crates/ml/src/cuda_pipeline/mapped_pinned.rs module so all kernel
test wrappers (Test 0.F, upcoming MoE tests) share one
implementation. Adds write_from_slice helper for direct host_ptr
write (no memcpy).
- Task 2.1 test wrapper rewritten to allocate mapped pinned buffers
+ write to host_ptr + read GPU-written output via host_ptr. No more
memcpy_stod / memcpy_dtov in test code.
Spec: new section 6.4 codifies the mapped-pinned-only constraint and
references the shared module + reference implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
