Layer A additive: three mapped-pinned buffers added to GpuDqnTrainer.
- wiener_state_buf (141 floats) — Pearl D state (47 producers × 3 floats:
sample_var, diff_var, x_lag). Per `feedback_no_htod_htoh_only_mapped_pinned`.
- clamp_engage_per_block_buf (2048 ints) — Pearl C engagement counters
(8 param-groups × 256 max blocks per Adam launch).
- producer_step_scratch_buf (47 floats) — per-producer per-step
step_observation output. Host applies Pearls A+D to map step_obs to
ISV bound slot via pearls_ad_update (Task A3, pending).
All three zero-initialized at construction (Pearl A sentinels). Reset
registry entries follow in Task A12.
No consumers wired yet — buffers are reserved but unread. Behavior
unchanged. cargo check clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 important + 1 optional findings from code quality review:
1. layout_fingerprint_seed() now lists all 40 SP4 slots and bumps
`ISV_TOTAL_DIM=131` -> `ISV_TOTAL_DIM=171`. Without this, the fail-fast
checkpoint-load guard would not detect the SP4 layout extension —
a binary built against new code would falsely compare-equal to old
checkpoints' fingerprints.
2. Added `SP4_BRANCH_COUNT=4` constant and `debug_assert!(branch <
SP4_BRANCH_COUNT)` in `atom_pos_bound`. Test extended to verify
atom_pos_bound max does not alias into WEIGHT_BOUND family.
3. Refreshed stale content in docs/isv-slots.md: ISV_TOTAL_DIM 96->171,
fingerprint location [37..39)/[47..49) -> [115..117), table entries
[94]/[95] -> [115]/[116].
4. Added `debug_assert!(group < SP4_PARAM_GROUP_COUNT)` to weight_bound,
adam_m_bound, adam_v_bound, wd_rate accessors (symmetric with the
atom_pos_bound change).
cargo check clean, cargo test slot_layout_is_contiguous_and_total_40 passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mech 10 with 100× multiplier (commit 55576d047) restored F1 to +81.16
and F2 to +80.55, both above baseline 55.87 — F1 cascade definitively
fixed. But F0 dropped to 21.09 (vs v2's 45.47). HEALTH_DIAG shows
Q-values O(0.01) and noisy-σ at 0.032 in F0 epoch 1 — natural h_s2
magnitudes should be tiny (~0.1-5), so 100×max(isv,1)=100 floor
shouldn't bind. Yet F0 plateaus at epoch-1's 21.09 with no improvement
in epochs 2-5 — same value as smoke-test-d25vq's over-clipped F0.
Two hypotheses to distinguish:
1. Clamp value is over-tightening h_s2 in F0 (despite the math).
2. The new launch in submit_forward_ops_main shifts captured-graph
kernel scheduling, changing TF32 sgemm accumulator order, which
pushes F0 into a different deterministic convergence basin.
Investigation: raise multiplier to 1e30. With 1e30 × max(isv,1) =
1e30 always, launch_clamp_finite_f32 degrades to a pure NaN/Inf
sanitizer — `isfinite(v) ? v : 0.0f` — leaves finite values
untouched (|v| < 1e30 always). The launch still happens
(scheduling preserved), value-clamping effectively removed.
Decision criteria for the smoke at this commit:
- F0 returns to ~45 + F1 still healthy → 100× was over-clamp; ship a
middle multiplier (1e3 or 1e4) that protects F1 without binding F0
- F0 returns to ~45 + F1 NaNs again → clamp value matters for F1;
search for a multiplier that protects F1 without over-tightening F0
- F0 stays at ~21 → kernel-scheduling artifact, not Mech 10 multiplier;
F0=21 is launch-presence-induced basin shift independent of clamp
value, and the right SP3 close-out is to keep Mech 10 at 100× and
document F0 RNG variance as residual
Temporary diagnostic commit; final SP3 close-out reverts to the proper
multiplier informed by the smoke result.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real root cause of the F1 step-1000 NaN, identified by smoke-test-2xrxk
diagnostic. With Mech 9 clamping weights at 100×Q_ABS_REF=5000, F0/F2
trained clean (weights stay at natural scale ~0.5) but F1's regime
shift drove weights toward the clamp ceiling. With weights pinned at
5000 and K=256 inner-dim, forward GEMM accumulators compound by
~80000× per layer; trunk depth ≥6 (encoder + VSN + GRN-blocks +
bottleneck) hits f32_max ≈ 3.4e38 in the chain.
Slot 12 (save_h_s2) firing was the smoking gun — h_s2 itself goes
non-finite mid-forward. Slots 39+43+48 (IQN-side) stayed clean,
disproving the close-out v2 hypothesis (IQN partial refactor) and
confirming the surface that wasn't protected: ACTIVATIONS.
Mech 1+2 clamp targets+atoms. Mech 9 clamps weights. Nothing clamped
the trunk's intermediate hidden state. Mech 10 closes that gap by
clamping h_s2 at 100 × H_S2_RMS_EMA.max(1.0) immediately after the
trunk + temporal pipeline finalises it, before any downstream consumer
reads it (branches, IQN, value head, replay save).
Implementation:
- Reuses existing launch_clamp_finite_f32 (dqn_utility_kernels.cu:462)
which does isfinite(v) ? clamp(v) : 0.0f — NaN→0, Inf→0, finite
clamped. No new kernel.
- ISV-driven via H_S2_RMS_EMA_INDEX=96 (already consumed by mag_concat
adaptive scale per P4.T2c.3c.6 — no new slot).
- ε on multiplier per SP1 pearl: isv[96].max(1.0). Cold-start RMS
near 0 floors the bound at 100×1=100, not at 0.
- Inline NaN check (slot 12) preserved BEFORE the clamp per SP1
diagnostic-ordering pearl — sentinel sees the original Inf/NaN
before sanitization replaces it with 0.
- Insertion site: end of submit_forward_ops_main 1c temporal-pipeline
block (after mamba2_step + apply_regime_dropout — the last writers
to save_h_s2), immediately before launch_curiosity_inference and the
loss path. Captured in the same forward child graph.
New diagnostic slot 49 = h_s2_max_abs at threshold 1e3 × H_S2_RMS_EMA.max(1.0).
Mirrors Mech 5 family pattern (clamp at 100×, diagnostic at 1000×) —
sentinel that NEVER fires under normal Mech 10 operation. If it does
fire, the activation clamp is disabled or H_S2_RMS_EMA itself drifted.
Buffer bumps: nan_flags_buf 49→50, metadata 25→26, fused-kernel block
count 25→26. Kernel signature gains second ISV-derived float param
h_s2_rms_ema_eff (distinct from q_abs_ref_eff — h_s2 lives in
activation space, not Q space; don't conflate the two ISV bases).
Both name tables in training_loop.rs (halt_nan + halt_grad_collapse)
extended to 50 entries.
No new ISV slots, no new kernels, no graph-topology surprise (one
extra launch in captured graph). cargo check clean. Audit docs
docs/dqn-wire-up-audit.md (Mech 10 entry prepended) and
docs/dqn-backward-nan-audit.md (Mech 10 row in mechanism table + slot
49 row in slot-allocation table + ISV bindings note + full Mech 10
section) updated per Invariant 7.
Validation: deferred to one L40S smoke. Expected: F1 trains past
step 1000; slots 36-43 + slot 49 stay quiet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The SP3 close-out v2 commit 1bcc70392 added the slot-48 "iqn_weight_max"
name to the halt_nan path's name table at training_loop.rs:2057 but missed
the symmetric halt_grad_collapse path's table at line 2143. With
nan_flags_buf now sized 49, the halt_grad_collapse formatter would
out-of-bounds index when slot 48 fires under that path.
One-line additive fix: append the same `"iqn_weight_max"` entry (with the
same SP3 close-out v2 comment) so both name tables match the buffer size.
Wire-up audit updated to document the symmetry requirement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real fix for the F1 step-3540 NaN. Commit 8956c2fb7 wired Mech 9
(post-Adam |p_val| clamp) into ONE Adam kernel (dqn_adam_update_kernel)
out of FIVE in the codebase. The slot-26 GEMM (apply_iqn_trunk_gradient
output) computes `grad_iqn @ W_iqn^T`; W_iqn lives in IQN's separate
param buffer, updated by iqn_adam_kernel — never reached by Mech 9.
Slots 44-45 only cover trunk + heads, leaving IQN weights without a
diagnostic. The chain: IQN weights drift via iqn_adam_kernel → finite
gradient × non-finite weight → slot-26 NaN. Same partial-refactor class
as Mech 4's missing GpuAttention reset (caught by B5 audit), corrected
the same way: every consumer of the contract migrates together.
Extended Mech 9 to all four remaining Adam kernels:
- iqn_adam_kernel (iqn_dual_head_kernel.cu)
- iql_adam_kernel (iql_value_kernel.cu)
- attn_adam_kernel (attention_backward_kernel.cu — used by GpuAttention + GpuTlob)
- curiosity_adam_step (curiosity_training_kernel.cu)
Same `if (weight_clamp_max_abs > 0.0f) p = fminf(fmaxf(p, -bound), bound)`
pattern. Same `100 × Q_ABS_REF.max(1.0)` ISV-driven bound. Each launch
site computes the bound host-side and passes it as the trailing kernel
arg, mirroring the existing dqn_adam_update_kernel pattern. DT launch
keeps the 0.0 disable (offline-RL, outside SP3 scope). Curiosity reads
ISV from FusedTrainingCtx in training_loop and threads through the
collector wrapper (collector doesn't own ISV).
Added IQN-weight diagnostic slot 48:
- nan_flags_buf 48 → 49
- Fused-kernel block count 24 → 25; new branch for absolute slot 48
(relative slot 24): threshold = 1e3 × q_abs_ref_eff (matches slot 44-45)
- Metadata buffers (nan_check_buf_ptrs/_lens) populate slot 48 with
IQN online_params pointer + length (new public accessors on GpuIqnHead)
- name table (halt_grad_collapse): 49 entries with "iqn_weight_max"
- Audit doc: slot 48 row in Mech 5 table; Mech 9 row updated to span
all 5 Adam kernels
No new ISV slots. No graph-topology change beyond the +1 block in the
already-fused NaN check. cargo check -p ml --lib clean (12 pre-existing
warnings, none new).
Validation: deferred to one L40S smoke at this HEAD. Expected: F1
trains past step 3720 (Mech-6-only ceiling) and slots 36-43 + new
slot 48 stay quiet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Coordinated close-out of SP3 Q-learning numerical-stability per
feedback_no_partial_refactor.
REVERT Mech 8 (slow_ema fold-boundary reset). smoke-test-rxhjh on b8a7ac6f7
showed F1 NaN at step 2300 — WORSE than the 3720 ceiling with Mech 6 alone.
Persistent slow_ema across folds was providing unintentional F1 protection
(anchored at F0's smaller grad scale → tighter Mech 6 upper_bound during F1
ramp-up). Resetting loosened the clip and accelerated Adam saturation.
Removed: reset_grad_norm_slow_ema() method + reset_for_fold call site.
Kept: grad_norm_slow_ema_pinned field (consumed by Mech 6).
Kept: Mech 6 multiplier at 100× (already restored from the 5× experiment
in the Mech 8 commit; correct steady-state value).
ADD Mech 9 (post-Adam ISV-driven weight clamp). Root cause being targeted:
cuBLAS sgemm f32 accumulator overflow at slots 26 (iqn_trunk_m) + 32
(bn_d_concat_buf) at F1 step ~3540 with INPUTS clean (slots 27, 35
unflagged). The matmul output saturates because the WEIGHT matrices have
drifted to extreme-but-finite magnitudes via Adam updates over thousands
of steps. Mech 1 (gradient clamp) and Mech 6 (grad-norm clip) bound
gradients, not parameters — neither prevents this drift.
Mech 9 clamps |p_val| ≤ 100 × Q_ABS_REF.max(1.0) inside dqn_adam_update_kernel
after the L1 proximal step. ISV-driven (slot 16, ε on multiplier per SP1
pearl). 1 OOM below slot 44-45 diagnostic threshold (1e3 × Q_ABS_REF) so
the regression sentinel retains 10× firing headroom — symmetric with
Mech 1's clamp/diagnostic ratio.
Implementation:
- dqn_adam_update_kernel: new trailing arg `weight_clamp_max_abs`.
Clamp `p_val = fminf(fmaxf(p_val, -bound), bound)` after L1 step.
- All Adam launch sites in gpu_dqn_trainer.rs: compute bound from
read_isv_signal_at(Q_ABS_REF_INDEX).max(1.0) × 100.0 host-side, pass
as final arg.
- No new ISV slots. No new kernels. No graph topology change.
Validation: deferred to one L40S smoke at the new HEAD. Expected:
- F1 trains past step 3720 (Mech-6-only ceiling) — Mech 9 closes SP3.
- F1 still NaNs at similar steps — close SP3 honestly with documented
residual pointing at SP4/SP5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User-identified root cause: grad_norm_slow_ema (α=0.001, half-life
~693 steps) persists across fold boundaries while every other
distribution-tracking signal (Adam m/v, target nets, atom positions)
gets reset. Mech 6's upper_bound formula (100 × slow_ema × isv) was
anchored to the WRONG scale during F1 ramp-up, driving the
non-monotonic multiplier-tuning dance across smokes:
- smoke-test-fxvkk (mult=100×): F0=44, F1 NaN @ 3720
- smoke-test-ftdjz (mult=100×, +Mech7): F0=38, F1 collapse @ 2040
- smoke-test-d25vq (mult=5×): F0=21, F1 collapse @ 2820
The multiplier was searching the wrong dimension — the anchor itself
was stale.
Three coordinated changes (per feedback_no_partial_refactor):
1. RESTORE Mech 6 multiplier 5× → 100×. The original 100× was correct
for steady-state; the F1 saturation was driven by anchor staleness,
not multiplier looseness. Tightening it harmed F0 (over-clip during
ramp-up when slow_ema lagged grad_norm_ema).
2. ADD reset_grad_norm_slow_ema method on GpuDqnTrainer. Zeroes the
mapped-pinned scalar. First step of new fold builds up slow_ema
fresh, with MIN_CLIP=1.0 floor active during the brief transient.
3. WIRE Mech 8 call in fused_training.rs::reset_for_fold, alongside
Mech 4's existing Adam resets. Mech 6's anchor now aligns with
the new fold's grad scale from step 1 — same philosophy as Mech 3
(target net hard-sync) and Mech 4 (Adam EMA reset).
Net SP3 design: Mech 6 stays at 100× multiplier (broad, principled
headroom), Mech 8 keeps the anchor honest. The pair is more robust
than either change alone:
- Without Mech 8: anchor is stale, multiplier tuning has no winning
setting (5× hurts F0 ramp, 100× lets F1 saturate at slot 36).
- With Mech 8: anchor is fresh per fold; 100× multiplier provides
legitimate per-step headroom over CURRENT fold's grad norm.
Mech 7 stays reverted (per-element clip was misdiagnosis — over-clipped
legitimate gradient outliers without addressing the saturation root
cause).
F0 risk: low — F0 starts with slow_ema=0 anyway (cold start), so Mech 8
is a no-op on F0. Only changes F1+F2 fold-boundary behavior.
F1+F2 expectation: Mech 6's upper_bound now scales with the CURRENT
fold's grad norm, providing legitimate ~10× headroom per step without
allowing Adam EMA saturation. Slot 36-42 should stay quiet.
Smoke smoke-test-ftdjz (commit d9a4d98a3, Mech 6 + Mech 7) regressed
both F0 (44 -> 38.82, per-element cap clipped legitimate outliers) and
F1 (grad-collapse at step 2040 vs 3720 with Mech 6 alone). Slots 36-42
STILL fired — Mech 7's per-element clip didn't prevent Adam EMA
saturation, just slowed training to grad-collapse.
Re-analysis: Mech 6's 100x multiplier on the upper-bound formula was
mismatched with the slot 36 threshold ratio. Per-element gradient max
<= adaptive_clip = 100 x slow_ema x isv ~= 200. Adam m_X steady-state
reaches 200, exceeding slot 36 threshold of 100*isv = 100. Mech 6 was
doing its job but the bound was wider than the diagnostic threshold.
Two coordinated changes (per feedback_no_partial_refactor):
1. REVERT Mech 7 (per-element clip in dqn_adam_update_kernel). The
per-element approach was misdiagnosis — clipping post-global-clip
gradients tighter than legitimate per-element variance harms F0
training without addressing Adam saturation root cause. Kernel
returns to its post-Mech-6 state (blob 546feee48).
2. TIGHTEN Mech 6's upper-bound multiplier from 100x to 5x. Standard
DL practice (5-10x steady-state grad norm). Per-element gradient
max becomes <= 5 x slow_ema x isv ~= 10, well below slot 36
threshold of 100. Adam m_X EMA stays bounded <= 10 — slots 36-42
should not fire.
Why 5x and not 10x: slot 36 threshold is 100 x isv. 5x slow_ema
(~=10 absolute) leaves 10x headroom against the threshold, providing
robust margin. 10x slow_ema would be 20 absolute, only 5x margin —
risk of fluctuations triggering slot 36.
Why not tighter (e.g., 2x): per-step gradient norms can legitimately
spike to 5x slow_ema in normal training (e.g., gradient resumption
after warmup); tighter bounds would over-clip.
F0 risk: low — F0 typical adaptive_clip values are bounded by Mech 6's
upper anchor only when the EMA-driven clip exceeds 5x slow_ema, which
is rare in steady F0 training. Cap should be a no-op for F0.
F1 risk: prevents the saturation pathway diagnosed by smoke
smoke-test-ftdjz. Validates by running the next smoke at this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke smoke-test-fxvkk (commit 48c25d999, Mech 6 anchored clip) F1-NaN'd
at step 3720 with slots 36-38, 40-42 still firing — Adam EMAs saturated
despite Mech 6 capping the global clip threshold.
Diagnosis: Mech 6 bounds the AGGREGATE L2 gradient norm, but Adam m/v
EMAs are PER-ELEMENT. A gradient with one large element (e.g.,
element_X = 100, rest small) has L2 norm ≈ 100, passing a clip
threshold of 1000 untouched. Adam m_X EMA accumulates the large
element; β1=0.9 steady-state gives m_X ≈ 1000, exceeding slot 36
threshold (100 × isv).
Fix: in dqn_adam_update_kernel, after the global L2 clip is applied,
clip EACH gradient element to ±per_element_cap where:
per_element_cap = 10 × sqrt(adaptive_clip / total_params)
Rationale:
- sqrt(adaptive_clip / N) is the average per-element contribution to
the L2 norm budget
- 10× allows legitimate per-element deviations up to 10× average
- Scales with adaptive_clip — when global clip is doing its job
(Mech 6), per_element_cap is tight enough to prevent saturation
- When global clip is loose (post-fold warmup), per_element_cap
scales with it — never tighter than the global clip's intent
Together with Mech 6, prevents Adam saturation at both the aggregate
(L2 norm) and per-element levels. Closes the residual pathology after
Mech 6's partial 3060→3720 improvement.
Adds an upper bound to update_adaptive_clip's new_clip formula to
prevent clip-drift pathology: consecutive elevated samples were
ratcheting the EMA-driven clip threshold upward without bound,
eventually rendering clipping a no-op against in-distribution drift.
Smoke smoke-test-5rqzs (commit b9edccfc1) F1-NaN'd at step 3060 with
Mech 5 diagnostic slots 36-38, 40-42 firing — DQN main Adam m + v
EMAs saturated. Diagnostic confirmed Adam saturation as the root cause.
Diagnostic chain identified: grad_norm_ema ratchets up via repeat-
sample compounding (winsorizer caps single-sample but not consecutive
elevated samples) -> clip threshold > actual grad_norm -> clip no-op
-> Adam m/v poisoned -> saturate at slot 36-42 thresholds -> cuBLAS
overflow at slots 26+32 -> loss=34.68 grad=inf.
Fix: cap new_clip by grad_norm_slow_ema * 100 * isv[Q_ABS_REF=16].max(1).
- grad_norm_slow_ema (existing alpha=0.001 slow EMA): anchors against
legitimate steady-state grad norm
- 100x: headroom for per-step deviations
- isv[Q_ABS_REF].max(1): adaptive scale per
feedback_isv_for_adaptive_bounds; epsilon on multiplier per SP1 pearl
- .max(MIN_CLIP): cold-start epsilon-floor
Standard DL practice — bounds the runaway EMA clip-drift while
preserving adaptive behavior. F0 risk: low — slow_ema * 100 is
broad headroom; F0-typical clip thresholds are well below this
cap. F1+F2 benefit by preventing the ratchet pathology.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces rsv36-rsv47 placeholders with the SP3 Mech 5 diagnostic
slot names. Both name table sites (halt_nan + halt_grad_collapse)
updated identically per shared-contract migration principle
(feedback_no_partial_refactor).
Slot names mirror the audit doc table:
- 36-39: Adam m max-abs (trunk/value/branch/IQN)
- 40-43: Adam v max-abs (trunk/value/branch/IQN)
- 44-45: Weight max-abs (trunk/heads)
- 46: target_q post-clip
- 47: atom_span_max
When the fused kernel sets a slot bit, the readback log line names
the buffer that exceeded its ISV-derived threshold — providing
direct observability for SP3 mechanism effectiveness.
Extends dqn_nan_check_fused_f32_kernel to handle 24 slots (12 NaN-only +
12 ISV-threshold). Per-slot thresholds computed inline in the kernel
from a single q_abs_ref_eff arg (no HtoD per step; no separate
thresholds buffer needed). Slot index → threshold mapping:
- 12-15 (slots 36-39): Adam m ≥ 100 × q_abs_ref_eff
- 16-19 (slots 40-43): Adam v ≥ 1e6 × q_abs_ref_eff²
- 20-21 (slots 44-45): Weight max ≥ 1e3 × q_abs_ref_eff
- 22 (slot 46): target_q ≥ 95 × q_abs_ref_eff (= 9.5 × max_abs_target_q)
- 23 (slot 47): atom span ≥ 190 × q_abs_ref_eff (= 9.5 × max_atom_abs × 2)
nan_check_buf_ptrs/lens resized 12→24 entries (mapped-pinned host
write at construction). populate_nan_check_meta extended with 4 new
args for IQN Adam m/v ptr+len (Option<u64>/Option<usize> — null when
IQN inactive).
Grid: 12 → 24 blocks. Single launch covers all 24 backward-path
diagnostic slots. Slot 31 (deferred) + slots 33-35 (inline elsewhere)
+ slots with null IQN entries no-op via the kernel's null-pointer
guard.
q_abs_ref_eff = max(isv[Q_ABS_REF=16], 1.0) — ε on multiplier per SP1
pearl; cold-start ISV (~0) gives q_abs_ref_eff = 1, so all thresholds
floor at their ε-floor multiplier × 1.
SP3 Mech 5 closes the diagnostic instrumentation loop — slots 36-47
fire when their threshold is exceeded, providing observability for
SP3's other 4 mechanisms' effectiveness. Fail-safe: if SP3 fix doesn't
fully resolve F1 NaN, slot 36-47 firing pattern guides the next
iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit of all Adam optimizer state in crates/ml/src/ against the existing
reset_adam_state calls at the fold-boundary in
fused_training.rs::reset_for_fold:
- DQN main (line 971) — covers m_buf/v_buf, IQN-trunk, q_attn, sel,
denoise, mamba2, ofi_embed, PopArt, Q-div EMA via
GpuDqnTrainer::reset_adam_state. Also covers VSN params (in main
params buffer slots) and aux next_bar/regime heads (slots 119-126).
- IQN head (line 1011)
- TLOB (line 1026)
- IQL-high (line 1031)
- IQL-low (line 1036)
Missing wire identified and added:
- GpuAttention (4-head feature attention on h_s2): owns a separate
(attn_m, attn_v, attn_adam_step) tuple in gpu_attention.rs, called
every training step via attn.adam_step at fused_training.rs:1962/1995,
but was never zeroed at fold boundary. Same pathology as IQN/TLOB/IQL
— fold N momentum oversizes fold N+1's first-epoch SDP + output
projection updates, compounding through the trunk gradient.
Implementation:
- Add GpuAttention::reset_adam_state mirroring the gpu_iqn_head and
gpu_tlob pattern (memset_zeros m/v + zero step counter + write 0
through pinned t_pinned for next adam_step launch).
- Wire it under `if let Some(ref mut attn) = self.gpu_attention` in
reset_for_fold, immediately after the IQL-low reset, with the same
warn-on-failure / info-on-success log pattern.
- Append SP3 Task B5 entry to docs/dqn-wire-up-audit.md documenting
the audit + fix (Invariant 7 contract).
Out-of-scope optimizers (documented in audit, not changed):
- DecisionTransformer: scoped within DT pretrain block, dropped at end
of pretrain (no fold leak possible).
- GpuCuriosityTrainer: owned by GpuExperienceCollector, external to
FusedTrainingCtx. Reset belongs at the collector layer if needed.
- MetaQNetwork: host-side observability MLP for collapse prediction
(does not feed LearningHealth or trunk gradients).
- GpuMoeHead: forward-only, no Adam state of its own.
Comprehensive coverage now per feedback_no_partial_refactor — every
in-context Adam state buffer zeros at fold transition. SP3 Mech 4
closes the "Adam EMA accumulation drives weight pathology" pathway.
Clamps C51 atom positions during all dynamic write paths to
±10 × ISV[Q_ABS_REF=16].max(1.0) via inline fminf(fmaxf(...)):
- atoms_update_kernel.cu — active GPU-driven shared atom grid
(all 4 branches via grid.x = branch_id)
- iql_value_kernel.cu::iql_compute_per_sample_support — per-sample
[v_min, v_max, delta_z] tile (delta_z recomputed from clamped span)
- experience_kernels.cu::adaptive_atom_positions — legacy single-branch
entry kept in lockstep per feedback_no_partial_refactor
Same ISV bound as Mech 1 (target_q clip) — atoms and target_q share
the magnitude scale. ε on the multiplier (`isv.max(1.0)`) per the SP1
ε-floor pearl; bound ≥ 10 even with cold-start ISV.
Closes the second of two Q-target inflation pathways: with Mech 1
capping the projection target and Mech 2 capping the atom support,
the C51 expected_q (mean of atom × prob) is bounded by
±10 × ISV[16].max(1.0) regardless of probability mass distribution.
Prevents the Adam EMA saturation → weight pathology → cuBLAS overflow
chain.
Reuses isv_signals already in scope at all three kernels — no new
kernel arg, no new launch site, no new ISV slot, no new buffer,
no new kernel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Clips denoise_target_q_buf to ±10 × ISV[Q_ABS_REF=16].max(1.0) after
compute_denoise_target_q. Reuses dqn_clamp_finite_f32_kernel from SP1
(no new kernel). Single-point clip covers all downstream consumers
(C51 / IQN / MSE).
ε on multiplier per SP1 pearl; bound at least 10 even with cold-start
ISV. F0 paper-review: F0-typical |target_q| ≤ 10; with Q_ABS_REF EMA
≈ 1-5 at F0, max_abs = 10-50 — F0 guard no-op. F1 inflation
(thousands+) gets clamped, breaking the Q-target inflation pathway
that drives Adam EMA saturation → weight pathology → cuBLAS overflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds pub(crate) accessor methods on GpuDqnTrainer and pub accessors
on GpuIqnHead exposing device pointers for the SP3 Mech 5 threshold
checks at slots 36-47:
GpuDqnTrainer (pub(crate)):
36: trunk_adam_m_ptr/_len — m_buf trunk slice (tensors [0..13))
37: value_adam_m_ptr/_len — m_buf value-head slice (tensors [13..17))
38: branch_adam_m_ptr/_len — m_buf branch-head slice (tensors [17..33))
40: trunk_adam_v_ptr/_len — v_buf trunk slice
41: value_adam_v_ptr/_len — v_buf value-head slice
42: branch_adam_v_ptr/_len — v_buf branch-head slice
44: trunk_params_ptr/_len — params_buf trunk slice
45: heads_params_ptr/_len — params_buf value+branch concat
46: target_q_ptr/_len — denoise_target_q_buf [B, 12]
47: atom_positions_ptr/_len — atom_positions_buf [4, num_atoms]
GpuIqnHead (pub):
39: adam_m_ptr/_len — IQN Adam first-moment buffer
43: adam_v_ptr/_len — IQN Adam second-moment buffer
DQN Adam state is UNIFIED in m_buf/v_buf (single TOTAL_PARAMS-sized
buffer covering trunk + heads at the same offsets as params_buf).
Slot 36/37/38 (and 40/41/42) accessors return pointers into the same
buffer at offsets computed via padded_byte_offset over the existing
compute_param_sizes layout. The kernel discriminates by slot index
for threshold checks. IQN Adam state lives separately on GpuIqnHead.
Trunk/heads param-buf split (slots 44/45) uses the same offset helper
— no new buffers, no new ISV slots, no DtoD/HtoD copies. Each ptr
accessor returns self.<field>.raw_ptr() (or +offset for slices); each
len accessor returns self.<field>.len() or computed from
compute_param_sizes.
Audit doc updated (docs/dqn-wire-up-audit.md SP3 Task B1 entry).
Used by populate_nan_check_meta_v2 (Task B6) to feed the fused
kernel's threshold-check entries when extended to 24 slots. Unused
yet — wired in B6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
run_nan_checks_post_backward now fires a single fused kernel launch
covering slots 24-35 in 12 blocks. Reduces graph-capture overhead from
8 launches to 1 per step. Slots 33-35 inline checks at backward
orchestration sites (gpu_dqn_trainer.rs:18580/:18601/:6939) unchanged
— multi-point localization preserved.
Method signature simplified — IQN pointers no longer per-step args.
populate_nan_check_meta (called once at construction in fused_training.rs)
baked Option<u64> nullity into the metadata buffer entries; the fused
kernel's null-pointer guard handles slots 27/28 when IQN inactive
without per-step branching at the Rust caller.
Both call sites in fused_training.rs (ungraphed + graph-captured paths)
updated together per feedback_no_partial_refactor.
This is the F0 regression fix — Phase B's per-step kernel-launch
overhead was the F0 cause across 3 SP1 smokes (F0 ~ 35 vs baseline
55.87). Gate 1 smoke (Task A6) validates F0 >= 53.08.
Replaces A2's CudaSlice<u64>/<i32> field types with MappedU64Buffer/
MappedI32Buffer per feedback_no_htod_htoh_only_mapped_pinned. Mapped-
pinned eliminates the HtoD copy entirely — the kernel reads via the
device-mapped pointer (cuMemHostGetDevicePointer_v2) while the trainer
writes through the same mapped pages on the host side.
Adds populate_nan_check_meta on GpuDqnTrainer (one-shot construction-
time write of 12 (ptr, len) tuples for slots 24-35). Slot 31 deferred
(null entry); slots 27/28 nullable on Option<u64> (None when IQN
inactive); slots 33-35 null (inline checks fire separately at backward
orchestration phases — kept individual for entry-point localization).
Adds launch_nan_check_fused_f32 (per-step kernel launch wrapper with
grid_dim=12, block_dim=256, base_flag_idx=24). Registers
dqn_nan_check_fused_f32_kernel in compile_training_kernels (tuple
43→44, info log 38→39 utility kernels) — same module as the per-buffer
dqn_nan_check_f32 to share the captured replay group.
Constructor-time wire-up lands in FusedTrainingCtx::new after gpu_iqn
construction (gpu_iqn is owned by FusedTrainingCtx, not GpuDqnTrainer
— mirrors the same Option<u64> arg pattern used by
apply_iqn_trunk_gradient and run_nan_checks_post_backward).
Wrapper unused yet — call-site replacement (8 individual check_nan_f32
calls in run_nan_checks_post_backward → single fused launch) lands in
A4.
Audit doc updated (Invariant 7).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 12-entry u64 ptr table + 12-entry i32 len table on GpuDqnTrainer
for the fused NaN-check kernel. Allocated as zeros; populated once
in next commit (slot pointers stable across trainer lifetime).
Slot 31 (deferred ensemble) entry will be 0 — kernel skips that block.
Two separate buffers chosen over packed-stride layout for direct
ABI match with the kernel signature (const float* const* buf_ptrs,
const int* buf_lens). Avoids stride-alignment concerns.
Unused yet — populate + wrapper + call-site replacement in next commits.
Smoke smoke-test-dr2bn (commit 19b008e1c) F1-NaN'd at step 240 — earlier
than pre-fix smoke smoke-test-xvzgk (step 890). The fix made things
worse, indicating the ε floor `(1e6 × isv).max(1e3)` is actively
destabilizing F1 startup.
Diagnosis: at fold boundary, ISV[H_S2_RMS_EMA_INDEX=96] and
ISV[Q_DIR_ABS_REF_INDEX=21] reset to 0 (per StateResetRegistry).
Formula `(1e6 × 0).max(1e3) = 1e3` makes max_abs aggressively narrow.
F1 startup gradients can have natural magnitudes > 1e3 (post-fold
Bellman-target shift); clamping them to ±1e3 destabilizes Adam EMAs,
which then drive cuBLAS GEMM accumulators into pathological inputs
that overflow → slot 26 + 32 NaN.
New formula: `1e6 × isv.max(1.0)` guarantees max_abs ≥ 1e6 regardless
of ISV state:
- ISV = 0 → max_abs = 1e6 × max(0, 1.0) = 1e6
- ISV = 0.5 → max_abs = 1e6 × max(0.5, 1.0) = 1e6
- ISV = 2.0 → max_abs = 1e6 × max(2.0, 1.0) = 2e6
- ISV = 100 → max_abs = 1e8
F0 no-op intent preserved (F0 inputs ≪ 1e6 in all states; ISV[96]≈1.0
for converged F0 → max_abs = 1e6, well above F0-typical |iqn_d_h_s2|
≤ ~10²). F1 startup gradients ≤ 1e6 are un-clipped.
The 1.0 ε floor is on the ISV multiplier (Invariant 1 carve-out per
`feedback_isv_for_adaptive_bounds`), not on the bound itself — bound
is still ISV-driven when ISV is meaningful (≥ 1.0).
Two edit sites:
- gpu_dqn_trainer.rs:6967 (apply_iqn_trunk_gradient, slot 26 path)
- gpu_dqn_trainer.rs:18631 (launch_cublas_backward_to, slot 32 path)
F0 regression to 35.24 still unsolved — separate investigation thread
within SP1 (no deferral; Phase B instrumentation timing impact suspected).
Quality-review follow-ups to commit 97f1d25f5:
1. CRITICAL: preserve regression sentinel for slots 24, 25, 26, 32.
The post-clamps in apply_iqn_trunk_gradient + launch_cublas_backward_to
ran BEFORE run_nan_checks_post_backward, silently zeroing the buffers
the diagnostic was supposed to observe. Fix: inline check_nan_f32 calls
IMMEDIATELY BEFORE each launch_clamp_finite_f32 — same pattern as the
existing slot 35 check at line 6949. Now the diagnostic fires on NaN
entry; the clamp then sanitizes (preventing propagation but preserving
the flag — flags are sticky/OR'd in dqn_nan_check_f32, never cleared).
2. CRITICAL: correct F0 paper-review line reference + reasoning. The
commit body cited "5.0 norm-clip at line 16747" — that's a memset, not
a clip. Real norm-clip lives at lines 16827-16868. Also tightened the
L2-vs-per-element reasoning: L2 norm <= 5.0 worst-case bounds per-element
|x| <= 5.0 (single-element edge), still several orders of magnitude
below the 1e6 max_abs guard.
3. Inline comment at line 6951 mislabeled the clamp target as 'slot 27
input' — actually clamps the slot 35 buffer (bw_d_h_s2 post-DtoD). Fixed.
4. Articulated pre-clamp rationale: defense-in-depth regression protection
for input buffers (slots 24, 25, 35) against future pathologies that
could create extreme-but-finite inputs. The F1 ep2 NaN was outputs
overflowing finite inputs, but the same fix template covers both
failure modes per feedback_no_partial_refactor.
5. Renamed clamp_finite_f32_kernel -> dqn_clamp_finite_f32_kernel for
consistency with sibling utility kernels (dqn_nan_check_f32,
dqn_zero_kernel, dqn_grad_norm_kernel).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Patches two cuBLAS GEMM backward operations identified by SP1 Phase B
smoke smoke-test-xvzgk (commit f139a63ee) as the F1 ep2 NaN sources:
1. apply_iqn_trunk_gradient (gpu_dqn_trainer.rs:6885+):
- Pre-call: clamp bw_d_h_s2 (the IQN DtoD-overwrite target that feeds
the GRN trunk encoder's cuBLAS Linear_a/Linear_b dW/dX/dB GEMMs;
symmetric with slot 27 source-side check) to ±1e6×ISV[96].
- Post-call: sanitize iqn_trunk_m (slot 26 output) — isfinite-or-zero
+ magnitude clamp; defence-in-depth.
2. launch_cublas_backward_to main backward path:
- Pre-call: clamp d_value_logits_buf + d_adv_logits_buf (slots 24/25
inputs that feed bw.backward_full's dueling/branch backward GEMMs)
to ±1e6×ISV[21].
- Post-call: sanitize bn_d_concat_buf (slot 32 output, gated on
bottleneck_dim > 0).
Both fixes use the new clamp_finite_f32_kernel utility (CUDA — replaces
NaN/Inf with 0 + clamps finite values to ±max_abs) + launch_clamp_finite_f32
(Rust wrapper). Kernel lives in dqn_utility_kernels.cu (already in
build.rs); loaded into the same module as the NaN check kernels in
compile_training_kernels — both call sites land inside captured children
(forward_child for slot 32, aux_child for slot 26) and use only stream-
bound launch_builder, so the kernel is graph-safe.
ISV-driven max_abs bound = 1e6 × isv[<slot>] with 1e3 ε floor for
uninitialized state (Invariant 1 carve-out per
feedback_isv_for_adaptive_bounds). Wide guard band — F0 inputs sit
several orders of magnitude below the threshold (F0 ISV[96] ≈ 1.0
LayerNorm RMS, F0 ISV[21] ≈ 1.0 Q-value scale; F0 |iqn_d_h_s2| ≤ ~10²,
F0 |d_value_logits| ≤ ~10³ post the existing 5.0 norm-clip at
line 16747), so guards are no-ops on F0; F1 overflows trigger clamp.
F0 paper-review pre-smoke confirmed.
Combined RELATED commit per feedback_no_partial_refactor — both kernels
share the same unsafe-pattern (cuBLAS GEMM extreme-intermediate-product
overflow) + the same fix template, so they ship together.
Phase B slots 24-35 remain as permanent diagnostic infrastructure;
will catch any future regression of this NaN class.
SP1 Phase D validation smoke pending.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires per-step NaN checks on backward-path kernel outputs. Coverage
per audit per-slot accessor table (docs/dqn-backward-nan-audit.md
:530-548).
GpuDqnTrainer::run_nan_checks_post_backward (NEW) — fire-once-at-end:
- 24 d_value_logits_buf (post-c51_grad value gradient)
- 25 d_adv_logits_buf (post-c51_grad branch advantage)
- 26 iqn_trunk_m (apply_iqn_trunk_gradient cuBLAS bwd output)
- 27 iqn_d_h_s2_buf (IQN backward dh_s2; arg from FusedTrainingCtx)
- 28 d_branch_logits_buf (IQN production backward; arg from caller)
- 29 cql_d_value_logits (CQL gradient output)
- 30 aux_dh_s2_nb_buf (Aux next-bar backward dh_s2)
- 32 bn_d_concat_buf (Bottleneck Linear backward dy)
Inline checks during backward orchestration:
- 33 bw_d_h_s2 post-main (in launch_cublas_backward_to, after
backward_full + branch concat accum,
BEFORE aux_heads_backward SAXPY)
- 34 bw_d_h_s2 post-aux (in launch_cublas_backward_to, after
aux_heads_backward SAXPY)
- 35 bw_d_h_s2 post-iqn (in apply_iqn_trunk_gradient, after
graph_safe_copy_f32 DtoD overwrite,
BEFORE encoder_backward_chain consumes)
Sequential 33→34→35 fire pattern localises the NaN entry point:
- 33 alone fires → main backward chain (c51 + MSE + branch concat)
- 34 fires after 33-clean → aux SAXPY (aux_heads_backward)
- 35 fires after 34-clean → IQN DtoD or per-sample IQN backward
Slot 31 (ensemble_d_logits_buf) cleanly deferred per Task 2 commit
387335e2b — owner is FusedDqnTraining (different struct); will be
un-deferred when ensemble Phase B saxpy guards are verified.
IQN pointers (slots 27, 28) passed as Option<u64> from FusedTrainingCtx
because GpuDqnTrainer does NOT own GpuIqnHead (Task 3 deviation
finding, audit lines 535-536). None case (when iqn_lambda == 0.0
and gpu_iqn = None) cleanly skips slots 27/28 — semantically honest
"buffer doesn't exist this run" rather than false-clean signal.
Pattern matches apply_iqn_trunk_gradient(iqn_d_h_s2_ptr: u64, ...)
at gpu_dqn_trainer.rs:6843.
Call sites (atomic — feedback_no_partial_refactor):
- fused_training.rs:1519 ungraphed step path
- fused_training.rs:2324 capture_training_graph closure (post_aux child)
- gpu_dqn_trainer.rs::launch_cublas_backward_to (slots 33, 34 — captured
in forward child via submit_forward_ops_main)
- gpu_dqn_trainer.rs::apply_iqn_trunk_gradient (slot 35 — captured in
aux child via submit_aux_ops)
Each check uses existing check_nan_f32(buf_ptr, len, flag_idx) —
single-block GPU reduce, no atomicAdd, no DtoD/HtoD/HtoH copies, no
per-step DtoH. Lengths use CudaSlice::len() where possible (auto-syncs
with allocator padding); inline arithmetic for slot 28 since
d_branch_logits_buf.len() is private to GpuIqnHead. Flags accumulate
within fold; reset at fold boundary via reset_nan_flags(). Readback
flow (commit d1808df14) consumes them in BOTH halt_nan and
halt_grad_collapse paths via name tables annotated in commit 387335e2b.
Permanent diagnostic infrastructure — stays as production-grade
regression sentinel after the surgical fix lands.
Adds 5 new pub(crate) accessor methods exposing backward-path buffer
device pointers for Task 4's per-step NaN checks (slots 24, 25, 28,
29, 30 per audit per-slot table at docs/dqn-backward-nan-audit.md
:530-548):
GpuDqnTrainer (4 new):
- d_value_logits_buf_ptr (slot 24 — post-c51_grad value gradient)
- d_adv_logits_buf_ptr (slot 25 — post-c51_grad branch advantage)
- cql_d_value_logits_ptr (slot 29 — CQL gradient output)
- aux_dh_s2_nb_buf_ptr (slot 30 — aux next-bar backward dh_s2)
GpuIqnHead (1 new):
- d_branch_logits_buf_ptr (slot 28 — production IQN backward output,
iqn_quantile_huber_loss)
Slot 27 (iqn_d_h_s2_buf) reuses existing GpuIqnHead::d_h_s2_raw_ptr()
at gpu_iqn_head.rs:1660 — no new method per feedback_no_legacy_aliases:
the existing accessor is already public and sufficient; renaming +
chasing the single call site adds churn without value.
Slots 26, 32, 33-35 reuse pre-existing handles:
- 26: self.ptrs.iqn_trunk_m
- 32: self.bn_d_concat_buf() (existing, returns &CudaSlice<f32>)
- 33-35: self.ptrs.bw_d_h_s2 (3 different Task 4 call sites)
Slot 31 (ensemble_d_logits_buf) deferred per Task 2 commit 387335e2b
(cross-struct on FusedDqnTraining).
DEVIATION FROM PLAN: the plan called for "delegate accessors on
GpuDqnTrainer for slots 27/28" — structurally invalid because
GpuDqnTrainer does NOT own GpuIqnHead. The IQN head is owned by
FusedTrainingCtx (fused_training.rs:289) alongside the trainer at
line 234. The audit's per-slot accessor table (lines 535-536) is
correct: accessors land on GpuIqnHead. Task 4's
run_nan_checks_post_backward will receive IQN pointers as u64
arguments from the FusedTrainingCtx call site — same pattern already
in use at gpu_dqn_trainer.rs:6843
(apply_iqn_trunk_gradient(&mut self, iqn_d_h_s2_ptr: u64, ...)).
NO new scratch buffer added — the plan's bw_d_h_s2_pre_saxpy scratch
+ DtoD-copy approach was superseded by the audit's 3-call-site
reformulation. Slots 33/34/35 are post-main / post-aux / post-iqn
snapshots of the same bw_d_h_s2 (one buffer, three Task 4 invocations).
Pattern follows commit e9096c7be's GRN-block accessors (concise
pub(crate) fn name_ptr(&self) -> u64 with doc-comment referencing
slot number + audit doc + buffer semantics). Additive — no behavioral
change; new accessors consumed by Task 4's NaN check call sites.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Quality-review follow-ups to commit 53bc0bc50 (nan_flags_buf 24->48):
1. Update run_nan_checks_post_forward docstring (gpu_dqn_trainer.rs:14842-14873):
replace 24-slot map with 48-slot range summary + audit-doc pointer.
Was actively misleading after the 24->48 expansion; partial-refactor
residue per feedback_no_partial_refactor.
2. Update '[24] system' comment in training_loop.rs (around line 2044):
reflect the 48-slot post-expansion state (slots 0-23 fwd, 24-35 bwd,
36-47 reserved). Also fix stale '0..11' tracing message to '0..47'.
3. Slot 31 (ensemble_d_logits_buf) annotation: flag DEFERRED + owner
on FusedDqnTraining (different struct than 24-30, 32-35). Prevents
Task 4 from blanket-launching check_nan_f32 on slot 31's null
accessor.
4. Both name-table header comments now reference the future
run_nan_checks_post_backward method (Task 4) plus the audit's
per-slot table — pre-empts contract drift when Task 4 lands a
3-way name-table dependency.
Audit-doc entry appended to docs/dqn-wire-up-audit.md SP1 Phase B
section. No behavioral change. Both name tables remain byte-identical.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expands the NaN flag buffer from 24 to 48 slots to make room for
backward-path NaN checks (slots 24-35 per audit doc per-slot table)
plus 12 reserved headroom slots (36-47) for SP2 framework + SP3
observer hooks.
Touches:
- gpu_dqn_trainer.rs: alloc size 24→48 (line ~11178); read_nan_flags
signature [i32; 24]→[i32; 48] (line ~14982); field docstring updated
(line ~2658) to reflect 48-slot layout
- fused_training.rs: pub(crate) read_nan_flags signature [i32; 24]→[i32; 48]
- training_loop.rs: BOTH name table sites (halt_nan + halt_grad_collapse
block from commit d1808df14) updated to 48 entries
Slot names use audit allocation (docs/dqn-backward-nan-audit.md per-slot
table), which supersedes the plan's placeholder names per the audit's
plan-supersession note. Slots 24-35 cover production buffers:
d_value_logits_buf, d_adv_logits_buf, iqn_trunk_m, iqn_d_h_s2_ptr,
d_branch_logits_buf, cql_d_value_logits, aux_dh_s2_nb_buf,
ensemble_d_logits_buf, bn_d_concat_buf, bw_d_h_s2 (×3 call sites).
Slots 36-47 are rsv36-rsv47 (headroom).
No behavioral change — new slots stay at zero until Task 4 wires the
kernel-output NaN checks. Buffer size reviewable by SP2.
R1: K's hardcoded shrink-and-perturb (m×0.1, v×0.01) at fold boundary
violated feedback_adaptive_not_tuned (untracked tunable knobs) AND
created a downstream pathology: tiny v_hat denominator → oversized
Adam updates 50+ steps post-reset → trunk param overshoot → save_h_s2
NaN at F1 ~step 1745 (smoke-test-bkdx5 diagnostic).
K was introduced (commit 4ef1d8ebb) BEFORE fold_warmup_factor existed
in the same commit's "K + adaptive warmup" pair. With warmup_factor
in place — ISV-driven, dampens lr+clip via lr_eff = lr_base ×
max(MIN_WARMUP_LR_FRAC, fold_warmup_factor) — K is redundant. Single
mechanism, ISV-driven, no hardcoded constants. Eliminating K leaves
m,v reset to 0 at fold boundary; warmup_factor handles cold-start.
P: expanded nan_flags_buf 16→24 with 5 GRN-stage checks
(linear_a_out, elu_out, linear_b_out, glu_sigmoid_out,
layernorm_var/out) for finer-grained source identification if R1
alone doesn't fix F1.
Predicted outcomes:
- If K's tiny-v_hat was the cause: F1 trains successfully (R1 alone)
- If different mechanism: new GRN-stage flags pinpoint which sub-
stage produces NaN, enabling layer-level fix
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Existing nan_flags_buf [16] covers 13 buffers with per-step NaN checks
inside the captured training graph. But read_nan_flags() was only
called when training guard's halt_nan fired — which checks pinned-
readback grad_norm. NaN-clamped-to-zero gradients reach the pinned
scalar as 0, not NaN, so halt_nan never fires for the explosion case.
The collapse path (halt_grad_collapse, grad < 0.01) was firing instead
without reading the flags. Add flag readback in that path when
gr.raw_grad_norm < 1e-6 (suspicious zero) — logs NaN-CLAMPED-TO-ZERO
with flagged buffer names. Genuine near-zero gradients get a separate
"no NaN flags set" log so we can distinguish the two cases.
This should pinpoint which kernel produces the F1 ep2 NaN first
(C51 KL projection? IQN aux? CQL? aux heads?). Diagnostic only —
keep after fix lands; correct gate for future regressions.
After 6+ intervention layers (A.1+A.2+A.3+F+H+K+warmup+N) F1 ep2 still
explodes with grad_norm=2.66T while F1 ep1 trains cleanly. None of our
defenses catch the explosion path. Need data to pinpoint WHICH signal
explodes first.
Per-step FOLD_EXPLOSION_DIAG in fold >= 1: when grad_norm > 1000 OR
jumps 5x from prior guard step, fire diagnostic warn with:
- loss decomposition: total / c51 / mse / iqn (pinned readback, no DtoH)
- Q range: q_min / q_mean / q_max via cold-path reduce
- atom positions: per_sample_support[0,d0] + [0,d2] (v_min, v_max, dz)
- prev grad norm + ratio for context
- trigger flag + post-trigger trajectory countdown
After trigger, continues emitting for 5 guard steps so the explosion
trajectory is captured (not just the first crossing).
Plus an unconditional FOLD_EXPLOSION_DIAG[F1_END_EP1] baseline log at
the end of fold 1 epoch 0 — healthy state immediately preceding the
explosion. Compare-and-contrast with per-step explosion frames pins
the runaway driver.
CQL and ensemble losses are not pinned-readback (transient device
buffers consumed inside the training graph) and are explicitly absent
from the decomposition. If none of {c51, iqn, mse} is the runaway
driver but total_loss still explodes, that implicates the unexposed
CQL/ens path — the absence is itself diagnostic information.
Implementation:
- Three diagnostic-only fields on DQNTrainer: current_fold,
last_logged_grad_norm, explosion_diag_steps_remaining. Reset
last_logged + remaining in reset_for_fold; current_fold
overwritten unconditionally at fold-loop entry.
- Three accessors on FusedTrainingCtx: explosion_diag_loss_components
(pinned), explosion_diag_atom_range (12-elem cold DtoH),
explosion_diag_q_range (reduce + 28B DtoH).
- Three accessors on GpuDqnTrainer: c51_loss_pinned_value,
mse_loss_pinned_value, stream_for_diag.
Diagnostic-only — remove once root cause identified. Not for prod.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke smoke-test-xw4c6 showed F1 ep2 grad_norm=8.4B polluting the
adaptive_clip EMA -> next adaptive_clip threshold became huge ->
subsequent extreme grads passed unclipped -> NaN propagation -> grad
collapse -> fold 1 fails despite K+warmup fixing the first-step problem.
Add Winsorized clamp on raw_grad_norm before the adaptive_clip EMA
update: clamped_sample = min(raw, K * previous_adaptive_clip) where
K = 100 (numerical-stability bound, not a tuned constant — explicit
"single sample can grow EMA at most 100x in one step" semantics).
Companion observability log emits GRAD_CLIP_OUTLIER warning per clamp
event so we can see when this fires in subsequent smokes.
Fast/slow grad_norm EMAs (driving warmup factor) intentionally NOT
winsorized — they're a stability signal that SHOULD respond to
outliers, providing extra warmup damping when the system is unstable.
Predicted impact on Plan C smoke F1 ep2: 8.4B grad -> clamped to
~3000 for EMA -> next adaptive_clip caps at ~1000 -> subsequent F1
batches get bounded gradients passed to Adam -> no NaN propagation.
F1+F2 should now train through, exposing whether the underlying
Q-target inflation requires further structural work or whether
clipping alone suffices.
Composes with K+warmup (4ef1d8ebb), A.2, A.1/A.3, F+H — fourth layer
of the adaptive gradient-stability scaffolding (now five total:
A.2 target-sync drift dampening, K+warmup post-reset transient
dampening, N persistent clip-EMA outlier defense).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the F1 ep1 catastrophic-overshoot gap exposed by smoke-test-s9h4h
(F0 succeeded with Best Sharpe = 36.03; F1 ep1 grad_norm = 355,009 — 5
orders of magnitude larger than F0 steady-state ~10 — leading to NaN
propagation and grad-clamp-to-zero early stop). Combined fix: (1) Adam
shrink-and-perturb at fold boundary (replace m=0/v=0 with m*=0.1, v*=0.01
to preserve direction while damping magnitude), and (2) a single
adaptive ISV signal driving BOTH lr_eff and clip_eff dampening over
the fold's first ~50 steps.
K — Adam shrink-and-perturb in `GpuDqnTrainer::reset_adam_state`:
- m *= 0.1, v *= 0.01 via existing `dqn_scale_f32_kernel` (loaded as
`scale_f32_ungraphed`); t_pinned still zeroed so bias correction
restarts. Architectural constants (preserve direction / lose magnitude
history) per `feedback_isv_for_adaptive_bounds.md` Invariant 1
carve-out — not tuned. Composes with existing param shrink-and-perturb
(`alpha=0.8`) in `FusedTrainingCtx::reset_for_fold`. Root cause for
F1 overshoot: m=0,v=0 → first Adam step ≈ lr × g / ε → 6 OoM
amplification.
New CUDA kernel `fold_warmup_factor_kernel.cu`:
- Single-block single-thread cold-path producer mirroring
`q_drift_rate_ema_kernel.cu` / `moe_lambda_eff_kernel.cu` shape.
- Reads two grad-norm EMAs (fast α=0.1, slow α=0.001) plus host-passed
step counter; writes ISV[FOLD_WARMUP_FACTOR_INDEX=130] = clamp(fast/slow, 0, 1).
- Bootstrap branches (steps_observed < 200, slow EMA < 1e-6) emit
factor=1.0 (no damping during cold-start). No atomicAdd; no DtoH.
New ISV slot Q_DRIFT_RATE_INDEX → FOLD_WARMUP_FACTOR_INDEX = 130:
- ISV_TOTAL_DIM 130 → 131; layout fingerprint shifts (checkpoint-
incompatible per `feedback_no_legacy_aliases.md`, expected for a
real architecture change).
- FoldReset entries: `isv_fold_warmup_factor` → 0.0 and companion
`isv_grad_norm_fast_ema` → 0.0 (lockstep reset per
`feedback_no_partial_refactor.md`); slow EMA persists across folds
as the cross-fold steady-state baseline.
- Two new mapped-pinned scalars on GpuDqnTrainer (grad_norm_fast_ema_pinned,
grad_norm_slow_ema_pinned) fed by `update_adaptive_clip` from the
same `gr.raw_grad_norm` observation source as the existing adaptive
clip EMA.
Two consumers, both monotone (only dampen, never excite):
- lr_eff = cosine_effective_lr × max(MIN_WARMUP_LR_FRAC=0.05, factor)
via `set_lr` per-step. New `cosine_effective_lr_base` field
on DQNTrainer composes the cosine schedule's per-epoch
baseline with the warmup factor's per-step damping (rather
than overriding the cosine schedule).
- clip_eff = clip_base × (MIN_CLIP_FRAC=0.1 + 0.9 × factor) via new
`set_active_clip` setter on FusedTrainingCtx + GpuDqnTrainer.
Composes with the EMA-derived `clip_base = grad_norm_ema × 2`
that `update_adaptive_clip` just wrote to the pinned slot.
Numerical-stability bounds 0.05 / 0.1 are Invariant 1
carve-outs.
Steady-state behaviour unchanged: factor=1 → lr_eff=lr_base,
clip_eff=clip_base. Fold-boundary behaviour: factor starts at 0 →
lr_eff = 0.05 × lr_base, clip_eff ≈ 0.1 × clip_base; rises to 1 over
~50 steps as the fast EMA catches up to the slow steady-state EMA.
Predicted impact on Plan C smoke F1: 355,009-magnitude transient grad
clipped to ~clip_base × 0.1 ≈ 1.0 (vs 10), Adam state shrunk instead
of zeroed → first step update bounded; grad recovers normally over
~50 steps. Companion to A.1 (prev_epoch_q_mean reset), A.2 (adaptive
Polyak-tau), A.3 (gradient_collapse_counter reset), F+H (kill-criterion
robustness) — completes the fold-boundary state-reset family.
Per `pearl_adaptive_moe_lambda.md` (kernel + ISV slot + bootstrap +
reset + observability template), `pearl_cold_path_no_exception_to_gpu_drives.md`
(GPU-stays-on-GPU even at cold-path cadence),
`pearl_blend_formulas_must_have_permanent_floor.md` (lr/clip floors
are permanent minimums), `feedback_adaptive_not_tuned.md` (lr+clip
ISV-driven), `feedback_isv_for_adaptive_bounds.md` (factor IS the
bound; consumers compose at runtime), `feedback_no_atomicadd.md`
(single-thread reduce), `feedback_cudarc_f64_f32_abi.md` (slot index
passed as i32), `feedback_no_partial_refactor.md` (kernel + slot +
reset + producer + 2 consumers all land together),
`feedback_no_quickfixes.md` (replaces brittle full-reset with
adaptive damping; not threshold relaxation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke smoke-test-vh9bj revealed: after F+H Q-drift kill fix, fold 0
trains successfully (5 epochs, Best Sharpe 36.03), but fold 1 fails
at epoch 1 with "Gradient collapse detected for 5 consecutive epochs"
despite epoch 1 grad_norm=296,417 (healthy). Root cause: the per-step
gradient_collapse_counter on DQN was already near patience (5) from
fold 0's late-epoch near-zero-grad steps, and DQNTrainer::reset_for_fold
didn't clear it. Plus, training_steps accumulating across folds made
the `past_warmup` gate always-true in fold 1+, removing the warmup
grace period for data distribution shifts.
Add DQN::reset_for_fold zeroing both. Wire from DQNTrainer::reset_for_fold
alongside the A.1 prev_epoch_q_mean reset. Pure additive — same
fold-boundary state-reset gap pattern as A.1, A.2, F.
Predicted impact: fold 1+ now starts with counter=0 and warmup window
restored; gradient collapse check has its full per-fold grace period.
Current criterion uses |q_mean| / |prev_q_mean| which explodes near
zero crossings (legitimate cold-start has |prev_q_mean| ≈ 0.01-0.1,
making any non-tiny current trigger the ratio threshold).
smoke-test-n9xzr fired with prev=−0.0795, curr=0.7647, ratio=9.62×
despite this being natural cold-start growth, not geometric runaway.
Replace with rolling 5-epoch window of q_means. Compute median +
MAD (Median Absolute Deviation, a 50%-breakdown estimator robust
to single outliers); kill condition becomes
|q_mean − median(window)| > 4.0 × max(MAD(window), 0.01)
AND the existing adaptive floor.
Constants are declared `const` near the kill block:
- Q_DRIFT_WINDOW_SIZE=5 (matches smoke fold length, gives MAD a
meaningful estimator without averaging across regime shifts)
- Q_DRIFT_WARMUP_SAMPLES=3 (skip until ≥ 3 priors — smaller window
degenerates to half-range MAD that trips on monotonic
trajectories)
- Q_DRIFT_DEVIATION_THRESHOLD=4.0 (4 MADs ≈ 2.7σ Gaussian-
equivalent; clear outlier without firing on every legitimate
dip; literal MAD count rather than σ because q_mean is non-
Gaussian during cold-start)
- Q_DRIFT_MIN_DEVIATION=0.01 (numerical-stability floor when
window is constant)
Window resets at fold boundary alongside prev_epoch_q_mean /
adaptive_tau (A.1 pattern — cross-fold q-stats are independent
training runs, mixing them would inflate MAD or shift the
median).
Predicted impact on smoke-test-n9xzr:
- Plan C smoke fold 0 ep2: window has only 2 priors, warmup gate
not yet satisfied → kill stays silent (was firing on ratio=9.62×)
- Genuine geometric runaway (q_mean → 62.5 from baseline 0.5 over
4 epochs): ep3 deviation = |62.5 − 2.5|/MAD=2.0 = 30 > 4 AND
|62.5| > floor=3×12=36 → kill fires correctly
Both conditions ANDed (floor + deviation), preserving the
production-safety semantics of the original criterion. The floor
check is unchanged; only the divergence detector is replaced.
Per feedback_no_quickfixes.md: this is a principled robust-
statistics replacement, not a threshold relaxation.
Per feedback_no_partial_refactor.md: window field, constants,
criterion site, and fold-boundary reset land in lockstep — the
kill criterion's contract is internally consistent across
trainer/mod.rs, constructor.rs, and training_loop.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Q-drift kill criterion's adaptive floor formula is
max(0.5, 3 × max(ISV[Q_ABS_REF=16], ISV[Q_DIR_ABS_REF=21]))
but smoke-test-n9xzr showed both ISVs stuck at bootstrap 0.05 even
when q_mean reached 0.76 — the producers (q_mag_bin_means_reduce
and q_dir_bin_means_reduce) only fired inside reduce_current_q_stats
at epoch boundary, while the per-step captured update_isv_signals
consumed the resulting scratch buffers each training step.
Concrete failure mode at α=0.05 EMA with raw ≈ 1.0:
- Epoch 0: scratch is zero-initialized, all per-step EMA pulls
toward zero, ISV[16,21] stay 0.0
- Epoch 0 boundary: reduce_current_q_stats fires, scratch becomes
q_abs_ref ≈ 1.0, single update_isv_signals writes ISV[16] = 0.05
- Epoch 1+: with stale scratch held constant between boundaries,
per-step EMA over hundreds of steps would saturate — but smoke
step counts are small enough (small batch=64, buffer=256) that
the slot stays near 0.05
- kill_floor degenerates to max(0.5, 3 × 0.05) = 0.5 forever, so
the criterion fires on legitimate cold-start growth
Fix: launch q_mag_bin_means_reduce + q_dir_bin_means_reduce in
both fused_training paths (captured adam_update_child at ~line 2228
AND ungraphed step-0 fallback at ~line 1465) immediately before
update_isv_signals. Per-step graph replays now read q_out_buf that
forward_child populated this same step, write fresh scratch, and
the captured update_isv_signals consumes it on the same stream.
Per pearl_cold_path_no_exception_to_gpu_drives.md: cold-path EMAs
fire alongside their consumers. Per feedback_no_partial_refactor.md:
graphed and ungraphed paths migrate together — same producer-
consumer contract.
After this fix, the floor will adapt: ISV[16,21] saturate to the
policy's actual q_abs_ref scale within ~60 steps at α=0.05, so by
epoch 2 the floor becomes 3 × ~0.5 = ~1.5, no longer firing on
legitimate cold-start growth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
C.4 timing bonus (experience_kernels.cu) and D.4b regime penalty
multiplied by the trade-cumulative |reward| / |final_pnl| as an
unbounded multiplicand. Across trades this compounds — bonus values
inflate as Q values inflate, then bonus rewards inflate Q values
further (Bellman bootstraps off shaped reward).
Per pearl_one_unbounded_signal_per_reward + feedback_isv_for_adaptive_bounds:
replace |reward|/|final_pnl| with ISV[Q_DIR_ABS_REF_INDEX]-bounded
variant. The unbounded multiplicand becomes the direction-branch
Q-scale EMA (already-tracked, gradient-decoupled), not the
trade-cumulative shaping output. Cap is adaptive (matches Q magnitude
as it evolves) and breaks the multi-trade compounding loop.
Discovered during Plan C Phase 2 smoke diagnosis (researcher report
2026-04-29 a25f669e9df953174). a52d99613 baseline also hits
bonus=235 — pre-existing structural pathology, not Plan-C-specific.
Other shaping sites (D.4a persist, B.2 novelty, D.4c stable) already
have all factors bounded — no fix needed.
Surgical change set:
- experience_env_step gains final `int q_dir_abs_ref_idx` param.
- C.4 (line 2521): pnl_unit = |final_pnl|/max(ISV[21], eps);
pnl_capped = min(pnl_unit, 1) * ISV[21].
- D.4b (line 2581): same pattern, reward_unit/reward_capped.
- gpu_experience_collector.rs:3800 wires Q_DIR_ABS_REF_INDEX as i32.
No new ISV slot, no new producer kernel, no layout-fingerprint shift —
ISV[21] already produced by q_stats_kernel.cu since Plan 1.
Predicted impact:
- bonus EMA drops O(100-256) -> O(0.05-2.0)
- rc[5] -> Bellman-target optimism loop broken
- Plan C smoke: Q-drift kill at F0 ep2 likely no longer fires
- a52d99613 baseline: same effect; validates pre-existing fix
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the q_drift_rate_ema kernel + ISV slot Q_DRIFT_RATE_INDEX=129
(landed in fc8dbb0a8) into the production training path:
- StateResetRegistry FoldReset entry isv_q_drift_rate -> 0.0
(companion to A.1's prev_epoch_q_mean reset; both ensure no
cross-fold leakage of drift state).
- reset_named_state dispatch arm in training_loop.rs.
- Per-epoch launch_q_drift_rate_ema call after the Q-drift kill
check, gated on the same `prev_epoch_q_mean.abs() > 1e-6`
cold-start guard. Both q_mean_curr and q_mean_prev are in scope
BEFORE the `prev_epoch_q_mean = q_mean` update, so the producer
sees the correct delta.
- tau_update_kernel.cu multiplies the cosine-scheduled,
health-coupled tau_eff by `1 / (1 + clip(ISV[129], 0, 4))` so
tau ranges [tau_base/5, tau_base] — monotone dampening under
drift; healthy runs (drift ≈ 0) unaffected.
- reset_for_fold comment in trainer/mod.rs notes the registry
entry handles the new ISV slot.
Predicted impact: Plan C fold 0 ep2 with q_mean(t)=0.82,
q_mean(t-1)=-0.018, ISV[16]+ISV[21]≈0.1 -> drift_rate ≈ 8.4 ->
clipped to 4 -> tau_eff = tau_base × 0.2 (5× slower target sync,
dampens Q-target optimism through the inflation spike).
a52d99613 fold 0 with q_mean staying ~0.21 -> drift_rate ≈ 0 ->
tau_eff = tau_base (unchanged).
Per pearl_adaptive_moe_lambda.md — canonical "EMA-tracked
diagnostic drives a controller" pattern. Per
pearl_cold_path_no_exception_to_gpu_drives.md — cold-path scalar
arithmetic stays on GPU. Per feedback_no_partial_refactor.md —
consumer (tau_update + state_reset + producer launch) all migrate
together in this commit.
Layout fingerprint already shifted by fc8dbb0a8 (slot + ISV_TOTAL_DIM
bump); no additional fingerprint shift in this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single-block single-thread ISV producer mirrors h_s2_rms_ema /
moe_lambda_eff pattern. Computes per-epoch
drift_rate = |q_mean(t) - q_mean(t-1)| /
max(|q_mean(t-1)|, ISV[Q_ABS_REF] + ISV[Q_DIR_ABS_REF], 1e-6)
clipped to [0, 4] and writes to ISV[Q_DRIFT_RATE_INDEX=129].
Includes the full ISV-contract shift required for the kernel to load:
- ISV_TOTAL_DIM 129 -> 130
- Q_DRIFT_RATE_INDEX = 129 (tail-appended after MOE_LAMBDA_EFF=128)
- Cold-start ISV[129] = 0.0 in constructor (no-op dampening factor)
- layout_fingerprint_seed entry Q_DRIFT_RATE=129 + ISV_TOTAL_DIM=130
- Cubin static, kernel field, kernel load, struct assignment
Per feedback_no_partial_refactor.md the ISV slot + dim + fingerprint
all migrate together (the wrapper references Q_DRIFT_RATE_INDEX so they
cannot be split). Tau consumer + state reset registry + per-epoch
producer launch land in the next commit.
Audit doc dqn-wire-up-audit.md updated with the kernel + ISV slot
description per Invariant 7.
No callers in this commit; layout fingerprint shifts so existing
checkpoints will fail-fast at load per feedback_no_legacy_aliases.md
(expected for a real architecture change).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan C Phase 2 T9. Verifies the production kernel's eval-mode argmax
exactly matches a Rust ground-truth E[Q] argmax on a 256-sample batch
with realistic per-direction non-degenerate C51 distributions, and
confirms Thompson explores Long+Short ≥ 40% in training mode.
The plan-prescribed approach (reuse Phase 0 Test 0.F's converged
checkpoint loader) was deferred — Test 0.F itself already exercises
the safetensors load + branching forward path. Replacement strategy
from the dispatch brief: synthetic batch via direct buffer write.
Setup:
- 256 samples, 21 atoms, per-(sample, direction, atom) C51 logits
drawn from a deterministic hash → uniform [-1, 1] (post-Xavier-init
scale of fresh C51 head outputs)
- Per-sample adaptive support [v_min ∈ [-1.0, -0.2], v_max ∈ [0.2, 1.0]]
matching the layout produced by `update_per_sample_support`
- Uniform q_values (mag/ord/urg fall through Boltzmann uniformly)
Rust ground-truth: softmax(b_logits[i, d]) · atom_vals[i, d] computed
with the numerically-stable subtract-max softmax matching the kernel's
softmax_c51_inline.
Assertions:
- Eval mode: per-sample dir_idx EXACTLY matches ground-truth argmax E[Q]
- Train mode: count(d ∈ {Long, Short}) / batch > 0.40
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan C Phase 2 T8. The plan-prescribed pre-T2 snapshot approach was
impossible (T2 had already landed); replacement strategy (ii) from the
dispatch brief — behavioral parity vs an analytical Boltzmann reference
computed in Rust — is used.
Setup forces direction = Long (d=2) deterministically via peaked C51
logits (Long peaked at v=+0.8 atom; other directions at v=-0.5), so
the kernel's Hold/Flat → mag_idx=0 short-circuit doesn't mask the
magnitude branch's Boltzmann sampling. q_values are crafted with each
branch (mag/ord/urg) peaked at a single bin with magnitude 1.0:
Mag Q = [0.0, 0.0, 1.0] peak at Full (mag=2)
Order Q = [1.0, 0.0, 0.0] peak at Market (ord=0)
Urgency Q = [0.0, 1.0, 0.0] peak at urg=1
With q_range=1.0 in all three branches, tau collapses to 1.0 and the
analytical Boltzmann probabilities are:
P(best) = 1/(1 + 2/e) ≈ 0.5767
P(other) = 1/e/(1 + 2/e) ≈ 0.2117
Tolerance: at batch=8192 the 1-σ Bernoulli noise is ~0.0055 for p≈0.58;
±5% absolute tolerance covers ~9σ. Algorithmic divergence (e.g. an
inadvertent strict-argmax substitution) would shift P(best) to 1.0 —
trivially detected by the ±5% tolerance.
Assertions:
- dir_idx == Long for every sample (eval argmax E[Q] over peaked C51)
- mag/ord/urg histograms each within ±5% of the Boltzmann reference
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan C Phase 2 T7. #[ignore]-gated GPU test that runs BOTH the
production experience_action_select kernel AND the standalone
direction_thompson_v2_test (added in commit 5de5e546a) on identical
Phase 0 Test 0.D failure-mode inputs and asserts the resulting
direction histograms are statistically indistinguishable.
Path (a) — bit-identical comparison via a shared pre-computed uniform
array — would require editing the standalone v2 kernel's signature to
accept a `float* uniforms` instead of generating its own LCG draws.
Out of scope for the 1-2-hour dispatch. Path (b) — KS-style histogram
comparison — is used.
Setup:
- Single per-direction logits tile (Phase 0 Test 0.D failure mode)
- Production: tile replicated across batch=100 000 samples; Philox
keyed on (i, timestep=0, ctr) per thread
- Standalone v2: same tile fed to single-tile launcher with n_seeds=100 000;
LCG keyed on (base_seed=0 + seed_idx) per thread
Assertion:
- KS distance over the {Short, Hold, Long, Flat} histograms ≤ 0.02
(n=100k empirical-CDF noise floor for matching distributions is
~4·sqrt(2/n) ≈ 0.018; algorithm divergences would shift mass by
O(10%) → KS ≈ O(0.1), easily detected)
- Sanity: both histograms have P(Long+Short) ≥ 0.20 (rules out
coincidental shared-mode collapse with KS ≈ 0)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
reset_for_fold cleared q_value_history (the q_mean source) but missed
the kill criterion's prev_epoch_q_mean baseline AND the adaptive_tau
modulation state. So the kill ratio at fold-N epoch 0 was computed
against fold-(N-1) epoch 5's q_mean — a stale cross-fold baseline.
Discovered while diagnosing why Plan C Phase 2 Thompson smoke triggers
Q-drift kill at fold 0 epoch 2 (researcher report 2026-04-29).
Pairs with the existing q_value_history clear; matches the
project_fold_boundary_q_drift_resolved.md pattern (kill criterion's
companion resets were partially missed).
Independent of Plan C — this is a bug in the kill criterion's
fold-boundary contract regardless of which exploration mechanism is
active.
Plan C Phase 2 T6. Adds an #[ignore]-gated GPU test that exercises
the production experience_action_select kernel directly through a
minimum-viable inline cudarc fixture (ProdActionSelectFixture). The
Plan B audit fixture (GpuExperienceCollector::new_for_test) was never
built, so each test in this Phase 2 batch builds its own buffers,
launches the kernel, and reads back action picks for assertions.
Setup mirrors Phase 0 Test 0.D failure-mode:
- Flat/Hold C51 logits = peaked at v=0 atom (δ(v=0) shape)
- Long/Short C51 logits = log-Gaussian centred at v=-0.001, σ=0.05
- Linear adaptive support [v_min=-0.5, v_max=+0.5, delta_z=0.05]
- Uniform q_values (mag/ord/urg fall through Boltzmann uniformly)
Assertions:
- Eval mode (eps=0): 10 launches at distinct timesteps produce
IDENTICAL per-sample dir_idx (deterministic argmax E[Q]).
Every sample picks Hold or Flat (E[Q] argmax bias reproduces).
- Training mode (eps_start=0.5): >=30% of 10 000 samples pick
Long or Short (Thompson explores directional alternatives).
Shared helpers (PROD_B0..3, decode_dir/mag/ord/urg, fill_peaked_logits,
fill_linear_support, ProdActionSelectFixture) added at the top of the
Phase 2 section to be reused by Tests 2.B-2.D.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The prior `test_eval_action_select_boltzmann_bounded` asserted
Boltzmann theory (P(best)≈0.366 at tau=q_range) on the direction
branch. Plan C T2 (commit 52a2663a2) replaced the direction-branch
Boltzmann path with single-distribution C51 Thompson + argmax-E[Q]
eval; the T2 amendment (5de5e546a) added 4 args to the kernel ABI
(b_logits_dir, per_sample_support, atom_positions, n_atoms). The old
test was launching with an outdated arg count and asserting a
distribution shape that no longer applies.
Migration (per feedback_no_partial_refactor):
- Rename to `test_eval_action_select_eval_argmax_picks_best`
- Build per-direction C51 logits PEAKED at distinct E[Q] values:
Short=-0.5, Hold=-0.1, Long=+0.5 (best), Flat=+0.1
- Linear adaptive support [v_min=-1, v_max=+1, delta_z=0.1] uniform
across all (sample, direction) pairs
- Run kernel in eval mode (eps_start=eps_end=0): assert P(Long)>=0.99
i.e. the kernel deterministically picks argmax(E[Q]) per sample
This validates the T2 amended ABI runs end-to-end and exercises the
production kernel's argmax-E[Q] eval branch with a controlled
distinct-E[Q] setup. Companion to Phase 2 Tests 2.A-2.D in
distributional_q_tests.rs which add training-mode assertions and
production-vs-standalone parity checks.
Verification: `SQLX_OFFLINE=true cargo check -p ml --lib --tests` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Counterpart to existing val_active_frac. Reports the fraction of
TRAINING rollout actions that were Long or Short. Required for L3
verification gate of Plan D (Phase 3) which checks Thompson is
generating directional exploration during training.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Threads b_logits_dir / per_sample_support / atom_positions / n_atoms
through experience_action_select kernel launch in
gpu_backtest_evaluator.rs's chunked val pipeline (line ~1722, inside
submit_dqn_step_loop_cublas).
The evaluator sources Q-values via the QValueProvider trait
(delegates forward to the trainer's CUDA-Graphed cuBLAS), so this
change extends the trait surface rather than duplicating the
forward:
- New trait method compute_q_and_b_logits_to(states_ptr, batch,
q_out_ptr, b_logits_out_ptr) — DtoD-copies trainer's
on_b_logits_buf into caller's chunked buffer per sub-batch
iteration alongside the existing q_out copy.
- New trait accessors per_sample_support_ptr / atom_positions_ptr /
num_atoms / total_branch_atoms (stable trainer-owned buffers).
- FusedTrainingCtx implements all of the above; trainer gains
on_b_logits_buf_ptr / atom_positions_buf_ptr /
per_sample_support_ptr_get pub accessors.
- Evaluator gains chunked_b_logits_buf field (sized
[n_windows * CHUNK_SIZE, total_actions * num_atoms] + 32*3 tail
safety), allocated in ensure_action_select_ready.
Phase 6's last-step plan_params forward keeps using the original
compute_q_values_to (b_logits not consumed there).
Plan C Task 4 — evaluator companion to T3 collector wire-up. After
this commit, all production callers of experience_action_select use
the amended kernel ABI (T2 amendment 5de5e546a) end-to-end
(feedback_no_partial_refactor).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Threads b_logits_dir (= exp_b_logits, first N*b0*NA region for direction
branch), per_sample_support, atom_positions (NULL = linear), and n_atoms
through experience_action_select kernel launch in gpu_experience_collector.
After amendment commit 5de5e546a, the kernel uses single-distribution C51
Thompson with production's adaptive per-direction support — no IQN
inference required at rollout.
Plan C Task 3 (corrected file path: collector, not trainer per
gpu_dqn_trainer.rs/gpu_experience_collector.rs distinction).
