Root cause of the step-4 NaN that has blocked the Phase 2.1 smoke gate
since session start.
`window_tensor_d` is allocated [B, K, ENCODER_INPUT_DIM=56]:
- Features [0..40) are per-snapshot market features (snap_feature_assemble_batched)
- Features [40..56) are per-batch broadcast context (rl_encoder_context_broadcast
writes via offset `idx * 56 + 40`)
But `variable_selection_fwd` and `_bwd` read x with stride
`VSN_FEATURE_DIM=40`. For row 0 the stride happens to align with the
[B, K, 56] layout. For row 1 onwards the kernel reads MIXED data
across row boundaries — broadcast context bleeds into VSN's "snap
features" view, and snap features bleed across K boundaries.
The bleed produces nonsense for most steps but is bounded enough to
train through. Around step 4, accumulated trade_context magnitudes
(time_in_trade, unrealized_R, position_lots) get large enough to
overflow VSN's softmax in the bleed-affected rows → NaN cascades
through the encoder forward → backward computes NaN gradients →
AdamW applies them → step 5's forward graph sees NaN weights → G8 abort.
Fix: introduce `VSN_X_ROW_STRIDE = 56` (= ENCODER_INPUT_DIM) and use it
for all input reads of `x` in both forward and backward:
- variable_selection_fwd:41 (x_row = x + row * 56)
- variable_selection_bwd:173 (x_i read at stride 56)
- variable_selection_bwd:204 (xj read at stride 56)
Output strides (gates_out, y, grad_W, grad_b, grad_x) remain at
VSN_FEATURE_DIM=40 because:
- gates_out / y feed Mamba2 L1 which reads at in_dim=40 stride
- grad_W / grad_b accumulate into [FEATURE_DIM, FEATURE_DIM] / [FEATURE_DIM]
- grad_x is write-only (audit: `vsn_grad_x_d` has zero downstream consumers
per grep of crates/ml-alpha/)
Verification:
- 10/10 runs clean at seed=16962, n-steps=10 (was 4-7/10 NaN before)
- 1000-step smoke clean at seed=16962: l_q≤1.76, l_v≤1.18, wr climbing
0.36→0.55 (the surfer pattern emerging as expected per
pearl_dd049d9a4_surfer_baseline_verified)
- Phase 2.1 smoke gate (l_q<3.0, l_v<2.0, no NaN over 1000 steps) PASSED
Per pearl_atomicadd_masks_v_instability commits 60e96bf55..4e629830b for
the full diagnostic chain (49 nan_scan labels, 4 sub-agent dispatches,
13 commits localizing the bug from "somewhere in the trainer" down to
this single line).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
104fe81ca8