Production training run train-multi-seed-bn42w showed label_scale=5481
(raw_close magnitude) at epoch 4 vs smoke's ~25 at the same commit
29b1d34c6. The 5-layer data-loading defense + DBN spread filter + target
stride/column fix all in place — yet column 0 of next_states_buf still
sees raw-price-magnitude values in production but not in the
multi_fold_convergence test path.
This one-shot diagnostic fires once at the first training batch and
prints:
- aux_nb_label_buf stats (the kernel input; should be ~0.8 z-norm)
- next_states_buf col 0 sample (what strided_gather reads)
- features_raw_cuda col 0 sample (the source feature buffer)
- targets_raw_cuda col 2 sample (raw_close — the suspected leak)
- Verdict line interpreting the 4 stats
Three numbers will pin the source. If aux_buf ~5500: raw_close leaked
into next_states. If features_raw_cuda col 0 ~5500: the writer didn't
normalize. If targets_raw_cuda col 2 ~5500 but aux_buf ~0.8: only
production reads raw_close, smoke doesn't.
One-shot via static AtomicBool. Pre-graph-capture, no perf impact.
Removal gate: delete once the leak source is identified and fixed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2683d46378