Phase 2A of the HEALTH_DIAG GPU port. Lands the simplest member of the
kernel family (single-block single-thread ISV scalar copies) plus the
orchestrator scaffolding (cubin loader, mapped-pinned snapshot owner,
launcher) that subsequent phases will reuse.
Why simplest first: 5 kernels each with different masking semantics
landing in one commit risks silent numerical mismatches that downstream
parsers (aggregate-multi-seed-metrics.py, smoke summarisers) would
mis-parse. Per feedback_no_quickfixes.md, kernel-by-kernel with
parallel-shadow validation. Phase 4 deletes CPU paths in one commit
once all kernels are bit-identical.
What this kernel does: copies 22 ISV signal-bus slots into the matching
HealthDiagSnapshot fields — reward-component EMAs (slots 63-68), VSN
attention focus EMAs (87, 88), target-drift EMAs (92, 93), aux-head
loss EMAs (113, 114, 117), MoE expert utilisation (118-126), gate
entropy (126), and λ_eff (128). 22 of 147 snapshot words populated
(reward_split[6] + noisy_vsn[2] + noisy_drift[2] + aux[3] + aux_moe[10]).
Producer-only — no consumer reads health_diag.snapshot() yet; Phase 3
wires the call site, Phase 4 makes the snapshot the sole emit source.
Architecture rules upheld:
- No HtoD / DtoH (writes through mapped-pinned device pointer).
- No atomicAdd (single-thread kernel; family-wide rule for 2B-2E too).
- No DtoD memcpy on the diag path (kernel pointer-load + store).
- Kernel WORD-index table inline; static_assert pins WORD_TOTAL=147 in
lockstep with snapshot_size_is_stable test.
- ISV indices passed as kernel args (not #define'd) so the kernel
never embeds the numbering — single source of truth lives in
gpu_dqn_trainer.rs constants.
Changes:
- new crates/ml/src/cuda_pipeline/health_diag_kernel.cu (311 lines).
- new crates/ml/src/cuda_pipeline/gpu_health_diag.rs (190 lines)
holds GpuHealthDiag orchestrator (cubin module + isv_mirror_kernel
handle + MappedHealthDiagSnapshot).
- mod.rs: pub mod gpu_health_diag.
- build.rs: registers health_diag_kernel.cu in kernels_with_common.
- gpu_dqn_trainer.rs:
- imports super::gpu_health_diag::GpuHealthDiag;
- adds health_diag: Option<GpuHealthDiag> field next to aux_heads_fwd;
- constructor calls GpuHealthDiag::new() after the cubin loads;
- adds pub fn launch_health_diag_isv_mirror() (uses compile-time
ISV index constants, single source of truth);
- adds pub fn health_diag_snapshot() accessor.
- docs/health_diag_inventory.md: Phase 2A entry + Phase 4 deletion
list pinned per Invariant 7.
- docs/dqn-wire-up-audit.md: new row for health_diag_kernel.cu under
"CUDA Kernels (.cu files)" — classified Pending wire-up (Phase 3).
Verification:
- cargo check -p ml: 0 new warnings (12 lib warnings pre-existing,
none touch health_diag/gpu_health_diag).
- cargo test -p ml --no-run: clean.
- cargo test -p ml --lib cuda_pipeline::health_diag::tests: 3/3 pass
(snapshot_size_is_stable, default_is_zeroed, alignment_is_4_bytes
— the WORD_TOTAL=147 static_assert in the kernel matches the host
size_of test bit-for-bit).
Scope discipline (per prompt's "if kernel #3 takes too long, stop"):
This commit lands the orchestrator scaffolding + simplest kernel only.
Phases 2B-2E (eval-histogram, q-mag-reduce, per-sample-reduce,
finalise) intentionally deferred — masking semantics for q_mag_reduce
in particular need careful per-CPU-getter inspection before kernel-side
implementation. Producer-only by design; no production caller of
launch_health_diag_isv_mirror in this commit (matches AuxHeadsForwardOps
landing pattern from Plan 4 Task 6 Commit A).
Phase 0 (commit 333ea7184): exhaustive inventory of every numeric value
emitted across the 7 HEALTH_DIAG sites in training_loop.rs and metrics.rs.
Classified each as GPU-already / CPU-bound / Mixed / Host-state /
ISV-already. Aggregate cost: ~50-150 MB/epoch DtoH + ~70 sec/epoch CPU
compute (the original observation that motivated the port). Inventory
lives at docs/health_diag_inventory.md.
Phase 1 (commit 60fd7de96): foundation only — no kernel, no producer
launch, no CPU code deletion yet.
* HealthDiagSnapshot #[repr(C)] POD with 147 fields (all f32 or u32)
covering every numeric value in the HEALTH_DIAG line family. Field
order matches the existing log format strings (downstream parsers
like aggregate-multi-seed-metrics.py depend on this).
* Controller fire-bools promoted from u8 to u32 to avoid #[repr(C)]
padding mismatch when followed by f32 fields. ~24-byte cost.
* MappedHealthDiagSnapshot wrapper around cuMemHostAlloc(DEVICEMAP|
PORTABLE) + cuMemHostGetDevicePointer_v2, mirroring the
MappedF32Buffer pattern.
* Default impl via MaybeUninit::zeroed() (allocator-free, valid
bit-pattern).
* 3 unit tests pinning size (588 bytes), alignment (4 bytes),
zeroed-default — all passing on local CPU host.
Phases 2-5 deferred (kernel family + wiring + CPU emit rewrite + old
code deletion). The 5 kernels each read different device buffers with
different masking semantics — getting any one wrong would silently
produce numerically-different HEALTH_DIAG output that downstream
parsers would mis-parse. Per the prompt's 'don't wedge yourself'
guidance and feedback_no_quickfixes.md, the safer move was to land
the typed wrapper cleanly so subsequent commits can iterate kernel
by kernel against the actual buffer-by-buffer reduction.
nsys profile of multi_fold_convergence on L40S identified compute_expected_q
as the #3 GPU consumer at 12.9% (207ms / 1382 calls — ~150 µs per call,
compute-bound at typical batch=8192 × num_atoms=51). The kernel computed each
per-action softmax over atoms in 3 passes (max → sum_exp → normalize+expected
+var+entropy+util), re-reading v_row[z]+adv_a[z] 3 times per atom for each of
the 4 × ~3.25 = 13 actions per sample.
Replaced with online softmax (running-max + running-sum) so expected_q,
sum_z_sq, and entropy all accumulate in a single forward pass over atoms.
Atom utilisation still needs a 2nd pass because it depends on the final
S = sum exp(logit-M) being known to compute prob_z = exp(logit_z-M)/S for
the threshold test.
Inner loop reduction: 3 → 2 passes, ~33% fewer global loads of branch
advantage logits (which dominate because v_row[z] is reused across 13
(action, branch) pairs and may stay in L1, while adv_a[z] flips per
action and is pure global).
Online accumulation pattern (Page-Olshen):
M, S, TZ, TZ², TLM = -inf, 0, 0, 0, 0
for z in 0..num_atoms:
logit = v_row[z] + adv_a[z]
if logit > M:
scale = exp(M - logit) # ≤ 1, no overflow
S, TZ, TZ², TLM *= scale
M = logit
w = exp(logit - M)
S += w
TZ += w * z_val
TZ² += w * z_val²
TLM += w * (logit - M)
expected_q = TZ / S
sum_z_sq = TZ² / S
entropy = log(S) - TLM / S # analytical: -sum(p log p)
Correctness: bit-stable when atoms processed in fixed order
(z = 0..num_atoms-1). The analytical entropy form is mathematically
identical to -sum(p log p):
-sum(p log p) = log(S) - (1/S) * sum(exp(logit-M) * (logit-M))
= log(S) - TLM / S
The previous code's `if (prob > 1e-10f)` underflow guard is no longer needed:
exp(logit - M) underflows cleanly to 0 when logit << M, and the analytical
form does not multiply tiny probs by very negative logs. Bit-stable per
feedback_stop_on_anomaly.md — no fuzzy tolerance change.
Atom-stat block-sum reduction unchanged (warp shfl + shared-mem bank,
deterministic).
Expected wall-clock saving: ~33% of 207ms = ~70ms across the smoke run; on
production batch=8192 × 1382 calls per fold, roughly proportional. Lower-bound
estimate — at low batch (smoke) the kernel may be launch-bound rather than
load-bound. nsys re-profile after deployment will quantify.
ABI unchanged; no Rust caller changes required. Per Invariant 7 the audit
doc dqn-gpu-hot-path-audit.md is updated with Fix 19 entry.
Build: SQLX_OFFLINE=true cargo check -p ml --lib clean (12 warnings, baseline).
Tests: cargo test -p ml --lib --no-run clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of the HEALTH_DIAG GPU port (Phase 0 inventory landed in
333ea7184). Lands the typed snapshot struct and the mapped-pinned
allocation that subsequent phases write into and read from — no kernel
and no caller wiring yet, deliberately so the type can stabilise before
Phase 2 invests in CUDA shmem reductions against its byte layout.
`HealthDiagSnapshot` (#[repr(C)] POD, 147×4 = 588 bytes) holds every
numeric field emitted across the 7 HEALTH_DIAG log sites. All fields
are `f32` or `u32` so CPU and GPU agree byte-for-byte with no padding
to reason about. Controller fire booleans (`fire_lr`/`fire_tau`/etc.)
are promoted from u8 → u32 per the Phase 0 design review's open-
question #3 — a `[u8; 6]` followed by `f32` would risk implementation-
defined padding mismatch between Rust's #[repr(C)] and CUDA's struct
layout rules; the +24 bytes of cost is acceptable.
`MappedHealthDiagSnapshot` wraps `cuMemHostAlloc(DEVICEMAP|PORTABLE)`
of `sizeof(HealthDiagSnapshot)` plus `cuMemHostGetDevicePointer_v2` —
mirrors the existing `MappedF32Buffer` allocator pattern but typed.
The kernel chain in Phase 2 will write through `dev_ptr()`; the host
emit code in Phase 4 reads through `host_ref()`. No `memcpy_dtoh`,
no `Vec` allocator on the path — consistent with
`feedback_no_htod_htoh_only_mapped_pinned.md`.
Three unit tests pin the layout (`snapshot_size_is_stable`,
`alignment_is_4_bytes`, `default_is_zeroed`) so any future field
add/reorder requires an explicit test update — guards the kernel-side
offset table from silent drift between commits. The size assertion
records the field count breakdown line by line in a comment so the
next maintainer can audit the math without re-deriving it.
Touched: cuda_pipeline/health_diag.rs (+339 LOC new),
cuda_pipeline/mod.rs (+8 re-exports). cargo check clean at 12 warnings
(workspace baseline). Three new unit tests pass on local CPU host (no
GPU required). Audit doc note added per Invariant 7 — explicitly notes
producer kernel + caller migrations land in Phase 2 / 4 commits.
No fingerprint change — separate mapped-pinned alloc, not part of the
param-tensor layout the fingerprint guards.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
nsys profile of multi_fold_convergence on L40S identified backtest_state_gather as
the #1 GPU consumer at 37.2% (596 ms / 60,588 calls — kernel launch latency
dominated compute) and the per-step TLOB cuBLAS gemvx calls as #2 at 25%
(242K calls). Both share the same per-step amplification: chunk_len=512 separate
gather launches + 512 separate TLOB.forward calls (4 SGEMMs each) per chunk
before any Q-values can be computed.
This commit replaces the per-step gather + DtoD pattern with a single batched
launch, and reuses the same chunked buffer for a single chunk-wide TLOB forward.
Per-chunk launch reduction: from 2*chunk_len + chunk_len*7 to 1 + 7 for the
gather+TLOB phase (4608 -> 8 with chunk_len=512, a 576x reduction).
New kernel `backtest_state_gather_chunk` (experience_kernels.cu):
- Writes [chunk_len, N, padded_sd] directly into chunked_states_buf
- chunk_len * N threads, 1 thread per output row
- Mathematically identical to per-step gather: portfolio_buf and plan_isv_buf
are CONSTANT within a chunk (env_step + plan_state_isv update at chunk
boundary only). Each thread reads independent feature offsets, no atomics,
no reordering.
Chunked val TLOB (gpu_backtest_evaluator.rs + metrics.rs):
- Val TLOB instance now sized to DQN_BACKTEST_CHUNK_SIZE * n_windows via new
GpuBacktestEvaluator::val_tlob_batch_size() helper.
- submit_dqn_step_loop_cublas calls tlob.forward(chunked_states, batch) ONCE
per chunk on the chunk-wide buffer instead of chunk_len times on states_buf.
- Partial last chunks reuse the same buffers (forward(b) accepts any
b <= construction_batch).
Borrow restructure:
- Removed top-of-function `let ch_states = self.chunked_states_buf.as_ref()?`
binding (TLOB needs &mut). Replaced with per-chunk ch_states_base raw u64
device pointer extracted in tight scope, reused by Phase 1 (gather) and
Phase 2+3 (compute_q_values_to + last_step_states_ptr). The pointer is
stable across the chunk because the Option<CudaSlice<f32>> does not
reallocate.
Per-step gather kernel `backtest_state_gather` retained unchanged for
evaluate() / evaluate_ppo() / evaluate_supervised() paths that still need
a single-step writer (closure-based callers with no chunked buffer).
Audit doc dqn-gpu-hot-path-audit.md updated with Fix 18 entry per Invariant 7.
Build: SQLX_OFFLINE=true cargo check -p ml --lib clean (12 warnings, baseline).
Tests: cargo test -p ml --lib --no-run clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Catalogue every numeric field in the HEALTH_DIAG family of log lines
(7 emit sites across training_loop.rs + metrics.rs) and classify each
source as GPU-already / CPU-bound / Mixed / Host-state / ISV-already.
Identifies the dominant cost contributors driving the observed
~70 s/epoch HEALTH_DIAG overhead on L40S after the eval async-split
(commit f815f7239) — they are all CPU-side reductions over multi-million
element per-sample buffers fetched via memcpy_dtoh:
- update_q_mag_means_cached (~786 KB DtoH + B×total_actions loop)
- var_scale_epoch_mean (N-element DtoH + conditional avg)
- trail_fire_and_hold_per_mag (4× N-element DtoH + per-mag loop)
- per_magnitude_winrate_and_variance (4× DtoH + sumsq loop)
- reward_contrib_fractions (6× large-buffer DtoH + 6 passes)
- read_eval_action_distribution_* (4 sites; CPU iter on pinned)
- branch_noisy_sigma_mean (NoisyNet param DtoH per branch)
Aggregate per-emit DtoH ≈ 50-150 MB depending on alloc_episodes ×
alloc_timesteps; fully consistent with the 70 s/epoch observation.
All inputs are already device-resident, so the migration is "stop
reducing on the host" plus a small kernel family writing into a single
mapped-pinned HealthDiagSnapshot struct.
Phase 0 deliverable per the dispatching prompt; subsequent commits
(struct + mapped wrapper, kernel family, CPU emit rewrite, audit-doc
updates) follow the phased plan documented at the bottom of the file
and gate-review on this inventory before touching code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per feedback_mbp10_mandatory.md, MBP-10 + trades are mandatory inputs.
multi_fold_convergence now reads FOXHUNT_MBP10_DATA / FOXHUNT_TRADES_DATA
env vars, forwards them to train_baseline_rl as --mbp10-data-dir /
--trades-data-dir, and hard-fails if either path is missing.
Argo sanitizer-test + nsys-test templates set the env vars to PVC paths
/data/test-data/{mbp10,trades} and FOXHUNT_TEST_DATA points at
/data/test-data/ohlcv (PVC layout has lost+found/trades/mbp10/ohlcv as
top-level subdirs; the symbol ES.FUT lives under ohlcv/).
Without this, the smoke would silently fall back to tick-rule proxy
classification and validate a degraded model variant (no real OFI),
defeating the purpose of L40S validation. Audit doc updated per
Invariant 7.
After Fix 1..16 migrated all 80+ production callers off
`super::htod_f32` and `super::clone_htod_f32`, the helper bodies in
`cuda_pipeline/mod.rs:129-145` had zero non-test consumers. Deleted
both function definitions per `feedback_no_legacy_aliases.md` (no
deprecated wrappers).
Per `feedback_no_partial_refactor.md` (when a shared contract is
deleted, every consumer migrates together — including tests), the
two surviving test-block callers in `gpu_tlob.rs::tests` (lines
1017 and 1132) are migrated to `mapped_pinned::upload_f32_via_pinned`
in the same commit. The other test-only callers in
`signal_adapter.rs::tests`, `gpu_action_selector.rs::tests`, and
`cuda_pipeline/mod.rs::tests` use bare `stream.memcpy_htod` /
`stream.memcpy_stod` against the cudarc handle directly (not the
deleted helpers) — no change needed.
A docstring was added at the deletion site recording when and why
the helpers were removed, pointing future readers at the canonical
replacements `mapped_pinned::clone_to_device_f32_via_pinned` and
`mapped_pinned::upload_f32_via_pinned`.
Final state of the HtoD migration sequence:
- production callers of `stream.memcpy_htod` / `memcpy_stod`: 0
- production callers of `htod_f32` / `clone_htod_f32`: 0
- helper definitions: removed from `mod.rs`
docs/dqn-gpu-hot-path-audit.md updated with Fix 17 entry.
cargo check -p ml --lib clean at 12 warnings.
cargo check -p ml --tests clean at 23 warnings (12 lib duplicates +
11 test-specific, baseline unchanged).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two `crate::cuda_pipeline::clone_htod_f32` callsites in
`training_loop.rs::set_raw_market_data` (lines 1250 and 1254) are
rewritten to `mapped_pinned::clone_to_device_f32_via_pinned` per
`feedback_no_htod_htoh_only_mapped_pinned.md`.
These run once per fold immediately before the step loop begins; the
`_raw` suffix indicates the un-normalized arrays kept on-GPU for the
eval-time critic.
Error-type continuity: the helper already returns `Result<_, String>`,
which feeds `anyhow::anyhow!("…raw upload: {e}")` directly — the prior
.map_err wrapper is preserved verbatim. 1:1 path swap.
docs/dqn-gpu-hot-path-audit.md updated with Fix 16 entry.
cargo check -p ml --lib clean at 12 warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The remaining `super::clone_htod_f32` call in `upload_ofi_features`
(`:4258`) is rewritten to `mapped_pinned::clone_to_device_f32_via_pinned`
per `feedback_no_htod_htoh_only_mapped_pinned.md`. Called from
training_loop's data-load phase before each fold's training run, so
the OFI tensor (4M bars × 32 dims) now stages through mapped-pinned
+ DtoD just like all other large data uploads.
Error-type shift: `mapped_pinned::clone_to_device_f32_via_pinned`
returns `Result<CudaSlice<f32>, String>` whereas `clone_htod_f32`
returned `Result<CudaSlice<f32>, MLError>`. Wrapped with
`.map_err(|e| MLError::ModelError(format!("upload_ofi_features: {e}")))`
to preserve the call-site label in error messages.
docs/dqn-gpu-hot-path-audit.md updated with Fix 15 entry.
cargo check -p ml --lib clean at 12 warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Residual COLD ctor sites missed by the original audit are now migrated
off explicit HtoD per `feedback_no_htod_htoh_only_mapped_pinned.md`:
- `:9922` sel_clip_buf 1-element init (sigmoid head clip-norm seed)
- `:10075-10078` spectral norm init_u_s1/v_s1/u_s2/v_s2 (4 calls)
- `:10095-10096` `alloc_spec_pair!` macro body — both u/v init uploads
inside the macro now go through `upload_f32_via_pinned`; the macro's
`$lbl_u`/`$lbl_v` are reused in the error-message format so per-pair
failures stay diagnostically distinct
- `:11276` graph_params_host (60 floats: cross-branch graph message
passing weights)
- `:11330` denoise_params_host (1800 floats: 2-step diffusion Q-refinement)
- `:11476` qlstm_weights_host (528 floats: QLSTM Xavier init)
All sites use `mapped_pinned::upload_f32_via_pinned` (the canonical
mapped-pinned + DtoD staging helper). The helper returns
`Result<_, String>` whereas this constructor returns
`Result<_, MLError>`, so each site wraps the error via
`.map_err(|e| MLError::ModelError(format!("<site> upload via pinned: {e}")))`.
Site labels preserved so backtraces remain readable.
docs/dqn-gpu-hot-path-audit.md updated with Fix 14 entry.
cargo check -p ml --lib clean at 12 warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Agent 1 — bulk COLD/WARM migration across init/loader files.
Sites: 39 production HtoD calls migrated to MappedF32Buffer / MappedI32Buffer /
new MappedU32Buffer. Cross-file infrastructure includes new staging helpers
(clone_to_device_{f32,i32}_via_pinned, upload_{f32,i32,u32}_via_pinned),
MappedU32Buffer type, and host_slice_mut accessor on MappedF32/U32Buffer.
Per feedback_no_htod_htoh_only_mapped_pinned.md — first of 3 parallel batches.
# Conflicts:
# docs/dqn-gpu-hot-path-audit.md
HOT sites:
- L2992 hindsight bar_indices: was `alloc + memcpy_htod(total i32s)` on
every collect_experiences_gpu when hindsight_fraction > 0. Added
persistent `bar_indices_pinned: MappedI32Buffer` (capacity =
alloc_episodes * alloc_timesteps * 2) allocated at constructor.
Per-call host writes go through host_ptr; relabel kernel reads via
dev_ptr u64.
- L3186 feature_mask: was alloc + memcpy_htod each epoch at t==0.
Promoted `feature_mask_buf` to `Option<MappedF32Buffer>`. Reallocates
only when mask size changes; CPU writes via host_ptr, state_gather
reads via dev_ptr.
- L3816 update_per_sample_support: per_sample_support_buf is read-only
by the kernel — promoted to MappedF32Buffer (kernel args now
dev_ptr u64). per-epoch tile fill becomes a direct host_ptr write.
WARM sites:
- L3097 episode_starts upload: episode_starts_buf is GPU-mutated by
domain_rand_episode_starts so must remain CudaSlice. Stage via
MappedI32Buffer + memcpy_dtod_async through new helper
`upload_host_to_cuda_i32_via_pinned`.
- L2009 upload_expert_actions: promoted expert_actions_gpu from
Option<CudaSlice<i32>> to Option<MappedI32Buffer>. Direct host_ptr
write replaces alloc + memcpy_htod.
COLD sites:
- L1076,3848 portfolio_states init/reset (GPU-mutated): use
upload_host_to_cuda_f32_via_pinned (mapped-pinned staging + DtoD).
- L1093 epoch_state init (GPU-mutated): replace clone_htod_f32 with
explicit alloc_zeros + upload_host_to_cuda_f32_via_pinned.
- L1341 saboteur_base init (GPU-mutated): use the same helper.
- L2784 trade_stats_buf pre-reduction zero: replaced htod_f32 of an
all-zeros vec with stream.memset_zeros — fully GPU-side, no PCIe.
Imports: added DevicePtrMut for memcpy_dtod_async pointer extraction.
Per `feedback_no_partial_refactor.md`, every consumer of the migrated
buffers is updated in this commit:
- per_sample_support_buf kernel args at L3426 (compute_expected_q) and
L3546 (quantile_q_select) now consume dev_ptr u64.
- feature_mask_buf accessor at L3195 reads via .dev_ptr.
- expert_actions_gpu field type change is contained (no external
consumers in this crate).
cargo check -p ml --lib: 11 warnings (unchanged from prior commit on
this branch; 2 below 13-warning baseline because two unrelated trivial
warnings disappeared as a side-effect of the refactor).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
Migrates 3 WARM HtoD sites:
- `fused_training.rs:505` HER source_indices Vec<i32> upload
- `fused_training.rs:511` HER reward_ones Vec<f32> upload
- `training_loop.rs:470` curriculum episode_starts Vec<i32> upload
For `fused_training.rs` adds module-level `staging_upload_{i32,f32}`
helpers. For `training_loop.rs` uses an inline staging+DtoD block
because the destination is reached via the out-of-scope accessor
`collector.episode_starts_buf_mut() -> &mut CudaSlice<i32>`.
All destination buffer types remain `CudaSlice<T>` so no downstream
consumer (incl. `gpu_her::relabel_batch_with_strategy` and any
`episode_starts_buf` reader) needs to change.
Note: if Agent 2's `gpu_experience_collector` merge changes
`episode_starts_buf_mut`'s return type to a mapped pinned buffer,
the inline staging block in `training_loop.rs:470` can be simplified
to a direct `write_from_slice` (follow-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
Both `upload_params` and `upload_target_params` previously did
`stream.memcpy_htod` of TOTAL_PARAMS f32 (~MB) at fold boundaries /
external weight loads. Now route through the helper
`update_via_mapped_f32` introduced in the previous commit — stages
CPU bytes through a transient mapped pinned buffer and DtoD-copies
into the existing `params_buf` / `target_params_buf` `CudaSlice<f32>`.
No public API change. The destination buffer types remain
`CudaSlice<f32>` so every downstream kernel-arg consumer is untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cold-path constructor sites (2):
- portfolio_states init: htod_f32 → upload_f32_via_pinned
- rng_states init: memcpy_htod → upload_u32_via_pinned
Warm per-collect sites (2):
- episode_starts_buf: memcpy_htod → upload_i32_via_pinned
- barrier_config: htod_f32 → upload_f32_via_pinned
Warm per-epoch reset (2 sites + persistent staging refactor):
The Vec<f32>/Vec<u32> host staging fields (portfolio_init_staging,
rng_seed_staging) replaced with persistent MappedF32Buffer /
MappedU32Buffer allocated once at construction. Resets fill via
host_slice_mut() (zero-allocation, zero-HtoD); a single async DtoD
seeds the persistent device-resident portfolio_states / rng_states
(which the kernel mutates each step and must remain in VRAM).
Adds host_slice_mut() accessor on MappedF32Buffer and MappedU32Buffer
to support structured per-element writes against the mapped pages.
Audit row appended (Fix 11) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
Adds module-level `upload_via_mapped_{f32,i32,u32,u64}` helpers and an
in-place `update_via_mapped_f32`. Each stages CPU data through a
transient mapped pinned buffer and DtoD-copies into the destination
`CudaSlice<T>`, then stream-syncs so the staging buffer is safe to
drop. Destination buffer types remain `CudaSlice<T>` so every existing
consumer (raw_ptr / kernel arg) is untouched, satisfying
`feedback_no_partial_refactor.md` for these one-shot init paths.
Migrated COLD sites in `GpuDqnTrainer::new`:
- weight_decay_mask (TOTAL_PARAMS f32)
- branch_slice_starts_dev, branch_slice_lens_dev ([i32; 4])
- branch_grad_scales_dev ([f32; 4])
- per_branch_gamma_base/max_dev ([f32; 4] each)
- q_quantile_branch_offsets/sizes_dev ([i32; 4] each)
- spectral_norm_descriptors_dev ([u64; 78])
- stochastic_depth_scale_buf ([f32; 3])
- stochastic_depth_rng_state ([u32; 1])
- vsn_group_begins_buf, vsn_group_ends_buf ([i32; num_groups] each)
- mamba2_params Xavier init (mamba2_param_count f32)
11 COLD HtoD sites eliminated. cargo check clean (15 warnings; +2 over
baseline 13 are the new MappedU32/U64 struct visibility warnings,
matching the existing MappedF32/I32 pattern).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 1 constructor site (rng_states u32 init) rewritten to
mapped_pinned::upload_u32_via_pinned.
- 2 warm-path sites (source_indices, donor_indices in relabel_batch)
rewritten to mapped_pinned::upload_i32_via_pinned. The warm sites
re-allocate pinned staging per relabel call — acceptable on this
cold path (HER relabel runs once per epoch).
Adds MappedU32Buffer type to mapped_pinned.rs mirroring MappedI32Buffer,
plus upload_u32_via_pinned helper. Manual Debug impl so warning count
stays at 13.
Audit row appended (Fix 10) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HOT site: `train_step_gpu` was uploading the 4-byte Adam step counter via
`stream.memcpy_htod(&[step], &mut scratch.adam_t)` on every backward pass.
Promoted `DtScratch.adam_t` from `CudaSlice<i32>[1]` to `MappedI32Buffer(1)`
allocated once in `alloc_dt_scratch`. Per-step CPU writes go through
`host_ptr`; the Adam kernel reads via `dev_ptr` u64. Trivial fix, massive
frequency — runs once per gradient step.
Other sites:
- L478 ctor params init: `params` is mutated by Adam (must stay CudaSlice).
Upload via mapped-pinned staging + `memcpy_dtod_async` through new
helper `upload_host_to_cuda_f32`.
- L501 ctor wd_mask init: wd_mask is read-only by Adam. Promoted to
`MappedF32Buffer`; CPU writes 1.0s through host_ptr, kernel reads dev_ptr.
Caller switched from `wd_mask.raw_ptr()` to `wd_mask.dev_ptr`.
- L1105 trajectory build episode_starts: one-time, replaced `clone_htod`
with `MappedI32Buffer` (direct host write, kernel read via dev_ptr).
- L1147 trajectory build ep_rewards: one-time, replaced `htod_f32` with
`MappedF32Buffer` (direct host write, return_to_go reads via dev_ptr).
Also added `#[allow(missing_debug_implementations)]` to MappedI32Buffer
and MappedF32Buffer in `mapped_pinned.rs` — raw pointers + CUdeviceptr
have no useful Debug impl, and the workspace `missing_debug_implementations`
lint now reaches them through the public DT type that holds them.
Audit doc: docs/dqn-gpu-hot-path-audit.md updated.
cargo check -p ml --lib: 11 warnings (down from 13 baseline; net-zero
regression, 2 minor unrelated warnings vanished as a side-effect).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 4 stream.clone_htod sites in IQN constructor: cos_features precompute,
online_taus + target_taus tile broadcast, init_iqn_xavier_weights
upload — all rewritten to mapped_pinned::clone_to_device_f32_via_pinned.
- clone_cuda_slice helper rewritten to a single DtoD copy via the
existing super::clone_cuda_slice_f32 helper. Previously did
device→host→device, double-violating both no-DtoH and no-HtoD rules.
Sole callsite is constructor seeding of target params from online.
Audit row appended (Fix 9) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
`GpuActionSelector::new` previously did one HtoD memcpy to upload
RNG seeds. Now allocates `MappedU32Buffer`, writes seeds via
host_ptr, and passes `dev_ptr` (CUdeviceptr) to the three kernels
that consume rng_states (epsilon_greedy, epsilon_greedy_routed,
branching_action_select).
Adds `MappedU32Buffer` and `MappedU64Buffer` to `mapped_pinned.rs`
mirroring the existing `MappedF32Buffer`/`MappedI32Buffer` API
(new/write_from_slice/read_all/Drop). The U64 variant is staged
for the upcoming spectral-norm host_desc[78] descriptor table
migration in `gpu_dqn_trainer::new`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- trainers/ppo.rs: 2 sites (features, targets in set_raw_market_data)
- hyperopt/adapters/ppo.rs: 3 sites (features, targets in upload path;
action_indices in run_gpu_backtest forward closure)
- hyperopt/adapters/mamba2.rs: 1 site (gpu_to_stream)
All clone_htod_f32 and bare stream.clone_htod calls rewritten to
mapped_pinned::clone_to_device_{f32,i32}_via_pinned.
Audit row appended (Fix 7) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HOT site at simulate_batch_gpu — `actions: &[i32]` was uploaded
per-batch via stream.memcpy_htod into a persistent CudaSlice. Promoted
the field to `MappedI32Buffer` allocated once at constructor; per-call
host writes now go through `host_ptr` (no HtoD copy). Kernel arg switched
to passing `dev_ptr` as a u64.
The two COLD sites (ctor init_state, reset init_state) writing into the
GPU-mutated `portfolio_state_buf` cannot replace the destination buffer
type (env_step kernel writes back to it). Use a mapped-pinned staging
buffer + memcpy_dtod_async via a new `upload_portfolio_state` helper.
Retains zero direct HtoD copies on the only path the violation rule
permits per `feedback_no_htod_htoh_only_mapped_pinned.md`.
Sites migrated:
- L122 ctor init_state (htod_f32 -> staging+DtoD via upload_portfolio_state)
- L174 simulate_batch_gpu actions upload (HOT: persistent MappedI32Buffer)
- L296 reset init_state (htod_f32 -> staging+DtoD)
Audit doc: docs/dqn-gpu-hot-path-audit.md updated with new MIGRATED entries.
cargo check -p ml --lib: 13 warnings, baseline preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates the 3 clone_htod_f32 calls in walk-forward GPU data upload
(features, targets, OFI) to mapped_pinned::clone_to_device_f32_via_pinned.
Audit row appended (Fix 6) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, mapped pinned
(cuMemHostAlloc DEVICEMAP) is the only allowed CPU↔GPU path.
`set_count_bonuses` is called per action-selection — converting the
three bonus buffers from CudaSlice<f32> (HtoD memcpy) to
MappedF32Buffer eliminates 3-6 HtoD memcpys per call.
Before: 3 reused-buffer htod_f32 (lines 92/96/100) + 3 first-call
clone_htod_f32 (lines 93/97/101) on every call = 3 HtoD steady-state.
After: zero HtoD — direct host_ptr writes via write_from_slice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates the 7 in-file callers of clone_htod_f32 in DqnGpuData/PpoGpuData
constructors and helpers to mapped_pinned::clone_to_device_f32_via_pinned.
Sites: features+targets+ofi upload (DqnGpuData::upload_slices,
::upload_ofi), portfolio scratch (build_batch_states), htod_copy_into,
PpoGpuData::upload.
Helper bodies htod_f32 / clone_htod_f32 left intact at lines 129-145 —
other parallel agents are still migrating files that call them.
Audit row appended (Fix 5) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds clone_to_device_{f32,i32}_via_pinned and upload_{f32,i32}_via_pinned
in `mapped_pinned.rs`. They allocate a temporary MappedXxxBuffer
(cuMemHostAlloc DEVICEMAP), write the payload via host_ptr, then async
DtoD into a target CudaSlice<T>. Used to migrate cold-path uploads off
explicit cudaMemcpy HtoD per `feedback_no_htod_htoh_only_mapped_pinned.md`
without changing struct field types where the kernel mutates the buffer
every step (e.g. portfolio_buf, weight buffers consumed by Adam).
Audit table updated in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- 660f02ff4 split compute_validation_loss into launch_validation_loss + consume_validation_loss; mirror split on GpuBacktestEvaluator (launch_metrics_and_record_event + consume_metrics_after_event); evaluate_dqn_graphed_async pipelined path; mapped-pinned mirrors for actions_history/intent_mag/picked_action; lazy eval_done_event re-recorded each epoch; post-loop drain so smoke tests see fresh data
The async-diag merge (3c0d26292, building on d9cb14f1b + 673b04a8d) wired the
GPU side of cross-stream eval pipelining correctly (training stream `cuda_stream`,
eval stream `validation_stream`, `cuStreamWaitEvent` barrier on `train_done_event`)
but left a host-side `stream.synchronize()` inside
`gpu_backtest_evaluator::launch_metrics_and_download` that blocked the CPU thread
for the FULL eval drain (~25-30s/epoch on L40S). The host thread is the same one
that submits the next epoch's training kernels via `run_full_step` — so until eval
drained, training submission was gated on it, defeating the dedicated
`validation_stream`.
Fix: split metrics readback into record/consume halves at the buffer-readback layer:
* `launch_metrics_and_record_event` — submits the metrics kernel + DtoD-async
copies of `actions_history_buf` / `intent_mag_buf` / `picked_action_history_buf`
into mapped-pinned mirrors, then records `eval_done_event`. Returns immediately.
* `consume_metrics_after_event` — `event.synchronize()` (the SOLE host wait per
epoch boundary), then `read_volatile`s mapped-pinned `metrics_buf`. After the
event syncs, the four action-distribution helpers read directly from the
now-coherent mapped-pinned mirrors — eliminating four synchronous DtoH copies
that violated `feedback_no_htod_htoh_only_mapped_pinned.md`.
Caller migration (per `feedback_no_partial_refactor.md`):
* `evaluate_dqn_graphed` (existing public API) becomes a thin sync wrapper:
launch_async → consume. ABI unchanged.
* New `evaluate_dqn_graphed_async` for the pipelined path.
* `evaluate` / `evaluate_ppo` / `evaluate_supervised` migrated inline to
launch+consume (no caller pipelines them).
* `compute_validation_loss` replaced by `launch_validation_loss` +
`consume_validation_loss`. Training loop now consumes pending at epoch start
(cached_async_val_loss = epoch N-1's val_sharpe) and launches at epoch end
(sentinel-only return). Final post-loop consume drains the LAST epoch's launch
so smoke tests / hyperopt that read `last_eval_direction_dist` see the most
recent epoch's data.
Mathematical identity preserved: every val_sharpe consumed is bit-identical to
what the prior synchronous flow would have produced for the same epoch — only
the timing of the host parse differs (one epoch later, matching the existing
`cached_async_val_loss` lag semantics; HEALTH_DIAG `val [...]` / `val_dir_dist` /
`val_picked_dir_dist` emit moves with the consume).
Audit: docs/dqn-gpu-hot-path-audit.md Fix 4. Out of scope: periodic chunk-level
`stream.synchronize` calls in `submit_dqn_step_loop_cublas` (lines ~1596 + ~1863)
for kernel-error detection — host-blocking but smaller in aggregate; future work.
Workspace baseline preserved: 13 warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the proposed static `include_str!` regression guard for the
fold-boundary IQN target hard-sync (issue #84, root-cause fix in commit
`7c19b5903`) with a real GPU runtime test that exercises the contract
end-to-end. The static guard only caught literal deletion of the call
line — a stub body returning `Ok(())`, a copy against the wrong buffer,
the wrong copy direction, or a queue against the wrong stream all pass
the textual assertion silently.
Test
(`cuda_pipeline::gpu_iqn_head::tests::iqn_sync_target_from_online_makes_target_equal_online`):
1. Construct a `GpuIqnHead` with default `GpuIqnConfig` on the
default CUDA stream.
2. Fill `online_params` ← 0.42 and `target_params` ← 0.99 via a
single mapped-pinned staging buffer + `cuMemcpyDtoDAsync`. No
HtoD copy is issued; the host write through `MappedF32Buffer::host_ptr`
reaches the GPU through the device-mapped alias and the DtoD
copies the staged values into each parameter buffer. The
witnesses 0.42 / 0.99 are arbitrary distinct fp32 constants —
the contract asserted is buffer equality, independent of magnitude.
3. Sanity: read both buffers back via fresh mapped-pinned destinations
+ DtoD, assert they differ pointwise.
4. Call `iqn.sync_target_from_online()`.
5. `stream.synchronize()` so the queued DtoD has retired.
6. Read both buffers back and assert bit-for-bit equality across all
`total_params` slots using `f32::to_bits` (so any future NaN-bearing
implementation also fails loud).
Per `feedback_no_htod_htoh_only_mapped_pinned.md`, all CPU↔GPU
communication routes through `cuMemHostAlloc(DEVICEMAP|PORTABLE)`
mapped pinned memory. Tests are not exempt — fills and read-backs
both use `MappedF32Buffer` + `cuMemcpyDtoDAsync`.
Buffer access exposed via four new `#[cfg(test)] pub(crate)` accessors
on `GpuIqnHead` (`online_params_slice`, `target_params_slice`,
`total_params_for_test`, `stream_for_test`) so the public API is not
widened.
Test carries `#[ignore = "gpu"]` matching the smoke-test convention
already used in `regression_detection.rs`. `cargo test -p ml --lib`
on a CPU-only host (the worktree environment) skips it cleanly; the
L40S smoke validation pool runs it via `--ignored`.
Paired with a strengthened doc-block at the call site in
`fused_training.rs::reset_for_fold` (boxed `DO NOT DELETE` warning +
reference to the new test name and issue #84) so anyone touching the
line sees the regression context inline before deleting.
Touched: `gpu_iqn_head.rs` (4 cfg(test) accessors + tests mod with
helpers + the runtime test, +217 LOC), `fused_training.rs` (boxed
comment + test reference, +16 LOC, no behaviour change),
`docs/dqn-wire-up-audit.md` (audit entry replacing the
static-test entry from the previous proposal, +33 LOC).
Verified:
* `cargo check -p ml --lib` — clean at 13 warnings (workspace
baseline).
* `cargo test -p ml --lib --no-run` — clean at 24 warnings (test
profile baseline).
* `cargo test -p ml --lib state_reset_registry` — 3/3 existing
tests pass (no 4th static-source test added).
* `cargo test -p ml --lib gpu_iqn_head` — 1 test discovered,
correctly reports `ignored, gpu` on this CPU-only worktree.
Local run not attempted — worktree environment lacks a GPU. The test
runs as part of L40S smoke validation via `--ignored`.
No fingerprint change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous absolute floor (`|q_mean| > 1.5`) in the Q-drift kill
criterion was a tuned constant in violation of
`feedback_adaptive_not_tuned.md` and
`feedback_isv_for_adaptive_bounds.md`. Replace with an ISV-driven
adaptive threshold:
kill_floor = max(0.5, 3.0 × max(ISV[Q_ABS_REF_INDEX=16],
ISV[Q_DIR_ABS_REF_INDEX=21]))
Both ISV slots are per-branch EMAs of `max(|Q_mean|)` already
maintained on-GPU by `q_stats_kernel.cu` and consumed by
`c51_loss_kernel`/`c51_grad_kernel`. The kill floor now anchors on
the same recently-observed healthy Q scale that the loss kernels
already use to normalise their collapse-fraction signals.
The 3.0× multiplier is architectural ("RL Q-divergence shows up at
2-4× healthy scale"); the 0.5 cold-start floor is an Invariant-1
numerical-stability bound active only while both ISV slots are still
≤ 1e-6 in fold-0 epoch-1, then dominated by the runtime formula. The
2× ratio gate is unchanged — already an architectural rate-of-change
bound.
ISV reads use the existing pinned/device-mapped path
(`fused.trainer().read_isv_signal_at`) — no HtoD/DtoH per
`feedback_no_htod_htoh_only_mapped_pinned.md`.
The bug-signature trajectory that motivated the original 1.5 floor
(F1 ep2 Q=+2.23 from F1 ep1 Q=+0.82) still trips: ISV[16] would have
been ~0.6 with α=0.05 EMA tracking, giving kill_floor ≈ 1.8, and
Q=+2.23 > 1.8 with ratio 2.7× trips both gates.
Touched: `training_loop.rs` (+70 LOC, two new use-list imports),
`docs/dqn-wire-up-audit.md` (Invariant 7 audit entry).
cargo check clean at 13 warnings (workspace baseline). No
fingerprint change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The IQN readiness gauge on `GpuDqnTrainer` is a streaming improvement
fraction `(iqn_loss_initial - iqn_loss_ema) / iqn_loss_initial`
(gpu_dqn_trainer.rs:5805-5818). `iqn_loss_initial` is captured once on
the first `update_iqn_readiness` call where it is still ~0 and never
resets — across folds it stays pinned to the fold-0 epoch-1 IQN loss.
The pinned device-mapped readiness slot is read by `c51_loss_kernel`
as the CVaR α and by IQN gradient-weight gating, so a stale anchor
either pins readiness near 1 (over-confident, full IQN gradient
weight on a still-recovering head) or near 0 (under-weighting a
converged head).
Fix:
- New `pub fn reset_iqn_readiness_state` on `GpuDqnTrainer` zeroes the
three coupled scalars (iqn_loss_initial / iqn_loss_ema /
iqn_readiness) and writes 0.0 through the device-mapped pinned host
slot — no HtoD copy, the mapping propagates the host write directly.
- New `pub(crate) fn reset_iqn_readiness_state` wrapper on
`FusedTrainingCtx` mirrors the existing `reset_eval_v_range_state`
pattern.
- Three new `FoldReset` registry entries (`isv_iqn_loss_initial`,
`isv_iqn_loss_ema`, `isv_iqn_readiness`) dispatch through one match
arm in `reset_named_state` (training_loop.rs).
`update_iqn_readiness` already handles `iqn_loss_initial < 1e-12` as
the bootstrap branch, so the new fold's first batch is recaptured as
the anchor on the first call after reset, by design.
Per `feedback_no_partial_refactor.md` — same fold-boundary
state-migration contract, all three coupled scalars migrate together.
This commit completes the 3-fix sequence (Fix 1 IQN target sync,
Fix 2 aux Adam states, Fix 3 IQN readiness) closing the fold-boundary
contract gap surfaced by the audit. Bug signature being resolved:
F0 ep5 Q=+0.79 (healthy)
F1 ep1 Q=+0.82 (boundary fine; c51_alpha warmup blends IQN low)
F1 ep2 Q=+2.23 (drift starts — IQN online↔target gap widens)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66 (geometric ~2.3×/epoch)
F1 ep5 NaN at step 5 (fp32 overflow past atom support)
Fix 1 (IQN target sync) is the load-bearing fix for the geometric
drift itself; Fix 2 prevents first-epoch Adam-step overshoot in
seven aux optimizers; Fix 3 closes the gauge-staleness path.
cargo check -p ml --lib clean at 13 warnings (workspace baseline).
cargo test -p ml --lib --no-run clean. All three existing
`state_reset_registry::tests` pass without modification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend the fold-boundary Adam reset contract to every auxiliary
optimizer added after the main DQN's `reset_adam_state` was wired:
- `GpuDqnTrainer::reset_adam_state` now also memsets q_attn /
sel / denoise / mamba2 / ofi_embed (m, v) buffer pairs and
zeroes the corresponding adam_step counters.
- New `pub(crate) fn reset_adam_state` on `GpuTlob` (memsets
adam_m/v, zeroes adam_step + writes 0 to the device-mapped
pinned host counter so the next adam_step kernel reads t=1
after `increment_adam_step`).
- New `pub fn reset_adam_state` on `GpuIqlTrainer` (same pattern,
m_buf/v_buf/adam_step/t_pinned).
- `FusedTrainingCtx::reset_for_fold` invokes the three new
helpers immediately after the existing IQN Adam reset block.
Both `gpu_iql` and `gpu_iql_low` are reset.
Per `feedback_no_partial_refactor.md` — the fold-boundary
state-migration contract has multiple consumers (main DQN trunk,
IQN head, aux Adam states across q_attn / sel / denoise / mamba2
/ ofi_embed / tlob / iql / iql_low). Adding `sync_target_from_online`
to the main DQN years ago without extending it to subsequently-
added optimizers left a "partial migration strictly worse than
the original" state. This commit + Fix 1 + Fix 3 close all three
remaining gaps surfaced by the fold-boundary audit.
All new resets follow the existing main-DQN pattern: DtoD memset-
zero only, pinned host counter written through the device-mapped
page (no HtoD/HtoH/DtoH paths). cargo check -p ml --lib clean
at 13 warnings (workspace baseline). cargo test -p ml --lib
--no-run clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `GpuIqnHead::sync_target_from_online` (DtoD copy of online_params →
target_params) and call it from `FusedTrainingCtx::reset_for_fold`
alongside the existing IQN Adam reset.
Root cause: at fold boundary the main DQN does shrink-and-perturb +
hard target sync (gpu_dqn_trainer.rs:12495 — the latter explicitly added
to prevent fold-1 grad explosion). The IQN head was added later but
only had Polyak EMA `target_ema_update` — no hard-sync existed. Polyak
at τ=0.005/step closes ~0.5%/step, far too slow to close the
fold-boundary online↔target gap before inflated Bellman TD-error
compounds geometrically through the IQN backward pass.
Bug signature this resolves:
F0 ep5 Q=+0.79 (healthy)
F1 ep1 Q=+0.82 (boundary fine; c51_alpha warmup blends IQN low)
F1 ep2 Q=+2.23 (drift starts — c51_alpha ramp + IQN online↔target gap widens)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66 (geometric ~2.3×/epoch)
F1 ep5 NaN at step 5 (fp32 overflow past atom support)
Per `feedback_no_partial_refactor.md`: when a shared contract
(fold-boundary state migration) changes, every consumer must migrate
together. The main DQN sync was added explicitly — the IQN, TLOB, IQL,
and aux Adam states were added later but never extended to that
contract. This commit closes the IQN gap (primary). Aux Adam + IQN
readiness gaps follow in subsequent commits.
Touched:
- gpu_iqn_head.rs: new `sync_target_from_online` (mirrors gpu_dqn_trainer.rs:12495 exactly)
- fused_training.rs: call the new sync inside the existing IQN reset block
- docs/dqn-wire-up-audit.md: Invariant 7 entry covering Fix 1 of 3
cargo check -p ml --lib clean at 13 warnings (workspace baseline). No
fingerprint change. DtoD memcpy only — no HtoD/HtoH/DtoH paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Best-checkpoint save (val Sharpe improvement, ~30% of epochs in
convergent runs) blocked the epoch loop for 20-40s on each improvement:
serialize_model() chained N (~26) per-tensor DtoH downloads via
GpuTensor::to_host → memcpy_dtoh, each forcing an implicit stream sync
on a busy training stream. The DtoH chain also violates
feedback_no_htod_htoh_only_mapped_pinned.md (only cuMemHostAlloc
DEVICEMAP allowed for CPU↔GPU paths).
Plan B:
- Introduce snapshot_model_to_pinned (mod.rs): allocate one
MappedF32Buffer sized for all named weight slices concatenated,
cuMemcpyDtoDAsync each slice into the buffer's device pointer
(which aliases the host page), single stream sync, copy bytes out
to a Send + 'static Vec<u8>. One sync per snapshot, replaces N.
- serialize_snapshot_bytes (mod.rs): pure-CPU safetensors construction
from CheckpointSnapshot. Static — callable without &self, so the
worker can move the snapshot across thread boundary.
- handle_epoch_checkpoints_and_early_stopping on val-Sharpe
improvement: save_best_gpu_params (DtoD, fast) + snapshot to pinned
+ tokio::task::spawn_blocking the safetensors construction +
checkpoint_callback invocation. JoinHandle parked on
pending_checkpoint_handles. Training loop continues immediately.
- await_pending_checkpoint_handles drains in-flight workers at
training end (success branch + early-stop branches) and before
any synchronous cold-path checkpoint write to keep disk ordering
deterministic.
- F bound on train / train_walk_forward / train_fold_from_slices
gains + 'static so the callback can be moved into the worker.
All public callers already use 'static-compatible move closures
(test fixtures with shared mutable state migrate to Arc<Mutex<T>>).
Internal pipeline uses CheckpointCallbackHandle =
Arc<std::sync::Mutex<Box<dyn FnMut + Send + 'static>>> so the
same callback flows through multi-fold walk-forward into every
fold's worker.
- serialize_model itself rewritten via the snapshot path: the
no-DtoH rule now holds across ALL checkpoint paths (best, periodic,
early-stop, plateau-exhausted). The pre-existing GpuTensor::to_host
path is no longer reachable from the DQN trainer.
The audit's spec called for an mpsc channel(1) drop-old worker, but
the multi-fold + &mut F pre-existing API made the simpler
fire-and-forget spawn_blocking pattern a cleaner fit (Mutex
serialises any concurrent invocations; Vec<JoinHandle> drain at end
guarantees disk writes complete before the trainer returns). Same
overlap benefit (training rolls while serialize+disk run on a
blocking thread); upper bound on in-flight work is one-per-improved-
epoch which approximates the spec's depth=1 in realistic training
runs.
Per feedback_no_partial_refactor: every site that constructs a
checkpoint payload migrated in lockstep — best-improvement uses
the worker; periodic / plateau-exhausted / early-stop call the
shared Arc<Mutex<F>> handle inline. All paths read params via
snapshot_model_to_pinned, so the no-DtoH rule applies uniformly.
Test fixtures (8 .rs files) updated for the + 'static bound (move
closures + cloned PathBufs / Arc<Mutex<T>> for shared mutable state).
Verified: SQLX_OFFLINE=true cargo check --workspace --tests clean
(warnings unchanged from baseline). cargo test -p ml --lib --no-run
clean. No fingerprint change.
Wire-up audit entry extended with Plan B file:line edit sites
(rides under the same Async-validation overlap section started by
the companion Plan A commit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The train-multi-seed-p526h repro (with MSE-clamp + PopArt-carry already
landed) reproduced a geometric Q-divergence in fold 1 that train_loss
can no longer hide:
F0 ep5 Q=+0.79 (healthy carryover)
F1 ep1 Q=+0.82 (boundary handoff fine)
F1 ep2 Q=+2.23 (2.7× jump — drift starts)
F1 ep3 Q=+4.05
F1 ep4 Q=+10.66
F1 ep5 NaN at step 5
A model with this Q dynamic is catastrophically unsafe for live trading:
Kelly cap floats with Q-confidence, so runaway Q drives oversized
positions and can blow up the strategy before any downstream safety
trips. This commit installs a production safety net — NOT a root-cause
fix — that hard-halts training the moment Q-drift is detected:
if |q_mean| > 2× prev AND |q_mean| > 1.5 production-unsafe floor
→ return Err, training halts, model rejected from deployment
Inserted in training_loop.rs immediately before the existing
`self.prev_epoch_q_mean = q_mean` update so the comparison uses the
same source-of-truth values that already feed downstream diagnostics.
The 2× ratio catches genuine geometric divergence (typical healthy
growth <30%/epoch); the 1.5 absolute floor prevents false positives
in early training where small Q magnitudes oscillate by large ratios
(verified: F0 ep4→ep5 ratio 5.3× would NOT trigger because |0.79|<1.5).
Both thresholds are numerical-stability bounds (Invariant 1 carve-
out), not tuned hyperparameters. Skips first epoch (no prev_q). Does
NOT skip fold-boundary epochs — those are exactly the failures
we're catching.
This is a safety net. The actual root cause is a fold-boundary reset
gap (which reset isn't firing correctly?). A systematic audit is
queued to identify it. Suspects: prev_grad_buf, PopArt GPU Welford
buffers (popart_count huge from fold 0 → new-fold stats track too
slowly), isv_q_abs_ref_* magnitude EMAs, spectral norm σ EMAs, TLOB
Adam state.
Per user's "can't happen at production" philosophy: production
deployments must have this safety net regardless of whether the
root-cause fix lands first.
Validation backtest was serialised on the training stream by passing
self.cuda_stream as parent_stream into GpuBacktestEvaluator::new (the
forked eval-stream still depends on training-stream completion, but the
metrics + plan_diag readbacks went through clone_dtoh / memcpy_dtoh,
each forcing an implicit sync that blocked behind the training stream's
pending work). The prior comment claimed single-stream eval was needed
"for cuBLAS determinism" — incorrect: cuBLAS bit-wise reproducibility
caveats apply per-handle, not across the process, and each stream
already owns its own PerStreamCublasHandles. Costs ~30-40s/epoch on L40S.
Plan A:
- Route eval through self.validation_stream (already forked at
constructor.rs:547 but unused on the eval path) and add a fresh
cuda_stream-recorded train_done_event so validation_stream +
evaluator's internal forked stream both wait on it before launching
eval kernels. GPU-side cuStreamWaitEvent — no CPU sync.
- Replace metrics_buf and plan_diag_buf in GpuBacktestEvaluator from
CudaSlice<f32> to MappedF32Buffer (cuMemHostAlloc DEVICEMAP). Kernels
write through the buffer's dev_ptr; host reads via volatile host_ptr
after a single eval_stream.synchronize() per evaluation. Replaces
the forbidden clone_dtoh / memcpy_dtoh path
per feedback_no_htod_htoh_only_mapped_pinned.md. The single sync
blocks only the eval stream — training rolls on uninterrupted.
- Update the misleading "single-stream cuBLAS determinism" comment
to explain the actual per-stream-handle architecture.
Per feedback_no_partial_refactor: every consumer of the
metrics_buf / plan_diag_buf contract migrated in lockstep (struct
fields, alloc, kernel-arg passing, host readout). Per-stream cuBLAS
handle parity with training preserved (val TLOB still uses
PerStreamCublasHandles::new(&eval_stream); evaluator forks its own
internal cuBLAS state from parent_stream).
Verified: cargo check -p ml --lib clean (13 warnings, workspace
baseline). cargo test -p ml --lib --no-run clean. No fingerprint
change — mapped-pinned is allocation-method orthogonal to kernel
buffer layout.
Companion commit lands Plan B (async best-checkpoint serialize).
Wire-up audit entry covers Plan A here; will be extended with Plan B
edit sites in the companion commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous fixes (PopArt carry-forward + iqr floor in d710f9d50) reduced
fold-1 ep-1 train loss 16× but ep-2/ep-3 still oscillated 10k-200k
while val Sharpe stayed healthy at 76 — confirming the model itself
trains fine and the inflated train_loss is a pure measurement artefact.
Root cause in mse_loss_kernel.cu:457: the Bellman target
target_q = reward + γ·(1-done)·target_eq
is NOT clamped to the C51 atom support [v_min, v_max], whereas
`online_eq` is naturally bounded (it's an expected value of softmax
probabilities over atoms in that support). When PopArt-normalised
reward lands outside support — most acutely at fold boundaries when
cached robust stats lag the new fold's distribution by one epoch —
td = online_eq - target_q
grows to O(10^3–10^4), and
mse = 0.5·td²
inflates to 10^5–10^6 per-sample, dwarfing the (correctly bounded)
C51 distributional loss in the blended warmup total
(1-α)·MSE + α·C51
C51 already handles this in c51_loss_kernel.cu:247
t_z = fminf(fmaxf(t_z, v_min), v_max)
because the categorical projection requires it. MSE was missing the
matching clamp.
Fix: one-line `target_q = fminf(fmaxf(target_q, v_min), v_max)` in
mse_loss_kernel.cu between the ensemble-disagreement adjustment and
the td compute. Mirrors C51's clamp exactly.
No information lost: anything outside [v_min, v_max] is unrepresentable
in the categorical distribution anyway, so MSE warmup tracking targets
that the network categorically CANNOT learn produces a meaningless
reading. With the clamp, train loss stays bounded by support × atoms,
matching healthy fold-0 readings.
Per pearl_blend_formulas_must_have_permanent_floor.md (numerical-
stability bound carve-out under Invariant 1).
L40S 15-epoch repro #2 (train-multi-seed-kdkdv) revealed train_loss
exploded 1000-2000× in fold 1 (3.1 → 318k-694k) while val Sharpe
stayed healthy and CLIMBING (73 → 114). Train/val disconnect = pure
measurement bug, not real instability.
Mechanism: reset_for_fold() zeroed cached_iqr/cached_median at every
fold boundary. The `if cached_iqr > 0.0` guard at fused_training.rs:1220
then forced fold N+1's first epoch to the Welford GPU path. Welford
running stats inherited from fold N, combined with fold N+1's slightly
different reward distribution post-S&P + adversarial regime, produced
normalized rewards far outside C51 atom support [-50, 50] — categorical
loss readings 10^5x inflated. On rare timing-sensitive paths the
near-zero divide overflowed to Inf → NaN (the run-1 flagged=[2=on_b_logits,
3=mse_loss, 6=grad_buf, 7=save_current_lp, 8=save_projected]
diagnostic — what we caught was downstream of THIS root cause).
The diagnostic infrastructure from 756b1ef31 + 32e5375ac worked perfectly:
its precise signal of "on_v_logits clean, on_b_logits NaN, but loss is
also NaN, params clean pre-forward" surfaced the train/val disconnect
that pointed at the reward normalization bug.
Fix A (carry-forward, fused_training.rs:919-922):
Stop resetting cached_iqr / cached_median at fold boundary. Carry
fold N's final-epoch median/IQR forward as fold N+1's epoch-1
default. Same instrument, similar reward distribution between
adjacent walk-forward folds, so the carry is safe and gets replaced
by fresh stats at the end of fold N+1's epoch 1.
prev_popart_var still resets (sole consumer is tau-change detection;
a fresh fold counts as a change point regardless).
Fix B (permanent floor, dqn_utility_kernels.cu:1657):
Raise iqr fmaxf floor 1e-6 → 1e-4. With iqr=1e-6 a trade-exit
reward of 5.0 normalizes to 5e6 (vs post-fix 5e4) — much harder to
hit fp32 overflow. Defensive bound for genuinely-pathological iqr
paths (e.g. genuinely degenerate data quantiles), not a tuned
knob — Invariant 1 carve-out for numerical-stability bounds.
Per pearl_blend_formulas_must_have_permanent_floor.md (the same
recipe that resolved Kelly cap warmup + var_scale collapse before).
Resolves task #84 ("Fold-boundary state reset gap causes fold 1 grad
explosion").
Previous L40S 15-epoch repro fired the wired NaN diagnostic with
flagged=[2=on_b_logits, 3=mse_loss_scalar, 6=grad_buf, 7=save_current_lp,
8=save_projected] while value stream (flag 1) and pre-forward params
(flags 4-5) stayed clean. That isolates the corruption to the branch
advantage GEMM/activation but leaves one open question: is the GEMM
input (h_s2, shared between value and branch heads) finite or already
NaN?
Flag 12 = save_h_s2 covers the post-trunk activation. Outcomes:
- flag 12 clean + flag 2 NaN → branch GEMM overflowed (advantage-side
fp32 overflow under post-S&P + adversarial reward stress).
- flag 12 NaN → trunk encoder corrupted upstream of both heads.
Mechanically: one new check_nan_f32 call against save_h_s2 in
run_nan_checks_post_forward, names array slot 12 updated rsv12 →
save_h_s2 in training_loop.rs error message. Same ~5µs cost as the
other 12 checks. Slots 13-15 still reserved.
Per `feedback_no_partial_refactor.md` (extending the NaN diagnostic
contract atomically) and `feedback_no_stubs.md` (rsv12 was a real
reservation; now consumed).
Migrates the remaining 4 forward sites:
- VSN Linear_1 (x6 groups): adds 6 per-group RELU_BIAS shapes to the
relu_bias cache; migrates sgemm_f32_ldb + launch_add_bias_relu_f32_raw
pairs to sgemm_f32_fused_relu_bias.
- VSN Linear_2 (x6 groups, single shape (1, B, VSN_HIDDEN, VSN_HIDDEN)):
migrates to sgemm_f32_fused_bias.
- GLU value head + gate (launch_vsn_glu_branch Steps 3-4): migrates to
sgemm_f32_fused_bias with per-stream workspace selection.
Also fixes a partial-refactor leak in forward_online_raw — the
sequential branch fallback (distinct_branches=false) was still on the
unfused sgemm_f32 + launch_add_bias_relu_f32_raw pair while its
multi-stream sibling already used the fused epilogue. Now both code
paths take the fused-first contract.
Layout fingerprint unchanged (no params buffer changes).
Migrates 6 branch FC adv_logits sites (decoder_forward_only,
forward_online_raw multi-stream + sequential, forward_online_f32
multi-stream + sequential, forward_target_raw multi-stream) from
sgemm_f32(_branch) + launch_add_bias_f32_raw pairs to
sgemm_f32_fused_bias.
The branch FC output projects [B, adv_h] -> [B, branch_size_d * num_atoms]
across 4 distinct branch shapes, all pre-cached. Multi-stream sites use
the per-branch workspace (branch_workspace_ptrs[d]) to avoid contention
with concurrent branch streams. Drops up to 4 add_bias_f32 launches per
training step (one per branch direction) on each of {online, target,
f32-collector} forwards.
Migrates 4 v_logits sites (forward_online_raw, forward_online_f32,
forward_value_head ensemble heads, forward_target_raw) from
sgemm_f32 + launch_add_bias_f32_raw pairs to the new
sgemm_f32_fused_bias helper. The value-head output projects
[B, value_h] -> [B, num_atoms] (C51 atom logits), no ReLU on output.
Drops 4 standalone add_bias_f32 kernel launches per training step.
Migrates 8 trunk sites (4 online + 4 target) from
sgemm_f32 + launch_add_bias_f32_raw pairs to the new
sgemm_f32_fused_bias helper. Covers all four GRN Linear layers per
encoder: h_s1 Linear_a, h_s1 Linear_b, h_s2 Linear_a, h_s2 Linear_b.
Linear_residual has no bias term and stays a plain sgemm_f32_ldb.
Drops 8 standalone add_bias_f32 kernel launches + 8 memory
round-trips per (online + target) forward step.
