From 840408ceb75194d4a946a44e7d11bd4c41aa3b80 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 28 Apr 2026 21:24:15 +0200 Subject: [PATCH] perf(htod): migrate training_loop.rs raw features/targets to mapped-pinned (2 sites) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two `crate::cuda_pipeline::clone_htod_f32` callsites in `training_loop.rs::set_raw_market_data` (lines 1250 and 1254) are rewritten to `mapped_pinned::clone_to_device_f32_via_pinned` per `feedback_no_htod_htoh_only_mapped_pinned.md`. These run once per fold immediately before the step loop begins; the `_raw` suffix indicates the un-normalized arrays kept on-GPU for the eval-time critic. Error-type continuity: the helper already returns `Result<_, String>`, which feeds `anyhow::anyhow!("…raw upload: {e}")` directly — the prior .map_err wrapper is preserved verbatim. 1:1 path swap. docs/dqn-gpu-hot-path-audit.md updated with Fix 16 entry. cargo check -p ml --lib clean at 12 warnings. Co-Authored-By: Claude Opus 4.7 (1M context) --- crates/ml/src/trainers/dqn/trainer/training_loop.rs | 4 ++-- docs/dqn-gpu-hot-path-audit.md | 3 +++ 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/crates/ml/src/trainers/dqn/trainer/training_loop.rs b/crates/ml/src/trainers/dqn/trainer/training_loop.rs index d03d59cb2..21839d1b5 100644 --- a/crates/ml/src/trainers/dqn/trainer/training_loop.rs +++ b/crates/ml/src/trainers/dqn/trainer/training_loop.rs @@ -1247,11 +1247,11 @@ impl DQNTrainer { .collect(); self.targets_raw_cuda = Some( - crate::cuda_pipeline::clone_htod_f32(&stream, &flat_targets) + crate::cuda_pipeline::mapped_pinned::clone_to_device_f32_via_pinned(&stream, &flat_targets) .map_err(|e| anyhow::anyhow!("targets_raw upload: {e}"))? ); self.features_raw_cuda = Some( - crate::cuda_pipeline::clone_htod_f32(&stream, &flat_features) + crate::cuda_pipeline::mapped_pinned::clone_to_device_f32_via_pinned(&stream, &flat_features) .map_err(|e| anyhow::anyhow!("features_raw upload: {e}"))? ); diff --git a/docs/dqn-gpu-hot-path-audit.md b/docs/dqn-gpu-hot-path-audit.md index e43d54abe..356c7e3dc 100644 --- a/docs/dqn-gpu-hot-path-audit.md +++ b/docs/dqn-gpu-hot-path-audit.md @@ -160,6 +160,9 @@ Out-of-scope (remaining host-side stream syncs in eval, not part of this fix): ` - `collect_experiences` (warm but per-collect): `episode_starts_buf` (memcpy_htod) → `upload_i32_via_pinned`; `barrier_config` (htod_f32) → `upload_f32_via_pinned`. - `reset_episodes` (warm, per-epoch): replaced the `Vec` / `Vec` host staging fields with persistent `MappedF32Buffer` / `MappedU32Buffer` allocated once at construction. Resets fill via `host_slice_mut()` (zero-allocation, zero-HtoD); a single async DtoD seeds the persistent device-resident `portfolio_states` / `rng_states` (which the kernel mutates each step). Adds `host_slice_mut()` accessor on both mapped-pinned types. +### Fix 16 — Migrate `training_loop.rs` raw features/targets uploads (2 sites, 2026-04-28) +`trainers/dqn/trainer/training_loop.rs:1250` (targets_raw) and `:1254` (features_raw): both `crate::cuda_pipeline::clone_htod_f32` callsites in the data-load helper rewritten to `mapped_pinned::clone_to_device_f32_via_pinned`. These run once per fold immediately before the step loop begins (the `_raw` suffix indicates the un-normalized arrays kept on-GPU for the eval-time critic). The `.map_err(|e| anyhow::anyhow!("…raw upload: {e}"))` wrapping is preserved verbatim — the helper already returns `Result<_, String>` which feeds `anyhow::anyhow!` directly, so the change is a 1:1 path swap. + ### Fix 15 — Migrate `gpu_experience_collector.rs::upload_ofi_features` (1 site, 2026-04-28) `gpu_experience_collector.rs:4258`: warm-path `super::clone_htod_f32` upload of OFI features rewritten to `mapped_pinned::clone_to_device_f32_via_pinned`. Called from training_loop's data-load phase before each fold's training run, so the OFI tensor (4M bars × 32 dims = ~512 MB) now stages through mapped-pinned + DtoD just like all other large data uploads. Error-type shift handled with `.map_err(|e| MLError::ModelError(format!("upload_ofi_features: {e}")))?` since the helper returns `Result<_, String>` and the function returns `Result<_, MLError>`.