feat(sp22-vnext): Phase F-3b — K=3 CE EMA launcher wireup + reset registry

Completes the Phase F-3 observability chain. Kernel + ISV slot landed
in F-3a; this commit wires the launcher into the trainer's per-step
training graph and registers the FoldReset entry.

Changes:
- gpu_dqn_trainer.rs:
  - New pub(crate) static AUX_OUTCOME_CE_EMA_CUBIN embed
  - New aux_outcome_ce_ema_kernel: CudaFunction field on GpuDqnTrainer
  - Constructor loads kernel handle + struct-init
  - New launch_aux_outcome_ce_ema() method (single-thread launch,
    ISV slot 538, α=0.05)
- training_loop.rs:
  - Launch call appended after launch_aux_heads_loss_ema()
  - New dispatch arm "aux_trade_outcome_ce_ema" in reset_named_state
- state_reset_registry.rs:
  - New RegistryEntry for "aux_trade_outcome_ce_ema" (FoldReset
    sentinel 0.0)

End-to-end observability now live: K=3 head's batch-mean sparse-CE
flows into ISV[538] every step via Pearl A-bootstrapped EMA.
Smoke runs can read this slot to verify learning:
- Cold-start: 0.0
- After first step with B_valid > 0: bootstrap to first observation
  (~1.098 = ln(3) for uniform K=3 prediction)
- Healthy learning: monotonic decrease toward 0.5-0.7 over epochs
- Falsification: pinned at ~1.098 for many epochs

Phase F prep complete. The trade-outcome aux head's:
- Forward chain (A2-A5 + B0-B4)
- Real labels reach trainer (B4b-1/2)
- K=3 → policy state (C-1/C-2)
- K=3 → Q-target atom-shift (D)
- Plan-conditioning (B5b)
- Smoke observability (F-3 + F-3b)
are all wired. Phase F deployment (argo-train.sh smoke) is next.

Verification:
- cargo check -p ml clean.
- cargo test -p ml --lib → 1016/0 green (incl. every_fold_and_soft_
  reset_entry_has_dispatch_arm pin test).

Audit: docs/dqn-wire-up-audit.md Phase F-3b section.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-14 11:34:32 +02:00
parent de64935b78
commit 02479c885d
4 changed files with 142 additions and 0 deletions

View File

@@ -700,6 +700,14 @@ pub(crate) static SP22_AUX_OUTCOME_SOFTMAX_TO_PER_ENV_CUBIN: &[u8] =
pub(crate) static AUX_TO_INPUT_CONCAT_CUBIN: &[u8] =
include_bytes!(concat!(env!("OUT_DIR"), "/aux_to_input_concat_kernel.cubin"));
/// SP22 H6 vNext Phase F-3 (2026-05-14): K=3 trade-outcome head's
/// sparse-CE EMA producer cubin. Single-thread direct-to-ISV writer.
/// Loaded by `GpuDqnTrainer` at construction; launched per training
/// step via `launch_aux_outcome_ce_ema`.
#[allow(dead_code)]
pub(crate) static AUX_OUTCOME_CE_EMA_CUBIN: &[u8] =
include_bytes!(concat!(env!("OUT_DIR"), "/aux_outcome_ce_ema_kernel.cubin"));
/// SP22 H6 vNext Phase B5b (2026-05-14): row-truncating SAXPY cubin for
/// backward dh_s2_aux accumulate. Loaded by `AuxTradeOutcomeBackwardOps`.
#[allow(dead_code)]
@@ -6425,6 +6433,14 @@ pub struct GpuDqnTrainer {
/// `[B, AUX_OUTCOME_K=3]` per-sample db2 partial. Reduced into the
/// flat `params_buf` Adam slot [166].
aux_partial_to_b2: CudaSlice<f32>,
/// SP22 H6 vNext Phase F-3 (2026-05-14): K=3 trade-outcome head's
/// sparse-CE EMA producer kernel handle. Loaded at trainer
/// construction from `AUX_OUTCOME_CE_EMA_CUBIN`. Launched per
/// training step via `launch_aux_outcome_ce_ema` (called from
/// `training_loop.rs` after `launch_aux_heads_loss_ema`). Writes
/// `ISV[AUX_TRADE_OUTCOME_CE_EMA_INDEX=538]` for smoke
/// observability.
aux_outcome_ce_ema_kernel: CudaFunction,
/// SP22 H6 vNext Phase B5b (2026-05-14): plan-conditioned input
/// buffer for the K=3 trade-outcome forward + backward. Sized
/// `[B, SH2 + PLAN_PARAM_DIM = 262]` f32 row-major. Populated each
@@ -18884,6 +18900,47 @@ impl GpuDqnTrainer {
Ok(())
}
/// SP22 H6 vNext Phase F-3 (2026-05-14): launch the K=3 trade-outcome
/// head's sparse-CE EMA producer. Reads `aux_to_loss_scalar_buf[0]`
/// (written by `aux_trade_outcome_loss_reduce` inside the captured
/// forward graph) and EMA-blends into
/// `ISV[AUX_TRADE_OUTCOME_CE_EMA_INDEX=538]` with α=0.05.
///
/// Cadence: same as `launch_aux_heads_loss_ema` (called after it in
/// training_loop's per-step block). Single-thread single-block
/// kernel; trivial cost.
pub fn launch_aux_outcome_ce_ema(&self) -> Result<(), MLError> {
use crate::cuda_pipeline::sp22_isv_slots::AUX_TRADE_OUTCOME_CE_EMA_INDEX;
if self.isv_signals_dev_ptr == 0 {
// ISV bus not wired (test scaffolds) — no-op.
return Ok(());
}
let loss_ptr = self.aux_to_loss_scalar_buf.raw_ptr();
let isv_ptr = self.isv_signals_dev_ptr;
let idx_i32 = AUX_TRADE_OUTCOME_CE_EMA_INDEX as i32;
let alpha: f32 = 0.05;
unsafe {
self.stream
.launch_builder(&self.aux_outcome_ce_ema_kernel)
.arg(&loss_ptr)
.arg(&isv_ptr)
.arg(&idx_i32)
.arg(&alpha)
.launch(LaunchConfig {
grid_dim: (1, 1, 1),
block_dim: (1, 1, 1),
shared_mem_bytes: 0,
})
.map_err(|e| MLError::ModelError(format!(
"sp22-vnext F-3: aux_outcome_ce_ema launch: {e}"
)))?;
}
Ok(())
}
/// HEALTH_DIAG GPU port — Phase 2A: launch `health_diag_isv_mirror`.
///
/// Copies a curated set of ISV signal-bus slots (reward-component EMAs,
@@ -23260,6 +23317,19 @@ impl GpuDqnTrainer {
let aux_to_valid_count_buf = alloc_f32(&stream, 1, "aux_to_valid_count_buf")?;
let aux_dh_s2_to_buf = alloc_f32(&stream, aux_b * aux_sh2_total, "aux_dh_s2_to_buf")?;
let aux_partial_to_w1 = alloc_f32(&stream, aux_b * aux_h * aux_sh2_total, "aux_partial_to_w1")?;
// SP22 H6 vNext Phase F-3 (2026-05-14): K=3 CE EMA producer
// kernel handle. Loaded once at trainer construction.
let aux_outcome_ce_ema_kernel = {
let module = stream.context()
.load_cubin(AUX_OUTCOME_CE_EMA_CUBIN.to_vec())
.map_err(|e| MLError::ModelError(format!(
"sp22-vnext F-3: aux_outcome_ce_ema cubin load: {e}"
)))?;
module.load_function("aux_outcome_ce_ema_update")
.map_err(|e| MLError::ModelError(format!(
"sp22-vnext F-3: aux_outcome_ce_ema_update load: {e}"
)))?
};
let aux_partial_to_b1 = alloc_f32(&stream, aux_b * aux_h, "aux_partial_to_b1")?;
let aux_partial_to_w2 = alloc_f32(&stream, aux_b * aux_kto * aux_h, "aux_partial_to_w2")?;
let aux_partial_to_b2 = alloc_f32(&stream, aux_b * aux_kto, "aux_partial_to_b2")?;
@@ -25704,6 +25774,9 @@ impl GpuDqnTrainer {
aux_partial_to_b1,
aux_partial_to_w2,
aux_partial_to_b2,
// SP22 H6 vNext Phase F-3 (2026-05-14): K=3 CE EMA producer
// kernel handle.
aux_outcome_ce_ema_kernel,
// SP22 H6 vNext Phase B5b (2026-05-14): plan-conditioned
// input buffer for the K=3 forward + backward.
aux_to_input_buf,

View File

@@ -2166,6 +2166,13 @@ impl StateResetRegistry {
category: ResetCategory::FoldReset,
description: "ISV[SP22_AUX_ALIGN_SCALE_INDEX=537] — SP22 H6 Phase 3 (2026-05-13) SP11-controller-emitted adaptive scale_β for the β producer at trade-close. Producer (eventual): extended `reward_subsystem_controller_kernel.cu` (7-weight output, anchor on `REWARD_AUX_ALIGN_EMA_INDEX=536`, target ~10% of total reward magnitude, bounds [0.05, 2.0]) — Phase B6, NOT yet landed. Consumers: `experience_env_step::segment_complete` (training-side β: `r_aux_align = scale_β × alignment × profit_pos`) and `backtest_env_kernel::segment_complete` (eval-side β with the same formula); HEALTH_DIAG `sp11_reward` printer (the `w_aux=…` column). FoldReset sentinel 0.5 — structural prior matching W's [-0.5, 0, +0.5, 0] init approach; activates β with fixed magnitude from epoch 0. The 0.5 prior was chosen over 0 (the original spec) because slot 537 has no producer yet (Phase B6 deferred); 0 would make β a runtime no-op. Once B6 lands and the controller emits adaptively per the bounds [0.05, 2.0], the prior is overwritten on first emit per `pearl_first_observation_bootstrap`; subsequent EMA-driven adaptation tracks the anchor signal.",
},
// SP22 H6 vNext Phase F-3 (2026-05-14) — K=3 trade-outcome
// head sparse-CE EMA for smoke validation observability.
RegistryEntry {
name: "aux_trade_outcome_ce_ema",
category: ResetCategory::FoldReset,
description: "ISV[AUX_TRADE_OUTCOME_CE_EMA_INDEX=538] — SP22 H6 vNext Phase F-3 (2026-05-14) batch-mean sparse cross-entropy EMA of the K=3 trade-outcome aux head. Producer: `aux_outcome_ce_ema_update` kernel (single-thread, single-block, direct-to-ISV with Pearl A first-observation bootstrap, α=0.05). Reads `aux_to_loss_scalar_buf[0]` (written by `aux_trade_outcome_loss_reduce` inside the captured forward graph). Cold-start sentinel 0.0; Pearl A bootstrap replaces with the first non-zero observation. FoldReset sentinel 0.0 — each fold's first non-zero loss observation re-bootstraps the EMA from scratch (the head's training state is preserved across folds via Adam moments, but the per-fold loss distribution can shift slightly with the new fold's data). Consumers (initial): smoke validation observability — operator reads the slot via HEALTH_DIAG or ISV inspector to verify the head is learning (CE drops from ~1.098=ln(3) toward 0.5-0.7 over epochs if signal-rich; pinned at ln(3) if signal-poor).",
},
// SP22 H6 vNext Phase A3 (2026-05-13) — save-for-backward
// buffers for the trade-outcome aux head's label producer.
// Both written at every segment_complete bar alongside

View File

@@ -4770,6 +4770,17 @@ impl DQNTrainer {
if let Err(e) = fused.trainer().launch_aux_heads_loss_ema() {
tracing::warn!("Plan 4 Task 6 Commit B aux_heads_loss_ema launch failed: {e}");
}
// SP22 H6 vNext Phase F-3 (2026-05-14): launch the
// K=3 trade-outcome head's sparse-CE EMA producer
// right after the K=2/K=5 EMA. Writes
// ISV[AUX_TRADE_OUTCOME_CE_EMA_INDEX=538] for
// observability during smoke runs (expected
// cold-start ~1.098 = ln(3) for uniform K=3
// prediction; learning signal = CE drop toward
// 0.5-0.7 over epochs).
if let Err(e) = fused.trainer().launch_aux_outcome_ce_ema() {
tracing::warn!("Phase F-3 aux_outcome_ce_ema launch failed: {e}");
}
}
// SP13 P0a.T4 + SP14 (2026-05-07 cadence fix, Phase C.1
@@ -10654,6 +10665,16 @@ impl DQNTrainer {
fused.trainer().write_isv_signal_at(SP22_AUX_ALIGN_SCALE_INDEX, 0.0);
}
}
// SP22 H6 vNext Phase F-3 (2026-05-14): K=3 trade-outcome
// sparse-CE EMA for smoke observability.
"aux_trade_outcome_ce_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp22_isv_slots::AUX_TRADE_OUTCOME_CE_EMA_INDEX;
fused.trainer().write_isv_signal_at(
AUX_TRADE_OUTCOME_CE_EMA_INDEX, 0.0,
);
}
}
_ => {
return Err(crate::MLError::ModelError(format!(
"StateResetRegistry reset dispatch: unknown name '{}'. \

View File

@@ -18253,3 +18253,44 @@ The existing K=2/K=5 aux-loss EMA writes through Pearls A+D's 2-stage producer s
- State reset registry entry: `AUX_TRADE_OUTCOME_CE_EMA_INDEX` as FoldReset sentinel 0
**Verification**: `cargo check -p ml` clean; `cargo test -p ml --lib`**1016/0 green**.
#### Phase F-3b — K=3 CE EMA launcher wireup + reset registry (2026-05-14)
Completes the Phase F-3 observability chain. The kernel + ISV slot landed in F-3a; this commit wires the launcher into the trainer's per-step training graph and registers the FoldReset entry.
**Changes**:
`gpu_dqn_trainer.rs`:
- New `pub(crate) static AUX_OUTCOME_CE_EMA_CUBIN` embed.
- New `aux_outcome_ce_ema_kernel: CudaFunction` field on `GpuDqnTrainer`.
- Constructor loads the kernel handle from the new cubin.
- New method `launch_aux_outcome_ce_ema()` on `GpuDqnTrainer` — single-thread, single-block launch with the ISV slot index + α=0.05.
`training_loop.rs`:
- New launch call appended after `launch_aux_heads_loss_ema()` in the per-step training block. Tracing-warn on failure (matches the sibling K=2 EMA's error handling).
- New dispatch arm `"aux_trade_outcome_ce_ema"` in `reset_named_state` — writes 0.0 to ISV slot 538 at fold boundary.
`state_reset_registry.rs`:
- New `RegistryEntry { name: "aux_trade_outcome_ce_ema", category: FoldReset, ... }` with detailed description of the smoke validation signal interpretation.
**Pin tests pass** including `every_fold_and_soft_reset_entry_has_dispatch_arm` — the new entry has its required dispatch arm.
**End-to-end observability now live**: at every training step the K=3 head's batch-mean sparse-CE flows into `ISV[538]` via Pearl A-bootstrapped EMA. Smoke runs can read this slot to verify the head is learning:
- Cold-start: 0.0
- After first step with `B_valid > 0`: bootstrap to first observation (~1.098 = ln(3) for uniform K=3 prediction)
- Healthy learning: monotonic-ish decrease toward 0.5-0.7 over epochs
- Falsification signal: pinned at ~1.098 for many epochs → head can't learn the labels
**Verification**:
- `cargo check -p ml` clean.
- `cargo test -p ml --lib`**1016/0 green** (including the registry pin test that catches missing dispatch arms).
**Phase F prep complete**. The trade-outcome aux head's:
- Forward chain: A2-A5 kernels + B0-B4 wireup ✓
- Real labels reach trainer: B4b-1/2 ✓
- K=3 → policy state: C-1/C-2 ✓
- K=3 → Q-target atom-shift: D ✓
- Plan-conditioning: B5b ✓
- Smoke observability: F-3 + F-3b ✓
**Phase F deployment** (next): push branch + `argo-train.sh` smoke run. Decisive test of the spec's hypothesis. Spec falsification criterion (per the original spec doc): if smoke produces WR ≈ 0.4346 ± 0.005, the trade-outcome hypothesis is falsified; the conclusion would be "WR is data-determined at this bar resolution + horizon, not a model architecture issue", and future work explores different data sampling / different prediction target.