plan(sp18 v2): Phase 0 Task 0.2 — B-leg V_SHARE trajectory + TD-error magnitude diagnostic
Adds the SP18 v2 Phase 0 B-leg observability scaffold per the plan's
Task 0.2 spec:
- New kernel `td_error_mag_ema_kernel.cu` (single-block × 256 threads):
reads `td_errors_buf [B]` (already populated by C51/MSE loss for PER
priority recomputation, post-train-step), block tree-reduces
`mean(|td_errors[b]|)` (no atomicAdd), and EMA-blends into
`ISV[TD_ERROR_MAG_EMA_INDEX=493]` via Pearl-A first-observation
bootstrap (sentinel 0.0 → REPLACE) + fixed α=`WELFORD_ALPHA_MIN=0.4`
per `pearl_wiener_alpha_floor_for_nonstationary`. The TDB_* Welford
accumulators in slots [498..504) are RESERVED for the Phase 4
q_next_target Wiener-α chain — not used in Phase 0.
- Cubin manifest entry in `crates/ml/build.rs` + `TD_ERROR_MAG_EMA_
CUBIN` re-export in `gpu_dqn_trainer.rs`.
- `sp18_td_error_mag_ema_kernel` field on `GpuDqnTrainer` + cubin load
on the trainer's stream + `launch_sp18_td_error_mag_ema_update()`
cold-path launcher + `read_sp18_td_error_mag_ema()` convenience
wrapper.
- New `sp18_v_share_history: [[f32; 4]; 5]` field on `DQNTrainer` —
fixed-size ring buffer of the last 5 epochs of per-branch V_SHARE
EMA readings (slots [478..482) per SP17 Phase 3.2). Initialised to
`[[NaN; 4]; 5]`; epochs 0–3 emit `nan` as the slope and skip the
ISV write; epoch 4 onward computes `(EMA[now] - EMA[now-4]) / 4`
per branch and writes the dir-branch slope to
`ISV[V_SHARE_TREND_DIAG_INDEX=496]`.
- Two new HEALTH_DIAG lines in `training_loop.rs` at the per-epoch
boundary (right after the SP18 reward_decomp line):
HEALTH_DIAG[N]: v_share_traj [dir_slope=X mag_slope=Y ord_slope=Z urg_slope=W]
HEALTH_DIAG[N]: td_error_pre [magnitude_ema=X]
V_SHARE slope is host-side computation against the ring buffer
(`(now - now_m4) / 4` per branch). TD-error magnitude is post-blend
ISV slot 493 read (producer fires inside `read_sp18_td_error_mag_
ema()`). Pre-fix baseline for the B-DD9 ratio gate
(`avg(|TD-error|) ratio post-fix / pre-fix ∈ [0.5, 5.0]`).
- New GPU oracle tests in `crates/ml/tests/sp18_hold_reward_oracle_
tests.rs`:
* `td_error_mag_ema_pearl_a_bootstrap` — synthetic td_errors with
closed-form mean(|td|)=1.125; pre-populate slot at sentinel;
assert post-launch slot equals the mean (Pearl-A direct-replace).
* `td_error_mag_ema_blend_post_bootstrap` — synthetic td_errors
with mean(|td|)=0.5; pre-populate slot at non-sentinel 1.0;
assert blend equals `(1 - 0.4) × 1.0 + 0.4 × 0.5 = 0.8`.
Pure observability — no production-path consumer in this commit. No
reward changes, no Bellman target changes, no kernel modifications to
the action-selection or training paths. Per `feedback_no_partial_
refactor` the kernel + cubin manifest + buffer + launcher + ring
buffer + HEALTH_DIAG emit + GPU oracle tests all land atomically.
Verification:
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check --workspace clean.
All 4 GPU oracle tests pass on RTX 3050 Ti (2.09s):
reward_decomp_per_action_gpu_oracle, reward_decomp_empty_bin,
td_error_mag_ema_pearl_a_bootstrap, td_error_mag_ema_blend_post_bootstrap.
Existing slot lock + state_reset_registry tests still pass.
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
§ Phase 0 Task 0.2.
Audit: docs/dqn-wire-up-audit.md § "SP18 v2 Phase 0 Task 0.2".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -123,6 +123,13 @@ pub(crate) static REWARD_COMPONENT_EMA_CUBIN: &[u8] = include_bytes!(concat!(env
|
||||
/// reduce — 4 blocks × 256 threads. Pure observability — no
|
||||
/// production-path consumer in this commit. Plan: SP18 Phase 0 Task 0.1.
|
||||
pub(crate) static REWARD_DECOMP_DIAG_CUBIN: &[u8] = include_bytes!(concat!(env!("OUT_DIR"), "/reward_decomp_diag_kernel.cubin"));
|
||||
/// SP18 v2 Phase 0 Task 0.2 (2026-05-08): B-leg TD-error magnitude
|
||||
/// EMA producer. Single-block 256-thread kernel computing
|
||||
/// mean(|td_errors[b]|) over [B] and EMA-blending into
|
||||
/// ISV[TD_ERROR_MAG_EMA_INDEX=493] via Pearl-A first-observation
|
||||
/// bootstrap + fixed α=0.4. Pure observability: pre-fix baseline for
|
||||
/// the B-DD9 ratio gate. Plan: SP18 Phase 0 Task 0.2.
|
||||
pub(crate) static TD_ERROR_MAG_EMA_CUBIN: &[u8] = include_bytes!(concat!(env!("OUT_DIR"), "/td_error_mag_ema_kernel.cubin"));
|
||||
/// Plan 3 Task 3 B.2: Flat→Positioned transition rate EMA into ISV[71].
|
||||
pub(crate) static TRADE_RATE_EMA_CUBIN: &[u8] = include_bytes!(concat!(env!("OUT_DIR"), "/trade_rate_ema_kernel.cubin"));
|
||||
/// Plan 3 Task 4 B.4: per-batch readiness EMA + derived plan_threshold.
|
||||
@@ -6813,6 +6820,20 @@ pub struct GpuDqnTrainer {
|
||||
/// once at construct, written-then-read inside the kernel each launch.
|
||||
sp17_advantage_clip_scratch: super::mapped_pinned::MappedF32Buffer,
|
||||
|
||||
// ── SP18 v2 Phase 0 Task 0.2 (2026-05-08): B-leg TD-error magnitude
|
||||
// EMA producer kernel handle ─────────────────────────────────────
|
||||
/// Kernel handle for `td_error_mag_ema_update`. Reads
|
||||
/// `td_errors_buf [B]` (post-train-step, populated by `c51_loss` /
|
||||
/// `mse_loss` per backward configuration) and EMA-blends
|
||||
/// `mean(|td_errors[b]|)` into `ISV[TD_ERROR_MAG_EMA_INDEX=493]`
|
||||
/// via Pearl-A first-observation bootstrap + fixed α=0.4. Loaded
|
||||
/// from `td_error_mag_ema_kernel.cubin`. Consumed by
|
||||
/// `launch_sp18_td_error_mag_ema_update()` (cold-path post-train-
|
||||
/// step launch from `training_loop.rs`). Pure observability —
|
||||
/// pre-fix baseline for the B-DD9 ratio gate. Plan: SP18 Phase 0
|
||||
/// Task 0.2.
|
||||
sp18_td_error_mag_ema_kernel: CudaFunction,
|
||||
|
||||
// ── SP18 v2 Phase 0 Task 0.1 (2026-05-08): D-leg per-action reward
|
||||
// decomposition diagnostic buffer ──────────────────────────────────
|
||||
/// 20-float mapped-pinned diagnostic buffer for the SP18 D-leg
|
||||
@@ -9435,6 +9456,82 @@ impl GpuDqnTrainer {
|
||||
Ok(self.read_isv_signal_at(ADVANTAGE_CLIP_BOUND_INDEX))
|
||||
}
|
||||
|
||||
/// SP18 v2 Phase 0 Task 0.2 (2026-05-08) — B-leg TD-error magnitude
|
||||
/// EMA producer launch.
|
||||
///
|
||||
/// Cold-path (post-train-step) launcher for
|
||||
/// `td_error_mag_ema_update`. Reads the trainer-owned
|
||||
/// `td_errors_buf [B]` (populated by the C51/MSE loss kernel each
|
||||
/// training step, consumed in-place by `seg_tree_update` for PER
|
||||
/// priority recomputation), block tree-reduces
|
||||
/// `mean(|td_errors[b]|)`, applies Pearl-A first-observation
|
||||
/// bootstrap (sentinel 0.0 → REPLACE) and fixed α=0.4 EMA blend,
|
||||
/// and writes the result to `ISV[TD_ERROR_MAG_EMA_INDEX=493]`.
|
||||
///
|
||||
/// Pure observability: pre-fix baseline for the B-DD9 ratio gate
|
||||
/// (`avg(|TD-error|) ratio post-fix / pre-fix ∈ [0.5, 5.0]`).
|
||||
/// Phase 0 does NOT yet use the Welford-derived α from the TDB_*
|
||||
/// accumulators in slots [498..504) — those are reserved for Phase
|
||||
/// 4 q_next_target Wiener-α blending.
|
||||
///
|
||||
/// Returns Ok(()) on launch success; the host-side HEALTH_DIAG emit
|
||||
/// reads the slot via `read_isv_signal_at(TD_ERROR_MAG_EMA_INDEX)`
|
||||
/// after stream sync.
|
||||
pub fn launch_sp18_td_error_mag_ema_update(&self) -> Result<(), MLError> {
|
||||
use super::sp14_isv_slots::{
|
||||
TD_ERROR_MAG_EMA_INDEX, SENTINEL_TD_ERROR_MAG_EMA, WELFORD_ALPHA_MIN,
|
||||
};
|
||||
let b = self.config.batch_size;
|
||||
if b == 0 || self.isv_signals_dev_ptr == 0 {
|
||||
return Ok(());
|
||||
}
|
||||
let td_ptr = self.td_errors_buf.raw_ptr();
|
||||
let isv_ptr = self.isv_signals_dev_ptr;
|
||||
let b_i32 = b as i32;
|
||||
let ema_idx = TD_ERROR_MAG_EMA_INDEX as i32;
|
||||
let alpha: f32 = WELFORD_ALPHA_MIN;
|
||||
let sentinel: f32 = SENTINEL_TD_ERROR_MAG_EMA;
|
||||
// Single-block, 256 threads. Dynamic shmem = BLOCK_SIZE × f32.
|
||||
const BLOCK_DIM: u32 = 256;
|
||||
let shmem_bytes: u32 = BLOCK_DIM * std::mem::size_of::<f32>() as u32;
|
||||
unsafe {
|
||||
self.stream
|
||||
.launch_builder(&self.sp18_td_error_mag_ema_kernel)
|
||||
.arg(&td_ptr)
|
||||
.arg(&b_i32)
|
||||
.arg(&isv_ptr)
|
||||
.arg(&ema_idx)
|
||||
.arg(&alpha)
|
||||
.arg(&sentinel)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
block_dim: (BLOCK_DIM, 1, 1),
|
||||
shared_mem_bytes: shmem_bytes,
|
||||
})
|
||||
.map_err(|e| MLError::ModelError(format!(
|
||||
"sp18 td_error_mag_ema_update launch: {e}"
|
||||
)))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// SP18 v2 Phase 0 Task 0.2 (2026-05-08) — B-leg TD-error magnitude
|
||||
/// EMA readback.
|
||||
///
|
||||
/// Convenience wrapper: launches the producer, syncs the stream,
|
||||
/// returns the post-blend ISV slot 493 value. Cold-path only
|
||||
/// (HEALTH_DIAG cadence).
|
||||
pub fn read_sp18_td_error_mag_ema(&self) -> Result<f32, MLError> {
|
||||
use super::sp14_isv_slots::TD_ERROR_MAG_EMA_INDEX;
|
||||
self.launch_sp18_td_error_mag_ema_update()?;
|
||||
self.stream
|
||||
.synchronize()
|
||||
.map_err(|e| MLError::ModelError(format!(
|
||||
"sp18 td_error_mag_ema sync: {e}"
|
||||
)))?;
|
||||
Ok(self.read_isv_signal_at(TD_ERROR_MAG_EMA_INDEX))
|
||||
}
|
||||
|
||||
/// SP18 v2 Phase 0 Task 0.1 (2026-05-08) — D-leg per-action reward
|
||||
/// decomposition diagnostic device pointer.
|
||||
///
|
||||
@@ -20620,6 +20717,20 @@ impl GpuDqnTrainer {
|
||||
let sp17_advantage_clip_bound_kernel = sp17_advantage_clip_bound_module
|
||||
.load_function("sp17_advantage_clip_bound_update")
|
||||
.map_err(|e| MLError::ModelError(format!("sp17_advantage_clip_bound_update: {e}")))?;
|
||||
|
||||
// SP18 v2 Phase 0 Task 0.2 (2026-05-08): B-leg TD-error
|
||||
// magnitude EMA producer load. Single block × 256 threads.
|
||||
// Pearl-A bootstrap on slot 493 (sentinel 0.0) + fixed
|
||||
// α=WELFORD_ALPHA_MIN=0.4. Reads `td_errors_buf [B]` post-
|
||||
// train-step. Pure observability.
|
||||
let sp18_td_error_mag_ema_module = stream.context()
|
||||
.load_cubin(TD_ERROR_MAG_EMA_CUBIN.to_vec())
|
||||
.map_err(|e| MLError::ModelError(format!(
|
||||
"sp18 td_error_mag_ema cubin: {e}")))?;
|
||||
let sp18_td_error_mag_ema_kernel = sp18_td_error_mag_ema_module
|
||||
.load_function("td_error_mag_ema_update")
|
||||
.map_err(|e| MLError::ModelError(format!(
|
||||
"td_error_mag_ema_update load: {e}")))?;
|
||||
let sp17_clip_scratch_capacity = config.batch_size
|
||||
.saturating_mul(
|
||||
config.branch_0_size + config.branch_1_size
|
||||
@@ -24901,6 +25012,11 @@ impl GpuDqnTrainer {
|
||||
sp17_v_share_kernel,
|
||||
sp17_advantage_clip_bound_kernel,
|
||||
sp17_advantage_clip_scratch,
|
||||
// SP18 v2 Phase 0 Task 0.2 (2026-05-08): B-leg TD-error
|
||||
// magnitude EMA producer kernel handle. Trainer-stream-
|
||||
// resident; launched from `launch_sp18_td_error_mag_ema_
|
||||
// update` post-train-step.
|
||||
sp18_td_error_mag_ema_kernel,
|
||||
// SP18 v2 Phase 0 Task 0.1 (2026-05-08): D-leg per-action
|
||||
// reward decomposition diagnostic buffer. Mapped-pinned 20
|
||||
// floats (4 bins × 5 stats). Written by collector kernel,
|
||||
|
||||
Reference in New Issue
Block a user