From c0fc28e4558243c5d093bfc5542e08b6d09fba88 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 5 May 2026 22:07:55 +0200 Subject: [PATCH] =?UTF-8?q?fix(sp14):=20delete=20warmup=5Fgate=20=E2=80=94?= =?UTF-8?q?=20let=20variance-driven=20k=5Faux/k=5Fq=20handle=20warmup=20(I?= =?UTF-8?q?SV-driven)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per `feedback_isv_for_adaptive_bounds`, the hardcoded `warmup_gate = (fold_step_counter / WARMUP_STEPS_FALLBACK).min(1.0)` ramp violated the rule: adaptive bounds in ISV, never hardcoded constants. The variance-driven k_aux/k_q sigmoid steepness already provides warmup behavior intrinsically: - High variance (cold-start, EMAs still moving) → k → K_MIN → flat sigmoid → gate ≈ 0.5 regardless of input. That IS the warmup. - Low variance (settled) → k → K_BASE → sharp sigmoid → gates respond correctly to driver signals. Adding a separate hardcoded step-counter multiplier on top was double-counting + tuning-driven (the 1000-step threshold had no principled basis). Removed entirely. Removed (per `feedback_no_partial_refactor`, all atomically): - `WARMUP_STEPS_FALLBACK` constant in `sp14_isv_slots.rs` - `warmup_gate: f32` parameter in `alpha_grad_compute_kernel.cu` - `gate1 * gate2 * warmup_gate` → `gate1 * gate2` in kernel - `warmup_gate` arg from `launch_sp14_alpha_grad_compute` - `fold_step_counter: usize` field on the trainer struct - `fold_step_counter = 0` reset in `reset_for_fold` - `fold_step_counter` init in trainer constructor - `let warmup_gate: f32 = 1.0;` and `.arg(&warmup_gate)` from B.4 oracle tests (4 launches: 2 in alpha_grad_schmitt_hysteresis, 20 in alpha_grad_adaptive_beta loop) Build: clean, 18 warnings (pre-existing baseline). Tests: cargo test --no-run on sp14_oracle_tests succeeds. Net result: EGF gate's warmup behavior now lives entirely in the variance-driven k_aux/k_q sigmoid steepness controller (ISV slots 388/var_aux, 389/var_q). No hardcoded step counter. Honors `feedback_isv_for_adaptive_bounds` and `pearl_controller_anchors_isv_driven`. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../alpha_grad_compute_kernel.cu | 28 +++++++++++++------ .../ml/src/cuda_pipeline/gpu_dqn_trainer.rs | 8 +++--- crates/ml/src/cuda_pipeline/sp14_isv_slots.rs | 11 ++++++-- .../src/trainers/dqn/trainer/constructor.rs | 6 ++-- crates/ml/src/trainers/dqn/trainer/mod.rs | 25 ++++++----------- .../src/trainers/dqn/trainer/training_loop.rs | 19 ++++++------- crates/ml/tests/sp14_oracle_tests.rs | 7 ----- docs/isv-slots.md | 16 +++++++++++ 8 files changed, 66 insertions(+), 54 deletions(-) diff --git a/crates/ml/src/cuda_pipeline/alpha_grad_compute_kernel.cu b/crates/ml/src/cuda_pipeline/alpha_grad_compute_kernel.cu index 235f7c2bd..71cf72215 100644 --- a/crates/ml/src/cuda_pipeline/alpha_grad_compute_kernel.cu +++ b/crates/ml/src/cuda_pipeline/alpha_grad_compute_kernel.cu @@ -39,7 +39,7 @@ // ISV[VAR_AUX = 388] updated aux_dir_acc variance EMA (SP14 B.11) // ISV[VAR_ALPHA = 390] updated α_grad_raw variance EMA // ISV[GATE1_OPEN = 391] updated Schmitt state (0 = closed, 1 = open) -// ISV[ALPHA_RAW = 392] α_grad_raw = gate1 × gate2 × warmup_gate +// ISV[ALPHA_RAW = 392] α_grad_raw = gate1 × gate2 (warmup_gate removed // ISV[ALPHA_SMOOTHED = 393] α_grad_smoothed = β × prev + (1-β) × raw // // ── Slot indices: must match `sp14_isv_slots.rs` ──────────────────────── @@ -64,7 +64,7 @@ // is invisible to the gradient flow it modulates. // // ── Bounded modifier per pearl_bounded_modifier_outputs_require_… ─────── -// α_grad_raw = sigmoid(·) × sigmoid(·) × warmup_gate is structurally +// α_grad_raw = sigmoid(·) × sigmoid(·) is structurally // bounded to [0, 1] — the two sigmoids guarantee [0, 1] each, the warmup // gate is supplied in [0, 1] by the host. No runtime clamp needed. Per // `pearl_bounded_modifier_outputs_require_structural_activation.md`. @@ -104,15 +104,17 @@ void alpha_grad_compute_kernel( * as the var_aux mean reference). All other slots untouched. */ float* __restrict__ isv, - /* Per-epoch warmup ramp value in [0, 1]. Host-computed (B.7+ wires - * this from the trainer's per-epoch step counter against the - * configured WARMUP_STEPS). 0.0 during cold-start (no gradient - * flow); 1.0 once warmup completes (gates fully control the flow). - */ - float warmup_gate, /* EMA decay coefficient for the α_grad_raw variance update (typical * 0.05; matches the `alpha_var` argument to * `q_disagreement_update_kernel`). + * + * SP14 L1+L2 follow-up (2026-05-05): the original `warmup_gate` + * parameter was deleted per `feedback_isv_for_adaptive_bounds`. The + * variance-driven k_aux/k_q already provide warmup behavior (high + * variance during cold-start collapses k → K_MIN → flat sigmoid → + * gate ≈ 0.5; once EMAs settle, k → K_BASE → sharp sigmoid → gates + * respond correctly). Adding a separate hardcoded step-counter + * warmup multiplier was double-counting and tuning-driven. */ float alpha_var) { @@ -218,7 +220,15 @@ void alpha_grad_compute_kernel( // gate1 ∈ [0, 1], gate2 ∈ [0, 1], warmup_gate ∈ [0, 1] → product is // structurally bounded to [0, 1]. No runtime clamp per // `pearl_bounded_modifier_outputs_require_structural_activation.md`. - const float alpha_raw = gate1 * gate2 * warmup_gate; + // SP14 L1+L2 follow-up: warmup_gate multiplier removed. Variance-driven + // k_aux/k_q (from var_aux, var_q EMAs) already provide warmup via the + // sigmoid steepness collapse during cold-start (high variance → flat + // sigmoid → gate ≈ 0.5 regardless of input). Per + // `feedback_isv_for_adaptive_bounds`, adaptive bounds live in ISV + // (here: variance EMAs); only Invariant-1 numerical-stability anchors + // are constants. The separate hardcoded WARMUP_STEPS_FALLBACK ramp was + // tuned, not signal-driven. + const float alpha_raw = gate1 * gate2; // ── Welford-style variance EMA on α_grad_raw ─────────────────────── // Reference is α_smoothed (the rate-limited tracking estimate); under diff --git a/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs b/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs index d708bd688..cddf16547 100644 --- a/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs +++ b/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs @@ -7263,8 +7263,10 @@ impl GpuDqnTrainer { /// (consumes slot 373 / 374). The launcher submits onto the trainer /// stream — same-stream submission ordering enforces these deps. /// - /// `warmup_gate ∈ [0, 1]` is the host-supplied per-epoch ramp factor - /// (see plan §2581: `clamp(steps_in_fold / WARMUP_STEPS, 0, 1)`). + /// SP14 L1+L2 follow-up (2026-05-05): the original `warmup_gate` parameter + /// was deleted per `feedback_isv_for_adaptive_bounds`. The variance-driven + /// k_aux/k_q already provide warmup behavior through ISV signals. + /// /// `alpha_var` is the Welford-EMA blend coefficient for both /// var_alpha (slot 390) and var_aux (slot 388, B.11 producer). /// @@ -7273,7 +7275,6 @@ impl GpuDqnTrainer { /// oracle test's launch shape. pub(crate) fn launch_sp14_alpha_grad_compute( &self, - warmup_gate: f32, alpha_var: f32, ) -> Result<(), MLError> { debug_assert!(self.isv_signals_dev_ptr != 0, @@ -7283,7 +7284,6 @@ impl GpuDqnTrainer { self.stream .launch_builder(&self.sp14_alpha_grad_compute_kernel) .arg(&isv_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), diff --git a/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs b/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs index dfd0a2434..15d28abb7 100644 --- a/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs +++ b/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs @@ -79,9 +79,14 @@ pub const BETA_MAX: f32 = 0.95; /// Schmitt-trigger hysteresis half-band: open at target+0.03, close at target-0.03. pub const SCHMITT_BAND: f32 = 0.03; -/// Per-epoch warmup ramp duration (steps). Set at runtime from config; this -/// constant is the FALLBACK if config lookup fails. -pub const WARMUP_STEPS_FALLBACK: usize = 1000; +// SP14 L1+L2 follow-up (2026-05-05): WARMUP_STEPS_FALLBACK constant deleted +// per `feedback_isv_for_adaptive_bounds`. The previous design had +// `warmup_gate = (steps_in_fold / WARMUP_STEPS_FALLBACK).min(1.0)` multiplied +// into α_raw — a hardcoded step-counter ramp. The variance-driven k_aux/k_q +// adaptive sigmoid steepness already provides warmup behavior (high variance +// during cold-start collapses k → K_MIN → flat sigmoid → gate ≈ 0.5; once +// EMAs settle, k → K_BASE → sharp sigmoid → gates respond correctly). +// Adding a separate hardcoded warmup multiplier was double-counting. /// Anti-gradient-hacking circuit breaker: aux_dir_acc must drop more than /// LOCKOUT_TRIGGER_DROP below open-threshold to suspect hacking. diff --git a/crates/ml/src/trainers/dqn/trainer/constructor.rs b/crates/ml/src/trainers/dqn/trainer/constructor.rs index 84dd7011e..ade03e076 100644 --- a/crates/ml/src/trainers/dqn/trainer/constructor.rs +++ b/crates/ml/src/trainers/dqn/trainer/constructor.rs @@ -608,9 +608,9 @@ impl DQNTrainer { // Wave 16 Portfolio Features (action masking always active) max_position, current_epoch: 0, - // SP14 B.11 (2026-05-05): EGF warmup-gate counter; reset by - // reset_for_fold, incremented per-step in run_training_steps_slices. - fold_step_counter: 0, + // SP14 L1+L2 follow-up (2026-05-05): fold_step_counter deleted + // per feedback_isv_for_adaptive_bounds; variance-driven k_aux/k_q + // provide warmup intrinsically. multi_asset_portfolio, stress_tester, diff --git a/crates/ml/src/trainers/dqn/trainer/mod.rs b/crates/ml/src/trainers/dqn/trainer/mod.rs index 71489d7d6..b3687aa0a 100644 --- a/crates/ml/src/trainers/dqn/trainer/mod.rs +++ b/crates/ml/src/trainers/dqn/trainer/mod.rs @@ -436,19 +436,12 @@ pub struct DQNTrainer { /// Current epoch (for Q-gap warm-up ramp) pub(crate) current_epoch: usize, - /// SP14 B.11 (2026-05-05): per-fold cumulative training-step counter - /// driving the EGF α_grad warmup gate. Reset to 0 in `reset_for_fold`; - /// incremented every iteration of `run_training_steps_slices`'s - /// per-step body so it accumulates across epochs within a fold. - /// `warmup_gate = clamp(fold_step_counter / WARMUP_STEPS_FALLBACK, 0, 1)` is - /// passed to `launch_sp14_alpha_grad_compute` per - /// `crate::cuda_pipeline::sp14_isv_slots::WARMUP_STEPS_FALLBACK`. The - /// gate ramps from 0 (cold-start, no EGF gradient flow) to 1 (warmup - /// complete, gates fully control flow) over `WARMUP_STEPS_FALLBACK` - /// per-step launches. Per `feedback_no_partial_refactor`, the trainer - /// owns this counter rather than fishing it out of the training-loop - /// async closure scope. - pub(crate) fold_step_counter: usize, + // SP14 L1+L2 follow-up (2026-05-05): `fold_step_counter` field deleted + // per `feedback_isv_for_adaptive_bounds`. Was driving the + // `warmup_gate = clamp(fold_step_counter / WARMUP_STEPS_FALLBACK, 0, 1)` + // ramp passed to `launch_sp14_alpha_grad_compute`. Both the counter + // and the warmup_gate parameter are gone — variance-driven k_aux/k_q + // (ISV[388, 389]) provide warmup behavior intrinsically. /// Entropy regularizer for preventing policy collapse (None if disabled) // entropy_regularizer removed — SAC-style entropy is computed directly on Q-value tensors in DQN::compute_loss_internal /// Multi-asset portfolio tracker (None if single-asset mode) @@ -1790,10 +1783,8 @@ impl DQNTrainer { // SP14 B.11 (2026-05-05): EGF warmup-gate counter resets at fold // boundary so the new fold cold-starts with α_grad force-closed // and ramps over WARMUP_STEPS_FALLBACK steps. Without this, fold N - // would inherit fold N-1's saturated warmup gate and the EGF wire - // would be hot from step 0 — defeating the cold-start safety - // contract documented in `sp14_isv_slots::WARMUP_STEPS_FALLBACK`. - self.fold_step_counter = 0; + // SP14 L1+L2 follow-up: fold_step_counter reset removed (counter + // deleted; variance-driven k_aux/k_q provide warmup intrinsically). // Reset loss/Q-value history self.loss_history.clear(); diff --git a/crates/ml/src/trainers/dqn/trainer/training_loop.rs b/crates/ml/src/trainers/dqn/trainer/training_loop.rs index cfbecf691..65d5a365e 100644 --- a/crates/ml/src/trainers/dqn/trainer/training_loop.rs +++ b/crates/ml/src/trainers/dqn/trainer/training_loop.rs @@ -4116,10 +4116,13 @@ impl DQNTrainer { // is still 0.0 → wire column zeroed; from step 1 onward the // gate value accrues. // - // Fold-cold-start contract: `fold_step_counter` is reset to 0 - // in `reset_for_fold`, ensuring each fold's first step sees - // `warmup_gate = 0` (closed gate) and ramps over - // `WARMUP_STEPS_FALLBACK = 1000` steps to 1. + // SP14 L1+L2 follow-up (2026-05-05): warmup_gate parameter + // and `fold_step_counter` ramp deleted per + // `feedback_isv_for_adaptive_bounds`. The variance-driven + // k_aux/k_q already provide warmup behavior — when var_aux + // and var_q EMAs are high (cold-start, EMAs still moving), + // k → K_MIN → flat sigmoid → gate ≈ 0.5; when EMAs settle, + // k → K_BASE → sharp sigmoid → gates respond correctly. // // Constants per plan §2581-2585: // * α_short = 0.3 (fast EMA, ~3-step half-life) @@ -4133,7 +4136,6 @@ impl DQNTrainer { // (Welford on aux_short vs aux_long), so the adaptive k_aux // branch is no longer degenerate at K_BASE_AUX = 20. if let Some(ref fused) = self.fused_ctx { - use crate::cuda_pipeline::sp14_isv_slots::WARMUP_STEPS_FALLBACK; let alpha_short_qd: f32 = 0.3; let alpha_long_qd: f32 = 0.05; let alpha_var: f32 = 0.05; @@ -4142,15 +4144,10 @@ impl DQNTrainer { ) { tracing::warn!("SP14 B.11 q_disagreement_update launch failed: {e}"); } - let warmup_gate = ((self.fold_step_counter as f32) - / (WARMUP_STEPS_FALLBACK as f32)).min(1.0); - if let Err(e) = fused.trainer().launch_sp14_alpha_grad_compute( - warmup_gate, alpha_var, - ) { + if let Err(e) = fused.trainer().launch_sp14_alpha_grad_compute(alpha_var) { tracing::warn!("SP14 B.11 alpha_grad_compute launch failed: {e}"); } } - self.fold_step_counter += 1; // Phase 3 T3.5: MoE expert-utilisation EMA + gate entropy EMA. // Reads `moe_gate_softmax_buf [B, K]` (valid after the captured diff --git a/crates/ml/tests/sp14_oracle_tests.rs b/crates/ml/tests/sp14_oracle_tests.rs index 1f96a8934..e2f6926c0 100644 --- a/crates/ml/tests/sp14_oracle_tests.rs +++ b/crates/ml/tests/sp14_oracle_tests.rs @@ -393,7 +393,6 @@ mod gpu { .expect("alloc isv buffer"); isv_buf.write_from_slice(&isv); - let warmup_gate: f32 = 1.0; let alpha_var: f32 = 0.05; // ── Step 1: aux_dir_acc = 0.55 < threshold_open = 0.58 → gate stays closed ── @@ -401,7 +400,6 @@ mod gpu { stream .launch_builder(&kernel) .arg(&isv_buf.dev_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), @@ -437,7 +435,6 @@ mod gpu { stream .launch_builder(&kernel) .arg(&isv_buf.dev_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), @@ -469,7 +466,6 @@ mod gpu { stream .launch_builder(&kernel) .arg(&isv_buf.dev_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), @@ -496,7 +492,6 @@ mod gpu { stream .launch_builder(&kernel) .arg(&isv_buf.dev_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), @@ -575,14 +570,12 @@ mod gpu { current[AUX_DIR_ACC_SHORT_EMA_INDEX] = acc; isv_buf.write_from_slice(¤t); - let warmup_gate: f32 = 1.0; let alpha_var: f32 = 0.05; unsafe { stream .launch_builder(&kernel) .arg(&isv_buf.dev_ptr) - .arg(&warmup_gate) .arg(&alpha_var) .launch(LaunchConfig { grid_dim: (1, 1, 1), diff --git a/docs/isv-slots.md b/docs/isv-slots.md index 0b9f3f7b6..2bc6f0a9c 100644 --- a/docs/isv-slots.md +++ b/docs/isv-slots.md @@ -410,3 +410,19 @@ values are omitted from the emit (recomputing host-side would violate **Why this slipped through:** SP4/SP5 had analogous regression tests (`all_sp4_slots_fit_within_isv_total_dim`); SP14's was missing. The bug was silent because OOB GPU reads return zeros (or adjacent-allocation contents) instead of crashing — sentinel values that look almost-plausible. **Open follow-up (separate from L1/L2):** `warmup_gate` uses hardcoded `WARMUP_STEPS_FALLBACK=1000` constant. Per `feedback_isv_for_adaptive_bounds`, this should be ISV-signal-driven (variance-driven) OR removed entirely since k_aux/k_q already provide variance-driven warmup behavior. Redesign post-re-smoke once bus-size fix is verified. + +## SP14 L1+L2 follow-up — warmup_gate deletion (2026-05-05, atomic with bus-fix) + +**Resolution of the open follow-up above:** Removed `warmup_gate` entirely instead of redesigning ISV-driven. The variance-driven k_aux/k_q already provide warmup behavior — when var_aux/var_q are high (cold-start), `k = K_BASE / (1 + var/VARIANCE_REF)` collapses toward `K_MIN`, which makes the sigmoid flat → gate ≈ 0.5 regardless of input. Once EMAs settle, k → K_BASE → sharp sigmoid → gates respond correctly. The separate hardcoded WARMUP_STEPS_FALLBACK ramp was double-counting and tuning-driven. + +**Removed:** +- `WARMUP_STEPS_FALLBACK` constant in `sp14_isv_slots.rs` +- `warmup_gate: f32` parameter in `alpha_grad_compute_kernel.cu` +- `gate1 * gate2 * warmup_gate` → `gate1 * gate2` in kernel +- `warmup_gate` argument from `launch_sp14_alpha_grad_compute` launcher +- `fold_step_counter: usize` field on the trainer struct (was the ramp driver) +- `fold_step_counter = 0` reset in `reset_for_fold` +- `fold_step_counter` init in trainer constructor +- `let warmup_gate: f32 = 1.0;` and `.arg(&warmup_gate)` from B.4 oracle tests + +**Net result:** EGF gate's warmup behavior now lives entirely in the variance-driven k_aux/k_q sigmoid steepness controller. ISV-signal-driven via slots 388 (var_aux) and 389 (var_q). No hardcoded step counter. Honors `feedback_isv_for_adaptive_bounds`.