fix(sp14): delete warmup_gate — let variance-driven k_aux/k_q handle warmup (ISV-driven)
Per `feedback_isv_for_adaptive_bounds`, the hardcoded `warmup_gate = (fold_step_counter / WARMUP_STEPS_FALLBACK).min(1.0)` ramp violated the rule: adaptive bounds in ISV, never hardcoded constants. The variance-driven k_aux/k_q sigmoid steepness already provides warmup behavior intrinsically: - High variance (cold-start, EMAs still moving) → k → K_MIN → flat sigmoid → gate ≈ 0.5 regardless of input. That IS the warmup. - Low variance (settled) → k → K_BASE → sharp sigmoid → gates respond correctly to driver signals. Adding a separate hardcoded step-counter multiplier on top was double-counting + tuning-driven (the 1000-step threshold had no principled basis). Removed entirely. Removed (per `feedback_no_partial_refactor`, all atomically): - `WARMUP_STEPS_FALLBACK` constant in `sp14_isv_slots.rs` - `warmup_gate: f32` parameter in `alpha_grad_compute_kernel.cu` - `gate1 * gate2 * warmup_gate` → `gate1 * gate2` in kernel - `warmup_gate` arg from `launch_sp14_alpha_grad_compute` - `fold_step_counter: usize` field on the trainer struct - `fold_step_counter = 0` reset in `reset_for_fold` - `fold_step_counter` init in trainer constructor - `let warmup_gate: f32 = 1.0;` and `.arg(&warmup_gate)` from B.4 oracle tests (4 launches: 2 in alpha_grad_schmitt_hysteresis, 20 in alpha_grad_adaptive_beta loop) Build: clean, 18 warnings (pre-existing baseline). Tests: cargo test --no-run on sp14_oracle_tests succeeds. Net result: EGF gate's warmup behavior now lives entirely in the variance-driven k_aux/k_q sigmoid steepness controller (ISV slots 388/var_aux, 389/var_q). No hardcoded step counter. Honors `feedback_isv_for_adaptive_bounds` and `pearl_controller_anchors_isv_driven`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -39,7 +39,7 @@
|
||||
// ISV[VAR_AUX = 388] updated aux_dir_acc variance EMA (SP14 B.11)
|
||||
// ISV[VAR_ALPHA = 390] updated α_grad_raw variance EMA
|
||||
// ISV[GATE1_OPEN = 391] updated Schmitt state (0 = closed, 1 = open)
|
||||
// ISV[ALPHA_RAW = 392] α_grad_raw = gate1 × gate2 × warmup_gate
|
||||
// ISV[ALPHA_RAW = 392] α_grad_raw = gate1 × gate2 (warmup_gate removed
|
||||
// ISV[ALPHA_SMOOTHED = 393] α_grad_smoothed = β × prev + (1-β) × raw
|
||||
//
|
||||
// ── Slot indices: must match `sp14_isv_slots.rs` ────────────────────────
|
||||
@@ -64,7 +64,7 @@
|
||||
// is invisible to the gradient flow it modulates.
|
||||
//
|
||||
// ── Bounded modifier per pearl_bounded_modifier_outputs_require_… ───────
|
||||
// α_grad_raw = sigmoid(·) × sigmoid(·) × warmup_gate is structurally
|
||||
// α_grad_raw = sigmoid(·) × sigmoid(·) is structurally
|
||||
// bounded to [0, 1] — the two sigmoids guarantee [0, 1] each, the warmup
|
||||
// gate is supplied in [0, 1] by the host. No runtime clamp needed. Per
|
||||
// `pearl_bounded_modifier_outputs_require_structural_activation.md`.
|
||||
@@ -104,15 +104,17 @@ void alpha_grad_compute_kernel(
|
||||
* as the var_aux mean reference). All other slots untouched.
|
||||
*/
|
||||
float* __restrict__ isv,
|
||||
/* Per-epoch warmup ramp value in [0, 1]. Host-computed (B.7+ wires
|
||||
* this from the trainer's per-epoch step counter against the
|
||||
* configured WARMUP_STEPS). 0.0 during cold-start (no gradient
|
||||
* flow); 1.0 once warmup completes (gates fully control the flow).
|
||||
*/
|
||||
float warmup_gate,
|
||||
/* EMA decay coefficient for the α_grad_raw variance update (typical
|
||||
* 0.05; matches the `alpha_var` argument to
|
||||
* `q_disagreement_update_kernel`).
|
||||
*
|
||||
* SP14 L1+L2 follow-up (2026-05-05): the original `warmup_gate`
|
||||
* parameter was deleted per `feedback_isv_for_adaptive_bounds`. The
|
||||
* variance-driven k_aux/k_q already provide warmup behavior (high
|
||||
* variance during cold-start collapses k → K_MIN → flat sigmoid →
|
||||
* gate ≈ 0.5; once EMAs settle, k → K_BASE → sharp sigmoid → gates
|
||||
* respond correctly). Adding a separate hardcoded step-counter
|
||||
* warmup multiplier was double-counting and tuning-driven.
|
||||
*/
|
||||
float alpha_var)
|
||||
{
|
||||
@@ -218,7 +220,15 @@ void alpha_grad_compute_kernel(
|
||||
// gate1 ∈ [0, 1], gate2 ∈ [0, 1], warmup_gate ∈ [0, 1] → product is
|
||||
// structurally bounded to [0, 1]. No runtime clamp per
|
||||
// `pearl_bounded_modifier_outputs_require_structural_activation.md`.
|
||||
const float alpha_raw = gate1 * gate2 * warmup_gate;
|
||||
// SP14 L1+L2 follow-up: warmup_gate multiplier removed. Variance-driven
|
||||
// k_aux/k_q (from var_aux, var_q EMAs) already provide warmup via the
|
||||
// sigmoid steepness collapse during cold-start (high variance → flat
|
||||
// sigmoid → gate ≈ 0.5 regardless of input). Per
|
||||
// `feedback_isv_for_adaptive_bounds`, adaptive bounds live in ISV
|
||||
// (here: variance EMAs); only Invariant-1 numerical-stability anchors
|
||||
// are constants. The separate hardcoded WARMUP_STEPS_FALLBACK ramp was
|
||||
// tuned, not signal-driven.
|
||||
const float alpha_raw = gate1 * gate2;
|
||||
|
||||
// ── Welford-style variance EMA on α_grad_raw ───────────────────────
|
||||
// Reference is α_smoothed (the rate-limited tracking estimate); under
|
||||
|
||||
@@ -7263,8 +7263,10 @@ impl GpuDqnTrainer {
|
||||
/// (consumes slot 373 / 374). The launcher submits onto the trainer
|
||||
/// stream — same-stream submission ordering enforces these deps.
|
||||
///
|
||||
/// `warmup_gate ∈ [0, 1]` is the host-supplied per-epoch ramp factor
|
||||
/// (see plan §2581: `clamp(steps_in_fold / WARMUP_STEPS, 0, 1)`).
|
||||
/// SP14 L1+L2 follow-up (2026-05-05): the original `warmup_gate` parameter
|
||||
/// was deleted per `feedback_isv_for_adaptive_bounds`. The variance-driven
|
||||
/// k_aux/k_q already provide warmup behavior through ISV signals.
|
||||
///
|
||||
/// `alpha_var` is the Welford-EMA blend coefficient for both
|
||||
/// var_alpha (slot 390) and var_aux (slot 388, B.11 producer).
|
||||
///
|
||||
@@ -7273,7 +7275,6 @@ impl GpuDqnTrainer {
|
||||
/// oracle test's launch shape.
|
||||
pub(crate) fn launch_sp14_alpha_grad_compute(
|
||||
&self,
|
||||
warmup_gate: f32,
|
||||
alpha_var: f32,
|
||||
) -> Result<(), MLError> {
|
||||
debug_assert!(self.isv_signals_dev_ptr != 0,
|
||||
@@ -7283,7 +7284,6 @@ impl GpuDqnTrainer {
|
||||
self.stream
|
||||
.launch_builder(&self.sp14_alpha_grad_compute_kernel)
|
||||
.arg(&isv_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
|
||||
@@ -79,9 +79,14 @@ pub const BETA_MAX: f32 = 0.95;
|
||||
/// Schmitt-trigger hysteresis half-band: open at target+0.03, close at target-0.03.
|
||||
pub const SCHMITT_BAND: f32 = 0.03;
|
||||
|
||||
/// Per-epoch warmup ramp duration (steps). Set at runtime from config; this
|
||||
/// constant is the FALLBACK if config lookup fails.
|
||||
pub const WARMUP_STEPS_FALLBACK: usize = 1000;
|
||||
// SP14 L1+L2 follow-up (2026-05-05): WARMUP_STEPS_FALLBACK constant deleted
|
||||
// per `feedback_isv_for_adaptive_bounds`. The previous design had
|
||||
// `warmup_gate = (steps_in_fold / WARMUP_STEPS_FALLBACK).min(1.0)` multiplied
|
||||
// into α_raw — a hardcoded step-counter ramp. The variance-driven k_aux/k_q
|
||||
// adaptive sigmoid steepness already provides warmup behavior (high variance
|
||||
// during cold-start collapses k → K_MIN → flat sigmoid → gate ≈ 0.5; once
|
||||
// EMAs settle, k → K_BASE → sharp sigmoid → gates respond correctly).
|
||||
// Adding a separate hardcoded warmup multiplier was double-counting.
|
||||
|
||||
/// Anti-gradient-hacking circuit breaker: aux_dir_acc must drop more than
|
||||
/// LOCKOUT_TRIGGER_DROP below open-threshold to suspect hacking.
|
||||
|
||||
@@ -608,9 +608,9 @@ impl DQNTrainer {
|
||||
// Wave 16 Portfolio Features (action masking always active)
|
||||
max_position,
|
||||
current_epoch: 0,
|
||||
// SP14 B.11 (2026-05-05): EGF warmup-gate counter; reset by
|
||||
// reset_for_fold, incremented per-step in run_training_steps_slices.
|
||||
fold_step_counter: 0,
|
||||
// SP14 L1+L2 follow-up (2026-05-05): fold_step_counter deleted
|
||||
// per feedback_isv_for_adaptive_bounds; variance-driven k_aux/k_q
|
||||
// provide warmup intrinsically.
|
||||
multi_asset_portfolio,
|
||||
stress_tester,
|
||||
|
||||
|
||||
@@ -436,19 +436,12 @@ pub struct DQNTrainer {
|
||||
/// Current epoch (for Q-gap warm-up ramp)
|
||||
pub(crate) current_epoch: usize,
|
||||
|
||||
/// SP14 B.11 (2026-05-05): per-fold cumulative training-step counter
|
||||
/// driving the EGF α_grad warmup gate. Reset to 0 in `reset_for_fold`;
|
||||
/// incremented every iteration of `run_training_steps_slices`'s
|
||||
/// per-step body so it accumulates across epochs within a fold.
|
||||
/// `warmup_gate = clamp(fold_step_counter / WARMUP_STEPS_FALLBACK, 0, 1)` is
|
||||
/// passed to `launch_sp14_alpha_grad_compute` per
|
||||
/// `crate::cuda_pipeline::sp14_isv_slots::WARMUP_STEPS_FALLBACK`. The
|
||||
/// gate ramps from 0 (cold-start, no EGF gradient flow) to 1 (warmup
|
||||
/// complete, gates fully control flow) over `WARMUP_STEPS_FALLBACK`
|
||||
/// per-step launches. Per `feedback_no_partial_refactor`, the trainer
|
||||
/// owns this counter rather than fishing it out of the training-loop
|
||||
/// async closure scope.
|
||||
pub(crate) fold_step_counter: usize,
|
||||
// SP14 L1+L2 follow-up (2026-05-05): `fold_step_counter` field deleted
|
||||
// per `feedback_isv_for_adaptive_bounds`. Was driving the
|
||||
// `warmup_gate = clamp(fold_step_counter / WARMUP_STEPS_FALLBACK, 0, 1)`
|
||||
// ramp passed to `launch_sp14_alpha_grad_compute`. Both the counter
|
||||
// and the warmup_gate parameter are gone — variance-driven k_aux/k_q
|
||||
// (ISV[388, 389]) provide warmup behavior intrinsically.
|
||||
/// Entropy regularizer for preventing policy collapse (None if disabled)
|
||||
// entropy_regularizer removed — SAC-style entropy is computed directly on Q-value tensors in DQN::compute_loss_internal
|
||||
/// Multi-asset portfolio tracker (None if single-asset mode)
|
||||
@@ -1790,10 +1783,8 @@ impl DQNTrainer {
|
||||
// SP14 B.11 (2026-05-05): EGF warmup-gate counter resets at fold
|
||||
// boundary so the new fold cold-starts with α_grad force-closed
|
||||
// and ramps over WARMUP_STEPS_FALLBACK steps. Without this, fold N
|
||||
// would inherit fold N-1's saturated warmup gate and the EGF wire
|
||||
// would be hot from step 0 — defeating the cold-start safety
|
||||
// contract documented in `sp14_isv_slots::WARMUP_STEPS_FALLBACK`.
|
||||
self.fold_step_counter = 0;
|
||||
// SP14 L1+L2 follow-up: fold_step_counter reset removed (counter
|
||||
// deleted; variance-driven k_aux/k_q provide warmup intrinsically).
|
||||
|
||||
// Reset loss/Q-value history
|
||||
self.loss_history.clear();
|
||||
|
||||
@@ -4116,10 +4116,13 @@ impl DQNTrainer {
|
||||
// is still 0.0 → wire column zeroed; from step 1 onward the
|
||||
// gate value accrues.
|
||||
//
|
||||
// Fold-cold-start contract: `fold_step_counter` is reset to 0
|
||||
// in `reset_for_fold`, ensuring each fold's first step sees
|
||||
// `warmup_gate = 0` (closed gate) and ramps over
|
||||
// `WARMUP_STEPS_FALLBACK = 1000` steps to 1.
|
||||
// SP14 L1+L2 follow-up (2026-05-05): warmup_gate parameter
|
||||
// and `fold_step_counter` ramp deleted per
|
||||
// `feedback_isv_for_adaptive_bounds`. The variance-driven
|
||||
// k_aux/k_q already provide warmup behavior — when var_aux
|
||||
// and var_q EMAs are high (cold-start, EMAs still moving),
|
||||
// k → K_MIN → flat sigmoid → gate ≈ 0.5; when EMAs settle,
|
||||
// k → K_BASE → sharp sigmoid → gates respond correctly.
|
||||
//
|
||||
// Constants per plan §2581-2585:
|
||||
// * α_short = 0.3 (fast EMA, ~3-step half-life)
|
||||
@@ -4133,7 +4136,6 @@ impl DQNTrainer {
|
||||
// (Welford on aux_short vs aux_long), so the adaptive k_aux
|
||||
// branch is no longer degenerate at K_BASE_AUX = 20.
|
||||
if let Some(ref fused) = self.fused_ctx {
|
||||
use crate::cuda_pipeline::sp14_isv_slots::WARMUP_STEPS_FALLBACK;
|
||||
let alpha_short_qd: f32 = 0.3;
|
||||
let alpha_long_qd: f32 = 0.05;
|
||||
let alpha_var: f32 = 0.05;
|
||||
@@ -4142,15 +4144,10 @@ impl DQNTrainer {
|
||||
) {
|
||||
tracing::warn!("SP14 B.11 q_disagreement_update launch failed: {e}");
|
||||
}
|
||||
let warmup_gate = ((self.fold_step_counter as f32)
|
||||
/ (WARMUP_STEPS_FALLBACK as f32)).min(1.0);
|
||||
if let Err(e) = fused.trainer().launch_sp14_alpha_grad_compute(
|
||||
warmup_gate, alpha_var,
|
||||
) {
|
||||
if let Err(e) = fused.trainer().launch_sp14_alpha_grad_compute(alpha_var) {
|
||||
tracing::warn!("SP14 B.11 alpha_grad_compute launch failed: {e}");
|
||||
}
|
||||
}
|
||||
self.fold_step_counter += 1;
|
||||
|
||||
// Phase 3 T3.5: MoE expert-utilisation EMA + gate entropy EMA.
|
||||
// Reads `moe_gate_softmax_buf [B, K]` (valid after the captured
|
||||
|
||||
@@ -393,7 +393,6 @@ mod gpu {
|
||||
.expect("alloc isv buffer");
|
||||
isv_buf.write_from_slice(&isv);
|
||||
|
||||
let warmup_gate: f32 = 1.0;
|
||||
let alpha_var: f32 = 0.05;
|
||||
|
||||
// ── Step 1: aux_dir_acc = 0.55 < threshold_open = 0.58 → gate stays closed ──
|
||||
@@ -401,7 +400,6 @@ mod gpu {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
@@ -437,7 +435,6 @@ mod gpu {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
@@ -469,7 +466,6 @@ mod gpu {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
@@ -496,7 +492,6 @@ mod gpu {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
@@ -575,14 +570,12 @@ mod gpu {
|
||||
current[AUX_DIR_ACC_SHORT_EMA_INDEX] = acc;
|
||||
isv_buf.write_from_slice(¤t);
|
||||
|
||||
let warmup_gate: f32 = 1.0;
|
||||
let alpha_var: f32 = 0.05;
|
||||
|
||||
unsafe {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&warmup_gate)
|
||||
.arg(&alpha_var)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (1, 1, 1),
|
||||
|
||||
@@ -410,3 +410,19 @@ values are omitted from the emit (recomputing host-side would violate
|
||||
**Why this slipped through:** SP4/SP5 had analogous regression tests (`all_sp4_slots_fit_within_isv_total_dim`); SP14's was missing. The bug was silent because OOB GPU reads return zeros (or adjacent-allocation contents) instead of crashing — sentinel values that look almost-plausible.
|
||||
|
||||
**Open follow-up (separate from L1/L2):** `warmup_gate` uses hardcoded `WARMUP_STEPS_FALLBACK=1000` constant. Per `feedback_isv_for_adaptive_bounds`, this should be ISV-signal-driven (variance-driven) OR removed entirely since k_aux/k_q already provide variance-driven warmup behavior. Redesign post-re-smoke once bus-size fix is verified.
|
||||
|
||||
## SP14 L1+L2 follow-up — warmup_gate deletion (2026-05-05, atomic with bus-fix)
|
||||
|
||||
**Resolution of the open follow-up above:** Removed `warmup_gate` entirely instead of redesigning ISV-driven. The variance-driven k_aux/k_q already provide warmup behavior — when var_aux/var_q are high (cold-start), `k = K_BASE / (1 + var/VARIANCE_REF)` collapses toward `K_MIN`, which makes the sigmoid flat → gate ≈ 0.5 regardless of input. Once EMAs settle, k → K_BASE → sharp sigmoid → gates respond correctly. The separate hardcoded WARMUP_STEPS_FALLBACK ramp was double-counting and tuning-driven.
|
||||
|
||||
**Removed:**
|
||||
- `WARMUP_STEPS_FALLBACK` constant in `sp14_isv_slots.rs`
|
||||
- `warmup_gate: f32` parameter in `alpha_grad_compute_kernel.cu`
|
||||
- `gate1 * gate2 * warmup_gate` → `gate1 * gate2` in kernel
|
||||
- `warmup_gate` argument from `launch_sp14_alpha_grad_compute` launcher
|
||||
- `fold_step_counter: usize` field on the trainer struct (was the ramp driver)
|
||||
- `fold_step_counter = 0` reset in `reset_for_fold`
|
||||
- `fold_step_counter` init in trainer constructor
|
||||
- `let warmup_gate: f32 = 1.0;` and `.arg(&warmup_gate)` from B.4 oracle tests
|
||||
|
||||
**Net result:** EGF gate's warmup behavior now lives entirely in the variance-driven k_aux/k_q sigmoid steepness controller. ISV-signal-driven via slots 388 (var_aux) and 389 (var_q). No hardcoded step counter. Honors `feedback_isv_for_adaptive_bounds`.
|
||||
|
||||
Reference in New Issue
Block a user