gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(sp14): B.2 — register 11 SP14 ISV slots for fold-boundary reset

Browse Source 
Each EGF pearl EMA / state slot resets to its Pearl-A sentinel at fold
boundary, mirroring sp13_aux_dir_acc_short_ema / long_ema entries.
Atomic refactor (feedback_no_partial_refactor): both halves land
together — registry entry + reset_named_state dispatch arm.

Reset slots (11 total, sentinel in parens):
  - Q_DISAGREEMENT_SHORT/LONG_EMA (slots 383, 384) → 0.5
  - K_AUX_ADAPTIVE (385) → K_BASE_AUX = 20.0
  - K_Q_ADAPTIVE (386) → K_BASE_Q = 15.0
  - BETA_RATE_LIMITER_ADAPTIVE (387) → BETA_BASE = 0.5
  - AUX_DIR_ACC_VARIANCE_EMA, Q_DISAGREEMENT_VARIANCE_EMA,
    ALPHA_GRAD_RAW_VARIANCE_EMA (388, 389, 390) → 0.0
    (initial k = k_base, β = β_base via ISV-driven controllers)
  - GATE1_OPEN_STATE (391) → 0.0 (closed)
  - ALPHA_GRAD_SMOOTHED (393) → 0.0
  - AUX_DIR_ACC_POST_OPEN_MIN (394) → 1.0 (no min observed)

ALPHA_GRAD_RAW (slot 392, recomputed every step from variance EMAs)
and GRADIENT_HACK_LOCKOUT_REMAINING (slot 395, decays at epoch
boundary) are NOT in the fold-reset registry; both naturally
re-initialise without explicit reset.

Also corrects the isv-slots.md SP14 table: slots 392 and 395 were
incorrectly marked FoldReset in the B.1 entry; corrected to reflect
their actual reset semantics (NOT reset / epoch-boundary decay).

Producer + consumer wiring lands in subsequent tasks (B.3-B.12);
this commit is additive infrastructure only — no behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-05 18:51:00 +02:00
parent d63cb7992e
commit 84de278dfe
3 changed files with 204 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

67
crates/ml/src/trainers/dqn/state_reset_registry.rs
View File

@@ -938,6 +938,73 @@ impl StateResetRegistry {
category: ResetCategory::FoldReset,
description: "ISV[HOLD_RATE_OBSERVED_EMA_INDEX=382] — SP13 v3 P0a.T3 per-fold EMA of the per-step Hold-pick rate `count(action_dir == DIR_HOLD) / batch_size`. Produced by `apply_fixed_alpha_ema_kernel` chained off `hold_rate_observer_kernel` (added in P0a.T3). Consumed by the host-side Hold-cost controller in `training_loop.rs`: `excess = max(0, observed − target); hold_cost = HOLD_COST_BASE × (1 + 5 × excess)` clamped to `[0.5×base, 5.0×base]`, written to ISV[HOLD_COST_INDEX=380]. The reward-composition site in `experience_kernels.cu` then subtracts the cost on every Hold-action bar — pricing the action so the policy uses Hold deliberately rather than as a free CQL-bias-anchored default. FoldReset sentinel 0.0 so Pearl A bootstraps from the new fold's first observation; the controller relaxes back to baseline cost (HOLD_COST_BASE × 0.5 floor) when observed ≤ target. Spec §'Change 1: Price Hold (replaces v2's Eliminate Hold)'.",
},
// ── SP14 fold-reset slots [383..396) ─────────────────────────────
// 11 of the 13 allocated slots reset at fold boundary; the two
// exceptions are:
// - ALPHA_GRAD_RAW_INDEX=392: recomputed every step from the
// three Welford variance EMAs (388, 389, 390); no stale value
// persists past the first step of a new fold.
// - GRADIENT_HACK_LOCKOUT_REMAINING_INDEX=395: epoch-boundary
// decay; the lockout counter naturally reaches 0 at the start
// of any new epoch/fold and must NOT be reset here (doing so
// would silently abort an active lockout mid-fold).
// Both naturally re-initialise without explicit reset.
// Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md
RegistryEntry {
name: "sp14_q_disagreement_short_ema",
category: ResetCategory::FoldReset,
description: "ISV[Q_DISAGREEMENT_SHORT_EMA_INDEX=383] — SP14 EGF Gate 2 fast EMA (α=0.3) of per-step K=4↔K=2 mapped argmax disagreement with Hold/Flat masked. Produced by `q_disagreement_update_kernel` (B.3) every training step. Consumed by `alpha_grad_compute_kernel` (B.4) as the Gate 2 sigmoid driver. FoldReset sentinel SENTINEL_Q_DISAGREEMENT=0.5 (analytic random alignment under masking) so Pearl A bootstraps from the new fold's first observation per `pearl_first_observation_bootstrap.md` — the EMA tracks the new fold's Q-head signal without bias from the previous fold's terminal value. Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_q_disagreement_long_ema",
category: ResetCategory::FoldReset,
description: "ISV[Q_DISAGREEMENT_LONG_EMA_INDEX=384] — SP14 EGF Gate 2 slow EMA (α=0.05) of per-step K=4↔K=2 mapped argmax disagreement with Hold/Flat masked. Produced by `q_disagreement_update_kernel` (B.3) every training step. Used alongside the short EMA for stagnation detection in the Gate 2 controller (persistent high disagreement despite training ↔ Q head stuck, not just noisy). FoldReset sentinel SENTINEL_Q_DISAGREEMENT=0.5 so Pearl A bootstraps from the new fold's first observation per `pearl_first_observation_bootstrap.md`. Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_k_aux_adaptive",
category: ResetCategory::FoldReset,
description: "ISV[K_AUX_ADAPTIVE_INDEX=385] — SP14 ISV-driven adaptive sigmoid steepness for Gate 1 (aux competence). Driven by AUX_DIR_ACC_VARIANCE_EMA_INDEX=388 via `k_aux = K_BASE_AUX × exp(-variance / VARIANCE_REF_AUX)` clamped to [K_MIN, K_BASE_AUX]; high aux variance → flatter sigmoid → more cautious gate opening. FoldReset sentinel K_BASE_AUX=20.0 (Invariant-1 anchor: start each fold with maximum steepness, decay only if the new fold's signal is noisy). Produced by `alpha_grad_compute_kernel` (B.4); consumed by Gate 1 sigmoid. Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_k_q_adaptive",
category: ResetCategory::FoldReset,
description: "ISV[K_Q_ADAPTIVE_INDEX=386] — SP14 ISV-driven adaptive sigmoid steepness for Gate 2 (Q-head disagreement). Driven by Q_DISAGREEMENT_VARIANCE_EMA_INDEX=389 via `k_q = K_BASE_Q × exp(-variance / VARIANCE_REF_Q)` clamped to [K_MIN, K_BASE_Q]; high Q-disagreement variance → flatter sigmoid → more cautious gate modulation. FoldReset sentinel K_BASE_Q=15.0 (Invariant-1 anchor: start each fold with maximum steepness). Produced by `alpha_grad_compute_kernel` (B.4); consumed by Gate 2 sigmoid. Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_beta_rate_limiter_adaptive",
category: ResetCategory::FoldReset,
description: "ISV[BETA_RATE_LIMITER_ADAPTIVE_INDEX=387] — SP14 ISV-driven adaptive rate-limiter β for the α_grad_smoothed EMA. Driven by ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX=390 via `β = BETA_BASE + (BETA_MAX − BETA_BASE) × (1 − exp(-variance / VARIANCE_REF_ALPHA))` so volatile α_grad_raw → heavier smoothing (β → BETA_MAX=0.95) and stable signal → light smoothing (β → BETA_BASE=0.5). FoldReset sentinel BETA_BASE=0.5 (Invariant-1 anchor: start each fold with light smoothing). Produced by `alpha_grad_compute_kernel` (B.4); consumed by the α_grad_smoothed EMA update. Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_aux_dir_acc_variance_ema",
category: ResetCategory::FoldReset,
description: "ISV[AUX_DIR_ACC_VARIANCE_EMA_INDEX=388] — SP14 Welford variance EMA of per-step aux directional accuracy (Welford one-pass, α=0.05). Drives K_AUX_ADAPTIVE_INDEX=385: low variance → high k_aux (confident gate), high variance → low k_aux (cautious gate). FoldReset sentinel SENTINEL_VARIANCE=0.0 so that k_aux initialises at K_BASE_AUX=20.0 (maximum steepness) at the start of each fold and decays only if the new fold's aux-acc signal is genuinely noisy. Produced by `aux_variance_update_kernel` (B.3). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_q_disagreement_variance_ema",
category: ResetCategory::FoldReset,
description: "ISV[Q_DISAGREEMENT_VARIANCE_EMA_INDEX=389] — SP14 Welford variance EMA of per-step Q-disagreement signal (Welford one-pass, α=0.05). Drives K_Q_ADAPTIVE_INDEX=386: low variance → high k_q (confident Gate 2 modulation), high variance → low k_q (cautious). FoldReset sentinel SENTINEL_VARIANCE=0.0 so that k_q initialises at K_BASE_Q=15.0 at the start of each fold. Produced by `q_disagreement_update_kernel` (B.3). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_alpha_grad_raw_variance_ema",
category: ResetCategory::FoldReset,
description: "ISV[ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX=390] — SP14 Welford variance EMA of per-step α_grad_raw (Welford one-pass, α=0.05). Drives BETA_RATE_LIMITER_ADAPTIVE_INDEX=387: low α_grad_raw variance → light smoothing (β → BETA_BASE=0.5, responsive), high variance → heavy smoothing (β → BETA_MAX=0.95, stable). FoldReset sentinel SENTINEL_VARIANCE=0.0 so that β initialises at BETA_BASE=0.5 at the start of each fold. Produced by `alpha_grad_compute_kernel` (B.4). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_gate1_open_state",
category: ResetCategory::FoldReset,
description: "ISV[GATE1_OPEN_STATE_INDEX=391] — SP14 Schmitt-trigger persistent state for Gate 1 (0.0=closed, 1.0=open). Hysteresis: open when aux_dir_acc crosses target+SCHMITT_BAND=0.03, close when it drops below target-SCHMITT_BAND. Prevents rapid open/close cycling near the threshold. FoldReset sentinel SENTINEL_GATE1_CLOSED=0.0 (gate starts closed at each fold boundary; Gate 1 must re-earn the right to open by demonstrating competence on new fold data before α_grad can be non-zero). Produced and consumed by `alpha_grad_compute_kernel` (B.4). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_alpha_grad_smoothed",
category: ResetCategory::FoldReset,
description: "ISV[ALPHA_GRAD_SMOOTHED_INDEX=393] — SP14 rate-limited (β-smoothed) α_grad, the actual scalar consumed by the backward path to modulate aux→Q gradient flow. Updated each step as `α_smoothed = β × α_smoothed + (1−β) × α_raw` where β = BETA_RATE_LIMITER_ADAPTIVE_INDEX=387. FoldReset sentinel SENTINEL_ALPHA_GRAD=0.0 (no gradient flow at fold start; the smoothed gate must re-ramp from zero so the policy's early fold exploration is not contaminated by the previous fold's terminal gate state). Produced by `alpha_grad_compute_kernel` (B.4); consumed by backward-pass gradient scaling (B.10). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
RegistryEntry {
name: "sp14_aux_dir_acc_post_open_min",
category: ResetCategory::FoldReset,
description: "ISV[AUX_DIR_ACC_POST_OPEN_MIN_INDEX=394] — SP14 anti-gradient-hacking circuit breaker: running minimum of aux_dir_acc since Gate 1 last opened. Initialised to 1.0 (no minimum observed) when Gate 1 opens; updated each step post-open as `min = fminf(min, current_aux_dir_acc)`. If `open_min − current_aux_dir_acc > LOCKOUT_TRIGGER_DROP=0.05` while q_disagreement also rises by >LOCKOUT_TRIGGER_DIS_RISE=0.10, the lockout circuit fires (B.5) and writes LOCKOUT_EPOCHS=2.0 to GRADIENT_HACK_LOCKOUT_REMAINING_INDEX=395. FoldReset sentinel SENTINEL_POST_OPEN_MIN=1.0 (no minimum observed at fold start; the circuit breaker must accumulate post-open observations before it can fire). Produced by `alpha_grad_compute_kernel` (B.4). Spec: docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md.",
},
// SP5 Task A1: Wiener-state companion reset. The wiener_state_buf
// covers SP4 (SP4_PRODUCER_COUNT=71 producers × 3 = 213 floats) +
// SP5 (SP5_PRODUCER_COUNT × 3 floats) = (71 + SP5_PRODUCER_COUNT) × 3

126
crates/ml/src/trainers/dqn/trainer/training_loop.rs
View File

@@ -7784,6 +7784,132 @@ impl DQNTrainer {
);
}
}
// SP14 (2026-05-05): EGF pearl fold-reset arms — 11 slots.
// Slots 392 (ALPHA_GRAD_RAW) and 395 (GRADIENT_HACK_LOCKOUT_REMAINING)
// are intentionally absent: raw is recomputed every step from the
// three Welford variance EMAs; lockout decays at epoch boundary and
// must NOT be force-reset mid-fold.
"sp14_q_disagreement_short_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
Q_DISAGREEMENT_SHORT_EMA_INDEX, SENTINEL_Q_DISAGREEMENT,
};
fused.trainer().write_isv_signal_at(
Q_DISAGREEMENT_SHORT_EMA_INDEX,
SENTINEL_Q_DISAGREEMENT,
);
}
}
"sp14_q_disagreement_long_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
Q_DISAGREEMENT_LONG_EMA_INDEX, SENTINEL_Q_DISAGREEMENT,
};
fused.trainer().write_isv_signal_at(
Q_DISAGREEMENT_LONG_EMA_INDEX,
SENTINEL_Q_DISAGREEMENT,
);
}
}
"sp14_k_aux_adaptive" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
K_AUX_ADAPTIVE_INDEX, K_BASE_AUX,
};
fused.trainer().write_isv_signal_at(
K_AUX_ADAPTIVE_INDEX,
K_BASE_AUX,
);
}
}
"sp14_k_q_adaptive" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
K_Q_ADAPTIVE_INDEX, K_BASE_Q,
};
fused.trainer().write_isv_signal_at(
K_Q_ADAPTIVE_INDEX,
K_BASE_Q,
);
}
}
"sp14_beta_rate_limiter_adaptive" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
BETA_RATE_LIMITER_ADAPTIVE_INDEX, BETA_BASE,
};
fused.trainer().write_isv_signal_at(
BETA_RATE_LIMITER_ADAPTIVE_INDEX,
BETA_BASE,
);
}
}
"sp14_aux_dir_acc_variance_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
AUX_DIR_ACC_VARIANCE_EMA_INDEX, SENTINEL_VARIANCE,
};
fused.trainer().write_isv_signal_at(
AUX_DIR_ACC_VARIANCE_EMA_INDEX,
SENTINEL_VARIANCE,
);
}
}
"sp14_q_disagreement_variance_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
Q_DISAGREEMENT_VARIANCE_EMA_INDEX, SENTINEL_VARIANCE,
};
fused.trainer().write_isv_signal_at(
Q_DISAGREEMENT_VARIANCE_EMA_INDEX,
SENTINEL_VARIANCE,
);
}
}
"sp14_alpha_grad_raw_variance_ema" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX, SENTINEL_VARIANCE,
};
fused.trainer().write_isv_signal_at(
ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX,
SENTINEL_VARIANCE,
);
}
}
"sp14_gate1_open_state" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
GATE1_OPEN_STATE_INDEX, SENTINEL_GATE1_CLOSED,
};
fused.trainer().write_isv_signal_at(
GATE1_OPEN_STATE_INDEX,
SENTINEL_GATE1_CLOSED,
);
}
}
"sp14_alpha_grad_smoothed" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
ALPHA_GRAD_SMOOTHED_INDEX, SENTINEL_ALPHA_GRAD,
};
fused.trainer().write_isv_signal_at(
ALPHA_GRAD_SMOOTHED_INDEX,
SENTINEL_ALPHA_GRAD,
);
}
}
"sp14_aux_dir_acc_post_open_min" => {
if let Some(ref fused) = self.fused_ctx {
use crate::cuda_pipeline::sp14_isv_slots::{
AUX_DIR_ACC_POST_OPEN_MIN_INDEX, SENTINEL_POST_OPEN_MIN,
};
fused.trainer().write_isv_signal_at(
AUX_DIR_ACC_POST_OPEN_MIN_INDEX,
SENTINEL_POST_OPEN_MIN,
);
}
}
// SP11 Task A2 (2026-05-04): novelty visit-count hash table
// reset arm — closes the A0 deferral per the registry entry's
// docstring. 1M f32 slots, zero-filled via mapped-pinned host

16
docs/isv-slots.md
View File

@@ -371,16 +371,22 @@ SP14 starts at 383 (shifted +2 from the original plan which documented
| 389 | `Q_DISAGREEMENT_VARIANCE_EMA_INDEX` | FoldReset (0.0) | Welford variance for k_q adaptation |
| 390 | `ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX` | FoldReset (0.0) | Welford variance for β adaptation |
| 391 | `GATE1_OPEN_STATE_INDEX` | FoldReset (0.0) | Schmitt-trigger state (0=closed, 1=open) |
| 392 | `ALPHA_GRAD_RAW_INDEX` | FoldReset (0.0) | Raw gate output (HEALTH_DIAG visibility) |
| 392 | `ALPHA_GRAD_RAW_INDEX` | NOT reset (recomputed every step) | Raw gate output (HEALTH_DIAG visibility) |
| 393 | `ALPHA_GRAD_SMOOTHED_INDEX` | FoldReset (0.0) | Rate-limited gate consumed by backward path |
| 394 | `AUX_DIR_ACC_POST_OPEN_MIN_INDEX` | FoldReset (1.0) | Post-open aux_dir_acc minimum (circuit breaker) |
| 395 | `GRADIENT_HACK_LOCKOUT_REMAINING_INDEX` | FoldReset (0.0) | Lockout steps remaining (anti-gradient-hacking) |
| 395 | `GRADIENT_HACK_LOCKOUT_REMAINING_INDEX` | NOT reset (epoch-boundary decay) | Lockout epochs remaining (anti-gradient-hacking) |
Slots 396-398 are conceptual buffer; no constants allocated.
All 13 slots are FoldReset per `pearl_first_observation_bootstrap.md`.
Producer + consumer wiring lands in B.2-B.12; B.1 is additive
infrastructure only (no behavior change at this commit).
11 of the 13 slots are FoldReset per `pearl_first_observation_bootstrap.md`.
Slots 392 (ALPHA_GRAD_RAW) and 395 (GRADIENT_HACK_LOCKOUT_REMAINING) are
intentionally excluded from fold-reset: raw is recomputed every step from
the three Welford variance EMAs; lockout decays at epoch boundary and must
not be force-reset mid-fold.
Producer + consumer wiring lands in B.3-B.12; B.1 is additive
infrastructure (slot constants), B.2 wires the 11 fold-reset registry
entries + dispatch arms — both are additive only (no behavior change).
**Spec:** `docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md`
**Plan:** `docs/superpowers/plans/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md`