From d243a6f0806268950bd5c2e3fc0c290aaf52d479 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 5 May 2026 17:11:04 +0200 Subject: [PATCH] =?UTF-8?q?docs(sp14):=20spec=20v2=20=E2=80=94=20critical?= =?UTF-8?q?=20review=20corrections?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 8 corrections from code-anchored critical review at HEAD eaf4adcb9: 1. Direction Q-head emits K=3 (Short/Flat/Long), not K=4. Aux head emits K=2 (down/up). Gate 2 q_disagreement now uses K=3↔K=2 mapping with Flat masking. Verified against gpu_dqn_trainer.rs:2438. 2. Forward launch order constraint added: aux forward must complete before direction Q-head forward (new serial dep). Cited pearl_canary_input_freshness_launch_order. 3. Adaptive sigmoid k formula fixed: v1 had unreachable k_max=50 because formula caps k ≤ k_base. v2 uses max(..., k_min) with k_max = k_base implicit. 4. α_grad rate limiter promoted from nice-to-have to v1. Schmitt state-flip introduces sigmoid discontinuity. β=0.9 EMA smoothing added; new ALPHA_GRAD_SMOOTHED_INDEX slot. 5. q_disagreement_baseline drop: v1 had adaptive baseline as long-EMA (feedback loop risk). v2 uses structural 0.5 (analytic K=3-with- Flat-masked random alignment). Drop BASELINE_INDEX slot. 6. Backward gradient scaling clarified: α_grad scales dL/dx (input gradient flowing back to aux), NOT dL/dW (Q-head's weight grad). Q-head learns to use the wire freely; gate only controls upstream flow. 7. 4 hard rules added: feedback_no_hiding, feedback_no_htod_htoh_only_mapped_pinned, feedback_kill_runs_on_anomaly_quickly, pearl_canary_input_freshness_launch_order. 8. Smoke A2 explicit kill criteria table added (8 triggers). Net ISV slot count unchanged (11), composition shifted: dropped BASELINE, added SMOOTHED. Total impl cost ~1150 LOC (was ~1060). Verified against current code: - TARGET_DIR_ACC_INDEX=372, AUX_DIR_ACC_SHORT_EMA_INDEX=373, AUX_DIR_PREDICTION_INDEX=375 (sp13_isv_slots.rs) - set_aux_weight clamp(0.05, 0.3) at gpu_dqn_trainer.rs:14722 (confirms Bug 3 from Smoke A diagnostic) - mag_concat_qdir precedent at experience_kernels.cu:4560 (direction-conditioning pattern; SP14's wire is the analog) - state_reset_registry pattern at lines 913-922 (canonical template for new EMA fold-reset entries) - branch_0_size = 3 in production config (the K=3 finding) Co-Authored-By: Claude Opus 4.7 (1M context) --- ...05-sp14-aux-q-wire-earned-gradient-flow.md | 177 +++++++++++++----- 1 file changed, 125 insertions(+), 52 deletions(-) diff --git a/docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md b/docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md index fdae0fb40..e5f7e2346 100644 --- a/docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md +++ b/docs/superpowers/specs/2026-05-05-sp14-aux-q-wire-earned-gradient-flow.md @@ -1,12 +1,27 @@ # SP14 — Aux→Q Wire + Earned Gradient Flow Pearl -**Status:** Design (awaiting user review) +**Status:** Design v2 (post-critical-review; awaiting final user approval) **Date:** 2026-05-05 **Author:** Brainstormed in session post-Smoke A -**Parent commit:** `6657e5626` (B1.1b — final piece of SP13 Layer B chain) -**Depends on:** SP13 chain (B0 → B0.1 → B1.0 → B1.1a → B1.1b) all landed +**Parent commit:** `eaf4adcb9` (v1 spec) → v2 with corrections +**Depends on:** SP13 chain (B0 → B0.1 → B1.0 → B1.1a → B1.1b) all landed at `6657e5626` **Branch target:** `sp11-reward-as-controlled-subsystem` +## v1 → v2 changelog + +Critical review against current code state at `eaf4adcb9` surfaced 8 corrections: + +1. **Action-class asymmetry corrected.** Direction Q-head emits K=3 (Short/Flat/Long), not K=4. Aux head emits K=2 (down/up). Gate 2 q_disagreement signal now uses K=3↔K=2 mapping with Flat masking (B.2.3). The 4-action `ExposureLevel` enum from SP13 is the *combined exposure decision*, not the direction Q-head output. +2. **Forward launch order constraint added** (B.1). Wire creates new serial dep: aux forward must complete before direction Q-head forward. Cited `pearl_canary_input_freshness_launch_order`. +3. **Adaptive sigmoid k formula fixed** (B.2.5). v1 had unreachable `k_max = 50` because formula caps `k ≤ k_base`. v2 uses `max(...)` instead of `clip(..., k_min, k_max)`, with `k_max = k_base` implicit. `k_base_aux = 20`, `k_base_q = 15`. +4. **α_grad rate limiter promoted from nice-to-have to v1** (new B.2.8). Schmitt state-flip introduces sigmoid discontinuity (~0.15-0.4 jump). Rate limiter (β=0.9 EMA smoothing) damps the transition. Added `ALPHA_GRAD_SMOOTHED_INDEX` slot. +5. **q_disagreement_baseline drop** (B.2.3). v1 had adaptive baseline as long-EMA — risk of feedback loop. v2 uses structural 0.5 (analytic K=3-with-Flat-masked random alignment). Drop `Q_DISAGREEMENT_BASELINE_INDEX`. +6. **Backward gradient scaling clarified** (B.4). α_grad scales `dL/dx` (input gradient flowing back to aux), NOT `dL/dW` (Q-head's weight gradient). Q-head learns to use the wire freely; gate only controls upstream gradient flow. +7. **4 hard rules added** (B.6): `feedback_no_hiding`, `feedback_no_htod_htoh_only_mapped_pinned`, `feedback_kill_runs_on_anomaly_quickly`, `pearl_canary_input_freshness_launch_order`. +8. **Smoke A2 explicit kill criteria table added** (B.7). + +Net slot count unchanged (11), composition shifted: dropped baseline, added smoothed. + --- ## Goal @@ -76,7 +91,11 @@ This is the architectural piece. Two coupled changes that must land atomically p **Slot 375 fate:** kept as-is (batch-mean scalar) for HEALTH_DIAG observability. The existing `aux_pred_to_isv_tanh_kernel` continues to write `mean(softmax[1] - softmax[0])` for diagnostic purposes. The wire bypasses slot 375 and reads `aux_nb_softmax_buf` directly per-bar — slot 375 becomes pure diagnostic, not a critical data path. -**Q-head SGEMM weight matrix:** `direction Q-head W` grows from `[SH2, branch_0_size]` to `[SH2 + 1, branch_0_size]`. The new row of weights is initialized to **zero** at trainer construction. Rationale: model starts ignoring the new input (no perturbation to existing learned behavior), then learns to use it through gradient descent. Xavier-init would inject noise on day 0 that the trunk would have to learn to denoise. +**Q-head SGEMM weight matrix:** the direction Q-head's first FC layer (`w_b0fc`-equivalent — implementer to identify exact tensor index in `compute_param_sizes()` against current code) grows its INPUT dimension from `SH2` to `SH2 + 1`. Output dim (advantage hidden / direction logits) unchanged. Production `branch_0_size = 3` (Short/Flat/Long per `gpu_dqn_trainer.rs:2438`) — output dim is unaffected by the wire. The new row of weights is initialized to **zero** at trainer construction. Rationale: model starts ignoring the new input (no perturbation to existing learned behavior), then learns to use it through gradient descent. Xavier-init would inject noise on day 0 that the trunk would have to learn to denoise. + +**Action-space context for the wire's downstream meaning:** the direction Q-head emits 3 actions: Short (0), Flat (1), Long (2). The aux head emits 2 classes: down (0), up (1) [plus -1 skip mask]. The wire's signal `softmax[1] - softmax[0]` is positive when aux thinks "up" (favoring Long), negative when aux thinks "down" (favoring Short). Flat is the abstention action — model picks it when neither directional commitment seems profitable. The wire makes aux's directional confidence available as input; the Q-head learns whether to trust it. + +**Forward launch order constraint** (per `pearl_canary_input_freshness_launch_order`): the wire creates a new serial dependency in the per-step graph capture. The aux head forward (`aux_next_bar_forward` writing `aux_nb_softmax_buf [B, K=2]`) must complete BEFORE the direction Q-head's first FC reads the softmax diff. Pre-SP14, aux and direction Q-head ran in parallel from `h_s2`; post-SP14, they're serial: trunk → aux forward → direction Q-head forward. Implementer must order the two launches in graph capture; failure to do so produces a stale-input bug (direction Q-head reads previous step's softmax, exactly the failure mode the canary pearl warns about). **Param tensor layout:** the direction Q-head weight tensor is one of the indices in `compute_param_sizes()` — implementer to identify exactly which index against current code at HEAD `6657e5626` per `feedback_trust_code_not_docs`. Its size grows by `branch_0_size`. Update `compute_param_sizes()` and the corresponding `layout_fingerprint_seed` entry. Per `pearl_build_rs_rerun_if_env_changed` and Invariant 8, fingerprint must bump (rename the entry suffix to indicate the new shape, e.g., `PARAM_W_DIR_QHEAD → PARAM_W_DIR_QHEAD_AUX1` or similar — implementer chooses convention matching the codebase). @@ -116,17 +135,32 @@ where: #### B.2.3 — Gate 2: Q-head disagreement (NEW signal) -**Driver signals:** `q_disagreement_short_ema` (NEW ISV slot) and `q_disagreement_baseline` (NEW ISV slot, long-EMA of disagreement to provide adaptive baseline). +**Driver signals:** `q_disagreement_short_ema` (NEW ISV slot, fast EMA α=0.3) and `q_disagreement_long_ema` (NEW ISV slot, slow EMA α=0.05 for HEALTH_DIAG drift visibility). Baseline is **structural 0.5** — not an ISV slot. Long-EMA-as-baseline was considered and rejected because adaptive baseline drift creates a feedback loop (disagreement drops → baseline drops → gate sees lower values as "above baseline" → opens spuriously). -**Per-step disagreement signal:** -``` -q_disagreement[t] = (argmax(Q_dir_logits[t]) != argmax(aux_softmax[t])) ? 1.0 : 0.0 -``` -EMA'd over batch, then over steps via standard Pearl-A bootstrap pattern. +**K=3 ↔ K=2 mapping** (the action-class asymmetry): aux head emits K=2 (down=0, up=1). Direction Q-head emits K=3 (Short=0, Flat=1, Long=2). Naive `argmax(Q) != argmax(aux)` doesn't apply across different cardinality. The disagreement signal masks Flat picks (model declined to commit; no signal): -**Behavior:** opens when persistent disagreement is meaningfully above baseline. Closed when alignment is high (no co-training needed; aux signal already being consumed). Specifically: -- High disagreement → aux signal is informative (Q-head WANTS to fight aux) → co-training would help align them -- Low disagreement → Q-head already aligned with aux → no benefit from gradient flow +``` +For each row b in batch: + aux_class = argmax(aux_softmax[b]) # 0 or 1 + q_dir_class = argmax(Q_dir_logits[b]) # 0, 1, or 2 + + if q_dir_class == 1 (Flat): # neutral; model declined + q_disagreement[b] = (skip from EMA — mask out) + contribution_count = 0 + else: + # Map Q to aux space: Short=0 ↔ down=0, Long=2 ↔ up=1 + q_dir_mapped = 0 if q_dir_class == 0 else 1 + q_disagreement[b] = (q_dir_mapped != aux_class) ? 1.0 : 0.0 + contribution_count = 1 + +q_disagreement_batch_mean = sum(q_disagreement) / max(1, sum(contribution_count)) +``` + +Then EMA over steps via standard Pearl-A bootstrap pattern. Skip-rows from Flat picks don't pollute the EMA — they're treated as "no information about disagreement", not "no disagreement." + +**Behavior:** opens when persistent disagreement is meaningfully above the structural 0.5 baseline. Specifically: +- `q_disagreement_short_ema > 0.5 + ε` → Q-head is fighting aux signal (with Flat masked out, this means the model is actively picking Short when aux says up, or Long when aux says down) → co-training would help align them +- `q_disagreement_short_ema ≤ 0.5` → Q-head agreement is at-or-better-than chance → no benefit from gradient flow **Argmax source:** use the deterministic `argmax(Q_dir_logits)` at both training and eval time — i.e., the Q-head's preferred direction *before* Thompson sampling. This tracks the Q-head's underlying directional belief; the disagreement signal should not be polluted by exploration noise. Thompson-sampled action selection diverges from argmax stochastically (especially early in training when Q values are flat), so using the post-Thompson selection would conflate two signals: directional belief and exploration. We want the belief. @@ -166,16 +200,19 @@ Once open, gate stays open until aux drops 0.06 below the open-threshold (i.e., Both `k_aux` and `k_q` self-adapt based on the variance of their respective driver signals: ``` -k_x = clip(k_base / (1 + variance_ema_x / variance_ref), [k_min, k_max]) +k_x = max(k_base / (1 + variance_ema_x / variance_ref), k_min) ``` - **Welford EMA variance** of the driver signal computed per-step - High variance → noisy signal → low k → smoother sigmoid → less reactive to noise -- Low variance → stable signal → high k → sharper sigmoid → more decisive +- Low variance → stable signal → k near k_base → sharper sigmoid → more decisive +- At variance = 0, k = k_base (structural maximum; the formula bounds k naturally above by k_base) +- At variance = variance_ref, k = k_base / 2 +- At variance >> variance_ref, k decays toward k_min (structural floor) -**Bounds:** `k_min = 1.0`, `k_max = 50.0`. These are structural numerical-stability anchors (Invariant 1 territory). The variance bound `variance_ref` is structural per-signal: e.g., `variance_ref_aux = 0.01` (1% deviation), `variance_ref_q = 0.05`. +**Bounds:** only `k_min = 1.0` is a separate constant (numerical-stability floor — prevents the sigmoid from becoming completely flat under huge variance). `k_max = k_base` is implicit in the formula — k can never exceed `k_base`. This is correct semantics: "k_base is the maximum steepness we trust when the signal is perfectly reliable; less reliability → less reactive sigmoid." Set `k_base_aux = 20.0` and `k_base_q = 15.0` (structural numerical-stability anchors per Invariant 1; calibrated so a 0.05 deviation from threshold gives sigmoid output ≈ 0.73 at full reactivity). The variance bound `variance_ref` is structural per-signal: e.g., `variance_ref_aux = 0.01` (1% deviation = signal becomes noisy), `variance_ref_q = 0.05`. -**Per-fold reset:** variance EMAs reset to a sentinel that gives `k = k_base` at fold boundary (e.g., `var_ema = variance_ref` so initial `k = k_base / 2`). Without reset, fold N's variance carries into fold N+1's sigmoid steepness during cold start. +**Per-fold reset:** variance EMAs reset to a sentinel that gives `k = k_base` at fold boundary (e.g., `var_ema = 0.0` so initial k = `k_base`). Without reset, fold N's variance carries into fold N+1's sigmoid steepness during cold start. (Note: starting at `var_ema = 0` deliberately gives maximum reactivity at fold start; the warmup ramp + Pearl-A bootstrap on EMAs prevent the high-k from acting on bad data.) #### B.2.6 — Per-epoch warmup ramp @@ -199,51 +236,70 @@ The pathological dynamic to defend against: Q-head learns that disagreeing with This is the "smart toddler with circuit breaker" pattern: the model can't game the wire if doing so triggers a forced reset. +#### B.2.8 — α_grad rate limiter (smoothing across state transitions) + +The Schmitt-trigger hysteresis (B.2.4) introduces a sigmoid-argument discontinuity at the moment `gate1_open_state` flips between {0, 1}. The argument shifts by 0.06 (the hysteresis band). With `k_aux ≈ 10`, the sigmoid output jumps by ~0.15; with high `k_aux ≈ 25` (low-variance regime), the jump is ~0.4. That's a meaningful gradient discontinuity at a single step boundary. + +**Mitigation:** exponentially-smoothed α_grad output: + +``` +α_grad_smoothed = β · α_grad_smoothed_prev + (1 − β) · α_grad_raw +where β = 0.9 (structural; ~10-step smoothing window) +``` + +The kernels reading α_grad for backward gradient scaling (B.4 launch site) consume `α_grad_smoothed`, NOT `α_grad_raw`. `α_grad_raw` remains in HEALTH_DIAG so we can see both the raw decision and the smoothed actuation. + +**Why β = 0.9:** ten-step smoothing window means a state flip is absorbed over ~10 backward passes, with α_grad transitioning roughly linearly. Long enough to dampen the discontinuity; short enough that the gate's directional intent (open vs closed) still expresses promptly. β is structural — not adaptive — because rate-limiter behavior should be consistent regardless of signal regime. + +**Per-fold reset:** `α_grad_smoothed` resets to 0 at fold boundary via state-reset-registry. Fresh fold starts with closed gate; the smoothing kicks in only when gate transitions occur during training. + ### B.3 — ISV slot allocations 11 new ISV slots required: | Slot name | Purpose | Reset at fold? | |---|---|---| -| `Q_DISAGREEMENT_SHORT_EMA_INDEX` | Fast EMA of per-step disagreement | Yes (sentinel 0.5) | -| `Q_DISAGREEMENT_LONG_EMA_INDEX` | Slow EMA, baseline source | Yes (sentinel 0.5) | -| `Q_DISAGREEMENT_BASELINE_INDEX` | Adaptive baseline (= long-EMA after warmup) | Yes (structural 0.5 sentinel) | -| `K_AUX_ADAPTIVE_INDEX` | Current `k_aux` value | Yes (sentinel `k_base`) | -| `K_Q_ADAPTIVE_INDEX` | Current `k_q` value | Yes (sentinel `k_base`) | -| `AUX_DIR_ACC_VARIANCE_EMA_INDEX` | Welford variance of aux_dir_acc | Yes (sentinel `variance_ref`) | -| `Q_DISAGREEMENT_VARIANCE_EMA_INDEX` | Welford variance of disagreement | Yes (sentinel `variance_ref`) | +| `Q_DISAGREEMENT_SHORT_EMA_INDEX` | Fast EMA (α=0.3) of per-step disagreement (Flat-masked) | Yes (sentinel 0.5) | +| `Q_DISAGREEMENT_LONG_EMA_INDEX` | Slow EMA (α=0.05) — HEALTH_DIAG drift visibility only, NOT used as adaptive baseline | Yes (sentinel 0.5) | +| `K_AUX_ADAPTIVE_INDEX` | Current `k_aux` value | Yes (sentinel `k_base_aux`) | +| `K_Q_ADAPTIVE_INDEX` | Current `k_q` value | Yes (sentinel `k_base_q`) | +| `AUX_DIR_ACC_VARIANCE_EMA_INDEX` | Welford variance of aux_dir_acc | Yes (sentinel 0.0 → initial k = k_base) | +| `Q_DISAGREEMENT_VARIANCE_EMA_INDEX` | Welford variance of disagreement | Yes (sentinel 0.0 → initial k = k_base) | | `GATE1_OPEN_STATE_INDEX` | Schmitt-trigger persistent state | Yes (0 — closed) | -| `ALPHA_GRAD_INDEX` | Computed α_grad value (HEALTH_DIAG visibility) | Yes (0) | +| `ALPHA_GRAD_RAW_INDEX` | Raw computed α_grad value (HEALTH_DIAG visibility) | Yes (0) | +| `ALPHA_GRAD_SMOOTHED_INDEX` | Rate-limited α_grad consumed by backward path (B.2.8) | Yes (0) | | `AUX_DIR_ACC_POST_OPEN_MIN_INDEX` | Anti-gradient-hacking detection | Yes (1.0 — no min observed) | | `GRADIENT_HACK_LOCKOUT_REMAINING_INDEX` | Circuit-breaker epoch counter | Yes (0 — no lockout) | -All slots register in `state_reset_registry.rs` with appropriate sentinels. Slot indices to be allocated in `sp14_isv_slots.rs` (NEW file; mirrors `sp13_isv_slots.rs` pattern). +**Baseline for Gate 2 is structural 0.5** — not an ISV slot. The K=2-aux ↔ K=3-Q-with-Flat-masked random alignment at random init is exactly 0.5; this is the analytic baseline. Long-EMA-as-baseline was rejected (feedback loop risk; see B.2.3). + +All slots register in `state_reset_registry.rs` with appropriate sentinels following the existing pattern (lines 913-922 for the two pre-existing aux_dir_acc EMAs are the canonical template). Slot indices to be allocated in `sp14_isv_slots.rs` (NEW file; mirrors `sp13_isv_slots.rs` pattern). ### B.4 — Kernels Three new GPU kernels: -1. **`q_disagreement_update_kernel`** — per-step compute argmax mismatch from `Q_dir_logits` and `aux_nb_softmax_buf`, EMA update for short and long, Welford variance update. Block tree-reduce per `feedback_no_atomicadd`. ~80 LOC. -2. **`alpha_grad_compute_kernel`** — read all driver signals, evaluate Schmitt-trigger logic for Gate 1, compute both sigmoids, multiply with warmup gate, write `ALPHA_GRAD_INDEX`. Includes adaptive k_aux/k_q computation from variance EMAs. ~60 LOC. +1. **`q_disagreement_update_kernel`** — per-step compute K=3↔K=2 mapped argmax mismatch (B.2.3 logic, with Flat masked) from `Q_dir_logits` and `aux_nb_softmax_buf`, EMA update for short (α=0.3) and long (α=0.05), Welford variance update. Block tree-reduce per `feedback_no_atomicadd`; tracks both numerator (sum of disagreements) and denominator (count of non-Flat picks) separately, then divides at the end. ~100 LOC. +2. **`alpha_grad_compute_kernel`** — read all driver signals, evaluate Schmitt-trigger logic for Gate 1 (B.2.4), compute adaptive `k_aux` and `k_q` from variance EMAs (B.2.5), evaluate both sigmoids, multiply with warmup gate (B.2.6), write `ALPHA_GRAD_RAW_INDEX`. Then apply rate-limiter (B.2.8): `α_grad_smoothed = β · α_grad_smoothed_prev + (1−β) · α_grad_raw`, write `ALPHA_GRAD_SMOOTHED_INDEX`. ~80 LOC. 3. **`gradient_hack_detect_kernel`** — read aux_dir_acc, gate1_open_state, q_disagreement, decide lockout state, write to `GRADIENT_HACK_LOCKOUT_REMAINING_INDEX` and force-clear `GATE1_OPEN_STATE_INDEX` if triggered. ~40 LOC. Plus modifications to: -- **Direction Q-head forward kernel** (or new fused concat-SGEMM): in_dim+1, read `softmax_diff` per-bar from `aux_nb_softmax_buf`. ~50-100 LOC. -- **Direction Q-head backward kernel**: scale the gradient column at the wire position by `α_grad` before SAXPY into aux head's softmax gradient. ~30 LOC. -- **`compute_param_sizes()`** and **`layout_fingerprint_seed()`**: bump direction Q-head weight tensor size by `branch_0_size`, rename layout entry. ~10 LOC. +- **Direction Q-head forward kernel** (or new pre-SGEMM concat kernel mirroring `mag_concat_qdir` precedent at `experience_kernels.cu:4560`): in_dim+1, read `softmax_diff = aux_nb_softmax_buf[b, 1] − aux_nb_softmax_buf[b, 0]` per-bar. **Launch order: aux forward must complete before this concat reads `aux_nb_softmax_buf`** (B.1 forward launch order constraint). ~50-100 LOC. +- **Direction Q-head backward gradient scaling**: scale `dL/dx[wire_col]` (the **input** gradient flowing back to aux head's softmax) by `α_grad_smoothed`. The Q-head's **weight** gradient `dL/dW` for the new column is NOT scaled — Q-head learns to use the wire freely; α_grad only gates gradient flow to aux. ~30 LOC. +- **`compute_param_sizes()`** and **`layout_fingerprint_seed()`**: bump the direction Q-head's first FC weight tensor (`w_b0fc` or equivalent — implementer to identify exact index) input dim by 1; corresponding layout-fingerprint entry renamed (e.g., `PARAM_W_DIR_FC1 → PARAM_W_DIR_FC1_AUX`) to force fingerprint hash bump per Invariant 8. ~10 LOC. - **Trainer construction**: zero-initialize the new weight column. ~5 LOC. ### B.5 — HEALTH_DIAG observability New per-epoch `pearl_egf_diag` line: ``` -pearl_egf_diag α=0.45 gate1=0.92 gate2=0.49 k_aux=15.3 k_q=8.7 - q_dis_short=0.523 q_dis_long=0.498 q_dis_baseline=0.50 +pearl_egf_diag α_smoothed=0.42 α_raw=0.45 gate1=0.92 gate2=0.49 k_aux=15.3 k_q=8.7 + q_dis_short=0.523 q_dis_long=0.498 gate1_state=open aux_post_open_min=0.55 lockout_remaining=0 warmup=1.0 ``` -Without these emit fields, diagnosing α_grad behavior in the smoke logs is forensic-only. All 11 ISV slots emit per epoch. +Both `α_raw` and `α_smoothed` emit so the rate limiter's effect is visible. `q_dis_long` is included even though it's not used as baseline — it's there to track whether disagreement is drifting (training-side directional alignment improving over folds). Without these emit fields, diagnosing α_grad behavior in the smoke logs is forensic-only. ### B.6 — Hard rules upheld @@ -251,26 +307,43 @@ Without these emit fields, diagnosing α_grad behavior in the smoke logs is fore - `feedback_no_atomicadd` — all three new kernels use block tree-reduce. Welford variance specifically benefits from this pattern (Welford's online algorithm is associative and parallelizable). - `feedback_cpu_is_read_only` — all signals computed GPU-side. CPU only reads ISV for HEALTH_DIAG emit. - `feedback_no_stubs` — every new ISV slot has a real producer and a real consumer in this commit. No zero-stub. -- `feedback_isv_for_adaptive_bounds` — adaptive bounds (k_aux, k_q, q_disagreement_baseline) live in ISV. Numerical-stability anchors (k_min, k_max, variance_ref, hysteresis ±0.03 band, lockout epochs) are structural constants per Invariant 1. -- `feedback_trust_code_not_docs` — implementer verifies all line numbers and structures against current code at HEAD `6657e5626` before editing. +- `feedback_no_hiding` — no `_` prefixes on unused params, no `#[allow]` to suppress warnings. Every new identifier serves its declared purpose. +- `feedback_no_htod_htoh_only_mapped_pinned` — all 11 new ISV slots live in the existing mapped-pinned ISV buffer (SP4 infrastructure). HEALTH_DIAG reads via the same mapped-pinned snapshot path used by all other ISV consumers; no new HtoD or HtoH copies. +- `feedback_isv_for_adaptive_bounds` — adaptive bounds (k_aux, k_q from variance) live in ISV. Numerical-stability anchors (k_min, k_base_aux, k_base_q, variance_ref, hysteresis ±0.03 band, lockout epochs, smoothing β) are structural constants per Invariant 1. +- `feedback_trust_code_not_docs` — implementer verifies all line numbers and tensor indices against current code at HEAD `eaf4adcb9` before editing. - `feedback_no_legacy_aliases` — fingerprint entry rename has no `_DEPRECATED` shim. -- `pearl_first_observation_bootstrap` — Pearl-A bootstrap applied to all new EMAs (q_disagreement, variances, post_open_min). +- `feedback_kill_runs_on_anomaly_quickly` — explicit kill criteria enumerated in B.7 below; smoke A2 monitor terminates the workflow at first signal of pathological behavior. +- `pearl_first_observation_bootstrap` — Pearl-A bootstrap applied to all new EMAs (q_disagreement short/long, variances, post_open_min). - `pearl_bounded_modifier_outputs_require_structural_activation` — softmax IS the structural activation; α_grad's sigmoid composition produces structurally-bounded `[0, 1]`. - `pearl_zscore_normalization_for_magnitude_asymmetric_signals` — variance-driven k adjustment is the appropriate analog. +- `pearl_canary_input_freshness_launch_order` — direction Q-head's input concat reads aux_nb_softmax_buf; aux forward MUST launch before direction Q-head forward in graph capture, otherwise stale-input bug manifests as fold-1-wire-works-fold-2-doesn't pattern. - `pearl_event_driven_reward_density_alignment` — N/A; this is a backward-pass gradient gate, not a per-step reward shaping. - `feedback_no_quickfixes` — the pearl is the canonical answer to "stop-gradient vs full-gradient" tradeoff, not a hack to make Q-head train. ### B.7 — Validation criteria -**Smoke A2 (5-epoch L40S, post-SP14):** -- `α_grad` emits per epoch in HEALTH_DIAG; observable across all folds -- `gate1_open_state` opens (=1) at least once during fold 1 (when aux_dir_acc reached 0.61 in pre-SP14 smoke) -- `q_disagreement_short_ema` non-zero (at random init, expect ~0.5; with training, expect drift toward agreement or persistent disagreement) -- `aux_dir_acc` reaches target (0.55) in fold 1 — confirms aux head still trains on its label even with new wire +**Smoke A2 (5-epoch L40S, post-SP14) — pass criteria:** +- `α_grad_smoothed` and `α_grad_raw` both emit per epoch in HEALTH_DIAG; observable across all folds +- `gate1_open_state` opens (=1) at least once during fold 1 (when aux_dir_acc reaches 0.55-0.60) +- `q_disagreement_short_ema` non-zero and varying across epochs (at random init, expect ~0.5; with training, expect drift) +- `aux_dir_acc` reaches target (0.55) in fold 1 — confirms aux head still trains on its label even with the new wire - `val_dir_dist` remains balanced (Hold-pricing still works) - `GRAD_CLIP_OUTLIER` count < 100 (down from 1109 — assumes A's C51 floor lands first) - No new pathological behaviors: aux_dir_acc doesn't drop catastrophically; circuit breaker `lockout_remaining` stays at 0 in normal operation +**Smoke A2 — explicit KILL criteria** (per `feedback_kill_runs_on_anomaly_quickly`; smoke monitor terminates the workflow at first signal): + +| # | Kill trigger | Mechanism implication | +|---|---|---| +| 1 | Workflow phase = Failed / Error | Already done; report + investigate | +| 2 | Build/compile failure (CUDA error, missing symbol, link error) | Wire infrastructure broken | +| 3 | `α_grad_smoothed` stuck at 0.0 across 5 epochs in fold 1 (when aux_dir_acc demonstrably reaches 0.55+) | Wire backward path not reaching the gate; either backward order wrong or scaling factor not applied | +| 4 | `gate1_open_state` never flips to 1 across full smoke despite aux_dir_acc above threshold_1_open=0.58 | Schmitt-trigger logic broken or state slot not persisting | +| 5 | `aux_dir_acc` collapses below 0.40 after gate1 opens AND `lockout_remaining` stays at 0 | Gradient hacking happening; circuit breaker not firing — bug in `gradient_hack_detect_kernel` | +| 6 | `val_sharpe` < 0 sustained across 2+ epochs | Full-stack regression; the wire is destabilizing training | +| 7 | `n_up = 0` in HEALTH_DIAG aux_b1_diag (regression of B1.1b producer) | Producer broke during SP14 implementation; should have been caught by unit tests but kill-on-detect is the safety | +| 8 | Forward stale-input pattern: `aux_dir_acc` healthy but `α_grad` oscillates wildly with no apparent driver | Launch order constraint violated — direction Q-head reading previous-step softmax (canary pearl signature) | + **30-epoch validation A (post-Smoke-A2-pass):** - `val_win_rate` rises above 0.50 by ep 20-25 — confirms the user's hypothesis that aux signal can move WR when wired - If WR stays at 45-48% across 30 epochs even with α_grad demonstrably opening: hypothesis falsified at this layer; next intervention is on the aux signal richness (sub-project C — adaptive LR, OR longer prediction horizons, OR microstructure feature additions) @@ -285,20 +358,20 @@ Without these emit fields, diagnosing α_grad behavior in the smoke logs is fore | Component | LOC | |---|---| | 11 new ISV slots + constants in `sp14_isv_slots.rs` | ~80 | -| `q_disagreement_update_kernel` (NEW) | ~80 | -| `alpha_grad_compute_kernel` (NEW) | ~60 | +| `q_disagreement_update_kernel` (NEW; K=3↔K=2 mapping with Flat masking) | ~100 | +| `alpha_grad_compute_kernel` (NEW; includes Schmitt + adaptive k + rate limiter) | ~80 | | `gradient_hack_detect_kernel` (NEW) | ~40 | -| Direction Q-head forward modification (in_dim+1, concat) | ~100 | -| Direction Q-head backward gradient scaling | ~30 | +| Direction Q-head forward modification (in_dim+1, concat reading aux_nb_softmax_buf) | ~100 | +| Direction Q-head backward gradient scaling (dL/dx scaling, NOT dL/dW) | ~30 | | `compute_param_sizes` + `layout_fingerprint_seed` updates | ~20 | | `state_reset_registry` entries (11 new) | ~50 | -| HEALTH_DIAG emit | ~30 | +| HEALTH_DIAG emit (pearl_egf_diag line) | ~30 | | Trainer construction (zero-init new weight column) | ~10 | -| Three orchestrator launchers (kernel launches + arg threading) | ~80 | -| Audit doc B section | ~80 | -| **Production code total** | **~660** | -| GPU oracle unit tests (12-15 tests) | ~400 | -| **Grand total** | **~1060** | +| Three orchestrator launchers (kernel launches + launch-order constraint enforcement) | ~80 | +| Audit doc SP14 section | ~80 | +| **Production code total** | **~700** | +| GPU oracle unit tests (12-15 tests covering each gate, hysteresis, rate limiter, K=3↔K=2 mapping) | ~450 | +| **Grand total** | **~1150** | 3-5 days of focused work for a fresh implementer with clear brief. Comparable in scope to SP13 B1.1a (which was 175 tool uses for the implementer agent — at the upper bound of what fits in a single session).