From 7a19c51522ce2c3cde4d6935fe4e783394e41dd3 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Mon, 4 May 2026 13:26:55 +0200 Subject: [PATCH] =?UTF-8?q?docs(sp11):=20B1b=20smoke-recovery=20close-out?= =?UTF-8?q?=20=E2=80=94=20audit=20+=203=20memory=20pearls?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Layer C close-out for SP11 B1b smoke-recovery work. Two atomic commits land the fix: - b3b4d0278: z-score normalization for mag-ratio canary (Bug 5) - fd24b5383: launch-order — reward_component_ema before canary (Bug 6) Audit doc (docs/dqn-wire-up-audit.md) adds the close-out entry covering all 6 bugs (slot 63 overload, stale rc[] init, cf_flip ordering, cf-component feedback loop, magnitude-asymmetric ratios, launch-order), the 6 new ISV variance EMA slots [361..367) producer/consumer wiring, and validation evidence from the 3 smoke-test workflows (smoke-test-6wd2c killed → smoke-test-4rbv9 killed → smoke-test-gwfn8 PASSED 15m11s on commit fd24b5383). 3 memory pearls written to user memory (not in repo): - pearl_zscore_normalization_for_magnitude_asymmetric_signals - pearl_canary_input_freshness_launch_order - pearl_controller_amplifies_dominant_magnitude_trap MEMORY.md index updated under Controllers and signals. Open follow-up flagged: residual within-fold sharpe degradation across epochs is a separate triage scoped to post-T10 validation, not part of B1b smoke-recovery. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/dqn-wire-up-audit.md | 140 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index e8f54aa11..d5278ac35 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -5432,3 +5432,143 @@ Verified post-fix: Follow-up: re-run B1b smoke on the patched commit; expectation is `w_pop` no longer pinning to 2.0 at ep1 because z[c] for c=1..5 now sees populated mag and var at the moment the canary fires. + +## SP11 B1b smoke-recovery — Layer C close-out (2026-05-04) + +Layer C close-out for the B1b smoke-recovery work. Two atomic commits +(per `feedback_no_partial_refactor`) land the fix: + +| Commit | Subject | Fixes | +|---|---|---| +| `b3b4d0278` | `fix(sp11): B1b smoke-recovery — z-score normalization for mag-ratio canary` | Bug 5 (magnitude-asymmetric ratios) — adds 6 variance EMA slots + Welford in producers + z-score in canary | +| `fd24b5383` | `fix(sp11): B1b launch-order — reward_component_ema before mag-ratio canary` | Bug 6 (launch-order stale slots) — moves `launch_reward_component_ema_inplace` to fire before the canary | + +Both build atop the prior B1b bug-hunt fix-up chain (`5e16b67ca`, +`b435d25be`, `61b2fa962`) which addressed Bugs 1-4 in the per-bar +reward-composition kernel. Together: 6 bugs × 4 fix-up commits over +the B1b smoke-recovery window. + +### Bug catalog + +| # | Symptom | Root cause | Fix commit | Audit entry above | +|---|---|---|---|---| +| 1 | Slot 63 (`REWARD_POPART_EMA_INDEX`) overload — controller saw inflated total-reward magnitude as "popart's signal" → `w_pop≈2.0` → 10× sharpe drop | Pre-SP11 `reward_component_ema` wrote `\|reward_components_per_sample[+0]\|` EMA into slot 63, which doubled as PopArt's input. Pre-SP11 `r[+0] ≈ r_popart` (popart dominated inline accumulation); B1b's structural decomposition surfaced the contamination | `5e16b67ca` (B1b follow-up) | "## SP11 B1b" via slot 360 separation (`POPART_COMPONENT_MAG_EMA_INDEX`) | +| 2 | Stale `reward_components_per_sample[+1..+5]` per-bar — bars whose execution path didn't fire retained values from previous batch's same `out_off`, contaminating SP4 EMA + SP11 mag-ratio canary | Conditional-write pattern on `rc[1..5]` (each slot written only when its component path executed); zero-init of locals didn't extend to the buffer | `b435d25be` (B1b bug-hunt fix-up) | "### SP11 B1b bug-hunt fix-up" §Bug 1 | +| 3 | cf_flip ordering — flipped-sample inventory/churn (subtractive penalties) ADDED to negated reward (penalty became bonus); conviction scaled wrong-signed value → asymmetric gradient signal between flipped and non-flipped batch samples | Per spec §3.4.4 the cf_flip multiplicand should be the LAST modifier (after clamp/drawdown/inventory/churn/conviction). Pre-fix-up `if (do_flip) reward *= -1.0f;` ran at the NaN-guard site BEFORE inventory/churn/conviction | `b435d25be` (B1b bug-hunt fix-up) | "### SP11 B1b bug-hunt fix-up" §Bug 2 | +| 4 | cf-component feedback loop in mag-ratio canary — `experience_env_step` wrote `cf_reward_weighted = w_cf × cf_reward` (POST-controller-weight) to `rc[+1]`, which SP4 EMA fed into ISV[64] (`REWARD_CF_EMA_INDEX`), which the canary read → `high w_cf → high slot 64 → high ratio[1] → controller raises w_cf` self-reinforcing loop | Single `rc[+1]` write site mistakenly used the controller-weighted value while the other 5 slots correctly wrote raw pre-weight values | `61b2fa962` (B1b bug-hunt fix-up #2) | "### SP11 B1b bug-hunt fix-up #2" | +| 5 | Magnitude-asymmetric linear ratios — `winner_weight = mag[c] / Σ mag` always elected popart (intrinsically O(100) per fire vs cf/trail/micro/opp_cost/bonus O(0.1-2)). `w_pop` cap-bound saturation at MAX_WEIGHT=3.0 by ep1; mean(weights) below 1.0 because cap binds; sharpe collapsed | Linear ratio formula doesn't see through intrinsic-magnitude asymmetry — see new pearl `pearl_controller_amplifies_dominant_magnitude_trap` | `b3b4d0278` (z-score amendment) | This entry — see "Resolution" below | +| 6 | Launch-order stale slots — `smoke-test-4rbv9` on `b3b4d0278` showed bit-identical `w_pop=2.000` at ep1 to pre-z-score B1b smoke. With `var[c] = sentinel-0` at ep1 the z-score formula degenerates to the linear-formula equivalent (`mag[c] × 1e6 / Σ mag × 1e6 = mag[c] / Σ mag`) | `launch_reward_component_ema_inplace` (populating ISV[64..68] mag + ISV[362..366] var for cf/trail/micro/opp_cost/bonus) ran at the C.2 site (line ~3707) AFTER `launch_sp11_mag_ratio_compute` at line 3465. Canary read sentinel-0 for c=1..5 → ratio[0]=1.0 → controller saturated | `fd24b5383` (launch-order) | "## SP11 B1b launch-order fix-up" entry above | + +### Resolution — Bugs 5 + 6 (this Layer C close-out) + +**Z-score normalization** (Bug 5, commit `b3b4d0278`). The mag-ratio +canary kernel changed from `ratio[c] = mag[c] / Σ mag` to: + +``` +z[c] = mag[c] / max(sqrt(var[c]), EPS_DIV) +ratio[c] = z[c] / Σ z +``` + +Each component contributes "shares of own scale" rather than "shares +of absolute magnitude". The controller's redistribution logic now +correctly amplifies the *currently-surprising* component, not the +intrinsically-largest. New pearl +`pearl_zscore_normalization_for_magnitude_asymmetric_signals` and +`pearl_controller_amplifies_dominant_magnitude_trap` capture the +generalisable lesson. + +**Launch-order fix** (Bug 6, commit `fd24b5383`). Moved +`launch_reward_component_ema_inplace` from line ~3707 to immediately +before `launch_sp11_popart_component_ema` in the SP11 producer chain +(line ~3441). The kernel itself unchanged — only the launch site. +Other launches at the original C.2 site (`launch_trade_attempt_rate +_ema_inplace`, `launch_plan_threshold_update_inplace`, +`launch_seed_step_counter_update_inplace`, +`launch_cql_alpha_seed_update_inplace`) STAY at C.2 — their consumers +have no pre-canary timing constraint. New pearl +`pearl_canary_input_freshness_launch_order` documents the producer- +before-consumer audit discipline. + +### ISV slot allocation — 6 new variance EMA slots + +| Slot | Constant | Producer kernel | Consumer kernel | +|---|---|---|---| +| 361 | `REWARD_COMPONENT_VAR_EMA_BASE + 0` (popart variance) | `popart_component_ema_kernel.cu` (extended with Welford) → `apply_pearls_ad_kernel` | `reward_component_mag_ratio_compute_kernel.cu` (paired with mag at slot 360) | +| 362 | `+ 1` (cf variance) | `reward_component_ema_kernel.cu` (extended with Welford) → `apply_pearls_ad_kernel` | same canary (paired with mag at slot 64) | +| 363 | `+ 2` (trail variance) | same | same canary (paired with mag at slot 65) | +| 364 | `+ 3` (micro variance) | same | same canary (paired with mag at slot 66) | +| 365 | `+ 4` (opp_cost variance) | same | same canary (paired with mag at slot 67) | +| 366 | `+ 5` (bonus variance) | same | same canary (paired with mag at slot 68) | + +`SP11_SLOT_END` 361 → 367; `SP11_PRODUCER_COUNT` 21 → 27. Layout +fingerprint shifted accordingly. State-reset registry: 5 new entries +(`sp11_reward_component_var` covering slots [361..367) — Welford +state per-component is paired with the magnitude EMA's existing +sentinel-0 reset path; FoldReset sentinel 0 lets Pearl A's first +observation define scale per-fold per `pearl_first_observation_bootstrap`). + +Variance is per-fold (NOT cross-fold persistent) — each new fold's +reward-component scale distribution may differ; resetting tracks the +new fold's z-score normalisation correctly. + +### Validation evidence + +Three smokes track the recovery: + +| Smoke ID | Commit | Result | Key metrics | +|---|---|---|---| +| `smoke-test-6wd2c` | `61b2fa962` (post Bug 1-4 fix-ups, pre z-score) | **KILLED ep1** | `w_pop=2.000`, `w_cf=0.661`, others=0.661; sharpe collapsed 10.7 → 2.4 inside ep1 | +| `smoke-test-4rbv9` | `b3b4d0278` (z-score landed, launch-order BUG present) | **KILLED ep1** | Bit-identical `w_pop=2.000` to `6wd2c` — proves z-score was structurally a no-op (Bug 6 diagnostic signature) | +| `smoke-test-gwfn8` | `fd24b5383` (z-score + launch-order both fixed) | **PASSED 15m11s** | 3 folds × 5 epochs; `w_pop` max=1.706 (down from 3.0 cap); `mean(weights) = 1.000 ± 0.015` across all 15 epochs; F1/F2 ep0 transitions clean (no MAX_WEIGHT saturation); F1 ep0 picked `w_mi=1.558` as largest (NOT popart) — proving adaptive component selection works through intrinsic-magnitude asymmetry | + +Empirical comparison ep1 across smokes: + +| Metric | `smoke-test-6wd2c` | `smoke-test-4rbv9` | `smoke-test-gwfn8` | +|---|---|---|---| +| `w_pop` ep1 | 2.000 | 2.000 (bit-identical) | < MAX (varies; max 1.706 across full run) | +| `mean(weights)` ep1 | < 1.0 (cap binds) | < 1.0 (cap binds) | 1.000 ± 0.015 | +| Sharpe trajectory | 10.7 → 2.4 (1 epoch) | unstable, killed | Stable across 15 epochs | +| Adaptive winner | always popart | always popart | varies per regime (F1 ep0: `w_mi`) | + +### Verification commands (HEAD `fd24b5383`) + +- `SQLX_OFFLINE=true cargo check -p ml --lib` — clean +- `SQLX_OFFLINE=true cargo test -p ml --lib trainers::dqn::monitors::reward_component_monitor` — 3/3 passing +- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — passes (variance-slot block above SP9 region) +- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset_registry` — 4/4 contract tests pass + +### Per-pearl provenance + +- `pearl_first_observation_bootstrap.md` — variance EMAs reset to + sentinel-0 per-fold; first observation defines z-score scale. +- `pearl_no_partial_refactor.md` — variance slot allocation + + producer Welford extension + canary z-score + state-reset registry + entries land in one atomic commit per fix. +- `pearl_isv_for_adaptive_bounds.md` — `EPS_DIV=1e-6` is the only + Invariant-1 anchor in the new code; every other quantity (variance + EMAs, magnitudes, ratios) is signal-driven. +- `pearl_no_atomicadd.md` — Welford variance computed via single-pass + two-tree-reduce inside the existing producer kernel, no atomicAdd. +- `pearl_no_cpu_compute_strict.md` — variance compute lives in GPU + producer; consumer reads via mapped-pinned ISV. +- `pearl_zscore_normalization_for_magnitude_asymmetric_signals.md` + (NEW) — z-score makes the ratio formula scale-invariant; one extra + Welford-variance per existing magnitude EMA. +- `pearl_canary_input_freshness_launch_order.md` (NEW) — producers + feeding a canary MUST launch before it; bit-identical-to-linear is + the smoking-gun diagnostic for stale-slot leakage. +- `pearl_controller_amplifies_dominant_magnitude_trap.md` (NEW) — + generic recipe: any "winner_weight = ratio" controller with + asymmetric input scales needs scale normalization first. + +### Open follow-up + +Within-fold sharpe degradation across epochs (e.g., F0 sharpe slips +across ep0..ep4 within `smoke-test-gwfn8`) is a **separate residual** +not part of B1b smoke-recovery scope. The B1b smoke-recovery success +criterion was "controller engages adaptively without saturating cap + +no early-epoch collapse"; that criterion is met. Within-fold residual +degradation will be triaged as a follow-up task post-T10 50-epoch +validation, scoped against fold-internal exploration/exploitation +dynamics (Kelly engagement at fold mid-epoch, NoisyNet σ schedule, +saboteur intensity drift), not against the controller.