docs(sp11): B1b smoke-recovery close-out — audit + 3 memory pearls
Layer C close-out for SP11 B1b smoke-recovery work. Two atomic commits land the fix: -b3b4d0278: z-score normalization for mag-ratio canary (Bug 5) -fd24b5383: launch-order — reward_component_ema before canary (Bug 6) Audit doc (docs/dqn-wire-up-audit.md) adds the close-out entry covering all 6 bugs (slot 63 overload, stale rc[] init, cf_flip ordering, cf-component feedback loop, magnitude-asymmetric ratios, launch-order), the 6 new ISV variance EMA slots [361..367) producer/consumer wiring, and validation evidence from the 3 smoke-test workflows (smoke-test-6wd2c killed → smoke-test-4rbv9 killed → smoke-test-gwfn8 PASSED 15m11s on commitfd24b5383). 3 memory pearls written to user memory (not in repo): - pearl_zscore_normalization_for_magnitude_asymmetric_signals - pearl_canary_input_freshness_launch_order - pearl_controller_amplifies_dominant_magnitude_trap MEMORY.md index updated under Controllers and signals. Open follow-up flagged: residual within-fold sharpe degradation across epochs is a separate triage scoped to post-T10 validation, not part of B1b smoke-recovery. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -5432,3 +5432,143 @@ Verified post-fix:
|
||||
Follow-up: re-run B1b smoke on the patched commit; expectation is
|
||||
`w_pop` no longer pinning to 2.0 at ep1 because z[c] for c=1..5
|
||||
now sees populated mag and var at the moment the canary fires.
|
||||
|
||||
## SP11 B1b smoke-recovery — Layer C close-out (2026-05-04)
|
||||
|
||||
Layer C close-out for the B1b smoke-recovery work. Two atomic commits
|
||||
(per `feedback_no_partial_refactor`) land the fix:
|
||||
|
||||
| Commit | Subject | Fixes |
|
||||
|---|---|---|
|
||||
| `b3b4d0278` | `fix(sp11): B1b smoke-recovery — z-score normalization for mag-ratio canary` | Bug 5 (magnitude-asymmetric ratios) — adds 6 variance EMA slots + Welford in producers + z-score in canary |
|
||||
| `fd24b5383` | `fix(sp11): B1b launch-order — reward_component_ema before mag-ratio canary` | Bug 6 (launch-order stale slots) — moves `launch_reward_component_ema_inplace` to fire before the canary |
|
||||
|
||||
Both build atop the prior B1b bug-hunt fix-up chain (`5e16b67ca`,
|
||||
`b435d25be`, `61b2fa962`) which addressed Bugs 1-4 in the per-bar
|
||||
reward-composition kernel. Together: 6 bugs × 4 fix-up commits over
|
||||
the B1b smoke-recovery window.
|
||||
|
||||
### Bug catalog
|
||||
|
||||
| # | Symptom | Root cause | Fix commit | Audit entry above |
|
||||
|---|---|---|---|---|
|
||||
| 1 | Slot 63 (`REWARD_POPART_EMA_INDEX`) overload — controller saw inflated total-reward magnitude as "popart's signal" → `w_pop≈2.0` → 10× sharpe drop | Pre-SP11 `reward_component_ema` wrote `\|reward_components_per_sample[+0]\|` EMA into slot 63, which doubled as PopArt's input. Pre-SP11 `r[+0] ≈ r_popart` (popart dominated inline accumulation); B1b's structural decomposition surfaced the contamination | `5e16b67ca` (B1b follow-up) | "## SP11 B1b" via slot 360 separation (`POPART_COMPONENT_MAG_EMA_INDEX`) |
|
||||
| 2 | Stale `reward_components_per_sample[+1..+5]` per-bar — bars whose execution path didn't fire retained values from previous batch's same `out_off`, contaminating SP4 EMA + SP11 mag-ratio canary | Conditional-write pattern on `rc[1..5]` (each slot written only when its component path executed); zero-init of locals didn't extend to the buffer | `b435d25be` (B1b bug-hunt fix-up) | "### SP11 B1b bug-hunt fix-up" §Bug 1 |
|
||||
| 3 | cf_flip ordering — flipped-sample inventory/churn (subtractive penalties) ADDED to negated reward (penalty became bonus); conviction scaled wrong-signed value → asymmetric gradient signal between flipped and non-flipped batch samples | Per spec §3.4.4 the cf_flip multiplicand should be the LAST modifier (after clamp/drawdown/inventory/churn/conviction). Pre-fix-up `if (do_flip) reward *= -1.0f;` ran at the NaN-guard site BEFORE inventory/churn/conviction | `b435d25be` (B1b bug-hunt fix-up) | "### SP11 B1b bug-hunt fix-up" §Bug 2 |
|
||||
| 4 | cf-component feedback loop in mag-ratio canary — `experience_env_step` wrote `cf_reward_weighted = w_cf × cf_reward` (POST-controller-weight) to `rc[+1]`, which SP4 EMA fed into ISV[64] (`REWARD_CF_EMA_INDEX`), which the canary read → `high w_cf → high slot 64 → high ratio[1] → controller raises w_cf` self-reinforcing loop | Single `rc[+1]` write site mistakenly used the controller-weighted value while the other 5 slots correctly wrote raw pre-weight values | `61b2fa962` (B1b bug-hunt fix-up #2) | "### SP11 B1b bug-hunt fix-up #2" |
|
||||
| 5 | Magnitude-asymmetric linear ratios — `winner_weight = mag[c] / Σ mag` always elected popart (intrinsically O(100) per fire vs cf/trail/micro/opp_cost/bonus O(0.1-2)). `w_pop` cap-bound saturation at MAX_WEIGHT=3.0 by ep1; mean(weights) below 1.0 because cap binds; sharpe collapsed | Linear ratio formula doesn't see through intrinsic-magnitude asymmetry — see new pearl `pearl_controller_amplifies_dominant_magnitude_trap` | `b3b4d0278` (z-score amendment) | This entry — see "Resolution" below |
|
||||
| 6 | Launch-order stale slots — `smoke-test-4rbv9` on `b3b4d0278` showed bit-identical `w_pop=2.000` at ep1 to pre-z-score B1b smoke. With `var[c] = sentinel-0` at ep1 the z-score formula degenerates to the linear-formula equivalent (`mag[c] × 1e6 / Σ mag × 1e6 = mag[c] / Σ mag`) | `launch_reward_component_ema_inplace` (populating ISV[64..68] mag + ISV[362..366] var for cf/trail/micro/opp_cost/bonus) ran at the C.2 site (line ~3707) AFTER `launch_sp11_mag_ratio_compute` at line 3465. Canary read sentinel-0 for c=1..5 → ratio[0]=1.0 → controller saturated | `fd24b5383` (launch-order) | "## SP11 B1b launch-order fix-up" entry above |
|
||||
|
||||
### Resolution — Bugs 5 + 6 (this Layer C close-out)
|
||||
|
||||
**Z-score normalization** (Bug 5, commit `b3b4d0278`). The mag-ratio
|
||||
canary kernel changed from `ratio[c] = mag[c] / Σ mag` to:
|
||||
|
||||
```
|
||||
z[c] = mag[c] / max(sqrt(var[c]), EPS_DIV)
|
||||
ratio[c] = z[c] / Σ z
|
||||
```
|
||||
|
||||
Each component contributes "shares of own scale" rather than "shares
|
||||
of absolute magnitude". The controller's redistribution logic now
|
||||
correctly amplifies the *currently-surprising* component, not the
|
||||
intrinsically-largest. New pearl
|
||||
`pearl_zscore_normalization_for_magnitude_asymmetric_signals` and
|
||||
`pearl_controller_amplifies_dominant_magnitude_trap` capture the
|
||||
generalisable lesson.
|
||||
|
||||
**Launch-order fix** (Bug 6, commit `fd24b5383`). Moved
|
||||
`launch_reward_component_ema_inplace` from line ~3707 to immediately
|
||||
before `launch_sp11_popart_component_ema` in the SP11 producer chain
|
||||
(line ~3441). The kernel itself unchanged — only the launch site.
|
||||
Other launches at the original C.2 site (`launch_trade_attempt_rate
|
||||
_ema_inplace`, `launch_plan_threshold_update_inplace`,
|
||||
`launch_seed_step_counter_update_inplace`,
|
||||
`launch_cql_alpha_seed_update_inplace`) STAY at C.2 — their consumers
|
||||
have no pre-canary timing constraint. New pearl
|
||||
`pearl_canary_input_freshness_launch_order` documents the producer-
|
||||
before-consumer audit discipline.
|
||||
|
||||
### ISV slot allocation — 6 new variance EMA slots
|
||||
|
||||
| Slot | Constant | Producer kernel | Consumer kernel |
|
||||
|---|---|---|---|
|
||||
| 361 | `REWARD_COMPONENT_VAR_EMA_BASE + 0` (popart variance) | `popart_component_ema_kernel.cu` (extended with Welford) → `apply_pearls_ad_kernel` | `reward_component_mag_ratio_compute_kernel.cu` (paired with mag at slot 360) |
|
||||
| 362 | `+ 1` (cf variance) | `reward_component_ema_kernel.cu` (extended with Welford) → `apply_pearls_ad_kernel` | same canary (paired with mag at slot 64) |
|
||||
| 363 | `+ 2` (trail variance) | same | same canary (paired with mag at slot 65) |
|
||||
| 364 | `+ 3` (micro variance) | same | same canary (paired with mag at slot 66) |
|
||||
| 365 | `+ 4` (opp_cost variance) | same | same canary (paired with mag at slot 67) |
|
||||
| 366 | `+ 5` (bonus variance) | same | same canary (paired with mag at slot 68) |
|
||||
|
||||
`SP11_SLOT_END` 361 → 367; `SP11_PRODUCER_COUNT` 21 → 27. Layout
|
||||
fingerprint shifted accordingly. State-reset registry: 5 new entries
|
||||
(`sp11_reward_component_var` covering slots [361..367) — Welford
|
||||
state per-component is paired with the magnitude EMA's existing
|
||||
sentinel-0 reset path; FoldReset sentinel 0 lets Pearl A's first
|
||||
observation define scale per-fold per `pearl_first_observation_bootstrap`).
|
||||
|
||||
Variance is per-fold (NOT cross-fold persistent) — each new fold's
|
||||
reward-component scale distribution may differ; resetting tracks the
|
||||
new fold's z-score normalisation correctly.
|
||||
|
||||
### Validation evidence
|
||||
|
||||
Three smokes track the recovery:
|
||||
|
||||
| Smoke ID | Commit | Result | Key metrics |
|
||||
|---|---|---|---|
|
||||
| `smoke-test-6wd2c` | `61b2fa962` (post Bug 1-4 fix-ups, pre z-score) | **KILLED ep1** | `w_pop=2.000`, `w_cf=0.661`, others=0.661; sharpe collapsed 10.7 → 2.4 inside ep1 |
|
||||
| `smoke-test-4rbv9` | `b3b4d0278` (z-score landed, launch-order BUG present) | **KILLED ep1** | Bit-identical `w_pop=2.000` to `6wd2c` — proves z-score was structurally a no-op (Bug 6 diagnostic signature) |
|
||||
| `smoke-test-gwfn8` | `fd24b5383` (z-score + launch-order both fixed) | **PASSED 15m11s** | 3 folds × 5 epochs; `w_pop` max=1.706 (down from 3.0 cap); `mean(weights) = 1.000 ± 0.015` across all 15 epochs; F1/F2 ep0 transitions clean (no MAX_WEIGHT saturation); F1 ep0 picked `w_mi=1.558` as largest (NOT popart) — proving adaptive component selection works through intrinsic-magnitude asymmetry |
|
||||
|
||||
Empirical comparison ep1 across smokes:
|
||||
|
||||
| Metric | `smoke-test-6wd2c` | `smoke-test-4rbv9` | `smoke-test-gwfn8` |
|
||||
|---|---|---|---|
|
||||
| `w_pop` ep1 | 2.000 | 2.000 (bit-identical) | < MAX (varies; max 1.706 across full run) |
|
||||
| `mean(weights)` ep1 | < 1.0 (cap binds) | < 1.0 (cap binds) | 1.000 ± 0.015 |
|
||||
| Sharpe trajectory | 10.7 → 2.4 (1 epoch) | unstable, killed | Stable across 15 epochs |
|
||||
| Adaptive winner | always popart | always popart | varies per regime (F1 ep0: `w_mi`) |
|
||||
|
||||
### Verification commands (HEAD `fd24b5383`)
|
||||
|
||||
- `SQLX_OFFLINE=true cargo check -p ml --lib` — clean
|
||||
- `SQLX_OFFLINE=true cargo test -p ml --lib trainers::dqn::monitors::reward_component_monitor` — 3/3 passing
|
||||
- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — passes (variance-slot block above SP9 region)
|
||||
- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset_registry` — 4/4 contract tests pass
|
||||
|
||||
### Per-pearl provenance
|
||||
|
||||
- `pearl_first_observation_bootstrap.md` — variance EMAs reset to
|
||||
sentinel-0 per-fold; first observation defines z-score scale.
|
||||
- `pearl_no_partial_refactor.md` — variance slot allocation +
|
||||
producer Welford extension + canary z-score + state-reset registry
|
||||
entries land in one atomic commit per fix.
|
||||
- `pearl_isv_for_adaptive_bounds.md` — `EPS_DIV=1e-6` is the only
|
||||
Invariant-1 anchor in the new code; every other quantity (variance
|
||||
EMAs, magnitudes, ratios) is signal-driven.
|
||||
- `pearl_no_atomicadd.md` — Welford variance computed via single-pass
|
||||
two-tree-reduce inside the existing producer kernel, no atomicAdd.
|
||||
- `pearl_no_cpu_compute_strict.md` — variance compute lives in GPU
|
||||
producer; consumer reads via mapped-pinned ISV.
|
||||
- `pearl_zscore_normalization_for_magnitude_asymmetric_signals.md`
|
||||
(NEW) — z-score makes the ratio formula scale-invariant; one extra
|
||||
Welford-variance per existing magnitude EMA.
|
||||
- `pearl_canary_input_freshness_launch_order.md` (NEW) — producers
|
||||
feeding a canary MUST launch before it; bit-identical-to-linear is
|
||||
the smoking-gun diagnostic for stale-slot leakage.
|
||||
- `pearl_controller_amplifies_dominant_magnitude_trap.md` (NEW) —
|
||||
generic recipe: any "winner_weight = ratio" controller with
|
||||
asymmetric input scales needs scale normalization first.
|
||||
|
||||
### Open follow-up
|
||||
|
||||
Within-fold sharpe degradation across epochs (e.g., F0 sharpe slips
|
||||
across ep0..ep4 within `smoke-test-gwfn8`) is a **separate residual**
|
||||
not part of B1b smoke-recovery scope. The B1b smoke-recovery success
|
||||
criterion was "controller engages adaptively without saturating cap +
|
||||
no early-epoch collapse"; that criterion is met. Within-fold residual
|
||||
degradation will be triaged as a follow-up task post-T10 50-epoch
|
||||
validation, scoped against fold-internal exploration/exploitation
|
||||
dynamics (Kelly engagement at fold mid-epoch, NoisyNet σ schedule,
|
||||
saboteur intensity drift), not against the controller.
|
||||
|
||||
Reference in New Issue
Block a user