From 2b91f587b76e5784cd6b0121faeb53c7646e6d43 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sat, 9 May 2026 00:39:16 +0200 Subject: [PATCH] plan(sp18 v2): 22 fold-reset registry entries + dispatch arms (PP.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds FoldReset registry coverage for all 22 SP18 ISV slots [483..505) landed in PP.2 (10 D-leg + 12 B-leg), in lockstep with matching match arms in `reset_named_state` per the existing `every_fold_and_soft_reset_entry_has_dispatch_arm` contract test (which catches the "add registry entry, forget dispatch arm → runtime panic at fold boundary" pattern surfaced twice historically — SP5 #281, SP7 T7 commit 6e479c55c). Sentinels match `sp14_isv_slots.rs` constants (single source of truth): D-leg POS/NEG caps → 5.0 / -10.0 (SP14 P0-A REWARD_*_CAP_ADAPTIVE pattern) D-leg HRC Welford → 0.0 (SENTINEL_WELFORD_ZERO) D-leg DIAG/FIRE → 0.0 (Pearl-A direct-replace) B-leg HEALTH_DIAG → 0.0 (Pearl-A) B-leg POPART_RESET → 1.0 (one-shot per B-DD11 — see below) B-leg TDB Welford → 0.0 (SENTINEL_WELFORD_ZERO) B-leg RESERVED → 0.0 Slot 497 POPART_RESET_FLAG semantics: each fold start, FoldReset writes 1.0; on the first epoch of that fold the Phase 5 PopArt-reset consumer reads 1.0, resets PopArt slot 63 EMA to identity normalization, and writes 0.0 back — one-shot per fold. Cheap insurance against the 1+ epoch PopArt adaptation lag when switching `q_next` source from rewards-distributed to Q-distributed in the Phase 5 atomic refactor. New lock test `sp18_fold_reset_entries_present` asserts all 22 entries exist with FoldReset category and a description containing "SP18 D-leg" or "SP18 B-leg" prefix marker. Existing `every_fold_and_soft_reset_entry_has_dispatch_arm` catches drift in either direction (registry adds without dispatch arms, or dispatch arms without registry entries). Audit doc updated per Invariant 7 with PP.3 entry + full sentinel mapping table. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md Co-Authored-By: Claude Opus 4.7 (1M context) --- .../src/trainers/dqn/state_reset_registry.rs | 192 +++++++++++++ .../src/trainers/dqn/trainer/training_loop.rs | 264 ++++++++++++++++++ docs/dqn-wire-up-audit.md | 30 ++ 3 files changed, 486 insertions(+) diff --git a/crates/ml/src/trainers/dqn/state_reset_registry.rs b/crates/ml/src/trainers/dqn/state_reset_registry.rs index f8929792c..768848470 100644 --- a/crates/ml/src/trainers/dqn/state_reset_registry.rs +++ b/crates/ml/src/trainers/dqn/state_reset_registry.rs @@ -1892,6 +1892,134 @@ impl StateResetRegistry { category: ResetCategory::FoldReset, description: "ISV[ADVANTAGE_CLIP_BOUND_INDEX=482] — SP17 adaptive |A_centered| upper bound, producer-tracked from `p99(|A_centered|) × ADVANTAGE_CLIP_SAFETY_FACTOR=1.5` (mirrors SP14 P0-A REWARD_POS_CAP producer pattern). FoldReset sentinel SENTINEL_ADVANTAGE_CLIP_BOUND=1.0 — Pearl-A bootstrap interpreted as cold-start `no effective clipping` (1.0 ≫ typical |A_centered| early in training when A → 0; clip is a no-op until the first valid observation lands a real bound). α=ADVANTAGE_CLIP_EMA_ALPHA=0.01 slow per-fold cadence — the bound shouldn't track per-batch noise. Bilateral clamp to `[ADVANTAGE_CLIP_BOUND_MIN=0.1, ADVANTAGE_CLIP_BOUND_MAX=100.0]` per `pearl_symmetric_clamp_audit.md` — Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`: floor 0.1 below which centered advantage is essentially noise (clipping would erase the action signal); ceiling 100.0 above which a single adversarial outlier dominates per-batch EMAs. NEVER hardcoded — adaptive bound. Producer + consumer wire in Phase 1-3 of SP17 (atomic) — not yet wired. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md.", }, + // ── SP18 v2 Pre-Phase Task PP.3 (2026-05-08): combined Hold-attractor fix ── + // 22 ISV slots [483..505) split across two atomic legs per the v2 + // combined plan. D-leg [483..493) — 10 slots driving the structural + // Hold opportunity-cost reward chain (replaces SP13 P0a + SP16 P2 + + // SP16 T3 reactive per-bar Hold-cost chain per DD7=c). B-leg + // [493..505) — 12 slots driving the TD(λ) Q(s') bootstrap chain + // (replaces self-bootstrap q_next = rewards at gpu_experience_ + // collector.rs:4143 with real per-branch DDQN target-Q via target- + // net forward pass on next_states). Producer + consumer kernels + // land in Phases 2-7 atomic per `feedback_no_partial_refactor`; + // this Pre-Phase commit is additive infrastructure only. All + // sentinels are Pearl-A first-observation bootstrap except slot + // 497 POPART_RESET_FLAG (sentinel 1.0 — one-shot per B-DD11, + // gates per-fold PopArt slot 63 EMA reset at SP18 deployment + // boundary). Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md. + RegistryEntry { + name: "sp18_hold_reward_pos_cap", + category: ResetCategory::FoldReset, + description: "ISV[HOLD_REWARD_POS_CAP_INDEX=483] — SP18 D-leg adaptive upper bound on the structural Hold opportunity-cost reward (Hold during a losing bar). Producer-tracked from `p99(|step_ret|)` over Long/Short trade closes × HOLD_REWARD_CAP_SAFETY_FACTOR=1.5 (DD3=b) by `hold_reward_cap_update_kernel` (lands in Phase 3) with Wiener-optimal α from Welford accumulators in [487..493) per `pearl_wiener_optimal_adaptive_alpha`. FoldReset sentinel SENTINEL_HOLD_REWARD_POS_CAP=5.0 — Pearl-A first-observation bootstrap matching the SP14 P0-A REWARD_POS_CAP_ADAPTIVE pattern for bit-identical cold-start before the cap producer's first valid launch. Consumer: `compute_sp18_hold_opportunity_cost` device function in `trade_physics.cuh` (added Phase 2) consumes pos_cap as the upper bound on the opportunity-cost reward returned to the 3 Hold-cost subtraction sites in `experience_kernels.cu`. Bounds [HOLD_REWARD_POS_CAP_MIN_BOUND=0.5, HOLD_REWARD_POS_CAP_MAX_BOUND=50.0] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`. Producer + consumer wire in Phase 2-3 of SP18 (atomic 3-site migration) — not yet wired in this Pre-Phase commit. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md.", + }, + RegistryEntry { + name: "sp18_hold_reward_neg_cap", + category: ResetCategory::FoldReset, + description: "ISV[HOLD_REWARD_NEG_CAP_INDEX=484] — SP18 D-leg adaptive lower bound on the structural Hold opportunity-cost reward (Hold during a winning bar — the missed-winner case). Producer writes NEG = -2 × POS (DD5=b mirrored 2:1 loss-aversion ratio matching SP14 P0-A REWARD_POS/NEG_CAP — preserves the Kahneman/Tversky prospect-theory asymmetry: missing a winner is the bigger pain, drives the policy toward engagement). FoldReset sentinel SENTINEL_HOLD_REWARD_NEG_CAP=-10.0 — Pearl-A first-observation bootstrap matching SP14 P0-A REWARD_NEG_CAP_ADAPTIVE for bit-identical cold-start. Same producer kernel as slot 483 (`hold_reward_cap_update_kernel`) writes both slots in one launch. Consumer: `compute_sp18_hold_opportunity_cost` device function applies the asymmetric clamp via `compute_asymmetric_capped_pnl(-2 × vol_normalized_log_return, neg_cap, pos_cap)`. Producer + consumer wire in Phase 2-3 of SP18 (atomic) — not yet wired. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md.", + }, + RegistryEntry { + name: "sp18_hold_reward_decomp_diag", + category: ResetCategory::FoldReset, + description: "ISV[HOLD_REWARD_DECOMP_DIAG_INDEX=485] — SP18 D-leg HEALTH_DIAG observability slot for the per-action-bin reward decomposition diagnostic introduced in Phase 0 (per-bar EMAs of r_micro / r_opp_cost / r_popart / fire_rate per Hold/Long/Short/Flat action bin, surfaced by the new `reward_decomp_diag_kernel.cu`). FoldReset sentinel SENTINEL_HOLD_REWARD_DIAG=0.0 — Pearl-A direct-replace on first valid observation. Diagnostic only — does NOT modify any consumer path. Phase 0 KILL CRITERION evaluator reads this slot to abort D-leg if the per-step Hold-cost magnitude is already disproportionately large pre-fix (the controller is winning). Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § Phase 0.", + }, + RegistryEntry { + name: "sp18_hold_opp_cost_fire_rate_ema", + category: ResetCategory::FoldReset, + description: "ISV[HOLD_OPP_COST_FIRE_RATE_EMA_INDEX=486] — SP18 D-leg engagement canary: fraction of bars on which the new Hold opportunity-cost reward fires (non-zero) — should be ≈ Hold% itself once the structural reward is wired. FoldReset sentinel SENTINEL_HOLD_OPP_FIRE_RATE=0.0 — Pearl-A direct-replace on first valid observation. Sanity-checks the structural-reward producer is actually emitting non-zero on Hold bars (pre-Phase-3 sentinel-bootstrap pre-fires the per-bar zero, post-Phase-3 should track the realised Hold rate). Producer kernel embeds the fire-rate EMA update into the same launch that consumes the new opportunity-cost reward. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § Phase 3.", + }, + // ── HRC_* Welford accumulators (D-leg, mirrors SP16 T3 HCS_* layout) + RegistryEntry { + name: "sp18_hrc_target_mean", + category: ResetCategory::FoldReset, + description: "ISV[HRC_TARGET_MEAN_INDEX=487] — SP18 D-leg Welford running mean of target_cap for the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; first valid observation initialises via Welford increment. Combined with the cold-start fallback (sample_count<3 → α=1.0 REPLACE), the first 3 epochs of each fold force REPLACE; from epoch 3 onward Wiener-optimal α emerges from Welford state per `pearl_wiener_optimal_adaptive_alpha`. Mirrors the SP16 T3 HCS_TARGET_MEAN_INDEX=462 pattern.", + }, + RegistryEntry { + name: "sp18_hrc_target_m2", + category: ResetCategory::FoldReset, + description: "ISV[HRC_TARGET_M2_INDEX=488] — SP18 D-leg Welford ΣΔ² (sample-variance accumulator) for target_cap in the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; tracks variance of `target_cap` series across each fold. Used to compute `sample_var = M2 / (n - 1)` once `n ≥ 3`. Mirrors HCS_TARGET_M2_INDEX=463.", + }, + RegistryEntry { + name: "sp18_hrc_diff_mean", + category: ResetCategory::FoldReset, + description: "ISV[HRC_DIFF_MEAN_INDEX=489] — SP18 D-leg Welford running mean of consecutive `target_cap` one-step diffs in the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; first diff observation lands when sample_count reaches 2. Mirrors HCS_DIFF_MEAN_INDEX=464.", + }, + RegistryEntry { + name: "sp18_hrc_diff_m2", + category: ResetCategory::FoldReset, + description: "ISV[HRC_DIFF_M2_INDEX=490] — SP18 D-leg Welford ΣΔ² for the consecutive-diff series in the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; provides `diff_var` half of the Wiener formula α = diff_var / (diff_var + sample_var + ε). Mirrors HCS_DIFF_M2_INDEX=465.", + }, + RegistryEntry { + name: "sp18_hrc_prev_target", + category: ResetCategory::FoldReset, + description: "ISV[HRC_PREV_TARGET_INDEX=491] — SP18 D-leg previous-epoch `target_cap` for the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; written every launch from the freshly-computed `target_cap` so the next launch's diff calculation has a basis. Mirrors HCS_PREV_TARGET_INDEX=466.", + }, + RegistryEntry { + name: "sp18_hrc_sample_count", + category: ResetCategory::FoldReset, + description: "ISV[HRC_SAMPLE_COUNT_INDEX=492] — SP18 D-leg Welford observation counter for the `hold_reward_cap_update_kernel` producer. Sentinel 0.0; incremented every launch (excluding cold-start sentinel-guard early-return). The cold-start fallback (sample_count < WELFORD_MIN_SAMPLE_COUNT=3 → α = 1.0 REPLACE) keeps the producer responsive in epochs 1-2 of each fold while Welford state stabilises. Mirrors HCS_SAMPLE_COUNT_INDEX=467.", + }, + // ── B-leg observability + reset gate slots [493..505) ───────────── + RegistryEntry { + name: "sp18_td_error_mag_ema", + category: ResetCategory::FoldReset, + description: "ISV[TD_ERROR_MAG_EMA_INDEX=493] — SP18 B-leg HEALTH_DIAG observable: `avg(|TD-error|)` computed post-fix from the per-batch L1 norm of `td_errors` (already computed for PER priority recomputation at training time). FoldReset sentinel SENTINEL_TD_ERROR_MAG_EMA=0.0 — Pearl-A direct-replace on first valid observation. The B-DD9 ratio gate compares post-fix / pre-fix at HD0–HD1 (target [0.5, 5.0]) — ratio outside that range = bootstrap change blew up loss magnitude. Producer + consumer wire in Phase 0 (diagnostic emit) + Phase 8 (combined HEALTH_DIAG emit) of SP18 — not yet wired in this Pre-Phase commit. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § B-DD9 + B-DD12.", + }, + RegistryEntry { + name: "sp18_q_next_target_p99", + category: ResetCategory::FoldReset, + description: "ISV[Q_NEXT_TARGET_P99_INDEX=494] — SP18 B-leg HEALTH_DIAG observable: p99 of the new target-Q bootstrap value (the per-branch DDQN sum-of-argmaxed target-Q across dir/mag/ord/urg branches that replaces `q_next = rewards`). FoldReset sentinel 0.0 — Pearl-A direct-replace. Bound check on the new path (catches blow-up if target net diverges). Producer + consumer wire in Phase 4-5 of SP18 — not yet wired. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § B-DD12.", + }, + RegistryEntry { + name: "sp18_q_next_minus_reward_p99", + category: ResetCategory::FoldReset, + description: "ISV[Q_NEXT_MINUS_REWARD_P99_INDEX=495] — SP18 B-leg HEALTH_DIAG observable: p99 of `|q_next_target − rewards|`. FoldReset sentinel 0.0 — Pearl-A direct-replace. Sanity check that the new bootstrap is doing real work — should be O(1) post-fix (NOT O(0) which would mean q_next_target ≈ rewards, i.e. the fix is doing nothing; NOT O(100) which would mean it's blowing up). Pre-fix this slot would be ~0 because q_next *was* rewards. Producer + consumer wire in Phase 4-5 of SP18 — not yet wired. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § B-DD12.", + }, + RegistryEntry { + name: "sp18_v_share_trend_diag", + category: ResetCategory::FoldReset, + description: "ISV[V_SHARE_TREND_DIAG_INDEX=496] — SP18 B-leg HEALTH_DIAG observable: slope of the SP17 V_SHARE EMA (slots [478..482)) across the last 4 epochs, computed as `(EMA[now] - EMA[now-4]) / 4`. FoldReset sentinel 0.0 — Pearl-A direct-replace. The B-leg synergy hypothesis (pearl_TBD: bootstrap-bias drives v_share elevation) predicts v_share monotone HD0→HD4 pre-fix; post-fix the slope should *flatten* or *reverse*. Slope < 0 OR slope ≤ 0.05 with v_share[HD3] > 0.7 = synergy hypothesis confirmed; otherwise B-leg KILL CRITERION fires. Producer + consumer wire in Phase 0 (diagnostic emit) of SP18 — not yet wired. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § Phase 0.", + }, + RegistryEntry { + name: "sp18_popart_reset_flag", + category: ResetCategory::FoldReset, + description: "ISV[POPART_RESET_FLAG_INDEX=497] — SP18 B-leg per-fold PopArt EMA reset gate per B-DD11. FoldReset sentinel SENTINEL_POPART_RESET_FLAG=1.0 (NOT 0.0 — this is the only non-zero diagnostic sentinel in the 22-slot SP18 block). Each fold start, the FoldReset machinery writes 1.0 here; on the first epoch of that fold the PopArt-reset consumer kernel (Phase 5) reads 1.0, clamps PopArt slot 63 EMA back to identity normalization (μ=0, σ=1), and writes 0.0 back to slot 497 — making it one-shot per fold. Switching `q_next` source from rewards-distributed to Q-distributed (Phase 5 atomic refactor) shifts the input distribution to PopArt; PopArt's α-EMA will adapt but may take 1+ epochs. The reset is cheap insurance. Producer + consumer wire in Phase 5 of SP18 — not yet wired in this Pre-Phase commit. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md § B-DD11.", + }, + // ── TDB_* Welford accumulators (B-leg, mirrors HRC_* layout) ────── + RegistryEntry { + name: "sp18_tdb_target_mean", + category: ResetCategory::FoldReset, + description: "ISV[TDB_TARGET_MEAN_INDEX=498] — SP18 B-leg Welford running mean of q_next_target for the post-fix Wiener-α blend on the TD-error magnitude EMA. Sentinel 0.0; same Welford algorithm as the D-leg HRC counterpart at slot 487 but tracks q_next_target instead of target_cap. Combined with the cold-start fallback (sample_count<3 → α=1.0), the first 3 epochs of each fold force REPLACE; from epoch 3 onward Wiener-optimal α emerges. Producer wire in Phase 4 of SP18 — not yet wired.", + }, + RegistryEntry { + name: "sp18_tdb_target_m2", + category: ResetCategory::FoldReset, + description: "ISV[TDB_TARGET_M2_INDEX=499] — SP18 B-leg Welford ΣΔ² for q_next_target. Sentinel 0.0; mirrors HRC_TARGET_M2_INDEX=488 and HCS_TARGET_M2_INDEX=463 modulo the producer it serves. Producer wire in Phase 4 — not yet wired.", + }, + RegistryEntry { + name: "sp18_tdb_diff_mean", + category: ResetCategory::FoldReset, + description: "ISV[TDB_DIFF_MEAN_INDEX=500] — SP18 B-leg Welford running mean of consecutive q_next_target diffs. Sentinel 0.0; mirrors HRC_DIFF_MEAN_INDEX=489 and HCS_DIFF_MEAN_INDEX=464. Producer wire in Phase 4 — not yet wired.", + }, + RegistryEntry { + name: "sp18_tdb_diff_m2", + category: ResetCategory::FoldReset, + description: "ISV[TDB_DIFF_M2_INDEX=501] — SP18 B-leg Welford ΣΔ² for the consecutive-diff series. Sentinel 0.0; provides `diff_var` half of the Wiener formula. Mirrors HRC_DIFF_M2_INDEX=490. Producer wire in Phase 4 — not yet wired.", + }, + RegistryEntry { + name: "sp18_tdb_prev_target", + category: ResetCategory::FoldReset, + description: "ISV[TDB_PREV_TARGET_INDEX=502] — SP18 B-leg previous-epoch q_next_target. Sentinel 0.0; mirrors HRC_PREV_TARGET_INDEX=491. Producer wire in Phase 4 — not yet wired.", + }, + RegistryEntry { + name: "sp18_tdb_sample_count", + category: ResetCategory::FoldReset, + description: "ISV[TDB_SAMPLE_COUNT_INDEX=503] — SP18 B-leg Welford observation counter. Sentinel 0.0; cold-start fallback (sample_count < 3 → α = 1.0) ensures the first 3 epochs of each fold force REPLACE so the Welford-derived α takes over from epoch 3 onward. Mirrors HRC_SAMPLE_COUNT_INDEX=492 and HCS_SAMPLE_COUNT_INDEX=467. Producer wire in Phase 4 — not yet wired.", + }, + RegistryEntry { + name: "sp18_b_leg_reserved", + category: ResetCategory::FoldReset, + description: "ISV[SP18_B_LEG_RESERVED_INDEX=504] — SP18 B-leg RESERVED slot for follow-up if needed (e.g. distributional flavour of B-DD8 if 5-epoch smoke shows atom-distribution drift). Sentinel 0.0. No producer or consumer in the Pre-Phase scope — preserves the 12-slot B-leg block aligned and the layout fingerprint stable.", + }, ]; Self { entries } } @@ -2035,4 +2163,68 @@ mod tests { missing ); } + + /// Lock SP18 v2 Pre-Phase Task PP.3: 22 fold-reset entries land for the + /// combined Hold-attractor fix slots [483..505). 10 D-leg + 12 B-leg. + /// Asserts each by name is present with FoldReset category and a + /// description containing "SP18 D-leg" or "SP18 B-leg" (sentinel / + /// description content checked separately for clarity). + #[test] + fn sp18_fold_reset_entries_present() { + let r = StateResetRegistry::new(); + let by_name: std::collections::HashMap<&str, &RegistryEntry> = + r.all().iter().map(|e| (e.name, e)).collect(); + + // 22 expected SP18 entries — D-leg 10 + B-leg 12. + let expected: &[(&str, &str)] = &[ + // D-leg [483..493) — 10 slots + ("sp18_hold_reward_pos_cap", "SP18 D-leg"), + ("sp18_hold_reward_neg_cap", "SP18 D-leg"), + ("sp18_hold_reward_decomp_diag", "SP18 D-leg"), + ("sp18_hold_opp_cost_fire_rate_ema", "SP18 D-leg"), + ("sp18_hrc_target_mean", "SP18 D-leg"), + ("sp18_hrc_target_m2", "SP18 D-leg"), + ("sp18_hrc_diff_mean", "SP18 D-leg"), + ("sp18_hrc_diff_m2", "SP18 D-leg"), + ("sp18_hrc_prev_target", "SP18 D-leg"), + ("sp18_hrc_sample_count", "SP18 D-leg"), + // B-leg [493..505) — 12 slots + ("sp18_td_error_mag_ema", "SP18 B-leg"), + ("sp18_q_next_target_p99", "SP18 B-leg"), + ("sp18_q_next_minus_reward_p99", "SP18 B-leg"), + ("sp18_v_share_trend_diag", "SP18 B-leg"), + ("sp18_popart_reset_flag", "SP18 B-leg"), + ("sp18_tdb_target_mean", "SP18 B-leg"), + ("sp18_tdb_target_m2", "SP18 B-leg"), + ("sp18_tdb_diff_mean", "SP18 B-leg"), + ("sp18_tdb_diff_m2", "SP18 B-leg"), + ("sp18_tdb_prev_target", "SP18 B-leg"), + ("sp18_tdb_sample_count", "SP18 B-leg"), + ("sp18_b_leg_reserved", "SP18 B-leg"), + ]; + + assert_eq!(expected.len(), 22, "expected 22 SP18 entries (10 D-leg + 12 B-leg)"); + + for (name, leg_marker) in expected { + let entry = by_name.get(name).unwrap_or_else(|| { + panic!( + "SP18 v2 PP.3: registry missing entry '{}'. Add a `RegistryEntry` to \ + `StateResetRegistry::new()` AND a dispatch arm in \ + `trainer/training_loop.rs::reset_named_state` (else \ + `every_fold_and_soft_reset_entry_has_dispatch_arm` fires too).", + name + ) + }); + assert_eq!( + entry.category, + ResetCategory::FoldReset, + "SP18 entry '{}' must be FoldReset (got {:?})", name, entry.category + ); + assert!( + entry.description.contains(leg_marker), + "SP18 entry '{}' description must contain '{}'; got: {}", + name, leg_marker, entry.description + ); + } + } } diff --git a/crates/ml/src/trainers/dqn/trainer/training_loop.rs b/crates/ml/src/trainers/dqn/trainer/training_loop.rs index 562188e9d..916469a62 100644 --- a/crates/ml/src/trainers/dqn/trainer/training_loop.rs +++ b/crates/ml/src/trainers/dqn/trainer/training_loop.rs @@ -8903,6 +8903,270 @@ impl DQNTrainer { ); } } + // ── SP18 v2 Pre-Phase Task PP.3 (2026-05-08): combined Hold-attractor fix ── + // 22 ISV slots [483..505) split across two atomic legs per the v2 + // combined plan. D-leg [483..493) — 10 slots driving the structural + // Hold opportunity-cost reward chain. B-leg [493..505) — 12 slots + // driving the TD(λ) Q(s') bootstrap chain. All FoldReset with + // Pearl-A first-observation bootstrap sentinels except slot 497 + // POPART_RESET_FLAG (sentinel 1.0 — one-shot per B-DD11, gates + // the per-fold PopArt slot 63 EMA reset at SP18 deployment + // boundary; the Phase 5 reset consumer reads 1.0, performs + // the PopArt reset, and writes 0.0 back, making it one-shot per + // fold). Producer + consumer kernels land in Phases 2-7 atomic + // per `feedback_no_partial_refactor`; this Pre-Phase commit is + // additive infrastructure only. Plan: + // docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md. + "sp18_hold_reward_pos_cap" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HOLD_REWARD_POS_CAP_INDEX, SENTINEL_HOLD_REWARD_POS_CAP, + }; + fused.trainer().write_isv_signal_at( + HOLD_REWARD_POS_CAP_INDEX, + SENTINEL_HOLD_REWARD_POS_CAP, + ); + } + } + "sp18_hold_reward_neg_cap" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HOLD_REWARD_NEG_CAP_INDEX, SENTINEL_HOLD_REWARD_NEG_CAP, + }; + fused.trainer().write_isv_signal_at( + HOLD_REWARD_NEG_CAP_INDEX, + SENTINEL_HOLD_REWARD_NEG_CAP, + ); + } + } + "sp18_hold_reward_decomp_diag" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HOLD_REWARD_DECOMP_DIAG_INDEX, SENTINEL_HOLD_REWARD_DIAG, + }; + fused.trainer().write_isv_signal_at( + HOLD_REWARD_DECOMP_DIAG_INDEX, + SENTINEL_HOLD_REWARD_DIAG, + ); + } + } + "sp18_hold_opp_cost_fire_rate_ema" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HOLD_OPP_COST_FIRE_RATE_EMA_INDEX, SENTINEL_HOLD_OPP_FIRE_RATE, + }; + fused.trainer().write_isv_signal_at( + HOLD_OPP_COST_FIRE_RATE_EMA_INDEX, + SENTINEL_HOLD_OPP_FIRE_RATE, + ); + } + } + // HRC_* Welford accumulators — D-leg, mirrors SP16 T3 HCS_* layout. + // All 6 sentinels = 0.0 (SENTINEL_WELFORD_ZERO). + "sp18_hrc_target_mean" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_TARGET_MEAN_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_TARGET_MEAN_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_hrc_target_m2" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_TARGET_M2_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_TARGET_M2_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_hrc_diff_mean" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_DIFF_MEAN_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_DIFF_MEAN_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_hrc_diff_m2" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_DIFF_M2_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_DIFF_M2_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_hrc_prev_target" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_PREV_TARGET_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_PREV_TARGET_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_hrc_sample_count" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + HRC_SAMPLE_COUNT_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + HRC_SAMPLE_COUNT_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + // ── B-leg observability + reset gate slots [493..505) ── + "sp18_td_error_mag_ema" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TD_ERROR_MAG_EMA_INDEX, SENTINEL_TD_ERROR_MAG_EMA, + }; + fused.trainer().write_isv_signal_at( + TD_ERROR_MAG_EMA_INDEX, + SENTINEL_TD_ERROR_MAG_EMA, + ); + } + } + "sp18_q_next_target_p99" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + Q_NEXT_TARGET_P99_INDEX, SENTINEL_Q_NEXT_TARGET_P99, + }; + fused.trainer().write_isv_signal_at( + Q_NEXT_TARGET_P99_INDEX, + SENTINEL_Q_NEXT_TARGET_P99, + ); + } + } + "sp18_q_next_minus_reward_p99" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + Q_NEXT_MINUS_REWARD_P99_INDEX, SENTINEL_Q_NEXT_MINUS_REWARD_P99, + }; + fused.trainer().write_isv_signal_at( + Q_NEXT_MINUS_REWARD_P99_INDEX, + SENTINEL_Q_NEXT_MINUS_REWARD_P99, + ); + } + } + "sp18_v_share_trend_diag" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + V_SHARE_TREND_DIAG_INDEX, SENTINEL_V_SHARE_TREND_DIAG, + }; + fused.trainer().write_isv_signal_at( + V_SHARE_TREND_DIAG_INDEX, + SENTINEL_V_SHARE_TREND_DIAG, + ); + } + } + // POPART_RESET_FLAG sentinel = 1.0 (one-shot per B-DD11). At + // each fold start the sentinel is rewritten to 1.0; the Phase 5 + // PopArt-reset consumer kernel reads 1.0, resets PopArt slot 63 + // EMA to identity normalization, and writes 0.0 back, making + // it one-shot per fold. + "sp18_popart_reset_flag" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + POPART_RESET_FLAG_INDEX, SENTINEL_POPART_RESET_FLAG, + }; + fused.trainer().write_isv_signal_at( + POPART_RESET_FLAG_INDEX, + SENTINEL_POPART_RESET_FLAG, + ); + } + } + // TDB_* Welford accumulators — B-leg, mirrors HRC_* layout. + // All 6 sentinels = 0.0 (SENTINEL_WELFORD_ZERO). + "sp18_tdb_target_mean" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_TARGET_MEAN_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_TARGET_MEAN_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_tdb_target_m2" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_TARGET_M2_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_TARGET_M2_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_tdb_diff_mean" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_DIFF_MEAN_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_DIFF_MEAN_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_tdb_diff_m2" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_DIFF_M2_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_DIFF_M2_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_tdb_prev_target" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_PREV_TARGET_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_PREV_TARGET_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_tdb_sample_count" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::{ + TDB_SAMPLE_COUNT_INDEX, SENTINEL_WELFORD_ZERO, + }; + fused.trainer().write_isv_signal_at( + TDB_SAMPLE_COUNT_INDEX, + SENTINEL_WELFORD_ZERO, + ); + } + } + "sp18_b_leg_reserved" => { + if let Some(ref fused) = self.fused_ctx { + use crate::cuda_pipeline::sp14_isv_slots::SP18_B_LEG_RESERVED_INDEX; + fused.trainer().write_isv_signal_at( + SP18_B_LEG_RESERVED_INDEX, + 0.0, + ); + } + } // SP15 Phase 1.2 (2026-05-06): cost-net sharpe slots. // OFI_IMPACT_LAMBDA_INDEX=407 is an Invariant-1 anchor (NOT // a stateful EMA) — rewrite the constructor's value at fold diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index ea3ff6f5b..b905d8975 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -2,6 +2,36 @@ **Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7. +## 2026-05-08 — SP18 v2 Pre-Phase Task PP.3: 22 fold-reset registry entries + dispatch arms + +Pre-Phase scaffolding completing the FoldReset wire-up for the 22 SP18 ISV slots from PP.2: + +- `crates/ml/src/trainers/dqn/state_reset_registry.rs` — 22 new `RegistryEntry` records (10 D-leg + 12 B-leg) with FoldReset category. Each entry's `description` includes the slot index, sentinel value, and the prefix marker "SP18 D-leg" or "SP18 B-leg" (asserted by the new `sp18_fold_reset_entries_present` lock test). +- `crates/ml/src/trainers/dqn/trainer/training_loop.rs::reset_named_state` — 22 new match arms, each rewriting the slot to its sentinel via `fused.trainer().write_isv_signal_at(, )`. Without these arms the existing `every_fold_and_soft_reset_entry_has_dispatch_arm` contract test fires (catches the "add registry entry, forget dispatch arm → runtime panic at fold boundary" bug pattern from SP5 #281 + SP7 T7). + +**Sentinel mapping** (matches `crates/ml/src/cuda_pipeline/sp14_isv_slots.rs` — single source of truth): + +| Slot | Name | Sentinel const | Value | +|---|---|---|---| +| 483 | sp18_hold_reward_pos_cap | SENTINEL_HOLD_REWARD_POS_CAP | 5.0 | +| 484 | sp18_hold_reward_neg_cap | SENTINEL_HOLD_REWARD_NEG_CAP | -10.0 | +| 485 | sp18_hold_reward_decomp_diag | SENTINEL_HOLD_REWARD_DIAG | 0.0 | +| 486 | sp18_hold_opp_cost_fire_rate_ema | SENTINEL_HOLD_OPP_FIRE_RATE | 0.0 | +| 487..492 | sp18_hrc_* (6 Welford slots) | SENTINEL_WELFORD_ZERO | 0.0 | +| 493 | sp18_td_error_mag_ema | SENTINEL_TD_ERROR_MAG_EMA | 0.0 | +| 494 | sp18_q_next_target_p99 | SENTINEL_Q_NEXT_TARGET_P99 | 0.0 | +| 495 | sp18_q_next_minus_reward_p99 | SENTINEL_Q_NEXT_MINUS_REWARD_P99 | 0.0 | +| 496 | sp18_v_share_trend_diag | SENTINEL_V_SHARE_TREND_DIAG | 0.0 | +| 497 | sp18_popart_reset_flag | SENTINEL_POPART_RESET_FLAG | **1.0** | +| 498..503 | sp18_tdb_* (6 Welford slots) | SENTINEL_WELFORD_ZERO | 0.0 | +| 504 | sp18_b_leg_reserved | (literal 0.0) | 0.0 | + +**Slot 497 POPART_RESET_FLAG semantics** (B-DD11): Sentinel = 1.0 (the only non-zero diagnostic sentinel in the 22-slot block). At each fold start, FoldReset writes 1.0; on the first epoch of that fold the Phase 5 PopArt-reset consumer kernel reads 1.0, resets PopArt slot 63 EMA back to identity normalization (μ=0, σ=1), and writes 0.0 back to slot 497 — making the reset one-shot per fold. Switching `q_next` source from rewards-distributed to Q-distributed (Phase 5 atomic refactor) shifts the PopArt input distribution; the per-fold reset is cheap insurance against the 1+ epoch adaptation lag. + +**Atomicity envelope**: Pre-Phase scaffolding only — registry + dispatch arms wired in lockstep per the `every_fold_and_soft_reset_entry_has_dispatch_arm` contract; no producer kernel, no HEALTH_DIAG emit, no consumer migration. Phase 0–7 lands the actual wiring. + +**Plan**: `docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md` § Pre-Phase Task PP.3. + ## 2026-05-08 — SP18 v2 Pre-Phase Task PP.2: 22 ISV slots [483..505) D-leg + B-leg Pre-Phase scaffolding for the combined SP18 v2 plan: allocates 22 ISV slots split across two atomic legs per `feedback_no_partial_refactor`.