spec(sp19+20): apply Q1(b) — Hold-reward EMA for Q-scale comparability

Q1 design decision (b): add explicit hold_reward_ema to center the per-bar
Hold reward, so Q(Hold) and Q(trade) targets are scale-comparable. The
marginal Q(trade) > Q(Hold) preference now comes from data variance in
each state (high-aux states pull Q(Hold) more negative), not from
structural scale asymmetry that depends on cost_scale magnitude.

Q2 decision: keep 4-quadrant fixed (no ramping partials). Already in spec.

Changes:
- §4.2 Hold opp-cost: dual emission documented — R_per_bar_centered
  (= R_per_bar - hold_reward_ema) for Q-target/replay tuple,
  R_per_bar uncentered for hold_baseline_buffer (Component 1 baseline)
- §4.5 Kernel 1: hold_reward_ema added (per-step on Hold-state bars only)
- §5 data flow: per-bar reward path shows the centered/uncentered split
- ISV slots: 9 → 10 (HOLD_REWARD_EMA_INDEX added)
- §8 footprint: Component 2 LoC 70 → 90 (+20 for dual emission)
- Total LoC estimate: 1620

docs/superpowers/specs/2026-05-09-sp19-20-wr-first-design.md

@@ -185,13 +185,24 @@ elif hold_pct_ema < target_hold_pct - 0.05:
 
 **Buffer sizing:** `hold_baseline_buffer` is a circular buffer of size `LOOKAHEAD_HORIZON_MAX = 30` bars (matches SP19's longest horizon and bounds max trade duration). Bars older than 30 are overwritten — fine, since trades > 30 bars don't sample beyond their last 30 bars.
 
-**Q-scale design note (intentional asymmetry between Hold and trade rewards):** Hold per-bar reward is **uncentered** (raw `-aux_conf × cost_scale`), whereas trade reward is centered via `R_used = alpha - alpha_ema`. This asymmetry is **deliberate**: it produces a marginal Q(trade) > Q(Hold) preference (Q(Hold) accumulates a small negative drift, Q(trade) is zero-mean), which counteracts the Q(Hold) attractor we are trying to break. Centering Hold rewards too would symmetrize the bias and remove this engineered preference. Behavioral test `sp20_pure_noise` validates the Hold attractor is still correctly induced when there is no signal — so the marginal trade preference doesn't override the no-signal Hold default.
+**Q-scale comparability — dual emission with centering:** Both Hold and trade rewards are centered via long-run EMA subtraction so Q(Hold) and Q(trade) are scale-comparable. Hold's `R_per_bar_centered` writes to the replay tuple for Q-target consumption; the **uncentered** `R_per_bar` writes to `hold_baseline_buffer` for Component 1's structural baseline subtraction. The two emissions serve different purposes — centered for Q-target stability, uncentered for "what would Hold have paid" baseline.
+
+```
+hold_reward_ema_t+1 ← (1-α) × hold_reward_ema_t + α × R_per_bar_i      # Wiener-α floor 0.4
+                                                                         # over Hold-state bars only
+
+R_per_bar_centered_i = R_per_bar_i - hold_reward_ema_t                  # for Q-target / replay tuple
+
+hold_baseline_buffer[i] = R_per_bar_i                                    # uncentered, for Component 1
+```
+
+The marginal Q(trade) > Q(Hold) preference now comes from the **data** rather than from structural scale asymmetry: in high-aux-confidence states, Hold's variance pulls Q(Hold) more negative (when aux is right); in low-aux states, Q(Hold) is near-zero and Q(trade) is near-zero, model has no preference. This is cleaner than scale-asymmetric drift and robust as cost_scale evolves.
 
 **Implementation:**
 - 1 new producer kernel `hold_cost_scale_compute_kernel.cu` (~50 LoC, follows SP4 Pearls A+D)
 - Modification at `experience_kernels.cu` per-bar reward path: add `R_per_bar_i` term, write to `hold_baseline_buffer`
 
-**ISV slots (3 new):** `HOLD_COST_SCALE_INDEX`, `TARGET_HOLD_PCT_INDEX`, `HOLD_PCT_EMA_INDEX`.
+**ISV slots (4 new):** `HOLD_COST_SCALE_INDEX`, `TARGET_HOLD_PCT_INDEX`, `HOLD_PCT_EMA_INDEX`, `HOLD_REWARD_EMA_INDEX`.
 
 ### 4.3 Component 3 — n-step credit distributor
 
@@ -283,6 +294,7 @@ Produces (Wiener-α EMAs with floor 0.4):
 - `aux_dir_acc_ema` — per-step (extends existing SP14-C signal)
 - `hold_pct_ema` — per-step
 - `alpha_ema` — per-trade
+- `hold_reward_ema` — per-step on Hold-state bars only (for Q-scale comparability)
 
 **Kernel 2: `sp20_controllers_compute`** (~70 LoC, depends on Kernel 1)
 
@@ -301,7 +313,7 @@ Consumes: aux_logits raw.
 Produces: `aux_conf_p50`, `aux_conf_std` (used by Kernel 1).
 Uses `sp4_histogram_p99.cuh` pattern for p50 (median, k=128).
 
-**Total new ISV slots: 9 in range [510..520).**
+**Total new ISV slots: 10 in range [510..520).**
 
 ### 4.6 Component 6 — Behavioral Test Suite
 
@@ -332,9 +344,12 @@ All reuse SP15 Phase 2A LobBar harness. ~250 LoC test code. Each runs <30s on RT
 [Step]
   trade_physics(a_t) → step_return, is_close, position state
   per-bar reward path:
-    if action == Hold:  R_per_bar = -aux_conf × cost_scale  (Component 2)
+    if action == Hold:
-    else:                R_per_bar = 0
+        R_per_bar          = -aux_conf × cost_scale                  # raw, for hold_reward_ema update
-    hold_baseline_buffer[t] = -aux_conf × cost_scale  (always, for Component 1)
+        R_per_bar_centered = R_per_bar - hold_reward_ema             # for Q-target via replay tuple
+    else:
+        R_per_bar_centered = 0
+    hold_baseline_buffer[t] = -aux_conf × cost_scale                  # uncentered, for Component 1
 
 [Trade close detected]
   R_event = 4-quadrant reward (Component 1)
@@ -385,7 +400,8 @@ Approximate sizing per component (atomic ship per `feedback_no_partial_refactor`
 | Component | Files modified | New files | Approx LoC |
 |---|---|---|---|
 | 1: Reward kernel | `experience_kernels.cu`, `gpu_experience_collector.rs` | — | 100 |
-| 2: Hold opp-cost | `experience_kernels.cu` | `hold_cost_scale_compute_kernel.cu` | 70 |
+| 2: Hold opp-cost | `experience_kernels.cu` | `hold_cost_scale_compute_kernel.cu` | 90 |
+|   (Q1(b) added: `hold_reward_ema` producer + dual emission centered/uncentered split — +20 LoC) | | | |
 | 3: n-step distributor | `gpu_experience_collector.rs` | `n_step_credit_kernel.cu` | 110 |
 | 4: Aux→Q gate | `gpu_dqn_trainer.rs` | — | 40 |
 | 5: Adaptive curriculum | `sp14_isv_slots.rs`, `mod.rs`, `build.rs` | `sp20_emas_compute_kernel.cu`, `sp20_controllers_compute_kernel.cu`, `sp20_stats_compute_kernel.cu` | 280 |
@@ -394,7 +410,7 @@ Approximate sizing per component (atomic ship per `feedback_no_partial_refactor`
 | 6: Behavioral tests | — | `sp19_20_behavioral_tests.rs` | 250 |
 | Per-component unit tests | — | various | 600 |
 | Audit doc + memory pearls | `audit/sp20_wireup.md` (new) | — | 100 |
-| **Total** | | | **~1,600** |
+| **Total** | | | **~1,620** |
 
 Comparable in scope to SP18 (~2,500 LoC). Smaller than SP15 (~5,000 LoC).