docs(dqn): R5 micro-reward — documented intentional disable (Phase 2 Task 2.3)

Per feedback_no_functionality_removal.md: R5 was originally scoped as
DELETE in the Phase 2 plan and the Track 2 triage because
reward_contrib[3] = 0.000 across 60 / 60 smoke epochs and
dqn-smoketest.toml sets micro_reward_scale = 0.0. Re-examination during
Phase 2 Task 2.3 rejected the DELETE path:

- dqn-production.toml already sets micro_reward_scale = 0.1, so R5 is
  load-bearing in production, not dead code. The 0.0 value in smoke is
  deliberate test-isolation (td_propagation / magnitude_distribution /
  reward_component_audit all want the sparse-reward TD path isolated).
- The state-vector OFI block at state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM)
  = [42..62) provides representation features for the encoder (policy
  side). R5 is a per-bar reward gradient on the critic (critic side).
  Different mechanisms — production deploys both together.
- R5 also reads PREV_MID (retrospective hold quality), which is NOT in
  the state vector. That signal exists only in the kernel branch.

Changes — pure documentation, no behavior change:
- experience_kernels.cu: ~30-line comment block at the R5 wiring site
  (~L1915) documenting the parameter-not-flag status, production vs
  smoke values, why state-vector OFI is complementary not redundant,
  and the feedback_no_functionality_removal.md seal.
- experience_kernels.cu: fix stale kernel-signature comment that claimed
  OFI was at state[66..74). Correct range is [42..62) per state_layout.cuh.
- config.rs: extend DQNHyperparameters::micro_reward_scale docstring and
  add a comment at the Default impl pointing back at the kernel site.
- gpu_experience_collector.rs: extend reward_contrib_fractions docstring
  to mark the micro=0.000 slot as a SEMANTIC value when the loaded profile
  has micro_reward_scale=0.0, not a wiring regression.
- track2-triage.md: R5 verdict changed from DELETE to FIX-documented-disable
  with the rationale above; "Proposed Phase 2 changes" section 1 and
  "Next Track 2 steps" updated accordingly.

Smoke tests: 3 / 4 pass (reward_component_audit, controller_activity,
exploration_coverage). magnitude_distribution is failing on baseline
HEAD c0fee5a9b as well — pre-existing H10 magnitude-head collapse
regression (eval dist_mag ef=0.000, eh=0.000), unrelated to R5.
Verified via git-stash round-trip: baseline fails, changes do not
introduce new failures.

Closes Track 2 R5 verdict (originally DELETE, now FIX-documented-disable).

crates/ml/src/cuda_pipeline/experience_kernels.cu

@@ -1238,7 +1238,7 @@ extern "C" __global__ void experience_env_step(
     const float* __restrict__ isv_signals_ptr,   /* [8] pinned device-mapped ISV signals. NULL = static hold. */
     const float* __restrict__ plan_params_ptr,   /* [N, 6] trade plan params. NULL = no plan. */
     const float* __restrict__ readiness_ptr,     /* [1] pinned model readiness ∈[0,1]. NULL=1.0. Plan enforcement scales with readiness. */
-    const float* __restrict__ full_batch_states, /* [N, full_state_dim] full state with OFI at [66..74). NULL = no OFI. */
+    const float* __restrict__ full_batch_states, /* [N, full_state_dim] full state with OFI at [SL_OFI_START..SL_OFI_START+SL_OFI_DIM) = [42..62). NULL = no OFI. */
     int full_state_dim,                          /* state_dim of full_batch_states (needed for stride) */
     float price_confirm_weight,      /* micro-reward: price confirmation weight (default 0.5) */
     float book_aggression_weight,    /* micro-reward: book aggression weight (default 0.3) */
@@ -1912,6 +1912,39 @@ extern "C" __global__ void experience_env_step(
      * Dense per-bar noise (OFI alignment, DSR, inventory penalty) REMOVED.
      * C51 atoms now model distribution of TRADE OUTCOMES, not per-bar noise.
      * ──────────────────────────────────────────────────────────────────── */
+    /* ── R5: Dense micro-reward (Phase 2 Task 2.3 — FIX-documented-disable) ──
+     *
+     * micro_reward_scale is a PARAMETER, not a feature flag:
+     *   - dqn-production.toml  : micro_reward_scale = 0.1   (ACTIVE)
+     *   - dqn-localdev.toml    : micro_reward_scale = 0.01
+     *   - dqn-hyperopt.toml    : micro_reward_scale = 0.01  (sweep [0.0, 0.05])
+     *   - dqn-smoketest.toml   : micro_reward_scale = 0.0   (intentionally off)
+     *
+     * Why this branch is intentionally DISABLED at smoke scale (scale=0.0):
+     *   Smoke fixtures isolate the sparse-reward TD path for reproducibility —
+     *   smoke tests (td_propagation, magnitude_distribution,
+     *   reward_component_audit) measure segment-complete rewards in isolation
+     *   without dense per-bar shaping. Smoke data is 5000 bars × 3 folds,
+     *   too small for PopArt normalization to cover the dense signal stably.
+     *
+     * Why this branch is NOT redundant with OFI state features at
+     * state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM) = state[42..62):
+     *   The state-vector OFI block provides representation features for the
+     *   encoder to learn from (policy side). This branch injects a per-bar
+     *   REWARD gradient signal (critic side). Representation features and
+     *   gradient-injection signals are different mechanisms — a sparse TD
+     *   target on trade completion does not in general push representation
+     *   information into Q-value magnitudes on the per-bar scale that
+     *   dense shaping does. Empirically, production deploys with scale=0.1
+     *   on top of the full OFI state vector, so the two are complementary.
+     *
+     * Additionally, this branch reads PREV_MID (retrospective hold quality)
+     * which is NOT in the state vector — that signal exists only here.
+     *
+     * Per feedback_no_functionality_removal.md: this code path MUST NOT be
+     * removed. To activate, set micro_reward_scale > 0 in the loaded TOML
+     * profile (no recompile needed).
+     * ──────────────────────────────────────────────────────────────────── */
     if (!segment_complete && fabsf(position) > 0.001f && micro_reward_scale > 0.0f
         && full_batch_states != NULL && full_state_dim >= SL_OFI_START + SL_OFI_DIM) {
         /* ── Dense micro-reward: OFI momentum + MBP-10 price confirmation ──

crates/ml/src/cuda_pipeline/gpu_experience_collector.rs

@@ -1806,6 +1806,13 @@ impl GpuExperienceCollector {
     ///                  flip) can shrink or flip the final `total_reward_per_sample`.
     ///                  Pre/post mismatch is intentional — we want micro magnitude
     ///                  relative to the final reward scale the network trains on.
+    ///                  **Semantic zero:** when the loaded profile sets
+    ///                  `micro_reward_scale = 0.0` (e.g. `dqn-smoketest.toml`),
+    ///                  the kernel branch is guarded off and `micro_reward_per_sample`
+    ///                  stays at its entry-default 0.0f → this slot reads 0.000
+    ///                  as a real semantic value ("profile intentionally disables
+    ///                  R5"), NOT a wiring regression. Phase 2 Task 2.3 kept the
+    ///                  kernel path live per feedback_no_functionality_removal.md.
     /// * `loss_aversion` — |Σ Δclamp| / |Σ post-transform total|. Same caveat
     ///                  as `micro`: numerator is the signed delta
     ///                  (`base_reward - clamped_reward`), kernel writes the signed

crates/ml/src/trainers/dqn/config.rs

@@ -491,7 +491,15 @@ pub struct DQNHyperparameters {
     /// Auto-scales with v_min/v_max: 0.05 = "need ≥5% of Q-spread conviction to trade."
     /// 0.0 = disabled. Typical range [0.0, 0.5].
     pub q_gap_threshold: f64,
-    /// v8: Dense directional micro-reward scale (adaptive vol).
+    /// v8: Dense directional micro-reward scale (R5 reward term).
+    /// Parameter, NOT a feature flag. Controls the tanh-bounded per-bar
+    /// shaping reward built from OFI momentum, price confirmation, and
+    /// retrospective hold quality (see experience_kernels.cu `R5: Dense
+    /// micro-reward` branch). Production profile uses 0.1; smoke profile
+    /// uses 0.0 to isolate the sparse-reward TD path for reproducibility.
+    /// Phase 2 Task 2.3 FIX-documented-disable — per
+    /// feedback_no_functionality_removal.md the kernel branch and buffer
+    /// infrastructure are preserved even when the scale is 0 in a given profile.
     pub micro_reward_scale: f64,
     /// Dense micro-reward: price confirmation weight (default 0.5).
     pub price_confirm_weight: f64,
@@ -1371,6 +1379,11 @@ impl DQNHyperparameters {
             // Reward — Counterfactual Branch Attribution
             cea_weight: 0.3,
             // v8: Dense micro-reward, TD(λ), PopArt, curriculum, hindsight
+            // R5 micro-reward: per-bar OFI-momentum + price-confirmation signal.
+            // Phase 2 Task 2.3 FIX-documented-disable: production uses 0.1,
+            // smoke uses 0.0 (intentional — see kernel comment at
+            // experience_kernels.cu `R5: Dense micro-reward` branch).
+            // DO NOT remove — feedback_no_functionality_removal.md applies.
             micro_reward_scale: 0.01,
             price_confirm_weight: 0.5,
             book_aggression_weight: 0.3,

docs/superpowers/specs/2026-04-21-policy-quality-track2-triage.md

@@ -14,7 +14,8 @@ Preliminary triage against spec §5.2 reward terms R1–R8. At smoke scale:
 
 - **3 KEEP** — R1 (step_return, base reward), R3 (CF-flip, 66% contribution is the dominant signal), R8 (raw_returns_out, required by Sharpe / MaxDD / risk-metric pipeline)
 - **1 KEEP-WITH-CAVEAT** — R4 (trailing-stop, 25 / 60 epochs above the 1 % floor; firing-rate diagnostic only, not a reward additive — re-validate at L40S where Full positions actually enter)
-- **2 DELETE** — R5 (micro-reward: `micro_reward_scale=0` in smoke, contribution 0 / 60 epochs — intentionally dormant, already matches Phase 2 deletion plan), R6 (loss-aversion asymmetric clamp: 6 / 60 epochs above 1 %, mean 0.26 % — sub-noise)
+- **1 DELETE** — R6 (loss-aversion asymmetric clamp: 6 / 60 epochs above 1 %, mean 0.26 % — sub-noise)
+- **1 FIX-documented-disable** — R5 (micro-reward: `micro_reward_scale=0` in smoke, contribution 0 / 60 epochs — smoke-scale-only disable; production toml has scale=0.1 so the code path is already load-bearing in prod. Per `feedback_no_functionality_removal.md` the DELETE verdict was rejected; Phase 2 Task 2.3 preserved the code path and documented the disable.)
 - **1 PENDING** — R2 (PopArt drift: 0 / 60 epochs nonzero; warmup-gated by design — requires production-scale run before a final verdict)
 - **1 ALREADY-REMOVED** — R7 (segment-patience multiplier: never wired in the kernel; slot deleted per Task 0.8 stub-remediation commit `83d524f86`)
 
@@ -137,13 +138,23 @@ The smoke-test assertion at `reward_component_audit.rs:105` enforces `trail_r >
 
 This is a **SEMANTIC zero** (intentional profile config, not a wiring regression), the same class as R2's warmup zero. Distinction from Phase 2's deletion plan: Phase 2 intends to DELETE the term outright (branch + config field + kernel variables) once the TD-propagation diagnostic lands.
 
-**Decision-matrix verdict:** **DELETE** — contribution < 1 % in 60 / 60 epochs AND the intent is better served by the feature layer (OFI is already in state) AND reward-inventory already tagged it for removal.
+**Decision-matrix verdict:** **FIX-documented-disable** (originally DELETE — rejected per `feedback_no_functionality_removal.md`).
 
-Unlike R2 (which stays dormant at smoke but ships to production warmed), R5 is explicitly disabled in the smoke profile AND the design direction is to remove it. The decision matrix applies cleanly.
+The original DELETE verdict relied on "contribution 0 / 60 epochs AND feature-layer redundancy AND reward-inventory tagged for removal". Re-examination during Phase 2 Task 2.3 found:
 
-**Caveat:** the deletion is gated on the Phase 2 Task #8 TD-propagation diagnostic landing first, per reward-inventory §"Open questions". Until that diagnostic confirms `cov(Q(s_entry, a), realized_trade_return) > 0.1`, the term stays as dead code in-tree even if it's deactivated by config. Track 2 verdict is DELETE; Phase 2 plan owns the sequencing.
+1. **The contribution is 0 at smoke ONLY** because `dqn-smoketest.toml` sets `micro_reward_scale = 0.0` for test-isolation reasons (td_propagation / magnitude_distribution / reward_component_audit all want the sparse-reward TD path isolated). `dqn-production.toml` sets `micro_reward_scale = 0.1` — R5 is load-bearing in production, not dead code.
+2. **Feature-layer "redundancy" was a category error.** State-vector OFI features at state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM) = [42..62) are representation inputs for the policy/value encoder. R5 is a per-bar reward gradient on the critic. Different mechanism; production deploys both together. R5 also reads `PREV_MID` (retrospective hold quality) which is NOT in the state vector — that signal exists only in the kernel branch.
+3. **Reward-inventory disposition** predates the no-functionality-removal rule; the inventory's DELETE tag is overridden.
 
-**Evidence strength:** STRONG on contribution (consistent zero). MEDIUM on timing (gated on unrelated diagnostic).
+Phase 2 Task 2.3 implementation (commit follows this triage amendment):
+- Preserves the kernel branch at `experience_kernels.cu:1915..2003`.
+- Preserves the `micro_reward_per_sample` buffer + `reward_contrib_fractions` reducer slot.
+- Preserves the `micro_reward_scale` field in `DQNHyperparameters`.
+- Adds a kernel-site comment documenting (a) the parameter-not-flag status, (b) the smoke-vs-production scale values, (c) why the state-vector OFI block is complementary not redundant, (d) the `feedback_no_functionality_removal.md` seal.
+- Adds a config-site comment at the `Default` impl.
+- Extends the `reward_contrib_fractions` docstring to mark `micro=0.000` as a SEMANTIC value when the profile's scale is 0 (not a wiring regression).
+
+**Evidence strength:** STRONG on "code path is load-bearing in production" (config evidence). STRONG on "state-vector OFI is representation not reward" (mechanism distinction).
 
 ---
 
@@ -246,7 +257,7 @@ VaR / CVaR are non-degenerate → `raw_returns_out` buffer is alive.
 
 Ordered by impact and independence:
 
-1. **DELETE R5 (micro-reward)** — after Phase 2 Task #8 (TD-propagation diagnostic) confirms sparse-reward TD works. Full removal: kernel branch at `experience_kernels.cu:1818-1902`, `micro_reward_scale` config field, supporting args in kernel signature, collector's `micro_reward_per_sample` buffer. Dead code reduction ~ 100 lines kernel + supporting Rust.
+1. **FIX-documented-disable R5 (micro-reward)** — DELETE verdict rejected per `feedback_no_functionality_removal.md`. Phase 2 Task 2.3 implementation: preserve the kernel branch at `experience_kernels.cu:1915..2003`, preserve the `micro_reward_scale` config field, preserve the `micro_reward_per_sample` buffer + reducer slot; add code-site comments explaining (a) scale=0 at smoke is intentional test-isolation, (b) production already deploys scale=0.1 so the code path is live in prod, (c) OFI state features are representation signal, micro_reward is gradient-injection — different mechanism, not redundant. Net code delta: +~50 LOC comments, zero deletion.
 
 2. **DELETE R6 (asymmetric soft-clamp as a reward-layer mechanism)** — move the negative-tail compression to Q-target smoothing (same pattern as reward-inventory's `reward_noise_scale` relocation to C51 Bellman). Preserve the +10 upper cap as a separate guard (numerical safety, not loss-aversion). Change surface: `experience_kernels.cu:1774-1783` deletion + equivalent in C51 loss kernel.
 
@@ -278,7 +289,7 @@ Ordered by impact and independence:
 
 - **L40S validation**: re-run the 20-epoch × 3-fold baseline on L40S scale (plan §Task 2.1). Extract the same `reward_contrib` trajectory. Promote R2 from PENDING to final verdict. Re-confirm R4, R6 (fold-3 dominance pattern was real signal or sample artifact).
 
-- **Phase 2 Task #8 (TD-propagation diagnostic)**: needed before R5 deletion lands, per reward-inventory §"Open questions". Measure `cov(Q(s_entry, a), realized_trade_return)` over 10-epoch sparse-reward run. If > 0.1 → sparse TD works, delete R5 unconditionally. If < 0.1 → fix `td_lambda` / n-step / reward clipping first.
+- **Phase 2 Task #8 (TD-propagation diagnostic)**: originally gating R5 DELETE. With R5 reclassified to FIX-documented-disable, the diagnostic is no longer a pre-requisite for this term — the code path stays regardless. The diagnostic remains useful for Track 4 E4 (C51 entropy) and for validating that the sparse TD path carries meaningful gradient; keep it on the plan under Task 2.0, but decouple from R5 sequencing.
 
 - **R6 relocation design**: sketch the Q-target smoothing mechanism in `c51_loss_kernel.cu` that subsumes the negative-tail compression currently done by `asymmetric_soft_clamp`. Document before deleting R6 from the reward kernel.