# SP22 H6 Phase 3 — Combined α (bypass-head) + β (event-driven aux bonus)

**Author**: jgrusewski + assistant pair
**Date**: 2026-05-12
**Branch**: `sp20-aux-h-fixed`
**Predecessors**:
- Phase 1 — `docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md` (aux→state wiring; falsified at cycle 1 WR = 50.21%)
- Phase 2 — `docs/plans/2026-05-12-sp22-h6-phase2-recenter.md` (slot-121 recentering [-1, +1]; falsified at cycle 1 WR = 50.20% with **bit-identical action distribution** to Phase 1, proving the encoder ignores state[121] regardless of encoding)

**Audit pointer**: append findings to `docs/dqn-wire-up-audit.md` under
`### Phase 3 (2026-05-12) — combined α + β routing aux around the cold encoder`.

## Why this is Phase 3, not a pivot to H3

The Phase 2 verdict gave **mechanistic evidence** that the encoder's weights for state[121] are essentially unused — the policy made bit-identical action choices regardless of slot 121's encoding (drift < 0.5% on every action bin, well within run-to-run noise). The aux signal IS reaching state[121] (`pred_tanh = 0.66` in both phases — strong directional bias) but the encoder doesn't gradient-couple to a new dim fast enough in 3 epochs.

Simply amplifying state[121] (Phase 2 spec's first-fallback suggestion) doesn't address this: if the encoder's weights for slot 121 are near-zero, larger inputs scale to larger zeros at the encoder output. The fix must **route aux around the cold encoder**.

Combined per `pearl_no_deferrals_for_complementary_fixes` (distinct mechanisms, non-overlapping refactor scopes):
- **α** routes aux through a **parallel skip connection** at the network output level (4 new params, trained end-to-end with Adam).
- **β** routes aux through the **training signal** (rewards aux-aligned profitable trades, raising the Q-target for trades that aux endorsed).

Both bypass the cold-encoder bottleneck on different axes (forward activation vs training signal). The pearl prescribes one atomic plan; falsification (or confirmation) of the combined intervention rules in or out both routes simultaneously.

## α — atom-position shift (architecturally revised 2026-05-13)

### Architectural correction: scalar Q bias is INCOMPATIBLE with C51

The original spec proposed `Q_dir[b, a] += W[a] * state_121[b]` as a post-encoder bypass. For non-distributional Q-learning (scalar Q per action, MSE loss against Bellman target), this works directly: W gets gradient via `dloss/dQ`.

This codebase uses **C51 distributional Q-learning**. Q is computed from a probability distribution over fixed atom positions: `E[Q_a] = Σ_n p_n(s, a) * z_n`. The C51 loss is KL divergence between predicted distribution and projected Bellman target. Adding a constant to logits is softmax-shift-invariant → probability distribution unchanged → loss unchanged → **W gradient = 0, never trains**. Verified mathematically and operationally during Phase 3b implementation (commit `cb80b74ce`).

### Corrected α: per-(b, a) atom-position shift

The mathematically principled fix is to shift the atom POSITIONS state-dependently:

```
z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b]    for all atoms n
```

Where `z_n_base = v_min + n * dz` (existing per-branch support), W is a 4-element learnable bias (one per direction action), and `state_121[b] = batch_states[b * STATE_DIM + 121]` (encoder INPUT — bypasses cold encoder).

Effects:

- **Expected Q shift**: `E[Q'_a] = Σ_n p_n * z_n_effective = E[Q_a] + W[a] * state_121[b]` (intended bias).
- **C51 Bellman projection**: target distribution projects onto SHIFTED atom positions. Projection arithmetic is continuous in atom positions → loss depends on `W[a] * state_121[b]` smoothly → **W gradient flows**.
- **dW gradient**: `dloss/dW[a] = Σ_b state_121[b] * Σ_n dloss/dz_n_effective` (per-sample, per-atom).
- **dstate_121 gradient**: feed back into encoder column for slot 121 (option-(i) routing).

### Initialization: structural prior

W is initialized to a domain-prior reflecting belief that aux conviction × position-sign biases toward aligned trades:

```
W_init = [-0.5, 0.0, +0.5, 0.0]    [Short, Hold, Long, Flat]
```

Interpretation: `state_121 ∈ [-1, +1]` is recentered aux p_up. When `state_121 = +1` (aux predicts up):
- `W[Short] * state_121 = -0.5` → Q_short atoms shift down by 0.5 → action selection avoids Short.
- `W[Long] * state_121 = +0.5` → Q_long atoms shift up by 0.5 → action selection biases toward Long.
- Hold / Flat unchanged.

Adam then refines W over training. Mathematically equivalent to `W = W_prior + W_learned` with W_learned starting at zero — the prior is just the initial value of the unified W tensor.

### Param storage

| Buffer | Shape | Purpose |
|---|---|---|
| `W_aux_to_Q_dir` | `[4]` f32 | Adam-trained weight; init `[-0.5, 0.0, +0.5, 0.0]` |
| `adam_m_W_aux` | `[4]` f32 | Adam first moment, zero-init |
| `adam_v_W_aux` | `[4]` f32 | Adam second moment, zero-init |
| `dW_aux_buf` | `[4]` f32 | Gradient accumulator, written by backward |

Total: 16 floats. Negligible.

### Kernel inventory (atom-position-shift implementation)

Threads `aux_atom_shift[b, a] = W[a] * state_121[b]` through every kernel that uses atom positions for the direction branch:

| Kernel | Direction-branch atom consumer? | Phase 3-final change |
|---|---|---|
| `compute_expected_q` (`experience_kernels.cu:4932`) | YES | New args: W, batch_states, aux_dir_prob_index, state_dim, b0_size. When `d == 0`, add `aux_atom_shift[b, a]` to `z_val` per atom. |
| `c51_loss_kernel` (`c51_loss_kernel.cu`) | YES — Bellman projection reads atom positions | Same atom-shift threading. Projection uses shifted positions for dir branch. |
| `c51_grad_kernel` (`c51_grad_kernel.cu`) | YES — gradients on logits via projection | Compute `dz_shift[b, a] = Σ_n dloss/dz_n_effective`. Accumulate `dW[a] += Σ_b state_121[b] * dz_shift[b, a]` and `dstate_121[b] += Σ_a W[a] * dz_shift[b, a]`. Per `feedback_no_atomicadd`: block tree-reduce. |
| `mag_concat_qdir` (`experience_kernels.cu:5505`) | YES — internal Q_dir compute uses atom positions | Apply same atom shift inline so magnitude-branch conditioning sees α-biased Q_dir consistent with action selection. |
| `quantile_q_select` / `iqn_dual_head_kernel` | Investigate during implementation | Same shift if applicable. |

The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` are **architecturally redundant** under this revised design. The dW + dstate gradient computations integrate into `c51_grad_kernel`'s projection backward. The Phase A kernels stay as compiled cubins (committed in `464bc5f7a`, dead code at runtime); a future cleanup commit may delete them.

### Adam wireup

Same generic Adam-step kernel as other params. Add `W_aux_to_Q_dir` + `adam_m_W_aux` + `adam_v_W_aux` to the Adam param list. β1=0.9, β2=0.999, ε=1e-8, lr from learning-rate schedule. No new kernel.

### Gradient routing to state[121]

Same option (i): `dbatch_states[b * state_dim + 121] += Σ_a W[a] * dz_shift[b, a]`. Both α and encoder paths get gradient signal.

## β — event-driven aux-aligned trade-close reward

### Formula

At trade-close events (existing event detection in `experience_env_step::segment_complete`):

```
position_sign = sign(prev_position)         // +1 long, -1 short, 0 hold/flat
aux_at_close  = state_121[env]              // Phase-2 recentered ∈ [-1, +1]

alignment     = max(0, aux_at_close * position_sign)   // ∈ [0, +1]; zero if anti-aligned
profit_term   = max(0, realized_pnl)                    // ∈ [0, +∞); zero on losing trades

r_aux_align   = scale_beta * alignment * profit_term
```

Where:
- `scale_beta` (β's reward-component weight) is **emitted by the SP11 controller** each step into `isv_signals_ptr[SP22_AUX_ALIGN_SCALE_INDEX]`. Cold-start sentinel = 0 → first controller emit replaces. The producer site reads the slot at the trade-close event. See "scale_β driven by SP11 controller" below for the anchor / target / bounds.
- `realized_pnl` is the close-bar P&L for this trade (already computed in `experience_env_step` via `unified_env_step_core`'s `step_ret_core`).
- `prev_position` is the position held just before `unified_env_step_core` mutates it (already snapshotted in `experience_env_step` by the existing SP21 Phase 1 pre-call snapshot block — locate by grepping for `pre_entry_price` or `prev_position` near the segment_complete branch).
- `state_121[env]` is read from the per-env buffer `prev_aux_dir_prob` (Phase 1 H6 introduced this; already populated by `aux_softmax_to_per_env_kernel`).

### Placement — 7th reward component (atomic contract change)

β fires as a **new 7th reward component** `r_aux_align`, not merged into r_trail. Reasons (revised from earlier YAGNI framing):
- Diagnosability — HEALTH_DIAG `reward_split` shows the component contribution explicitly; we can read off whether β is firing at the expected rate and magnitude per trade.
- SP11 controller integration is the canonical pattern for adaptive component weighting (see "scale_β driven by SP11 controller" below); a separate component slot is the natural home for the adaptive weight.
- Future tuning, ablations, and removal are cleaner when β lives in its own slot.
- Per `feedback_no_partial_refactor`, the contract change cascades through every consumer of `reward_components_per_sample[6]` — they all migrate to `[7]` atomically in one commit.

Component index assignment: `r_aux_align` becomes `rc[6]` (extending the existing 0..5 → 0..6 layout). The current 6-component layout in `experience_kernels.cu` line ~2050 area documents `[0] popart, [1] cf, [2] trail, [3] micro, [4] opp_cost, [5] bonus`. New slot:

```
[6] aux_align — H6 Phase 3 event-driven aux-aligned trade-close bonus.
                Fires ONLY at segment_complete events. Per
                `pearl_event_driven_reward_density_alignment` and
                `pearl_one_unbounded_signal_per_reward` (realized_pnl
                is the single unbounded multiplicand).
```

Producer site (in `experience_env_step::segment_complete` branch):

```c
/* SP22 H6 Phase 3 β (2026-05-12): aux-aligned trade-close bonus.
 * Fires when policy direction aligns with aux conviction AND trade
 * was profitable. Non-negative-only (anti-alignment + losses get
 * zero, NOT a penalty — let r_trail's loss carry the negative
 * signal). Bounded by `pearl_one_unbounded_signal_per_reward`:
 * realized_pnl is the natural per-trade scale; alignment ∈ [0, 1];
 * scale_beta is controller-bound (see SP11 wiring below). */
float position_sign = (prev_position > 0.001f) ? 1.0f
                   : (prev_position < -0.001f) ? -1.0f
                   : 0.0f;
float aux_at_close = (aux_dir_prob_per_env != NULL)
                   ? aux_dir_prob_per_env[i]
                   : 0.0f;  /* NULL → A3 fallback → β no-op */
float alignment = fmaxf(0.0f, aux_at_close * position_sign);
float profit_pos = fmaxf(0.0f, realized_pnl);  /* realized_pnl from step_ret_core */
float scale_beta = (isv_signals_ptr != NULL)
                 ? isv_signals_ptr[SP22_AUX_ALIGN_SCALE_INDEX]
                 : 0.0f;  /* NULL → β no-op (test scaffolds without ISV) */
float r_aux_align = scale_beta * alignment * profit_pos;
reward_components_per_sample[out_off * 7 + 6] = r_aux_align;
```

### Consumer migration (6 → 7 components, all in this commit)

Per `feedback_no_partial_refactor` — every consumer of the component buffer migrates atomically:

| Site | Change |
|---|---|
| `experience_kernels.cu::experience_env_step` (producer) | Allocate `out_off * 7 + …` instead of `out_off * 6 + …`; write `rc[6] = r_aux_align`. Document the 7-component layout in the kernel preamble. |
| `crates/ml/src/cuda_pipeline/reward_component_ema_kernel.cu` | Iterate `[0..7)` instead of `[0..6)`; consume a 7-slot var-scratch base (extend `var_scratch_first_index` allocation by 1 slot if needed). |
| `crates/ml/src/cuda_pipeline/reward_decomp_diag_kernel.cu` | Emit per-action `r_aux_align` column 6 in the diagnostic tile. |
| `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu` | Add `w_aux_align` adaptive weight to the SP11 weight vector (the 6→7 controller output expansion — see "scale_β driven by SP11 controller" below). |
| `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu` | Extend the axis map to include axis 6 (aux_align). |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Allocate `reward_components_per_sample` with 7-slot stride. Update HEALTH_DIAG print of `sp11_reward [w_pop w_cf w_tr w_mi w_oc w_bn w_aux]` and `reward_split [popart cf trail micro opp_cost bonus aux_align]`. |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Match collector's allocation + emit format. |
| `crates/ml/src/trainers/dqn/monitors/reward_component_monitor.rs` | Add `REWARD_AUX_ALIGN_EMA_INDEX` ISV slot for the EMA; consume in HEALTH_DIAG path. |
| `crates/ml/src/cuda_pipeline/health_diag_kernel.cu` | Add `[6] REWARD_AUX_ALIGN_EMA_INDEX → snap.reward_split_aux_align` in the snap layout. |

The 7-component layout becomes the new contract. All future reward-decomposition consumers see 7 components.

### NULL-safety

Two NULL-fallback paths in the producer:
- `aux_dir_prob_per_env == NULL` (eval-time A3 fallback) → `aux_at_close = 0` → `alignment = 0` → `r_aux_align = 0`.
- `isv_signals_ptr == NULL` (test scaffolds without ISV) → `scale_beta = 0` → `r_aux_align = 0`.

Either NULL makes β a no-op without crashing. Consistent with the existing A3 NULL-fallback pattern from Phase 1.

### scale_β driven by SP11 controller (adaptive ISV slot)

Replace the fixed `0.5` with a controller-emitted ISV slot:

**New ISV slot**: `SP22_AUX_ALIGN_SCALE_INDEX = <next available slot in sp22_isv_slots>`. Producer: extension of `reward_subsystem_controller_kernel.cu` (which already emits the 6 SP11 reward weights). Consumer: the β producer site above.

**Controller emit**: extend the SP11 weight vector from 6 → 7 outputs. The new 7th output `w_aux_align` is anchored using the same diagnostic-driven pattern as the other weights (per `pearl_controller_anchors_isv_driven`). Specifically:

- **Anchor signal**: the EMA of `r_aux_align` itself (slot `REWARD_AUX_ALIGN_EMA_INDEX` produced by `reward_component_ema_kernel`).
- **Target**: a fraction of the total reward magnitude. Initial target: `0.10 × Σ|reward_components|` so β contributes ~10% of total reward signal at steady state. Adaptive per the existing controller pattern.
- **Bounds**: floor 0.05 (so β doesn't vanish), ceiling 2.0 (so β can't dominate per `pearl_one_unbounded_signal_per_reward`'s component-balancing intent). Bounds enforced inside the controller kernel.
- **Cold-start sentinel**: 0.0 per `pearl_first_observation_bootstrap` → first observation replaces. The producer above's `scale_beta = isv_signals_ptr[…]` will see 0.0 at cold-start → `r_aux_align = 0` until the controller's first emit. This is acceptable; the controller emits after the first reward EMA observation lands (typically within the first few hundred steps).

**Implementation footprint**:
- New ISV slot in `crates/ml/src/cuda_pipeline/sp22_isv_slots.rs` (or whichever sp22 slots module exists; extend or create as needed).
- Controller kernel `reward_subsystem_controller_kernel.cu`: extend the emit list from 6 to 7 weights; add the `w_aux_align` anchor/target/bound logic.
- `state_reset_registry.rs`: register the new ISV slot as `FoldReset` category (cold-start to 0 each fold per the pearl).
- HEALTH_DIAG emit: extend the `sp11_reward [w_pop … w_bn w_aux_align]` line to include the new weight.

### Encoder-state[121] gradient routing — decision

Per the earlier "Gradient routing to state[121]" tri-choice, **commit to option (i)** — both paths open. Reasons:

- **Simplest implementation**: write `dbatch_states[121] += Σ_a W[a] * dQ_dir[b, a]` and let the existing encoder backward chain consume it. No stop-gradient masking, no special handling.
- **Robust long-term**: the encoder's state[121] weight initially near-zero means its gradient contribution is small; α dominates the Q_dir adjustment during the cold-encoder period. As training progresses, IF the encoder learns useful weights for state[121], both paths contribute additively. This is the "belt + suspenders" architecture that survives architectural changes downstream.
- **Empirical observable**: post-smoke telemetry compares `W_aux_to_Q_dir` magnitudes against the encoder's column-norm for state[121]. If they diverge cleanly (W contributes, encoder stays near-zero), α is doing the work. If both grow, they reinforce. If they oscillate / fight, we have a diagnostic signal that something is off and can revisit.

Option (ii) (block gradient into encoder via state[121]) was considered. Trade-off: cleaner experimental attribution at the cost of permanently blinding the encoder to state[121]. Rejected because the long-term right answer is for the encoder to learn from any input dim available; α's role is to bridge the cold-start phase, not to be the only path forever.

### Why this formulation

**Non-negative only** (alignment ≥ 0, profit ≥ 0): rewards correlation without anti-correlation penalty. If aux says up and policy goes long but trade loses, no bonus and no penalty — the loss-of-r_trail itself carries the negative signal. Avoids confusing the policy when aux is wrong.

**Both terms scale linearly**: bonus magnitude tracks BOTH aux confidence AND trade quality. A barely-confident aux signal that aligns with a tiny-profit trade gives a tiny bonus. A confident aux signal that aligns with a big-profit trade gives a meaningful one. Self-modulating.

**Event-driven** (fires only at trade-close): aligns with `pearl_event_driven_reward_density_alignment`. Per-step shaping would be the anti-pattern.

**Bounded by `pearl_one_unbounded_signal_per_reward`**: `aux_align ∈ [0, 1]`, `profit_term` is the natural per-trade scale (no artificial cap), `scale_beta = 0.5`. The only unbounded factor is `profit_term`, which IS the natural per-trade scale and is the single unbounded multiplicand allowed by the pearl.

## Implementation file map

| File | Change | Responsibility |
|---|---|---|
| `crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu` (new) | Forward kernel: `Q_dir[b, a] += W[a] * state_121[b]` | α forward |
| `crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu` (new) | Backward kernel: `dW[a] = Σ_b state_121[b] * dQ_dir[b, a]`; `dstate_121[b] += Σ_a W[a] * dQ_dir[b, a]` | α backward |
| `crates/ml/build.rs` | Register the 2 new kernel files | α cubin build |
| `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` | Allocate `W_aux_to_Q_dir`, `adam_m_W_aux`, `adam_v_W_aux`; load 2 kernels; wire forward into the captured graph; wire backward into the captured backward graph; wire Adam-step update; extend reward-components allocation to 7-slot stride; HEALTH_DIAG emit format updated to 7-weight sp11_reward line | α plumbing + 7-component contract |
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | Wire α forward into rollout-time forward graph (reads trainer's W); extend reward-components allocation to 7-slot stride; HEALTH_DIAG emit format updated | α rollout-side + 7-component contract |
| `crates/ml/src/cuda_pipeline/experience_kernels.cu` | Add β producer site in `experience_env_step::segment_complete` (writes `rc[6] = r_aux_align`); migrate every `out_off * 6 + …` reference to `out_off * 7 + …`; update kernel-preamble documentation of component layout | β producer + 7-component contract |
| `crates/ml/src/cuda_pipeline/reward_component_ema_kernel.cu` | Iterate 0..7 components; consume 7-slot var-scratch base | 7-component EMA |
| `crates/ml/src/cuda_pipeline/reward_decomp_diag_kernel.cu` | Emit per-action `r_aux_align` column 6 in the diagnostic tile | 7-component diag |
| `crates/ml/src/cuda_pipeline/reward_subsystem_controller_kernel.cu` | Extend the SP11 weight emit from 6 → 7 outputs; add the `w_aux_align` anchor / target / bounds logic; consume `REWARD_AUX_ALIGN_EMA_INDEX` for the target | scale_β SP11 controller |
| `crates/ml/src/cuda_pipeline/reward_component_mag_ratio_compute_kernel.cu` | Extend axis map to include axis 6 (aux_align) for magnitude-ratio computation | 7-component mag-ratio |
| `crates/ml/src/cuda_pipeline/sp22_isv_slots.rs` (new or extend existing) | Define `SP22_AUX_ALIGN_SCALE_INDEX` (controller output) + `REWARD_AUX_ALIGN_EMA_INDEX` (EMA producer output) | New ISV slots |
| `crates/ml/src/cuda_pipeline/health_diag_kernel.cu` | Add `REWARD_AUX_ALIGN_EMA_INDEX → snap.reward_split_aux_align` mapping; extend snap layout by one slot | HEALTH_DIAG snap layout |
| `crates/ml/src/trainers/dqn/monitors/reward_component_monitor.rs` | Consume `REWARD_AUX_ALIGN_EMA_INDEX` in HEALTH_DIAG print path; extend EMA reader to 7 components | 7-component HEALTH_DIAG |
| `crates/ml/src/trainers/dqn/state_reset_registry.rs` | Register `SP22_AUX_ALIGN_SCALE_INDEX` as FoldReset category (sentinel 0) and `REWARD_AUX_ALIGN_EMA_INDEX` as FoldReset (sentinel 0, Pearl A bootstrap) | Slot registry |
| `crates/ml/src/trainers/dqn/trainer/training_loop.rs` | Extend HEALTH_DIAG `sp11_reward [w_pop w_cf w_tr w_mi w_oc w_bn w_aux]` and `reward_split [popart cf trail micro opp_cost bonus aux_align]` lines to 7 components | HEALTH_DIAG printer |
| `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu` | Migrate reward-component stride 6 → 7; add β producer site at segment_complete; document the 7-component layout in the kernel preamble | β eval-side producer + 7-component contract (eval) |
| `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` | Aux trunk weight loading at init; per-window `prev_aux_dir_prob` buffer alloc + fill_f32(0.0); aux trunk forward at per-step (reuse `AuxHeadsForward` if API allows); `aux_softmax_to_per_env_kernel` launch; α post-Q_dir bias launch; pass non-NULL `prev_aux_dir_prob.raw_ptr()` to all 3 state-gather launchers; reset per `evaluate()` call | A2 — eval aux trunk + α + state-gather wiring |
| `docs/dqn-wire-up-audit.md` | Append Phase 3 entry (Invariant 7) covering α + β + 7-component contract + SP11 controller extension + A2 eval-side wiring | Audit doc |

## W-sharing between trainer and rollout-time collector

The trainer holds the authoritative `W_aux_to_Q_dir` parameter buffer (Adam updates it). The collector's rollout-time forward also needs to use the SAME W (to make consistent action choices for the rollouts that feed training).

Existing pattern: rollout-time forward in the collector uses the **same** GEMM weight buffers as the trainer (single source of truth — see `CublasForward::forward_online` reading from the trainer's weight set). The α weight is conceptually a small extension of the network weights.

Implementation: store `W_aux_to_Q_dir` in the same `GpuWeights` structure that holds the Q-network weights. The collector's forward call reads the buffer pointer from the trainer's weight set (already wired). No separate W in the collector.

Backward: only happens on the trainer side. Collector's α launch is forward-only. This is the same pattern as e.g. the GEMM weights — collector forwards them, trainer backwards them.

## Verification gates (all required clean before commit)

1. `SQLX_OFFLINE=true cargo check -p ml --features cuda` → 0 errors. Expect 21 pre-existing warnings (parity with Phase 2 baseline). May see 1-2 new warnings if the new kernel files have any (unused vars from initial scaffolding); investigate and resolve.

2. `cargo test -p ml --test gpu_backtest_validation --features cuda --release` → 6/6 pass (post-Phase-2-test-fix baseline).

3. `/usr/local/cuda/bin/compute-sanitizer --tool=memcheck <test_bin>` → `ERROR SUMMARY: 0 errors`.

Additionally, because Phase 3 introduces new captured-graph contents:

4. **Captured-graph integrity check**: run the smoke for ≥ 100 training steps (or a tiny unit-test invocation) and verify the log contains `CAPTURE_PHASE_FORWARD_DONE`, `CAPTURE_PHASE_DDQN_DONE`, and the parent graph compose line. Failure to capture = `pearl_no_host_branches_in_captured_graph` violation; fix kernel to remove any host branches.

5. **α weight movement check** (smoke cycle 1): after epoch 1, `W_aux_to_Q_dir` should be non-zero (Adam updates accumulated). If still at `[0, 0, 0, 0]`, backward gradient flow is broken — investigate before reading verdict.

6. **7-component contract sanity** (smoke cycle 1): HEALTH_DIAG `sp11_reward` line shows 7 weights including `w_aux=…`; `reward_split` line shows 7 components including `aux_align=…`. Both `w_aux` and `aux_align` should become non-zero within the first epoch after the SP11 controller's first emit. If still zero after epoch 1 → either the SP11 controller's `w_aux_align` emit is broken OR the β producer site isn't firing (check trade-close detection + NULL-fallback gates).

7. **A2 eval-side aux activity check** (smoke + post-smoke validation eval): post-epoch-1 validation eval (the per-cycle backtest the training loop runs) should show non-zero `r_aux_align` accumulating in eval-side `WindowMetrics` reward decomposition. If eval `r_aux_align ≈ 0` while training-side `r_aux_align > 0` → A2 wiring failure (likely eval-time aux trunk forward not firing OR `prev_aux_dir_prob` not flowing through to backtest_state_gather). `gpu_backtest_validation` tests pass post-A2 (with potential numerical shift in extended metrics — the action-constant-fixed test set may need adjusted tolerances if β's bonus shifts CVaR/Omega meaningfully; the four directional tests should remain bit-identical because their `constant_action_model` bypasses Q-network and β bonus is multiplied by sentinel-zero scale_β at test-time since tests don't run the SP11 controller).

## Smoke dispatch + verdict criteria

```bash
./scripts/argo-train.sh dqn --baseline --branch sp20-aux-h-fixed \
  --epochs 3 --gpu-pool ci-training-l40s
```

### Primary

**WR > 50.5%** within 3 epochs → Phase 3 confirmed → the bypass routing is sufficient → justify A2 (eval-side α + β integration) for production parity.

### Secondary (mechanistic)

- **`W_aux_to_Q_dir` magnitudes after epoch 1**: any element > 1e-3 → α gradient flowed and aux has measurable Q-contribution. Magnitudes near zero → α didn't fire (debug); fall back to amplitude scaling.

- **Action distribution diverges from Phase 1/2 baselines**: S_Small drops below 3.5% OR rises above 4.5%; H_Half OR H_Full move outside their Phase-1 bands. Indicates the policy is actually choosing differently — the Phase 1/2 finding "policy makes same choices regardless of state[121]" no longer holds.

- **`a_var [m, o, u]` > 1e-3**: sub-branches gradient-coupled (β's reward signal should propagate through r_trail magnitudes large enough to break the sub-branch starvation pattern).

### Tertiary

- **`reward_split` shows non-trivial r_trail magnitude** (vs Phase 1's 0.0006 baseline): β bonuses are stacking onto r_trail; the aggregate trail-component magnitude should grow when aux-aligned profitable trades fire.

### Falsification path

**If WR pinned at 50.1–50.2% AND `W_aux_to_Q_dir` is non-zero AND action distribution moved**: the bypass routing IS getting the aux signal to the policy, but the aux signal itself isn't actionable on the current price fixtures + reward shape. This points downstream of "policy can see aux" — likely the deeper "directional alpha is genuinely too weak for 3-epoch baseline to extract" finding. Pivot to:
- Per-branch optimization re-audit (`pearl_per_branch_iqn_tau_schedule` + family)
- Reward density / decomposition rewrite (the harder H3)
- Longer training horizon (10+ epochs)

**If WR pinned AND `W_aux_to_Q_dir` is near zero**: gradient flow is broken. Kernel debug.

**If WR pinned AND action distribution is bit-identical to Phase 1/2**: α isn't influencing decisions despite W moving. Investigate Q-value scale — maybe `W * state[121]` is dwarfed by `Q_dir` natural magnitude. Amplitude scaling becomes relevant (multiply W initialization or input).

## Risks + mitigations

| Risk | Mitigation |
|---|---|
| α gradient amplifies Q-values aggressively → divergence | Zero-init W; standard Adam clipping (already in place; see GRAD_CLIP_OUTLIER in Phase 1/2 logs) |
| β bonus dwarfs r_trail and distorts training | scale_beta = 0.5 (subtle); bonus magnitude tracks realized_pnl natural scale so it can't pathologically dominate |
| State[121] gradient via α + via encoder might interfere | Both paths are independent; α dominates initially (encoder weight near-zero). Empirical check via W magnitudes. |
| Aux is systematically wrong; β rewards wrong-direction trades | β requires aux-position alignment AND profitable trade. If aux is wrong direction, alignment AND profit rarely co-occur → bonus rarely fires. Self-correcting. |
| Captured-graph violation in new kernels | Pure per-thread compute, no host branches, no atomics. Verified via captured-graph integrity check (gate 4 above). |
| α W-sharing race between trainer and collector | Same pattern as existing GEMM weights — collector reads, trainer reads+writes. Adam step happens between rollouts, not during. No race. |
| Rollout-time forward graph might not contain α | Add to the rollout forward graph at the same site as the trainer's forward (post-Q_dir). Verify via CAPTURE_PHASE_FORWARD_DONE in collector logs. |

## A2 — eval-side α + β + aux trunk forward (in scope)

User-directed inclusion. Brings the eval pipeline to full parity with the training pipeline so the verdict is no longer training-side-only. A2 is the architectural addition that lets eval-time `total_pnl` / `sharpe` / `WindowMetrics` reflect aux-guided policy behavior; without A2 the eval is observing whatever policy the rollout-time-aux-amplified network produces but eval itself sees `state[121] = 0.0` (A3 NULL fallback), making eval-time α a no-op (`W * 0 = 0`) and β a no-op (alignment * profit = 0 because alignment uses aux_at_close = 0).

A2 has five components (eval-time forward parity with training-time rollout-time forward):

### A2.1 — Aux trunk weight loading at eval init

The trainer holds the aux trunk weights as part of its parameter set. Eval currently doesn't load them. Add:
- At `GpuBacktestEvaluator::new` (or wherever weights are loaded from the trainer's checkpoint / shared-weight buffer): pull the aux trunk weight buffers (input projection, hidden, output projection) into eval's weight set. **Same source-of-truth pattern as the existing GEMM weights** — eval reads, doesn't update.
- The aux trunk has its own buffer (separate from policy trunk per `pearl_separate_aux_trunk_when_shared_starves`). Eval needs both: the shared encoder up to `h_s2_aux` (already loaded for the Q-network) AND the aux trunk weights that consume `h_s2_aux`.

**Discovery dependency**: locate where training initializes `exp_h_s2_aux` and the aux trunk weights, mirror the load path. Likely at the trainer's checkpoint-loading entry. Implementation will need to grep for `aux_trunk` / `aux_heads` / `h_s2_aux` references.

### A2.2 — Per-window `prev_aux_dir_prob` buffer at eval

New `GpuBacktestEvaluator` field:

```rust
/// SP22 H6 Phase 3 A2 (2026-05-13) — per-window recentered aux p_up cache.
/// Sized [n_windows] f32. Filled by `aux_softmax_to_per_env_kernel` after
/// the eval-time aux forward. Read by `backtest_state_gather` +
/// `backtest_state_gather_chunk` at the next step's state assembly.
/// Cold-start + per-evaluate() reset: 0.0 sentinel via fill_f32.
prev_aux_dir_prob: cudarc::driver::CudaSlice<f32>,
```

Init in `new()`: allocate `[n_windows]`, fill with `0.0` via `fill_f32` (same kernel as training-side; already pub(crate) via `EPSILON_GREEDY_CUBIN`).

Reset on each `evaluate()` call: re-fill to `0.0` so consecutive evaluations don't carry state across runs.

### A2.3 — Aux trunk forward at eval per-step

Inside `GpuBacktestEvaluator::evaluate`'s per-step loop, between encoder forward and action selection:

```
encoder_forward(states) → h_s2_shared, h_s2_aux
aux_trunk_forward(h_s2_aux) → aux_logits  [n_windows, K=2]
aux_softmax(aux_logits) → aux_softmax     [n_windows, K=2]
aux_softmax_to_per_env_kernel(aux_softmax → prev_aux_dir_prob)  [reuse from training]
```

`aux_trunk_forward` and `aux_softmax` can either:
- **(a)** Reuse the existing `AuxHeadsForward` orchestrator from the trainer (if its lifecycle allows eval-time use — i.e., no training-only state coupling).
- **(b)** Define a thin eval-only aux forward that calls the same underlying cuBLAS kernels with eval-side buffer pointers.

Choice in spec: **(a) reuse the trainer's orchestrator** if its API permits. Falls back to **(b)** if the orchestrator is too tightly coupled. Implementation phase determines which after reading `AuxHeadsForward` source.

### A2.4 — α at eval (post-Q_dir bias)

Inside the eval forward, after Q_dir is computed but before action selection:

```
aux_to_q_dir_bias_kernel(Q_dir, prev_aux_dir_prob, W_aux_to_Q_dir, n_windows, b0_size)
```

The same kernel as training-side α forward. W is read from the trainer's weight buffer (same shared-source-of-truth pattern). No backward at eval-time (eval doesn't update params).

### A2.5 — β at eval (trade-close bonus + 7-component migration in backtest_env_kernel)

`backtest_env_kernel.cu` has its own reward-component-writing logic (mirroring `experience_env_step`). Two changes:

- Migrate `backtest_env_kernel.cu` reward-component stride from 6 → 7 atomically with the training-side migration (`feedback_no_partial_refactor` — same buffer layout across both paths).
- Add the β producer site at `backtest_env_kernel.cu::segment_complete` (parallel to `experience_env_step::segment_complete`):

```c
/* SP22 H6 Phase 3 A2 β (2026-05-13): aux-aligned trade-close bonus at eval.
 * Same formula and NULL-fallback semantics as the training-side producer
 * in experience_env_step. Reads scale_beta from the same SP11 controller
 * ISV slot (training-emitted; eval reads). */
float position_sign = (prev_position > 0.001f) ? 1.0f
                   : (prev_position < -0.001f) ? -1.0f
                   : 0.0f;
float aux_at_close = (aux_dir_prob_per_env != NULL)
                   ? aux_dir_prob_per_env[w]
                   : 0.0f;  /* NULL → A3 fallback; should be non-NULL post-A2 */
float alignment = fmaxf(0.0f, aux_at_close * position_sign);
float profit_pos = fmaxf(0.0f, realized_pnl);
float scale_beta = (isv_signals != NULL)
                 ? isv_signals[SP22_AUX_ALIGN_SCALE_INDEX]
                 : 0.0f;
float r_aux_align = scale_beta * alignment * profit_pos;
reward_components_per_sample[w_off * 7 + 6] = r_aux_align;
```

### A2.6 — Wire `prev_aux_dir_prob` into the 3 state-gather launchers

Update `gpu_backtest_evaluator.rs::launch_gather`, the secondary `launch_gather` site, and `launch_gather_chunk` to pass `self.prev_aux_dir_prob.raw_ptr()` instead of `&0u64` for the `aux_dir_prob_per_env` argument.

### A2 NULL-safety

With A2.1–A2.6 wired, the eval-time NULL-fallbacks become unreachable in production (state-gather, state[121] from non-NULL buffer; β reads scale_beta from non-NULL ISV). The kernel-side NULL checks STAY for defensive programming (test scaffolds that don't initialize A2 still work). All 3 backtest state-gather kernels keep their `(ptr != NULL) ? ptr[w] : 0.0f` guards.

### A2 risks

| Risk | Mitigation |
|---|---|
| Aux trunk weight loading at eval requires checkpoint-format coupling | Reuse the same load path as the policy weights (already implemented for Q-network); aux trunk follows the same pattern per `pearl_separate_aux_trunk_when_shared_starves` |
| Eval-time aux forward adds latency to every backtest step | Aux trunk is a small head (per the trainer's `exp_h_s2_aux` allocation pattern); per-step cost is ~µs. Compute-sanitizer-verified at training cycle 1; eval-time parity expected. |
| Eval's `prev_aux_dir_prob` reset semantics don't match training's FoldReset | Resetting to 0 per `evaluate()` call gives eval a clean cold-start; matches Phase 2 sentinel semantics. Documented explicitly. |
| chunk-path evaluator runs aux forward at the wrong cadence | The chunk-path evaluator updates portfolio_buf + plan_isv_buf at chunk boundary only. Aux trunk should also fire at chunk boundary (not per-step within a chunk). The aux signal is then constant within a chunk — matching the existing portfolio/plan_isv constancy assumption in `backtest_state_gather_chunk`. |
| Eval-side reward-component allocation might not match the 7-slot training-side allocation | Caught by the 7-component contract migration (cargo check would fail on stride mismatch); same atomic migration covers eval-side `reward_components_per_sample`. |

## Out of scope (with explicit reasoning)

- **Aux trunk gradient flow back through state[121]**: not a deferral; a property statement. The aux trunk has stop-grad to policy per H6 design. The gradient that flows back from α through state[121] lands on the ENCODER's input weight column for slot 121, NOT through the aux trunk. The aux trunk stays untouched. Preserves `pearl_separate_aux_trunk_when_shared_starves`.

Everything else from the brainstorming "decision points" — β as 7th component, SP11 controller for scale_β, encoder-state[121] gradient routing, AND A2 eval-side wiring — is **in scope** for this spec per the user directives "no follow ups include in spec" + "do eval to[o]".

## Scope estimate (with A2 eval-side wiring)

| Phase | Effort | Risk |
|---|---|---|
| α: 2 new kernels (fwd, bwd) + build.rs reg | 4–6 hr | Medium (kernel correctness, backward shape) |
| α: trainer param + Adam wireup + 2 captured graphs (rollout + training) | 4–6 hr | Medium (graph capture for new kernels) |
| β: producer site in experience_env_step (writes rc[6] = r_aux_align) | 1–2 hr | Low |
| 7-component contract migration (atomic): reward_component_ema_kernel, reward_decomp_diag_kernel, reward_component_mag_ratio_compute_kernel, gpu_experience_collector buffer alloc + HEALTH_DIAG print, gpu_dqn_trainer buffer alloc + HEALTH_DIAG print, reward_component_monitor, backtest_env_kernel (eval-side reward stride) | 8–10 hr | Medium (cross-file contract; every consumer migrates atomically per `feedback_no_partial_refactor`) |
| SP11 controller extension: new `w_aux_align` output, anchor/target/bounds logic, new ISV slots (`SP22_AUX_ALIGN_SCALE_INDEX`, `REWARD_AUX_ALIGN_EMA_INDEX`), state_reset_registry entries, HEALTH_DIAG sp11_reward line extension | 4–6 hr | Medium (controller tuning sensitivity; cold-start verification) |
| A2.1–A2.3: aux trunk eval loading + forward + per-window buffer + softmax-to-per-env launch | 8–12 hr | Higher (eval-side aux infrastructure is greenfield; checkpoint-load coupling; aux orchestrator reuse vs eval-only thin variant) |
| A2.4: α at eval (post-Q_dir bias launch) — reuses training kernel | 1 hr | Low |
| A2.5: β at eval in backtest_env_kernel (mirror of training producer; covered in 7-component migration row above for the stride aspect) | 1–2 hr | Low |
| A2.6: wire non-NULL `prev_aux_dir_prob.raw_ptr()` into 3 backtest state-gather launchers | 30 min | Low |
| Verification gates (cargo check + tests + sanitizer + capture check + W-movement check + sp11_reward 7th-weight check + A2 eval-aux capture-pulse check) | 1.5 hr | Low |
| Audit doc + atomic commit + push | 30 min | Low |
| Smoke + verdict | ~37 min wall | Low |
| **Total** | **~34–47 hr engineering + smoke wait** | |

Roughly 5–6 working days of engineering before smoke dispatch. The bulk of the additional scope (vs the earlier "21–31 hr" within-phase-follow-ups version) is the A2 eval-side aux trunk infrastructure, which is genuinely greenfield code — the eval pipeline had no aux infrastructure before. A2 lets the eval verdict pipeline be a real production-parity check rather than running a partially-aux-blind backtest.

## Decision points carried forward from brainstorming

| Question | Decision (this spec) | Rationale |
|---|---|---|
| α: scalar bias vs 4-weight vector? | 4-weight vector | Aux conviction is directionally signed; per-action weight lets network learn `W_long > 0`, `W_short < 0` independently. 4 params is YAGNI-minimum for direction-aware bias. |
| β: non-negative-only vs signed? | Non-negative-only | Avoids confusing policy when aux is wrong (let r_trail's loss carry the negative). Self-correcting under correlation. |
| β: fixed scale vs SP11 controller? | **SP11 controller** (`SP22_AUX_ALIGN_SCALE_INDEX`) | Per the directive "no follow-ups": canonical pattern per `pearl_controller_anchors_isv_driven`. Anchor on `REWARD_AUX_ALIGN_EMA_INDEX`, target ~10% of total reward magnitude, bounds [0.05, 2.0]. Cold-start sentinel 0 per `pearl_first_observation_bootstrap` — β is no-op until first controller emit. |
| State[121] grad routing? | Option (i) — both paths open | Simplest implementation; α dominates initially due to near-zero encoder weights; encoder may eventually learn additively (belt + suspenders). Empirical observable: compare W magnitudes vs encoder column-norm for state[121]. |
| β placement: 7th reward component vs added to r_trail? | **Separate 7th component** `r_aux_align = rc[6]` | Per the directive "no follow-ups": full contract migration across reward_component_ema_kernel + reward_decomp_diag + reward_component_mag_ratio + gpu_experience_collector + gpu_dqn_trainer + reward_component_monitor + health_diag_kernel atomically per `feedback_no_partial_refactor`. Diagnosability (HEALTH_DIAG attribution), ablation support, and SP11 controller integration all become clean. |
| α W-sharing trainer ↔ collector? | Trainer holds; collector reads | Mirrors existing GEMM weight pattern; no race because Adam fires between rollouts. |
| A2 (eval-side α + β + aux trunk forward)? | **In scope** — full eval-side aux trunk forward + per-window `prev_aux_dir_prob` buffer + α post-Q_dir bias launch + β at backtest_env_kernel + non-NULL ptr to 3 state-gather launchers + 7-component migration to eval reward path | Per user directive "do eval to[o]". A2 brings the eval pipeline to production parity with training-time rollout — eval-time `total_pnl` / `sharpe` / `WindowMetrics` now reflect aux-guided policy behavior. Without A2, eval observed a "what the trained policy DOES choose" snapshot but with state[121]=0 (the policy's action selection at eval would still implicitly use α via the policy weights, but the eval-time forward could not benefit from real-time α reactivity to per-window aux signal). A2 closes that gap. |
