# SP22 H6 vNext — Trade-Outcome Aux Head

**Status**: SPEC. Implementation pending.
**Date**: 2026-05-13
**Branch (start point)**: `sp20-aux-h-fixed` @ commit after Phase 3 → dormant cleanup.

---

## Motivation: the architectural mismatch

The H6 Phase 3 mechanism (atom-shift via aux-predicted direction + β reward bonus) was implemented, debugged, and tested. Final smoke `train-xrkb7` @ `ebc7144434` produced val WR=0.4346 — statistically identical to all baselines (~0.4338-0.4346). The hypothesis is **falsified at this data regime + horizon**.

**Root cause** is not the wiring (it works) and not capacity (aux head 4× enlarged from 32 → 128 hidden units, accuracy improved 28% → 70% on imbalanced data). The root cause is **the aux head predicts the wrong thing**:

| What the policy decides | What aux predicts |
|-------------------------|-------------------|
| 4-branch action (direction × magnitude × order × urgency) | Binary next-bar direction (up/down) |
| Multi-bar trade with target_bars, profit_target, stop_loss | Single-bar close-price comparison at H bars ahead |
| Plan-conditioned (uses 6 plan_params) | Plan-ignorant (only sees h_s2_aux) |
| Operates over trade lifecycle (entry → hold → exit) | Operates at a single horizon |

The aux head answers "will close be higher in H bars?" when the policy asks "should I enter THIS trade with these plan parameters, expecting these outcomes?"

A binary direction predictor at a fixed horizon cannot provide the trade-success signal the policy needs. The 70% accuracy on imbalanced data dropped to 52% on balanced data — confirming aux's signal is **mostly majority-class prior + minimal per-bar skill**.

## Hypothesis (vNext)

**If the aux head predicts trade-level outcomes** (Profit / Stop / Timeout) **conditioned on the trade plan being considered**, and the policy uses this prediction to bias action selection toward likely-profitable trades, the WR will improve.

Specifically:
- Aux input includes `plan_params [6]` so it knows what trade is being considered
- Aux output is K=3 softmax over {Profit, Stop, Timeout}
- Labels computed at trade close (sparse but unambiguous)
- Atom-shift uses per-action × per-outcome W (4×3 = 12 weights)
- β reward shaping unchanged — already trade-level, just reads new aux signal

## Architecture

### Aux head input change

Current: `aux_next_bar_forward(h_s2_aux [256])`
vNext:   `aux_trade_outcome_forward(h_s2_aux [256] || plan_params [6])` = 262-dim input.

The concat happens BEFORE the aux head's Linear_1 layer. plan_params produced by the existing trade plan MLP (see `gpu_dqn_trainer.rs::launch_trade_plan_forward`) on the same forward pass.

Architecture:
```
Input  [262]  (h_s2_aux 256 + plan_params 6)
   ↓ Linear (W1 [128, 262]) + ELU
Hidden [128]  (matches AUX_HIDDEN_DIM)
   ↓ Linear (W2 [3, 128])
Logits [3]    (Profit, Stop, Timeout)
   ↓ Softmax
Probs  [3]
```

### Output: K=3 softmax over trade outcomes

| Class | Definition |
|-------|------------|
| 0 = **Profit** | Trade hit `unrealized ≥ profit_target × equity` before stop or timeout |
| 1 = **Stop** | Trade hit `unrealized ≤ -stop_loss × equity` before profit or timeout |
| 2 = **Timeout** | Trade reached `hold_time = target_bars` without hitting either |

### Label producer kernel: `trade_outcome_label_kernel.cu` (NEW)

Fires at every trade-close event (sparse — only a small fraction of bars have trades closing). Reads:
- `trade_close_per_sample [N*L]` flag from `experience_kernels.cu`
- `unrealized` at close
- `profit_target × equity` and `stop_loss × equity` thresholds
- `hold_time` and `target_bars`

Writes: `aux_outcome_label_per_step [n_episodes]` (i32 in `{-1, 0, 1, 2}` — `-1` = mask for bars without trade close).

### Aux loss kernel: `aux_trade_outcome_loss_reduce` (NEW or extend existing K-aware version)

Standard CE over K=3 classes with `-1` mask. Mirrors `aux_next_bar_loss_reduce`'s pattern. Sparse (most bars get -1 mask) — needs careful gradient scaling.

### Aux softmax → state slots

Replace single `state[121] = 2*p_up - 1` with three slots:
- `state[121] = p_Profit`
- `state[122] = p_Stop`
- `state[123] = p_Timeout`

(Reuses 3 of the 7 padding slots [121..128). The remaining 4 stay zero.)

Producer kernel: `aux_outcome_softmax_to_per_env_kernel.cu` (NEW, mirrors `aux_softmax_to_per_env_kernel`).

### Atom-shift: per-action × per-outcome W

`W_outcome [b0_size=4, K=3]` matrix. Atom-shift becomes:
```
aux_atom_shift[b, a] = Σ_k W_outcome[a, k] * p_outcome[k, b]
                     = W_outcome[a, 0] * p_Profit
                     + W_outcome[a, 1] * p_Stop
                     + W_outcome[a, 2] * p_Timeout
```

Structural prior:

| Action | W[Profit] | W[Stop] | W[Timeout] |
|--------|-----------|---------|------------|
| Short | +0.5 | −0.5 | 0 |
| Hold | 0 | +0.5 | 0 |
| Long | +0.5 | −0.5 | 0 |
| Flat | 0 | +0.5 | 0 |

Semantic: trade-directional actions (Short, Long) get boost when aux thinks the trade will be Profitable, suppressed when aux thinks it'll Stop out. Hold/Flat get the opposite bias — encouraged when Stop-prob high.

### β reward shaping

Unchanged structurally. The producer at `experience_kernels.cu::3614-3628` already fires at trade close. The change is `aux_at_close` now reads `p_Profit` (state slot 121 at trade-entry time) instead of `p_up`. Semantic becomes: "extra reward if aux predicted Profit at entry AND trade was profitable."

## Implementation phases

### Phase A — Foundation (kernels, label producer)

A1. Define new ISV slots (none needed — re-uses padding slots 121-123 for outputs)
A2. Add `aux_outcome_label_per_step_kernel.cu` — fires at trade close, writes K=3 label
A3. Add `aux_trade_outcome_forward_kernel.cu` (or extend `aux_next_bar_forward` with K=3 mode)
A4. Add `aux_trade_outcome_loss_reduce_kernel.cu`
A5. Add `aux_trade_outcome_backward_kernel.cu`
A6. Register all in `build.rs`

### Phase B — Plan-params concat input

B1. Modify aux head input to concat h_s2_aux + plan_params (262-dim)
B2. Update W1 shape `[AUX_HIDDEN_DIM, 262]` (was `[AUX_HIDDEN_DIM, 256]`)
B3. Update Xavier init for new fan_in
B4. Update aux trunk backward to receive plan_params gradient (or use stop-grad if simpler)
B5. Verify cargo check + nvcc clean

### Phase C — Multi-slot state assembly

C1. Modify `aux_outcome_softmax_to_per_env_kernel` — writes 3 slots instead of 1
C2. Update `state_layout.rs` constants (rename or add `AUX_TRADE_OUTCOME_PROFIT_INDEX=121`, etc.)
C3. Update `experience_state_gather` to populate slots 121-123 from the 3-element per-env buffer
C4. Update `assemble_state` to write 3 values, zero remaining 4 padding slots

### Phase D — Atom-shift extension to 12-weight W

D1. Change `w_aux_to_q_dir` from `[4]` to `[4, 3]` = 12 floats
D2. Update `aux_w_prior_init_kernel` to write 12 structural prior values
D3. Update `compute_expected_q` atom-shift computation to use per-outcome × per-action
D4. Update `c51_loss_kernel` atom-shift (eq_per_action shift + effective_reward)
D5. Update `mag_concat_qdir` atom-shift
D6. Update `quantile_q_select` atom-shift

### Phase E — dW backward + Adam for 12-weight W

E1. Update `c51_aux_dw_kernel` to compute dW[4, 3] = 12 partial gradients
E2. Update `adam_w_aux_kernel` to update 12 weights (grid (12,1,1), block (1,1,1))
E3. Update `dw_aux_buf` allocation [4] → [12]

### Phase F — Validation

F1. Verify cargo check + cargo build clean
F2. Smoke at structural prior (W populated, β=0.5)
F3. Capture aux_outcome_acc HEALTH_DIAG (NEW diagnostic — should land in P)
F4. If WR moves above 0.4346 baseline → validate; if not → re-evaluate aux signal source

### Estimated effort

**~2-3 days focused engineering**, dominated by Phase A (new kernels) and Phase E (12-weight dW reduce). Phase B-D are mostly re-shaping existing wiring.

## Key design decisions to validate before starting

1. **Should aux see plan_params or trade_plan output?**
   - Plan_params (6 numbers) is the proposal. Could also use full trade_plan hidden (256) for more context.
   - Recommendation: plan_params (6) — keeps aux head light, focuses on decision-relevant features.

2. **Sparse labels — how to handle?**
   - Most bars have no trade close → most aux labels are -1 mask.
   - Aux loss only fires on label != -1 bars. Effective batch size ~1-5% of nominal.
   - Per-trade labels are signal-rich though. Net signal/noise ratio likely better than per-bar direction.

3. **W structural prior magnitudes — symmetric?**
   - Current proposal symmetric: ±0.5. Could go larger since trade-outcome signal is more confident than per-bar.
   - Recommendation: start at ±0.5 symmetric, let Adam learn from there.

4. **Should aux predict trade outcome before OR after the policy commits?**
   - If aux predicts BEFORE: lookahead — aux can imagine "if I enter this trade now, what happens?"
   - If aux predicts AFTER: validates the policy's choice retrospectively
   - Recommendation: BEFORE — that's what makes it useful as a confidence gate.

## Open questions

1. Does the rollout collect trade_close events accurately for label production? (Verify via `experience_kernels.cu::trade_close_per_sample`)
2. What's the trade-close event rate? Affects gradient signal strength.
3. Should regime classification (existing `aux_regime_kernel`) be incorporated as an additional gating signal? Could enhance further but adds scope.

## Relationship to current infrastructure

vNext **reuses** ~80% of Phase 3 infrastructure:
- Atom-shift kernel infrastructure (compute_expected_q, c51_loss, mag_concat_qdir, quantile_q_select all already wired)
- Defensive NaN guards
- Step 8 (c51_aux_dw_kernel) — needs reshaping for 12-weight W but pattern proven
- Step 11 (adam_w_aux_kernel) — same
- Phase C1 (collector W ptr setter)
- β reward producer (`experience_kernels.cu:3614-3628`)
- HEALTH_DIAG diagnostic infrastructure

vNext **replaces**:
- `aux_next_bar_*` kernels → `aux_trade_outcome_*` kernels (Phase A)
- Single state slot 121 → three slots 121-123 (Phase C)
- W [4] → W [4, 3] (Phase D)

## Falsification criteria

If full vNext implementation produces WR ≈ 0.4346 ± 0.005, the trade-outcome hypothesis is also falsified. At that point, the conclusion is **WR is data-determined at this bar resolution + horizon, not a model architecture issue**. Future work should explore:
- Different bar sampling (volume bars instead of imbalance bars)
- Lower-resolution data (1-min OHLC instead of MBP-10)
- Different prediction target entirely (volatility regime, regime transitions, etc.)
