# SP10 — Unconditional Thompson Selector + ISV-driven Temperature

**Date:** 2026-05-03
**Status:** Draft (user review pending)
**Authors:** session-1777830566225
**Related pearls:**
- `pearl_thompson_for_distributional_action_selection.md` (existing — selector/target asymmetry)
- `pearl_controller_anchors_isv_driven.md` (anchors must be ISV-driven)
- `pearl_blend_formulas_must_have_permanent_floor.md` (permanent floor pattern)
- `pearl_cold_start_exit_signal_or.md` (OR'd cold-start signals)

## 1. Overview

Eliminate argmax-at-eval bias in the direction-branch action selector by
making the selector unconditional Thompson sampling with an ISV-driven
temperature parameter. The temperature is signal-driven from SP9's
existing `intent_eval_divergence` canary: high divergence (eval collapsed)
→ raise τ (more exploration); low divergence (healthy) → lower τ (closer
to argmax-like exploitation).

This is the next link in the Fix 33→37 chain. Fix 33-37 addressed
training-side controller dynamics; SP10 addresses the eval-side selector
that was producing val-Flat-collapse regardless of controller state.

## 2. Pathology

T10 train-multi-seed-khr7c ep2 (commit `8a25b330f`, all SP9 fixes
applied):

```
val: trade_count=1 in 214,654 bars
     active_frac=0.0 dir_entropy=0.0 sharpe=0.0
sp9_kelly_warmup [floor=0.25 (engaged)
                  divergence=500000+ (eval collapsed)
                  conf [stat=1.0 bhv=0.0 tmp=0.2]]
```

Despite Kelly warmup floor=0.25 (allowing Half-mag exposure), Kelly
controllers all healthy, IQN warnings absent — the model picks ONE
direction (Hold) at every val bar. dir_entropy=0 means deterministic
single-action over the whole 214k window.

Root cause: `experience_action_select` kernel uses `argmax(E[Q])` at
eval (line 1208). With Hold's E[Q] ≈ 0 (no position cost) and
Short/Long's E[Q] = ε (small edge minus tx costs), argmax wins Hold
deterministically every bar. Thompson sampling would let positive-Q
directions win proportional to their posterior overlap with Hold.

The `pearl_thompson_for_distributional_action_selection` was about
selector/target asymmetry in the **Bellman target** (DDQN target Q
computation). It does NOT prohibit Thompson at the rollout selector;
only at the target. SP10 honors that distinction.

## 3. Architecture

### 3.1 Kernel change

Replace the `if (eval_mode)` branch in `experience_action_select` with
unconditional Thompson sampling, modulated by an ISV-driven temperature:

```cuda
// Read ISV-driven temperature
float temp = isv_signals[EVAL_THOMPSON_TEMP_INDEX];

// Per-direction Thompson sample with temperature blend
float best_q_eff = -1e30f;
int best_d = 0;
for (int d = 0; d < b0_size; d++) {
    softmax_c51_inline(logits_i + d * n_atoms, n_atoms, probs_d);
    compute_atom_values_inline(per_sample_support, atom_positions,
                               i, d, b0_size, n_atoms, atom_vals_d);
    float u = philox_uniform(i, timestep, rng_ctr++);
    float q_sample = sample_c51_inverse_cdf(probs_d, atom_vals_d, n_atoms, u);

    // Temperature blend: τ=0 → argmax of E[Q]; τ=1 → pure Thompson;
    // τ>1 → exaggerated exploration (corrective during collapse).
    float q_eff = e_dir[d] + temp * (q_sample - e_dir[d]);

    if (q_eff > best_q_eff) { best_q_eff = q_eff; best_d = d; }
}
dir_idx = best_d;
```

The `if (eval_mode)` branch and its argmax body are deleted entirely.
No conditional path remains — one selector for both training and eval.

### 3.2 Temperature formula (ISV producer)

Extended from SP9's existing `intent_eval_divergence_compute_kernel`. The
existing kernel already computes `divergence`. Add a second scratch write:

```cuda
// Existing SP9 output:
scratch_buf[scratch_div_idx] = divergence;

// SP10 addition: temperature output, using same divergence + target
float div_target = isv_signals[KELLY_DIVERGENCE_TARGET_INDEX];  // = 2.0 (Invariant-1)
float temp = clamp(divergence / max(div_target, EPS_DIV),
                   THOMPSON_TEMP_MIN, THOMPSON_TEMP_MAX);
scratch_buf[scratch_temp_idx] = temp;
```

Where:
- `THOMPSON_TEMP_MIN = 0.5f` (Invariant 1: permanent stochasticity floor per
  `pearl_blend_formulas_must_have_permanent_floor`; never fully deterministic)
- `THOMPSON_TEMP_MAX = 2.0f` (Invariant 1: cap on exploration boost; prevents
  pathological all-uniform sampling)
- `EPS_DIV = 1e-6f` (Invariant 1: divide-by-zero protection)

Behavior:
- Healthy (`divergence ≈ target = 2.0`): `temp ≈ 1.0` → pure Thompson
- Collapsed (`divergence ≫ target`, e.g., 1e6 sentinel): `temp = 2.0` → exploration boost
- Healthy and converging (`divergence < target`): `temp = 0.5` → closer to argmax behavior, but never fully deterministic

The MIN_TEMP=0.5 floor is the permanent-stochasticity guarantee per
`pearl_blend_formulas_must_have_permanent_floor`. Eval action selection
ALWAYS has at least 50% Thompson contribution.

### 3.3 ISV slot allocation

After SP9's last slot at 338:

| Slot | Purpose | Reset semantics |
|---|---|---|
| `EVAL_THOMPSON_TEMP_INDEX = 339` | Action selector temperature | FoldReset (sentinel 0, Pearl A bootstrap to MIN_TEMP) |

Bump `SP5_SLOT_END = 340` and `ISV_TOTAL_DIM = 340`.

Add 1 new scratch slot for the temperature output.

### 3.4 Producer kernel — extend or new?

**Decision: extend existing `intent_eval_divergence_compute_kernel`** rather
than create a separate producer. Reasons:

1. The temperature is a deterministic function of `divergence` (no new
   inputs needed)
2. Both outputs come from the same compute (cheap to add one extra write)
3. Atomic commit per `feedback_no_partial_refactor` — slot + kernel
   extension + consumer migration all together
4. Matches `pearl_engagement_rate_self_correction` pattern (one signal,
   multiple consumers)

The kernel signature gains 2 new parameters (scratch_temp_idx and
divergence_target_isv_index) but otherwise unchanged.

## 4. Pearl amendments

### 4.1 `pearl_thompson_for_distributional_action_selection.md`

Update wording to clarify the selector/target distinction:
- BEFORE: "Thompson at training, argmax at eval"
- AFTER: "Thompson at all times for the rollout selector; argmax ONLY for the
  Bellman target Q computation (DDQN target). Temperature on the Thompson
  sample is ISV-driven from intent_eval_divergence per
  pearl_controller_anchors_isv_driven."

### 4.2 No new pearl

The principle here is captured by combining existing pearls:
- `pearl_thompson_for_distributional_action_selection` (amended) — selector vs target
- `pearl_controller_anchors_isv_driven` — the temperature is ISV-driven
- `pearl_blend_formulas_must_have_permanent_floor` — MIN_TEMP=0.5 floor
- `pearl_engagement_rate_self_correction` — temperature self-corrects via divergence

## 5. Test changes

`crates/ml/src/cuda_pipeline/mod.rs:1242` — `test_eval_action_select_eval_argmax_picks_best`:

- **Rename** to `test_eval_action_select_thompson_picks_proportionally`
- **Update** assertion: instead of "best direction always wins", assert
  "best direction wins with probability ≥ posterior weight of best
  direction" (allowing the temperature blend's stochasticity)
- **Histogram check**: with τ=1.0 and a clearly-best direction (large
  Q gap), best wins ≥ 80% of samples; with smaller gap, ≥ 50%

Existing comment at `mod.rs:1370` ("Thompson-eval histogram: short=...
long=... (expected: P(Long)=1.0)") suggests prior preparation for this
case — test framework already aware of Thompson-eval semantics.

## 6. Implementation phases (atomic per feedback_no_partial_refactor)

Single commit:

1. Allocate 1 new ISV slot (EVAL_THOMPSON_TEMP_INDEX=339), bump
   SP5_SLOT_END and ISV_TOTAL_DIM
2. Extend `intent_eval_divergence_compute_kernel.cu` with temperature
   output (~10 LOC addition + signature change)
3. Update launcher in `gpu_dqn_trainer.rs` for new kernel signature
4. Modify `experience_action_select` kernel: delete `if (eval_mode)` argmax
   branch, replace Thompson branch with temperature-blended version, add
   `EVAL_THOMPSON_TEMP_INDEX` ISV read at top of direction-selection block
5. Update FoldReset registry: 1 new slot entry, 1 new dispatch arm
6. Update test `test_eval_action_select_eval_argmax_picks_best` →
   `test_eval_action_select_thompson_picks_proportionally`
7. Update pearl `pearl_thompson_for_distributional_action_selection.md`
   in MEMORY (selector/target wording)
8. Audit doc Fix 38 entry

Total estimated: ~300-500 LOC across ~6 files. ~45-60 min focused work.

## 7. Validation criteria

After SP10 lands:
- Smoke (5-epoch L40S `multi_fold_convergence`):
  - val_active_frac > 0.10 with τ ≥ 0.5 (always some action variety)
  - dir_entropy at eval > 0.3 (multi-direction selection)
  - HEALTH_DIAG sp10 line shows τ evolving with divergence
- T10 50-epoch:
  - val_trade_count > 1000 in 214k-bar window (was 1)
  - val_active_frac > 0.20 across folds
  - sharpe positive net of fold transitions

## 8. Risks and open questions

- **Risk 1**: Pure Thompson eval introduces PnL variance — short windows
  show more sharpe noise. Mitigation: τ floor at 0.5, not full Thompson;
  diagnostic is multi-fold mean sharpe, not single-fold spot value.
- **Risk 2**: Production trading reproducibility — Philox RNG + state +
  timestep makes Thompson deterministic given seed. Same model + same
  seed = same actions. Reproducibility is preserved at the seed level.
- **Risk 3**: The pearl amendment may conflict with intent at the time it
  was written. Per `feedback_trust_code_not_docs`, the smoke evidence
  trumps the original wording; pearl gets updated to reflect reality.

## 9. Followups (NOT in this commit)

- Adaptive temperature beyond the divergence canary (e.g., per-state
  uncertainty from Q variance, regime-dependent τ)
- Apply same Thompson selector logic to mag/ord/urg branches if their
  argmax-at-eval pathways are found to bias similarly (currently mag
  uses Boltzmann softmax, ord/urg also stochastic — likely fine)
- Bellman target Q computation review: confirm DDQN target argmax stays
  argmax (no SP10 change there)
