# AUC-Regret Controller Design Spec

**Status:** ⛔ SUPERSEDED — empirically falsified by the diagnostic this spec proposed. See §S below.
**Date:** 2026-05-18 (drafted), 2026-05-18 (superseded same day)
**Branch:** `ml-alpha-phase-a`
**Owner:** ml-alpha team

## S. Superseded — empirical falsification (2026-05-18)

The spec's core motivation was that `λ` saturating at the BCE-z-score ceiling **does not predict AUC gain**, citing a 50% hit-rate observed on Option-2 fold 0 (gm67g). That was a 4-saturated-event sample on a config (Option 2 / commit `004b662c8`) that subsequently proved to be a *regression* vs the actual production baseline (ISV-σ at `410ab6b0e`).

When the same diagnostic is run on the **real ISV-σ baseline** (rxm5t/r57lx/x24d6 — 3 folds, 215 total horizon-epoch observations, logs retrieved from `argo-logs` bucket on MinIO):

| | n | AUC up next epoch | Hit rate | Median ΔAUC |
|---|---|---|---|---|
| λ saturated (≥1.99) | 13 | 8 / 5 down | **61.5%** | **+0.00245** |
| λ not saturated | 202 | 96 / 106 down | 47.5% | −0.00120 |

**+14 pp positive saturation→AUC-up signal.** Median ΔAUC sign is correct (positive on the boosted horizon, negative when no boost). The BCE-z-score controller is doing what it's supposed to do — boosting horizons whose AUC then improves.

The diagnostic n=13 saturated events is small; a binomial z-test against the null hypothesis (p=0.484) gives p≈0.34 — not statistically significant at p<0.05. But the conservative reading is clear: **no evidence the controller is misaligned with AUC**. The premise of this spec ("controller is fighting AUC") is unsupported by data.

**Decision**: do not implement. The BCE-z-score controller is retained at production. The production HEAD (`e004d6c21`) keeps the existing ISV-σ + adaptive Z_SCALE design.

This spec is kept in tree as a historical record of an architectural hypothesis that was tested and falsified. Future controller work (if pursued) should:
1. Re-verify the diagnostic on the latest production config before redesigning.
2. Require statistically significant evidence (n ≥ 50 saturated events, p < 0.05) before committing implementation cycles.
3. Consider the AUC-regret framework only if BCE-z-score saturation→AUC correlation drops below 50%.

The methodology of the diagnostic itself is valuable and worth preserving — see `/tmp/isv_sigma_logs/` and the saturation-hit-rate Python snippet in the conversation transcript for the analytical pattern.

---

## 0. Context (original draft preserved below as record)

The per-horizon λ controller in `horizon_lambda.cu` scales each horizon's trunk-grad contribution in `heads_grn_bwd` by `λ[h] ∈ [1.0, 2.0]`. Per `pearl_adam_normalizes_loss_weights`, scaling head-grad would be Adam-cancelled; scaling **trunk-grad works** because trunk parameters are shared across horizons — boosting one horizon's contribution shifts the trunk's effective update direction in that horizon's favor while Adam normalizes the aggregate.

The current controller computes `λ[h]` from a **z-score of `loss_ema[h]`** (per-horizon EMA of unweighted BCE):

```
z[h]          = (loss_ema[h] - mean) / std
Z_SCALE_ISV   = 1.0 / max(z_max_ema, 0.1)
λ[h]          = clamp(1.0, 2.0, 1.0 + Z_SCALE_ISV * z[h])
```

**Empirical problem (gm67g fold-0, 20 epochs):** when `λ_h6000` saturated at 2.0 (4 of 20 epochs), h6000 AUC went UP on the next epoch in 2/4 cases and DOWN in 2/4. Saturation does not predict AUC gain — the BCE signal is misaligned with the optimization objective.

**Root cause:** BCE measures distance from labels, not predictability. A horizon can have high BCE for three reasons, only one of which benefits from boosting:

| Cause | Boost helps? |
|---|---|
| Genuinely under-trained — encoder hasn't found the signal yet | **Yes** |
| Intrinsically harder horizon (e.g., h6000 has more inherent price-walk uncertainty) | **No** — BCE floor is structural |
| Calibration drift (probs miscalibrated but ranking fine — `pearl_auc_accuracy_gap_means_miscalibration`) | **No** — AUC unchanged by ranking-preserving recalibration |

For our 5 horizons {30, 100, 300, 1000, 6000}, cause #2 dominates h6000. The current controller almost always identifies h6000 as the "worst" horizon and boosts it — but the BCE floor is structural, so the boost spends encoder capacity on a horizon whose loss can't get materially lower.

**This spec replaces BCE-z-score with AUC-regret as the controller signal.** The new controller boosts horizons that have fallen below their *personal-best* AUC — i.e., horizons that demonstrably *were* learnable and now aren't.

Out of scope (separate work):
- Changing the consumption point (still `heads_grn_bwd` per-horizon trunk-grad scaling).
- Changing the λ envelope `[1.0, 2.0]`.
- Adding a meta-learner over λ. Per `pearl_controller_anchors_isv_driven`, the controller stays signal-driven.
- Trainable controllers. The whole point is principled signal-driven adaptation.

## 1. Goal

Per-epoch validation AUC drives λ. A horizon that's at its personal-best AUC contributes uniform grad (λ=1). A horizon that's regressed from its personal-best AUC contributes boosted grad (λ up to 2). The historical-max regret across horizons maps exactly to the ceiling (Z_SCALE_ISV self-calibrating).

**Falsifiable success criterion:** at this commit, on the same `seed=16962` 3-fold CV at 30 epochs as the Option-2 baseline, **mean_auc lifts by ≥ +0.005** (one full noise σ from prior runs) AND **the median saturation→next-epoch-AUC delta is positive** (i.e., when the controller boosts, the boosted horizon actually improves more often than not).

## 2. Math

### State (carried across training steps)

| Variable | Shape | Description |
|---|---|---|
| `best_auc[h]` | [5] | Running max of per-horizon validation AUC since training start |
| `regret_max_ema` | [1] | EMA of `max_h regret[h]` across epochs (the adaptive scale anchor) |

Both initialised to 0 (sentinel — first-observation bootstrap per `pearl_first_observation_bootstrap`).

### Per-epoch update (after validation completes)

Input: `current_auc[5]` from the validation pass (already computed for the `validation` log line).

```text
# 1. Update best_auc (running max — strict, no decay).
for h in 0..5:
    best_auc[h] = max(best_auc[h], current_auc[h])

# 2. Compute regret. ≥ 0 by construction.
regret[h] = max(0.0, best_auc[h] - current_auc[h])

# 3. Update regret_max_ema (first-observation bootstrap, fixed α=0.1).
r_max = max_h regret[h]
if regret_max_ema <= 0:
    regret_max_ema = r_max                    # sentinel bootstrap
else:
    regret_max_ema = (1-α) * regret_max_ema + α * r_max

# 4. Adaptive scale — by design, observed-max-regret maps to ceiling.
Z_SCALE_ISV = (LAMBDA_CEILING - LAMBDA_FLOOR) / max(regret_max_ema, REGRET_FLOOR)
            = 1.0 / max(regret_max_ema, 0.001)

# 5. Per-horizon λ.
for h in 0..5:
    raw      = 1.0 + Z_SCALE_ISV * regret[h]
    λ[h]     = clamp(LAMBDA_FLOOR=1.0, LAMBDA_CEILING=2.0, raw)
```

### Properties

- **At personal best**: `regret[h]=0` → `λ[h]=1.0`. Uniform encoder grad.
- **Max regret**: `regret[h]=regret_max_ema` → `λ[h]=2.0`. Full boost.
- **No regret anywhere**: `r_max=0` → `regret_max_ema` floor at `0.001` keeps `Z_SCALE_ISV` finite. All `regret[h]=0` → all `λ[h]=1.0`. Controller dormant.
- **Cold start (e0)**: `best_auc` is 0 → after first validation, `best_auc[h] = current_auc[h]` for all h → `regret[h]=0` → uniform λ. Controller engages only at e1+.
- **Persistent regression**: if a horizon stays below its best for many epochs, regret stays high, λ stays boosted. The controller doesn't time out.
- **No hardcoded magic numbers** beyond bootstrap epsilons (`REGRET_FLOOR=0.001`, EMA `α=0.1` matching the existing loss-EMA cadence) and the clamp envelope (kept at `[1.0, 2.0]` for direct comparability with the current controller).

## 3. Removed: BCE-z-score signal

The existing kernel computes `loss_ema[h]` from `loss_per_horizon` emitted by the BCE kernel, z-scores across horizons, EMA-tracks `z_max`, and produces λ.

**This pipeline is fully replaced.** Specifically:

- `loss_ema[5]` state → **REMOVED** from the controller. (Still emitted by the BCE kernel for telemetry, but no longer consumed.)
- `z_max_ema[1]` state → **REMOVED**.
- `log_sigma_h[5]` output → **KEPT as-is**. The closed-form Kendall σ from loss_ema is preserved for telemetry (BCE kernel ignores it per the σ-sidecar commit `17eb82511`, so it's a pure observability output). σ still uses `loss_ema` internally; that loss_ema computation moves to a dedicated `horizon_loss_ema` kernel.

After this commit:
- `horizon_loss_ema.cu` (new, tiny): maintains `loss_ema[5]` and `log_sigma_h[5]`. Called from the trainer at the same point in the step as today.
- `horizon_lambda.cu` (rewritten): consumes `current_auc[5]` + maintains `best_auc[5]` + `regret_max_ema[1]`, emits `lambda[5]`. Called from the trainer **once per epoch** (after validation), not per step.

The per-step call frequency change is significant. See §5 for trainer wiring.

## 4. Inputs / outputs

### `horizon_loss_ema` kernel (new, split off from current)

```cuda
extern "C" __global__ void horizon_loss_ema(
    const float* __restrict__ loss_per_horizon,  // [5] from BCE kernel
    float* __restrict__ loss_ema,                 // [5] state read+write
    float* __restrict__ log_sigma_h               // [5] output (sidecar)
);
```

Single thread, single block. Updates `loss_ema[h]` with sentinel bootstrap + fixed α=0.1; computes `log_sigma_h[h] = max(log(0.5), 0.5 * log(loss_ema[h] + ε))` (preserves the existing σ-sidecar behavior). Called per-step from the trainer hot loop (same place `horizon_ema_and_lambda` is called today).

### `horizon_lambda` kernel (rewritten)

```cuda
extern "C" __global__ void horizon_lambda(
    const float* __restrict__ current_auc,     // [5] from this epoch's validation
    float* __restrict__ best_auc,               // [5] state read+write
    float* __restrict__ regret_max_ema,         // [1] state read+write
    float* __restrict__ lambda                  // [5] output
);
```

Single thread, single block. Implements the math in §2. Called **once per epoch** from `alpha_train.rs` after validation completes.

The trainer passes `current_auc` from the existing per-horizon AUC computation (the same values that populate the `auc_h30..auc_h6000` keys on the `validation` log line and `best_val_auc` in the summary). No new computation needed.

## 5. Trainer wiring (`crates/ml-alpha/src/trainer/perception.rs` + `alpha_train.rs`)

### New `PerceptionTrainer` state

```rust
// AUC-regret controller state.
best_auc_d:        CudaSlice<f32>,   // [5], zero-init (sentinel)
regret_max_ema_d:  CudaSlice<f32>,   // [1], zero-init (sentinel)
horizon_lambda_fn: CudaFunction,     // existing handle, same name
// New:
horizon_loss_ema_fn: CudaFunction,   // split off from horizon_lambda
```

### Removed state

```rust
z_max_ema_d:  CudaSlice<f32>,   // [1] — DROPPED (was used by BCE z-score controller)
```

`loss_ema_d` and `log_sigma_h_d` remain (both still used by the new `horizon_loss_ema` kernel + BCE kernel's σ-sidecar signature stability).

### Per-step trainer call (in K-loop body)

```rust
// Was: launch horizon_ema_and_lambda(loss_per_h, loss_ema, z_max_ema, lambda, log_sigma_h)
// Now: launch horizon_loss_ema(loss_per_h, loss_ema, log_sigma_h)
self.stream.launch_builder(&self.horizon_loss_ema_fn)
    .arg(&self.loss_per_horizon_d)
    .arg(&mut self.loss_ema_d)
    .arg(&mut self.log_sigma_h_d)
    .launch(cfg_single)?;
```

λ does NOT update per-step anymore. It uses the value computed at the last epoch boundary.

### Per-epoch trainer call (new entry point, called from `alpha_train.rs`)

```rust
impl PerceptionTrainer {
    pub fn update_lambda_from_auc(&mut self, current_auc: &[f32; N_HORIZONS]) -> Result<()> {
        // Upload current_auc to device, launch horizon_lambda kernel,
        // λ buffer is now populated for the next training epoch's K-loop.
    }
}
```

In `alpha_train.rs` after computing `per_horizon_auc` for the validation pass:

```rust
trainer.update_lambda_from_auc(&per_horizon_auc)
    .context("update lambda from validation AUC")?;
```

### Cold-start handling

At e0, before validation runs, `best_auc_d = [0; 5]` and `regret_max_ema_d = [0]`. The trainer hot loop's first epoch reads `lambda` which is also zero-initialised → all-zero λ → kernel sentinel logic (`l <= 0 ? 1.0 : l`) in `grn_bwd` already converts to uniform λ=1.0 per `s_lambda` cooperative-load logic. **No special-case code needed in the trainer**; the existing sentinel handles e0 transparently.

After validation at end of e0, `update_lambda_from_auc` runs with `current_auc[h] = first-observation`. Since `best_auc[h] = 0` initially, `best_auc[h] := max(0, current_auc[h]) = current_auc[h]` → regret=0 everywhere → λ=1.0. Same uniform behavior as the sentinel, just emitted from the kernel.

E1 onwards, regret can become non-zero whenever a horizon dips below its observed maximum.

## 6. Telemetry / observability

The existing `isv snapshot` log line at validation end currently logs `ema_h*` (loss_ema) and `lam_h*` (the λ values). **Add `best_auc_h*` and `regret_h*` to that log line** so the per-epoch trajectory in `/tmp/alpha_status.py wf <name>` shows the full controller state.

```rust
tracing::info!(
    epoch,
    // existing:
    ema_h30 = ema[0], /* ... */ ema_h6000 = ema[4],
    lam_h30 = lam[0], /* ... */ lam_h6000 = lam[4],
    // new (AUC-regret state):
    best_auc_h30 = best_auc[0], /* ... */ best_auc_h6000 = best_auc[4],
    regret_h30 = regret[0],     /* ... */ regret_h6000 = regret[4],
    "isv snapshot"
);
```

The monitor's `parse_train_log` already handles unknown extra fields gracefully (regex only captures explicit captures); add a parse pass for these new fields.

## 7. Failure modes & mitigations

1. **All horizons at personal best forever** — `regret=0` everywhere, λ=1.0 uniform forever. The controller is dormant. *This is the desired behavior* — if the model never regresses on any horizon, the controller has nothing useful to do. The encoder learns uniformly.

2. **Best-AUC drift on noisy validation** — if a validation seed quirk produces an outlier-high AUC on some horizon at e3, `best_auc[h]` pins to that outlier, and every subsequent epoch shows positive regret. Boost stays on indefinitely.

   **Mitigation**: validation uses a deterministic seed (`val_loader` seed = `train_seed + 0xC0FFEE`); same validation data every epoch. So outliers are not from sampling noise — they're real model behavior on that fixed eval set. If the model genuinely degraded, persistent regret is the correct signal.

   Open question: should `best_auc` decay with time? Possible failure: model finds a high-AUC local optimum at e3 then diverges. With strict-max best_auc, the controller will keep trying to drag it back; with EMA-smoothed best_auc, it would let go. **Spec choice: strict max (no decay)**. Rationale: the strict signal is more interpretable; if the model genuinely diverges, early-stop fires (patience=5 on mean_auc) and we end the run anyway.

3. **Saturation rate too high / too low** — if `regret_max_ema` is too small (most regrets near 0), `Z_SCALE_ISV` is large and even tiny regrets saturate λ. The `REGRET_FLOOR=0.001` clamps this; effectively the smallest regret that produces λ=2.0 saturation is ≥ 0.001 in absolute AUC. That's ~0.1 percentage points — reasonable. If you think it's too sensitive, raise the floor.

4. **K-loop reads λ before per-epoch update** — the K-loop reads λ values from `self.lambda_d`. After epoch N's validation, we update λ via `update_lambda_from_auc`. The next training epoch's K-loop reads the new λ. Race-free because validation happens between training epochs on the same stream.

5. **Cuda-graph capture compatibility** — the K-loop runs inside a captured CUDA graph. The graph reads `lambda_d` device pointer; we update the *contents* of that buffer between captures (per epoch), not the pointer. Compatible with graph capture (data update is allowed; pointer change isn't). Same model the current controller uses.

## 8. Test plan

### Local (sm_86, RTX 3050)

1. **Build + unit test**: `cargo test -p ml-alpha --release --test perception_overfit` — all 9 tests pass. The overfit smoke validates the K-loop end-to-end including the new controller wiring.

2. **Synthetic controller test** (new): construct synthetic `current_auc` sequences and assert the controller produces expected λ values.
   - All-zero current_auc → all λ = 1.0 (cold start).
   - Monotonically-improving AUC → all λ = 1.0 (no regret).
   - One horizon regresses by 0.05 from best → that horizon's λ ≈ 2.0 (saturates at observed-max-regret), others ≈ 1.0.
   - All horizons regress equally → all λ ≈ 2.0 (uniform max).
   - File: `crates/ml-alpha/tests/horizon_lambda_auc_regret.rs`.

### Cluster

3. **5-epoch single-fold smoke** at the new commit. Check the `isv snapshot` log lines show non-zero regret values from e1 onwards, and `best_auc_h*` is monotonically non-decreasing. Confirm λ is correlated with regret (high regret → high λ).

4. **3-fold validation** at same `seed=16962`, 30 epochs. Compare to Option-2 baseline (`004b662c8`):
   - Primary success criterion: `mean_auc` lifts by ≥ +0.005 averaged over 3 folds.
   - Secondary: median saturation→next-epoch-AUC delta is positive.
   - If primary fails but secondary positive: the controller is doing the right thing per-horizon but doesn't move the mean — possible the per-horizon AUC distribution is fine and the gain is in stability (std reduction). Report and discuss.

## 9. Implementation surface (file-level summary)

| File | Change |
|---|---|
| `crates/ml-alpha/cuda/horizon_loss_ema.cu` | NEW — small kernel (loss_ema EMA + closed-form log σ) |
| `crates/ml-alpha/cuda/horizon_lambda.cu` | REWRITE — AUC-regret math (signature change: now consumes `current_auc[5]`) |
| `crates/ml-alpha/build.rs` | Add `horizon_loss_ema` to kernel list |
| `crates/ml-alpha/src/trainer/perception.rs` | Replace `z_max_ema_d` with `best_auc_d` + `regret_max_ema_d`. Replace per-step `horizon_ema_and_lambda` launch with per-step `horizon_loss_ema`. Add per-epoch `update_lambda_from_auc(per_horizon_auc)` method. |
| `crates/ml-alpha/examples/alpha_train.rs` | Call `trainer.update_lambda_from_auc(&per_horizon_auc)` after the existing `validation` log line. Extend the `isv snapshot` log line with `best_auc_h*` and `regret_h*` fields. |
| `crates/ml-alpha/tests/horizon_lambda_auc_regret.rs` | NEW — synthetic controller correctness tests |
| `/tmp/alpha_monitor.py` | Extend `parse_train_log` to capture `best_auc_h*` and `regret_h*` from the `isv snapshot` line; expose in the wf-detail status view |

Estimate: ~250 LOC across kernels + Rust, plus ~50 LOC of new tests.

## 10. Open questions for review

1. **Best-AUC decay vs strict-max?** Spec choice: strict max. Alternative: EMA-smoothed best (e.g., α=0.05) lets the controller "forget" outliers. *Recommendation*: strict-max for the first round; revisit if outlier-pinning becomes a problem.

2. **Per-epoch update vs per-K-step?** Spec choice: per-epoch (the cheapest, most-aligned cadence). Alternative: per-step λ update using a running estimate of validation AUC (e.g., online AUC from the training mini-batch's logits). *Recommendation*: per-epoch is sufficient — λ is consumed 64 times per training step in the K-loop, so even per-epoch λ gets ~512K applications per fold.

3. **Should the existing σ sidecar move out of horizon_loss_ema too?** Currently σ is wired into `loss_ema` computation. The σ kernel produces `log_sigma_h` from `loss_ema` in closed form (the σ-sidecar from `17eb82511`). Cleanest: bundle σ into `horizon_loss_ema` (one fused kernel). Alternative: split σ into its own kernel for clarity. *Recommendation*: bundle for now (one fewer kernel launch per step); split if it becomes a clarity issue.

4. **`REGRET_FLOOR` value?** Spec choice: `0.001` (0.1pp AUC). Sensitivity: smaller floor → controller saturates on smaller regressions (more reactive); larger floor → only big regressions trigger boost. *Recommendation*: 0.001 is the right order of magnitude; tunable but no need to make ISV-driven for the first round.

5. **Should we also expose `is_personal_best[5]` as a per-epoch boolean log field?** Useful for monitoring "which horizons just hit a new high?" Tiny addition. *Recommendation*: yes, costs 5 booleans / epoch.

## 11. References

- `pearl_controller_anchors_isv_driven` — every controller anchor/target/cap is signal-driven, never hardcoded.
- `pearl_adam_normalizes_loss_weights` — Adam cancels uniform per-loss weight lifts; effective lever is trunk-grad scaling, not loss aggregate.
- `pearl_first_observation_bootstrap` — sentinel = 0, first observation replaces (not blends).
- `pearl_audit_unboundedness_for_implicit_asymmetry` — asymmetric clamp (boost-only) preserved.
- `pearl_zscore_normalization_for_magnitude_asymmetric_signals` — historical motivation for the prior z-score design; explicitly REPLACED here because the signal (BCE) was wrong, not the math.
- `pearl_input_feature_redundancy_loadbearing` (2026-05-18) — the Option-2 input-feature fix this builds on top of.

The current controller's saturation pattern (gm67g fold-0 ISV trajectory in `/tmp/alpha_monitor/workflows/alpha-perception-gm67g.json`) is the empirical motivation for this redesign.
