fix(rl): surfer-scaffold v5.1 — decay re-engagement requires EXCESS alertedness

alpha-rl-zf6s5 (38a4aa15b b=1024 fold-1 step 5719 verdict, 2026-06-02):
the v5 scaffold_weight stayed pinned at 1.0 even when wr crossed
break-even (0.306 > 0.30). Root cause: `w_decay = min(1, 2 × frac_alerted)`
saturated at 1.0 because train-time Page-Hinkley naturally alerts on
75-95% of batches (per pearl_edge_decay_detector_phase1_validated_train_also_decays
discovered 2026-06-01); `max(w_competence, w_decay)` then kept the
scaffold engaged forever, defeating the fade mechanism.

FIX: `w_decay = max(0, 2 × frac_alerted − 1)` so re-engagement measures
EXCESS alertedness above 50% baseline, not absolute level:

  frac_alerted   v5 w_decay    v5.1 w_decay
  -----------   -----------   ------------
  0.50          1.0           0.00      ← v5 broken here
  0.78          1.0           0.56
  0.90          1.0           0.80
  1.00          1.0           1.00

At zf6s5 step 5719 (frac_alerted=0.78), v5.1 gives w_decay=0.56 not 1.0;
w_competence=0.455 wins via max(), so w drops to ~0.56 letting the fade
begin properly.

Health signals from zf6s5 (informative even though terminated):
  step 5719: wr=0.306, hold=14.4, entropy=1.82, realized=+$152M
  trajectory: wr crossing 0.30 ✓, hold growing 12→14 ✓, entropy ↓ ✓
  the agent IS becoming competent; the scaffold just couldn't fade.

Local invariant test:
  reward_alignment_surfer_scaffold_invariants OK: 251 rows validated
  train[0] bootstrap = 0.966 (unchanged — boot path identical)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-06-02 00:22:23 +02:00
parent 38a4aa15b3
commit 4a4953b01f

View File

@@ -22,12 +22,16 @@
// Math:
// competence = confidence(n_trades) × sigmoid(k_sharp × max(0, wr_ema break_even))
// w_competence = 1 competence
// w_decay = min(1, 2 × frac_alerted)
// w_decay = max(0, 2 × frac_alerted 1) // FIX v5.1: only fires above 50%
// w = clamp(max(w_competence, w_decay), 0, 1)
//
// At training start (n_trades=0, wr_ema=0): competence ≈ 0, w_competence ≈ 1 → w=1 (full scaffold)
// After warmup and wr_ema crossing break-even: competence → 1, w → 0 (pure pnl)
// If edge-decay PH alerts on >50% of batches: w_decay forces scaffold back up
// If edge-decay PH alerts on >50% of batches: w_decay re-engages PROPORTIONAL to
// excess above 50% baseline (alpha-rl-zf6s5 step 5719 verdict 2026-06-02 showed
// train-time PH inherently alerts on 75-95% of batches, so the v5 original
// `min(1, 2×frac)` pinned w=1 forever even when wr crossed break-even; the new
// `max(0, 2×frac 1)` only fires above 50% and reaches 1.0 only at 100% alert).
//
// Per `feedback_no_atomicadd`: single-thread launch (1×1×1), no atomic ops.
// Per `feedback_cpu_is_read_only`: pure device-side.
@@ -76,8 +80,12 @@ extern "C" __global__ void rl_surfer_scaffold_controller(float* isv) {
// Inverse: scaffold weight from competence.
const float w_competence = 1.0f - competence;
// Edge-decay re-engagement: if >50% of batches are PH-alerted, force w back up.
const float w_decay = fminf(1.0f, 2.0f * frac_decay);
// Edge-decay re-engagement: only fires ABOVE 50% baseline. Train-time PH
// naturally alerts on a steady fraction of batches even during healthy
// learning (pearl_edge_decay_detector_phase1_validated_train_also_decays);
// re-engagement must measure EXCESS alertedness, not absolute level, or it
// pins the scaffold at w=1 forever (alpha-rl-zf6s5 step 5719 failure mode).
const float w_decay = fmaxf(0.0f, 2.0f * frac_decay - 1.0f);
const float w_raw = fmaxf(w_competence, w_decay);
const float w = fmaxf(0.0f, fminf(1.0f, w_raw));