fix(rl): surfer-scaffold v5.1 — decay re-engagement requires EXCESS alertedness
alpha-rl-zf6s5 (38a4aa15b b=1024 fold-1 step 5719 verdict, 2026-06-02):
the v5 scaffold_weight stayed pinned at 1.0 even when wr crossed
break-even (0.306 > 0.30). Root cause: `w_decay = min(1, 2 × frac_alerted)`
saturated at 1.0 because train-time Page-Hinkley naturally alerts on
75-95% of batches (per pearl_edge_decay_detector_phase1_validated_train_also_decays
discovered 2026-06-01); `max(w_competence, w_decay)` then kept the
scaffold engaged forever, defeating the fade mechanism.
FIX: `w_decay = max(0, 2 × frac_alerted − 1)` so re-engagement measures
EXCESS alertedness above 50% baseline, not absolute level:
frac_alerted v5 w_decay v5.1 w_decay
----------- ----------- ------------
0.50 1.0 0.00 ← v5 broken here
0.78 1.0 0.56
0.90 1.0 0.80
1.00 1.0 1.00
At zf6s5 step 5719 (frac_alerted=0.78), v5.1 gives w_decay=0.56 not 1.0;
w_competence=0.455 wins via max(), so w drops to ~0.56 letting the fade
begin properly.
Health signals from zf6s5 (informative even though terminated):
step 5719: wr=0.306, hold=14.4, entropy=1.82, realized=+$152M
trajectory: wr crossing 0.30 ✓, hold growing 12→14 ✓, entropy ↓ ✓
the agent IS becoming competent; the scaffold just couldn't fade.
Local invariant test:
reward_alignment_surfer_scaffold_invariants OK: 251 rows validated
train[0] bootstrap = 0.966 (unchanged — boot path identical)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -22,12 +22,16 @@
|
||||
// Math:
|
||||
// competence = confidence(n_trades) × sigmoid(k_sharp × max(0, wr_ema − break_even))
|
||||
// w_competence = 1 − competence
|
||||
// w_decay = min(1, 2 × frac_alerted)
|
||||
// w_decay = max(0, 2 × frac_alerted − 1) // FIX v5.1: only fires above 50%
|
||||
// w = clamp(max(w_competence, w_decay), 0, 1)
|
||||
//
|
||||
// At training start (n_trades=0, wr_ema=0): competence ≈ 0, w_competence ≈ 1 → w=1 (full scaffold)
|
||||
// After warmup and wr_ema crossing break-even: competence → 1, w → 0 (pure pnl)
|
||||
// If edge-decay PH alerts on >50% of batches: w_decay forces scaffold back up
|
||||
// If edge-decay PH alerts on >50% of batches: w_decay re-engages PROPORTIONAL to
|
||||
// excess above 50% baseline (alpha-rl-zf6s5 step 5719 verdict 2026-06-02 showed
|
||||
// train-time PH inherently alerts on 75-95% of batches, so the v5 original
|
||||
// `min(1, 2×frac)` pinned w=1 forever even when wr crossed break-even; the new
|
||||
// `max(0, 2×frac − 1)` only fires above 50% and reaches 1.0 only at 100% alert).
|
||||
//
|
||||
// Per `feedback_no_atomicadd`: single-thread launch (1×1×1), no atomic ops.
|
||||
// Per `feedback_cpu_is_read_only`: pure device-side.
|
||||
@@ -76,8 +80,12 @@ extern "C" __global__ void rl_surfer_scaffold_controller(float* isv) {
|
||||
// Inverse: scaffold weight from competence.
|
||||
const float w_competence = 1.0f - competence;
|
||||
|
||||
// Edge-decay re-engagement: if >50% of batches are PH-alerted, force w back up.
|
||||
const float w_decay = fminf(1.0f, 2.0f * frac_decay);
|
||||
// Edge-decay re-engagement: only fires ABOVE 50% baseline. Train-time PH
|
||||
// naturally alerts on a steady fraction of batches even during healthy
|
||||
// learning (pearl_edge_decay_detector_phase1_validated_train_also_decays);
|
||||
// re-engagement must measure EXCESS alertedness, not absolute level, or it
|
||||
// pins the scaffold at w=1 forever (alpha-rl-zf6s5 step 5719 failure mode).
|
||||
const float w_decay = fmaxf(0.0f, 2.0f * frac_decay - 1.0f);
|
||||
|
||||
const float w_raw = fmaxf(w_competence, w_decay);
|
||||
const float w = fmaxf(0.0f, fminf(1.0f, w_raw));
|
||||
|
||||
Reference in New Issue
Block a user