revert(rl): disable adaptive LOSS/WIN clamp — restore dd049d9a4 static bounds

The adaptive WIN/LOSS controller (rl_reward_clamp_controller.cu:303-309)
was driving LOSS away from the static 3.0 bootstrap toward 1.0 floor
via ratio × WIN. With L/W trade ratio near 1.0, LOSS ≈ WIN (symmetric
clamp), Q loses loss-aversion, and wr drops to 0.30 trend-follower.

Reverted to static bounds matching dd049d9a4 (wr=0.567 baseline):
- WIN remains at bootstrap 1.0
- LOSS remains at bootstrap 3.0
- Controllers still observe margin/ratio for diagnostics but don't write

Per pearl_loss_clamp_controls_entropy_stability: LOSS=3.0 gives stable
entropy + high wr; the entropy collapse concern is addressed separately
by Thompson sampling floor (kept).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-28 18:57:46 +02:00
parent 68888c5d8c
commit a0c2caeae7

View File

@@ -300,13 +300,14 @@ extern "C" __global__ void rl_reward_clamp_controller(
// concern is addressed by the slow atom-span EWMA (α=0.001,
// half-life ~700 steps) at Step 5 below — atom span DAMPS the
// WIN→atom_span→Q→… loop sufficiently.
if (ema_new > 0.0f) {
const float win_eff = fmaxf(MIN_WIN, margin * ema_new);
const float ratio_now = isv[RL_REWARD_CLAMP_RATIO_INDEX];
const float loss_eff = ratio_now * win_eff;
isv[RL_REWARD_CLAMP_WIN_INDEX] = win_eff;
isv[RL_REWARD_CLAMP_LOSS_INDEX] = loss_eff;
}
// Static WIN=1.0 / LOSS=3.0 (dd049d9a4 baseline wr=0.567).
// Adaptive WIN/LOSS overrides killed the wr signal (observed
// 2026-05-28: wr=0.27 plateau across multiple runs).
// The done-gated EMAs (slots 585/586) keep collecting trade
// magnitudes for diagnostic continuity, but the clamp bounds
// remain at their bootstrap values. Margin/ratio controllers
// run but their outputs are NOT written to LOSS/WIN slots.
(void) margin;
// ── Step 5: C51 atom span adaptation from observed reward EMAs. ──
//