jgrusewski 9c6c280bd8 fix(rl): anti-collapse probability floor + argo b_size=16 default ...

Two fixes for alpha-rl-9k9x6 (commit 3737feb66) findings:

(1) rl_pi_action_kernel.cu — per-action probability floor P_MIN=0.02
    with renorm after softmax. Structural guarantee H(π_sample)
    bounded below regardless of logits. 9k9x6 evidence:
      - π collapsed to action 1 (100% sample rate) by step 2500
      - action_entropy EMA → 0.001 by step 10000 (uniform=2.197)
      - entropy_coef controller pegged at COEF_MAX without effect
        because PPO update couldn't escape δ-function fixed point
      - 0 trades closed in 50000 steps; position monotonically
        accumulated to -45871 lots
    With P_MIN=0.02, worst-case H(π_sample) ≈ 0.77 nats — well
    above the observed collapse and well below uniform.
    Per `pearl_blend_formulas_must_have_permanent_floor`.

(2) argo template + script — N_BACKTESTS default 1 → 16. The CLI
    default lift in commit 3737feb66 was overridden by both the
    argo script and the wftmpl `value: "1"`. 9k9x6 actually ran
    b_size=1 despite being framed as the b_size=16 test.
    Per `pearl_b_size_1_signal_starvation_blocks_q_learning`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-24 11:26:37 +02:00

9.7 KiB

Executable File

Raw Blame History

View Raw