fix(rl): anti-collapse probability floor + argo b_size=16 default
Two fixes for alpha-rl-9k9x6 (commit3737feb66) findings: (1) rl_pi_action_kernel.cu — per-action probability floor P_MIN=0.02 with renorm after softmax. Structural guarantee H(π_sample) bounded below regardless of logits. 9k9x6 evidence: - π collapsed to action 1 (100% sample rate) by step 2500 - action_entropy EMA → 0.001 by step 10000 (uniform=2.197) - entropy_coef controller pegged at COEF_MAX without effect because PPO update couldn't escape δ-function fixed point - 0 trades closed in 50000 steps; position monotonically accumulated to -45871 lots With P_MIN=0.02, worst-case H(π_sample) ≈ 0.77 nats — well above the observed collapse and well below uniform. Per `pearl_blend_formulas_must_have_permanent_floor`. (2) argo template + script — N_BACKTESTS default 1 → 16. The CLI default lift in commit3737feb66was overridden by both the argo script and the wftmpl `value: "1"`. 9k9x6 actually ran b_size=1 despite being framed as the b_size=16 test. Per `pearl_b_size_1_signal_starvation_blocks_q_learning`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -32,7 +32,11 @@ BRANCH=$(git symbolic-ref --short HEAD 2>/dev/null || echo "ml-alpha-phase-a")
|
||||
GPU_POOL=ci-training-l40s
|
||||
N_STEPS=1000
|
||||
SEQ_LEN=32
|
||||
N_BACKTESTS=1
|
||||
# Default 16 — fixes b_size=1 signal starvation per
|
||||
# `pearl_b_size_1_signal_starvation_blocks_q_learning`. 16 parallel
|
||||
# transitions per Adam step give Q proper gradient variance reduction.
|
||||
# Override with `--n-backtests N` for sweeps.
|
||||
N_BACKTESTS=16
|
||||
PER_CAPACITY=4096
|
||||
SEED=16962
|
||||
INSTRUMENT_MODE=front-month
|
||||
|
||||
Reference in New Issue
Block a user