Files
foxhunt/crates
jgrusewski b1ef6664ab fix(rl): reward_scale floor uses cumulative dones, not closed-trade-steps
Follow-up to the 2026-05-30 adaptive controller floor refactor (commit
083a88f7c). Spec Special case R wired the bootstrap-fraction floor to
release after `RL_TRADE_DUR_VAR_COUNT_INDEX < MIN_TRADES_FOR_RELEASE
(=100)`. That counter increments via the Welford trade-duration-EMA
producer ONCE PER STEP where any trade closed — not once per closed
trade. At b=1024 with typical ~7 dones/step, the gate released after
step ~100 (= ~700 actual closed trades, far short of the intended
"100 trades of confidence"). After release reward_scale crashed to
~0.001 within 200 more steps, identical to the pre-fix fold 0/1
behavior the spec was supposed to prevent.

Fix: track REAL cumulative closed trades by summing `dones[b]` per
step in the fused kernel's reward_scale block. New ISV slots:
  - RL_CUMULATIVE_DONES_INDEX (660) — running total
  - RL_MIN_TRADES_FOR_RELEASE_INDEX (661) — gate threshold,
    ISV-driven per `feedback_adaptive_not_tuned`, bootstrap 5000

At ~7 dones/step (typical) the floor now holds for ~715 steps before
release — enough for the reward magnitude EMA to stabilize against
genuine trade outcomes rather than early-training noise. Threshold is
ISV-resident so production runs can tune without recompile.

Verified locally (500-step b=128 smoke):
  step 50:  reward_scale = 0.384 (decaying toward floor)
  step 100: reward_scale = 0.140 (approaching floor)
  step 200: reward_scale = 0.100 (AT FLOOR, held)
  step 500: reward_scale = 0.100 (still at floor)

Without this fix (pre-commit): reward_scale crashed to 0.001 by step 500.

Tests: integrated_trainer_smoke + signal_variance_kernel +
controller_adaptive_floors all pass. Release build clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-30 16:38:17 +02:00
..