Closes defect #1 from the flawed Phase F+G arc: ISV[400..406] were
left at alloc_zeros sentinel 0 in production, causing
bellman_target_projection (γ=0), ppo_clipped_surrogate (ε=0, entropy=0),
and the C51 backward to train against degenerate targets that the
MockLobEnv toy fixture (done=true every step, horizon=1) intrinsically
could not detect.
Three changes:
1. Port crates/ml-alpha/cuda/rl_reward_scale_controller.cu from the
ml-alpha-phase-f-g-flawed reference branch (93 lines, unchanged).
Add to build.rs KERNELS list; bump cache-bust to v25.
2. Extend src/rl/isv_slots.rs: add 7 new EMA-input slot constants
(RL_MEAN_TRADE_DURATION_EMA_INDEX..RL_MEAN_ABS_PNL_EMA_INDEX),
RL_SLOTS_END goes 417 -> 424. These are reserved for the EMA
producer kernels Phase R3 lands; in R1 they stay at sentinel 0
(asserted by the G1 test).
3. Wire all 7 RL adaptive controllers (γ / τ / ε / entropy_coef /
n_rollout_steps / per_α / reward_scale) into IntegratedTrainer:
- 7 cubin includes + 7 module/function fields
- All 7 loaded in new() via the existing load_cubin pattern
- New fn launch_isv_controller_3arg() centralises the shared
(isv*, alpha, scalar_input) launch signature
- New fn with_controllers_bootstrapped() consumes self and fires
each controller once against the freshly-zeroed isv_d; each
kernel's first-observation-bootstrap path (per
pearl_first_observation_bootstrap) sees sentinel zero in its
slot and writes its canonical *_BOOTSTRAP value:
ISV[400] γ = 0.99
ISV[401] τ = 0.005
ISV[402] ε = 0.2
ISV[403] entropy_coef = 0.01
ISV[404] n_rollout_steps= 2048
ISV[405] per_α = 0.6
ISV[406] reward_scale = 1.0
- new() ends with `.with_controllers_bootstrapped()?` so every
trainer construction site picks this up automatically.
This replaces the flawed Phase F approach of host memcpy_htod-ing
canonical constants into ISV, which violated
feedback_no_htod_htoh_only_mapped_pinned (tests not exempt) AND
short-circuited the canonical pearl_first_observation_bootstrap
pattern every other adaptive controller in the codebase uses.
The launch_isv_controller_3arg helper is reused by Phase R5's
per-step controller launches with real EMA inputs sourced from
ISV[417..424] — at that point the Wiener-α blend kicks in and the
slots adapt away from the R1 bootstrap defaults.
Gate G1 (crates/ml-alpha/tests/isv_bootstrap.rs):
- Construct IntegratedTrainer
- memcpy_dtoh full ISV slice to host
- Assert ISV[400..406] equal each kernel's #define *_BOOTSTRAP
- Assert ISV[417..424] still at sentinel 0 (R3 wires producers)
Per feedback_no_cpu_test_fallbacks: the oracle is the kernel's own
*_BOOTSTRAP constant, not a CPU computation. Per
pearl_tests_must_prove_not_lock_observations: the test asserts an
invariant (bootstrap path wrote the canonical value defined by the
kernel), not a tuned magic number.
Build clean: cargo check + cargo build --test isv_bootstrap on
ml-alpha both green. CUDA-required, #[ignore]'d for non-GPU CI.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
0840fcfe64