Closes defects #3 (controllers never launched) and #4 (target net
never soft-updated) from the flawed Phase F+G arc.
CONTROLLER SIGNATURE CHANGE (all 7 .cu files):
Scalar input arg → int input_slot. Each controller now reads its EMA
input from ISV[input_slot] directly inside the kernel, eliminating
the 7 DtoH-per-step host roundtrips a scalar-arg signature would have
required — per feedback_cpu_is_read_only (hot-path must be GPU-pure).
Bootstrap path unchanged: kernel reads its output slot, sees sentinel
zero, writes *_BOOTSTRAP, and returns BEFORE the input_slot read. So
R1's launch_isv_controller_3arg(controller_fn, alpha=0.4, input_slot)
works both at bootstrap (input read deferred via early return) and
at per-step (input slot has real EMA observation from R3 producers).
PER-STEP CONTROLLER LAUNCHER:
New IntegratedTrainer::launch_rl_controllers_per_step() fires all 7
controllers in sequence, each with its dedicated EMA input slot:
ISV[400] γ ← ISV[417] MEAN_TRADE_DURATION_EMA
ISV[401] τ ← ISV[418] Q_DIVERGENCE_EMA
ISV[402] ε ← ISV[419] KL_PI_EMA
ISV[403] entropy_coef ← ISV[420] ENTROPY_OBSERVED_EMA
ISV[404] n_rollout_steps← ISV[421] ADVANTAGE_VAR_RATIO_EMA
ISV[405] per_α ← ISV[422] TD_KURTOSIS_EMA
ISV[406] reward_scale ← ISV[423] MEAN_ABS_PNL_EMA
R1's with_controllers_bootstrapped also updated to pass the input
slot indices (the bootstrap path still ignores them via early return).
TARGET-NET SOFT UPDATE (defect #4):
New cuda/dqn_target_soft_update.cu — element-wise
target[i] = (1-τ)·target[i] + τ·current[i]
reading τ from ISV[401]. Trivially parallel, no atomicAdd. DqnHead
gains target_soft_update_fn + _target_soft_update_module fields +
soft_update_target(&isv_d) method that fires the kernel twice
(weights + biases). R6 calls this from step_with_lobsim after the
Q-head Adam update.
GATE TESTS (tests/r5_controllers_and_soft_update.rs):
G3: g3_per_step_controllers_move_isv_outputs_when_fed_real_emas
- Verifies R1 bootstrap pre-conditions (all 7 output slots at
documented bootstrap values; all 7 EMA-input slots at sentinel 0).
- Populates each EMA-input slot with a distinct non-zero value via
R3's ema_update_per_step bootstrap path (different values per slot
so a wrong-slot wiring bug would produce out-of-range outputs).
- Verifies the EMA producers wrote what we expected (sanity).
- Fires launch_rl_controllers_per_step.
- Asserts each output slot moved off its bootstrap value (catches
"controller doesn't fire" / "reads wrong slot" / "dead kernel").
G4: g4_dqn_target_soft_update_implements_polyak_formula
- Force-overwrite w_d with all-ones (breaks the w==target init
symmetry so soft_update has something to blend).
- Snapshot w_target (Xavier init values).
- Fire dqn_head.soft_update_target with R1-bootstrapped τ=0.005.
- For sample indices: assert target_after[i] equals
(1-τ)·target_before[i] + τ·1.0 within 1e-6 (exact algebraic
identity, not a CPU reference — kernel IS the kernel).
- Negative invariant: at least one element changed.
Per feedback_no_cpu_test_fallbacks: G3 oracle is the invariant
"output != bootstrap after non-trivial input"; G4 oracle is the
algebraic identity (1-τ)·a + τ·b applied to the SAME numbers the
kernel saw — not a parallel CPU implementation.
Build cache-bust v28. cargo check + cargo build --tests on ml-alpha
green for all R-phase tests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
0a32a3bb89