Replaces Phase E.1's placeholder host-side loss scalars with actual GPU
kernel calls. Each of Q, π, V heads now runs:
- Forward kernel on h_t → head outputs (q_logits / pi_logits / v_pred)
- Backward kernel chain → grad_w / grad_b / grad_h_t (per-batch scratch
reduced by reduce_axis0)
- Adam step using the existing project-wide adamw_step cubin via the
AdamW wrapper (each head owns 2 instances — one for w, one for b)
New CUDA kernels:
- v_head_fwd_bwd.cu: scalar V head forward + MSE backward. Per-batch
loss scratch + per-batch grad_w / grad_b scratch — caller reduces
via reduce_axis0. Per-(batch, c) grad_h_t writes are sole-writer so
no atomicAdd needed.
- grad_h_accumulate.cu: element-wise grad_h_encoder += λ × grad_h_head
combiner. Loaded and ready; consumed by Phase E.3 once the encoder
backward entry point lands.
Extends existing kernels (additive, doesn't break prior callers):
- dqn_distributional_q.cu: adds `dqn_grad_w_b_h_t` that maps the C51
bwd kernel's grad_logits output into per-batch grad_w / grad_b
scratch + OVERWRITE grad_h_t. Mirrors aux_heads_bwd's thread-role
pattern (one thread per HIDDEN_DIM channel, sole writer per slot).
- ppo_clipped_surrogate.cu: adds `ppo_policy_logits_fwd` (standalone
linear-projection forward producing pi_logits for the surrogate
kernel to consume) AND `ppo_grad_w_b_h_t` (same backward pattern as
the DQN extension, with output dim N_ACTIONS=9).
All new kernels honour feedback_no_atomicadd: per-batch grad scratch
reduced by the existing reduce_axis0 path, never atomics. The Phase C/D
loss-scalar atomicAdds (dqn / ppo bwd accumulators) stay as their
loud-flagged deferral.
Per-head Adam state:
- DqnHead: 2 AdamW instances (w_d, b_d). Re-uses the existing
adamw_step cubin from `trainer::optim::AdamW`.
- PolicyHead: same.
- ValueHead: same.
- Each AdamW owns independent m / v buffers and a device-resident
step counter (per pearl_no_host_branches_in_captured_graph: counter
advancement is a captured kernel, not host scalar).
PerceptionTrainer: adds `h_t_view()` accessor — borrows the h_t_d
field that `forward_encoder` populates. IntegratedTrainer uses this
to dispatch Q/π/V head kernels on the same encoder representation
without re-borrowing self.perception mutably between the encoder
forward and the head dispatches (disjoint field borrows).
ENCODER BACKWARD: Phase E.2 DEFERS the encoder-side gradient combine
to Phase E.3 (where the LobSim integration provides a clean entry
point for a new PerceptionTrainer backward-encoder method). The Q/π/V
kernels DO emit grad_h_t but it is not yet wired into the encoder
backward path. The encoder still learns from BCE+aux only in E.2;
the new heads update their own weights via per-head Adam. E.3 lands
the missing piece via `launch_grad_h_accumulate` (wired and ready).
Bellman target distribution: Phase E.2 uses a deterministic
single-atom-mass projection (host-side, mapped-pinned upload). Phase
E.3 replaces with the proper categorical projection kernel that
consumes γ from ISV[400] and the target net's bootstrap atom values.
build.rs: registers `v_head_fwd_bwd` + `grad_h_accumulate` cubins.
Cache-bust v21 → v22 with a verbose changelog entry covering this
phase's contract changes.
The integrated_trainer_smoke #[ignore]-gated GPU test remains gated;
Phase E.3 activates it alongside LobSim. Non-GPU host tests
(loss_balance default + bootstrap) continue to pass.
Companion to Phases A-E.1 (commits 6a46ded7d9ec43fdb956efd96cb9732a667c729f110e0).
2665669b58