feat(rl): IQN loss + backward + ensemble action selection
Completes the IQN integration: - rl_iqn_backward.cu: backprop through quantile embedding + output projection to produce per-batch weight gradients for all 4 weight tensors. Block per batch, HIDDEN_DIM threads. - rl_iqn_loss.cu (already existed): quantile Huber loss feeds grad_online_q into the backward kernel. - rl_ensemble_action_value.cu: E_ensemble = α×C51 + (1-α)×IQN, ISV-driven α at slot 544. - Adam updates for IQN weights after backward. - Ensemble Q feeds rl_q_pi_agree_b diagnostic. Full stack smoke: 200 steps, l_q=2.43, no crash. Both C51 and IQN learning simultaneously from the same replay transitions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -93,6 +93,8 @@ const KERNELS: &[&str] = &[
|
||||
"rl_session_risk_check", // session-level loss limit circuit breaker
|
||||
"rl_iqn_forward", // IQN distributional Q-head: quantile embedding + action-value projection; complementary to C51
|
||||
"rl_iqn_loss", // IQN quantile Huber loss: ρ_τ(δ) = |τ - 1(δ<0)| × Huber(δ, κ=1.0); forward + backward
|
||||
"rl_iqn_backward", // IQN backward through forward pass: grad_output → grad_w_out/b_out/w_embed/b_embed per-batch scratch
|
||||
"rl_ensemble_action_value", // C51+IQN ensemble: E_ensemble = α×E_C51 + (1-α)×E_IQN; α from ISV[544]
|
||||
];
|
||||
|
||||
// Cache bust v31 — five new reduce / derive kernels populate the input
|
||||
|
||||
Reference in New Issue
Block a user