From c1dc84a34583a8cc7124f2e21ed95331b48bb7f1 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Mon, 1 Jun 2026 12:25:47 +0200 Subject: [PATCH] fix(rl): B-10 wire G1 Q-distribution launch into step_with_lobsim_gpu MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Initial B-10 commit placed `launch_q_distribution_stats` after the `dqn_head.forward(h_t)` at line 6615 — but that's inside `step_with_lobsim`, the legacy non-GPU path. The actual GPU path exercised by `alpha_rl_train` is `step_with_lobsim_gpu` (line 8402), which has its own h_t-based Q forward at line 8632. The G1 diag fields (q_dist_entropy_mean, q_value_range_mean, q_value_abs_max) returned 0 locally because the launch never ran in the GPU path. Added the same launch_q_distribution_stats call after line 8632. The non-GPU launch at 6615 is left in place — `step_with_lobsim` is still callable; per `feedback_no_partial_refactor.md` both paths are instrumented consistently. Validated locally (RTX 3050 Ti, 200+50 b=16 fold-1): q_dist_entropy_mean = 2.6482 (was 0) q_value_range_mean = 4.9257 (was 0) q_value_abs_max = 3.8715 (was 0) The entropy value 2.65 is consistent with a near-uniform softmax over Q_N_ATOMS=21 atoms with some learned concentration (ln(21)=3.04 = full uniform, 0 = one-hot delta). Range and abs_max in 4-5 unit scale match the locally-adapted atom support. Co-Authored-By: Claude Opus 4.7 --- crates/ml-alpha/src/trainer/integrated.rs | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/crates/ml-alpha/src/trainer/integrated.rs b/crates/ml-alpha/src/trainer/integrated.rs index 15a3341e2..31ea1bac3 100644 --- a/crates/ml-alpha/src/trainer/integrated.rs +++ b/crates/ml-alpha/src/trainer/integrated.rs @@ -8631,6 +8631,24 @@ impl IntegratedTrainer { self.dqn_head .forward(h_t_borrow, b_size, &mut self.q_logits_d) .context("step_with_lobsim_gpu: dqn_head.forward(h_t)")?; + // B-10 G1 (2026-06-01): Q-distribution informativeness diagnostic + // on online Q logits — emits ISV slots 730/731/732. Pure + // observability; reads q_logits_d immediately after the forward, + // before any downstream consumer (distill / Thompson / PPO). + // (Sister-launch added at line 6622 in `step_with_lobsim` — the + // GPU path here is the canonical one that the alpha_rl_train + // example actually exercises.) + self.dqn_head + .launch_q_distribution_stats( + &self.isv_dev_ptr, + &self.q_logits_d, + &self.atom_supports_d, + b_size, + &mut self.q_dist_entropy_pb_d, + &mut self.q_dist_range_pb_d, + &mut self.q_dist_abs_max_pb_d, + ) + .context("step_with_lobsim_gpu: dqn_head.launch_q_distribution_stats")?; self.dqn_head .forward_target(h_t_borrow, b_size, &mut self.q_logits_target_st_d) .context("step_with_lobsim_gpu: dqn_head.forward_target(h_t) for distillation")?;