feat(sp17-3.2): V_share + advantage_clip_bound producers + extended emit
Phase 3.2 lands the remaining two SP17 dueling-Q diagnostic producers
atomically with kernel + launcher + Rust wrapper + extended HEALTH_DIAG
emit + GPU oracle tests per `feedback_wire_everything_up`.
V_share[d] = |E[V]| / (|E[V]| + |E[A_centered, picked]|)
where picked = argmax_a Σ_z A_raw[i, a, z] (max-Q semantic — tractable
per-batch without depending on actions_history_buf which is collector-
time state stale relative to the cuBLAS forward at HEALTH_DIAG cadence).
Pearl-A bootstrap (sentinel 0.5) + α=WELFORD_ALPHA_MIN=0.4 + bilateral
[0, 1] clamp per `pearl_symmetric_clamp_audit`. 4 blocks × 256 threads.
advantage_clip_bound = p99(|A_centered|) × ADVANTAGE_CLIP_SAFETY_FACTOR=1.5
via sp4_histogram_p99 (block tree-reduce + per-warp tile binning, NO
atomicAdd per `pearl_fused_per_group_statistics_oracle`). EMA α=0.01
slow per-fold + bilateral clamp [0.1, 100.0] per
`pearl_symmetric_clamp_audit`. Pearl-A bootstrap (sentinel 1.0).
Single block × 256 threads + flat |A_centered| scratch buffer
(mapped-pinned, sized to B × Σ_d b_d × NA).
Observability-only — the actual clipping wire-up is Phase 5 follow-up.
The Phase 1 mean-zero contract (commits eabcf8d52..6f53d676f) makes
A_centered a meaningful signal; this commit observes it.
Extended HEALTH_DIAG line:
HEALTH_DIAG[N]: dueling [v_share=(d=X m=Y o=Z u=W)]
[a_var=(d=A m=B o=C u=D)] [clip=K]
GPU oracle tests on RTX 3050 Ti (all pass, 13/13 SP17 tests):
- v_share_per_branch_matches_closed_form: synthetic V=2.0 + linear A
per branch; closed-form V_share = 2/(2 + |K_d × (n_d-1)/2|);
ε=1e-4. Pearl-A bootstrap REPLACES on first launch.
- advantage_clip_bound_tracks_p99_safety: synthetic A with action-
dominant + per-(i,z) jitter (the jitter is REQUIRED — pathologically
lockstep values undercount in sp4_histogram_p99's non-atomic warp
tile binning per the kernel's documented "1/(256×32) loss for
uniformly distributed signals" qualifier; concentrated values violate
the assumption. Real |A_centered| in production is continuous, so
this is a test-data-only effect.) ε=0.20 (jitter + linear histogram
quantization).
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Phase 3.2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -941,6 +941,38 @@ fn main() {
|
||||
// `read_a_var_ema_per_branch()` in gpu_dqn_trainer.rs. See
|
||||
// docs/dqn-wire-up-audit.md § "SP17 Phase 3.1" for rationale.
|
||||
"sp17_a_var_ema_kernel.cu",
|
||||
// SP17 Phase 3.2 (2026-05-08): per-branch V_share EMA producer.
|
||||
// 4 blocks × 256 threads. Reads `on_v_logits_buf [B, NA]` and
|
||||
// BRANCH-MAJOR `on_b_logits_buf`; computes per-branch
|
||||
// `|E[V]| / (|E[V]| + |E[A_centered, picked]|)` where picked is
|
||||
// argmax_a Σ_z A_raw[i, a, z] (max-Q action — tractable
|
||||
// per-batch without depending on actions_history_buf). Writes
|
||||
// to ISV[V_SHARE_DIR/MAG/ORD/URG_INDEX] (slots 478..482) with
|
||||
// Pearl-A bootstrap (sentinel 0.5 = healthy 50/50 cold-start) +
|
||||
// α=WELFORD_ALPHA_MIN=0.4 EMA blend. Bilateral clamp [0, 1]
|
||||
// before write per `pearl_symmetric_clamp_audit`. Block
|
||||
// tree-reduce (no atomicAdd) per `feedback_no_atomicadd`.
|
||||
// Diagnostic observability — does NOT modify any consumer path.
|
||||
// Consumed by HEALTH_DIAG `dueling [v_share=...]` via
|
||||
// `read_v_share_per_branch()`. See
|
||||
// docs/dqn-wire-up-audit.md § "SP17 Phase 3.2" for rationale.
|
||||
"sp17_v_share_kernel.cu",
|
||||
// SP17 Phase 3.2 (2026-05-08): adaptive advantage_clip_bound
|
||||
// producer. Single block × 256 threads. Reads BRANCH-MAJOR
|
||||
// `on_b_logits_buf`; for each branch d, computes per-atom mean
|
||||
// of A across actions and writes |A_raw - mean[z]| into a
|
||||
// pre-allocated flat scratch buffer of size Σ_d B×b_d×NA;
|
||||
// calls `sp4_histogram_p99` over the full flat scratch (block
|
||||
// tree-reduce + per-warp tile binning, no atomicAdd per
|
||||
// `pearl_fused_per_group_statistics_oracle` and
|
||||
// `feedback_no_atomicadd`); blends `bound = p99 × 1.5` into
|
||||
// ISV[ADVANTAGE_CLIP_BOUND_INDEX=482] with Pearl-A bootstrap
|
||||
// (sentinel 1.0 = no effective clipping cold-start) + α=0.01
|
||||
// slow per-fold cadence + bilateral clamp [0.1, 100.0] per
|
||||
// `pearl_symmetric_clamp_audit`. Observability-only in this
|
||||
// commit — the actual clip wire-up is Phase 5 follow-up. See
|
||||
// docs/dqn-wire-up-audit.md § "SP17 Phase 3.2" for rationale.
|
||||
"sp17_advantage_clip_bound_kernel.cu",
|
||||
// SP14 Layer C Phase C.1 (2026-05-08): cubin entries for
|
||||
// alpha_grad_compute_kernel.cu, gradient_hack_detect_kernel.cu,
|
||||
// and sp14_scale_wire_col_kernel.cu deleted atomically with
|
||||
|
||||
Reference in New Issue
Block a user