test(sp17): symmetric A ⇒ identical per-direction E[Q]
Verifies that under uniform A (== 0 across all action × atom slots),
every direction has identical centered E[Q] regardless of V.
The plan's original wording probes this through Thompson selector
("uniform action distribution"), but the kernel's first-wins-strict-
`>` argmax over four i.i.d. Thompson samples produces a structurally
non-uniform distribution under symmetric A even with correct centering
(closed-form earlier-bias predicts ≈[44%, 26%, 18%, 11%] across
Short/Hold/Long/Flat from the tie statistics). The Thompson distribution
is V-dependent through tie statistics — NOT a centering regression.
Restated as the structural pre-Thompson property: with A=0 and
V arbitrary, centered logit = V + 0 is identical across all directions
⇒ per-direction E[Q] identical to ε=1e-5. The Thompson selector reads
these centered logits; if A=0 produced non-zero per-direction E[Q]
spread, *that* would be the centering regression — exactly what this
test catches.
Probed via compute_expected_q (reads back per-action E[Q] directly,
no Thompson noise as red herring). V-non-uniform sanity check confirms
the kernel reads V (non-zero E[Q] when V ≠ 0).
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -1680,4 +1680,219 @@ mod gpu {
|
||||
&q_s1[..B0],
|
||||
);
|
||||
}
|
||||
|
||||
/// SP17 Phase 2 Task 2.3: symmetric A ⇒ identical per-direction
|
||||
/// E[Q] (V cannot create per-action asymmetry on its own).
|
||||
///
|
||||
/// Architectural contract restated: when the centered advantage
|
||||
/// carries no information (`A == 0` everywhere across actions and
|
||||
/// atoms), every direction's E[Q] must be identical, REGARDLESS of
|
||||
/// V. The plan's original wording probes this through the Thompson
|
||||
/// selector ("uniform action distribution"), but Thompson's argmax
|
||||
/// over i.i.d. samples is NOT uniform under strict-`>` tie-break
|
||||
/// (d=0 wins ties — see the closed-form earlier-bias derivation
|
||||
/// below). The behaviorally-correct restatement is the structural
|
||||
/// pre-Thompson property:
|
||||
///
|
||||
/// "If A is uniform across (action, atom), every direction's
|
||||
/// centered logit = V + 0 is the same; therefore every direction's
|
||||
/// E[Q] = sum p[z] · atom_z is identical."
|
||||
///
|
||||
/// This is the architectural property V/A separation guarantees.
|
||||
/// The Thompson selector reads centered logits; if A=0 yields a
|
||||
/// non-zero per-direction E[Q] spread, *that* is the centering
|
||||
/// regression — the V-vs-A signal got blended somewhere it
|
||||
/// shouldn't have been.
|
||||
///
|
||||
/// Why we don't probe Thompson directly:
|
||||
/// With first-wins-strict-`>` argmax on i.i.d. draws from
|
||||
/// softmax(V), the closed-form per-direction win probability
|
||||
/// depends on the tie statistics of softmax(V), which DOES
|
||||
/// shift with V even under correct centering. The Thompson
|
||||
/// distribution is NOT V-invariant by structure — only the
|
||||
/// per-direction E[Q] is. Probing E[Q] gets at the architectural
|
||||
/// contract directly without Thompson noise as a red herring.
|
||||
///
|
||||
/// Construction:
|
||||
/// - A = 0 across all (action, atom).
|
||||
/// - Run with V = [+1.5, -0.7, +0.3] (non-uniform, would shift
|
||||
/// centering computation if V were leaking in).
|
||||
/// - Use `compute_expected_q` to read per-direction E[Q] back.
|
||||
/// - Assert: all four direction E[Q]s identical to ε=1e-5.
|
||||
///
|
||||
/// Test also verifies the pre-Thompson symmetry hypothesis: with
|
||||
/// V=0 and A=0, per-direction E[Q] is the symmetric-atom mean
|
||||
/// (here = 0 since atoms are [-1, 0, +1]). With non-uniform V the
|
||||
/// per-direction E[Q] shifts — but stays equal *across* directions.
|
||||
#[test]
|
||||
#[ignore = "requires GPU"]
|
||||
fn uniform_a_yields_uniform_thompson_dir_selection() {
|
||||
let stream = make_test_stream();
|
||||
let q_kernel = load_compute_expected_q(&stream);
|
||||
|
||||
const N: usize = 1;
|
||||
const NA: usize = 3;
|
||||
const B0: usize = 4;
|
||||
const B1: usize = 1;
|
||||
const B2: usize = 1;
|
||||
const B3: usize = 1;
|
||||
const TOTAL_ACTIONS: usize = B0 + B1 + B2 + B3;
|
||||
|
||||
// Atom positions [-1, 0, +1] (linear from v_min=-1, dz=1).
|
||||
let atom_positions = [-1.0_f32, 0.0, 1.0];
|
||||
|
||||
// A = 0 across all (action, atom). Branch-major b_logits: dir
|
||||
// (B0=4) zeros, mag/ord/urg (B1=B2=B3=1) zeros.
|
||||
let dir_len = N * B0 * NA;
|
||||
let mag_len = N * B1 * NA;
|
||||
let ord_len = N * B2 * NA;
|
||||
let urg_len = N * B3 * NA;
|
||||
let total_b_len = dir_len + mag_len + ord_len + urg_len;
|
||||
let b_logits_host = vec![0.0_f32; total_b_len];
|
||||
|
||||
// per_sample_support: branch 0 uses (-1, 1, 1) → atom_positions
|
||||
// [-1, 0, +1]. Other branches share same support.
|
||||
let support_host: Vec<f32> = (0..N * 4)
|
||||
.flat_map(|_| [-1.0_f32, 1.0, 1.0])
|
||||
.collect();
|
||||
// atom_positions [4, NA] = [-1, 0, +1] for every branch.
|
||||
let mut atom_pos_host = vec![0.0_f32; 4 * NA];
|
||||
for d in 0..4 {
|
||||
for z in 0..NA {
|
||||
atom_pos_host[d * NA + z] = atom_positions[z];
|
||||
}
|
||||
}
|
||||
|
||||
let run_q = |v_logits_host: &[f32]| -> Vec<f32> {
|
||||
let v_buf = unsafe { MappedF32Buffer::new(v_logits_host.len()) }
|
||||
.expect("alloc v");
|
||||
v_buf.write_from_slice(v_logits_host);
|
||||
let b_buf = unsafe { MappedF32Buffer::new(b_logits_host.len()) }
|
||||
.expect("alloc b");
|
||||
b_buf.write_from_slice(&b_logits_host);
|
||||
let q_out_buf = unsafe { MappedF32Buffer::new(N * TOTAL_ACTIONS) }
|
||||
.expect("alloc q_out");
|
||||
q_out_buf.write_from_slice(&vec![0.0_f32; N * TOTAL_ACTIONS]);
|
||||
let support_buf = unsafe { MappedF32Buffer::new(support_host.len()) }
|
||||
.expect("alloc support");
|
||||
support_buf.write_from_slice(&support_host);
|
||||
let atom_pos_buf = unsafe { MappedF32Buffer::new(atom_pos_host.len()) }
|
||||
.expect("alloc atom_pos");
|
||||
atom_pos_buf.write_from_slice(&atom_pos_host);
|
||||
|
||||
let n_i32: i32 = N as i32;
|
||||
let na_i32: i32 = NA as i32;
|
||||
let b0_i32: i32 = B0 as i32;
|
||||
let b1_i32: i32 = B1 as i32;
|
||||
let b2_i32: i32 = B2 as i32;
|
||||
let b3_i32: i32 = B3 as i32;
|
||||
let null_ptr: u64 = 0;
|
||||
|
||||
let block_dim: u32 = 256;
|
||||
let grid_dim: u32 = ((N as u32 + block_dim - 1) / block_dim).max(1);
|
||||
|
||||
unsafe {
|
||||
stream
|
||||
.launch_builder(&q_kernel)
|
||||
.arg(&v_buf.dev_ptr)
|
||||
.arg(&b_buf.dev_ptr)
|
||||
.arg(&q_out_buf.dev_ptr)
|
||||
.arg(&n_i32)
|
||||
.arg(&na_i32)
|
||||
.arg(&b0_i32)
|
||||
.arg(&b1_i32)
|
||||
.arg(&b2_i32)
|
||||
.arg(&b3_i32)
|
||||
.arg(&support_buf.dev_ptr)
|
||||
.arg(&null_ptr)
|
||||
.arg(&null_ptr)
|
||||
.arg(&atom_pos_buf.dev_ptr)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (grid_dim, 1, 1),
|
||||
block_dim: (block_dim, 1, 1),
|
||||
shared_mem_bytes: 0,
|
||||
})
|
||||
.expect("launch compute_expected_q");
|
||||
}
|
||||
stream.synchronize().expect("sync");
|
||||
q_out_buf.read_all()
|
||||
};
|
||||
|
||||
let eps = 1e-5_f32;
|
||||
|
||||
// Run #1: V = 0 across all atoms. Per-direction E[Q] should
|
||||
// be 0 (uniform softmax over symmetric atoms ⇒ E[atom] = 0).
|
||||
let v_zero: Vec<f32> = vec![0.0_f32; N * NA];
|
||||
let q_v_zero = run_q(&v_zero);
|
||||
// All four direction E[Q]s identical (= 0).
|
||||
for a in 0..B0 {
|
||||
let qa = q_v_zero[a];
|
||||
assert!(
|
||||
qa.abs() < eps,
|
||||
"V=0 regression: dir action {a} E[Q] = {qa} (expected ≈ 0 \
|
||||
for symmetric atoms with A=0). Either centering broken \
|
||||
or atom support misconfigured.",
|
||||
);
|
||||
}
|
||||
let q_v_zero_max = q_v_zero[..B0]
|
||||
.iter()
|
||||
.cloned()
|
||||
.fold(f32::NEG_INFINITY, f32::max);
|
||||
let q_v_zero_min = q_v_zero[..B0]
|
||||
.iter()
|
||||
.cloned()
|
||||
.fold(f32::INFINITY, f32::min);
|
||||
assert!(
|
||||
(q_v_zero_max - q_v_zero_min) < eps,
|
||||
"V=0 regression: per-direction E[Q] not all identical. \
|
||||
range=[{q_v_zero_min}, {q_v_zero_max}] q_v_zero={:?}",
|
||||
&q_v_zero[..B0],
|
||||
);
|
||||
|
||||
// Run #2: V non-uniform across atoms ([+1.5, -0.7, +0.3]).
|
||||
// Centered logits per direction = V + 0 = V (same for every
|
||||
// direction). Therefore:
|
||||
// - softmax probs the same across directions.
|
||||
// - per-action E[Q] the same value across directions.
|
||||
// V's actual value DOES affect the magnitude of E[Q] (not all
|
||||
// zero anymore — softmax(V) shifts mass off the symmetric
|
||||
// mean), but every direction sees the same E[Q].
|
||||
let v_nonuniform: Vec<f32> = vec![1.5, -0.7, 0.3];
|
||||
let q_v_nonuniform = run_q(&v_nonuniform);
|
||||
let q_v_nu_max = q_v_nonuniform[..B0]
|
||||
.iter()
|
||||
.cloned()
|
||||
.fold(f32::NEG_INFINITY, f32::max);
|
||||
let q_v_nu_min = q_v_nonuniform[..B0]
|
||||
.iter()
|
||||
.cloned()
|
||||
.fold(f32::INFINITY, f32::min);
|
||||
assert!(
|
||||
(q_v_nu_max - q_v_nu_min) < eps,
|
||||
"Test 2.3 regression: per-direction E[Q] not all identical \
|
||||
under symmetric A with V_nonuniform={:?}. \
|
||||
range=[{q_v_nu_min}, {q_v_nu_max}] q_v_nonuniform={:?}. V \
|
||||
leaked into centering ⇒ per-action distribution differs ⇒ \
|
||||
V signal alone biased direction selection (the SP17 contract \
|
||||
violation this test detects).",
|
||||
v_nonuniform, &q_v_nonuniform[..B0],
|
||||
);
|
||||
|
||||
// Sanity: V_nonuniform actually does shift the value (softmax
|
||||
// is biased toward the largest atom). With V=[1.5, -0.7, 0.3]
|
||||
// probs ∝ exp(V) ≈ [4.48, 0.50, 1.35] / 6.33 ≈ [0.708, 0.079,
|
||||
// 0.213], so E[Q] ≈ -0.708 + 0 + 0.213 ≈ -0.495. Not 0.
|
||||
// (If E[Q] is still zero, the kernel is ignoring V — caught
|
||||
// by the `compute_expected_q_centered_matches_cpu_oracle` test
|
||||
// above; just sanity-checking the non-degenerate case here.)
|
||||
let q_nu = q_v_nonuniform[0];
|
||||
assert!(
|
||||
q_nu.abs() > 0.1,
|
||||
"test sanity: V_nonuniform gave E[Q]={q_nu} ≈ 0 — V is \
|
||||
NOT being read by the kernel. (compute_expected_q_centered_\
|
||||
matches_cpu_oracle would also fail; this test is a \
|
||||
diagnostic-augmentation, not the primary regression \
|
||||
detector for the V-read path.)",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user