test(sp17): asymmetric A → deterministic argmax under MIN_TEMP=0.5
Verifies that when one direction's centered advantage dominates
strongly enough, action selection picks it deterministically across
all N=1000 Philox-keyed runs.
Plan specifies "Thompson temp = 0.0 → pure argmax E[Q]", but the
kernel floors thompson_temp at MIN_TEMP=0.5 per
pearl_blend_formulas_must_have_permanent_floor — pure τ=0 isn't
ISV-accessible. With τ=0.5 the blend is q_eff = 0.5·E[Q] +
0.5·q_sample; deterministic argmax across all τ=0.5 draws requires
the E[Q] gap to exceed atom_span/2.
Construction: NA=3 atoms [-0.1, 0, +0.1] (atom_span=0.2). A_dir
puts +300 at z=2 for Long, -100 at z=2 for others (per-atom mean
zero ⇒ centering preserves shape). Long centered E[Q] ≈ +0.1, others
≈ -0.05 (gap 0.15 > 0.1). q_sample[Long] = +0.1 with prob ≈ 1
(softmax(300) ≈ delta at z=2); q_sample[other] ∈ {-0.1, 0} (zero
prob for +0.1). q_eff[Long] = 0.1; q_eff[other] ≤ -0.025. Long
strictly wins all draws.
If the centering breaks (Long no longer dominant under centered E[Q])
or τ blends a non-Long sample over Long, this test fires.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -1895,4 +1895,222 @@ mod gpu {
|
||||
detector for the V-read path.)",
|
||||
);
|
||||
}
|
||||
|
||||
/// SP17 Phase 2 Task 2.4: asymmetric A → deterministic argmax under
|
||||
/// kernel-floored Thompson temperature.
|
||||
///
|
||||
/// Architectural contract: when one direction's centered advantage
|
||||
/// dominates strongly enough that *any* Thompson sample combined
|
||||
/// with the per-direction E[Q] cannot be beaten by another
|
||||
/// direction's draw, action selection picks the dominant direction
|
||||
/// in 100% of runs.
|
||||
///
|
||||
/// Restated under kernel constraints:
|
||||
/// The plan specifies "Thompson temp = 0.0 → pure argmax E[Q]",
|
||||
/// but the kernel floors `thompson_temp` at MIN_TEMP=0.5 per
|
||||
/// `pearl_blend_formulas_must_have_permanent_floor`. With τ=0.5
|
||||
/// the blend is `q_eff[d] = 0.5·E[Q][d] + 0.5·q_sample[d]`. We
|
||||
/// engineer the A asymmetry tightly enough that even the worst-
|
||||
/// case Long sample (lowest atom in its support) still beats the
|
||||
/// best-case other-direction sample (highest atom in its support).
|
||||
/// That tightness requires:
|
||||
/// E[Q][Long] − E[Q][other] > atom_max − atom_min
|
||||
/// under τ=0.5 — a 2× E[Q] gap relative to atom span.
|
||||
///
|
||||
/// Construction:
|
||||
/// - NA=3 atoms at v_min=-0.1, dz=0.1 ⇒ atoms [-0.1, 0, +0.1]
|
||||
/// (atom_span = 0.2).
|
||||
/// - V = 0 (V is irrelevant — equal across atoms means equal
|
||||
/// across actions; only A asymmetry drives the per-direction
|
||||
/// E[Q] difference under centering).
|
||||
/// - A_dir constructed so per-atom mean is zero (so centered
|
||||
/// A == raw A) and Long's logit at z=+0.1 is +300 vs others'
|
||||
/// -100. Centered Long's softmax → ~100% mass at z=+0.1
|
||||
/// ⇒ E[Q]≈+0.1; centered other's softmax → ~50/50 mass at
|
||||
/// z={-0.1, 0} ⇒ E[Q]≈-0.05. Gap ≈ 0.15 > atom_span/2 = 0.1
|
||||
/// under τ=0.5.
|
||||
/// - q_sample[Long] is +0.1 with probability ≈ 1.
|
||||
/// q_sample[other] is in {-0.1, 0} (zero probability for +0.1).
|
||||
/// q_eff[Long] = 0.5·0.1 + 0.5·0.1 = 0.1.
|
||||
/// q_eff[other] = 0.5·(-0.05) + 0.5·{-0.1, 0} ∈ {-0.075, -0.025}.
|
||||
/// Long wins all draws.
|
||||
/// - N=1000 samples in one launch (Philox keys vary by `i`,
|
||||
/// giving N independent Thompson trajectories).
|
||||
///
|
||||
/// Assertion: dir = DIR_LONG (= 2) in 100% of N=1000 runs. The
|
||||
/// engineered gap leaves only a sub-1e-44 probability of any
|
||||
/// Long sample landing at z=-0.1 (tail of softmax(L=300)), so a
|
||||
/// strict 100% pass is statistically robust.
|
||||
#[test]
|
||||
#[ignore = "requires GPU"]
|
||||
fn asymmetric_a_yields_deterministic_argmax() {
|
||||
let stream = make_test_stream();
|
||||
let kernel = load_experience_action_select(&stream);
|
||||
|
||||
const N: usize = 1000;
|
||||
const NA: usize = 3;
|
||||
const B0: usize = 4;
|
||||
const B1: usize = 3;
|
||||
const B2: usize = 3;
|
||||
const B3: usize = 3;
|
||||
const TOTAL_ACTIONS: usize = B0 + B1 + B2 + B3;
|
||||
const ISV_DIM: usize = 483;
|
||||
// DIR_LONG per state_layout.cuh.
|
||||
const DIR_LONG: usize = 2;
|
||||
|
||||
// q_values: zeros for mag/ord/urg slots — these branches are
|
||||
// not under test. (Kernel writes dir slots via Thompson on
|
||||
// b_logits_dir + v_logits_dir; pre-fill them as zeros; the
|
||||
// kernel's Pass 1 over e_dir feeds them.)
|
||||
let q_values = vec![0.0_f32; N * TOTAL_ACTIONS];
|
||||
|
||||
// V = 0 across all atoms (V irrelevant — common to every
|
||||
// direction under centering).
|
||||
let v_logits = vec![0.0_f32; N * NA];
|
||||
|
||||
// A_dir: only z=2 differs across actions; per-atom mean is 0
|
||||
// (so centering doesn't shift the logits). Long (a=DIR_LONG=2)
|
||||
// has +300 at z=2; others have -100 at z=2.
|
||||
// z=0: [0, 0, 0, 0] → mean 0 → centered [0,0,0,0]
|
||||
// z=1: [0, 0, 0, 0] → mean 0 → centered [0,0,0,0]
|
||||
// z=2: [-100, -100, +300, -100] → mean 0 → centered same
|
||||
let mut b_logits = vec![0.0_f32; N * B0 * NA];
|
||||
for i in 0..N {
|
||||
for a in 0..B0 {
|
||||
let off = i * B0 * NA + a * NA;
|
||||
if a == DIR_LONG {
|
||||
b_logits[off + 2] = 300.0;
|
||||
} else {
|
||||
b_logits[off + 2] = -100.0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// per_sample_support: branch 0 uses (-0.1, 0.1, 0.1) → atoms
|
||||
// [-0.1, 0, +0.1]. Tight atom span lets τ=0.5 still produce
|
||||
// deterministic argmax (E[Q] gap > atom_span/2).
|
||||
let mut support_host = vec![0.0_f32; N * 4 * 3];
|
||||
for i in 0..N {
|
||||
for d in 0..4 {
|
||||
support_host[(i * 4 + d) * 3 + 0] = -0.1; // v_min
|
||||
support_host[(i * 4 + d) * 3 + 1] = 0.1; // v_max
|
||||
support_host[(i * 4 + d) * 3 + 2] = 0.1; // delta_z
|
||||
}
|
||||
}
|
||||
|
||||
// ISV: cooldown OFF (435=0), plasticity warm OFF (438=0),
|
||||
// hold floor disabled (alpha=0 default). Thompson temp slot
|
||||
// 339 left at 0 → kernel floors to MIN_TEMP=0.5.
|
||||
let isv_host = vec![0.0_f32; ISV_DIM];
|
||||
|
||||
let v_buf = unsafe { MappedF32Buffer::new(v_logits.len()) }
|
||||
.expect("alloc v_logits");
|
||||
v_buf.write_from_slice(&v_logits);
|
||||
let q_buf = unsafe { MappedF32Buffer::new(q_values.len()) }
|
||||
.expect("alloc q_values");
|
||||
q_buf.write_from_slice(&q_values);
|
||||
let b_buf = unsafe { MappedF32Buffer::new(b_logits.len()) }
|
||||
.expect("alloc b_logits");
|
||||
b_buf.write_from_slice(&b_logits);
|
||||
let support_buf = unsafe { MappedF32Buffer::new(support_host.len()) }
|
||||
.expect("alloc support");
|
||||
support_buf.write_from_slice(&support_host);
|
||||
let isv_buf = unsafe { MappedF32Buffer::new(ISV_DIM) }.expect("alloc isv");
|
||||
isv_buf.write_from_slice(&isv_host);
|
||||
|
||||
let actions_buf = unsafe { MappedI32Buffer::new(N) }.expect("alloc actions");
|
||||
actions_buf.write_from_slice(&vec![0_i32; N]);
|
||||
let q_gaps_buf = unsafe { MappedF32Buffer::new(N) }.expect("alloc q_gaps");
|
||||
q_gaps_buf.write_from_slice(&vec![0.0_f32; N]);
|
||||
let portfolio_buf = unsafe { MappedF32Buffer::new(N * 23) }.expect("alloc portfolio");
|
||||
portfolio_buf.write_from_slice(&vec![0.0_f32; N * 23]);
|
||||
let conviction_buf = unsafe { MappedF32Buffer::new(N) }.expect("alloc conviction");
|
||||
conviction_buf.write_from_slice(&vec![0.0_f32; N]);
|
||||
let mag_conv_buf = unsafe { MappedF32Buffer::new(N) }.expect("alloc mag conv");
|
||||
mag_conv_buf.write_from_slice(&vec![0.0_f32; N]);
|
||||
|
||||
let null_atom_positions: u64 = 0;
|
||||
let null_per_sample_eps: u64 = 0;
|
||||
let null_intent_mag: u64 = 0;
|
||||
|
||||
let n_i32: i32 = N as i32;
|
||||
let b0_i32: i32 = B0 as i32;
|
||||
let b1_i32: i32 = B1 as i32;
|
||||
let b2_i32: i32 = B2 as i32;
|
||||
let b3_i32: i32 = B3 as i32;
|
||||
let na_i32: i32 = NA as i32;
|
||||
let block_dim: u32 = 256;
|
||||
let grid_dim: u32 = ((N as u32 + block_dim - 1) / block_dim).max(1);
|
||||
|
||||
unsafe {
|
||||
stream
|
||||
.launch_builder(&kernel)
|
||||
.arg(&q_buf.dev_ptr)
|
||||
.arg(&actions_buf.dev_ptr)
|
||||
.arg(&q_gaps_buf.dev_ptr)
|
||||
.arg(&0.0_f32) // eps_start = 0 (eval mode)
|
||||
.arg(&0.0_f32) // eps_end = 0
|
||||
.arg(&0_i32) // current_epoch
|
||||
.arg(&1_i32) // total_epochs
|
||||
.arg(&n_i32)
|
||||
.arg(&b0_i32)
|
||||
.arg(&b1_i32)
|
||||
.arg(&b2_i32)
|
||||
.arg(&b3_i32)
|
||||
.arg(&0.0_f32) // q_gap_threshold = 0
|
||||
.arg(&portfolio_buf.dev_ptr)
|
||||
.arg(&0_i32) // min_hold_bars
|
||||
.arg(&1.0_f32) // max_position
|
||||
.arg(&1.0_f32) // eps_exp_mult
|
||||
.arg(&1.0_f32) // eps_ord_mult
|
||||
.arg(&1.0_f32) // eps_urg_mult
|
||||
.arg(&0_i32) // timestep — fixed, samples vary by `i`
|
||||
.arg(&null_per_sample_eps)
|
||||
.arg(&isv_buf.dev_ptr)
|
||||
.arg(&0_i32) // contrarian_active
|
||||
.arg(&null_intent_mag)
|
||||
.arg(&conviction_buf.dev_ptr)
|
||||
.arg(&mag_conv_buf.dev_ptr)
|
||||
.arg(&b_buf.dev_ptr)
|
||||
.arg(&v_buf.dev_ptr)
|
||||
.arg(&support_buf.dev_ptr)
|
||||
.arg(&null_atom_positions)
|
||||
.arg(&na_i32)
|
||||
.arg(&0.0_f32) // plasticity_m_warm = 0 (disabled)
|
||||
.launch(LaunchConfig {
|
||||
grid_dim: (grid_dim, 1, 1),
|
||||
block_dim: (block_dim, 1, 1),
|
||||
shared_mem_bytes: 0,
|
||||
})
|
||||
.expect("launch experience_action_select");
|
||||
}
|
||||
stream.synchronize().expect("sync");
|
||||
|
||||
// Decode action → dir: action = dir * (b1*b2*b3) + ...
|
||||
let actions = actions_buf.read_all();
|
||||
let mut counts = [0_usize; B0];
|
||||
for &a in actions.iter() {
|
||||
let dir = (a as usize) / (B1 * B2 * B3);
|
||||
assert!(dir < B0, "decoded dir {dir} out of range, raw={a}");
|
||||
counts[dir] += 1;
|
||||
}
|
||||
|
||||
// Strict assertion: 100% Long picks.
|
||||
assert_eq!(
|
||||
counts[DIR_LONG], N,
|
||||
"Test 2.4 regression: dir=DIR_LONG ({DIR_LONG}) picked \
|
||||
{}/{N} times under asymmetric A (Long-dominant centered \
|
||||
E[Q]). counts={:?}. Expected 100% Long; any non-Long pick \
|
||||
indicates either (a) centering broke and Long isn't \
|
||||
actually dominant in centered E[Q], or (b) the τ=0.5 \
|
||||
temperature blend let a Thompson sample beat the \
|
||||
deterministic gap. The Long-dominant gap is engineered \
|
||||
to exceed atom_span/2 under τ=0.5 (E[Q]_Long ≈ +0.1, \
|
||||
E[Q]_other ≈ -0.05; gap=0.15; atom_span=0.2; q_eff_Long_min \
|
||||
= 0.5*0.1 + 0.5*(-0.1) = 0 vs q_eff_other_max = \
|
||||
0.5*(-0.05) + 0.5*0 = -0.025 — Long always strictly > \
|
||||
other).",
|
||||
counts[DIR_LONG], counts,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user