test(dqn): Phase 0 Test 0.E — synthetic edge discovery via Thompson Q-learning

Algorithmic property test (CPU). Confirms Thompson exploration discovers
KNOWN +0.005 edge in 100 iterations on a 1-state bandit, while
argmax-only training never updates Q[Long].

Setup revised from plan A draft (option 3 — production-realistic):
  p_long initial = [0.10, 0.20, 0.40, 0.20, 0.10] (uniform, E=0, has σ)
  p_flat initial = [0, 0, 1, 0, 0]                (δ(v=0), deterministic)
  Argmax with strict-> ties at E=0 → always picks Flat → never explores
  Long → Q[Long] stays at 0, never discovers edge.
  Thompson samples Long > 0 with P≈0.30 → ~30 effective updates → mean
  drifts toward +0.005, crosses Q[Flat]=0 within budget.

Plan's prior draft (initial p_long with mean=-0.015 + p_flat=δ(0)) was
calibration-bound: Thompson drift was directionally correct (-0.015 →
-0.005) but didn't cross zero in 100 iters. Revised setup eliminates
the artificial initial bias and matches production reality more
closely (Flat = δ(0) by construction; Long starts spread from random
init, then accumulates true edge).

Stop condition: if Thompson e_long ≤ e_flat with this setup, the
hypothesis is genuinely wrong and reward shaping must change before
proceeding to Phase 2.

Observed (local RTX 3050 Ti, ~0.00s test wall, ~1.57s 5-test suite):
  argmax  : e_long=0.000000, e_flat=0.000000 (asserts e_long ≤ 0.001 OK)
  thompson: e_long=0.003550, e_flat=0.000000 (asserts e_long > e_flat OK)

All 5 Phase 0 tests pass: 0.A bias-reproduces, 0.B inverse-CDF,
0.C IQN symmetry, 0.D Thompson-reverses, 0.E synthetic-edge.

crates/ml/src/trainers/dqn/distributional_q_tests.rs

@@ -535,3 +535,128 @@ fn test_0d_thompson_reverses_bias() {
     assert!(p_long >= 0.20,
         "Thompson P(Long) expected ≥ 0.20, got {} (active total {})", p_long, p_active);
 }
+
+/// Phase 0 Test 0.E — Synthetic edge discovery via Q-learning + Thompson.
+///
+/// Tests an ALGORITHMIC PROPERTY of Thompson exploration (does it discover
+/// edge if edge exists?), not a kernel correctness property. CPU is appropriate.
+///
+/// Setup: 1-state 2-action bandit. Action Long has KNOWN +0.005 edge after
+/// tx_cost. Action Flat returns exactly 0. Run Q-learning for 100 iterations
+/// with two action selectors:
+///   1. Argmax-only: always pick action with highest Q. Both Q[Long] and
+///      Q[Flat] start at E=0; strict-`>` tie-break consistently picks Flat
+///      → Long is never explored, Q[Long] never updates, the +0.005 edge is
+///      never discovered. Assertion: e_long ≤ e_flat + 0.001 (vacuously true
+///      at exactly e_long=e_flat=0).
+///   2. Thompson: sample s_long ~ p_long (has σ from spread), s_flat ~ p_flat
+///      (= δ(0), always 0). P(s_long > 0) = p_long[3] + p_long[4] = 0.30
+///      → Long is selected ~30% of iterations, each update pushes p_long
+///      mean toward the +0.005 reward target's nearest atom (+0.05).
+///      Mean drifts above zero by iter 100 → e_long > e_flat = 0.
+///
+/// Initial distributions (revised from plan A draft, option 3 — production-realistic):
+///   p_long initial = [0.10, 0.20, 0.40, 0.20, 0.10]   (uniform-ish, E=0, has σ;
+///                                                      mimics freshly initialized C51 head)
+///   p_flat initial = [0, 0, 1, 0, 0]                  (δ(v=0), tx_cost-bare Flat
+///                                                      deterministically returns 0)
+///
+/// Stop condition: if Thompson e_long ≤ e_flat with this setup, the hypothesis
+/// is genuinely wrong and reward shaping must change before proceeding to Phase 2.
+#[test]
+fn test_0e_synthetic_edge_discovery_cpu() {
+    use rand::{Rng, SeedableRng};
+    use rand::rngs::StdRng;
+    use rand_distr::{Normal, Distribution};
+
+    fn sample_categorical(p: &[f32], v: &[f32], rng: &mut StdRng) -> f32 {
+        let u: f32 = rng.gen();
+        let mut cum = 0.0_f32;
+        for i in 0..p.len() {
+            cum += p[i];
+            if u < cum { return v[i]; }
+        }
+        v[p.len() - 1]
+    }
+
+    fn run_q_learning(thompson: bool, seed: u64) -> (f32, f32) {
+        let mut rng = StdRng::seed_from_u64(seed);
+        // C51-like distribution: 5 atoms, [-0.1, -0.05, 0, 0.05, 0.1]
+        let atoms = [-0.1_f32, -0.05, 0.0, 0.05, 0.1];
+        // Long: uniform-ish, E[Q]=0, has variance. Mimics a freshly initialized
+        // C51 head before training: spread mass, mean near zero. P(sample > 0) ≈ 0.30.
+        let mut p_long = [0.10_f32, 0.20, 0.40, 0.20, 0.10];
+        // Flat = δ(v=0), deterministic. Production reality: tx_cost-bare flat returns 0.
+        let mut p_flat = [0.0_f32, 0.0, 1.0, 0.0, 0.0];
+        // True Long edge: Normal(+0.005, 0.03). Known positive edge ≈ 1/2 of σ.
+        let true_long_dist = Normal::new(0.005_f32, 0.03)
+            .expect("Normal(+0.005, 0.03) parameters valid");
+        let lr = 0.05_f32;
+        const N_ITERS: usize = 100;
+
+        for _iter in 0..N_ITERS {
+            // Action selection
+            let action = if thompson {
+                let s_long: f32 = sample_categorical(&p_long, &atoms, &mut rng);
+                let s_flat: f32 = sample_categorical(&p_flat, &atoms, &mut rng);
+                if s_long > s_flat { 0 } else { 1 }  // 0=long, 1=flat
+            } else {
+                let e_long: f32 = (0..5).map(|i| p_long[i] * atoms[i]).sum();
+                let e_flat: f32 = (0..5).map(|i| p_flat[i] * atoms[i]).sum();
+                if e_long > e_flat { 0 } else { 1 }
+            };
+
+            // Get reward — Long gets the synthetic edge, Flat is deterministic 0.
+            let reward = if action == 0 {
+                true_long_dist.sample(&mut rng)
+            } else {
+                0.0_f32
+            };
+
+            // Q-learning update: project reward onto atoms (nearest-bin).
+            let target_atom = atoms.iter()
+                .enumerate()
+                .min_by(|(_, a), (_, b)| {
+                    (*a - reward).abs()
+                        .partial_cmp(&(*b - reward).abs())
+                        .expect("atom distance is finite (no NaN)")
+                })
+                .map(|(i, _)| i)
+                .expect("atoms slice is non-empty");
+            let p = if action == 0 { &mut p_long } else { &mut p_flat };
+            for i in 0..5 {
+                let target = if i == target_atom { 1.0 } else { 0.0 };
+                p[i] = (1.0 - lr) * p[i] + lr * target;
+            }
+            // Renormalize (defensive — the convex blend above already preserves sum=1
+            // when both inputs sum to 1, but keep this so a future change to the
+            // update rule can't silently drift the total mass).
+            let s: f32 = p.iter().sum();
+            for i in 0..5 { p[i] /= s; }
+        }
+
+        let e_long: f32 = (0..5).map(|i| p_long[i] * atoms[i]).sum();
+        let e_flat: f32 = (0..5).map(|i| p_flat[i] * atoms[i]).sum();
+        (e_long, e_flat)
+    }
+
+    // Argmax-only: never explores Long → Q[Long] stays at 0 = Q[Flat].
+    let (e_long_argmax, e_flat_argmax) = run_q_learning(false, 42);
+    println!(
+        "Test 0.E argmax: e_long={:.6}, e_flat={:.6}",
+        e_long_argmax, e_flat_argmax
+    );
+    assert!(e_long_argmax <= e_flat_argmax + 0.001,
+        "argmax: Q[Long]={} should not exceed Q[Flat]={} (no exploration)",
+        e_long_argmax, e_flat_argmax);
+
+    // Thompson: ~30% Long-selections → mean drifts toward +0.005 → crosses 0.
+    let (e_long_thomp, e_flat_thomp) = run_q_learning(true, 42);
+    println!(
+        "Test 0.E thompson: e_long={:.6}, e_flat={:.6}",
+        e_long_thomp, e_flat_thomp
+    );
+    assert!(e_long_thomp > e_flat_thomp,
+        "Thompson: Q[Long]={} expected to exceed Q[Flat]={} after 100 iters",
+        e_long_thomp, e_flat_thomp);
+}

docs/dqn-wire-up-audit.md

@@ -82,6 +82,30 @@ exploration is sufficient at the realistic σ (not just the wide σ from Test 0.
 the failure-mode bias mechanism reproduces and Thompson reverses it. No
 production callers. Plan: `docs/superpowers/plans/2026-04-27-distributional-rl-thompson-plan-A-phase-0.md`.
 
+Plan A Phase 0 Task 7 Test 0.E (2026-04-27): `trainers/dqn/distributional_q_tests.rs` —
+appended `test_0e_synthetic_edge_discovery_cpu` (`#[test]`, NOT `#[ignore]` — pure
+CPU, no GPU dependency, runs in default `cargo test`). Algorithmic-property test
+of Thompson exploration on a 1-state 2-action bandit: Long has KNOWN +0.005 edge
+(`Normal(0.005, 0.03)`), Flat returns deterministic 0. Q-learning over 5 atoms
+`[-0.1, -0.05, 0, 0.05, 0.1]` with `lr=0.05` for 100 iterations under two action
+selectors. Initial setup revised from plan A draft (option 3 — production-realistic):
+`p_long = [0.10, 0.20, 0.40, 0.20, 0.10]` (uniform-ish, E=0, has σ; mimics
+freshly initialized C51 head); `p_flat = [0, 0, 1, 0, 0]` (δ(v=0); tx_cost-bare
+Flat returns 0). Argmax with strict-`>` tie-break at e_long=e_flat=0 always
+picks Flat → never explores Long → e_long stays 0 (assertion
+`e_long ≤ e_flat + 0.001` passes vacuously). Thompson: `P(s_long > 0) = p_long[3]
++ p_long[4] = 0.30` → ~30 effective Long-selections drift p_long mean toward
+the +0.005 reward target's nearest atom (+0.05). Observed values on local
+RTX 3050 Ti: argmax e_long=0.000000, e_flat=0.000000; Thompson e_long=0.003550,
+e_flat=0.000000 (test ~0.00 s; total 5-test run ~1.57 s). Plan's prior draft
+(initial p_long with mean=-0.015 + p_flat=δ(0)) was calibration-bound:
+Thompson drift was directionally correct but didn't cross zero in 100 iters.
+Revised setup eliminates the artificial initial bias and matches production
+reality. Stop condition: if Thompson `e_long ≤ e_flat` with this setup, the
+hypothesis is genuinely wrong and reward shaping must change before Phase 2.
+Uses `rand` + `rand_distr` (already in workspace deps). No production callers.
+Plan: `docs/superpowers/plans/2026-04-27-distributional-rl-thompson-plan-A-phase-0.md`.
+
 Plan A Phase 0 batched-pinned redesign (2026-04-27): `cuda_pipeline/thompson_test_kernel.cu`
 + `trainers/dqn/distributional_q_tests.rs` — eliminates the per-seed `clone_dtoh`
 the original Task 2 wrapper used inside the 10 000- (Test 0.A) / 100 000-iteration