foxhunt/docs/superpowers/notes/2026-06-02-determinism-phase1-diagnosis.md

Determinism Phase 1 — Diagnosis

Date: 2026-06-02 Branch: ml-alpha-adaptive-controller-floors Spec: docs/superpowers/specs/2026-06-02-determinism-foundation.md §1 Status: Phase 1 SHIPPED — diagnosis complete, fix is Phase 2 (separate dispatch).

TL;DR

scripts/determinism-check.sh --quick (200 train + 50 eval at b=128, seed=42) on ml-alpha-adaptive-controller-floors produces:

FIRST_DIVERGENCE: step=2 leaf=checksums.replay_sample_indices
                  run_A=709847.0 run_B=2851599.0 delta=2.14175e+06

Culprit: PER (Prioritized Experience Replay) sampling. Spec §2.F is the fix.

Steps 0-1 are bit-equivalent across runs. Divergence appears at step 2 the moment the replay buffer transitions from "trivially deterministic" (small, all entries equally probable) into "PER-sampled with priority weights".

Phase 1 deliverables

1. Kernel — crates/ml-alpha/cuda/rl_deterministic_checksum.cu

   • Two entry points: rl_deterministic_checksum_f32 + rl_deterministic_checksum_i32
   • Both are (1,1,1)/(1,1,1) launches — a single thread sums data[i]² into a double-precision accumulator. NO atomics, NO multi-block reduce, NO non-deterministic patterns by construction. Per feedback_no_atomicadd.
   • Cost per launch: ~5-20 μs launch overhead + sequential f64 accumulation (~1ns per element on RTX 3050). Largest tensor we checksum is the Q-head weight ss_q_grad_w_d at ~24k floats — 24 μs sequential.
   • Registered in crates/ml-alpha/build.rs (AOT cubin compile per feedback_no_nvrtc).

2. Diag emission — crates/ml-alpha/src/trainer/integrated.rs

   • New method IntegratedTrainer::compute_diag_checksums(&mut self) → Result<[f64; 15]>
   • Allocates a 28-f64 scratch buffer (checksum_scratch_d) once at trainer construction; 15 final-leaf slots + 12 Adam staging slots (the optimizer's m and v live in six separate sub-buffers for the DQN/π/V w+b parameter groups; host-folded into adam_m_sum / adam_v_sum after the dtoh).
   • build_diag_value is now &mut self; it calls compute_diag_checksums up-front and inserts the 15 leaves under a new top-level checksums JSON object.
   • 15 leaves emitted per spec §1.2: encoder_output, q_logits, pi_logits, v_value, replay_sample_indices, rewards_after_shape, advantages, q_grad, pi_grad, v_grad, encoder_grad, adam_m_sum, adam_v_sum, isv_state, popart_sigma_state.
   • tests/eval_diag_emission.rs EXPECTED_LEAVES bumped 679 → 694.
   • Call-site fix in crates/ml-alpha/examples/alpha_rl_train.rs: the train-loop let isv = trainer.isv_host_slice() borrow conflicted with the new &mut self on build_diag_value because the slice was referenced past the call in an eprintln!. Fix is local — pre-extract the 4 ISV values into f32 locals before the diag call. Eval loop was unaffected (NLL drops the borrow before the diag call there).

3. Diagnostic — scripts/determinism-check.sh rewritten

   • Iterates over EVERY row of diag.jsonl (was: last 5 rows)
   • Compares only the checksums.* subtree leaf-by-leaf (other diag fields are derived quantities; localising to component checksums gives a cleaner signal)
   • Reports first (step, leaf) divergence at rel-tol=1e-5 + abs-tol=1e-7
   • Also lists ALL leaves diverging at the first divergent step (multiple components frequently co-diverge — the script ranks them by Δ magnitude so the top entry is the most-likely root cause)
   • Maps each known leaf to its spec §2 fix recommendation
   • Exit 0 = deterministic / 1 = drift detected / 2 = inputs missing

Diagnostic findings — step-by-step

Output of ./scripts/determinism-check.sh --quick:

Step	Diverging leaves	Top Δ
0	0 (bit-equivalent)	—
1	0 (bit-equivalent)	—
2	5 simultaneously: replay_sample_indices, q_grad, adam_m_sum, q_logits, adam_v_sum	2.14e6 on replay_sample_indices
3	11 (drift cascades through forward + grads + ISV)	—
4-199	10-15 leaves drift on every step	—

Total divergent rows: 198 / 200.

Interpretation

The cascade ordering is structurally clean:

1. Step 0-1: PER buffer has not accumulated enough transitions for priority-weighted sampling to actually choose different indices (or the priorities are still degenerate). Both runs produce identical indices, identical sampled h_t/h_tp1, identical Q forward, identical gradients, identical Adam state. Bit-equivalent.

2. Step 2 — divergence enters via replay_sample_indices. The two runs choose different replay transitions to sample. From this moment the Q gradient (which depends on the sampled batch) diverges, which immediately diverges Q-head Adam m+v, which on the same step diverges q_logits (via the immediately-following Q forward in the next step's prefill). The 5-simultaneous divergence at step 2 is consistent with a single root cause (PER sampling) propagating to all downstream buffers in the same training step.

3. Step 3+: now divergent gradients have been Adam-applied to parameters → encoder forward diverges (encoder_output), π and V forward diverge, advantages diverge, controllers' EMA inputs diverge, ISV state diverges. Cascade complete.

The fact that replay_sample_indices is the FIRST and LARGEST-magnitude divergence at step 2, while every other field is bit-equivalent up through step 1, is decisive: the bug is in the PER sampling kernel chain.

Candidate root causes (Phase 2 investigation scope)

Spec §2.F lists the entry points to audit:

1. rl_per_sample kernel — stratified proportional sampling via sum-tree walk + gather. The walk uses a device-side xorshift32 PRNG (gpu_replay.sample_prng_d). Audit whether:

   • The PRNG state is seeded deterministically and not reset between runs (must use cli.seed.wrapping_add(...) per the existing CPU-side audit in items 1-5 impl log).
   • The kernel's launch configuration (Grid=B) reads state in a deterministic order (each block owns one batch slot → fixed).
   • The sum-tree walk converges deterministically — IEEE-754 sum-tree comparisons can produce different paths if the tree values were accumulated in a non-deterministic order (PER push_flush uses prefix-sum allocation, which is non-deterministic on parallel warps if the input order varies).

2. rl_per_push_flush and rl_per_tree_rebuild — coordinated coalesced replay writes + bottom-up parallel sum-tree rebuild. Both "have no atomics" per their build.rs comments, but the rebuild's bottom-up parallelism may produce different floating-point sums if the tree leaf order differs across runs. Audit: does the tree rebuild deterministically reduce children before parents?

3. rl_per_update_priority — writes |TD|^α to tree leaves + block- wide max reduction. The block-wide max should be deterministic, but any precision-sensitive aggregation could be the culprit. Less likely.

4. Priority update sequencing — pearl_rl_sample_tau_xorshift32 notes the PRNG seeding pattern. Verify the seed reaches sample_prng_d AND is not perturbed by any unrelated state at construction time.

Recommendation for Phase 2

Apply spec §2.F (PER sampling determinism audit). Concrete steps:

1. Add per-batch dumps of sample_prng_d, priority_tree_d, and sample_indices_d at step 0, 1, 2 of two same-seed runs. Compare PRNG state at step 1's exit / step 2's entry — if PRNG already diverges → fix the seeding; if PRNG matches but indices differ → the priority-tree walk is the culprit.
2. If priority tree is suspect, audit rl_per_tree_rebuild for any parallel reduction whose addition order depends on warp scheduling. The "no atomics" claim doesn't preclude order-of-addition variance in a parallel scan.
3. Apply the deterministic-by-construction fix (single-block reduction for small trees, or fixed-warp ordering for large trees).
4. Re-run scripts/determinism-check.sh --quick after the fix. Success = "DETERMINISTIC: all checksums.* leaves match across all 200 rows".

Once PER is fixed, the next divergence (if any) localises automatically. Phase 1 infrastructure is reusable — Phase 2 just runs the same script.

Files touched (Phase 1, no commits per dispatch instruction)

Path	Change
crates/ml-alpha/cuda/rl_deterministic_checksum.cu (NEW)	Sum-of-squares kernel, two entry points (f32 + i32)
crates/ml-alpha/build.rs	Registered the new kernel in KERNELS
crates/ml-alpha/src/trainer/optim.rs	Added immutable m() / v() accessors on AdamW
crates/ml-alpha/src/trainer/integrated.rs	New cubin const + struct fields, module loading in new(), compute_diag_checksums() method, build_diag_value switched to &mut self + emits checksums JSON object
crates/ml-alpha/examples/alpha_rl_train.rs	Pre-extracted ISV values into locals to release the isv_host_slice() borrow before calling build_diag_value (now &mut self)
crates/ml-alpha/tests/eval_diag_emission.rs	EXPECTED_LEAVES 679 → 694
scripts/determinism-check.sh	Rewritten to compare all rows × checksums.* leaves; reports FIRST_DIVERGENCE + spec §2 fix recommendation
docs/superpowers/notes/2026-06-02-determinism-phase1-diagnosis.md (NEW, this file)	Diagnosis log

Discipline checklist

• No new files in repo root
• No new docs outside docs/superpowers/notes/
• CUDA kernel registered in build.rs per feedback_no_nvrtc
• Kernel uses NO atomics per feedback_no_atomicadd
• Result read from CPU only; computation entirely on device per feedback_cpu_is_read_only
• All 15 checksums end-to-end wired — no stubs per feedback_no_stubs
• Local RTX 3050 smoke validation only — NO cluster submission
• Read-before-Edit on all touched existing files
• No fix applied — diagnosis only, per dispatch instruction (Phase 2 is separate)
• No commit — left uncommitted for human review

Reproduction

# Build
SQLX_OFFLINE=true cargo build --release --example alpha_rl_train -p ml-alpha

# Run
./scripts/determinism-check.sh --quick

# Expected output (current branch ml-alpha-adaptive-controller-floors):
# FIRST_DIVERGENCE: step=2 leaf=checksums.replay_sample_indices
#                   run_A=709847.0 run_B=2851599.0 delta=2.14175e+06