# Determinism Foundation — Stop Chasing Ghosts

**Date**: 2026-06-02
**Status**: Spec draft — MANDATORY PREREQUISITE for any further controller / architecture work
**Priority**: BLOCKER for multi-head policy (`2026-06-02-multi-head-policy-with-r-multiple.md`), R-multiple reward, and the entire regime-invariance roadmap
**Empirical motivation**: alpha-rl-determinism-check.sh on `ml-alpha-regime-observer @ 63fc16f17` showed same-seed runs diverging at **step 2**, eval pnl Δ=**$364,925**, run A produced **0 trades** while run B produced **347 trades**. Bit-equivalence requirement: same seed → same diag rows to fp32 rel-tol 1e-5.

## §0. Motivation — the going-in-circles diagnosis was incomplete

The session of 2026-06-01/02 cycled through 4 controller iterations (v4 → v5.2) plus today's grwwh eval. Each iteration produced a different result. We attributed the variance to:
1. Bad controller designs (the v5/v5.1 bugs)
2. Architectural mismatch (single policy can't generalize)
3. Reward signal contamination (Phase 5 hold-bonus)

What we did NOT consider: **part of the variance was non-determinism**.

The 2026-06-02 mid-smoke determinism check revealed:
- Same seed, same code, same hardware → divergence at step 2
- By step 199: 654 diag leaves diverging
- Eval pnl Δ=$365k between identical configurations
- Behavioral Δ: run A=0 trades vs run B=347 trades (different agent entirely)

**Implications**:
1. Every controller iteration today might have been measuring seed-noise, not controller effect
2. The "scaffold v5 stuck at w=1.0" might be partially seed-dependent (would different seeds also stick?)
3. The "grwwh eval +$74M then collapse to -$160M" might partially be intra-run variance
4. **Until determinism is fixed, every future iteration is also seed-dependent and unfalsifiable**

Per `feedback_going_in_circles_pattern`: "audit the foundation". The foundation has a bug. **Fix the foundation before adding more layers**.

## §1. Investigation strategy — localize divergence to ONE component

Per `feedback_systematic_debugging` Phase 1 "Gather evidence in multi-component systems": instrument every boundary, find FIRST divergence.

### §1.1 Deterministic checksum kernel (NEW infrastructure)

Add `crates/ml-alpha/cuda/rl_deterministic_checksum.cu`:
- Computes sum-of-squares of any device tensor with **provably deterministic** accumulation
- Single-block, single-thread accumulation if tensor < 1024 elements (trivially deterministic)
- For larger tensors: block-tree-reduce with **fixed accumulation order** (block 0 always reads block 1's result, never vice versa; `__threadfence_system` + counter `cudaDeviceSynchronize` if needed; NO atomic ops, NO "whichever-block-finishes-last" patterns)
- Returns one f64 per tensor (sum-of-squares, full precision)
- Compile-time guaranteed determinism via construction, not configuration

### §1.2 Per-step component checksums in diag

Instrument every major component output in `build_diag_value`:

```rust
"checksums": {
    "encoder_output": checksum(encoder_out_d, b_size × seq_len × hidden_dim),
    "q_logits": checksum(q_logits_d, b_size × n_actions × n_atoms),
    "pi_logits": checksum(pi_logits_d, b_size × n_actions),
    "v_value": checksum(v_d, b_size),
    "replay_sample_indices": checksum_int(per_sampled_indices_d, b_size),
    "rewards_after_shape": checksum(rewards_d, b_size),
    "advantages": checksum(advantages_d, b_size),
    "q_grad": checksum(q_grad_d, all_params),
    "pi_grad": checksum(pi_grad_d, all_params),
    "v_grad": checksum(v_grad_d, all_params),
    "encoder_grad": checksum(encoder_grad_d, all_params),
    "adam_m_sum": checksum(adam_first_moments_d, all_params),
    "adam_v_sum": checksum(adam_second_moments_d, all_params),
    "isv_state": checksum(isv_dev_ptr, RL_SLOTS_END),
}
```

Cost: ~15 small kernel launches per step, each ~10us → 150us/step overhead. Acceptable in dev mode; toggle off in production.

### §1.3 Run determinism-check + localization

`scripts/determinism-check.sh` now compares checksums leaf-by-leaf. Output: first leaf to diverge + step number + Δ value.

Expected outcomes (one of these wins):
- `checksums.encoder_output` diverges first → **mamba2 culprit** (most likely)
- `checksums.q_logits` diverges first (encoder same) → Q head GEMM split-K culprit
- `checksums.pi_logits` first → π head GEMM culprit
- `checksums.replay_sample_indices` first → PER sampling RNG culprit
- `checksums.q_grad` first (forward same) → backward kernel reduction culprit
- `checksums.adam_m_sum` first (grads same) → optimizer iteration-order culprit

## §2. Standard fixes by component (apply based on §1 localization)

### §2.A Mamba2 selective scan (most-likely culprit)

If encoder checksum diverges first:

- **Chunk-size determinism**: mamba2 selective scan uses chunk-parallel scan. Setting `CHUNK_SIZE=1` reduces to sequential scan (trivially deterministic but ~10× slower). Use as ground-truth oracle.
- **Reduction-order fix**: within each chunk, the scan reduction must be fixed-order. Audit `mamba2_selective_scan_fwd.cu` for any `atomicAdd`, any "block 0 OR block 1 writes final" patterns, any non-deterministic chunk ordering.
- **Reference**: pearl `pearl_mamba2_selective_scan_nan_source` (Phase 2 v2 work) — same code path was extensively debugged for NaN issues. May have introduced non-det then.

### §2.B cuBLAS GEMM split-K

If Q/π/V head checksums diverge first (encoder same):

- Set `cublasSetMathMode(cublas_handle, CUBLAS_PEDANTIC_MATH)` — disables auto-selection of non-deterministic algorithms
- Disable TF32 (`cublasSetMathMode(CUBLAS_DEFAULT_MATH)`) — TF32 split-K is non-deterministic per NVIDIA docs
- This REVERSES tasks 30/31 (TF32 enable). Document the perf cost trade-off (~10-15% on H100 GEMMs).
- Alternative: use FP32 explicitly via `cublasSgemmStridedBatched` with explicit algorithm selection

### §2.C cuDNN deterministic mode

If conv/RNN kernels involved:
- `cudnnSetMathMode(CUDNN_DEFAULT_MATH)` (no TF32)
- Force `CUDNN_DETERMINISTIC` via algorithm selection
- Foxhunt may not use cuDNN heavily (most ops are custom CUDA kernels) — verify before applying

### §2.D Custom kernel block-tree-reduce audit

Per `feedback_no_atomicadd`, all reductions use block-tree-reduce. Audit each kernel for:
- ANY use of "last block to finish writes final" patterns — must become "block 0 always writes final" with proper synchronization
- ANY use of `cudaDeviceSynchronize` inside kernels (rare) — implies non-graph-capturable patterns
- The `block_reduce_to_global_with_counter` pattern (if used) — non-deterministic by construction

### §2.E Adam optimizer iteration order

If `adam_m_sum` checksum diverges (grads same):
- Audit AdamW step in `crates/ml-alpha/src/networks/*` for iteration order
- HashMap iteration is non-deterministic in Rust — use `BTreeMap` or pre-sorted `Vec<(key, value)>` for parameter groups
- Per-tensor iteration MUST be a fixed order

### §2.F PER (Prioritized Experience Replay) sampling

If `replay_sample_indices` checksum diverges:
- The PER index sampling kernel must use a deterministic RNG (device-side xorshift32 seeded from host per `pearl_rl_sample_tau_xorshift32`)
- Verify the seed is correctly propagated and not re-randomized between runs
- Check that priority updates and reads are properly synchronized via events (not just stream ordering)

## §3. Test infrastructure — make determinism a CONTRACT

### §3.1 Local determinism gate

`scripts/determinism-check.sh` (already created by fast-dev-cycle items 1-5) is the gate. Add:
- Exit 0 if all checksums match to rel-tol 1e-5
- Exit 1 with first-divergent-leaf report otherwise
- Verbose mode shows checksum trajectory side-by-side

### §3.2 GPU-oracle determinism invariant test

Add `crates/ml-alpha/tests/determinism_invariants.rs`:
- Test 1: 200-step mid-smoke twice with seed=42 → all checksums match
- Test 2: 200-step mid-smoke with seed=42 vs seed=43 → checksums DIFFER (sanity: seed actually does something)
- Test 3: 200-step mid-smoke with seed=42 on RTX 3050 → checksum at step 199 matches a frozen golden value (catches regression to non-det)
- All 3 must pass before any cluster smoke can submit (CI gate)

### §3.3 Pre-commit hook

`.git/hooks/pre-commit` (or `.claude/hooks/`):
- If any `.cu` or `.rs` file in `crates/ml-alpha/` changed → run 50-step micro-determinism-check
- If divergence → BLOCK commit
- Override with `git commit --no-verify` (logged + requires justification)

### §3.4 Cluster determinism regression test

Once local determinism is fixed:
- One re-run of any cluster smoke with same SHA should produce identical eval_summary.json
- Add to `scripts/argo-alpha-rl.sh`: optional `--regression-pair` flag that submits TWO identical workflows and verifies they match

## §4. Speed cost — document the dev-vs-prod trade-off

Determinism almost always costs throughput:
- TF32 → FP32: ~10-15% on H100 GEMMs (per NVIDIA docs)
- Mamba2 chunk-1: ~10× slower (if needed; should not be needed if reduction-order fix works)
- cuDNN deterministic: ~5-20% varies
- Checksum kernels: ~150us/step overhead

**Acceptable in dev** (user explicitly said "we are in dev, determism is most important").

**Production toggle** via env var `FOXHUNT_DETERMINISTIC` (default 1 for dev, 0 for production):
```rust
let pedantic = std::env::var("FOXHUNT_DETERMINISTIC").unwrap_or_else(|_| "1".to_string()) == "1";
if pedantic {
    cublasSetMathMode(handle, CUBLAS_PEDANTIC_MATH);
} else {
    cublasSetMathMode(handle, CUBLAS_TF32_TENSOR_OP_MATH);
}
```

Production runs (live trading, walk-forward production) MAY use non-deterministic mode for the 10-15% throughput gain. ALL dev / cluster smokes MUST use deterministic mode.

## §5. CI guard

Add to GitLab CI (or whatever CI foxhunt uses — see `feedback_argo_workflows_primary`):
- Job: `determinism-gate`
  - Triggers: PR to main, or any branch with `ml-alpha-*` prefix
  - Runs: `scripts/determinism-check.sh` on the PR commit
  - Fails the PR if any divergence > rel-tol 1e-5
- Argo workflow template: add `pre-train-determinism-check` step to `alpha-rl-template.yaml` that runs determinism-check on the cluster GPU before submitting the actual training run

## §6. Falsification criteria

PASS criteria (ALL must hold):
- `determinism-check.sh` passes (exit 0) for 5 different seeds (42, 43, 44, 45, 46) on RTX 3050
- One b=1024 cluster smoke re-run produces eval_summary.json `total_pnl_usd` matching to ±$1 (true bit-exact at scalar level)
- `determinism_invariants.rs` test 1 passes (twin run match)
- `determinism_invariants.rs` test 3 passes (golden checksum match) — proves no regression

FAIL criteria:
- Any test above fails after applying §2 fixes
- If failure persists after auditing all 6 candidate causes (mamba2, GEMM, cuDNN, custom kernels, Adam, PER) → escalate to NVIDIA support OR accept multi-seed averaging as the workaround (5× cluster cost per verdict, but trustworthy)

## §7. Sequencing

Phase 1 ships first (the diagnostic):

1. **§1.1** Write `rl_deterministic_checksum.cu` + register in `build.rs` (~2 hours)
2. **§1.2** Instrument 15 component checksums into `build_diag_value` (~3 hours)
3. **§1.3** Modify `determinism-check.sh` to compare checksums leaf-by-leaf, report first-divergent (~1 hour)
4. **§3.1** Local determinism gate output (~1 hour)
5. **Run** the diagnostic: identify first-divergent component (~30 min)

**Phase 1 total: ~8 hours / 1 day.**

Phase 2 fixes depend on Phase 1 results (apply the relevant §2.X). Cost: ~1-3 days depending on which component is the culprit.

Phase 3 validation (§3.2, §3.3, §3.4) + Phase 4 documentation (§4): ~1 day.

**Total: ~3-5 days to ship the foundation.** This is the prerequisite for multi-head policy, R-multiple reward, and all subsequent regime-invariance work.

## §8. Out of scope (defer)

- Multi-GPU determinism (foxhunt is single-GPU per process; defer until production)
- Cross-hardware determinism (RTX 3050 vs L40S vs H100 may produce different exact bits even with all fixes; rel-tol 1e-5 should hold but bit-exact across hardware is not required)
- Cross-CUDA-version determinism (pin CUDA 12.4 per current setup; if upgraded, re-audit)
- Floating-point order across compiler versions (pin rustc + cudart versions in build env)

## §9. Why this comes BEFORE multi-head policy

Multi-head policy adds 3× the policy head parameters + a gating network + new aux loss term. ANY of these new components could introduce its own non-determinism (more GEMM calls, more block-tree-reduces, possibly more PER samples).

If we ship multi-head on a non-deterministic foundation:
- Multi-head eval shows +$3M → we cheer, but it's a lucky seed
- Multi-head eval shows -$5M → we revert, but it was an unlucky seed
- Multiple seeds → 5× cluster cost
- We never know if the architecture worked

If we ship determinism FIRST:
- Multi-head eval shows +$3M → reproducible, trust it, build on it
- Multi-head eval shows -$5M → reproducible, abandon it, learn from it
- Single seed per iteration → fast cluster cadence
- **Every iteration produces real signal**

The 3-5 day investment in determinism pays back on the FIRST controller iteration. Over the next 10 iterations, it saves 50% of cluster compute (no need for multi-seed averaging) and 100% of the "is this real or noise?" confusion that drove today's going-in-circles.

## §10. Memory pearls to create after Phase 1 lands

- `pearl_determinism_root_cause` — documents which component(s) were the actual culprit + fix applied
- `pearl_determinism_speed_cost` — documents the actual measured throughput cost on H100 + L40S + RTX 3050
- `pearl_determinism_contract` — the rule going forward: every commit passes determinism gate, no exceptions
