Files
foxhunt/crates/ml/tests
jgrusewski 13ce783853 test(sp20): post-fix runtime diagnostic — read-back accessors + producer test scaffold
Adds infrastructure for verifying the SP20 Phase 2 Task 2.2-fix
(`is_win_per_env`) producer side at runtime. The consumer-side oracle
test passes; the producer-side wire-up has not been independently
verified end-to-end. Production HEALTH_DIAG observed `wr_ema=0.0000`
across all training epochs even after `64bbbe418` landed, with
`alpha_ema` evolving normally — strongly suggesting either (a) all
profitable trade closes had `hold_time == 0` so the producer write
didn't fire, or (b) the producer write fired but `(segment_return >
0.0f)` evaluated to false in the production data.

Code-inspection audit (this commit, summarized in audit-doc):

- Verified 81 kernel args (launcher) match 81 kernel signature args
  for `experience_env_step`. `is_win_per_env` at position 81;
  `alpha_per_env` at position 78. cudarc `PushKernelArg<&mut
  CudaSlice<T>>` impl pushes `&arg.cu_device_ptr` (stable address
  for the lifetime of the chained statement).
- Verified buffer allocation at construction (alloc_zeros), pub(crate)
  field, FoldReset entry + dispatch arm, kernel-arg pass at both
  experience_env_step and sp20_aggregate_inputs launch sites.
- Verified producer site — both alpha + is_win writes inside the SAME
  `if (segment_complete && segment_hold_time > 0.0f)` block at
  experience_kernels.cu:3196, both NULL-guarded, race-free.
- Verified consumer site — aggregate kernel reads `is_win_per_env[env]`
  + `alpha_per_env[env]` side-by-side inside `if (tc != 0)`.
- Verified stream ordering — both kernels run on `self.stream` in the
  same `for t in 0..timesteps` loop; experience kernel at line 6069,
  aggregate at 6577. Stream-implicit producer→consumer ordering.

**No code-level bug surfaced by inspection.** The `is_win_per_env`
write is structurally identical to the `alpha_per_env` write 14
lines earlier in the same gating block.

Files modified:

- `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` —
  `read_is_win_per_env_for_test()` and `read_alpha_per_env_for_test()`
  pub fns. Mirror `read_epoch_dsr_state()` pattern: dtoh memcpy via
  the collector's stream, returns Vec<i32> / Vec<f32>. Cost-free in
  production (only called from tests) and benign for invariants.
- `crates/ml/tests/sp20_is_win_producer_test.rs` — production-path
  producer test scaffold. Builds 6-float-per-bar synthetic OHLCV
  matching `experience_env_step`'s tgt[0..6] contract, runs
  `collect_experiences_gpu`, reads back both buffers, asserts the
  three structural invariants:
    1. alpha_per_env has at least one non-zero slot (precondition —
       the synthetic upward-trend data must produce ≥ 1 trade close).
    2. is_win_per_env has at least one slot == 1 (regression-pin for
       the Task 2.2-fix producer wire-up).
    3. Co-occurrence: every env where alpha > 0 must have is_win == 1
       (R_event > 0 ⇒ segment_return > 0 ⇒ is_win = 1; structural
       guarantee from the kernel's two writes at the same gate).
  Currently `#[ignore]` because the collector's cuBLAS forward chain
  requires a real `trainer_params_ptr` (NUM_WEIGHT_TENSORS=163
  weights). Activation requires either trainer-params plumbing in the
  test scaffold OR moving the test inline as `#[cfg(test)]` in
  gpu_experience_collector.rs.
- `docs/dqn-wire-up-audit.md` — audit-doc entry per Invariant 7.
  Documents the inspection scope, the unverified producer-side
  pre-conditions, and the open-issue framing for the wr_ema=0.0000
  discrepancy.

Verification:
- SQLX_OFFLINE=true cargo check -p ml --tests passes
- 9 state_reset_registry tests pass (incl.
  sp20_is_win_per_env_registered_fold_reset)
- The new test file compiles cleanly under `--features cuda`

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 02:22:15 +02:00
..