gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(sp20): post-fix runtime diagnostic — read-back accessors + producer test scaffold

Browse Source 
Adds infrastructure for verifying the SP20 Phase 2 Task 2.2-fix
(`is_win_per_env`) producer side at runtime. The consumer-side oracle
test passes; the producer-side wire-up has not been independently
verified end-to-end. Production HEALTH_DIAG observed `wr_ema=0.0000`
across all training epochs even after `64bbbe418` landed, with
`alpha_ema` evolving normally — strongly suggesting either (a) all
profitable trade closes had `hold_time == 0` so the producer write
didn't fire, or (b) the producer write fired but `(segment_return >
0.0f)` evaluated to false in the production data.

Code-inspection audit (this commit, summarized in audit-doc):

- Verified 81 kernel args (launcher) match 81 kernel signature args
  for `experience_env_step`. `is_win_per_env` at position 81;
  `alpha_per_env` at position 78. cudarc `PushKernelArg<&mut
  CudaSlice<T>>` impl pushes `&arg.cu_device_ptr` (stable address
  for the lifetime of the chained statement).
- Verified buffer allocation at construction (alloc_zeros), pub(crate)
  field, FoldReset entry + dispatch arm, kernel-arg pass at both
  experience_env_step and sp20_aggregate_inputs launch sites.
- Verified producer site — both alpha + is_win writes inside the SAME
  `if (segment_complete && segment_hold_time > 0.0f)` block at
  experience_kernels.cu:3196, both NULL-guarded, race-free.
- Verified consumer site — aggregate kernel reads `is_win_per_env[env]`
  + `alpha_per_env[env]` side-by-side inside `if (tc != 0)`.
- Verified stream ordering — both kernels run on `self.stream` in the
  same `for t in 0..timesteps` loop; experience kernel at line 6069,
  aggregate at 6577. Stream-implicit producer→consumer ordering.

**No code-level bug surfaced by inspection.** The `is_win_per_env`
write is structurally identical to the `alpha_per_env` write 14
lines earlier in the same gating block.

Files modified:

- `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` —
  `read_is_win_per_env_for_test()` and `read_alpha_per_env_for_test()`
  pub fns. Mirror `read_epoch_dsr_state()` pattern: dtoh memcpy via
  the collector's stream, returns Vec<i32> / Vec<f32>. Cost-free in
  production (only called from tests) and benign for invariants.
- `crates/ml/tests/sp20_is_win_producer_test.rs` — production-path
  producer test scaffold. Builds 6-float-per-bar synthetic OHLCV
  matching `experience_env_step`'s tgt[0..6] contract, runs
  `collect_experiences_gpu`, reads back both buffers, asserts the
  three structural invariants:
    1. alpha_per_env has at least one non-zero slot (precondition —
       the synthetic upward-trend data must produce ≥ 1 trade close).
    2. is_win_per_env has at least one slot == 1 (regression-pin for
       the Task 2.2-fix producer wire-up).
    3. Co-occurrence: every env where alpha > 0 must have is_win == 1
       (R_event > 0 ⇒ segment_return > 0 ⇒ is_win = 1; structural
       guarantee from the kernel's two writes at the same gate).
  Currently `#[ignore]` because the collector's cuBLAS forward chain
  requires a real `trainer_params_ptr` (NUM_WEIGHT_TENSORS=163
  weights). Activation requires either trainer-params plumbing in the
  test scaffold OR moving the test inline as `#[cfg(test)]` in
  gpu_experience_collector.rs.
- `docs/dqn-wire-up-audit.md` — audit-doc entry per Invariant 7.
  Documents the inspection scope, the unverified producer-side
  pre-conditions, and the open-issue framing for the wr_ema=0.0000
  discrepancy.

Verification:
- SQLX_OFFLINE=true cargo check -p ml --tests passes
- 9 state_reset_registry tests pass (incl.
  sp20_is_win_per_env_registered_fold_reset)
- The new test file compiles cleanly under `--features cuda`

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-10 02:22:15 +02:00
parent 64bbbe4181
commit 13ce783853
3 changed files with 421 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
View File

@@ -6817,6 +6817,41 @@ impl GpuExperienceCollector {
/// Number of timesteps per episode in the pre-allocated output buffers.
pub fn alloc_timesteps(&self) -> usize { self.alloc_timesteps }
/// SP20 Phase 2 Task 2.2-fix (2026-05-10) — read-back of the per-env
/// `is_win_per_env` `[alloc_episodes]` i32 trade-close win flag scratch.
/// Producer site is the segment_complete branch in `experience_env_step`
/// (writes `(segment_return > 0.0f) ? 1 : 0` per close); consumer is
/// `sp20_aggregate_inputs_kernel` for the WR_EMA wins_count reduction.
/// This getter exists for the production-path diagnostic test that
/// validates the producer side end-to-end (the existing kernel oracle
/// test only validates the consumer side).
pub fn read_is_win_per_env_for_test(&self) -> Result<Vec<i32>, MLError> {
let mut host = vec![0_i32; self.alloc_episodes];
self.stream
.memcpy_dtoh(&self.is_win_per_env, &mut host)
.map_err(|e| MLError::ModelError(format!(
"read_is_win_per_env_for_test memcpy_dtoh: {e}"
)))?;
Ok(host)
}
/// SP20 Phase 2 Task 2.2-fix (2026-05-10) — read-back of the per-env
/// `alpha_per_env` `[alloc_episodes]` f32 trade-close alpha scratch.
/// Companion to `read_is_win_per_env_for_test` — both are written at
/// the same segment_complete site so the test asserts the producer
/// fires by checking that BOTH buffers received writes (alpha as f32
/// non-zero, is_win as i32 with at least one slot == 1 if any trade
/// closed profitably).
pub fn read_alpha_per_env_for_test(&self) -> Result<Vec<f32>, MLError> {
let mut host = vec![0_f32; self.alloc_episodes];
self.stream
.memcpy_dtoh(&self.alpha_per_env, &mut host)
.map_err(|e| MLError::ModelError(format!(
"read_alpha_per_env_for_test memcpy_dtoh: {e}"
)))?;
Ok(host)
}
/// Raw device pointer to the portfolio_state buffer [alloc_episodes * PS_STRIDE].
/// Used by the kelly_cap_update kernel (Plan 1 Task 11) to aggregate Kelly
/// win/loss stats across environments at epoch boundaries without a GPU→CPU roundtrip.

289
crates/ml/tests/sp20_is_win_producer_test.rs Normal file
View File

@@ -0,0 +1,289 @@
//! SP20 Phase 2 Task 2.2-fix (2026-05-10) — production-path producer
//! diagnostic for the `is_win_per_env` segment-level win-flag buffer.
//!
//! The aggregate-kernel oracle test
//! (`wr_ema_uses_segment_pnl_via_is_win_per_env_predicate` in
//! `sp20_aggregate_inputs_test`) validates the **consumer** side: when the
//! buffer is pre-populated with synthetic 1s, the kernel correctly
//! aggregates them into `wins_count`. That test passes.
//!
//! This test validates the **producer** side: when `experience_env_step`
//! runs the production launch path with synthetic OHLCV that forces
//! profitable trade closes, the kernel MUST write `1` into
//! `is_win_per_env[i]` at the segment_complete branch.
//!
//! Bug class targeted: production HEALTH_DIAG observed `wr_ema=0.0000`
//! across all training epochs even AFTER the kernel-body fix landed in
//! commit `64bbbe418`. Two runs (pre-fix vs post-fix) produced near
//! bit-identical EMA values for `alpha_ema` (which is written at the
//! SAME site), strongly suggesting the `is_win_per_env` write site is
//! NOT firing in production despite passing the consumer-side oracle
//! test. Companion test pins a parallel invariant: the `alpha_per_env`
//! producer DOES fire (alpha_ema evolves with non-zero values in
//! production), so any test where alpha_per_env got writes but
//! is_win_per_env did not would be a smoking gun for the bug.
//!
//! Run on a GPU host:
//!
//! SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 \
//! cargo test -p ml --test sp20_is_win_producer_test --features cuda \
//! -- --ignored --nocapture
#![allow(clippy::tests_outside_test_module)]
#[cfg(feature = "cuda")]
#[allow(unsafe_code)] // CUDA kernel launch + mapped-pinned memory.
mod gpu {
use std::sync::Arc;
use ml::cuda_pipeline::gpu_experience_collector::{
ExperienceCollectorConfig, GpuExperienceCollector,
};
use ml::cuda_pipeline::mapped_pinned::MappedF32Buffer;
use ml::cuda_pipeline::shared_cublas_handle::PerStreamCublasHandles;
use ml_core::device::MlDevice;
type CudaStream = cudarc::driver::CudaStream;
fn try_cuda() -> Option<Arc<CudaStream>> {
match MlDevice::cuda(0) {
Ok(dev) => Some(dev.cuda_stream().expect("CUDA stream required").clone()),
Err(_) => None,
}
}
/// Build a 6-float-per-bar target buffer matching
/// `experience_kernels.cu::experience_env_step`'s contract:
/// `[preproc_close, preproc_next, raw_close, raw_next, raw_open, mid_open]`
///
/// `prices[i]` is used as the raw close at bar i; raw_open is set to
/// `prices[i]` (intra-bar move = 0 — fine for synthetic test).
/// preproc_* are set to 0.0 (the Q-network reads them but our test
/// doesn't depend on Q-values).
fn build_targets(prices: &[f32]) -> MappedF32Buffer {
let n = prices.len();
let mut data = vec![0.0_f32; n * 6];
for i in 0..n {
let p = prices[i];
let p_next = if i + 1 < n { prices[i + 1] } else { p };
data[i * 6] = 0.0; // preproc_close
data[i * 6 + 1] = 0.0; // preproc_next
data[i * 6 + 2] = p; // raw_close
data[i * 6 + 3] = p_next; // raw_next
data[i * 6 + 4] = p; // raw_open
data[i * 6 + 5] = p; // mid_open
}
let buf = unsafe { MappedF32Buffer::new(n * 6) }.expect("alloc targets");
buf.write_from_slice(&data);
buf
}
/// Build a feature buffer where every bar has features that yield a
/// strong "long" signal (positive market values across all 51 dims).
/// Combined with `epsilon=1.0` random action selection, this gives
/// enough action diversity to trigger trade openings + closings.
fn build_market_features(n_bars: usize) -> MappedF32Buffer {
const MARKET_DIM: usize = 51;
let mut data = vec![0.0_f32; n_bars * MARKET_DIM];
for i in 0..n_bars {
for j in 0..MARKET_DIM {
// Slightly-positive features so the network's untrained
// Q-values lean toward longs/shorts roughly equally with
// epsilon=1 random override.
data[i * MARKET_DIM + j] = (i as f32 * 0.001).sin() * 0.5;
}
}
let buf = unsafe { MappedF32Buffer::new(n_bars * MARKET_DIM) }
.expect("alloc market");
buf.write_from_slice(&data);
buf
}
/// SP20 Phase 2 Task 2.2-fix — production-path producer diagnostic.
///
/// Constructs a synthetic monotonic upward price series so that any
/// long position with positive hold time closes profitably
/// (`segment_return > 0`). With `epsilon=1.0` (full random action),
/// the experience collector explores enough action space to open AND
/// close trades within `timesteps_per_episode=200` bars.
///
/// Asserts:
/// 1. `alpha_per_env` has at least one non-zero slot — the producer
/// site at the segment_complete branch fires for at least one env
/// (the existing reference behavior — alpha_ema evolves in
/// production).
/// 2. `is_win_per_env` has at least one slot == 1 — the SP20 Phase 2
/// Task 2.2-fix write site (`is_win_per_env[i] = (segment_return >
/// 0.0f) ? 1 : 0`) fires for at least one profitable trade close.
///
/// **Without the fix**: assertion 2 FAILS even though assertion 1
/// passes. With the fix wired correctly: BOTH pass.
///
/// **The two assertions are at the SAME site in the kernel** (both
/// inside `if (segment_complete && segment_hold_time > 0.0f) {...}`),
/// so the alpha-passes-but-is-win-fails outcome is the structural
/// signature of the kernel-arg-threading or buffer-wireup bug
/// described in the audit-doc 2026-05-10 entry.
#[test]
#[ignore = "requires GPU"]
fn is_win_per_env_producer_fires_at_segment_complete() {
let Some(stream) = try_cuda() else {
eprintln!("SKIP: CUDA not available");
return;
};
// Strong upward trend — every long position with hold > 0 closes
// profitably (segment_return > 0). This forces is_win_per_env=1
// for at least one env if the producer fires.
let n_bars = 500_usize;
let mut prices = Vec::with_capacity(n_bars);
for i in 0..n_bars {
// Linear ramp from 100 to 200 — robust >0 segment_return on
// any long position closed after at least one bar.
prices.push(100.0 + (i as f32) * 0.2);
}
let market_buf = build_market_features(n_bars);
let target_buf = build_targets(&prices);
let shared_cublas = Arc::new(
PerStreamCublasHandles::new(&stream).expect("cublas handles"),
);
// Run multiple episodes to maximize chance of trade close
// happening in at least one env. alloc_episodes >= n_episodes is
// required by the collector contract. Match the existing smoke
// test's (4, 50) allocation pattern that's known to work without
// explicit weight wiring.
let n_episodes = 4_usize;
let timesteps = 50_i32;
let mut collector = GpuExperienceCollector::new(
stream.clone(),
shared_cublas,
0, // bottleneck_dim
51, // market_dim
100_000.0,
0.01, // tx_cost_multiplier
0.05, // max_position
(64, 64, 32, 32),
(51, 51),
n_episodes, // alloc_episodes
timesteps as usize, // alloc_timesteps
0.0, // curiosity_weight
)
.expect("create collector");
let episode_starts: Vec<i32> = (0..n_episodes)
.map(|i| (i * (n_bars / n_episodes)) as i32)
.collect();
let mut exp_config = ExperienceCollectorConfig::default();
exp_config.n_episodes = n_episodes as i32;
exp_config.timesteps_per_episode = timesteps;
exp_config.total_bars = n_bars as i32;
exp_config.episode_length = timesteps;
exp_config.epsilon = 1.0; // full random — maximizes trade diversity
exp_config.gamma = 0.99;
let _batch = collector
.collect_experiences_gpu(&market_buf, &target_buf, &episode_starts, &exp_config)
.expect("collect_experiences_gpu");
// ── Read back per-env producer buffers ──────────────────────────
let alpha = collector
.read_alpha_per_env_for_test()
.expect("read alpha_per_env");
let is_win = collector
.read_is_win_per_env_for_test()
.expect("read is_win_per_env");
// Assertion 1 — alpha producer fires (reference behavior).
let alpha_nonzero = alpha.iter().any(|&v| v != 0.0);
let alpha_writes = alpha.iter().filter(|&&v| v != 0.0).count();
assert!(
alpha_nonzero,
"PRECONDITION FAILED — alpha_per_env has zero non-zero slots \
out of {} after {} episodes × {} steps. The synthetic \
upward-trend data must produce at least one trade close \
with hold_time > 0 (alpha=R_event != 0). If this assertion \
fails, the test scaffold is broken (no trade closes \
happened); fix the test, not the production code.",
alpha.len(),
n_episodes,
timesteps,
);
eprintln!(
"alpha_per_env: {}/{} slots non-zero, sum={:.4}, max={:.4}",
alpha_writes,
alpha.len(),
alpha.iter().sum::<f32>(),
alpha.iter().cloned().fold(f32::NEG_INFINITY, f32::max),
);
// Assertion 2 — is_win producer fires (SP20 Phase 2 Task 2.2-fix).
let is_win_count = is_win.iter().filter(|&&v| v == 1).count();
let is_win_max: i32 = is_win.iter().copied().max().unwrap_or(0);
eprintln!(
"is_win_per_env: {} slots == 1 (out of {}), max={}",
is_win_count,
is_win.len(),
is_win_max,
);
assert!(
is_win_max > 0,
"SP20 Phase 2 Task 2.2-fix REGRESSION — is_win_per_env has \
ZERO slots set to 1 across {} alloc_episodes, but \
alpha_per_env has {} non-zero slots. Both buffers are \
written at the SAME segment_complete site in \
experience_env_step (line 3322 alpha, line 3337 is_win). \
If alpha was written and is_win was not, the kernel-arg \
threading for is_win_per_env is broken — likely the \
pointer is being passed as NULL despite the launcher \
alloc'ing the buffer. This is the production WR_EMA=0.0000 \
pinning bug — the fix in commit `64bbbe418` did NOT \
actually fire at runtime.",
is_win.len(),
alpha_writes,
);
// Tighter contract: every slot where alpha_per_env has a
// **positive** value (R_event > 0 ⇒ segment_return > 0) MUST
// have is_win_per_env == 1 at the same env index. This is the
// structural co-occurrence the kernel guarantees:
//
// alpha[i] = R_event ∈ {-loss_cap, -0.5, 0, +0.5, +1.0}
// is_win[i] = (segment_return > 0) ? 1 : 0
//
// R_event > 0 implies segment_return > 0 implies is_win[i] = 1.
//
// Skip slot 0 (first-write may not yet be set if env didn't
// close), check ANY env where alpha > 0 has is_win = 1.
let positive_alpha_slots: Vec<usize> = alpha
.iter()
.enumerate()
.filter_map(|(i, &v)| if v > 0.0 { Some(i) } else { None })
.collect();
if !positive_alpha_slots.is_empty() {
let mismatched: Vec<usize> = positive_alpha_slots
.iter()
.copied()
.filter(|&i| is_win[i] == 0)
.collect();
assert!(
mismatched.is_empty(),
"SP20 Phase 2 Task 2.2-fix CO-OCCURRENCE VIOLATION — \
envs with alpha_per_env > 0 (R_event positive ⇒ \
segment_return > 0) MUST have is_win_per_env == 1. \
Found {} envs where alpha > 0 but is_win == 0: {:?}. \
This is the structural smoking gun for the kernel-arg \
wiring bug (the producer site writes both fields in \
succession at the same gating; if one fires the other \
must too).",
mismatched.len(),
mismatched,
);
}
}
}

97
docs/dqn-wire-up-audit.md
View File

@@ -2,6 +2,103 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
## 2026-05-10 — SP20 Phase 2 Task 2.2-fix: post-fix runtime diagnostic infrastructure (read-back + producer test scaffold)
Adds infrastructure for verifying the SP20 Phase 2 Task 2.2-fix
(`is_win_per_env`) producer side at runtime. The consumer-side
`sp20_aggregate_inputs_kernel` oracle test
(`wr_ema_uses_segment_pnl_via_is_win_per_env_predicate`) passes — when
the buffer is pre-populated with synthetic 1s, the kernel correctly
aggregates them into `wins_count`. **The producer-side wire-up has
not been independently verified** — runtime evidence
(production HEALTH_DIAG observation that `wr_ema=0.0000` across all
training epochs even after `64bbbe418` landed, with `alpha_ema`
evolving normally) suggests the producer write at
`experience_kernels.cu::3337` may not be firing in production despite
all visible code paths appearing correctly wired.
### Code-inspection audit (this commit)
Verified:
1. Kernel signature for `experience_env_step` — 81 args, matching the
Rust launcher's 81 `.arg(...)` calls. `is_win_per_env` at position
81; `alpha_per_env` at position 78; both pass through cudarc's
`PushKernelArg<&mut CudaSlice<T>>` impl which pushes
`&arg.cu_device_ptr` (a stable address for the lifetime of the
chained statement).
2. Buffer allocation — `alpha_per_env: CudaSlice<f32> [alloc_episodes]`
and `is_win_per_env: CudaSlice<i32> [alloc_episodes]` — both
`alloc_zeros`, both pub(crate) on `GpuExperienceCollector`.
3. Producer site — both writes inside the SAME
`if (segment_complete && segment_hold_time > 0.0f)` block at
`experience_kernels.cu:3196`, both NULL-guarded, race-free
(per-thread `[i]` writes), 14 lines apart.
4. Consumer site — aggregate kernel reads `is_win_per_env[env]` and
`alpha_per_env[env]` side-by-side inside the SAME
`if (tc != 0)` block.
5. State reset — `is_win_per_env` registered FoldReset, dispatch arm
present in `training_loop.rs::9572`. `every_fold_and_soft_reset_
entry_has_dispatch_arm` test passes.
6. Stream ordering — both kernels run on `self.stream` in the same
`for t in 0..timesteps` loop; experience kernel launches at line
6069, aggregate kernel at line 6577. Stream-implicit
producer→consumer ordering.
**No code-level bug surfaced by inspection.** The `is_win_per_env`
write site is structurally identical to the working `alpha_per_env`
write site.
### Files modified
| File | Status | Purpose |
|------|--------|---------|
| `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` | +2 pub fns | `read_is_win_per_env_for_test` + `read_alpha_per_env_for_test` device→host read-back accessors. dtoh memcpy gated behind public methods so future tests + production diagnostics can verify per-env producer firing without leaking field internals. |
| `crates/ml/tests/sp20_is_win_producer_test.rs` | +scaffold | Production-path producer test scaffold: builds 6-float-per-bar synthetic OHLCV, runs `collect_experiences_gpu`, reads back both per-env buffers. **Currently `#[ignore]`** because the collector's cuBLAS forward chain requires a real `trainer_params_ptr` (NUM_WEIGHT_TENSORS=163 weights) which the standalone test doesn't yet wire. The scaffold structure documents the intended invariants (alpha non-zero ⇒ is_win has at least one `1`; per-env alpha > 0 ⇒ same env's is_win == 1) for the future end-to-end verification. |
| `docs/dqn-wire-up-audit.md` | +entry | This entry |
### Behavioural pre-conditions (post-fix)
If the SP20 Phase 2 Task 2.2-fix is firing correctly, the following
invariants hold (they are currently **NOT** verified end-to-end —
only consumer-side and code-inspection):
1. After any rollout step where at least one env had a profitable
trade close (segment_pnl > 0) AND held the position for ≥ 1 bar:
- `alpha_per_env[i]` for that env is in {+0.5, +1.0} (R_event from
`sp20_compute_event_reward` 4-quadrant table).
- `is_win_per_env[i]` for that env is `1`.
2. The aggregate kernel's `wins_count` reduction over closed envs is
strictly equal to `sum(is_win_per_env[env] != 0 ; trade_close_
per_env[env*L + t] != 0)`.
3. The EMAs kernel's `WR_EMA` slot evolves with non-zero values when
`EmaInputs.is_close == 1` AND `2 * wins_count >= closed_count` for
the rollout step.
### Outstanding verification
The bit-similar-but-not-identical evidence (production runs:
alpha_ema=0.0555 → 0.0195 → 0.0143 (BUGGY) vs 0.0555 → 0.0194 →
0.0143 (FIX) — diverging at the 4th decimal at epoch 1 — implies the
kernel-arg threading is running differently in the FIX branch (extra
work / register pressure shifts float ordering). However, `wr_ema=
0.0000` in BOTH runs implies that EITHER:
a) The producer write at `experience_kernels.cu:3337` is firing but
`(segment_return > 0.0f)` evaluates to false EVERY time it
evaluates (no profitable trade closes happen with `hold_time > 0`
in the production data — possible if the policy mostly executes
immediate-reversal trades or the policy collapses to
pre-segment-close exits).
b) The producer write is somehow not firing despite the apparently-
correct code path.
The producer test scaffold landed in this commit will verify
hypothesis (a) vs (b) once the trainer-params plumbing is wired (or
moved to inline `#[cfg(test)]` in `gpu_experience_collector.rs`
with full visibility). Until that lands, the discrepancy remains an
**open issue** flagged via this audit-doc entry.
## 2026-05-10 — SP20 Phase 2 Task 2.2-fix: WR_EMA pinning bug — fix commit (kernel body + producer + buffer + reset)
Closes the fix gap left by the failing-test commit. The aggregation