diag(dqn): val_picked_dir_dist — read raw Boltzmann picks pre-env_step

Adds a paired diagnostic to localise the eval-mode collapse mechanism.

The existing `val_dir_dist` line reads `actions_history_buf` which env_step
overwrites with the POST-physics `actual_dir` (signed position bucket). When
the policy picks Long but the Kelly cap zeroes target_position, the bar shows
up as `actual_dir=Flat` — indistinguishable in the existing diagnostic from a
case where the kernel itself produced a Flat pick.

Cluster runs `ddrpr` (without Kelly fix) and `txdz9` (with Kelly fix
2c97e0436) produced bit-identical val_dir_dist patterns — Boltzmann math
should give `P(best) ≤ 47.5%` and `P(any direction) ≥ 13.5%` for 4 actions
with `tau=q_range`, yet observed `P(Long) + P(Short) ≈ 0.0007`. One of
those bounds is being violated and the existing diagnostic can't tell which.

The new `val_picked_dir_dist` reads `chunked_actions_buf` — the kernel's
RAW Boltzmann action_idx output BEFORE env_step computes actual_dir.
Divergence between the two distributions answers the gating question:
  - both flat → kernel itself produces collapsed picks (Q-values degenerate)
  - val_picked diverse, val_dir_dist flat → env_step drains to Flat
                                            (Kelly / margin / trail stop)

Sample is the most recent chunk (~64 bars on 1-window val). Noisier than
the full-window post-physics histogram but enough for qualitative
comparison against Boltzmann theory bounds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-26 20:06:31 +02:00
parent 2c97e0436c
commit 89ece2e366
3 changed files with 77 additions and 0 deletions

View File

@@ -1164,6 +1164,50 @@ impl GpuBacktestEvaluator {
])
}
/// Read the per-direction histogram of the policy's RAW Boltzmann picks
/// from the most recent chunk (the last `chunk_len` bars of the eval
/// rollout). Reads `chunked_actions_buf` BEFORE env_step touches it —
/// `actions_history_buf` records the POST-physics `actual_dir`, so a
/// divergence between the two distributions localises whether the eval
/// collapse is upstream (kernel produces degenerate picks) or downstream
/// (env_step / Kelly cap drains active picks to Flat).
///
/// Returns `[short, hold, long, flat]` summing to 1.0 over the chunk.
/// `Ok([0.0; 4])` if `chunked_actions_buf` has not been allocated yet.
pub fn read_chunked_actions_direction_distribution(&self) -> Result<[f32; 4], MLError> {
let buf = match self.chunked_actions_buf.as_ref() {
Some(b) => b,
None => return Ok([0.0; 4]),
};
let len = buf.len();
let mut host = vec![0_i32; len];
self.stream
.memcpy_dtoh(buf, &mut host)
.map_err(|e| MLError::ModelError(format!("chunked_actions dtoh: {e}")))?;
let mut counts = [0_u64; 4];
let mut total = 0_u64;
for &a in &host {
if a < 0 {
continue;
}
let dir = (a / 27) as usize;
if dir < 4 {
counts[dir] += 1;
total += 1;
}
}
if total == 0 {
return Ok([0.0; 4]);
}
let t = total as f32;
Ok([
counts[0] as f32 / t,
counts[1] as f32 / t,
counts[2] as f32 / t,
counts[3] as f32 / t,
])
}
/// Reset mutable evaluation state: portfolio, done flags, step returns, actions history.
/// Must be called before each `evaluate_dqn_graphed` when reusing the evaluator
/// across epochs — otherwise done_flags=1 from the previous evaluation causes

View File

@@ -796,6 +796,24 @@ impl DQNTrainer {
"eval direction dist readback failed (non-fatal): {e}"
),
}
// Raw Boltzmann pick histogram from the most recent chunk's
// chunked_actions buffer — BEFORE env_step touches `actual_dir`.
// A divergence from `val_dir_dist` (post-physics) localises whether
// the eval collapse is upstream (kernel produces collapsed picks
// → both lines flat) or downstream (kernel diverse, env_step /
// Kelly cap drains active picks to Flat → only val_dir_dist flat).
// Sample is a single chunk (~64 bars on 1-window val), so noise
// is higher than the full-window val_dir_dist; useful for
// qualitative comparison against Boltzmann theory bounds.
match ev.read_chunked_actions_direction_distribution() {
Ok(rd) => tracing::info!(
"HEALTH_DIAG[{}]: val_picked_dir_dist [short={:.4} hold={:.4} long={:.4} flat={:.4}]",
epoch, rd[0], rd[1], rd[2], rd[3],
),
Err(e) => tracing::warn!(
"raw Boltzmann pick dist readback failed (non-fatal): {e}"
),
}
}
Ok(-val_sharpe)

View File

@@ -2,6 +2,21 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
`val_picked_dir_dist` HEALTH_DIAG diagnostic (2026-04-26): `gpu_backtest_evaluator.rs`
gains a `read_chunked_actions_direction_distribution()` reader that decodes the
per-direction histogram from `chunked_actions_buf` (the buffer experience_action_select
writes to BEFORE env_step computes `actual_dir`). Pairs with the existing
`val_dir_dist` reader (reads `actions_history_buf` AFTER env_step) — divergence
between the two localises whether eval collapse is upstream (kernel produces
degenerate Boltzmann picks → both lines flat) or downstream (kernel diverse,
env_step / Kelly cap drains active picks to Flat → only val_dir_dist flat).
Cluster runs ddrpr + txdz9 showed val_dir_dist ≈ 80% Flat / 20% Hold across
30+ epochs while Boltzmann math caps `P(best) ≤ 47.5%` and `P(any) ≥ 13.5%`
for 4 actions with `tau=q_range`; one of those bounds is being violated and
this diagnostic identifies which path. Sample is one chunk (~64 bars on
1-window val), noisier than the full-window post-physics histogram but
sufficient to disambiguate the gating mechanism.
Kelly cap warmup_floor health-coupled (2026-04-26): `trade_physics.cuh::kelly_position_cap`
previously computed `warmup_floor = clamp(conviction, 0, 1)`. At cold start
(maturity = 0) `effective_kelly = warmup_floor`, so a zero conviction collapsed