diag(dqn): val_picked_dir_dist — read raw Boltzmann picks pre-env_step
Adds a paired diagnostic to localise the eval-mode collapse mechanism.
The existing `val_dir_dist` line reads `actions_history_buf` which env_step
overwrites with the POST-physics `actual_dir` (signed position bucket). When
the policy picks Long but the Kelly cap zeroes target_position, the bar shows
up as `actual_dir=Flat` — indistinguishable in the existing diagnostic from a
case where the kernel itself produced a Flat pick.
Cluster runs `ddrpr` (without Kelly fix) and `txdz9` (with Kelly fix
2c97e0436) produced bit-identical val_dir_dist patterns — Boltzmann math
should give `P(best) ≤ 47.5%` and `P(any direction) ≥ 13.5%` for 4 actions
with `tau=q_range`, yet observed `P(Long) + P(Short) ≈ 0.0007`. One of
those bounds is being violated and the existing diagnostic can't tell which.
The new `val_picked_dir_dist` reads `chunked_actions_buf` — the kernel's
RAW Boltzmann action_idx output BEFORE env_step computes actual_dir.
Divergence between the two distributions answers the gating question:
- both flat → kernel itself produces collapsed picks (Q-values degenerate)
- val_picked diverse, val_dir_dist flat → env_step drains to Flat
(Kelly / margin / trail stop)
Sample is the most recent chunk (~64 bars on 1-window val). Noisier than
the full-window post-physics histogram but enough for qualitative
comparison against Boltzmann theory bounds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1164,6 +1164,50 @@ impl GpuBacktestEvaluator {
|
||||
])
|
||||
}
|
||||
|
||||
/// Read the per-direction histogram of the policy's RAW Boltzmann picks
|
||||
/// from the most recent chunk (the last `chunk_len` bars of the eval
|
||||
/// rollout). Reads `chunked_actions_buf` BEFORE env_step touches it —
|
||||
/// `actions_history_buf` records the POST-physics `actual_dir`, so a
|
||||
/// divergence between the two distributions localises whether the eval
|
||||
/// collapse is upstream (kernel produces degenerate picks) or downstream
|
||||
/// (env_step / Kelly cap drains active picks to Flat).
|
||||
///
|
||||
/// Returns `[short, hold, long, flat]` summing to 1.0 over the chunk.
|
||||
/// `Ok([0.0; 4])` if `chunked_actions_buf` has not been allocated yet.
|
||||
pub fn read_chunked_actions_direction_distribution(&self) -> Result<[f32; 4], MLError> {
|
||||
let buf = match self.chunked_actions_buf.as_ref() {
|
||||
Some(b) => b,
|
||||
None => return Ok([0.0; 4]),
|
||||
};
|
||||
let len = buf.len();
|
||||
let mut host = vec![0_i32; len];
|
||||
self.stream
|
||||
.memcpy_dtoh(buf, &mut host)
|
||||
.map_err(|e| MLError::ModelError(format!("chunked_actions dtoh: {e}")))?;
|
||||
let mut counts = [0_u64; 4];
|
||||
let mut total = 0_u64;
|
||||
for &a in &host {
|
||||
if a < 0 {
|
||||
continue;
|
||||
}
|
||||
let dir = (a / 27) as usize;
|
||||
if dir < 4 {
|
||||
counts[dir] += 1;
|
||||
total += 1;
|
||||
}
|
||||
}
|
||||
if total == 0 {
|
||||
return Ok([0.0; 4]);
|
||||
}
|
||||
let t = total as f32;
|
||||
Ok([
|
||||
counts[0] as f32 / t,
|
||||
counts[1] as f32 / t,
|
||||
counts[2] as f32 / t,
|
||||
counts[3] as f32 / t,
|
||||
])
|
||||
}
|
||||
|
||||
/// Reset mutable evaluation state: portfolio, done flags, step returns, actions history.
|
||||
/// Must be called before each `evaluate_dqn_graphed` when reusing the evaluator
|
||||
/// across epochs — otherwise done_flags=1 from the previous evaluation causes
|
||||
|
||||
@@ -796,6 +796,24 @@ impl DQNTrainer {
|
||||
"eval direction dist readback failed (non-fatal): {e}"
|
||||
),
|
||||
}
|
||||
// Raw Boltzmann pick histogram from the most recent chunk's
|
||||
// chunked_actions buffer — BEFORE env_step touches `actual_dir`.
|
||||
// A divergence from `val_dir_dist` (post-physics) localises whether
|
||||
// the eval collapse is upstream (kernel produces collapsed picks
|
||||
// → both lines flat) or downstream (kernel diverse, env_step /
|
||||
// Kelly cap drains active picks to Flat → only val_dir_dist flat).
|
||||
// Sample is a single chunk (~64 bars on 1-window val), so noise
|
||||
// is higher than the full-window val_dir_dist; useful for
|
||||
// qualitative comparison against Boltzmann theory bounds.
|
||||
match ev.read_chunked_actions_direction_distribution() {
|
||||
Ok(rd) => tracing::info!(
|
||||
"HEALTH_DIAG[{}]: val_picked_dir_dist [short={:.4} hold={:.4} long={:.4} flat={:.4}]",
|
||||
epoch, rd[0], rd[1], rd[2], rd[3],
|
||||
),
|
||||
Err(e) => tracing::warn!(
|
||||
"raw Boltzmann pick dist readback failed (non-fatal): {e}"
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
Ok(-val_sharpe)
|
||||
|
||||
@@ -2,6 +2,21 @@
|
||||
|
||||
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
|
||||
|
||||
`val_picked_dir_dist` HEALTH_DIAG diagnostic (2026-04-26): `gpu_backtest_evaluator.rs`
|
||||
gains a `read_chunked_actions_direction_distribution()` reader that decodes the
|
||||
per-direction histogram from `chunked_actions_buf` (the buffer experience_action_select
|
||||
writes to BEFORE env_step computes `actual_dir`). Pairs with the existing
|
||||
`val_dir_dist` reader (reads `actions_history_buf` AFTER env_step) — divergence
|
||||
between the two localises whether eval collapse is upstream (kernel produces
|
||||
degenerate Boltzmann picks → both lines flat) or downstream (kernel diverse,
|
||||
env_step / Kelly cap drains active picks to Flat → only val_dir_dist flat).
|
||||
Cluster runs ddrpr + txdz9 showed val_dir_dist ≈ 80% Flat / 20% Hold across
|
||||
30+ epochs while Boltzmann math caps `P(best) ≤ 47.5%` and `P(any) ≥ 13.5%`
|
||||
for 4 actions with `tau=q_range`; one of those bounds is being violated and
|
||||
this diagnostic identifies which path. Sample is one chunk (~64 bars on
|
||||
1-window val), noisier than the full-window post-physics histogram but
|
||||
sufficient to disambiguate the gating mechanism.
|
||||
|
||||
Kelly cap warmup_floor health-coupled (2026-04-26): `trade_physics.cuh::kelly_position_cap`
|
||||
previously computed `warmup_floor = clamp(conviction, 0, 1)`. At cold start
|
||||
(maturity = 0) `effective_kelly = warmup_floor`, so a zero conviction collapsed
|
||||
|
||||
Reference in New Issue
Block a user