gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(dqn): #212 — intent-side magnitude distribution HEALTH_DIAG metric

Browse Source 
Adds intent_dist_q/h/f alongside eval_dist_q/h/f to separate policy-
learning quality from Kelly-enforcement reality. Per memory pearl
project_magnitude_eval_collapse_kelly_capped.md: EVAL_DIST Quarter
dominance is a downstream artefact of Kelly cap × warmup_floor ×
safety_multiplier math, NOT a policy-learning failure. The intent
metric exposes the policy's pre-Kelly-cap chosen mag bucket so
operators can distinguish "policy isn't learning Full" from "Kelly
cap is suppressing Full" — the latter being an operationally-correct
state during cold-start positions.

Pure observability: no Kelly-math tweaks, no new tuned constants, no
reward-bias mechanisms (all explicitly forbidden by the memory pearl).
The intent_mag bucket comes straight from the factored action's
mag_idx (0/1/2) which already maps 1:1 to Quarter/Half/Full buckets.
The pre-cap mag_idx is already captured by the action-select kernel
into intent_mag_buf and exposed via the trainer's pre-existing
last_eval_intent_magnitude_dist host field — this commit only wires
that signal into HEALTH_DIAG.

Wired through (one coordinated commit per feedback_no_partial_refactor):
  - health_diag.rs: HealthDiagSnapshot gains intent_dist_q/h/f fields
    adjacent to eval_dist_*; size assertion bumped 147 → 150 fields.
  - health_diag_kernel.cu: 3 new WORD_INTENT_DIST_* slots at [77..80);
    every downstream WORD_* shifted by +3 in lockstep; WORD_TOTAL +
    static_assert bumped 147 → 150. (The slots are reserved identically
    to WORD_EVAL_DIST_* — both currently inert; HEALTH_DIAG GPU port
    Phases 2/3 will populate them in lockstep with their sibling slots.)
  - training_loop.rs: new adjacent tracing::info!() emitting
    "intent_dist [iq=... ih=... if=...]" from the existing host-side
    last_eval_intent_magnitude_dist field, immediately after the big
    HEALTH_DIAG line containing eval_dist [eq=... eh=... ef=...].
  - docs/dqn-wire-up-audit.md: new top-of-file entry per Invariant 7.

Behavior: zero change. Only adds observability.

Verification:
  - cargo check -p ml --offline: clean.
  - cargo test -p ml --lib --offline -- health_diag: 3/3 pass.
  - cargo test -p ml --test sp4_producer_unit_tests --release
    --ignored: 16/16 GPU tests pass.

Closes #212.
This commit is contained in:
jgrusewski
2026-05-01 17:11:19 +02:00
parent 6d0ac7beb3
commit d6eca73e52
4 changed files with 90 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
crates/ml/src/cuda_pipeline/health_diag.rs
View File

@@ -171,6 +171,20 @@ pub struct HealthDiagSnapshot {
pub eval_dist_h: f32,
pub eval_dist_f: f32,
// ── Intent per-magnitude distribution (3 floats) ─────────────────────────
/// Pure-policy magnitude bucket BEFORE Kelly cap clipping. Layout
/// `[Quarter, Half, Full]` in `[0, 1]`. Sourced from the trainer's
/// `last_eval_intent_magnitude_dist`, which reads `intent_mag_buf`
/// (the action-select kernel's pre-Kelly-cap mag_idx). Paired with
/// `eval_dist_*` so operators can distinguish "policy isn't learning
/// Full" (both metrics near zero) from "Kelly cap is suppressing
/// Full" (intent_dist_f ≫ eval_dist_f). Per memory pearl
/// `project_magnitude_eval_collapse_kelly_capped.md`: pure observability,
/// the Kelly cap math itself stays unchanged.
pub intent_dist_q: f32,
pub intent_dist_h: f32,
pub intent_dist_f: f32,
// ── Reward contributions (4 floats) ──────────────────────────────────────
pub reward_contrib_popart: f32,
pub reward_contrib_cf: f32,
@@ -377,16 +391,16 @@ mod tests {
// 8 health + 8 effective + 7 novels + 2 diag + 1 gem
// + 10 mag + 4 grad_bwd + 5 grad_aux + 9 grad_trunk + 2 grad_abs
// + 6 trail + 6 magstats + 6 noisy + 3 eval_dist
// + 4 reward_contrib + 7 controller + 3 explore
// + 3 intent_dist + 4 reward_contrib + 7 controller + 3 explore
// + 16 val + 4 val_dir + 4 val_picked
// + 6 reward_split + 4 aux + 10 aux_moe
// + 12 action_counts
// = 8+8+7+2+1+10+4+5+9+2+6+6+6+3+4+7+3+16+4+4+6+4+10+12 = 147 fields
let expected_bytes = 147 * 4;
// = 8+8+7+2+1+10+4+5+9+2+6+6+6+3+3+4+7+3+16+4+4+6+4+10+12 = 150 fields
let expected_bytes = 150 * 4;
assert_eq!(
std::mem::size_of::<HealthDiagSnapshot>(),
expected_bytes,
"HealthDiagSnapshot must be {} bytes (147 × 4); update test if struct changes",
"HealthDiagSnapshot must be {} bytes (150 × 4); update test if struct changes",
expected_bytes,
);
}

94
crates/ml/src/cuda_pipeline/health_diag_kernel.cu
View File

@@ -137,61 +137,69 @@
#define WORD_EVAL_DIST_H 75
#define WORD_EVAL_DIST_F 76
/* Reward contributions (4 words, [77..81)). */
#define WORD_REWARD_CONTRIB_POPART 77
#define WORD_REWARD_CONTRIB_CF 78
#define WORD_REWARD_CONTRIB_TRAIL 79
#define WORD_REWARD_CONTRIB_MICRO 80
/* Intent dist (3 words, [77..80)). Pure-policy magnitude bucket BEFORE
* Kelly cap clipping; sibling of eval_dist for separating "policy isn't
* learning Full" from "Kelly cap is suppressing Full". Per memory pearl
* `project_magnitude_eval_collapse_kelly_capped.md`. */
#define WORD_INTENT_DIST_Q 77
#define WORD_INTENT_DIST_H 78
#define WORD_INTENT_DIST_F 79
/* Controller fire bools + frac (7 words, [81..88)). */
#define WORD_CTRL_FIRE_ANTI_LR 81 /* u32 */
#define WORD_CTRL_FIRE_TAU 82 /* u32 */
#define WORD_CTRL_FIRE_GAMMA 83 /* u32 */
#define WORD_CTRL_FIRE_CLIP 84 /* u32 */
#define WORD_CTRL_FIRE_CQL 85 /* u32 */
#define WORD_CTRL_FIRE_COST 86 /* u32 */
#define WORD_CTRL_FIRE_FRAC 87
/* Reward contributions (4 words, [80..84)). */
#define WORD_REWARD_CONTRIB_POPART 80
#define WORD_REWARD_CONTRIB_CF 81
#define WORD_REWARD_CONTRIB_TRAIL 82
#define WORD_REWARD_CONTRIB_MICRO 83
/* Explore (3 words, [88..91)). */
#define WORD_EXPLORE_ENT_MAG 88
#define WORD_EXPLORE_ENT_DIR 89
#define WORD_EXPLORE_SIGMA_MEAN 90
/* Controller fire bools + frac (7 words, [84..91)). */
#define WORD_CTRL_FIRE_ANTI_LR 84 /* u32 */
#define WORD_CTRL_FIRE_TAU 85 /* u32 */
#define WORD_CTRL_FIRE_GAMMA 86 /* u32 */
#define WORD_CTRL_FIRE_CLIP 87 /* u32 */
#define WORD_CTRL_FIRE_CQL 88 /* u32 */
#define WORD_CTRL_FIRE_COST 89 /* u32 */
#define WORD_CTRL_FIRE_FRAC 90
/* Validation block (16 words, [91..107)). */
#define WORD_VAL_BASE 91
/* Explore (3 words, [91..94)). */
#define WORD_EXPLORE_ENT_MAG 91
#define WORD_EXPLORE_ENT_DIR 92
#define WORD_EXPLORE_SIGMA_MEAN 93
/* Validation per-direction action distribution (4 words, [107..111)). */
#define WORD_VAL_DIR_DIST_BASE 107
/* Validation block (16 words, [94..110)). */
#define WORD_VAL_BASE 94
/* Validation per-direction picked distribution (4 words, [111..115)). */
#define WORD_VAL_PICKED_DIR_DIST_BASE 111
/* Validation per-direction action distribution (4 words, [110..114)). */
#define WORD_VAL_DIR_DIST_BASE 110
/* Reward split (6 words, [115..121)). */
#define WORD_REWARD_SPLIT_POPART 115
#define WORD_REWARD_SPLIT_CF 116
#define WORD_REWARD_SPLIT_TRAIL 117
#define WORD_REWARD_SPLIT_MICRO 118
#define WORD_REWARD_SPLIT_OPP_COST 119
#define WORD_REWARD_SPLIT_BONUS 120
/* Validation per-direction picked distribution (4 words, [114..118)). */
#define WORD_VAL_PICKED_DIR_DIST_BASE 114
/* Aux block (4 words, [121..125)). */
#define WORD_AUX_NEXT_BAR_MSE 121
#define WORD_AUX_REGIME_CE 122
#define WORD_AUX_WEIGHT 123
#define WORD_AUX_LABEL_SCALE 124
/* Reward split (6 words, [118..124)). */
#define WORD_REWARD_SPLIT_POPART 118
#define WORD_REWARD_SPLIT_CF 119
#define WORD_REWARD_SPLIT_TRAIL 120
#define WORD_REWARD_SPLIT_MICRO 121
#define WORD_REWARD_SPLIT_OPP_COST 122
#define WORD_REWARD_SPLIT_BONUS 123
/* Aux MoE (10 words, [125..135) — 8 utils + ent + λ_eff). */
#define WORD_AUX_MOE_UTIL_BASE 125
#define WORD_AUX_MOE_ENT 133
#define WORD_AUX_MOE_LAMBDA_EFF 134
/* Aux block (4 words, [124..128)). */
#define WORD_AUX_NEXT_BAR_MSE 124
#define WORD_AUX_REGIME_CE 125
#define WORD_AUX_WEIGHT 126
#define WORD_AUX_LABEL_SCALE 127
/* Action-count histogram (12 words, [135..147), all u32). */
#define WORD_ACTION_COUNTS_BASE 135
/* Aux MoE (10 words, [128..138) — 8 utils + ent + λ_eff). */
#define WORD_AUX_MOE_UTIL_BASE 128
#define WORD_AUX_MOE_ENT 136
#define WORD_AUX_MOE_LAMBDA_EFF 137
#define WORD_TOTAL 147
/* Action-count histogram (12 words, [138..150), all u32). */
#define WORD_ACTION_COUNTS_BASE 138
#define WORD_TOTAL 150
/* Pin the layout: matches `snapshot_size_is_stable` on the Rust side. */
static_assert(WORD_TOTAL == 147,
static_assert(WORD_TOTAL == 150,
"HealthDiagSnapshot layout drift — update kernel WORD_* and the "
"Rust struct in lockstep, then bump snapshot_size_is_stable.");

19
crates/ml/src/trainers/dqn/trainer/training_loop.rs
View File

@@ -3846,6 +3846,25 @@ impl DQNTrainer {
ent_mag, ent_dir, sigma_mean_overall,
);
// Task #212 — pure-policy magnitude intent distribution BEFORE
// Kelly cap clipping. Sourced from `last_eval_intent_magnitude_dist`
// (cached by `metrics.rs::compute_validation_loss` from
// `gpu_evaluator.read_eval_intent_magnitude_distribution`, which
// reads `intent_mag_buf` — the action-select kernel's pre-cap
// mag_idx). Paired with the `eval_dist [eq=...]` segment in the
// big info! line above so operators can distinguish "policy isn't
// learning Full" (both metrics near zero) from "Kelly cap is
// suppressing Full" (intent_dist_f ≫ eval_dist_f). Per memory
// pearl `project_magnitude_eval_collapse_kelly_capped.md`: pure
// observability — Kelly cap math itself is unchanged.
tracing::info!(
"HEALTH_DIAG[{}]: intent_dist [iq={:.3} ih={:.3} if={:.3}]",
epoch,
self.last_eval_intent_magnitude_dist[0],
self.last_eval_intent_magnitude_dist[1],
self.last_eval_intent_magnitude_dist[2],
);
// C.2 Plan 3 Task 1 (spec §4.C.2): reward_split HEALTH_DIAG line.
// Reads 6 ISV EMA slots updated by the GPU reward_component_ema kernel
// launched just above. CPU-side code only reads; GPU wrote the values.

2
docs/dqn-wire-up-audit.md
View File

File diff suppressed because one or more lines are too long