diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 626fa749b..004a9bc99 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -17251,3 +17251,43 @@ If the trainer-side mechanism MOVES WR (training-time, not eval-time), then Phas - D4 SUPERSEDED: eval-side `compute_expected_q` already inherits atom-shift via `QValueProvider::compute_q_and_b_logits_to` → `replay_forward_for_q_values`. No separate eval-side launcher needed. This refines the runbook's original Phase D estimate from "~25-35 hr" to "~3-4 hr" engineering, contingent on the trainer-side smoke validating the mechanism first. + +#### Smoke verdict template — SP22 H6 Phase 3 α (2026-05-13) + +Workflow: `train-qsltr` @ commit `5106e3b117e89ae5a7e6f0de9f09ed0fe600ad87`, 3 epochs, L40S pool, baseline mode. + +**Verdict criteria** (read HEALTH_DIAG output across 3 epochs): + +**A. Mechanism activity (does atom-shift engage?)** + +1. `eval_dist [eq, eh, ef]` — eval action distribution. If atom-shift biases Q toward aux-aligned dir actions, the distribution should TIGHTEN around the dominant direction. +2. `intent_dist [iq, ih, if]` — rollout action distribution. Atom-shift biases per-env action selection, so intent_dist should diverge from baseline when state_121 ≠ 0. +3. `dist_q / dist_h / dist_f` per HEALTH_DIAG mag line — magnitude dimension action distribution. Atom-shift only affects direction; magnitude should stay relatively stable. + +**B. Win rate signal (does mechanism translate to performance?)** + +1. `last_epoch_win_rate` (logged per epoch in financials path) + - Baseline: ~46% per `pearl_per_bar_vs_segment_pnl_signal_mismatch` (segment-level, post-bug-fix) + - Mechanism-positive: ≥ 50% in any of epochs 1-3 + - Strong signal: ≥ 53% by epoch 3 +2. `mag_stats wr_q / wr_h / wr_f` — per-magnitude-bin win rates. If atom-shift's direction bias is correct, the dominant magnitude bin's WR should lift. + +**C. Stability (no regression)** + +1. `health` value — overall training health. Should stay ≥ 0.5; collapse < 0.3 triggers plasticity injection. +2. `action_entropy` — policy entropy. Should NOT collapse below 0.3 (action-collapse trigger). +3. No `TERMINATED_BY_REGRESSION` log lines. +4. `c51_loss` shouldn't diverge — atom-shift changes the loss landscape, so SOME shift in c51_loss magnitude is expected, but >10× jump signals instability. + +**Possible outcomes**: + +| Pattern | Interpretation | Next action | +|---------|----------------|-------------| +| WR ≥ 53% epoch 3, no regression | Mechanism works end-to-end on trainer/rollout | Phase D for eval validation | +| WR 50-53% epoch 3, dist changes visible | Mechanism partially working; dir bias too weak | Tune W init magnitudes (0.5 → 1.0?) | +| WR ~46% but action dist shifts | Mechanism running but no WR translation | Investigate β reward signal disconnect | +| WR ~46% no dist shift | Atom-shift not active (state_121 always 0?) | Check aux head producing recentered state[121] in rollout | +| Training collapse / regression | Loss landscape change destabilized training | Reduce α magnitude or revert Phase 3 | + +**Diagnostic to add for next smoke** (deliberately deferred this run): +- W[0..4] direct readback per epoch via mapped pinned [4] f32 + DtoD copy outside captured graph + HEALTH_DIAG log line. Would directly show Adam updates on W.