Files
foxhunt/docs
jgrusewski 9deda5f65b feat(dqn): ISV-unified per-branch Q-support range (spec 2026-04-23)
Unifies the Q-support range source across atom grid / warm-start quantile
clamp consumers via the ISV signal bus. One broadcast written at epoch
boundary from per-branch Q-stats EMAs, read by the two consumers that
previously held disagreeing ranges. Target observation: atom utilisation
≥40% (up from 11-15% on train-fpxnw).

Phase 0 — per-branch Q-stats kernel Rust plumbing:
* Load q_stats_per_branch_reduce alongside legacy q_stats_reduce
* Add per_branch_q_stats_pinned (28 f32 = 4 × 7, device-mapped)
* PerBranchQValueStats struct: [QValueStatsResult; 4]
* reduce_current_q_stats_per_branch launches the new kernel with the
  four branch (off, size) pairs derived from config.branch_N_size

Phase 1 — ISV v-range plumbing (zero behavioural change at epoch 1):
* ISV_NETWORK_DIM=23 preserved for w_isv_fc1 sizing; ISV_TOTAL_DIM=31
  allocates 8 additional slots for per-branch (centre, half-width)
* Slot constants V_CENTER_DIR..V_HALF_URG covering slots 23..30
* eval_q_mean_ema / eval_q_std_ema / eval_ema_initialized promoted
  to [f32; 4] / [bool; 4]; scalar setters preserved for trajectory
  backtracking (broadcast same value to all branches)
* Bootstrap at construction: centre=0, half=(v_max-v_min)/2 → the
  byte-identical [config.v_min, config.v_max] span per branch before
  any Q observations arrive
* reset_eval_v_range_state resets the 4 per-branch EMAs AND the 8 ISV
  slots to bootstrap values; legacy eval_v_range_pinned[2] still reset
  (deferred removal — spec Phase 3)
* update_eval_v_range reworked: signature takes PerBranchQValueStats and
  per_branch_q_gaps. Maintains 4 independent adaptive-rate EMAs,
  computes (centre, half) per branch with min_half_floor=0.1×(v_max-v_min)
  and clamps to config bounds, writes 8 ISV slots. Branch-0 (direction)
  centre±half is also mirrored into the legacy eval_v_range_pinned for
  consumers that have not yet migrated to the per-branch bus.

Phase 2a/2b — atom grid per-branch v-range:
* adaptive_atom_positions kernel signature changed from
  (v_min: float, v_max: float) to (branch_idx: int, isv_signals: float*);
  reads centre/half from ISV slots 23+2·b, 24+2·b. Eliminates the f64→f32
  ABI trap (spec Phase 2 side-effect) since the only per-branch range
  path is now pointer-based.
* recompute_atom_positions passes branch_idx + isv_signals_dev_ptr per
  branch; no scalar v_min/v_max arg remains.

Phase 2c — warm-start quantile clamp per-branch from ISV:
* warm_start_atom_positions reads per-branch (centre, half) from pinned
  ISV host memory, clamps shared reward-quantile vector into each
  branch's adaptive range before tiling into atom_positions_buf.
  Bootstrap makes this equivalent to the pre-spec config.v_{min,max}
  clamp until the first Q observation lands.

Deviations from spec:
* Phase 2d (per_sample_support_buf → [N, 4, 3]) NOT implemented. The
  spec's premise was that per_sample_support is host-tiled from
  eval_v_range, but the active path in this codebase has it filled by
  iql_compute_per_sample_support (V(s)-centered, per-sample, already
  adaptive) — orthogonal to the ISV bus. Migrating that kernel to
  per-branch output would require rewriting iql_value_kernel +
  iql_support_floor + C51/MSE loss kernel indexing in lockstep, which
  the "no unrelated refactoring" constraint disallows. The loss-kernel
  Bellman projection today uses V-centered bounds that are themselves
  adaptive; the ISV v-range fix still lands the primary win (atom grid
  + warm-start agreement) without touching IQL.

Compile verified: cargo check -p ml + --workspace pass (SQLX_OFFLINE,
CARGO_INCREMENTAL=0, sccache). No TODO/FIXME/XXX introduced.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 21:18:19 +02:00
..