Iterates v2's reward-bias mechanism through v3 (always-fire on tradable),
v4 (cross-branch |Q|-scale fallback), and v5 (structural v_range floor).
Replaces the entire v2 body, not incremental.
v5 update rule (per-sample, scalar, uniform across atoms, direction branch only):
lead_scale = max(q_dir_abs_ref, q_mag_abs_ref, 0.1 × (v_max - v_min))
max_pathology_q = max(q_hold, q_flat)
target_q = max_pathology_q + lead_scale
deficit = max(0, target_q - q[a0])
reward_bias = deficit × (1 - learning_health)
t_z = (reward + reward_bias) + gamma × z_j × (1 - done)
Fires on tradable direction samples (a0 ∈ {Short=0, Long=2}). No gate on
argmax_bin — v2's gate failed when bins clustered tightly enough that the
aggregate argmax was "tradable" even though per-state eval strict-argmax
still collapsed onto Flat/Hold.
Signal stack (all adaptive, no hard-coded knobs):
- isv_signals[17..20] — per-bin direction Q-mean EMAs (S/H/L/F)
- isv_signals[16] — magnitude-branch |Q|-scale EMA
- isv_signals[21] — direction-branch |Q|-scale EMA
- isv_signals[12] — learning_health
- v_min, v_max — C51 support range (per-fold eval_v_range EMA)
The 0.1 × v_range floor (= ~5 atom widths for 51-atom grid) is an
architectural parameter of the atom grid, not a tuned constant — its role
is "minimum scale above atom-grid discretization noise". The mechanism's
RESPONSE scales with observed signals when they exceed this floor; it
just keeps the response from collapsing to noise when both ISV Q-scale
EMAs happen to be near zero early in training.
Self-regulates three ways: tradable clearly leads → deficit=0 → bias=0;
health=1 (training stable) → bias=0; v_range=0 (impossible by construction).
Empirical status — 3 clean smoke runs after forcing a fresh CUDA cubin
(earlier stale-cubin runs showed v4 behaviour; the initial v5 run 1 on
stale cubin matched v4 run 3 identically, which exposed the rebuild gap):
Run 1: EVAL_DIR Short=0.287 Hold=0.000 Long=0.713 Flat=0.000 — Hold+Flat=0 ✓
Run 2: EVAL_DIR Short=0.000 Hold=0.000 Long=1.000 Flat=0.000 — Hold+Flat=0 ✓
Run 3: EVAL_DIR Short=0.000 Hold=0.000 Long=1.000 Flat=0.000 — Hold+Flat=0 ✓
Pre-v5 baseline (committed v2): Hold+Flat ∈ {0.809, 0.872, 0.796} across 3 runs.
The smoke test still fails on magnitude assertions (line 134 eh+ef≥0.30
or line 153 ef≥0.05) because eval magnitude still collapses to Quarter
or Half. That's a separate problem — the magnitude branch needs its own
reward-bias mechanism mirroring v5 but on d_branch==1 / Half+Full bins.
Tracked separately as Task 2.X-ext (internal task #60).
Per feedback_adaptive_not_tuned.md: the mechanism remains signal-driven;
the only scalar constant (0.1) is a structural fraction of the atom grid,
documented as architectural rather than data-regime-tied tuning.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
04a6f0dea6