jgrusewski
e41a732081
feat(ml-alpha): Phase 4-A — No-transaction-band foundation (ISV slot 799 gate)
Foundation of the no-transaction-band architectural turnover regulator
described in
docs/superpowers/specs/2026-06-03-no-transaction-band-architecture.md.
Davis-Norman (1990) / Imaki-Imajo-Ito (2021, arXiv:2103.01775) — when the
current position lies inside a learned band [b_l, b_u], the architectural
default is "do nothing", complementing Phase 3D's reward-side fixes which
the literature (Goodhart-Skalse 2024) bounds the effectiveness of.
Scope: ISV slots 799-808 + BandHead forward + ±|tanh|·N_max_eff
activation + rl_band_mask action override + rl_band_turnover_loss
(Option b, fixed-target) + 5 GPU-oracle invariants + diag emission.
Master gate `RL_BAND_ENABLED_INDEX` (slot 799) bootstraps to 0.0 (OFF)
so the foundation preserves bit-equality with Phase 3D `bd811a774` until
operator opt-in via `FOXHUNT_BAND_ENABLED=1`.
Adaptive controller, encoder backward chain, and cluster verification
are Phase 4-B / 4-C, not in scope here.
ISV slots (799-808, bumps RL_SLOTS_END to 809):
799 RL_BAND_ENABLED_INDEX (master gate, bootstrap 0.0)
800 RL_BAND_LOWER_INIT_INDEX (b_l init, -0.5 in tanh space)
801 RL_BAND_UPPER_INIT_INDEX (b_u init, +0.5 in tanh space)
802 RL_BAND_LOSS_WEIGHT_INDEX (λ_turnover, 0.01)
803 RL_BAND_TURNOVER_TARGET_INDEX (target frac unmasked, 0.05)
804 RL_BAND_GRAD_SHARPNESS_INDEX (sigmoid surrogate, 4.0)
805 RL_BAND_FLAT_RECENTER_RATE_INDEX (reserved for 4-B controller)
806 RL_BAND_WIDTH_MIN_INDEX (collapse detection, 0.1)
807 RL_BAND_WIDTH_MAX_INDEX (saturation detection, 1.8)
808 RL_BAND_DIAG_LAUNCH_EVERY_INDEX (diag cadence, 1.0)
CUDA kernels (3 new):
rl_band_head_forward.cu — 2-stage forward: linear projection
(tree-reduce over HIDDEN_DIM, matches
ppo_policy_logits_fwd shape) + asymmetric
±|tanh|·N_max_eff activation enforcing
b_l ≤ 0 ≤ b_u (Davis-Norman invariant).
rl_band_mask.cu — overrides actions[b]→Hold when
position_lots[b] ∈ [b_l, b_u]. Master-
gated at slot 799; no atomicAdd; runs
OUTSIDE graph capture per spec §9.5.
rl_band_turnover_loss.cu — Option (b) turnover regularizer with
sigmoid surrogate. Per-batch loss +
per-batch grad on (b_l, b_u). Phase 4-A
wires kernel + test; full encoder grad
fold is Phase 4-B.
Rust glue:
crates/ml-alpha/src/rl/band_head.rs — BandHead struct, forward,
launch_mask, launch_turnover_loss.
Xavier-0.01 init under
scoped_init_seed(seed+0xBA_5EED).
trainer/integrated.rs — BandHead field + construction
(bias init = atanh(0.5)) +
ISV bootstrap row + FOXHUNT_BAND_
ENABLED override + forward call
at both step_with_lobsim and
step_with_lobsim_gpu_body sites
+ mask launch BEFORE confidence
gate + per-step band aggregate
diag (lower/upper/width means,
frac_in_band, frac_masked,
collapse_warning).
Tests (5 GPU-oracle invariants, all PASS):
band_activation_clamps_correctly — b_l ≤ 0 ≤ b_u, |·| ≤ N_max_eff
band_mask_forces_hold_when_in_band — pos 0 ∈ [-4,+4] → action becomes Hold
band_mask_passes_through_when_out_of_band — pos 5 ∉ [-1,+1] → action unchanged
band_turnover_loss_correct — loss + grad match analytical form
band_disabled_means_no_mask — slot 799 = 0.0 → mask no-op
Verification:
cargo build --release --example alpha_rl_train: exit 0
cargo test band_invariants --release -- --ignored: 5/5 PASS
cargo test multi_head_policy_invariants -- --ignored: 18/18 PASS (regression)
determinism-check.sh --quick (band OFF): exit 0
FOXHUNT_BAND_ENABLED=1 determinism-check.sh --quick: exit 0
FOXHUNT_USE_MULTI_HEAD_POLICY=1 determinism-check.sh --quick: exit 0
FOXHUNT_USE_MULTI_HEAD_POLICY=1 FOXHUNT_BAND_ENABLED=1 …: exit 0
Phase 4-A local mid-smoke (b=128, 2000+500 steps, band enabled):
exit 0, no NaN observed
frac_masked = 1.0 throughout (band wide at [-4,+4] init)
total_trades = 0 (vs Phase 3D 11,767) — primary kill criterion PASS
band width drifts 8.02 → 7.58 over 2000 steps (slight narrowing from
encoder shared-h_t shift; band-head's own backward chain is Phase 4-B)
G_no_band_collapse FAILS (frac_masked saturates at 1.0) — expected at
Phase 4-A because turnover-loss gradient is not yet folded into the
encoder (per spec §7.1 / §9.1 Mitigation 1, which requires Phase 4-B
adaptive controller + backward chain).
Pearls:
pearl_bootstrap_must_respect_clamp_range (every bootstrap ∈ clamp)
pearl_scoped_init_seed_for_reproducibility (band-head init guard)
pearl_determinism_achieved (no PRNG / no atomicAdd in kernels)
pearl_foxhunt_pi_trained_by_q_distillation_not_ppo (band is structural,
not reward-side — sidesteps Goodhart-Skalse attenuation)
pearl_fleet_fraction_not_aggregate (frac_in_band / frac_masked emitted)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 00:50:27 +02:00
..
2026-03-10 15:49:40 +01:00
2026-03-11 22:00:13 +01:00
2026-03-13 11:10:01 +01:00
2026-03-14 19:41:40 +01:00
2026-03-16 16:11:48 +01:00
2026-03-21 10:23:17 +01:00
2026-03-21 15:33:00 +01:00
2026-03-21 23:22:58 +01:00
2026-03-22 20:30:35 +01:00
2026-03-22 00:39:34 +01:00
2026-03-22 10:32:37 +01:00
2026-03-23 08:45:18 +01:00
2026-04-22 22:03:20 +02:00
2026-03-23 12:45:06 +01:00
2026-03-23 09:35:55 +01:00
2026-03-25 19:43:26 +01:00
2026-03-25 22:53:13 +01:00
2026-03-26 00:43:58 +01:00
2026-03-27 19:52:44 +01:00
2026-03-27 19:52:44 +01:00
2026-03-27 19:52:44 +01:00
2026-03-27 20:42:37 +01:00
2026-03-31 23:30:42 +02:00
2026-04-01 21:36:40 +02:00
2026-04-04 11:49:13 +02:00
2026-04-04 11:07:47 +02:00
2026-04-04 09:28:42 +02:00
2026-04-04 00:57:49 +02:00
2026-04-04 23:00:16 +02:00
2026-04-04 17:30:18 +02:00
2026-04-05 21:25:05 +02:00
2026-04-06 22:48:07 +02:00
2026-04-06 11:15:24 +02:00
2026-04-07 13:58:11 +02:00
2026-04-06 11:15:24 +02:00
2026-04-07 13:43:04 +02:00
2026-04-08 01:26:09 +02:00
2026-04-07 23:14:40 +02:00
2026-04-07 22:21:11 +02:00
2026-04-08 20:05:22 +02:00
2026-04-08 01:49:01 +02:00
2026-04-08 08:48:36 +02:00
2026-04-08 12:04:23 +02:00
2026-04-08 22:23:19 +02:00
2026-04-08 20:26:40 +02:00
2026-04-10 20:26:55 +02:00
2026-04-10 22:16:24 +02:00
2026-04-11 11:23:23 +02:00
2026-04-11 13:30:12 +02:00
2026-04-13 09:26:14 +02:00
2026-04-13 14:51:30 +02:00
2026-04-13 20:13:39 +02:00
2026-04-13 11:29:05 +02:00
2026-04-14 21:40:03 +02:00
2026-04-14 23:46:27 +02:00
2026-04-14 22:14:49 +02:00
2026-04-15 20:46:05 +02:00
2026-04-15 18:44:18 +02:00
2026-04-15 08:52:17 +02:00
2026-04-16 01:24:03 +02:00
2026-04-16 23:51:45 +02:00
2026-04-16 01:29:11 +02:00
2026-04-16 17:30:01 +02:00
2026-04-16 23:43:55 +02:00
2026-04-16 01:24:03 +02:00
2026-04-17 09:01:53 +02:00
2026-04-16 21:44:35 +02:00
2026-04-18 14:59:57 +02:00
2026-04-18 16:03:45 +02:00
2026-04-18 23:59:35 +02:00
2026-04-18 22:04:41 +02:00
2026-04-19 12:10:49 +02:00
2026-04-19 23:47:04 +02:00
2026-04-20 18:58:35 +02:00
2026-04-20 08:51:42 +02:00
2026-04-20 08:17:34 +02:00
2026-04-20 22:03:14 +02:00
2026-04-20 14:29:53 +02:00
2026-04-21 02:05:28 +02:00
2026-04-22 08:42:50 +02:00
2026-04-21 21:06:33 +02:00
2026-04-22 11:43:17 +02:00
2026-04-22 08:51:51 +02:00
2026-04-22 16:56:36 +02:00
2026-04-22 09:09:09 +02:00
2026-04-22 09:06:50 +02:00
2026-04-21 11:04:51 +02:00
2026-04-23 10:15:16 +02:00
2026-04-23 21:18:19 +02:00
2026-04-23 09:12:01 +02:00
2026-04-24 21:03:57 +02:00
2026-04-29 16:02:53 +02:00
2026-04-27 18:22:40 +02:00
2026-04-29 22:59:00 +02:00
2026-04-30 08:36:32 +02:00
2026-05-01 14:07:10 +02:00
2026-05-01 19:54:02 +02:00
2026-05-02 01:36:48 +02:00
2026-05-03 00:11:26 +02:00
2026-05-03 20:33:36 +02:00
2026-05-03 22:24:14 +02:00
2026-05-04 11:30:52 +02:00
2026-05-04 18:41:57 +02:00
2026-05-04 23:18:58 +02:00
2026-05-05 17:21:17 +02:00
2026-05-06 08:58:28 +02:00
2026-05-09 00:25:06 +02:00
2026-05-09 20:49:07 +02:00
2026-05-10 22:19:28 +02:00
2026-05-15 20:32:47 +02:00
2026-05-15 20:42:35 +02:00
2026-05-15 22:08:31 +02:00
2026-05-16 23:17:33 +02:00
2026-05-17 23:13:32 +02:00
2026-05-18 23:24:01 +02:00
2026-05-18 13:20:26 +02:00
2026-05-18 10:20:38 +02:00
2026-05-18 01:51:11 +02:00
2026-05-20 00:40:14 +02:00
2026-05-19 16:27:38 +02:00
2026-05-20 08:10:00 +02:00
2026-05-19 00:54:45 +02:00
2026-05-19 00:54:45 +02:00
2026-05-20 20:15:28 +02:00
2026-05-20 22:59:49 +02:00
2026-05-21 15:05:28 +02:00
2026-05-21 15:05:28 +02:00
2026-05-21 15:05:28 +02:00
2026-05-21 15:05:28 +02:00
2026-05-24 16:28:39 +02:00
2026-05-25 21:41:10 +02:00
2026-05-25 23:00:40 +02:00
2026-05-26 00:00:00 +02:00
2026-05-25 20:09:14 +02:00
2026-05-26 09:39:17 +02:00
2026-05-26 08:59:51 +02:00
2026-05-26 00:37:30 +02:00
2026-05-26 01:30:29 +02:00
2026-06-02 17:56:00 +02:00
2026-05-30 15:41:51 +02:00
2026-05-30 20:52:28 +02:00
2026-05-31 00:10:05 +02:00
2026-05-31 01:13:47 +02:00
2026-05-31 16:57:43 +02:00
2026-05-31 22:25:24 +02:00
2026-06-02 17:56:00 +02:00
2026-06-02 17:56:00 +02:00
2026-05-31 23:09:59 +02:00
2026-06-02 17:56:00 +02:00
2026-05-31 01:31:46 +02:00
2026-06-01 10:15:36 +02:00
2026-06-01 10:31:18 +02:00
2026-06-01 12:20:42 +02:00
2026-06-01 21:04:47 +02:00
2026-06-01 01:13:27 +02:00
2026-06-01 22:38:47 +02:00
2026-06-02 17:56:00 +02:00
2026-06-02 17:56:00 +02:00
2026-06-03 01:41:57 +02:00
2026-06-02 17:56:00 +02:00
2026-06-02 22:10:16 +02:00
2026-06-04 00:50:27 +02:00