-
6ded2c55c2
style(rl): F1.3 code review fixes — comments + spec annotations
jgrusewski
2026-05-31 12:01:50 +02:00
-
b8272221db
feat(rl): rl_regime_observer kernel (F1.3)
jgrusewski
2026-05-31 11:58:02 +02:00
-
8ba9417837
style(rl): F1.2 code review fixes — comment + const + power-of-2 doc
jgrusewski
2026-05-31 11:55:06 +02:00
-
c76d960645
feat(rl): rl_regime_flat_count kernel (F1.2)
jgrusewski
2026-05-31 11:51:29 +02:00
-
6353deed15
style(rl): F1.1 code review fixes — separator + comments
jgrusewski
2026-05-31 11:49:37 +02:00
-
c1a0143311
feat(rl): regime_observer ISV slot allocation (F1.1)
jgrusewski
2026-05-31 11:44:35 +02:00
-
baf971ba54
diag(rl): emit v9 eval_warmup state to JSONL + cleanup lints
ml-alpha-adaptive-controller-floors
jgrusewski
2026-05-31 02:12:04 +02:00
-
0c8cb6ad5b
fix(rl): v9 defensive eval-boundary calibration
jgrusewski
2026-05-31 01:52:29 +02:00
-
6c4945fe16
docs(spec): v9 defensive eval-boundary calibration (completes adaptive principle)
jgrusewski
2026-05-31 01:31:46 +02:00
-
ad5b29e652
diag(rl): emit atom-span calibration signals (no behavior change)
jgrusewski
2026-05-31 01:20:37 +02:00
-
82572ff3bd
docs(spec): C51 atom span math validation + empirical proof
jgrusewski
2026-05-31 01:13:47 +02:00
-
d57bee0542
docs(spec): atom resolution design alternatives (post Fix F failure)
jgrusewski
2026-05-31 00:10:05 +02:00
-
6d4a962e5c
Revert "fix(rl): decouple C51 atom span from reward-clamp ceiling"
jgrusewski
2026-05-31 00:01:19 +02:00
-
0fe825a8c5
fix(rl): decouple C51 atom span from reward-clamp ceiling
jgrusewski
2026-05-30 23:52:43 +02:00
-
1a05af803d
fix(cuda): force-close existing position on DD trip / cooldown entry
jgrusewski
2026-05-30 23:35:54 +02:00
-
7064c9269e
fix(rl): un-freeze adaptive WIN/LOSS clamp + reset per-batch state on fold
jgrusewski
2026-05-30 23:19:20 +02:00
-
5e4c2e62b6
fix(rl): CMDP DD recovery + IQN τ reads mean-of-active (not worst)
jgrusewski
2026-05-30 22:40:36 +02:00
-
39efacf77d
fix(rl): CMDP gates per-batch (one independent session per b)
jgrusewski
2026-05-30 22:06:22 +02:00
-
6e0f568160
diag(rl): emit risk-stack ISVs to JSONL
jgrusewski
2026-05-30 21:24:22 +02:00
-
285d42aa7b
feat(rl): adaptive risk-management stack — 5 layers, all ISV-driven
jgrusewski
2026-05-30 20:52:28 +02:00
-
448c5189cf
fix(cuda): confidence gate honors pos_state (skip non-flat positions)
jgrusewski
2026-05-30 20:39:58 +02:00
-
b1ef6664ab
fix(rl): reward_scale floor uses cumulative dones, not closed-trade-steps
jgrusewski
2026-05-30 16:38:17 +02:00
-
083a88f7c3
feat(rl): adaptive controller floors — 12 controllers, all signal-driven
jgrusewski
2026-05-30 15:41:51 +02:00
-
a3dfcd63f5
test(ml-alpha): migrate integration tests to post-Phase-4 trainer API
worktree-agent-a1d1d79db8df05af1
worktree-agent-a1055c4935b0bb281
jgrusewski
2026-05-30 11:52:36 +02:00
-
6695785666
feat(rl): Phase 4.5 — per-batch advantage normalization
jgrusewski
2026-05-30 10:22:24 +02:00
-
12635bd708
feat(rl): Phase 4.4 — ISV-adaptive V blend controller
jgrusewski
2026-05-30 10:09:32 +02:00
-
25f5ce99b6
feat(rl): Phase 4.3 — V_dq → PPO advantage swap + target net soft-update
jgrusewski
2026-05-30 09:14:46 +02:00
-
acdafe508e
feat(rl): Phase 4.2 — DuelingQHead trainer integration (diagnostic mode)
jgrusewski
2026-05-30 08:48:35 +02:00
-
13bf277cd6
feat(rl): Phase 4.1 — DuelingQHead loss + Bellman target + decompose backward
jgrusewski
2026-05-30 08:33:18 +02:00
-
af35bc778e
feat(rl): Phase 4.0 — DuelingQHead forward kernel + struct skeleton
jgrusewski
2026-05-30 08:26:00 +02:00
-
fd31742627
fix(cuda): Plan A v2 —
dd049d9a4 + 3 kernel-only bug fixes (no Phase 2.0)
jgrusewski
2026-05-29 21:34:29 +02:00
-
-
d8247034f8
fix(cuda): delay mega-graph capture until PER is full
ml-alpha-phase-a
jgrusewski
2026-05-28 01:03:41 +02:00
-
346e6670f5
perf(rl): precompute tree_rebuild_levels — eliminate host loop in mega-graph
jgrusewski
2026-05-28 01:01:53 +02:00
-
54de55d4bc
fix(cuda): rewrite 3 PER kernels — unlock sustained mega-graph 300+ sps
jgrusewski
2026-05-28 00:58:29 +02:00
-
d76919d6a2
perf(rl): gate diag staging to every 10th step
jgrusewski
2026-05-28 00:28:25 +02:00
-
9c1b70edec
perf(cuda): mega-graph pipeline — 10-12× speedup (6.4 → 68-76 sps)
jgrusewski
2026-05-28 00:20:56 +02:00
-
7c96504155
spec: mega-graph CUDA pipeline — single graph launch per step
jgrusewski
2026-05-27 23:26:34 +02:00
-
f7427b27de
revert(cuda): restore baseline reward shaping + gate behavior
jgrusewski
2026-05-27 23:17:27 +02:00
-
5717bc07fe
fix(cuda): remove quick-exit bonus — too generous, caused exit spam
jgrusewski
2026-05-27 23:11:26 +02:00
-
ff8cacb0e9
feat(cuda): asymmetric reward shaping — reward fast loss-cutting
jgrusewski
2026-05-27 23:05:22 +02:00
-
de378c5f62
revert(cuda): remove gate exit exemption — peak_equity doesn't track unrealized
jgrusewski
2026-05-27 23:02:11 +02:00
-
6fca6a1d9b
fix(cuda): gate-exempt exits only when position is losing
jgrusewski
2026-05-27 22:56:12 +02:00
-
dfbc916227
fix(rl): disable drawdown penalty, loosen stop-loss to 10×
jgrusewski
2026-05-27 22:46:04 +02:00
-
90f178ae9e
perf(rl): remove sync_training_event + 24 K-loop memsets
jgrusewski
2026-05-27 22:44:01 +02:00
-
2bdb55cc5b
fix(rl): loosen loss defense — penalty 0.01→0.001, stop-loss 2.0→5.0
jgrusewski
2026-05-27 22:35:59 +02:00
-
fa36d55384
feat(cuda): hard stop-loss via action override in confidence gate
worktree-agent-a60dea23f313641e7
jgrusewski
2026-05-27 22:18:40 +02:00
-
fab9c0e324
fix(cuda): disable hard stop-loss — dones override creates state mismatch
jgrusewski
2026-05-27 22:17:12 +02:00
-
35d21cb42f
feat(rl): four-layer loss defense + perf sync removal
jgrusewski
2026-05-27 22:06:59 +02:00
-
49dd4146ee
perf(cudarc): enable async alloc — eliminates 934 stream syncs/200 steps
jgrusewski
2026-05-27 21:45:44 +02:00
-
1c0c246ddf
perf(rl): decouple diagnostic JSON writer to background thread
jgrusewski
2026-05-27 21:30:41 +02:00
-
25ec8c7bcf
perf(cuda): fuse 5 label gathers into sample_and_gather — 95.6% GPU time eliminated
jgrusewski
2026-05-27 21:14:26 +02:00
-
f385558fdb
fix(argo): set FOXHUNT_CUDA_ARCH from detected compute cap
jgrusewski
2026-05-27 20:53:16 +02:00
-
7f4cd86421
feat(rl): wire checkpoint save/resume into training loop
jgrusewski
2026-05-27 20:30:50 +02:00
-
66ec7f75f4
feat(rl): IntegratedTrainer checkpoint save/load
jgrusewski
2026-05-27 20:29:26 +02:00
-
a1277af6c7
feat(rl): AdamW save/load for checkpoint persistence
jgrusewski
2026-05-27 20:23:25 +02:00
-
4fa9da9fc1
perf(cuda): enable TF32 Tensor Core math on all cuBLAS handles
jgrusewski
2026-05-27 20:19:45 +02:00
-
e730b5cbd1
plan: alpha-rl perf + checkpoint + walk-forward implementation
jgrusewski
2026-05-27 20:16:48 +02:00
-
6e641b934c
spec: alpha-rl perf + checkpoint + walk-forward design
jgrusewski
2026-05-27 20:10:47 +02:00
-
-
dd049d9a4c
fix(rl): reward clamp bootstrap WIN=1.0 LOSS=3.0 (was 0.5/0.5)
jgrusewski
2026-05-27 15:46:59 +02:00
-
18e19b4733
fix(rl): wire reward_clamp_controller + atom_support_update in GPU path
jgrusewski
2026-05-27 15:25:20 +02:00
-
17b426ba5b
feat(cuda): re-enable C51 atom span EWMA anchored on clamp bounds
jgrusewski
2026-05-27 15:02:49 +02:00
-
0a066a469d
fix(rl): C51 atom span ±0.5 → ±1.0 to match reward clamp range
jgrusewski
2026-05-27 14:46:50 +02:00
-
3b1265bc20
fix(rl): wire apply_reward_scale into step body — was dead code
jgrusewski
2026-05-27 14:35:55 +02:00
-
ad3e8d1528
fix(cuda): confidence gate exploration floor + symmetric threshold decay
jgrusewski
2026-05-27 14:03:19 +02:00
-
69d8038a80
fix(cuda): SAC co-tuning reads ACTION entropy, not policy entropy
jgrusewski
2026-05-27 13:44:10 +02:00
-
cfc89313bb
fix(cuda): SAC co-tuning uses batch-average entropy + asymmetric rates
jgrusewski
2026-05-27 13:29:57 +02:00
-
e3ca1a7113
feat(rl): co-tune τ with SAC α — adapts to batch size automatically
jgrusewski
2026-05-27 12:55:16 +02:00
-
d011676d75
tune(rl): distillation τ=5.0 — softer target, entropy stabilized
jgrusewski
2026-05-27 12:31:04 +02:00
-
2959e06ef2
tune(rl): SAC α_max=2.0, distillation λ=0.01 — give entropy more room
jgrusewski
2026-05-27 12:25:43 +02:00
-
4a08696128
feat(rl): target-Q distillation + SAC entropy — proper π architecture
jgrusewski
2026-05-27 12:20:30 +02:00
-
01c9cce9f8
feat(rl): distillation-only π — remove PPO surrogate entirely
jgrusewski
2026-05-27 12:00:25 +02:00
-
e5ced809aa
feat(rl): split KL into static reward + dynamic advantage
jgrusewski
2026-05-27 11:46:13 +02:00
-
844412f1df
feat(rl): KL penalty as REWARD not gradient — fixes structural imbalance
jgrusewski
2026-05-27 11:38:09 +02:00
-
c67b58d0d0
perf(lobsim): convert step_fill + step_pnl_track to raw_launch
jgrusewski
2026-05-27 11:25:48 +02:00
-
9e2c036c5e
tune(rl): lower KL β=0.0005 + reward_kl=0.001 for b=1024
jgrusewski
2026-05-27 11:14:58 +02:00
-
6b89dbfcb8
perf(rl): eliminate 2 of 3 per-step lobsim syncs — 21ms → 10ms/step
jgrusewski
2026-05-27 11:07:57 +02:00
-
8f9e4b269d
fix: rename RL_KL_TARGET_INDEX → RL_KL_REF_TARGET_INDEX (avoid dupe)
jgrusewski
2026-05-27 09:59:55 +02:00
-
1a2268b036
feat(rl): KL-based β controller (zero-lag signal, SAC auto-tune pattern)
jgrusewski
2026-05-27 09:57:39 +02:00
-
a7ec00bcab
feat(rl): ISV-driven reward KL β + gentle gradient β=0.003
jgrusewski
2026-05-27 09:50:28 +02:00
-
7894585ff4
feat(rl): KL-augmented reward aligns Q and π + cleanup dead code
jgrusewski
2026-05-27 09:20:25 +02:00
-
ee6bb6b7e8
feat(rl): adaptive KL reference β — maintains 50% Hold target
jgrusewski
2026-05-27 02:08:43 +02:00
-
c965d549d8
feat(rl): KL reference policy — proven RLHF pattern for Hold preservation
jgrusewski
2026-05-27 01:58:01 +02:00
-
8dfd49bda1
fix(rl): move gates + log_pi OUTSIDE CUDA Graph capture
jgrusewski
2026-05-27 01:36:59 +02:00
-
9395075a19
feat(rl): confidence gate on ALL non-Hold + post-gate log_pi + surfer fixes
jgrusewski
2026-05-27 01:30:01 +02:00
-
878c8897b6
wip(rl): Hold prior + entropy controller fixes — investigating gates
jgrusewski
2026-05-27 01:15:16 +02:00
-
5c7cc4ec32
wip(rl): entropy gradient + controller fix — still collapsing
jgrusewski
2026-05-27 01:05:42 +02:00
-
eeb0a829bf
feat(rl): done-gated π update — fixes qpa crash with sparse rewards
jgrusewski
2026-05-27 00:28:49 +02:00
-
e8b64a3b24
wip(rl): mask non-done advantages + remove normalization dead code
jgrusewski
2026-05-27 00:23:52 +02:00
-
514b04bee3
fix(rl): remove advantage normalization — sparse rewards diluted signal
jgrusewski
2026-05-27 00:20:27 +02:00
-
d743336060
feat(rl): revert to V-advantage + normalization, ISV-driven TAU_MAX
jgrusewski
2026-05-27 00:10:57 +02:00
-
5f272db5c0
feat(rl): /B normalize loss reporting — batch-invariant JSONL metrics
jgrusewski
2026-05-26 23:41:03 +02:00
-
8dac9f5f00
feat(rl): /B normalize encoder gradient accumulation — batch invariant
jgrusewski
2026-05-26 23:25:28 +02:00
-
958d39c2aa
feat(rl): batch-normalize pi gradient — scale-invariant to batch size
jgrusewski
2026-05-26 23:14:33 +02:00
-
62a7613a73
feat(rl): advantage normalization — stabilizes Q-advantage PPO
jgrusewski
2026-05-26 23:10:00 +02:00
-
008ea14a82
feat(rl): Q-advantage for PPO — fixes q_pi_agree anti-correlation
jgrusewski
2026-05-26 22:57:40 +02:00
-
6ce61deef0
fix(rl): normalize aux loss by n_valid in step_batched_from_device
jgrusewski
2026-05-26 22:36:06 +02:00
-
0e70bf96fe
fix(rl): wire BCE + aux losses into GPU loader path + event-based sync
jgrusewski
2026-05-26 22:18:30 +02:00
-
6f645df11f
feat(rl): wire lobsim book data from GPU SoA — proper rewards in GPU loader path
jgrusewski
2026-05-26 21:29:05 +02:00
-
3552d08501
fix(rl): streaming per-file GPU upload — prevent host OOM at 45M snaps
jgrusewski
2026-05-26 21:04:30 +02:00
-
1f672c3b09
perf(rl): GPU-resident data loader — zero CPU per step
jgrusewski
2026-05-26 20:51:06 +02:00