43c4bddd8e5fc7fccdea7b10fbbfa649955e23cd
82 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
0e15899670 |
feat(rl): exhaustive diag JSONL for all trade-management mechanics
Surfaces full per-unit per-batch state in the per-step diag output: - units: entry_price, entry_step, lots, trail_distance, active_mask, unit_count (all [B × MAX_UNITS] arrays) - trail: fired/tightened/loosened counts (step + cumulative) - pyramid: added count (step + cumulative), units_distribution, max_units_reached flag - partial_flat: fired count (step + cumulative), long/short split, close_unit_index per batch - confidence_gate: gated count (step + cumulative) - frd_gate: gated count (step + cumulative) - position_heat: capped count (step + cumulative), max_lots ISV - anti_martingale: per-batch outcome_ema, kappa ISV Replaces the minimal pyramid/heat_cap diag from P7. Every mechanic is now fully observable in post-hoc analysis. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
jgrusewski
|
a583bb508c |
feat(rl): pyramiding semantics — add/partial-flat/anti-martingale sizing
Implements the full pyramid trade-management suite:
P7.a: actions_to_market_targets gates pyramid adds on ISV-driven
threshold (slot 506); rl_unit_state_update allocates sequential
unit slots on position growth, deactivates oldest on shrink.
P7.b: HalfFlat (a9/a10) closes oldest unit's lots when pyramid>1;
trail-stop routes breaches through HalfFlat + close_unit_index
override instead of nuclear full-flat.
P7.c: Anti-martingale sizing on opening actions via signed outcome EMA
(slot 508) — size_eff = base × clamp(1 + κ × ema, MIN, MAX).
Diag: pyramid { units_count, add_count, outcome_ema } in step JSONL.
ISV slots: 506 threshold, 507 add_count, 508 outcome_ema,
509 κ, 510 MIN, 511 MAX. RL_SLOTS_END → 512.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
45a2041db4 |
feat(rl): SP20 P6 position heat cap — force-flat on over-leverage
Last-defense guard: if |position_lots| exceeds the ISV-driven
RL_HEAT_CAP_MAX_LOTS (slot 504, default 8 = MAX_UNITS × max_order_size),
the kernel overrides actions[b] to FlatFromLong (a3) or FlatFromShort
(a4) — full flatten, no partial. Catches runaway pyramid accumulation
before it reaches actions_to_market_targets.
Override stack ordering (step_with_lobsim):
1. rl_trail_mutate (a7/a8)
2. rl_trail_stop_check → may override to FlatFromLong/Short
3. rl_position_heat_check (THIS) → may override to FlatFromLong/Short
4. actions_to_market_targets → reads final actions[b]
Kernel `cuda/rl_position_heat_check.cu`:
* 1 block, b_size threads (grid-stride for b_size > 256)
* Reads position_lots from pos_state at offset 0 (PosFlat layout)
* Cap read from ISV[504]; if cap ≤ 0 → no-op (guard disabled)
* Per feedback_no_atomicadd: fired-count diagnostic uses shared-mem
flag array + thread-0 serial count (b_size ≤ 256 in practice)
* Writes fired-count to ISV[505] for diag
ISV slots:
* 504: RL_HEAT_CAP_MAX_LOTS_INDEX (seed 8.0)
* 505: RL_HEAT_CAP_FIRED_COUNT_INDEX (diagnostic, written per step)
* RL_SLOTS_END bumped 505 → 506
Diag (alpha_rl_train):
* "heat_cap": { "fired_count": N, "max_lots": 8 }
GPU oracle test (trade_management_kernels.rs):
* position_heat_cap_overrides_on_breach — long 5 > cap 4 → a3;
short -5 < -cap -4 → a4; long 3 ≤ cap 4 → untouched (Hold)
Verification (RTX 3050 Ti):
* cargo check -p ml-alpha --examples → clean
* integrated_trainer_smoke 1/1 → ok
* trade_management_kernels 6/6 (was 5/5, +1 heat cap) → ok
* audit-rust-consts → 0 flags
|
||
|
jgrusewski
|
7df7c81d37 |
refactor(rl): FRD horizons + range_σ are ISV-driven, not literals
Closes the literal/const drift gap that F.5 introduced. Per
feedback_isv_for_adaptive_bounds + feedback_single_source_of_truth_no_duplicates:
adaptive bounds belong in ISV (or in a single canonical const that
ISV references), never duplicated as literals across modules.
Single canonical source: `crate::rl::common::FRD_HORIZON_TICKS` +
`FRD_BUCKET_RANGE_SIGMA` (already declared in F.5).
Producer-side fixes:
* Trainer ISV bootstrap (integrated.rs): the seed values for slots
500-503 now dereference the canonical consts instead of hardcoded
60.0/300.0/1800.0/3.0 literals. Future tuning of the consts
automatically propagates to both ISV seeds and loader-side
labels — no manual sync required, no drift possible.
* compute_frd_labels (loader.rs): takes `horizon_ticks` and
`range_sigma` as parameters instead of reading consts directly.
Caller (the file-load closure) sources them from the new
MultiHorizonLoaderConfig fields.
Consumer-side fixes — 8 MultiHorizonLoaderConfig literal sites now
provide the two new fields, all defaulting to the canonical consts:
* crates/ml-alpha/src/data/loader.rs (2 internal test-fixture sites)
* crates/ml-alpha/tests/multi_horizon_loader.rs (2 sites)
* crates/ml-alpha/examples/alpha_train.rs (2 sites)
* crates/ml-alpha/examples/alpha_rl_train.rs (2 sites)
* crates/ml-backtesting/src/harness.rs (1 site)
* crates/ml-backtesting/tests/{trainer_parity,ring3_replay}.rs (2 sites)
The "optimal by default" property is preserved: every caller that
doesn't explicitly override gets the spec-recommended 60/300/1800
ticks + ±3σ. Callers that need to retune set the config fields, and
the trainer's ISV slots provide a runtime knob for the same numerics.
Verification (RTX 3050 Ti):
* cargo check -p ml-alpha -p ml-backtesting --examples --tests → clean
* cargo test --lib (6/6 unit tests for FRD label gen + loss_balance) → pass
* frd_head 10/10 + integrated_trainer_smoke 1/1 + trade_mgmt 5/5 → pass
* audit-rust-consts → 0 flags
The two new MultiHorizonLoaderConfig fields are required (no Default
impl) — callers MUST opt in to the FRD label-generation contract by
naming the fields. This is the same discipline applied across other
config consumers; making them Option<...> would silently default to
"no FRD labels" and break F.4's expected supervised signal.
|
||
|
jgrusewski
|
125c667a34 |
feat(rl): FRD label generation in loader + per-step write (F.5)
Activates the FRD head's supervised training signal that F.4 wired
through the trainer. Per-file forward-return σ-bucketed labels
computed at load time + per-step write into trainer.frd_labels_d
before each step_with_lobsim.
Loader-side label generation (`compute_frd_labels` in data/loader.rs):
* Mid-price series from snapshots[i].levels[0]
* Per-file σ_per_step = sample-std of single-tick mid increments
* For each FRD_HORIZON h ∈ {60, 300, 1800}:
- r = (mid[i+h] - mid[i]) / (σ_per_step × sqrt(h)) ← Brownian scaling
- bucket = round(r × (FRD_N_ATOMS-1) / (2 × FRD_BUCKET_RANGE_SIGMA)
+ (FRD_N_ATOMS-1) / 2)
- clamp to [0, FRD_N_ATOMS-1] for tail returns
- sentinel -1 if i + h >= n
* Cached in LoadedFile.frd_labels_full alongside sigma_k_full /
outcome_*_full
* Per-anchor slice into LabeledSequence.frd_labels (length-1 vec
per horizon at the newest-snapshot index — h_t aligns with the
rightmost K position, the only one the FRD head supervises)
New structural constants in rl/common.rs:
* FRD_HORIZON_TICKS = [60, 300, 1800] ← matches ISV slots 500/501/502 defaults
* FRD_BUCKET_RANGE_SIGMA = 3.0 ← matches ISV slot 503 default
Per pearl_glm_fitter_link_must_match_inference: bucket-edge math
here MUST match the trainer-side softmax+CE atom interpretation.
Both reference the same const so they can't drift.
alpha_rl_train per-step wiring:
* Stage frd_labels_bh[b_idx × FRD_N_HORIZONS + h] from
s_t.frd_labels[h][0] (the per-batch label at this step's anchor)
* write_slice_i32_d_pub into trainer.frd_labels_d BEFORE
step_with_lobsim → bwd chain reads real labels in step_synthetic
Tests (3 new in loader::frd_label_tests, total 3/3 passing):
* frd_labels_flat_price_maps_to_mid_bucket — constant mid → all
non-sentinel labels = 10 (FRD_N_ATOMS/2 rounded); sentinel range
[n-h, n) tested exhaustively
* frd_labels_monotonic_ramp_lands_in_upper_buckets — linear ramp
mid[i] = 100 + 0.01×i produces forward returns way above 3σ at
every horizon → clamp to top bucket (FRD_N_ATOMS-1=20)
* frd_labels_short_input_below_h_ticks_all_sentinel — n=10 < h_ticks
for all 3 horizons → every label is -1 (no leak in the sentinel path)
Existing tests still pass:
* loss_balance lib tests 3/3
* frd_head GPU tests 10/10
* integrated_trainer_smoke 1/1
* trade_management_kernels 5/5
The full FRD head pipeline is now active end-to-end. Cluster smoke
will show FRD entropy_mean drift below ln(21) ≈ 3.044 once the bwd
gradient signal accumulates — the observable proof that supervised
learning is happening. The "frd" diag block from F.2 was always
prepared for this; F.5 just feeds it real signal.
F.6+ scope (deferred, separate sessions):
* P9 FRD gate — override action to Hold when entry_quality < THR
* Loss-balance controller integration for λ_frd (currently 1.0 default)
* Per-horizon Sharpe attribution in diag
|
||
|
jgrusewski
|
935433850c |
feat(rl): FRD head trainer integration — Adam + bwd chain + loss (F.4)
Wires the F.3a/b/c backward kernels into IntegratedTrainer's per-step
flow so the FRD head trains end-to-end as a 6th loss-balanced head
alongside BCE/Q/π/V/aux. With labels currently sentinel-initialized to
-1 (F.5 loader will populate from forward-snapshot lookahead), the
chain produces zero gradients + zero loss — Adam steps are no-ops
modulo β decay, and the encoder receives no FRD-derived signal yet.
The wiring is complete and the path is exercised end-to-end; F.5 just
needs to swap the labels in for the head to start training.
IntegratedTrainer state additions:
* frd_w1_adam / frd_b1_adam / frd_w2_adam / frd_b2_adam — AdamW
instances for the 4 FRD weight tensors (LR mirrored per-step from
ISV[RL_FRD_LR_INDEX=499], seed 1e-3 per F.1).
* frd_labels_d — owned [B × FRD_N_HORIZONS] i32 buffer, sentinel-
initialized to -1 (every entry "missing horizon" → softmax_ce_grad
zeros loss + grad for every row). F.5 loader integration overwrites
pre-step from forward-return-bucketed labels.
LossLambdas extension:
* Added `frd: f32` field, default 1.0
* read_loss_lambdas_from_isv reads slot 498 (RL_FRD_LAMBDA_INDEX)
with the standard zero-sentinel bootstrap path
* Doc-comment updated: "5 heads / 5.0" → "6 heads / 6.0"
IntegratedStepStats extension:
* Added `l_frd: f32` — mean CE across (B × FRD_N_HORIZONS) rows
* step_synthetic returns the real l_frd from the bwd chain; the
new combined l_total formula includes `lambdas.frd × l_frd / 6`
step_synthetic bwd chain — inserted between Step 9 (Q/π/V Adam) and
Step 10 (grad_h_t_combined zero+accumulate):
1. softmax_ce_grad → frd_grad_logits_d + frd_loss_per_b_h_d
2. layer2_bwd → frd_grad_w2_pb_d, frd_grad_b2_pb_d, frd_grad_hidden_d
3. layer1_bwd → frd_grad_w1_pb_d, frd_grad_b1_pb_d, frd_grad_h_t_d
4. 4× reduce_axis0 to collapse per-batch scratch → final grads
5. 4× AdamW.step on w1/b1/w2/b2
6. read loss_per_b_h via mapped-pinned, average → l_frd_host
Step 10 grad_h_t_combined accumulation adds a third λ-weighted call:
accumulate_grad_h(frd_grad_h_t_d, lambdas.frd, &mut combined)
With sentinel labels (F.4 state) this contributes zero gradient to the
encoder backward — the wiring is exercised but silent. F.5 makes it
active by providing real labels.
alpha_rl_train diag JSON gains:
* "loss": { ..., "frd": stats.l_frd, ... }
* "lambdas": { ..., "frd": stats.lambdas.frd, ... }
Verification (RTX 3050 Ti):
* cargo check -p ml-alpha + --examples → clean
* integrated_trainer_step_with_lobsim_runs_without_panic → ok
(l_total 0.5073 vs prior 0.6087 — ÷6 instead of ÷5 expected;
l_frd=0 confirms sentinel labels are passing through cleanly)
* frd_head 10/10 tests still pass (no regression)
* trade_management_kernels 5/5 → no regression
* audit-rust-consts → 0 flags
F.5 (next, separate scope):
* Loader-side forward-return label generation (mid[i+h] - mid[i])/σ
bucketed into FRD_N_ATOMS=21 atoms over the ISV-driven ±range_σ
* Populate trainer.frd_labels_d before each step_with_lobsim call
* That unlocks the supervised learning signal; FRD entropy_mean
should start dropping below ln(21) in diag as the head trains.
|
||
|
jgrusewski
|
119c3a15f4 |
feat(rl): wire FRD head forward into trainer + diag (F.2 integration)
IntegratedTrainer now owns an FrdHead instance and per-step buffers
(frd_hidden_d [B × FRD_HIDDEN_DIM=64], frd_logits_d [B × FRD_OUT_DIM=63]).
The forward kernel runs in step_with_lobsim immediately after the
current-snapshot encoder forward, reading h_t_borrow and producing the
3-horizon × 21-atom return-bucket logits.
step_with_lobsim FRD forward placement rationale: it has to read
self.perception.h_t_view() AFTER the second forward_encoder(snapshots)
call (which lands h_t at slot K-1), but BEFORE any downstream
consumer of the encoder state — so right between Step 1b and Step 2.
This keeps the FRD output aligned with the same h_t that the Q / π /
V heads see for action sampling.
alpha_rl_train diag emits a new "frd" block per step:
"frd": { "h1": {"entropy_mean", "argmax_mean"}, "h2": ..., "h3": ... }
At init (Xavier × 0.1, b1=b2=0) the per-horizon softmax is near-
uniform → entropy_mean ≈ ln(21) = 3.044 and argmax_mean drifts around
the uniform expectation of 10. As supervised training kicks in (F.3),
entropy drops and argmax tracks the realized forward-return mode per
horizon — this is the observable signal that lets us catch a broken
backward kernel before cluster smoke.
Verification:
* cargo check -p ml-alpha --examples → clean
* integrated_trainer_step_with_lobsim_runs_without_panic → ok
(1.66s, b_size=1, full step path through encoder + FRD + Q/π/V)
* audit-rust-consts → 0 flags
* trade_management_kernels (5/5) + frd_head (3/3) → still pass
F.3 (backward kernel + finite-diff tests + label generation in loader
+ λ_frd-weighted loss accumulation into stats.l_total) is the next
chunk. FRD-gate (P9) and FRD label-cache wiring are separate scope.
|
||
|
jgrusewski
|
d3175711b9 |
feat(rl): SP20 P4 — N_ACTIONS 9→11 with HalfFlat actions
Action enum extended:
a9 = HalfFlatLong (close ⌈|pos|/2⌉ of long position, no-op if not long)
a10 = HalfFlatShort (close ⌈|pos|/2⌉ of short position, no-op if not short)
`actions_to_market_targets.cu` extended with a9/a10 handlers:
HalfFlatLong (pos > 0): side=1 sell, size=max(1, (position_lots+1)/2)
HalfFlatShort (pos < 0): side=0 buy, size=max(1, (|position_lots|+1)/2)
Round-up division ensures min 1 lot closes — single-lot positions
fully close on HalfFlat (the half rounds up to 1).
N_ACTIONS=9 → 11 propagated to all 10 .cu kernels:
argmax_expected_q, bellman_target_projection, dqn_distributional_q,
log_pi_at_action, ppo_clipped_surrogate, rl_action_kernel,
rl_entropy_coef_controller, rl_pi_action_kernel, rl_q_pi_agree_b,
rl_q_pi_distill_grad
Rust-side N_ACTIONS const bumped to 11 in src/rl/common.rs.
CLI alpha_rl_train.rs action_hist + windowed_act_hist refactored
to reference `N_ACTIONS` const instead of literal 11. Caught DURING
this commit's dogfood: an intermediate state had `[0u32; 11]` but
left `(0..9).contains(&a)` unchanged — HalfFlat samples silently
dropped (a9/a10 showed 0% in diag despite P_MIN=0.02 floor
guaranteeing 2% each). Fix uses N_ACTIONS const everywhere; new
pearl `feedback_use_consts_not_literals_for_structural_dims`
codifies the meta-pattern (Rust code mirroring kernel structural
dims must reference the const, NEVER duplicate the literal —
audit-isv only scans .cu files, this class of bug is currently
unaudited in .rs).
Audits PASS:
audit-isv: all kernel #defines allowlisted (BOOK_LEVELS,
ACTION_*, structural dims)
audit-wiring: all 4 actions in manifest (TrailTighten, TrailLoosen,
HalfFlatLong, HalfFlatShort) have consumers
Local 1k-step smoke (RTX 3050 Ti, 13.5s):
* Exit 0, 0 NaN/inf, 16000/16000 samples accounted for
* Action distribution: all 11 used in 7-11% range
* HalfFL=7.99%, HalfFS=7.51% — π samples them under multinomial
* Trail=19.34% — agent continues to value trail-stop actions
Trail-stop check (rl_trail_stop_check.cu) currently still routes
force-close through a3/a4 (FlatFromLong/Short) rather than per-unit
partial-flat via a9/a10. That routing upgrade is SP20 P5b follow-up
work — it requires the per-unit close_unit_index buffer wiring per
spec §3 P5. Adding a9/a10 to the action space is the foundational
prerequisite; consumer kernel uses come with P5b.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
e87e0b0774 |
feat(rl): adaptive λ_distill controller + reward_scale MIN ISV
Two architectural fixes from rljzl in-flight analysis (ultrathink
deep dive on actions a7/a8 + per-action calibration):
(1) λ_distill: static → controller-driven via Schulman bounded step
wwcsz showed Q→π KL EMA dropped 2.10 → 0.30 with λ=0.01, then
rljzl bumped to 0.05. Static λ is design intuition; KL is the
natural feedback signal:
if KL > target × 1.5 → λ *= 1.2 (Q not landing, pull harder)
if KL < target / 1.5 → λ /= 1.2 (Q absorbed, relax)
Bounds [MIN=0.001, MAX=1.0]. Target KL seeded 0.1 (slot 491).
New kernel `rl_q_distill_lambda_controller.cu`. Runs after the
distill kernel writes KL_EMA each step.
(2) REWARD_SCALE_MIN: hardcoded 1e-3 → ISV-driven 1e-4
wwcsz audit (mean_abs_pnl_ema mean=920, max=49437, p99=high):
the controller wanted scale ≈ 3.5e-4 when EMA spiked to 2871
but pegged at 1e-3, letting scaled rewards exceed unit support
and wasting C51 atom resolution on outliers. ISV slot 492
permits runtime re-tuning; default 1e-4 admits one more order
of magnitude before pegging. Per user-stated "floors and clamp
bounds" exemption — ISV-resident for tunability, not because
required.
Diag exposes q_distill_kl_target + reward_scale_min so the new
adaptation chains are observable.
Investigation (ultrathink): actions 7/8 (TrailTighten/TrailLoosen)
have ZERO consumers across the codebase. Spec'd as "ISV mutation"
in actions_to_market_targets.cu header but no slot, no mutation
kernel, no stop-check kernel. ~10% of wwcsz policy mass goes to
dead no-ops. Documented in
`pearl_dead_trail_stop_actions_a7_a8.md` — implementation
deferred to its own SP (per-batch trail_distance buffer +
mutation kernel + stop-check integration with LobSim apply_fill_to_pos
per `pearl-stop-checks-run-at-deadline-cadence`). N_ACTIONS=9
preserved; alternative refactor to 7 actions captured as
"Path B" in the pearl.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
185add7dc8 |
feat(rl): adaptive RATIO + EWMA V_MIN/V_MAX + λ_distill bump
wwcsz analysis identified atom-resolution starvation + asymmetric
RATIO mismatch as the empirical ceiling on win rate (38.56% vs
break-even 45.3%). Three coupled fixes shipped in one pass per
"no deferrals":
(a) Adaptive RATIO from observed |loss|/|win| EMAs:
- apply_reward_scale tracks max(-scaled, 0) per step (slot 489)
- rl_reward_clamp_controller maintains neg_max_ema (slot 490,
sparse-aware like pos_max_ema)
- RATIO = clamp(MIN=1.0, neg_ema/pos_ema, MAX=3.0); writes to
slot 481
- Removes built-in 3:1 loss-aversion bias when reality is
symmetric (wwcsz: actual avg|loss|/avg(win) = 0.83). Floor
1.0 prevents inverted asymmetry; ceiling 3.0 preserves
original loss-aversion as the worst case.
(b) C51 V_MAX/V_MIN: ratchet → slow EWMA (α=0.001, half-life ~700
steps):
- Static ratchet wasted atom resolution on rare tails — wwcsz
had V_MIN=-60, V_MAX=20 but realized rewards mostly in [-5, +5]
(Δz=4 vs typical reward magnitude 1-5)
- Slow EWMA lets atom span shrink toward active reward range,
gaining resolution where data lives. Floors at [-1, +1]
preserve original C51 baseline as the worst case.
- Slow α gives Q's atom mapping time to be valid across
encoder/head co-adaptation (vs aggressive EWMA which would
invalidate Q's learned distribution every step)
(c) Q→π distillation λ bumped 0.01 → 0.05:
- wwcsz showed KL dropped 2.10 → 0.30 with λ=0.01 — Q signal
landing but conservatively. Bump tests whether stronger Q
pull translates to better policy → better R/done.
Diag exposes neg_scaled_max + neg_scaled_max_ema so the RATIO
adaptation chain is observable.
apply_reward_scale shared_mem doubled from 2× to 3× block × f32
to fit the three parallel reductions (abs, pos, neg).
Companion to investigation (e) — n_rollout_steps controller was
suspected of misalignment (256-8192 vs trade_duration ≈ 6 steps)
but turned out to be a K-loop param, not used in Bellman target.
1-step Bellman with γ-bootstrap is the actual mechanism; closed
without code change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
79756a2153 |
fix(rl): sparse-aware EMA + Q→π distillation breaks defensive trap
Two coupled fixes addressing vj5f6 findings:
(1) WIN_clamp oscillation — sparse-aware EMA
vj5f6 showed WIN_clamp oscillating 1.0 ↔ 67.0 across 40k steps.
Root cause: the Wiener-α blend in rl_reward_clamp_controller
treated pos_max=0 as "no win this step ≡ win magnitude is zero,"
exponentially decaying the EMA toward 0 during dry-spell windows
(no closed winning trades). With α=0.4, ten dry steps decayed EMA
by 0.6^10 ≈ 0.006, collapsing WIN back to MIN_WIN=1.0 floor.
Fix: only update pos_max_ema AND clip_rate_ema AND MARGIN when
pos_max > 0. A dry step is "no signal," not "zero signal." The
EMA retains its last winning-period estimate; the controller
doesn't ratchet on stale data.
(2) Q→π distillation — couples Q's improved calibration to π
vj5f6 showed l_q dropping 100× (2.37 → 0.02) but reward economics
IDENTICAL to 8xwq8 (no C51 V_MAX lift). Per Option B, π drives
action selection but is trained by PPO surrogate using advantage
= returns - V. V regression doesn't benefit from C51 calibration,
so Q's improved knowledge stays trapped in the critic head.
Deep audit revealed a self-reinforcing defensive trap:
Q learned "big positions lose money" → π_target favors small
actions → π picks a3+a4 (tiny long / Hold) → position lots ≈ 0
→ rewards mostly 0 → V learns "everything is 0" → V_pred ≈ 0
→ advantage = returns - V_pred ≈ 0 → PPO gradient ≈ 0 → π
frozen at defensive attractor → loop. Trade count dropped 3×
(rdgzl 25k → 8xwq8/vj5f6 9k closes per 10k steps), win rate
inversely correlated with l_q (50% early → 22% late) because
only forced closes happen (stops = losses).
Fix: new rl_q_pi_distill_grad.cu computes
π_target = softmax(E_Q[s,*] / τ)
∂L/∂logits[a] = λ × (π_new(a) - π_target(a))
and ADDS this gradient to pi_grad_logits AFTER the PPO surrogate
backward. Couples Q's preferences directly into π's update without
going through advantage. λ=0.01 (small, PPO dominant), τ=1.0
(canonical Boltzmann). 3 new ISV slots (λ + τ + KL_ema diag).
Diag exposes c51_v_max/v_min, q_distill_lambda/temperature, and
q_distill_kl_ema so the adaptation + distillation loop is observable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
2d498bec3a |
feat(rl): adaptive C51 atom span ratchet to lift Q learning ceiling
rdgzl follow-up — chain hypothesis layer 2:
reward clamp lift unlocked V regression + PPO advantage (R/done
-$1.39 → -$0.48), but Q's distributional learning was structurally
capped at hardcoded V_MAX=1.0 in bellman_target_projection.cu —
any Bellman target > 1.0 categorically projected to atom 20 (top)
regardless of clamp. Even with WIN=3.8 clamp, Q never saw a +3.8
reward signal as distinct from a +1.0 reward signal.
This commit makes V_MIN/V_MAX ISV-driven with monotone-grow ratchet
coupled to the reward clamp. The C51 distribution support adapts
WITHOUT destabilising Q's learned values — atom 20 always represents
at least the widest WIN we've ever admitted (only grows, never shrinks).
Implementation:
- 2 new ISV slots (484 V_MAX, 485 V_MIN) with [-1, +1] floors
seeded by rl_isv_write
- rl_reward_clamp_controller.cu also ratchets these slots:
V_MAX_new = max(V_MAX_prev, max(1.0, WIN_clamp))
V_MIN_new = min(V_MIN_prev, min(-1.0, -LOSS_clamp))
- bellman_target_projection.cu reads V_MIN/V_MAX from ISV, derives
DELTA_Z inline (was #define)
- New rl_atom_support_update.cu (21-thread block) refreshes
atom_supports_d = linspace(V_MIN, V_MAX, 21) per step so
downstream C51 kernels (argmax_expected_q, rl_action_kernel,
dqn_distributional_q) see the current span
- Trainer launches atom-support updater after each reward-clamp
controller launch (both helper + step_with_lobsim inline paths)
- Diag exposes c51_v_max + c51_v_min for adaptation visibility
Floors at [-1, +1] preserve original C51 design as hard minimum —
the atom support can only become wider, never narrower than the
baseline.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
13084f7746 |
feat(rl): MARGIN adaptive from clip-rate + remove MAX_WIN cap
rdgzl follow-up — chain hypothesis test:
clip rate stayed at 25-40% across windows (target ~5%)
win rate oscillated 27-47% with no clear trend
positive-tail distribution: p50=1.85 p90=10.1 p99=76.9 max=2230
MAX_WIN=20 hit ceiling in EVERY window (load-bearing cap)
static MARGIN=1.5 couldn't chase the tail
Two interventions in one commit:
(1) MARGIN is now adaptive in rl_reward_clamp_controller.cu via a
Schulman bounded-step on clip-rate EMA vs target:
clip_indicator = (pos_max > current_WIN && pos_max > 0) ? 1 : 0
clip_rate_ema = (1-α) * prev + α * indicator (α=0.05)
if clip_rate_ema > target × 1.5 → MARGIN *= 1.2 (up to MAX_MARGIN=5)
if clip_rate_ema < target / 1.5 → MARGIN /= 1.2 (down to MIN_MARGIN=1)
Target clip rate seeded at 0.05 — accept 5% tail outliers, capture
the rest. Two new ISV slots (482 clip-rate EMA, 483 target).
(2) MAX_WIN cap REMOVED — the hardcoded ceiling defeated the purpose
of adaptation. Safety reasoning: WIN = MARGIN × pos_max_ema with
MARGIN ∈ [1, 5] and pos_max_ema bounded by reward_scale × raw_PnL
(both finite). MIN_WIN=1.0 floor retained.
Diag exposes clip_rate_ema + reward_clamp_clip_rate_target so the
adaptation loop is observable in the JSONL.
KNOWN DOWNSTREAM CEILING: bellman_target_projection.cu hardcodes C51
atom span at V_MIN=-1.0, V_MAX=+1.0. Any Bellman target outside this
range is categorically clipped regardless of our reward clamp. So
lifting WIN > 1.0 helps V regression + PPO advantage (which see real
magnitude) but Q's distributional learning is structurally capped at
V_MAX=1.0. A separate intervention to lift C51 V_MAX would be needed
to unlock Q's atom-distribution learning beyond +1.0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
51b9f46364 |
feat(rl): adaptive reward clamp from positive-tail EMA
alpha-rl-rmgm5 (commit
|
||
|
jgrusewski
|
a776fab31f |
fix(rl-cli): build B×K snapshot tensor per step at b_size>1
Crash in alpha-rl-ljn8k (commit
|
||
|
jgrusewski
|
3737feb664 |
audit: π drives actions (proper actor-critic) + bump b_size 1 → 16
Two coordinated architectural fixes addressing the deepest blockers
exposed by the audit:
## Option B: π-driven action selection
Per `pearl_q_thompson_actor_makes_pi_dead_weight`: the prior
architecture had Q acting as BOTH actor (via Thompson sample) AND
critic (via Bellman target). π trained by PPO surrogate against
Q's actions but never drove any decision — `q_pi_agree_ema`
decayed to 0 by step 5000 in every smoke because π converged to
Q's Thompson SAMPLING distribution, not Q's argmax. π was
dead-weight: 4 dedicated controllers (ε, ratio_clamp,
entropy_coef, KL EMA), shared encoder gradient interference, and
zero contribution to actor decisions.
### New kernel: rl_pi_action_kernel.cu
Single-thread-per-batch CUDA kernel that:
1. Computes numerically-stable softmax(pi_logits[b, :])
2. Draws u ∈ [0, 1) from per-batch xorshift32 PRNG
3. CDF-walks to pick the multinomial-sampled action
Per-batch xorshift32 PRNG state is the SAME `prng_state_d` buffer
already used by rl_action_kernel — no new state needed. Sampling
deterministic given (seed, b_size, pi_logits).
### Trainer wiring (1 site change in step_with_lobsim)
Replaced `rl_action_kernel(q_logits, atom_supports, ...)`
(Q-Thompson) with `rl_pi_action_kernel(pi_logits, ...)`
(π-multinomial). The argmax_expected_q call on h_{t+1} is
unchanged — Q remains the critic via canonical Double-DQN target.
PPO importance-ratio surrogate now has its canonical actor-critic
semantics: π_new(a|s) / π_old(a|s) where `a` was actually sampled
from π_old. Was nonsensical before (a was sampled from Q-Thompson,
not π, so the ratio measured something incoherent).
The rl_action_kernel (Q-Thompson) cubin + function field are kept
loaded for backward-compat tests and diagnostic comparison; no
longer in the hot path.
## b_size: 1 → 16
Per `pearl_b_size_1_signal_starvation_blocks_q_learning`: at
b_size=1 with 11% done-step rate and 70% loss rate per trade, Q
stayed at uniform baseline ln(21)=3.04 across all 16+ smokes
regardless of controller fixes. The architecture was structurally
signal-starved — 1 gradient sample per Adam step is fundamentally
too noisy.
LobSimCuda already supports b_size>1 (n_backtests parameter at
`crates/ml-backtesting/src/sim/mod.rs:355`). Trainer code is
already b_size-parametric throughout. The blocker was just the
CLI default at `--n-backtests=1`.
Default bumped to 16 (matches the doc note "production sweep at
32-64; L40S 48GB"). 16× more gradient samples per Adam step
gives Q proper batch variance reduction. The K-loop multiplier
(`isv[404]/2048`) will likely settle at K=1 since the
advantage_var_ratio drops with batch size.
## Expected behaviour
* `q_pi_agree_ema` becomes tautological/dropped (π IS the
policy now — comparing argmax(Q) to argmax(π) doesn't measure
a real consistency invariant any more)
* π gradient flows naturally drive π toward an actor that
optimises the PPO surrogate — Q's encoder gradient is no
longer competing with a different policy's gradient
* l_q should drop meaningfully below 3.04 for the first time
(was stuck at 2.7-2.9 across all prior smokes)
* reward/trade should approach 0 (was -$0.5 to -$0.8 across
every prior run)
* Wall-clock per env step ~16× slower (b_size=16) but training
cost per gradient step similar (denser sample = more
progress per step)
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
## Caveat: integrated_trainer_smoke runs at b_size=1
The default for the CLI is bumped to 16, but the local
`integrated_trainer_smoke` test passes its own b_size=1 to
verify the trainer mechanics. Real-world signal verification
happens via cluster smokes which now use b_size=16 by default.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
705d6c156b |
audit: ISV-ify 10 more design constants — Schulman + bootstraps + streaming α
Per `feedback_isv_for_adaptive_bounds` + user "do all except floors
and clamp bounds": 10 more constants moved from kernel-side `#define`s
into ISV slots (78 slots total now).
## Slot additions (468-477)
RL_SCHULMAN_TOLERANCE_INDEX (468, =1.5) — shared by 4 controllers
RL_SCHULMAN_ADJUST_RATE_INDEX (469, =1.5) — shared by 4 controllers
RL_STREAM_ALPHA_INDEX (470, =0.05) — shared by var + kurt streaming
RL_KURT_GAUSSIAN_INDEX (471, =3.0)
RL_KURT_NOISE_FLOOR_INDEX (472, =1.0)
RL_TAU_BOOTSTRAP_INDEX (473, =0.005)
RL_EPS_BOOTSTRAP_INDEX (474, =0.2)
RL_ROLLOUT_BOOTSTRAP_INDEX (475, =2048)
RL_REWARD_SCALE_BOOTSTRAP_INDEX (476, =1.0)
RL_PPO_RATIO_CLAMP_BOOTSTRAP_INDEX (477, =10.0)
## Skipped (per user "do all except floors and clamp bounds")
* `*_MIN`/`*_MAX` clamp bounds (algebraic domain — risk γ=1.5 nonsense)
* Numerical floors: ABS_MEAN_FLOOR=1e-6, M2_SQ_FLOOR=1e-12, EPS_PNL=1e-3
(risk div-by-zero if mis-tuned)
* C51 atom layout (V_MIN/V_MAX) — architecture, not config
## Wiring
* Shared Schulman pattern: 4 controllers (ppo_clip, target_tau,
rollout_steps, plus per_α independent KURT slots) now read TOLERANCE
+ ADJUST_RATE from the same 2 ISV slots. Single source of truth.
* Each controller's bootstrap (1st-emit on sentinel-zero) reads
isv[*_BOOTSTRAP_INDEX] instead of #define value. The `prev ==
BOOTSTRAP` first-observation replace-direct check also reads from
ISV.
* 2 streaming kernels (var + kurt) share RL_STREAM_ALPHA_INDEX.
## Diag bake-in
JSONL `isv_config` block grows by 10 new fields: schulman_tolerance,
schulman_adjust_rate, stream_alpha, kurt_gaussian, kurt_noise_floor,
tau_bootstrap, eps_bootstrap, rollout_bootstrap,
reward_scale_bootstrap, ppo_ratio_clamp_bootstrap. Total isv_config
fields: 26.
Also includes windowed action_entropy fix (was structurally 0 at
b_size=1) — accumulates EMA-smoothed action distribution over
~1k-step window, computes entropy on the windowed dist. Makes the
exploration metric meaningful at b_size=1.
## Slot total
RL_SLOTS_END: 468 → 478. **78 total ISV slots.**
## Verified gates (local sm_86)
G1 isv_bootstrap ✅ (with 10 new assertions)
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
827a0e9416 |
fix(rl): ISV-ify ALL remaining tunable design constants (10 new slots)
Per `feedback_isv_for_adaptive_bounds`: every controller design knob
that's genuinely tunable now lives in ISV instead of as a kernel-side
`#define`. Tuning is a re-seed (kernel launch with new arg) rather
than a recompile.
## New ISV slots (10 design constants)
RL_REWARD_CLAMP_WIN_INDEX (452, =1.0) apply_reward_scale
RL_REWARD_CLAMP_LOSS_INDEX (453, =3.0) apply_reward_scale
RL_KL_TARGET_INDEX (454, =0.01) rl_ppo_clip_controller
RL_IMPROVEMENT_THRESHOLD_INDEX (455, =0.99) rl_lr_controller
RL_PLATEAU_PATIENCE_INDEX (456, =1000.0) rl_lr_controller
RL_DIV_TARGET_INDEX (457, =0.01) rl_target_tau_controller
RL_ENTROPY_TARGET_FRAC_INDEX (458, =0.7) rl_entropy_coef_controller
RL_KURT_LIFT_SCALE_INDEX (459, =7.0) rl_per_alpha_controller
RL_PPO_CLAMP_MARGIN_INDEX (460, =10.0) rl_ppo_ratio_clamp_controller
RL_LR_WARMUP_STEPS_INDEX (461, =2000.0) rl_lr_controller
RL_SLOTS_END: 452 → 462.
## Constants NOT converted (truly fundamental)
* All `*_INDEX` (ABI)
* All `*_MIN`/`*_MAX` clamp bounds (algebraic domain)
* All `*_BOOTSTRAP` (one-shot init)
* `WIENER_ALPHA_FLOOR` (per pearl_wiener_alpha_floor_for_nonstationary)
* Schulman pattern parameters (`*_TOLERANCE`/`*_ADJUST_RATE`)
* C51 (`Q_N_ATOMS`, `V_MIN/MAX`, `N_ACTIONS`)
* Kernel numerics (`STREAM_ALPHA`, `ABS_MEAN_FLOOR`, `EPS_PNL`)
* `KURT_GAUSSIAN` (statistical constant = 3.0 for Gaussian)
* `KURT_NOISE_FLOOR` (defensive)
* `LR_BOOTSTRAP`/`LR_MIN`/`LR_MAX`/`LR_LOSS_EMA_ALPHA`/`DECAY_FACTOR`
## New infrastructure
New CUDA kernel `rl_isv_write.cu` — generic single-thread device-side
seeder taking `(int slot, float value)`. Trainer loops calling it
once per design constant at init. Replaces the prior pattern of
extending `rl_streaming_clamp_init`'s arg list every time a new
constant was added.
## Ordering fix
Design constants must be seeded BEFORE controllers bootstrap — the
controllers' bootstrap paths read these slots (e.g.
`rl_entropy_coef_controller` reads `RL_ENTROPY_TARGET_FRAC_INDEX`
to derive its target). Without correct ordering, controllers see
sentinel 0.0 and bootstrap to wrong values (caught by failing G1
test before commit). Seed loop runs at TOP of
`with_controllers_bootstrapped`.
## Diag bake-in
JSONL gains `isv_config` block exposing all 10 design constants per
step:
isv_config.{reward_clamp_win, reward_clamp_loss, kl_target,
improvement_threshold, plateau_patience, div_target,
entropy_target_frac, kurt_lift_scale, ppo_clamp_margin,
lr_warmup_steps}
Post-hoc analysis can correlate any controller's behaviour with the
exact design constants it saw, without grepping the source for
`#define` defaults.
## Test updates
G1 (isv_bootstrap) + G3 (r5_controllers) — skip 10 new design-
constant slots in sentinel-zero loop, assert seeded values
separately.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅ (with 10 new assertions)
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
644fbe0348 |
fix(rl): ISV-driven K-loop divisor + max ceiling (slot 450, 451)
f2ggr confirmed K-loop wiring works mechanically but K=8 firing on
22 % of steps over-trained at b_size=1: KL excursions to 12.44
(vs prior 3.4e-4), policy overshoot, reward/trade -$0.585 → -$0.723.
Per `feedback_isv_for_adaptive_bounds` the K-loop config must live
in ISV, not as hardcoded values in the trainer. Two new slots:
RL_K_LOOP_DIVISOR_INDEX (450) — divides n_rollout_steps to get K
Default 2048 (matches ROLLOUT_BOOTSTRAP
so K=1 at controller bootstrap)
RL_K_LOOP_MAX_INDEX (451) — clamp ceiling on K
Default 4 (was hardcoded 8; halved
to prevent gradient overtraining)
K computation in step_with_lobsim now reads both from ISV:
K = clamp(isv[404] / isv[450], 1, isv[451])
Halves worst-case overtraining while preserving the controller
cascade activation (KL above noise floor, ε actively adapting,
ratio_clamp firing). Distribution shifts from K=8 @ 22% → K=4 @ 22%
(half the gradient updates in the high-noise case).
## Wiring
`rl_streaming_clamp_init.cu` extended to seed 5 ISV-resident design
constants (was 3): adv_var_clamp, td_kurt_clamp, adv_var_target,
k_loop_divisor, k_loop_max. Still one kernel call, no HtoD.
## Diag bake-in
JSONL `k_updates` field replaced with `k_loop` block:
k_loop.k_updates — actual K used this step
k_loop.divisor — current divisor (reads isv[450])
k_loop.max — current max (reads isv[451])
Post-hoc analysis can verify the K-computation by independently
recomputing K from isv[404] / k_loop.divisor.
## Slot allocation
RL_SLOTS_END: 450 → 452 (+2 new config slots).
## Test updates
G1 + G3 skip slots 450, 451 in sentinel-zero loop and assert seeded
values (2048.0 + 4.0) separately.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
1d8ef94848 |
fix(rl): wire n_rollout_steps as K-loop + raise LR_MIN to 1e-4
Two coordinated fixes for the alpha-rl-frt7s findings:
## Issue 1: n_rollout_steps controller was write-only
ISV consumer audit confirmed: 7 of 8 RL controllers had a non-
controller consumer in the per-step path; n_rollout_steps had ZERO.
The controller adapted its output between 256-8192 but nothing read
it. Bit-identical losses between cvf86 and frt7s confirmed: even
fixing the target (0.1 → 5.0) and putting the controller into
healthy HOLD/SHRINK/WIDEN distribution had zero behavioral impact
because no downstream code gated on the emitted value.
### Fix: wire as DQN-replay + PPO+V K-loop multiplier
step_with_lobsim now wraps (sample_and_gather + step_synthetic +
PER priority update) in a K-loop where:
K = clamp(isv[RL_N_ROLLOUT_STEPS_INDEX] / 1024, 1, 8)
Mapping:
* isv[404] = 256 (MIN) → K = 1 (current behavior)
* isv[404] = 2048 (BOOTSTRAP) → K = 2
* isv[404] = 8192 (MAX) → K = 8
Each iteration re-samples PER (different transitions per Adam step)
and runs full Q + π + V forward + backward + Adam. Adapts the
training:env ratio so noisy-advantages regimes get more gradient
samples per env step without slowing env stepping. Directly
addresses the b_size=1 gradient starvation that left l_q stuck at
2.82 in frt7s.
Semantic fit: n_rollout_steps's design intent ("noisy advantages →
need more samples per update") now drives "more training updates
per env step" — equivalent semantics, fits the b_size=1
architecture without requiring a PPO rollout buffer refactor.
`last_k_updates` field tracks the per-step K value for diag.
## Issue 2: LR plateau-decay Q-lock
frt7s deep dive showed:
* Q best=2.3230 locked at step ~783 from a brief downward
excursion during early-training noise
* loss_ema range across 50k steps: [2.323, 3.113]; mean 2.819,
std 0.104
* ZERO steps had loss_ema < best in entire run (let alone <
best × 0.99 = 2.30 threshold)
* 7 LR halvings drove all heads to LR_MIN = 1e-5 by step 7783
* At 1e-5, Q's per-step Adam update is too small to escape;
l_q stayed at ~2.82 for 42k more steps
The plateau-decay is CORRECTLY identifying "model has stopped
improving" — the fix isn't to make plateau detection less
sensitive (loosening threshold to 0.95/0.90 still finds zero
improvements). The fix is to raise the floor LR so the model
has enough learning rate to escape the noise-locked best.
### Fix: LR_MIN 1e-5 → 1e-4 + WARMUP_STEPS 500 → 2000
* LR_MIN raised 10× — even at the plateau-decay floor the model
gets meaningful gradient. Still 10× below LR_BOOTSTRAP=1e-3
so the controller has full dynamic range.
* WARMUP_STEPS raised 4× — gives loss_ema 2000 observations
(≈145 EMA half-lives at α=0.05) to settle BEFORE best is
locked. Prevents the "lucky early excursion locks unreachable
bar" failure mode.
## Diag bake-in
JSONL gains `k_updates` field (per-step K value from the n_rollout
loop) so post-hoc analysis can correlate the K-multiplier with
loss trajectories.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
## Quality-first scope decision
User requested "quality over speed". Considered alternatives:
* Building a proper PPO rollout buffer (Issue 1) — significant
refactor, ~1-2 days. K-loop interpretation chosen instead
because it (a) matches the controller's design intent, (b)
requires no buffer/gradient-accumulation infrastructure, (c)
directly addresses Q learning starvation by giving more
gradient samples per env step.
* Encoder LR decoupling (Issue 2) — encoder receives gradient
from all head backward kernels with their own LRs; treating
the encoder separately would require restructuring all
backward kernels. LR_MIN raise + WARMUP extension gives the
same benefit at the head level without that scope.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
95dcc4e312 |
fix(rl): ISV-driven ADV_VAR_RATIO_TARGET for rl_rollout_steps_controller
cvf86 controller_branch diag (commit
|
||
|
jgrusewski
|
708c121f20 |
fix(rl): bounded multiplicative step + noise-floor on rollout_steps + per_α
kc2h9 confirmed: clamping streaming-kernel outputs to [≤100, ≤30]
had ZERO behavioral impact because rl_rollout_steps_controller's
prior design used `scale = clamp(input/target, 0.5, 2.0)` — the
scale saturated to ±2× on the SIGN of (input − target), not the
magnitude. With target=0.1 and typical input=1–10 the controller
slammed to MAX in ≤4 steps regardless of whether input was 4 or
3e5. Bit-identical losses between gxhr8 and kc2h9 confirmed the
saturation.
## Fix 1: rl_rollout_steps_controller — same Schulman pattern as ppo_clip
* input > TARGET × 1.5 → scale = 1.5 (widen)
* input < TARGET / 1.5 → scale = 1/1.5 (shrink)
* in-band → scale = 1.0 (hold)
* input < TARGET × 0.01 → return (noise floor — hold prev)
Per-step adjustment bounded at 1.5×, so rollout_steps drifts
smoothly toward MIN/MAX rather than slamming there. The noise-floor
gate matches the pattern from
`pearl_multiplicative_controllers_need_bounded_step_and_noise_floor`
applied to the ε and τ controllers earlier in R9.
## Fix 2: rl_per_alpha_controller — noise-floor gate (defensive)
per_α uses a LINEAR lift `0.4 + 0.2·(kurt-3)/7` (not multiplicative),
so it doesn't have the saturation bug. But added a noise-floor gate
at KURT_NOISE_FLOOR = 1.0 so a sub-Gaussian kurtosis reading from
the streaming estimator's startup window (when per-step batch-mean
deviations are small before tails develop) doesn't drag α toward
PER_ALPHA_MIN on cold-start.
## Diag bake-in (per user request "bake in diags")
JSONL gains a `controller_branch` block exposing the
multiplicative-controller inputs alongside their design targets:
controller_branch: {
rollout_steps_input: isv[421], rollout_steps_target: 0.1,
ppo_clip_input: isv[419], ppo_clip_target: 0.01,
target_tau_input: isv[418], target_tau_target: 0.01,
per_alpha_input: isv[422], per_alpha_target: 0.6,
}
Post-hoc analysis can compute the branch each step (WIDEN / HOLD /
SHRINK / NOISE) by comparing input/target against the ±33%
tolerance band, revealing whether each controller is being driven
by real signal or sitting in the in-band hold zone. Targets are
reflected from the kernel #defines (synchronised by code review at
the controller-cu file level — there's no ISV slot for these
design constants because they're fundamental to the controller's
behaviour, not adaptive).
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
66115007ab |
fix(rl): ISV-driven output clamp on streaming var/kurtosis kernels
gxhr8 confirmed the streaming kernels work — both formerly-dead
controllers (rl_rollout_steps, rl_per_alpha) now adapt instead of
pegging at MIN. But the unclamped streaming outputs reached
advantage_var_ratio = 3e5 (when streaming-mean passed through zero
and `var/|mean|` blew up under the 1e-6 denominator floor) and
td_kurtosis = 50.6, pegging both downstream controllers at MAX
instead. Per_α at MAX over-concentrates PER sampling on outliers,
which hurts distributional Q learning (best l_q window regressed
from 2.41 → 2.69 between pdgxn and gxhr8).
## Fix: ISV-resident output clamp ceilings
Two new ISV slots hold the streaming-kernel output ceilings:
RL_ADV_VAR_RATIO_CLAMP_INDEX = 447 (default 100.0)
RL_TD_KURTOSIS_CLAMP_INDEX = 448 (default 30.0)
* 100.0 for var_ratio = 1000× ADV_VAR_RATIO_TARGET (= 0.1) — wide
enough that healthy signal (typical 1-10) passes through, tight
enough that 3e5 outliers don't peg rollout_steps.
* 30.0 for kurtosis = 3× (KURT_GAUSSIAN + KURT_LIFT_SCALE) — lets
the full per_α response range engage on heavy-tailed signal
(≤ 10), bounds runaway above that.
Per `feedback_isv_for_adaptive_bounds`: the clamps live in ISV
(visible in diag, modifiable at runtime via re-launching the init
kernel or a future adaptive controller) rather than as kernel-side
`#define`s.
## Seeding (no HtoD per feedback_no_htod_htoh_only_mapped_pinned)
New device kernel `rl_streaming_clamp_init.cu` — single thread,
writes both clamp ceilings directly to ISV. Launched once at the
end of `with_controllers_bootstrapped` alongside the 8 existing
controller-bootstrap launches. Zero host→device transfer.
## Diag bake-in (per user request "ensure to bake in diags")
JSONL gains a new `streaming` block exposing:
* `streaming.adv_var.{mean, m2, clamp}`
* `streaming.td_kurt.{mean, m2, m4, clamp}`
Cross-check: when consumer-input slot (RL_ADVANTAGE_VAR_RATIO_EMA_INDEX
or RL_TD_KURTOSIS_EMA_INDEX) reads exactly the same value as
`streaming.*.clamp`, the clamp fired this step.
## Test updates
G1 (isv_bootstrap) + G3 (r5_controllers) blanket-assert that
ISV[417..END] is sentinel-zero at bootstrap. Both new slots are
seeded to non-zero values by rl_streaming_clamp_init during
bootstrap, so both tests skip these slots in the loop and assert
the seeded values separately.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
53aeef099b |
feat(rl): ISV-driven PPO importance-ratio clamp + log-ratio diagnostic
pt67l confirmed reward-scale + V-target clamp eliminate V regression
spikes — but exposed a residual: |l_pi| max=586 with mean 0.22. Root
cause: PPO's clip(r, 1-ε, 1+ε) bounds the loss only when surr2 is
the active min. The unclipped branch IS active when A<0,r>1+ε
(surr1=A·r is then more negative than surr2=A·(1+ε), so min selects
surr1) and when A>0,r<1-ε. In the first case `r` can blow up: we've
seen r reach 1e10 from policy drift over a multi-step rollout
producing l_pi=O(1e10) spikes that contaminate the loss-balance
controller and the LR controller's plateau detection.
## Fix: ISV-driven ratio clamp
Per `feedback_isv_for_adaptive_bounds` and
`pearl_controller_anchors_isv_driven`: the clamp ceiling lives in
ISV[RL_PPO_RATIO_CLAMP_MAX_INDEX = 440], not as a hardcoded #define.
New controller `rl_ppo_ratio_clamp_controller.cu`:
* Anchors on the (already KL-adaptive) PPO clip ε at ISV[402]
* target = (1 + ε) × PPO_CLAMP_MARGIN (MARGIN = 10.0)
* Wiener-α blend with floor 0.4 per
pearl_wiener_alpha_floor_for_nonstationary (ε is non-stationary)
* Permanent floor 2.0 / ceiling 1000 per
pearl_blend_formulas_must_have_permanent_floor
* Bootstrap 10.0, replace-directly on first non-bootstrap ε
observation per pearl_first_observation_bootstrap
When ε is small (rl_ppo_clip_controller seeing low KL → tight clip
band), the ratio clamp tightens — outliers should be rare anomalies.
When ε widens (large KL → wide clip band), the clamp widens
proportionally — outliers are expected so we permit more
magnitude before bounding.
## Wiring
ppo_clipped_surrogate_fwd and _bwd both read
isv[RL_PPO_RATIO_CLAMP_MAX_INDEX] and clamp ratio to
[1/ratio_max, ratio_max] before forming surr1/surr2. The clamp is
forward-only in effect (bwd gates pg_grad inside [1-ε, 1+ε] anyway
so gradients were already bounded), but bounding the FORWARD ratio
keeps l_pi sane for the controllers downstream.
The new controller is wired into both:
* `with_controllers_bootstrapped` — bootstrap launch alongside
the other 7 R1 controllers
* `launch_rl_controllers_per_step` — per-step refresh alongside
the other 7 R5 controllers
## Diagnostic: per-step max |log_ratio|
New kernel `ppo_log_ratio_abs_max_b.cu` (same tree-reduce shape as
rl_kl_approx_b) writes per-batch max(|log π_new − log π_old|) to
ISV[RL_PPO_LOG_RATIO_ABS_MAX_INDEX = 441]. Launched right after
rl_kl_approx_b (uses the same log_pi_old_d + pi_log_prob_d inputs).
Surfaces in diag JSONL as:
"ppo": {
"ratio_clamp_max": isv[440], # adaptive ceiling
"log_ratio_abs_max": isv[441] # per-step observed max
}
The clamp fires when log_ratio_abs_max > ln(ratio_clamp_max).
For ratio_clamp_max = 10, ln = 2.30. Healthy training has
log_ratio_abs_max well below this most steps; outliers touch or
exceed it on rare excursions which the clamp bounds before they
pollute l_pi.
## Slot allocation
RL_PPO_RATIO_CLAMP_MAX_INDEX = 440 (controller output)
RL_PPO_LOG_RATIO_ABS_MAX_INDEX = 441 (per-step diag)
RL_SLOTS_END = 442 (was 440)
## Test updates
G1 (isv_bootstrap) + G3 (r5_controllers) blanket-assert ISV[417..END]
== 0.0 to catch slot-wiring bugs. Slot 440 is now a controller
OUTPUT bootstrapped to 10.0, so both tests skip it in the loop and
assert == 10.0 separately.
## Verified gates (local sm_86)
G1 isv_bootstrap ✅ (with new slot-440 assertion)
G3 controllers ✅
G4 target_update ✅
G6 r7d_per_wiring ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
20c7852b66 |
fix(rl): asymmetric clamp on scaled reward + pre-clamp |max| diag
The xv66n smoke (commit
|
||
|
jgrusewski
|
d5c29fb4fa |
fix(rl): warmup window in plateau-decay LR controller fixes V cold-start
`alpha-rl-rzltn` exposed a bug in the plateau-decay design: V head's
`best` got bootstrapped to 7.12e-10 (machine epsilon) at step 1
because V regression had no reward signal yet — no trade had closed,
the bootstrap V target was 0, so the first V loss was effectively 0.
Every subsequent V loss EMA was orders of magnitude higher (4.07
at step 100, 1.15 at step 1000), so the improvement check
`loss_ema < best * 0.99` evaluated false FOREVER. The controller
then decayed lr_v every 1000 steps purely on the patience clock,
not because the model genuinely plateaued.
Cross-check across the 50k-step rzltn run:
* V best unique values: {0.0, 7.12e-10} — ONLY 2 across 50000 rows
* V best max: 7.12e-10
* V best-improvements: 0 (Q: 12, π: 12)
* V decays still fired: 7 (one every 1000 steps from step 1001)
The plateau-decay mechanics worked correctly — the controller counted
to 999 then halved LR exactly as designed. The bug was that "first
observation defines best forever" is degenerate for sparse-signal
heads whose first loss is a cold-start artifact.
## Fix: LR_WARMUP_STEPS
Three new ISV slots (one per head — Q, π, V at 436/437/438) hold a
monotonic warmup counter clamped at LR_WARMUP_STEPS = 500. During
warmup the controller:
* always overwrites `best` with current loss_ema (tracks the EMA
as it converges)
* holds the plateau counter at 0 (no decay fires during warmup)
* increments warmup_counter
Once warmup_counter >= LR_WARMUP_STEPS, the controller switches to
standard plateau detection — `best` then locks in at the
post-warmup loss_ema value (representative of the head's converged
loss scale), and patience counting begins.
At α=0.05 the EMA half-life is ~14 steps; 500 updates leaves ~35
half-lives, well past convergence. This gives V time to see its
first actual losses after trades start closing.
## Slot allocation
RL_SLOTS_END: 436 → 439 (adds 3 warmup counter slots).
## Wiring
* rl_lr_controller.cu — adds warmup_slot param to
plateau_decay_head, kernel takes 12
slot ints (was 9)
* isv_slots.rs — 3 new constants, RL_SLOTS_END += 3
* integrated.rs — launch_rl_lr_controller passes 12
slot ints
* alpha_rl_train.rs — diag JSONL emits new
lr_plateau.{head}.warmup field
## Verified gates (local sm_86)
G1 isv_bootstrap ✅
G3 controllers ✅
G4 target_update ✅
G6 r7d_per_wiring ✅
integrated_smoke ✅
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
13d81dc5e6 |
diag(rl): emit grad_norm_ema + lr_plateau state in alpha_rl_train JSONL
Adds two new top-level keys to each diag.jsonl row:
"grad_norm_ema": {q, pi, v} — slots 424-426
"lr_plateau": {q,pi,v} × {loss_ema, best, stale} — slots 427-435
With these in place we can independently verify each plateau-decay
event in `mjgsj`'s diag (and all future runs):
* `loss_ema` traces the controller's slow EMA of head loss
(α=0.05); confirms the EMA actually moves and isn't stuck on the
bootstrap zero
* `best` shows the rolling minimum the controller compares against;
confirms it improves early then plateaus
* `stale` is the steps-since-best counter; should hit
PLATEAU_PATIENCE = 1000 exactly when an LR halving fires; reset to
0 after every decay event or every improvement
The `grad_norm_ema` block is kept because the grad-norm producers are
still wired (commit
|
||
|
jgrusewski
|
87a22d12c9 |
feat(rl): walk-forward G8 eval phase + fold split (MVP, manual fan-out)
Adds the minimum-viable implementation of the R9 multi-fold G8 gate
per `pearl_single_window_oos_is_not_oos` ("a single window is NOT
out-of-sample"). The trainer can now:
1. Slice the MBP-10 file list into K equal-sized blocks
(`--n-folds K --fold-idx k`).
2. Train on blocks [0..=k] (passed to MultiHorizonLoader).
3. Run a separate eval phase of `--n-eval-steps` on block [k+1]
using a second loader instance.
4. Drain LobSim trade records gated by a pre-eval head checkpoint
so train-phase trades don't contaminate the eval summary.
5. Compute profit_factor + sharpe + drawdown via existing
`ml_backtesting::artifacts::compute_summary`.
6. Write `eval_summary.json` alongside `alpha_rl_train_summary.json`.
## Manual fan-out (this MVP)
The dispatcher (`scripts/argo-alpha-rl.sh`) gains three new flags
that thread through the Argo template into the CLI: `--fold-idx`,
`--n-folds`, `--n-eval-steps`. To run a 3-fold G8:
./scripts/argo-alpha-rl.sh --n-folds 3 --fold-idx 0 --n-eval-steps 200
./scripts/argo-alpha-rl.sh --n-folds 3 --fold-idx 1 --n-eval-steps 200
(With n_folds=3 the valid fold indices are 0 and 1 — the third block
is the eval window for fold 1. n_folds=K accepts fold_idx ∈ [0, K-2].)
Each submission produces one `eval_summary.json` at the resolved
output dir; the per-fold profit_factor is the value to aggregate.
Manual aggregation for now — automated DAG matrix fan-out + an
in-cluster aggregator pod is a follow-up commit. The aggregator
will mean ± SD the per-fold PFs and gate on `PF > 1.0`.
## What's NOT pure eval
The eval loop calls `step_with_lobsim` (same as train) — Adam steps,
PER updates, controller adaptations all still fire during eval. At
b_size=1 the per-step learning effect is small relative to the
train-phase-accumulated policy, so the eval PF approximates the
OOS performance of the train-end policy. A clean pure-eval mode
(forward + LobSim step only, no backward/Adam/PER) is a follow-up
architectural change; documented inline at the eval phase block.
## Default behaviour unchanged
`--n-folds=1` (default) skips the eval split entirely and uses all
files for training — identical to the prior single-window smoke.
The R9 prior smokes ran in this mode. Default `--fold-idx=0` and
`--n-eval-steps=0` keep prior smoke runs binary-compatible.
## Template + dispatcher changes
* `alpha-rl-template.yaml`: adds 3 new workflow parameters
(`fold-idx`, `n-folds`, `n-eval-steps`) and threads them into
the train container's `alpha_rl_train` invocation.
* `argo-alpha-rl.sh`: adds matching CLI flags with explicit
documentation of the multi-fold dispatching pattern.
## Verified gates
Local sm_86 build + dispatcher syntax clean. Tests unchanged
(the walk-forward path is exercised by cluster smokes, not unit
tests — the loader-slicing logic is straightforward index math).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
ce1e13519b |
fix(rl): mapped-pinned for all R7d/R8 CPU↔GPU paths + diag JSONL + guard
Two concerns in one commit since they're entangled:
## 1. feedback_no_htod_htoh_only_mapped_pinned violations
R7d (PER push/sample) + R8 (CLI binary) + the new per-step diag dump
shipped with 8 raw `stream.memcpy_htod` / `stream.memcpy_dtoh` calls.
The rule is explicit: "mapped-pinned only for CPU↔GPU; tests not
exempt." A raw `stream.memcpy_*` on a regular `&[T]` / `&mut [T]` is
NOT mapped-pinned — the source/dest slice isn't page-locked, so the
CUDA driver does an internal blocking HtoD/DtoH that stalls the
stream.
Refactored all 8 violations to use the mapped-pinned + DtoD pattern
(cuMemHostAlloc DEVICEMAP — host writes via `host_ptr`, kernel reads
`dev_ptr`, DtoD between them via `cudarc::driver::result::memcpy_dtod_async`).
New shared helpers in `trainer/integrated.rs`:
* `read_slice_i32_d` — DtoH for `i32` device buffers via
`MappedI32Buffer` staging. Counterpart to the existing
`read_slice_d` (f32 version).
* `write_slice_f32_d` — CPU→GPU upload for `f32` via
`MappedF32Buffer.write_from_slice` + DtoD into destination.
* `write_slice_i32_d` — CPU→GPU upload for `i32` via
`MappedI32Buffer.host_slice_mut().copy_from_slice` + DtoD.
`pub fn` wrappers (`read_slice_*_d_pub`) expose the f32/i32 helpers
to the CLI binary so the per-step diag DtoH uses the same canonical
pattern.
Call-site refactors:
* `push_to_replay`: 4× `stream.memcpy_dtoh` → `read_slice_*_d`.
* `sample_and_gather`: 3× `stream.memcpy_htod` → `write_slice_*_d`.
* `step_with_lobsim` pre-PER ISV refresh: raw `memcpy_dtoh` →
`read_slice_d` (424 floats per step).
* `step_with_lobsim` post-Q PER priority TD readback: raw
`memcpy_dtoh` → `read_slice_d` (b_size floats per step).
* `step_synthetic` ISV mirror refresh: raw `memcpy_dtoh` →
`read_slice_d` (pre-existing pre-R9 violation; fixed in the
same commit since it's the same pattern in the same file).
* Init-time (one-shot) `prng_state` upload: raw `memcpy_htod` →
inline mapped-pinned DtoD (custom because cast through i32 for
the u32 buffer).
* Init-time (one-shot) `atom_supports` upload: raw `memcpy_htod`
→ `write_slice_f32_d`.
* `examples/alpha_rl_train.rs` per-step diag DtoH (3 calls) →
`read_slice_*_d_pub`.
## 2. Pre-commit guard gap — diff-aware HtoD/DtoH check
The existing GPU hot-path guard (`scripts/gpu-hotpath-guard.sh`)
EXPLICITLY skips memcpy_htod/dtoh on the assumption that such calls
only appear in `cuda_pipeline/` (where mapped-pinned is the
convention). That assumption was falsified by R7d/R8 — the guard
shipped 8 violations green.
Added `check_no_raw_htod_dtoh` to `scripts/pre-commit-hook.sh` (the
real file behind the `.git/hooks/pre-commit` symlink). The check is
DIFF-AWARE: it greps only the `+` lines of `git diff --cached -U0`,
so pre-existing violations elsewhere (143 sites across the codebase)
don't block commits touching unrelated files. NEW additions of
`\.memcpy_(htod|dtoh)\(` are flagged with a clear error pointing at
the mapped-pinned alternative. Suppress per-line with `// gpu-ok:
<reason>` (same convention as the existing guards).
Pre-existing violations in `ml-alpha/src/aux_heads.rs`,
`mamba2_block.rs`, `cfc/`, `data/`, etc. are a separate cleanup —
not blocked by this commit's check because the diff-aware filter
ignores anything that was already on `HEAD~1`.
## Verified gates (post-fix, local sm_86)
G1 isv_bootstrap ✅ unchanged
G3 controllers_emit ✅ unchanged
G4 target_soft_update ✅ unchanged
G6 r7d_per_wiring ✅ unchanged (PER round-trips all
mapped-pinned now)
R3 ema/advantage (3 tests) ✅ unchanged
R4 action kernels (3 tests) ✅ unchanged
end integrated_trainer_smoke ✅ unchanged
Mapped-pinned is semantically equivalent to raw memcpy_htod/dtoh —
just routed through page-locked staging so the driver doesn't have
to do its own internal pinning. Behaviour identical; the cost shifts
from "driver hidden HtoD per call" to "mapped-pinned alloc + DtoD
per call." For the smoke (b_size=1, 1000 steps), the cost difference
is in the microseconds.
## Per-step diag JSONL (separate concern, same commit)
Added `--diag-jsonl <PATH>` flag to `alpha_rl_train.rs` (default:
`<out>/diag.jsonl`). After each `step_with_lobsim`, writes one JSON
record capturing:
* step number, elapsed wall time
* all 5 head losses + λs
* all 7 RL controller outputs (γ τ ε coef n_roll per_α scale)
* all 5 per-head learning rates (lr_bce/q/pi/v/aux)
* all 7 EMA inputs the controllers consume
* replay buffer length
* per-step reward stats (sum, max, min, abs_max)
* per-step done count
* per-step action histogram (9 action classes)
Critical for cluster smoke debugging — the prior CLI only flushed
an `eprintln` progress line every N steps (default 100), making
in-flight controller drift / replay stagnation / reward explosion
invisible until they produced a NaN abort. The JSONL is line-
buffered + flushed every `log_every` steps so `tail -f` shows
progress live.
The stderr progress line is also beefed up to include γ / ε / per_α /
reward_scale / dones / rew_sum at each tick so a casual `argo logs`
inspection sees the controller behaviour without parsing JSONL.
## Why R9 cluster submission needs this
Without the diag dump, an R9 1000-step smoke is "blind" — only the
final summary tells us what happened. With the dump, post-hoc
analysis can answer:
* Did the controllers adapt or stay at bootstrap?
* Did the reward scale stabilise or saturate?
* Did the PER buffer fill?
* Was the action histogram dominated by any one action?
* Where did the per-head losses converge to?
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
1168f3ea83 |
feat(rl): R8 — alpha_rl_train CLI + Argo template + dispatcher
Closes the rebuild plan's R8 scope: production runner shape for the
integrated RL trainer. Three artifacts wired end-to-end:
1. `crates/ml-alpha/examples/alpha_rl_train.rs` — clap CLI driving
`IntegratedTrainer::step_with_lobsim` against MBP-10 windows
loaded via `MultiHorizonLoader::next_sequence_pair` (R2) for
true `(s_t, s_{t+1})` adjacency. Per `feedback_mbp10_mandatory`,
`--mbp10-data-dir` is required — no synthetic-data fallback in
the production path.
2. `infra/k8s/argo/alpha-rl-template.yaml` — WorkflowTemplate
mirroring alpha-perception's DAG (check-cache → ensure-binary →
train; warmup-gpu parallel). Binary cache slot is
`/data/bin/<sha>/alpha_rl_train` (distinct from `alpha_train`
so the two binaries coexist at the same SHA).
3. `scripts/argo-alpha-rl.sh` — dispatcher with three rebuild-plan
guards baked in.
## Dispatcher guards (per the rebuild plan's feedback list)
`feedback_default_to_l40s_pool` (2026-05-09): default `--gpu-pool`
is `ci-training-l40s` (sm_89). H100 (sm_90) is opt-in for production
scale-up only. Cubins must match the device, so the dispatcher
derives `cuda-compute-cap` from the pool name and threads it into
the workflow params.
`feedback_argo_template_must_apply` (2026-05-21 canonical incident):
`argo submit --from=wftmpl/<name>` reads the cluster CRD, NOT the
on-disk YAML; unknown `-p` parameters silently no-op without a prior
`kubectl apply`. Dispatcher applies the local template BEFORE every
submission (overrideable via `--skip-template-apply` for the rare
case where you've already applied manually).
`feedback_push_before_deploy` (2026-05-20 canonical incident): the
in-cluster `ensure-binary` pod fetches source from `origin/<branch>`,
NOT the local working tree. Submitting before `git push` deploys the
last-pushed SHA, which can lag local diff by N commits. Dispatcher
verifies `git rev-parse HEAD == git rev-parse origin/<branch>` and
hard-errors with the explicit push command otherwise. Bypass via
`--skip-push-check` (only when intentionally deploying a previously-
pushed SHA via `--sha`).
## CLI: gate G8 (NaN abort)
Per `feedback_stop_on_anomaly` + `feedback_kill_runs_on_anomaly_quickly`,
the CLI checks every per-head loss (l_bce / l_q / l_pi / l_v / l_aux
/ l_total) for finiteness after each `step_with_lobsim` call.
Non-finite at any step → write summary with `nan_abort_step` set →
`process::exit(2)`. R9's cluster smoke tail-watcher kills the
workflow on the non-zero exit code, satisfying gate G8 from the
rebuild plan.
## CLI: knobs that ARE on the CLI
Structural / boundary parameters only (per
`pearl_controller_anchors_isv_driven`: every adaptive knob lives in
ISV, not CLI flags):
* `--mbp10-data-dir / --predecoded-dir / --out` — I/O paths.
* `--n-steps` — wall-budget control (1000 R9 smoke / 50k+ prod).
* `--seq-len / --n-backtests / --per-capacity` — structural
sizing. seq_len threads into the loader's multi-resolution
`1:<seq_len>` config; n_backtests into both LobSimCuda and
PerceptionTrainerConfig.n_batch.
* `--seed` — reproducibility per
`pearl_scoped_init_seed_for_reproducibility` (forks deterministic
sub-seeds for dqn / ppo / per).
* `--instrument-mode` — MBP-10 filter (all / front-month / id=N).
* `--gpu-idx` — CUDA device selection.
What's NOT on the CLI: γ / τ / ε / entropy_coef / per_α / reward_scale
/ per-head LRs — all live in ISV[400..417] and are driven by R5's
controllers from EMA-tracked diagnostics. Per the rebuild plan
A1: "every adaptive bound is signal-driven, not tuned."
## Cluster smoke entry point
```bash
# R9 validation smoke (after pre-cluster local CUDA tests green).
./scripts/argo-alpha-rl.sh --n-steps 1000 --instrument-mode front-month
# Production scale-up (gated by R9's multi-fold pass).
./scripts/argo-alpha-rl.sh --n-steps 50000 --n-backtests 32 \
--per-capacity 100000 # GPU sum-tree R-future when capacity > 4096
```
## What's NOT in this commit
The R9 cluster smoke run itself is out of band — this commit ships
the entry points. R9 will execute the pre-cluster validation
checklist + first 1000-step smoke + multi-fold walk-forward G8 gate
per the rebuild plan §"Cluster smoke discipline".
The summary JSON's schema is intentionally narrow (final-step losses
+ replay len + completion state + NaN abort marker). R-future may
add per-epoch breakdowns + per-ISV-slot snapshots once the cluster
smoke tells us which diagnostics are actually load-bearing for kill
decisions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
f68e0a1d0d |
revert(loader): multi-resolution default '1:32' (single-scale) after htpp6 falsification
The Phase 1 multi-resolution layout (10 raw + 10 agg@30 + 12 agg@100) regressed ALL horizons in alpha-perception-htpp6 (2026-05-22): - auc_h100: 0.681 -> 0.512 (-0.169) - auc_h300: 0.617 -> 0.506 (-0.111) - auc_h1000: 0.576 -> 0.526 (-0.050) Hypothesis falsified. Root cause: Mamba2+CfC SSM encoder already used all 32 raw ticks effectively via state-recurrence; replacing 22 raw ticks with arithmetic-mean aggregates destroyed within-window microstructure variance (the actual h100 signal) AND broke temporal continuity that the recurrence relies on. Δt Fourier encoder couldn't compensate. Architectural pearl: SSM/RNN/CfC + multi-resolution input is incompatible without separate-encoder-per-scale or explicit scale tokens. Transformer-style positional encoding tolerates scale-mixing; recurrent state updates assume consecutive positions. Reverts default to '1:32'. Adds explicit single_scale_32() constructor for callers (harness, tests). Keeps default_three_scale() in code with deprecation note for future sub-variant experiments. Production defaults across alpha_train CLI, Argo template, dispatcher script, ml-backtesting harness now match the proven baseline. |
||
|
jgrusewski
|
7e438aeba0 |
feat(alpha_train): --multi-resolution CLI flag
Replaces --seq-len. Default '1:10,30:10,100:12' = 32 positions covering 1510 ticks of context, the Phase 1 fix for temporal receptive field mismatch. Logs the active config at preload start so Argo logs surface the per-run choice. |
||
|
jgrusewski
|
78a9e08358 |
feat(loader): InstrumentFilter::FrontMonth for cross-quarter ES.FUT data
Replaces Option<u32> instrument_id_filter with InstrumentFilter enum {All,
Id(u32), FrontMonth}. FrontMonth runs a two-pass detect over the DBN
stream: pass 1 counts instrument_ids and collects SymbolMapping records,
picks the dominant id, validates it resolves to an ES contract via regex
ES[FGHJKMNQUVXZ]\d{1,2}; pass 2 streams the filtered records.
Motivated by alpha-perception-k54wd: a single-id filter on parent-symbol
ES.FUT data caught Q1 2024 (kept=73M) but kept=0 for Q2-Q9 because ES
front-month rolls quarterly (ESH4 -> ESM4 -> ESU4 -> ESZ4 ...). FrontMonth
self-tunes across the rolls without needing a per-file id table.
Sidecar keys distinguish modes: mbp10 / mbp10_instr<id> / mbp10_front_month.
CLI flag renamed --instrument-id -> --instrument-mode {all,id=N,front-month}
with matching parameter rename in argo-alpha-perception.sh + template.
|
||
|
jgrusewski
|
783297e002 | feat(loader): instrument_id filter + outdated test fix + sp18 fingerprint | ||
|
jgrusewski
|
f065cfbaa2 | diag(aux): log pos_fraction once at epoch=0, step=0 (diagnose pw bit-identity) | ||
|
jgrusewski
|
0f5d5c7b4a |
feat(aux-supervision): wire BCE + conditional-Huber actual calls (CB5)
CB3+CB4 shipped the kernels with a holding-pattern (grad-zero) call site.
CB5 wires the real calls + per-loss ISV signals.
perception.rs:
- step_batched body: pos_weight computed host-side from
self.last_pos_fraction (n_neg/n_pos clamped to [1.0, 50.0] per E3),
uploaded mapped-pinned to stg_aux_pos_weight_{long,short}
- 4 kernel calls per step (slab mode, K*B*N_AUX_HORIZONS):
* aux_bce_loss_gpu × 2 (prof_long, prof_short) — class-weighted
* aux_huber_masked_loss_gpu × 2 (size_long, size_short) — NaN-mask
from CB1's y_size=NaN at y_prof=0 gives conditional-Huber for free
- Per-loss EMAs replace single aux_huber_ema:
* aux_prof_bce_ema_per_h (BCE EMA per horizon)
* aux_size_huber_ema_per_h (Huber EMA per horizon)
* aux_dir_acc_ema_per_h (unchanged)
- Stop-grad lift condition: aux_prof_bce_ema < 0.4 AND aux_dir_acc > 0.85
for ALL horizons (uses BCE not Huber per E3 — size Huber is
observability-only since the regression scale varies more than the
binary classification quality)
- aux_lift_huber_threshold renamed to aux_lift_prof_bce_threshold
alpha_train.rs:
- AlphaTrainSummary: final_aux_huber_ema_per_h split into
final_aux_prof_bce_ema_per_h + final_aux_size_huber_ema_per_h
- Per-epoch tracing: aux_prof_bce_h{100,300,1000} + aux_size_huber_h{...}
+ aux_dir_acc_h{...} + stop_grad_aux_to_encoder
perception_overfit.rs: synthetic test asserts BCE finite + below ln(2)
chance baseline, size Huber finite, dir_acc >= 0.5.
Synthetic test on RTX 3050:
aux_prof_bce_ema = [0.0142, 0.0142, 0.0142] (30× below threshold)
aux_size_huber_ema = [0.1213, 0.1213, 0.1213] (small)
aux_dir_acc_ema = [1.0, 1.0, 1.0] (perfect)
stop_grad lifted = false (lift fired)
Known limitation (follow-up CB6): both kernels return single joint scalar
over the [K × B × N_AUX_HORIZONS] slab — all per-horizon EMA entries
carry the broadcast joint mean. Per-h gating would require kernel split.
cargo check --workspace --all-targets clean.
cargo test -p ml-alpha --lib: 43 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9ed882e740 |
feat(aux-labels): replace D with A+B paired labels + loader/trainer migration (CB1+CB2)
Smoke 1 v2 empirically falsified the D-style labels: 99.99% of K=100 D-labels
were negative on real ES MBP-10 (cost+1.5×MaxDD dominates every typical
100-tick move). Aux head couldn't escape predicting the mean.
CB1 — replace label generator:
- generate_outcome_labels_d → generate_outcome_labels_ab returning
OutcomeLabelsAB { y_prof_{long,short}, y_size_{long,short}, sigma_k,
cost_price_units, pos_fraction }
- y_prof = 1 iff signed_pnl > 2×cost (binary, class-weighted BCE target)
- y_size = signed_pnl / σ_K (σ-normalized regression target,
conditional-masked when y_prof=0)
- σ_K: rolling 1000-bar Welford std of K-step log-returns, floored at
cost/4 per pearl_trade_level_vol_for_stop_distance
- pos_fraction: per-(direction, horizon) positive class fraction for
downstream BCE pos_weight balancing
- 10 unit tests validating sign-correctness, NaN edges, balance,
cost-threshold, sigma-floor, per-horizon independence, error paths
CB2 — atomic caller migration:
- LabeledSequence + LoadedFile: outcome_long/short (2 arrays) → 5 arrays
(prof_long, prof_short, size_long, size_short, sigma_k) + pos_fraction
- Loader splits new generator's outputs + propagates pos_fraction
file-level into each yielded LabeledSequence
- step / step_batched signatures widened to 7 params + pos_fraction
- alpha_train.rs: 4 separate batches + per-snapshot row build for each
- last_pos_fraction stashed on PerceptionTrainer
- aux test fixture (synthetic_aux_outcomes) updated to emit 4 arrays +
pos_fraction matching the constant-up-ramp test signal
- HOLDING PATTERN: step_batched body validates new arrays + stages
prof-binary through the existing D-era Huber kernel so training
produces a real gradient. CB3 rewrites the kernel; CB4 widens head
outputs; CB5 wires BCE+Huber proper loss with class weighting.
cargo check --workspace --all-targets: clean (only pre-existing cudarc
cupti + ml insert_batch unrelated errors).
cargo test -p ml-alpha --lib: 43 passed (15 multi_horizon_labels).
cargo test -p ml-backtesting --lib: 33 passed.
stacked_trainer_aux_supervision RTX 3050: pass (huber=0.0004, dir_acc=1.0,
stop_grad lifted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
2e06297671 |
feat(aux-supervision): per-epoch aux observability in alpha_train (B6.5)
Smoke 1 at HEAD
|
||
|
jgrusewski
|
21e7dfd63c |
feat(aux-supervision): wire AuxTrunk + AuxHeads + Huber loss into PerceptionTrainer (B5)
Wires the aux supervision path parallel to BCE: - AuxTrunk (64-hidden single-bucket CfC) consumes the same encoder output as the main BCE trunk - AuxHeads (linear regression long/short) maps aux_trunk output to per-(direction, horizon) predicted outcomes - AuxHuberLoss supervises against D-style labels from MultiHorizonLoader Backward path with asymmetric stop-grad at encoder boundary: - aux_trunk gets gradient signal into its OWN params at all times - aux_trunk's encoder-boundary gradient is INITIALLY blocked (stop_grad_aux_to_encoder = true) - Conditional lift per E3 design: if aux_huber_ema < 0.4 AND aux_dir_acc_ema > 0.85 within 200 steps, lift the stop-grad - When lifted, aux_vec_add kernel folds aux's grad_x into the main grad_h_enriched_seq slot (element-wise += per feedback_no_atomicadd) ISV signals added: aux_huber_per_h, aux_dir_acc_per_h (per pearl). Per-trunk scratch + reduced grad buffers (no Adam state sharing per pearl_adam_normalizes_loss_weights — opt_aux is its own Adam group). New helper kernel cuda/aux_vec_add.cu: position-local dst += src for the asymmetric stop-grad lift accumulation. New synthetic test stacked_trainer_aux_supervision_converges_on_constant_signal validates end-to-end: aux_huber_ema_per_h = [0.087, 0.087, 0.087] (converged) aux_dir_acc_ema_per_h = [1.0, 1.0, 1.0] (perfect on constant) stop_grad_aux_to_encoder = false (lift fired) All 5 stacked_trainer tests pass on RTX 3050 (lib still converges, no regression from parallel aux wiring). Not yet consumed by decision policy (B7) — aux output flows through training only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
859d0c738b |
feat(aux-labels): wire D-style outcome labels into MultiHorizonLoader
Extends MultiHorizonLoaderConfig with `outcome_label_cost` (default DEFAULT_OUTCOME_LABEL_COST_ES = 0.5 = 2 ticks for ES round-trip). LabeledSequence + LoadedFile gain outcome_long + outcome_short arrays of [Vec<f32>; N_HORIZONS] populated by generate_outcome_labels_d during LoadedFile construction (same !inference_only branch as BCE labels). CLI exposure: alpha_train.rs gains --outcome-label-cost flag with the ES default. All 6 MultiHorizonLoaderConfig consumers updated atomically per feedback_no_partial_refactor: alpha_train (train + val loaders), in-file test fixtures (×2), multi_horizon_loader.rs (×2), harness.rs, trainer_parity.rs, ring3_replay.rs. cargo check --workspace clean; ml-alpha lib tests 41 passed. B3+ tasks not yet touched: outcome labels are computed and propagated to LabeledSequence but no consumer reads them yet (aux trunk Tasks B3-B5 will wire that). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
81e9970d4c |
refactor(per-horizon): N_HORIZONS 5→3 — alpha_train example
Renames all *_h6000 references to *_h1000 (new longest horizon): - AlphaTrainSummary struct fields: best_auc_h6000_* → best_auc_h1000_*, best_h6000_ckpt_path → best_h1000_ckpt_path - State vars: best_auc_h6000*, h6000_no_improvement → h1000 equivalents - Checkpoint filename: trunk_best_h6000.bin → trunk_best_h1000.bin - CLI early-stop metric: "auc_h6000" → "auc_h1000" - Tracing log fields: w_h30..w_h6000 → w_h10/w_h100/w_h1000 - 4 hardcoded [seq.labels[0..4][k]] literals → std::array::from_fn - MultiHorizonLoaderConfig.horizons + AlphaTrainSummary.horizons: literal [30,100,300,1000,6000] → HORIZONS constant - Doc comments: "5 horizons", "h6000-snapshot" → "N_HORIZONS", "longest-horizon" KNOWN FOLLOW-UP (Task 7): 5 sweep YAMLs reference trunk_best_h6000.bin filename — must update atomically when applying workflow template changes. KNOWN FOLLOW-UP (Task 8): crates/ml-alpha/src/gpu_log.rs:176-192 still decodes 5-horizon JSON fields (raw_h30..h6000, ema_h30..h6000, etc.). cargo check --example alpha_train: PASS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2f69bb1fc0 |
feat(per-horizon-cfc): wire Controller A + Phase 1→2 transition in trainer
Per spec §2.3 + §3.1 and Task 9 of the plan.
Adds two device kernels to bucket_transition_kernels.cu:
- tau_change_frobenius_kernel: ‖tau_t − tau_{t-1}‖_F² via block-tree
reduction (no atomicAdd per feedback_no_atomicadd)
- slack_factor_apply_kernel: (Q1, Q3) → (Q1/slack, Q3*slack) where
slack = sqrt(Q3/Q1), ISV-derived per feedback_isv_for_adaptive_bounds
Adds to PerceptionTrainer:
- TrainingPhase enum (Phase1Warmup / Phase2Routed)
- ControllerA state machine instance
- prev_tau_d device buffer + tau_change_d + tau_change_staged mapped-pinned
- bucket_warmup_cap_override CLI diagnostic field
- bucket_routing_metadata stored after transition fires
- heads_w_skip_compact_d (HIDDEN_DIM floats) populated by transition
- Cached function handles for the two new kernels
Per-step in step_batched (Phase 1 only):
- Launch tau_change_frobenius_kernel → mapped-pinned scalar shadow
- DtoD prev_tau ← tau_all_d for next step
- Sync + host scalar read
- controller_a.update(tau_change, bucket_warmup_cap_override)
- On trigger: stage tau into scratch buffer (avoids in/out aliasing in
tau_reorder_kernel), execute_transition writes routing metadata +
reorders tau_all_d + populates heads_w_skip_compact_d,
slack_factor_apply widens IQR bounds, invalidate CUDA Graph,
latch phase = Phase2Routed, log via tracing::info.
Phase 2 dispatch + Controllers B/C/D wiring deferred to Tasks 10–12.
In Task 9's transient state, the trainer enters Phase 2 but continues
Phase 1 dispatch path; the reordered tau_all_d still produces a valid
forward pass since CfC's per-channel decay math is order-invariant.
Per pearl_no_host_branches_in_captured_graph: transition fires OUTSIDE
the captured graph (cached graph invalidated → recaptured next step).
Per pearl_cudarc_disable_event_tracking_for_graph_capture: event tracking
is already disabled trainer-wide; recapture on next step is safe.
Per feedback_no_htod_htoh_only_mapped_pinned: host scalar read goes via
mapped-pinned DtoD shadow, not bulk DtoH.
CLI: --bucket-warmup-cap-steps added to alpha_train (Option<u64>,
diagnostic override of Controller A's ISV-derived cap).
Tests: 7 GPU oracle tests pass on RTX 3050 (5 existing + 2 new for
Frobenius + slack_factor). 33 ml-alpha lib tests pass. Workspace
cargo check clean modulo pre-existing cupti / sp15 / gpu_per_integration
errors unrelated to this task.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
jgrusewski
|
f13d6c6dcf |
refactor(per-horizon-cfc): atomically remove MTER scaffolding
Per spec §2.5 and feedback_no_partial_refactor. Removes: - LoaderMode enum (single-variant after Sequential removal → deleted entirely) - next_sequence_sequential + sequential_file_idx + sequential_anchor_idx + last_call_was_file_boundary + is_file_boundary - last_seen_file_boundary, train_graph_boundary_state fields - notify_file_boundary method + boundary-state graph recapture logic - need_attn_pool_bootstrap match — always true now (random mode always bootstraps) - --loader-mode CLI flag + notify_file_boundary() call - loader-mode Argo template param Additional consumers migrated atomically (beyond the 4 files listed in the plan): ml-alpha/tests/perception_overfit.rs, ml-alpha/tests/multi_horizon_loader.rs, ml-backtesting/src/harness.rs, ml-backtesting/tests/trainer_parity.rs, ml-backtesting/tests/ring3_replay.rs — all referenced LoaderMode or the removed config fields. Workspace builds clean at this commit (pre-existing cudarc-cupti example and ml-crate test errors are unrelated to MTER removal — they fail at HEAD too). New per-horizon CfC arch lands in subsequent tasks of the same plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7dd953aa2b |
feat(intervention-b): add LoaderMode::Sequential + attn_pool gate
Loader can now serve sequences in temporal order (Sequential mode); when active, the trainer skips attn_pool reset at non-file-boundary sequence boundaries (next commit will use this for stateful CfC + MTER). Random mode preserved as default; ZERO behavioral change for existing runs. Phased commit 1/8 of intervention B per docs/superpowers/specs/2026-05-21-crt-train-intervention-b-multi-timescale-readout.md |
||
|
jgrusewski
|
2d5a66f6e4 |
feat(kernel-step-trace): runtime --kernel-step-trace CLI flag + JSONL drain
The compile-time feature kernel-step-trace gates inclusion of the ring
code. NEW: --kernel-step-trace <PATH> CLI flag on alpha_train gates
RUNTIME activation:
- Path provided -> ring allocated, drain spawns, JSONL records written
to <PATH> (truncated). One record per kernel emission.
- Path omitted -> ring not allocated, drain not spawned, zero overhead
even with the feature compiled in.
JSONL schema: {"step": u32, "kid": u8, "kname": str, "rt": u8,
"rt_name": str, "payload": {field: f32, ...}}. The payload field names
come from the existing per-(kid, rt) decoders in gpu_log.rs (ported to
serde_json::json! in decode_to_json).
Trainer ring fields are now Option<_>; populated only when the runtime
trace path is Some. Tick kernel, step-counter shadow, smoothness
controller pointer-passing, and Drop all become path-conditional.
The tracing::info!-based drain variant is removed (file-writer is
strictly more useful; tests migrated to assert JSONL file contents).
Argo template:
- New parameter kernel-step-trace-enable (build-time feature opt-in)
- New parameter kernel-step-trace-path (runtime CLI value)
- ensure-binary cache-busts when feature toggles
- training step passes --kernel-step-trace flag conditionally
Per feedback_no_feature_flags: compile-time gate retained because the
ring carries real memory cost (~2 MiB pinned) and per-step write overhead;
the specific name kernel-step-trace narrows scope to this mechanism.
Runtime gate is Option<PathBuf>, not a boolean enable_*.
|
||
|
jgrusewski
|
60d0b62b8d | feat(crt-train): add --smoothness-base-lambda CLI flag + telemetry | ||
|
jgrusewski
|
4a32186a5f |
fix(crt-train): migrate all PerceptionTrainerConfig literals to new smoothness_base_lambda field
Task 5 added smoothness_base_lambda to PerceptionTrainerConfig but only updated the struct definition + Default impl. Eight test sites in perception_overfit.rs and one site in alpha_train.rs construct the config via explicit field list (no ..Default::default() spread), so they broke with E0063 missing-field errors under cargo check --all-targets. Per feedback_no_partial_refactor.md: when a contract changes, every consumer migrates atomically. This commit adds smoothness_base_lambda: 0.0 to all 9 sites so the workspace compiles cleanly. The alpha_train.rs value of 0.0 is a placeholder — Task 7 will replace it with cli.smoothness_base_lambda once the CLI flag is added. |
||
|
jgrusewski
|
045850e8f3 |
arch(crt-a): delete decision_stride field — greenfields atomic refactor
Per spec 2026-05-20-continuous-reasoning-trader-design.md §3.3 and §8:
decision_stride is REMOVED, not deprecated. No backwards-compat shim,
no fallback. Every consumer migrates in this commit per
feedback_no_partial_refactor.
Removed from:
- bin/fxt-backtest: RunArgs CLI flag, SweepBase field, SweepCell
override field, default_decision_stride() function, all three
BacktestHarnessConfig and RunArgs construction sites
- crates/ml-backtesting/src/harness.rs: BacktestHarnessConfig field,
MultiHorizonLoaderConfig decision_stride initializer, `let stride`
local, `if event_count % stride == 0` gate around
step_decision_with_latency; forward_step_into + step_decision now
share a single window-full guard (merged into one `if` block)
- crates/ml-alpha/src/data/loader.rs: MultiHorizonLoaderConfig field,
next_sequence stride logic simplified to stride=1 (consecutive
snapshots only)
- crates/ml-alpha/src/trainer/perception.rs: PerceptionTrainerConfig
field and Default impl; all four dt_s locals replaced with 1.0_f32
(training K-loop, graph-capture K-loop, forward_step_into CfC step,
eval K-loop)
- crates/ml-alpha/examples/alpha_train.rs: CLI flag, trainer_cfg and
both loader configs
- crates/ml/examples/alpha_baseline.rs: CLI flag, train + eval stride
gates replaced with unconditional read_all()
- config/ml/*.yaml: decision_stride: lines removed from
sweep_smoke, sweep_threshold_tuning, sweep_deployability,
sweep_decision_stride_example (file repurposed as generic example)
- tests: forward_step_golden, perception_overfit (×7 structs including
the stride=4 smoke repurposed as a second convergence check),
multi_horizon_loader (stride=4 spacing test repurposed as
ts_ns monotonicity check), ring3_replay, trainer_parity
Harness loop now invokes BOTH forward_step_into AND
step_decision_with_latency on every event whenever the snapshot window
is full. forward_step_into advances SSM state and writes alpha_probs_d;
step_decision_with_latency reads alpha_probs_d immediately after —
no CPU roundtrip, no stride gate.
n_decisions ≈ events_processed - seq_len + 1 after this commit
(vs ~9999 at stride=200 in the S2 baseline).
cargo check --workspace: clean
cargo test -p ml-backtesting --lib: 33 passed
cargo test -p ml-alpha --lib: 33 passed (6 ignored)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
87b8303950 |
chore(ml-alpha): drop 'v2' naming — greenfield, no backward compat
Per project_ml_alpha_starting_capital greenfield posture: there is no V1 to differentiate from (the V1 trunk forward was dead code, no V1 checkpoint files exist in the wild). The 'v2' prefix on every identifier was historical baggage from the migration period. Renames: - CheckpointV2 -> Checkpoint (also drops the version: u32 field — bincode either deserialises a current envelope or errors; no migration path needed) - CheckpointVersionProbe removed (was only for V1 rejection) - LAYER_NORM_CUBIN_V2 / VARIABLE_SELECTION_CUBIN_V2 / ATTENTION_POOL_CUBIN_V2 -> LAYER_NORM_CUBIN / VARIABLE_SELECTION_CUBIN / ATTENTION_POOL_CUBIN - _ln_module_v2 / _vsn_module_v2 / _attn_module_v2 -> drop _v2 suffix - smoke_load_v2_checkpoint test -> smoke_load_checkpoint - config/ml/sweep_v2_*.yaml -> config/ml/sweep_*.yaml - migration-era 'V2 weight skeleton' / 'V2 fields' / etc. comments cleaned to remove the v2 prefix Pre-existing 'v2' references in ml-backtesting CUDA files (decision_policy.cu, pnl_track.cu) are NOT touched — those refer to future planned 'v2' refinements (Portfolio mode, multi-fill averaging) from the C1-C19 commits and reflect aspirational features unrelated to this session's trunk-grows work. Verification: ml-alpha + ml-backtesting + fxt-backtest all build clean. perception_forward_golden bit-exact (max_diff = 0.000000). |
||
|
jgrusewski
|
7545651bce |
feat(ml-alpha): PerceptionTrainer.save_checkpoint + alpha_train wiring (X13+X14)
X13: Adds PerceptionTrainer::save_checkpoint as a thin delegate to self.trunk.save_checkpoint. Inference-only serialization — grads + AdamW state aren't included. X14: Inside the existing auc_h6000_improved block in alpha_train.rs, calls trainer.save_checkpoint(out_dir / 'trunk_best_h6000.bin') so the trained trunk lands alongside alpha_train_summary.json. Extends AlphaTrainSummary with best_h6000_ckpt_path (Option<String>) so downstream tooling (fxt-backtest --checkpoint) can locate the file without re-deriving the path. After this commit, every alpha-perception Argo workflow run produces a CheckpointV2 file at every new-best-h6000 epoch, ready for backtest consumption. Verification: - ml-alpha lib tests: 34 pass - alpha_train example builds clean (release) Per spec §1.1 (X13+X14). |