Commit Graph

17 Commits

Author SHA1 Message Date
jgrusewski
859d0c738b feat(aux-labels): wire D-style outcome labels into MultiHorizonLoader
Extends MultiHorizonLoaderConfig with `outcome_label_cost` (default
DEFAULT_OUTCOME_LABEL_COST_ES = 0.5 = 2 ticks for ES round-trip).
LabeledSequence + LoadedFile gain outcome_long + outcome_short arrays
of [Vec<f32>; N_HORIZONS] populated by generate_outcome_labels_d during
LoadedFile construction (same !inference_only branch as BCE labels).

CLI exposure: alpha_train.rs gains --outcome-label-cost flag with the
ES default. All 6 MultiHorizonLoaderConfig consumers updated atomically
per feedback_no_partial_refactor: alpha_train (train + val loaders),
in-file test fixtures (×2), multi_horizon_loader.rs (×2), harness.rs,
trainer_parity.rs, ring3_replay.rs.

cargo check --workspace clean; ml-alpha lib tests 41 passed.

B3+ tasks not yet touched: outcome labels are computed and propagated to
LabeledSequence but no consumer reads them yet (aux trunk Tasks B3-B5
will wire that).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 08:53:27 +02:00
jgrusewski
2efedcd6b8 refactor(per-horizon): N_HORIZONS 5→3 — remaining ml-alpha tests
Five test files migrated to clear the last cargo check --all-targets errors:

- tests/multi_horizon_loader.rs:22,64 — hardcoded [usize;5] horizons literal
  → ml_alpha::heads::HORIZONS; for-loop bounds 0..5 → 0..N_HORIZONS
- tests/output_smoothness_grad_finite_diff.rs:198 — [0.1, 0.3, 1.0, 3.0, 10.0]
  → [0.1, 1.0, 10.0] preserving 100× span across horizons
- src/data/loader.rs:560,640 (inline lib tests) — hardcoded [30,100,300,
  1000,6000] literal → crate::heads::HORIZONS
- tests/perception_overfit.rs:318,358 (audit-discovered 5-isms) —
  cfg.seq_len * 5 → cfg.seq_len * N_HORIZONS
- tests/gpu_log_ring_invariants.rs:173 (audit-discovered) — payload field
  name v["payload"]["raw_h30"] → "raw_h10" (matches gpu_log.rs schema
  migrated in Task 5)

cargo check --workspace --all-targets: clean (only pre-existing third-party
cudarc cupti example error, unrelated).
cargo test -p ml-alpha --lib: 33 passed, 0 failed, 6 ignored — baseline.

Golden fixtures deferred to runtime regeneration:
- tests/fixtures/perception_forward_golden.bin (644 → 388 bytes post-rebase).
  Test is #[ignore]-d and rewrites if missing; regenerate during Task 9
  local validation by deleting the .bin and re-running with --ignored.

gpu_log.rs migration verified complete by Task 5 (no remnant 5-horizon
field names in payload_json decoders for RT_INPUT/RT_STATE/RT_OUTPUT).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 01:45:45 +02:00
jgrusewski
1e8188c947 refactor(per-horizon): N_HORIZONS 5→3, HORIZONS=[10,100,1000] — library + data layer
First commit of the horizon-rebase refactor (full plan at
docs/superpowers/plans/2026-05-22-horizon-rebase-n3.md). Motivated by
empirical evidence from this session's investigations:

- C1: ES MBP-10 input ACF dies by L=300; K=1000/6000 are below noise floor
- D3: binarized labels ρ(30,6000)=0.06; K=6000 carries no signal
- Heads_w1 mask experiment: h6000 AUC collapsed 0.73→0.54, confirming
  long-horizon prediction needs cross-channel integration that 5-horizon
  bucket structure couldn't provide

New horizon set [10, 100, 1000] matches C1's empirical autocorrelation
elbows (~50ms, ~3.5s, ~45s wall-clock at 74µs median Δt).

Scope of this commit:
- crates/ml-alpha/src/heads.rs: N_HORIZONS 5→3, HORIZONS=[10,100,1000]
- crates/ml-alpha/src/cfc/bucket_routing.rs: MAX_BUCKET_DIM 28→96,
  BUCKET_DIM_K=[43,43,42], BUCKET_CHANNEL_OFFSET=[0,43,86,128]
- crates/ml-alpha/src/data/loader.rs: 5 hardcoded [T;5] types migrated
- crates/ml-backtesting/src/sim/mod.rs: broadcast_alpha signature,
  local N_HORIZONS re-exports ml_alpha::heads::N_HORIZONS
- crates/ml-backtesting/src/harness.rs: local N_HORIZONS re-export,
  per-horizon log labels, MultiHorizonLoaderConfig.horizons

Tests + examples + CUDA kernels still need migration (subsequent commits
per plan tasks 2-8).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 00:56:16 +02:00
jgrusewski
fe5d7a1ad7 fix(loader): rename EMA bandwidth constants to reflect actual half-lives
Empirical ES MBP-10 median inter-event Δt is 74µs (not 250ms as
documented in loader.rs:25-26). The constants ALPHA_MED=0.02 and
ALPHA_SLOW=0.0005 were named/commented as "9s @ 250ms" and "6min @
250ms" but at the actual 74µs Δt their half-lives are ~2.6ms and ~100ms
— a 3000× scale mismatch.

Rename for truth-in-naming (values unchanged to preserve all trained
checkpoints):
- ALPHA_MED → ALPHA_FAST_2P6MS (half-life ~2.6ms @ 74µs Δt)
- ALPHA_SLOW → ALPHA_MED_100MS (half-life ~100ms @ 74µs Δt)

The regime[0..5] feature naming in snap_features.rs reflects the
actual microstructure-burst and 1-2-second-clustering time scales, NOT
macro 9s/6min regimes. True macro-regime EMAs (if needed) are a
separate addition.

ml-alpha lib: 33 passed, unchanged from baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 00:09:25 +02:00
jgrusewski
f13d6c6dcf refactor(per-horizon-cfc): atomically remove MTER scaffolding
Per spec §2.5 and feedback_no_partial_refactor. Removes:
- LoaderMode enum (single-variant after Sequential removal → deleted entirely)
- next_sequence_sequential + sequential_file_idx + sequential_anchor_idx + last_call_was_file_boundary + is_file_boundary
- last_seen_file_boundary, train_graph_boundary_state fields
- notify_file_boundary method + boundary-state graph recapture logic
- need_attn_pool_bootstrap match — always true now (random mode always bootstraps)
- --loader-mode CLI flag + notify_file_boundary() call
- loader-mode Argo template param

Additional consumers migrated atomically (beyond the 4 files listed in
the plan): ml-alpha/tests/perception_overfit.rs,
ml-alpha/tests/multi_horizon_loader.rs, ml-backtesting/src/harness.rs,
ml-backtesting/tests/trainer_parity.rs,
ml-backtesting/tests/ring3_replay.rs — all referenced LoaderMode or the
removed config fields.

Workspace builds clean at this commit (pre-existing cudarc-cupti example
and ml-crate test errors are unrelated to MTER removal — they fail at
HEAD too). New per-horizon CfC arch lands in subsequent tasks of the
same plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 15:36:28 +02:00
jgrusewski
7dd953aa2b feat(intervention-b): add LoaderMode::Sequential + attn_pool gate
Loader can now serve sequences in temporal order (Sequential mode);
when active, the trainer skips attn_pool reset at non-file-boundary
sequence boundaries (next commit will use this for stateful CfC + MTER).

Random mode preserved as default; ZERO behavioral change for existing
runs. Phased commit 1/8 of intervention B per
docs/superpowers/specs/2026-05-21-crt-train-intervention-b-multi-timescale-readout.md
2026-05-21 11:30:50 +02:00
jgrusewski
045850e8f3 arch(crt-a): delete decision_stride field — greenfields atomic refactor
Per spec 2026-05-20-continuous-reasoning-trader-design.md §3.3 and §8:
decision_stride is REMOVED, not deprecated. No backwards-compat shim,
no fallback. Every consumer migrates in this commit per
feedback_no_partial_refactor.

Removed from:
  - bin/fxt-backtest: RunArgs CLI flag, SweepBase field, SweepCell
    override field, default_decision_stride() function, all three
    BacktestHarnessConfig and RunArgs construction sites
  - crates/ml-backtesting/src/harness.rs: BacktestHarnessConfig field,
    MultiHorizonLoaderConfig decision_stride initializer, `let stride`
    local, `if event_count % stride == 0` gate around
    step_decision_with_latency; forward_step_into + step_decision now
    share a single window-full guard (merged into one `if` block)
  - crates/ml-alpha/src/data/loader.rs: MultiHorizonLoaderConfig field,
    next_sequence stride logic simplified to stride=1 (consecutive
    snapshots only)
  - crates/ml-alpha/src/trainer/perception.rs: PerceptionTrainerConfig
    field and Default impl; all four dt_s locals replaced with 1.0_f32
    (training K-loop, graph-capture K-loop, forward_step_into CfC step,
    eval K-loop)
  - crates/ml-alpha/examples/alpha_train.rs: CLI flag, trainer_cfg and
    both loader configs
  - crates/ml/examples/alpha_baseline.rs: CLI flag, train + eval stride
    gates replaced with unconditional read_all()
  - config/ml/*.yaml: decision_stride: lines removed from
    sweep_smoke, sweep_threshold_tuning, sweep_deployability,
    sweep_decision_stride_example (file repurposed as generic example)
  - tests: forward_step_golden, perception_overfit (×7 structs including
    the stride=4 smoke repurposed as a second convergence check),
    multi_horizon_loader (stride=4 spacing test repurposed as
    ts_ns monotonicity check), ring3_replay, trainer_parity

Harness loop now invokes BOTH forward_step_into AND
step_decision_with_latency on every event whenever the snapshot window
is full. forward_step_into advances SSM state and writes alpha_probs_d;
step_decision_with_latency reads alpha_probs_d immediately after —
no CPU roundtrip, no stride gate.

n_decisions ≈ events_processed - seq_len + 1 after this commit
(vs ~9999 at stride=200 in the S2 baseline).

cargo check --workspace: clean
cargo test -p ml-backtesting --lib: 33 passed
cargo test -p ml-alpha --lib: 33 passed (6 ignored)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 18:48:23 +02:00
jgrusewski
453a22f47f feat(ml-alpha): CUDA Graph capture of forward_only (P5)
X11's original plan said "capture_graph_a covers full v2 forward" but
the X11 commit (4f888abbf) only shipped forward_only + from_checkpoint.
Graph capture is now actually implemented for the inference path.

Mirrors the pattern already in step_batched (perception.rs:1221-1257):
- First call: eager dispatch + set forward_warmed flag.
- Second call: begin_capture -> dispatch_forward_kernels -> end_capture
  -> store CudaGraph.
- Subsequent calls: graph.launch() — captured replay.

forward_only now performs its own staging-fill of the mapped-pinned
host buffers (input data varies per call), then dispatches through
the three-state machine. The captured region is the new private
dispatch_forward_kernels helper: a copy of evaluate_batched's
forward chain (VSN -> Mamba2 x2 -> LN x2 -> attn-pool -> CfC K-loop
-> heads) that omits labels, BCE, and any stream syncs. The final
sync + dtoh of probs_per_k_d happens OUTSIDE capture in forward_only.

Per pearl_no_host_branches_in_captured_graph: no host branches /
scalar-arg-changes / host-mallocs inside the captured region; all
kernel launches use pre-bound device pointers stable across replays.
Per pearl_cudarc_disable_event_tracking_for_graph_capture: event
tracking is already disabled for the trainer's lifetime at
construction (see PerceptionTrainer::new ~line 529), so the captured
region is free of cuStreamWaitEvent / event.record() insertions.

Vestigial loader.rs:272 doc comment referencing the never-shipped
CfcTrunk::capture_graph_a updated to point at the now-real
PerceptionTrainer::forward_only warmup path.

Regression: forward_captured_matches_uncaptured — eager (call 1) vs
captured replay (call 3) agree within 1e-5 relative tolerance per
element. NOT strict bit-identity because CUDA Graph capture can
reorder kernel launches and flip f32 reductions by 1 ULP harmlessly.
Local RTX 3050 Ti run: 160 elements, max rel_err = 0e0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 17:32:28 +02:00
jgrusewski
19986c8d96 feat(ml-alpha): tick-rule signed trade-flow inference at L1 (C16)
Replaces the placeholder `trade_signed_vol = trade_count_delta` (always
non-negative, sign-neutral) with a proper Databento-standard tick-rule
inference applied to L1 size + price deltas across consecutive MBP-10
snapshots:

  • ask_px[0] unchanged AND ask_sz[0] decreased → aggressive buys
    consumed ask depth; add (prev.ask_sz − cur.ask_sz).
  • bid_px[0] unchanged AND bid_sz[0] decreased → aggressive sells hit
    the bid; subtract (prev.bid_sz − cur.bid_sz).
  • ask_px[0] moved up → previous best ask cleared; add prev.ask_sz.
  • bid_px[0] moved down → previous best bid hit; subtract prev.bid_sz.
  • Pure cancellations (size shrank but price moved AWAY from us) =
    ambiguous; ignore.

Convention matches `Mbp10RawInput::trade_signed_vol`: positive =
buyer-initiated, negative = seller-initiated. This is a LOWER-BOUND
estimator — won't catch trades that cleared multiple levels (those
manifest only via deeper-level deltas) or trades against hidden /
off-book liquidity. Acceptable for v1 queue-decay signal; production
deployments can layer the trades-stream loader for ground-truth flow.

Wired into BacktestHarness::run() → sim.step_resting_orders(ts, vol)
so the queue-decay branch of resting_orders.cu finally fires with
non-zero input. Previously the harness passed 0.0 unconditionally,
which meant resting limits could only fill via the price-cross
marketability branch — same-price queue-decay was inert.

Six new unit tests cover each branch of the inference (pure cancel,
ask-shrank, bid-shrank, ask-px-up, bid-px-down, mixed both-sides).
34 ml-alpha lib tests + 33 ml-backtesting lib + 12 GPU fixtures + 3
fuzz still green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 10:08:20 +02:00
jgrusewski
e9c4acfb12 feat(ml-alpha): MultiHorizonLoader inference-only mode
Add inference_only flag to MultiHorizonLoaderConfig that skips per-file
forward-label precomputation (~half the file-load cost), plus
peek_first() and next_inference_input() chronological-streaming methods
for the ml-backtesting LOB harness.

- min_size relaxed to cfg.seq_len when inference_only=true (training
  still requires seq_len + max_horizon + 1 for label generation)
- New cursor fields (inference_file_idx, inference_snap_idx) walk every
  loaded snapshot in chronological order; reset() zeros both
- peek_first() seeds CfcTrunk::capture_graph_a with cur==prev semantics
  (prev_ts_ns==ts_ns, trade_signed_vol=0) — natural stream-start
- next_inference_input() errors if cfg.inference_only=false (guard
  against accidental mixing of training/inference paths)
- All trainer call-sites (alpha_train example + multi_horizon_loader
  tests) updated with inference_only: false (zero behaviour change)
- Inline test module exercises both modes; tests skip gracefully when
  fixture data isn't populated rather than panicking

See docs/superpowers/specs/2026-05-18-real-lob-integration-design.md
§1 (trainer parity) + §7 (orchestrator).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 08:08:38 +02:00
jgrusewski
70d5fc29cf feat(ml-alpha): decision-stride loader + CLI + Mamba2 dt_s scaling (Phase 2A)
Decision-stride S lets a length-K sequence span ((K-1)*S + 1) raw
snapshots instead of K consecutive ones — expands the effective
time-window covered by each sequence at the same K-positions compute
cost. With K=64 and S=4, the window covers 256 ticks (~5s on ES MBP-10
at 20ms-tick) instead of 64 ticks (~1.3s).

Loader (crates/ml-alpha/src/data/loader.rs):
- `MultiHorizonLoaderConfig.decision_stride: usize` (default 1, must
  pre-existing call sites add the new field).
- `next_sequence` reads snapshot at `anchor + k * stride`; labels at the
  same indices (labels stay in absolute-snapshot horizons regardless of
  stride, e.g. h=6000 always means "predict 6000 raw snapshots forward").
- `prev` snapshot for microstructure features (prev_mid, prev_ts_ns)
  now points to the prior K-position (`anchor + (k-1)*stride`), NOT the
  consecutive-snapshot prior, so `Δt = ts_ns - prev_ts_ns` carries the
  actual elapsed time between K-positions (consumed by Mamba2's dt_s and
  the planned Phase 2C TGN Fourier features).
- New `#[ignore]` real-data test: `loader_stride_4_yields_correct_spacing`
  asserts Δt monotonicity at stride=4.

Mamba2 dt_s (crates/ml-alpha/src/trainer/perception.rs):
- `PerceptionTrainerConfig.decision_stride: usize` plumbs the stride
  through. dispatch_train_step + evaluate_batched now use
  `dt_s = decision_stride as f32` so Mamba2's selective scan
  `exp(-dt * sigmoid(a))` reflects the real elapsed time. With stride=1
  the behaviour is identical to before.

CLI (crates/ml-alpha/examples/alpha_train.rs):
- `--decision-stride <S>` flag (default 1) wired into both train and val
  loaders + PerceptionTrainerConfig.

Argo workflow:
- `decision-stride` parameter on the template (default "1") +
  `--decision-stride` script flag + propagation into the train pod's
  alpha_train invocation.

Synthetic smoke (tests/perception_overfit.rs):
- `stacked_trainer_loss_shrinks_with_stride_4` proves the trainer-level
  dt_s=4.0 keeps the Mamba2+LN+CfC+GRN chain numerically stable.
  Converges 0.32 → 0.0000 (matches stride=1 smoke trajectory — dt_s
  scaling didn't break the SSM dynamics).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 22:04:38 +02:00
jgrusewski
eb51c0f9cd feat(ml-alpha): walk-forward CV via file-list-driven loader
mhzs7 reported val mean_auc=0.726 on 3a196382f — but the trainer
constructed both train and val MultiHorizonLoader with the SAME
`mbp10_root: cli.mbp10_data_dir`. The two loaders only differed by
seed. So val sequences were held-out-by-anchor from the same files
train sampled from; not temporally OOS. Per
`pearl_single_window_oos_is_not_oos.md` a single-window result that
doesn't enforce time-ordered separation can collapse across true
walk-forward folds.

Refactor: drop `mbp10_root` from `MultiHorizonLoaderConfig` (which
forced caller to share the dir between train and val). New API takes
an explicit `files: Vec<PathBuf>` — the loader preserves the order
given and does no internal shuffle, so callers control temporal
ordering. Added `discover_mbp10_files_sorted(root)` helper that
enumerates a dir and sorts by filename (chronological under the
`ES.FUT_<YEAR>-Q<n>.dbn.zst` convention).

alpha_train.rs splits the discovered files by 3 new CLI flags:
  --cv-fold <k>            (default 0)
  --cv-n-folds <N>         (default 1 — single fold)
  --cv-train-window <W>    (default 0 — auto)

Single-fold default (cv_n_folds=1): train on all files except the
last, val on the last file. This replaces the old "same files for
both" bug; even runs that don't think about CV now get a temporal
split by default.

Sliding-window CV (cv_n_folds > 1): fold k trains on files
[k..k+W] and validates on file [k+W]. With 9 quarterly files
(2024-Q1..2026-Q1) and `--cv-n-folds 3`, the natural layout is:

  fold 0: train 2024-Q1..2024-Q4 (W=4) → val 2025-Q1
  fold 1: train 2024-Q2..2025-Q1       → val 2025-Q2
  fold 2: train 2024-Q3..2025-Q2       → val 2025-Q3
  blind holdout: 2025-Q4, 2026-Q1

Threaded the flags through scripts/argo-alpha-perception.sh and
infra/k8s/argo/alpha-perception-template.yaml so each fold submits
as an independent workflow.

Updated tests/multi_horizon_loader.rs to the new API:
  loader_yields_seq_with_valid_labels — exercises discover + load.
  loader_errors_on_empty_files       — replaces missing-root test.
  discover_errors_on_missing_root    — pinpoints the discover step.

Honors:
  - feedback_no_partial_refactor.md — every consumer migrated atomically.
  - feedback_no_legacy_aliases.md — no `mbp10_root` shim left behind.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 16:36:48 +02:00
jgrusewski
3f089210f3 perf(ml-alpha): preload all MBP-10 files into RAM, reset loaders per epoch
cnjfl wall-time analysis: 80% of training time was disk IO. The
MultiHorizonLoader was constructed fresh per epoch in the CLI loop,
forcing 9 file deserializations from bincode (~50s/file × 9 = 7-8
min) per epoch. For a 6-epoch run that was ~45 min of pure IO out
of ~50 min total.

Now loaders preload ALL files into RAM at construction (one-time
~7 min startup) and expose a `reset(seed: u64)` method that
re-seeds anchor sampling per epoch — no disk IO between epochs.

Memory cost: ~13-15 GB for the 9-quarter ES.FUT dataset (45M
snapshots × ~280 bytes). Well under the training pod's 64 GB
limit; current 16 GB request remains sufficient since the resident
set fits.

Expected wall-time impact at K=96, B=8, 16K seqs:
  6 epochs:  ~50 min → ~13 min  (~3.7×)
  15 epochs: ~120 min → ~23 min (~5×)

The internal LoadedFile-cache-with-cycling logic is gone — the
loader now holds Vec<LoadedFile> with all files resident. Per-call
`next_sequence` picks a uniformly random file + uniformly random
anchor inside it (instead of cycling files with a per-file budget).
Distribution is equivalent: each file contributes ~n_max_sequences /
n_files samples per epoch in expectation.

77 ml-alpha tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 11:57:07 +02:00
jgrusewski
248d8fe510 feat(ml-alpha): full architectural pass — recurrent CfC, GPU BCE, regime features, training discipline
Comprehensive fix for the issues identified after the BPTT-unroll cluster
run plateaued at val_loss ~0.692 with oscillating AUCs:

ARCHITECTURE
  - CfC h_old is now RECURRENT across positions. Previously reset to
    zero every step → CfC degenerated to a per-cell tanh-FC layer.
    New: h_old at step k IS h_new at step k-1. Heads still operate
    on h_new_k, but now the CfC actually carries state. Reverse-order
    backward through the K positions accumulates grad_h_old → grad_h_new
    via the new optional `grad_h_carry` arg on multi_horizon_heads_backward.
  - tau is TRAINED. cfc_step_backward now writes grad_tau (per-cell decay
    constant derivative), trainer gets a 7th AdamW group at 0.1× cfc lr.
  - 6 NEW regime features (EMA cascade computed loader-side per file)
    fill slots out[20..26] of snap_features. Gives the model multi-minute
    trend / volatility / liquidity context that is structurally unreachable
    inside the K-snapshot BPTT window. Slots: mid-z (med/slow), trend
    signal, log-vol slow, log-spread med, log-trade-rate med. All bounded
    via log1p / signed-log so no tuned constants leak in.

PERFORMANCE (NVIDIA-style)
  - GPU-fused multi-horizon BCE for the entire [K, N_HORIZONS] grid in
    ONE launch (was K host roundtrips). Native NaN-label masking.
  - K-loop is fully GPU-resident: pre-allocated per-K scratch
    (h_new_per_k, probs_per_k, labels_per_k, grad_probs_per_k), zero
    device allocs inside step(). Only TWO syncs per sequence (after
    forward, after backward) vs previously 2K+1.
  - Stream-ordered kernel launches with pointer-offset addressing into
    per-K buffers — host doesn't wait between K iterations.
  - cfc_step_backward / multi_horizon_heads_backward both use += grad
    semantics; trainer pre-zeroes accumulators once per step().
  - MAMBA2_ALPHA_MAX_K capped at 96 (was temporarily at 256). 96 covers
    h=30/100/300 with room; regime features handle h=1000/h=6000.

TRAINING DISCIPLINE
  - LR schedule: linear warmup (default 200 steps) + cosine decay to
    lr * lr_min_factor (default 0.1). Applied per training step to both
    CfC and Mamba2 AdamW groups via new set_lr_cfc/set_lr_mamba2.
  - Best-checkpoint tracking by val_loss; recorded in summary
    (best_epoch, best_val_loss, best_val_auc).
  - Early stopping on val_loss plateau (default patience = 3).
  - CRITICAL BUG FIX: validation now uses new `evaluate()` method
    (forward-only) instead of `step()`. Previous CLI called step()
    on val data, which ran the full backward + AdamW update on the
    validation set. With per-step BPTT that's ~K× more pressure than
    the old comment ("statistically negligible") assumed.

Synthetic overfit: 0.6442 → 0.1233 in 250 steps (81% drop, sharper
than the previous 70%). 77 ml-alpha tests pass.

Local 2Q smoke (seq_len=64, 600 train seqs/epoch, 4 epochs):
  val_loss 0.7011 → 0.6990, best epoch=1, h300 AUC 0.565 in epoch 0.

Phase E.3 callers (ml/examples/alpha_baseline.rs,
alpha_dqn_h600_smoke.rs) use the LEGACY Mamba2 forward_train +
backward_from_h_enriched path — unaffected by these changes (their
kernels are pre-zeroed via alloc_zeros, so the += grad semantics
remain correct in single-call mode).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 09:59:37 +02:00
jgrusewski
586d1e7820 perf(ml-alpha): cache loaded file across next_sequence calls
The MultiHorizonLoader was calling load_or_predecode_mbp10 on every
next_sequence() call, deserializing millions of MBP-10 snapshots per
sequence. With 8000 sequences and 9 files this gave ~50+ hour
training time on what should be IO-trivial work.

Now keep one file cached (LoadedFile { snapshots, labels_full }),
yield ceil(n_max_sequences / n_files) sequences from it before
advancing. Per-horizon labels are computed once per file load and
sliced cheaply per anchor. With 8000/9: ~890 loads → 9 loads.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 00:19:06 +02:00
jgrusewski
1bb878b6a7 wip(ml-alpha): PerceptionTrainer scaffold + rename drops phase_a naming
Per user feedback "no phase naming, give proper naming to files and
functions": renames src/trainer/phase_a.rs -> perception.rs,
src/data/phase_a_loader.rs -> data/loader.rs, and the corresponding
types (PhaseATrainer -> PerceptionTrainer, PhaseALoader ->
MultiHorizonLoader, PhaseAConfig -> MultiHorizonLoaderConfig,
PhaseASequence -> LabeledSequence). Test files renamed in lock-step.

Adds PerceptionTrainer.step() — full end-to-end forward + heads
backward + cfc_step_backward (K=1 truncated BPTT) + 5 AdamW param
groups (W_in, W_rec, b, heads_w, heads_b). reset_hidden_state()
zeros h_old between independent samples.

KNOWN ISSUE — synthetic-overfit smoke (tests/perception_overfit.rs)
does NOT yet show loss shrinkage on the 200-step budget:
  initial_avg=0.6914, final_avg=0.6955 (random-baseline ln(2)=0.693)

The kernels are individually correct (heads_bwd + cfc_bwd finite-diff
at 5% rel, AdamW invariant ‖θ‖ 40->1 in 200 steps). The end-to-end
chain doesn't converge — most likely due to half the CfC cells having
near-1 decay from log-uniform tau init on a zeroed hidden state, so
only the fast cells carry signal. Fix candidates for next session:
  - tau init narrower / per-task tuned for the smoke
  - longer step budget (1000+) with adjusted LR
  - validate end-to-end with explicit print of grad/probs across iters

Task 13 is NOT complete (the gate criterion isn't met). Subsequent
work continues from here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 22:29:32 +02:00
jgrusewski
a6f4772617 feat(ml-alpha): Phase A data loader — predecoded MBP-10 -> seq + labels
Reuses ml-features::predecoded::load_or_predecode_mbp10 (no cycle —
ml-features doesn't depend on ml-alpha). Yields seq_len-sized windows
of Mbp10RawInput plus 5-horizon binary labels via
multi_horizon_labels::generate_labels.

Per-snapshot prev_mid / prev_ts_ns / trade_signed_vol come from the
prior snapshot in the source stream (not from the anchor), so the
CfC trunk sees a continuous-time signal across the entire seq.

Labels: NaN at edge positions (no forward window) or tied prices;
BCE kernel masks these (Task 10).

Tests:
  - loader_errors_on_missing_root: passes (1/1 inline)
  - loader_yields_seq_with_valid_labels: --ignored, runs at gate time
    with FOXHUNT_TEST_DATA set

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 22:17:51 +02:00