Plumbs the batched cfc + heads kernels added in 829ddfa62 through
PerceptionTrainer, evaluator, and the alpha_train CLI:
PerceptionTrainerConfig.n_batch — batch size, default 1
PerceptionTrainer::step_batched — process B sequences per
optimizer step using
cfc_step_batched / heads_batched
PerceptionTrainer::evaluate_batched — forward-only batched eval
PerceptionTrainer::step / evaluate — thin B=1 wrappers preserving
existing single-sequence
test/inference APIs (assert
cfg.n_batch == 1)
alpha_train CLI: --batch-size N — accumulates B sequences per
optimizer step in train loop;
val loop also batches and uses
evaluate_batched
Per-K scratch buffers all grow to [K, B, dim] layout (K-major, slot-k
contiguous). Mamba2's [B, K, H] output is transposed once after
forward via the new transpose_3d_swap_01 kernel, and grad_h_enriched_seq_t
is transposed back to [B, K, H] before Mamba2 backward. Two transposes
per training step; negligible (1.5MB at B=32).
Dead unbatched kernel handles removed from the trainer (step_fn,
step_bwd_fn, heads_fn, heads_bwd_fn, grad_x_d) — all training and
inference now go through the batched variants for B ≥ 1. The
single-sample kernels remain in CUDA for the standalone test helpers
in cfc/step.rs and heads.rs.
Local 2Q smoke (seq_len=32, B=4, --auto-horizon-weights, 800 train
seqs × 2 epochs):
epoch 0: val_loss=0.7138 AUC h30/h100/h300/h1000/h6000 = .55/.55/.57/.61/.51
epoch 1: val_loss=0.6558 AUC h30/h100/h300/h1000/h6000 = .72/.68/.75/.68/.65
vs the in-flight qf5mj baseline (B=1, K=96, no horizon weighting) which
had val_loss=0.6933 best and AUCs oscillating at ~0.50 — this batched
run hits AUC 0.75 (h300) and 0.72 (h30) in just 2 epochs of 200
optimizer updates. Batching + horizon-weighting unblocks the model.
77 ml-alpha tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
c3ee5e165a