Allocates the device buffers + AdamW optimizers + kernel bindings for
the per-horizon attention-pool training path from C21 + C22, bundled
into a single PerceptionTrainer.per_horizon field.
crates/ml-alpha/src/trainer/per_horizon_state.rs (NEW):
PerHorizonTrainState — owns:
Learnable params (zero-init for α + bias, Xavier-scale for Q_h,
0.1× scale for w_res):
q_h_d [N_HORIZONS, HIDDEN_DIM] — attention queries
w_res_d [N_HORIZONS, HIDDEN_DIM] — residual head weights
bias_res_d [N_HORIZONS] — residual bias
alpha_d [N_HORIZONS] — learnable gate (init 0)
Forward intermediates (per-batch):
context_d [B, N_HORIZONS, HIDDEN_DIM]
attn_weights_d [B, N_HORIZONS, K]
residual_d [B, N_HORIZONS]
Backward grad scratch + reduced grads + AdamW state.
Kernel bindings: PerHorizonAttentionPool + PerHorizonResidualHead.
PerHorizonTrainState::new(dev, n_batch, k_seq, lr, seed)
Allocates all buffers, runs Xavier-style init, constructs four
AdamW optimizers (q_h, w_res, bias_res at param-LR; α at 0.25× LR
per spec §5 open Q2 default — slow gate ramp). Captures the seed
via wrapping_add(0xA110C00A) from cfg.seed for determinism.
PerHorizonTrainState::zero_grads()
Clears all grad-scratch buffers between training steps.
trainer/mod.rs:
pub mod per_horizon_state — module export.
trainer/perception.rs:
PerceptionTrainer gains one field:
pub per_horizon: PerHorizonTrainState
Initialised in new() with cfg.n_batch + cfg.seq_len + cfg.lr_cfc.
α-gate init=0 ⇒ tanh(0)=0 ⇒ contribution to per-batch logits is exactly
0 at step 0 (per C23 alpha_zero_init_is_identity_to_baseline byte-equality
test). Adopting this commit produces bit-identical training behaviour
to the previous commit until C25 wires the forward+backward calls into
step_batched; even then the α=0 init means a one-epoch smoke against
existing baseline should match within FP rounding noise.
All 34 ml-alpha lib tests + 4 per-horizon GPU tests (numgrad pair +
end-to-end pipeline pair) green.
Next:
C25 — forward + backward integration into step_batched. The new path
runs as ADDITIONAL kernel launches before/after the existing
captured graph (not inside it) so the graph stays unchanged
and the integration risk is contained.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
5eb6567c4c