Commit Graph

1 Commits

Author SHA1 Message Date
jgrusewski
9ec43fdb9d feat(rl): expose forward_encoder() for RL head consumption (Phase B)
Adds PerceptionTrainer::forward_encoder() returning a borrowed slice
to h_t — the CfC h_new at the FINAL window position (K-1), where the
trade decision is made. The integrated RL trainer (ml_alpha::rl,
Phase E) consumes this to dispatch its 5 loss heads (BCE direction,
C51 Q, PPO pi, PPO V, aux prof+size) on the same encoder
representation that a supervised step() would have used.

Implementation strategy: reuse the existing forward_only() captured
graph (snap_features -> VSN -> Mamba2 x2 -> CfC K-loop -> BCE GRN
heads) rather than splitting the encoder forward out of the captured
graph. The BCE heads still run but their output (probs_per_k_d) is
discarded; the overhead is one fused GRN kernel per k (small vs.
Mamba2+CfC) and avoids the risk of breaking
pearl_cudarc_disable_event_tracking_for_graph_capture or
pearl_no_host_branches_in_captured_graph. Phase E end-to-end
profiling can revisit if needed.

A new dedicated h_t_d: CudaSlice<f32> field of size [B * HIDDEN_DIM]
receives a stream-ordered DtoD copy from h_new_per_k_d at slot K-1
after each forward_encoder() call. This gives RL callers a stable
borrowed reference even if a subsequent forward_encoder() overwrites
the per-K scratch.

Phase B (this commit) lands forward only. The backward path
(per-head loss -> lambda-weighted combine -> encoder backward via
existing step_backward_* machinery) is wired in Phase E once the
heads (Phases C+D) exist.

Existing step()/step_batched()/forward_only() semantics are
unchanged — the new field is initialised once at construction and
written only by forward_encoder(). All 56 ml-alpha lib tests still
pass; the new tests/encoder_gradient.rs locks the API contract
(borrow length B*HIDDEN_DIM, captured-graph determinism across two
calls, finite values, at least one non-zero element).
2026-05-22 22:28:19 +02:00