jgrusewski 1bb878b6a7 wip(ml-alpha): PerceptionTrainer scaffold + rename drops phase_a naming ...

Per user feedback "no phase naming, give proper naming to files and
functions": renames src/trainer/phase_a.rs -> perception.rs,
src/data/phase_a_loader.rs -> data/loader.rs, and the corresponding
types (PhaseATrainer -> PerceptionTrainer, PhaseALoader ->
MultiHorizonLoader, PhaseAConfig -> MultiHorizonLoaderConfig,
PhaseASequence -> LabeledSequence). Test files renamed in lock-step.

Adds PerceptionTrainer.step() — full end-to-end forward + heads
backward + cfc_step_backward (K=1 truncated BPTT) + 5 AdamW param
groups (W_in, W_rec, b, heads_w, heads_b). reset_hidden_state()
zeros h_old between independent samples.

KNOWN ISSUE — synthetic-overfit smoke (tests/perception_overfit.rs)
does NOT yet show loss shrinkage on the 200-step budget:
  initial_avg=0.6914, final_avg=0.6955 (random-baseline ln(2)=0.693)

The kernels are individually correct (heads_bwd + cfc_bwd finite-diff
at 5% rel, AdamW invariant ‖θ‖ 40->1 in 200 steps). The end-to-end
chain doesn't converge — most likely due to half the CfC cells having
near-1 decay from log-uniform tau init on a zeroed hidden state, so
only the fast cells carry signal. Fix candidates for next session:
  - tau init narrower / per-task tuned for the smoke
  - longer step budget (1000+) with adjusted LR
  - validate end-to-end with explicit print of grad/probs across iters

Task 13 is NOT complete (the gate criterion isn't met). Subsequent
work continues from here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-16 22:29:32 +02:00

284 KiB

Raw Blame History

View Raw