jgrusewski
b5530b551b
docs(plans): model capacity scale-up — 3-phase implementation plan
Three sequenced architectural capacity additions to push h6000 AUC
from current ~0.72-0.74 cross-fold plateau toward ≥0.78 deployment
target. Each phase independently deployable + measurable.
Phase 1 — Per-horizon specialisation (~1.5hr code):
- LayerNorm between Mamba2 trunk and CfC K-loop (with backward
+ per-row param-grad reduction kernel; no atomicAdd).
- 2-layer GELU MLP heads [hidden=128 → mid=64 → 1] per horizon;
ISV lambda integrates into the trunk-grad component of head
backward. Tasks 1.1-1.8 fully detailed (kernel source +
integration + numerical grad check + smoke + deploy).
Phase 2A — Decision-stride sampling (~1.5hr code):
- User-prioritised lever. Yield every S-th snapshot per training
sequence; K=64 with stride=4 covers 256 ticks of context for
the same compute as 64 ticks at stride=1. Mamba2's dt_s scalar
becomes stride-aware. Tasks 2A.1-2A.5 fully detailed (loader
refactor + test + CLI plumbing + synthetic smoke + deploy).
Phase 2B — 2-stack Mamba2 (~2hr code, sketch):
- Two Mamba2Block instances; forward chain
snap_feat → mamba2_l1 → LN → mamba2_l2 → LN → CfC.
- Acceptance criteria + key implementation notes documented;
bite-sized tasks elaborated when Phase 1+2A results land.
Phase 3 — Attention pool over Mamba2 K-positions (~4-6hr code,
sketch):
- Replace CfC's zero-init initial state with an attention-
pooled context vector over all K Mamba2 outputs. Design
decision documented (Option A: attention sets initial state,
preserving CfC's recurrent path).
Cross-phase deployment loop documented: fold-1 smoke → 3-fold
CV → per-fold comparison + isv-snapshot trajectory archive.
Honors `superpowers:writing-plans` skill: exact file paths,
complete code in every step, exact commands with expected output,
TDD-style steps, frequent commits, no placeholders in Phase 1
tasks. Self-review pass complete.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 21:18:49 +02:00
..
2026-02-23 11:19:58 +01:00
2026-05-14 19:21:25 +02:00
2026-05-17 21:18:49 +02:00
2026-04-30 17:26:25 +02:00
2026-05-07 23:36:07 +02:00
2026-04-27 22:19:41 +02:00
2026-05-16 10:20:03 +02:00
2026-04-25 12:29:47 +02:00
2026-04-28 23:54:45 +02:00
2026-05-16 09:22:20 +02:00
2026-05-09 13:52:00 +02:00
2026-04-26 10:36:03 +02:00
2026-05-09 11:26:17 +02:00