docs(plans): model capacity scale-up — 3-phase implementation plan

Browse Source

Three sequenced architectural capacity additions to push h6000 AUC
from current ~0.72-0.74 cross-fold plateau toward ≥0.78 deployment
target. Each phase independently deployable + measurable.

Phase 1 — Per-horizon specialisation (~1.5hr code):
  - LayerNorm between Mamba2 trunk and CfC K-loop (with backward
    + per-row param-grad reduction kernel; no atomicAdd).
  - 2-layer GELU MLP heads [hidden=128 → mid=64 → 1] per horizon;
    ISV lambda integrates into the trunk-grad component of head
    backward. Tasks 1.1-1.8 fully detailed (kernel source +
    integration + numerical grad check + smoke + deploy).

Phase 2A — Decision-stride sampling (~1.5hr code):
  - User-prioritised lever. Yield every S-th snapshot per training
    sequence; K=64 with stride=4 covers 256 ticks of context for
    the same compute as 64 ticks at stride=1. Mamba2's dt_s scalar
    becomes stride-aware. Tasks 2A.1-2A.5 fully detailed (loader
    refactor + test + CLI plumbing + synthetic smoke + deploy).

Phase 2B — 2-stack Mamba2 (~2hr code, sketch):
  - Two Mamba2Block instances; forward chain
    snap_feat → mamba2_l1 → LN → mamba2_l2 → LN → CfC.
  - Acceptance criteria + key implementation notes documented;
    bite-sized tasks elaborated when Phase 1+2A results land.

Phase 3 — Attention pool over Mamba2 K-positions (~4-6hr code,
sketch):
  - Replace CfC's zero-init initial state with an attention-
    pooled context vector over all K Mamba2 outputs. Design
    decision documented (Option A: attention sets initial state,
    preserving CfC's recurrent path).

Cross-phase deployment loop documented: fold-1 smoke → 3-fold
CV → per-fold comparison + isv-snapshot trajectory archive.

Honors `superpowers:writing-plans` skill: exact file paths,
complete code in every step, exact commands with expected output,
TDD-style steps, frequent commits, no placeholders in Phase 1
tasks. Self-review pass complete.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

...

This commit is contained in:

jgrusewski

2026-05-17 21:18:49 +02:00

parent 00da163078

commit b5530b551b

1 changed files with 1568 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1568

docs/superpowers/plans/2026-05-17-model-capacity-scaleup.md Normal file

File diff suppressed because it is too large Load Diff