spec(ml-alpha): gate-baseline ablation strategy amendment
Defines the Mamba2-only baseline for the stacked-vs-baseline gate verdict as an ablation of the SAME PerceptionTrainer (a --bypass-cfc flag), not a separate model. Apples-to-apples; same data window, same hyperparameters, same code path. The only difference is whether the CfC step is in the loop. Three ablation options evaluated: 1. --bypass-cfc flag (recommended): Mamba2 -> heads directly 2. --mamba2-state-dim 2 (crippled Mamba2, CfC stays) 3. Frozen CfC initialized to identity (no code branch needed) Option 1 wins on clarity: it answers "is CfC additive on top of Mamba2" unambiguously, with the same Mamba2 capacity and same training regime in both arms. Concrete next-session work documented (1-2 hours): - PerceptionTrainerConfig.bypass_cfc: bool + step() branch - alpha_train --bypass-cfc CLI flag - alpha-perception-template.yaml workflow parameter + bash branch - submit both runs, fetch summaries, alpha_gate, commit verdict gate_verdict logic unchanged — the cfc/mamba2 naming in the report becomes stacked/bypass at the binding layer; the verdict math is generic. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -19,6 +19,67 @@ The original spec framed CfC as a *replacement* for Mamba2 at the perception tru
|
||||
|
||||
The CfC kernels (`cfc_step.cu`), heads, projection, BCE, AdamW, and Graph A are unchanged. Only the trunk's forward path adds a Mamba2 prefix.
|
||||
|
||||
## 2026-05-16 Amendment: Gate baseline strategy
|
||||
|
||||
The stacked-architecture amendment moved the gate criterion from
|
||||
"CfC AUC ≥ Mamba2 AUC at every horizon" to
|
||||
"**stacked AUC ≥ Mamba2-only baseline AUC at every horizon**."
|
||||
This amendment defines what the Mamba2-only baseline actually is, since the field is no longer a separate model — it's an *ablation of the same trainer*.
|
||||
|
||||
### Why an ablation is the right baseline
|
||||
|
||||
Apples-to-apples — same data window, same hyperparameter pipeline, same code path. The only difference is whether the CfC layer is in the loop. That makes the verdict "did adding CfC help" unambiguous.
|
||||
|
||||
The alternative — training a separate "Mamba2 + MLP stacker" model like Phase 1d.3 — would re-introduce the disjoint-model comparison the stacked amendment already rejected (different code paths, different optimizers, different data preprocessing). Using prior session AUC numbers as a static reference is even weaker: different data, different training regime, no scientific basis for declaring delta>0.
|
||||
|
||||
### Three concrete ablation options
|
||||
|
||||
1. **`--bypass-cfc` flag.** Add a config flag on `PerceptionTrainer` that, when set, runs Mamba2 → heads directly (skips the CfC step entirely). Two binary args differ; everything else identical. Cleanest "is CfC additive?" test.
|
||||
|
||||
2. **`--mamba2-state-dim 2`**. Minimum allowed Mamba2 state (kernel requires ≥2). CfC stays in the loop but Mamba2's capacity is crippled. Tests "does richer Mamba2 help once CfC is present?" — useful but answers a different question.
|
||||
|
||||
3. **Frozen-CfC.** Initialize CfC with identity-ish weights (W_in≈I, W_rec≈0, tau→0 so decay→1) and freeze. Mamba2 + heads still train. Effectively "Mamba2 → pass-through → heads". Cleaner than (1) because it doesn't require a code path branch but harder to implement correctly.
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Option 1** for v1. Add `--bypass-cfc` to `alpha_train` and a corresponding `bypass_cfc: bool` to `PerceptionTrainerConfig`. When set:
|
||||
- Forward: `Mamba2.forward_train(window) → h_enriched → heads → BCE`
|
||||
- Backward: `BCE → heads_backward → Mamba2.backward_from_h_enriched`
|
||||
- Skip both `cfc_step` and `cfc_step_backward`
|
||||
- Skip the 3 CfC AdamWs (W_in, W_rec, b); keep the 2 heads AdamWs (heads_w, heads_b) and the Mamba2AdamW.
|
||||
|
||||
Estimated effort: ~80 lines added to `perception.rs` (one branch in `step()`, one short config field, three optional optimizer instances). Same finite-diff validation applies (already covered by `backward_finite_diff.rs` for the kept pieces).
|
||||
|
||||
### Cluster submission protocol
|
||||
|
||||
```bash
|
||||
# Submit twice on the same SHA — same compile cache, same data, same seed.
|
||||
./scripts/argo-alpha-perception.sh --watch # stacked
|
||||
./scripts/argo-alpha-perception.sh --bypass-cfc --watch --seed 0x4242 # ablation
|
||||
# Both summaries land at /feature-cache/alpha-perception-runs/<sha>/
|
||||
# (the bypass run goes to a sibling subdir to avoid clobbering).
|
||||
alpha_gate \
|
||||
--cfc-summary /feature-cache/.../stacked/alpha_train_summary.json \
|
||||
--mamba2-summary /feature-cache/.../bypass/alpha_train_summary.json \
|
||||
--tolerance 0.01 \
|
||||
--out /feature-cache/.../phase_a_gate.json
|
||||
```
|
||||
|
||||
`--bypass-cfc` plumbing in the Argo template adds one workflow parameter and one flag conditional in the `train` step's bash. Both runs share the `ensure-binary` cache.
|
||||
|
||||
### Verdict semantics (unchanged from `gate_verdict`)
|
||||
|
||||
`gate_verdict` already implements the right logic: pass if `cfc_auc[h] ≥ mamba2_auc[h] − tolerance` at every horizon, fail otherwise with per-horizon delta + failing-horizons list. Tolerance default 0.01 stays. The "cfc" / "mamba2" naming in the verdict report becomes the "stacked" / "bypass" naming in this amendment; the gate logic doesn't care which side is which as long as both summaries have the same horizon array.
|
||||
|
||||
### Open work (next session, before cluster gate)
|
||||
|
||||
1. Add `bypass_cfc: bool` to `PerceptionTrainerConfig` + branching in `step()`
|
||||
2. Add `--bypass-cfc` CLI flag to `alpha_train`
|
||||
3. Add workflow parameter + bash conditional to `alpha-perception-template.yaml`
|
||||
4. Submit both runs, fetch summaries, run `alpha_gate`, commit `phase_a_gate.json` to artifacts
|
||||
|
||||
Total estimated effort: 1-2 hours focused, mostly code-mechanical.
|
||||
|
||||
## TL;DR
|
||||
|
||||
A clean greenfield rebuild of `ml-alpha` as a full closed-loop alpha trading system for ES.FUT on IBKR with $35k starting capital. Three components: (1) a Rust model library that owns a CfC perception trunk + 5 multi-horizon alpha heads + a CfC actor-critic policy, (2) an offline training binary, (3) an `AlphaPpoStrategy` integration in the existing `trading_agent_service` for live runtime. Everything GPU-resident on the hot path. Online learning with EWC anchoring + replay buffer (80% offline / 20% live) + kill-switch on σ-band metric deviations. Hyperparameters that vary during training are ISV-driven (signal-driven controllers, no magic numbers). Reuse the existing IBKR adapter, data ingestion, execution engine, and risk manager.
|
||||
|
||||
Reference in New Issue
Block a user