From faa9a73c102f64f8afd536eb9254703d5fb1b4fa Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Wed, 20 May 2026 22:59:49 +0200 Subject: [PATCH] spec(crt-train): output smoothness retraining intervention (intervention A) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per project_crt_diag_findings.md — CRT.diag + diag.2 empirically falsified all three hypotheses about why the model's per-event output doesn't track horizon-specific dynamics. Labels ARE differentiated per horizon, AUC=0.66 IS per-event measured, but per-event predictions behave identically across all 5 horizons (2.5-event mean run length raw, 3.3-event after aggressive Wiener-α EMA smoothing). h6000 predictions should change every thousands of events, not every 3.3. Diagnosis: training dynamics issue. The BCE loss provides no signal pushing toward horizon-coherent predictions. Adjacent-event labels at horizon K share K-1/K of the forward window so SHOULD produce similar predictions, but the loss doesn't require this. Intervention A (this spec, minimum-scope): output smoothness regularizer L_smooth[h] = λ[h] × mean_t (p[h](t) - p[h](t-1))² With horizon-weighted λ (stronger for h6000 than h30): forces h6000 predictions to change slowly while leaving h30 responsive. Implementation surface: - multi_horizon_heads.cu backward: extend grad_probs with smoothness gradient - perception.rs trainer step: prev_probs_d buffer, per-horizon (p[t] - p[t-1])² accumulation - heads.rs: SMOOTHNESS_LAMBDA constant per horizon - alpha_train.rs: --smoothness-base-lambda CLI flag Validation gate (CRT.train Gate): MUST: per-horizon AUC ≥ baseline - 0.02 (no signal destruction) WIN: h6000 mean_run_len ≥ 100 events; h6000/h30 ratio ≥ 10× (horizons actually differentiated post-training) STRETCH: CRT.1 controller smoke shows mean PnL CORRELATED with conviction (vs anti-correlated in lnfwd) Future work if intervention A doesn't pass WIN: B: horizon-conditional output structure (architecture change) C: curriculum on horizon (multi-day training) D: different label generation (majority-vote vs single-point) Status: Design — awaiting user review before plan. Co-Authored-By: Claude Opus 4.7 (1M context) --- ...5-20-crt-train-output-smoothness-design.md | 137 ++++++++++++++++++ 1 file changed, 137 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-20-crt-train-output-smoothness-design.md diff --git a/docs/superpowers/specs/2026-05-20-crt-train-output-smoothness-design.md b/docs/superpowers/specs/2026-05-20-crt-train-output-smoothness-design.md new file mode 100644 index 000000000..de28ca3f0 --- /dev/null +++ b/docs/superpowers/specs/2026-05-20-crt-train-output-smoothness-design.md @@ -0,0 +1,137 @@ +# CRT.train — Output Smoothness Retraining Intervention (Design) + +**Date:** 2026-05-20 +**Branch context:** `ml-alpha-phase-a` (HEAD `0e17fd4f2`; CRT.1 implementation done; CRT.diag + diag.2 empirically falsified the input-side fix) +**Status:** Design — pending user review before plan +**Predicate:** `project_crt_diag_findings.md` (empirical conclusion that retraining is required) + +--- + +## 1. Motivation + +The CRT.1 controller architecture is implemented and correct. But the underlying perception model produces per-event outputs that don't track horizon-specific dynamics: + +| Empirical finding (lnfwd smoke @ 0e17fd4f2) | Implication | +|---|---| +| All 5 horizons have identical 2.5-event mean run length BEFORE smoothing | Heads collapsed to the same output behavior | +| All 5 horizons have 3.0-3.3 mean run length AFTER aggressive smoothing | Even with smoothing, h6000 doesn't track its supposed horizon (should be thousands) | +| Win rate flat 20-23% across all conviction buckets | Conviction is not predictive | +| mean PnL anti-correlated with conviction (-45 → -84 monotonically worsening) | The "high conviction" prediction is the WRONG direction more often | + +The model's per-event output has training-time AUC=0.66, but per-event predictions don't behave as horizon-differentiated multi-step predictors. No controller-side intervention can fix this. + +## 2. Hypothesis + +The per-event output collapse is a TRAINING DYNAMICS issue, not a data or architecture issue: + +- Labels per horizon ARE differentiated (`multi_horizon_labels::generate_labels` produces `sign(price[t+K]-price[t])` per K). Refuted as label-side issue. +- AUC=0.66 IS measured per-event (Mann-Whitney U). Refuted as metric-side issue. +- Architecture HAS 5 separate heads × 128 hidden weights. Heads can in principle differentiate. + +What's missing: the loss function provides no signal pushing toward per-horizon-coherent predictions. BCE on per-event labels alone: +1. Doesn't penalize per-event jitter (adjacent labels share K-1/K of the forward window so SHOULD produce similar predictions, but the model isn't required to) +2. Doesn't differentiate horizons structurally (the 5 BCE losses are summed/averaged; nothing in the gradient says "h6000 should change slower than h30") + +The minimal-scope intervention: add an **output smoothness regularizer** that penalizes per-event prediction jitter, weighted by horizon (stronger for h6000 than h30). + +## 3. Architectural Intervention + +### 3.1 Output smoothness regularizer + +For each horizon h, add a regularization term to the training loss: + +``` +L_smooth[h] = λ[h] × mean_t (p[h](t) - p[h](t-1))² +``` + +Where: +- `p[h](t)` is the model's per-event sigmoid output at event t for horizon h +- `λ[h]` is a horizon-specific weight, **strongly increasing with K**: e.g. `λ[h] = base_λ × (K_h / K_min)` +- For HORIZONS = [30, 100, 300, 1000, 6000]: λ ratios = 1, 3.3, 10, 33, 200 (or some chosen scale) + +Composite loss: `L_total = L_BCE_per_horizon + L_smooth_per_horizon` + +The intent: h6000 predictions are required to change very slowly (high penalty for fast changes); h30 predictions are allowed to be more responsive. + +### 3.2 Why this is the minimum-scope intervention + +Other options considered: +- **Architecture change**: horizon-conditional output structure. Larger code change; not necessary if regularizer suffices. +- **Curriculum**: train h6000 first then add shorter horizons. Multi-day training cycle; not needed if regularizer suffices. +- **Different label generation**: majority-vote over K-window instead of single-point K-ahead. Requires label regeneration; not needed if regularizer suffices. + +Order of preference: A (this spec) → B (architecture) → C (curriculum) → D (labels), each only if the prior didn't work. + +## 4. Implementation Surface + +**Files to modify:** +- `crates/ml-alpha/cuda/multi_horizon_heads.cu` — backward kernel: extend the existing `grad_probs` computation to include the smoothness gradient term. Forward pass unchanged (still produces 5 per-event sigmoid probs). +- `crates/ml-alpha/src/trainer/perception.rs` — training step: maintain a per-horizon `prev_probs_d` buffer; after forward, compute `(p[t] - p[t-1])²` per horizon × per batch position; add to total loss with `λ[h]` weighting; backward passes through the smoothness term. +- `crates/ml-alpha/src/heads.rs` — add `pub const SMOOTHNESS_LAMBDA: [f32; N_HORIZONS] = [...]` config constants. +- `crates/ml-alpha/examples/alpha_train.rs` — CLI flag `--smoothness-base-lambda ` (default e.g. 0.01); log per-horizon `(L_BCE, L_smooth)` separately. + +**Files NOT to modify:** +- `multi_horizon_labels.rs` — labels unchanged (this is intervention A, not D) +- `heads.rs` forward path — output structure unchanged (intervention A, not B) +- Any inference-time code (`forward_only` / `forward_step` / CRT.1 controller) — they consume whatever the trained model produces; if smoother outputs help, they help. + +## 5. Validation Gate (CRT.train Gate) + +After retraining a fresh checkpoint with the smoothness regularizer, run CRT.diag smoke with the existing CRT.1 controller (no controller changes). Acceptance criteria: + +**MUST (no-regression on signal quality):** +- Per-horizon AUC ≥ baseline_AUC − 0.02 (smoothness shouldn't destroy AUC) +- Training loss converges within reasonable epoch count (regularizer doesn't destabilize) +- No NaN propagation + +**WIN (signal differentiation by horizon):** +- h6000 mean_run_len ≥ 100 events (vs 3.3 in lnfwd) +- h30 mean_run_len ≤ 10 events (faster, as intended) +- h6000 mean_run_len / h30 mean_run_len ≥ 10× ratio (horizons actually differentiated) + +**STRETCH (alpha-realization):** +- CRT.1 controller smoke run shows mean PnL CORRELATED with conviction (not anti-correlated as in lnfwd) +- OR win rate increasing with conviction (vs flat 22%) + +If WIN passes but STRETCH doesn't, the model output is now horizon-differentiated but still not predictive of trade outcomes — this would point to interventions B/C/D. + +If WIN fails, regularizer wasn't enough — escalate to intervention B (architecture change). + +## 6. Risks + +| Risk | Severity | Mitigation | +|---|---|---| +| Over-smoothing destroys AUC | High | Per-horizon `λ` tunable; start at small `base_λ` and sweep | +| Smoothness gradient explodes with high λ | Medium | Standard gradient clipping; existing optimizer handles this | +| Training time bloats from extra ops | Low | (p[t] - p[t-1])² is O(B × K × N_HORIZONS) — negligible vs trunk fwd cost | +| The smoothness intervention fixes flip rate but not predictiveness | Medium | This is the "WIN without STRETCH" scenario; spec includes the escalation path to interventions B/C/D | + +## 7. Open Questions + +1. **What's the right `base_λ`?** Will be a sweep — start with 0.01 / 0.001 / 0.1 and pick the best on holdout AUC vs h6000 mean_run_len trade-off. + +2. **Should smoothness be measured per-sequence-position or per-time?** The trainer processes K-windows. Within a window, all 64 positions are present. Smoothness can be: (a) within-window adjacent-position differences, OR (b) cross-window for the last position of consecutive windows. (a) is cheaper and dominant. + +3. **Per-batch vs per-fold smoothness target?** If the model has access to `t` and `t-1` predictions in the same batch (sequential window), regularization is natural. If batches are random samples, regularization needs adjacent-event pairs as input. Check loader.rs structure. + +4. **Does the existing training pipeline support adjacent-event pairs?** If not, training data structure may need adjustment (sequential vs random batches). + +## 8. Out of Scope (deferred to interventions B/C/D if needed) + +- Architecture change to horizon-conditional outputs +- Curriculum training (h6000 first) +- Label regeneration (majority-vote vs single-point K-ahead) +- Multi-task losses (e.g., predicting both direction and magnitude) +- Online weight adaptation (was Phase CRT.3 of the controller plan; would shift downstream) + +## 9. Predecessor and Successor + +**Predecessor:** `project_crt_diag_findings.md` (empirical falsification of the input-side fix) + +**Successor (if WIN passes):** retrain with smoothness regularizer, validate via CRT.diag re-run, then resume CRT controller work (CRT.1 retest with new checkpoint, or proceed to CRT.2 envelope work). + +**Successor (if WIN fails):** intervention B (architecture change) spec. + +--- + +**End of CRT.train (Intervention A) design. Awaiting user review before implementation plan.**