spec(crt-train): output smoothness retraining intervention (intervention A)
Per project_crt_diag_findings.md — CRT.diag + diag.2 empirically
falsified all three hypotheses about why the model's per-event output
doesn't track horizon-specific dynamics. Labels ARE differentiated per
horizon, AUC=0.66 IS per-event measured, but per-event predictions
behave identically across all 5 horizons (2.5-event mean run length
raw, 3.3-event after aggressive Wiener-α EMA smoothing). h6000
predictions should change every thousands of events, not every 3.3.
Diagnosis: training dynamics issue. The BCE loss provides no signal
pushing toward horizon-coherent predictions. Adjacent-event labels at
horizon K share K-1/K of the forward window so SHOULD produce similar
predictions, but the loss doesn't require this.
Intervention A (this spec, minimum-scope): output smoothness
regularizer
L_smooth[h] = λ[h] × mean_t (p[h](t) - p[h](t-1))²
With horizon-weighted λ (stronger for h6000 than h30): forces h6000
predictions to change slowly while leaving h30 responsive.
Implementation surface:
- multi_horizon_heads.cu backward: extend grad_probs with smoothness
gradient
- perception.rs trainer step: prev_probs_d buffer, per-horizon (p[t]
- p[t-1])² accumulation
- heads.rs: SMOOTHNESS_LAMBDA constant per horizon
- alpha_train.rs: --smoothness-base-lambda CLI flag
Validation gate (CRT.train Gate):
MUST: per-horizon AUC ≥ baseline - 0.02 (no signal destruction)
WIN: h6000 mean_run_len ≥ 100 events; h6000/h30 ratio ≥ 10× (horizons
actually differentiated post-training)
STRETCH: CRT.1 controller smoke shows mean PnL CORRELATED with
conviction (vs anti-correlated in lnfwd)
Future work if intervention A doesn't pass WIN:
B: horizon-conditional output structure (architecture change)
C: curriculum on horizon (multi-day training)
D: different label generation (majority-vote vs single-point)
Status: Design — awaiting user review before plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -0,0 +1,137 @@
|
||||
# CRT.train — Output Smoothness Retraining Intervention (Design)
|
||||
|
||||
**Date:** 2026-05-20
|
||||
**Branch context:** `ml-alpha-phase-a` (HEAD `0e17fd4f2`; CRT.1 implementation done; CRT.diag + diag.2 empirically falsified the input-side fix)
|
||||
**Status:** Design — pending user review before plan
|
||||
**Predicate:** `project_crt_diag_findings.md` (empirical conclusion that retraining is required)
|
||||
|
||||
---
|
||||
|
||||
## 1. Motivation
|
||||
|
||||
The CRT.1 controller architecture is implemented and correct. But the underlying perception model produces per-event outputs that don't track horizon-specific dynamics:
|
||||
|
||||
| Empirical finding (lnfwd smoke @ 0e17fd4f2) | Implication |
|
||||
|---|---|
|
||||
| All 5 horizons have identical 2.5-event mean run length BEFORE smoothing | Heads collapsed to the same output behavior |
|
||||
| All 5 horizons have 3.0-3.3 mean run length AFTER aggressive smoothing | Even with smoothing, h6000 doesn't track its supposed horizon (should be thousands) |
|
||||
| Win rate flat 20-23% across all conviction buckets | Conviction is not predictive |
|
||||
| mean PnL anti-correlated with conviction (-45 → -84 monotonically worsening) | The "high conviction" prediction is the WRONG direction more often |
|
||||
|
||||
The model's per-event output has training-time AUC=0.66, but per-event predictions don't behave as horizon-differentiated multi-step predictors. No controller-side intervention can fix this.
|
||||
|
||||
## 2. Hypothesis
|
||||
|
||||
The per-event output collapse is a TRAINING DYNAMICS issue, not a data or architecture issue:
|
||||
|
||||
- Labels per horizon ARE differentiated (`multi_horizon_labels::generate_labels` produces `sign(price[t+K]-price[t])` per K). Refuted as label-side issue.
|
||||
- AUC=0.66 IS measured per-event (Mann-Whitney U). Refuted as metric-side issue.
|
||||
- Architecture HAS 5 separate heads × 128 hidden weights. Heads can in principle differentiate.
|
||||
|
||||
What's missing: the loss function provides no signal pushing toward per-horizon-coherent predictions. BCE on per-event labels alone:
|
||||
1. Doesn't penalize per-event jitter (adjacent labels share K-1/K of the forward window so SHOULD produce similar predictions, but the model isn't required to)
|
||||
2. Doesn't differentiate horizons structurally (the 5 BCE losses are summed/averaged; nothing in the gradient says "h6000 should change slower than h30")
|
||||
|
||||
The minimal-scope intervention: add an **output smoothness regularizer** that penalizes per-event prediction jitter, weighted by horizon (stronger for h6000 than h30).
|
||||
|
||||
## 3. Architectural Intervention
|
||||
|
||||
### 3.1 Output smoothness regularizer
|
||||
|
||||
For each horizon h, add a regularization term to the training loss:
|
||||
|
||||
```
|
||||
L_smooth[h] = λ[h] × mean_t (p[h](t) - p[h](t-1))²
|
||||
```
|
||||
|
||||
Where:
|
||||
- `p[h](t)` is the model's per-event sigmoid output at event t for horizon h
|
||||
- `λ[h]` is a horizon-specific weight, **strongly increasing with K**: e.g. `λ[h] = base_λ × (K_h / K_min)`
|
||||
- For HORIZONS = [30, 100, 300, 1000, 6000]: λ ratios = 1, 3.3, 10, 33, 200 (or some chosen scale)
|
||||
|
||||
Composite loss: `L_total = L_BCE_per_horizon + L_smooth_per_horizon`
|
||||
|
||||
The intent: h6000 predictions are required to change very slowly (high penalty for fast changes); h30 predictions are allowed to be more responsive.
|
||||
|
||||
### 3.2 Why this is the minimum-scope intervention
|
||||
|
||||
Other options considered:
|
||||
- **Architecture change**: horizon-conditional output structure. Larger code change; not necessary if regularizer suffices.
|
||||
- **Curriculum**: train h6000 first then add shorter horizons. Multi-day training cycle; not needed if regularizer suffices.
|
||||
- **Different label generation**: majority-vote over K-window instead of single-point K-ahead. Requires label regeneration; not needed if regularizer suffices.
|
||||
|
||||
Order of preference: A (this spec) → B (architecture) → C (curriculum) → D (labels), each only if the prior didn't work.
|
||||
|
||||
## 4. Implementation Surface
|
||||
|
||||
**Files to modify:**
|
||||
- `crates/ml-alpha/cuda/multi_horizon_heads.cu` — backward kernel: extend the existing `grad_probs` computation to include the smoothness gradient term. Forward pass unchanged (still produces 5 per-event sigmoid probs).
|
||||
- `crates/ml-alpha/src/trainer/perception.rs` — training step: maintain a per-horizon `prev_probs_d` buffer; after forward, compute `(p[t] - p[t-1])²` per horizon × per batch position; add to total loss with `λ[h]` weighting; backward passes through the smoothness term.
|
||||
- `crates/ml-alpha/src/heads.rs` — add `pub const SMOOTHNESS_LAMBDA: [f32; N_HORIZONS] = [...]` config constants.
|
||||
- `crates/ml-alpha/examples/alpha_train.rs` — CLI flag `--smoothness-base-lambda <f32>` (default e.g. 0.01); log per-horizon `(L_BCE, L_smooth)` separately.
|
||||
|
||||
**Files NOT to modify:**
|
||||
- `multi_horizon_labels.rs` — labels unchanged (this is intervention A, not D)
|
||||
- `heads.rs` forward path — output structure unchanged (intervention A, not B)
|
||||
- Any inference-time code (`forward_only` / `forward_step` / CRT.1 controller) — they consume whatever the trained model produces; if smoother outputs help, they help.
|
||||
|
||||
## 5. Validation Gate (CRT.train Gate)
|
||||
|
||||
After retraining a fresh checkpoint with the smoothness regularizer, run CRT.diag smoke with the existing CRT.1 controller (no controller changes). Acceptance criteria:
|
||||
|
||||
**MUST (no-regression on signal quality):**
|
||||
- Per-horizon AUC ≥ baseline_AUC − 0.02 (smoothness shouldn't destroy AUC)
|
||||
- Training loss converges within reasonable epoch count (regularizer doesn't destabilize)
|
||||
- No NaN propagation
|
||||
|
||||
**WIN (signal differentiation by horizon):**
|
||||
- h6000 mean_run_len ≥ 100 events (vs 3.3 in lnfwd)
|
||||
- h30 mean_run_len ≤ 10 events (faster, as intended)
|
||||
- h6000 mean_run_len / h30 mean_run_len ≥ 10× ratio (horizons actually differentiated)
|
||||
|
||||
**STRETCH (alpha-realization):**
|
||||
- CRT.1 controller smoke run shows mean PnL CORRELATED with conviction (not anti-correlated as in lnfwd)
|
||||
- OR win rate increasing with conviction (vs flat 22%)
|
||||
|
||||
If WIN passes but STRETCH doesn't, the model output is now horizon-differentiated but still not predictive of trade outcomes — this would point to interventions B/C/D.
|
||||
|
||||
If WIN fails, regularizer wasn't enough — escalate to intervention B (architecture change).
|
||||
|
||||
## 6. Risks
|
||||
|
||||
| Risk | Severity | Mitigation |
|
||||
|---|---|---|
|
||||
| Over-smoothing destroys AUC | High | Per-horizon `λ` tunable; start at small `base_λ` and sweep |
|
||||
| Smoothness gradient explodes with high λ | Medium | Standard gradient clipping; existing optimizer handles this |
|
||||
| Training time bloats from extra ops | Low | (p[t] - p[t-1])² is O(B × K × N_HORIZONS) — negligible vs trunk fwd cost |
|
||||
| The smoothness intervention fixes flip rate but not predictiveness | Medium | This is the "WIN without STRETCH" scenario; spec includes the escalation path to interventions B/C/D |
|
||||
|
||||
## 7. Open Questions
|
||||
|
||||
1. **What's the right `base_λ`?** Will be a sweep — start with 0.01 / 0.001 / 0.1 and pick the best on holdout AUC vs h6000 mean_run_len trade-off.
|
||||
|
||||
2. **Should smoothness be measured per-sequence-position or per-time?** The trainer processes K-windows. Within a window, all 64 positions are present. Smoothness can be: (a) within-window adjacent-position differences, OR (b) cross-window for the last position of consecutive windows. (a) is cheaper and dominant.
|
||||
|
||||
3. **Per-batch vs per-fold smoothness target?** If the model has access to `t` and `t-1` predictions in the same batch (sequential window), regularization is natural. If batches are random samples, regularization needs adjacent-event pairs as input. Check loader.rs structure.
|
||||
|
||||
4. **Does the existing training pipeline support adjacent-event pairs?** If not, training data structure may need adjustment (sequential vs random batches).
|
||||
|
||||
## 8. Out of Scope (deferred to interventions B/C/D if needed)
|
||||
|
||||
- Architecture change to horizon-conditional outputs
|
||||
- Curriculum training (h6000 first)
|
||||
- Label regeneration (majority-vote vs single-point K-ahead)
|
||||
- Multi-task losses (e.g., predicting both direction and magnitude)
|
||||
- Online weight adaptation (was Phase CRT.3 of the controller plan; would shift downstream)
|
||||
|
||||
## 9. Predecessor and Successor
|
||||
|
||||
**Predecessor:** `project_crt_diag_findings.md` (empirical falsification of the input-side fix)
|
||||
|
||||
**Successor (if WIN passes):** retrain with smoothness regularizer, validate via CRT.diag re-run, then resume CRT controller work (CRT.1 retest with new checkpoint, or proceed to CRT.2 envelope work).
|
||||
|
||||
**Successor (if WIN fails):** intervention B (architecture change) spec.
|
||||
|
||||
---
|
||||
|
||||
**End of CRT.train (Intervention A) design. Awaiting user review before implementation plan.**
|
||||
Reference in New Issue
Block a user