diff --git a/docs/superpowers/specs/2026-04-16-trade-plan-head-design.md b/docs/superpowers/specs/2026-04-16-trade-plan-head-design.md index 6e7308bd6..dcb3c3f61 100644 --- a/docs/superpowers/specs/2026-04-16-trade-plan-head-design.md +++ b/docs/superpowers/specs/2026-04-16-trade-plan-head-design.md @@ -349,6 +349,62 @@ The adaptive hold (Task 17) becomes the FALLBACK — if no plan is active (shoul --- +## Phase 2: Advanced Planning — Learning from Every Trade + +### PEARL: Counterfactual Plan Comparison (P11) +Every bar during a trade, the plan head computes a NEW plan from current state. Compare to entry plan. Feature in observation: `conviction_drift = current_conviction - entry_conviction`. Large negative drift = "I wouldn't enter this trade anymore" → strong exit signal. The model sees its thesis weakening in real-time. + +Implementation: store entry conviction in ps[27] (already done). Current conviction from plan_params[4]. Observation feature: `out[portfolio_base + X] = plan_params[i*6+4] - ps[27]`. + +### PEARL: Regime-Plan Alignment (P12) +The plan was created under a specific ISV regime state. If regime shifts mid-trade, the plan may be invalid. Feature: `regime_shift = |ISV_regime_now - ISV_regime_at_entry|`. Add ISV regime_stability at entry as ps[29] (overload counter_plan_q which we're not using). Feature: `out[X] = |isv_signals[11] - ps[29]|`. + +### GEM: Temporal Plan Decay (G11) +Plan conviction decays over time. A 20-bar-old plan is less reliable than a 3-bar-old one. The model sees `effective_conviction = conviction * exp(-0.05 * hold_time)`. Encourages re-commitment or exit. Already computable from existing features (conviction × decay). + +### GEM: Multi-Exit Strategy — Partial Profit Taking (G12) +Instead of all-or-nothing exits, the magnitude branch learns partial exits. When `pnl_vs_target > 0.5` (halfway to profit), the model can reduce from Full→Half (taking partial profit). When `pnl_vs_target > 1.0`, exit fully. The plan_conviction in observation enables this — the model sees "I'm 80% to target, conviction is fading → take half off." + +No new kernel needed — the magnitude branch already handles Half/Quarter positions. The model just needs to LEARN this behavior from the plan-aware observation. + +### NOVEL: Recursive Plan Revision (N17) +When readiness ≥ 0.5 and plan is active, the CURRENT plan output can REVISE the stored plan if the model's assessment changed significantly. Every bar: `if |current_plan - stored_plan| > threshold → update ps[23-28]`. The plan becomes a living document. The model learns WHEN to revise: "regime shifted → shorten target_bars from 15 to 5." + +Implementation: in env_step, when plan is active AND readiness ≥ 0.7 (high confidence in revision): +```cuda +if (plan_active && readiness >= 0.7f) { + float conv_now = plan_params_ptr[i*6+4]; + float conv_entry = ps[27]; + if (conv_now > conv_entry * 1.2f) { + // Conviction increased → extend plan (confident) + ps[23] = fmaxf(ps[23], plan_params_ptr[i*6+0]); // don't shorten + } else if (conv_now < conv_entry * 0.5f) { + // Conviction halved → shorten plan (abort) + ps[23] = fminf(ps[23], hold_time + 2.0f); // exit in 2 bars + } +} +``` + +### NOVEL: Hindsight Plan Labels — Learn from Every Trade (N18) +After each trade completes, compute the OPTIMAL plan: +- `optimal_target_bars = actual_hold_duration` +- `optimal_profit_target = max_unrealized_pnl_during_trade` +- `optimal_stop_loss = max_unrealized_drawdown_during_trade` +- `optimal_conviction = |actual_pnl| / expected_pnl` (how right was the thesis) + +Store in the replay buffer alongside the trade. In the NEXT epoch, use as supervised targets for the plan head: `L_plan = MSE(plan_params, optimal_plan)` at 0.01× weight. The plan head learns from hindsight: "for this state, the optimal plan was hold=12 bars with 0.4% target." + +Implementation: track `max_pnl_during_trade` and `max_dd_during_trade` in portfolio state (ps already has intra_trade_max_dd at ps[20]). At trade exit, write optimal plan to a hindsight buffer. + +### NOVEL: Plan Ensemble — Confidence from Agreement (N19) +Run plan_forward 3× with different noise (dropout or input perturbation). 3 candidate plans. Ensemble agreement = confidence signal: +- All 3 agree on direction + similar target_bars → high conviction +- Divergent plans → low conviction → don't enter + +Uses existing ensemble infrastructure (already 3 ensemble heads for Q-values). Apply same pattern to plans. Feature in observation: `plan_agreement = 1 - std(3_convictions) / mean(3_convictions)`. + +--- + ## Risks and Mitigations | Risk | Mitigation |