# Trade Plan Head — Design Spec

**Goal:** Replace per-bar independent action selection with a hierarchical plan-based system. The model outputs a full trade plan (direction, duration, profit target, stop loss, scale schedule, conviction) at entry, then commits to executing it. Per-bar decisions are constrained by the active plan. This eliminates coin-flip exits, enforces risk/reward discipline, and reduces trade count to high-conviction setups only.

**Core Principle:** A trade is not a bar-by-bar decision — it's a coordinated sequence with entry, hold, and exit. The model should plan the whole sequence, not improvise each step.

---

## Architecture

```
                    ┌──────────────────────────────┐
                    │  Plan Head (fires at entry)   │
                    │  h_s2 → plan_params [6]       │
                    │                               │
                    │  target_bars    (1-20)        │
                    │  profit_target  (0-1%)        │
                    │  stop_loss      (0-0.5%)      │
                    │  scale_aggression (0-1)       │
                    │  conviction     (0-1)         │
                    │  asymmetry      (0.5-3.0)     │
                    └──────────┬────────────────────┘
                               │ activates when Flat → Positioned
                               │ stored in portfolio state ps[16-21]
                               ▼
    ┌──────────────────────────────────────────────────┐
    │  Existing 4-Branch DQN (every bar)                │
    │  CONSTRAINED by active plan:                      │
    │                                                   │
    │  Direction: LOCKED to plan direction               │
    │  Magnitude: follows scale_aggression ramp          │
    │  Order: FREE (market/limit/IOC)                    │
    │  Urgency: overridden by plan target_bars           │
    │                                                   │
    │  AUTO-EXIT when:                                   │
    │  • unrealized P&L >= profit_target                 │
    │  • unrealized P&L <= -stop_loss                    │
    │  • bars_held >= target_bars                        │
    │  • ISV regime_stability < 0.3 (emergency)          │
    │  • Counter-plan advantage detected (N14)           │
    └──────────────────────────────────────────────────┘
```

---

## Component Catalog

### PEARLS

**P8: Learned Stop Loss + Profit Target**
Every professional trader uses stops and targets. The plan head outputs state-dependent stop/target — tight in choppy markets, wide in trends. The model learns the optimal exit thresholds for each market state instead of using fixed rules. The C51 distributional head naturally models the return distribution, making Q-value estimation of plan outcomes accurate.

**P9: Conviction-Weighted Position Sizing (Kelly Criterion)**
The plan's `conviction` parameter is a learned Kelly fraction. The model outputs how much of its risk budget to deploy. High conviction = full position. Low conviction = skip or minimal. `effective_size = conviction * risk_budget * magnitude`. When conviction < 0.3, the model doesn't enter at all — natural trade count reduction.

**P10: Temporal Commitment (Options Framework)**
The trade plan IS a Sutton option: initiation = entry from flat, internal policy = hold with scale schedule, termination = target/stop/time. The model learns WHEN to commit and WHEN to terminate. Q-values become Q(state, plan) — the expected return of the entire trade lifecycle, not a single bar.

### GEMS

**G8: Asymmetric Risk/Reward Ratio**
The `asymmetry` parameter controls profit_target / stop_loss ratio. The model learns context-dependent R/R: trending → asymmetry=2.5 (let winners run), ranging → asymmetry=1.0 (quick scalp). `effective_profit_target = base_target * asymmetry`. `effective_stop_loss = base_target / asymmetry`. This is how professional traders think — not in absolute levels but in ratios.

**G9: Scale Schedule — Pyramiding**
`scale_aggression` ∈ [0, 1] controls position ramp speed:
- aggression=0: enter 33%→66%→100% over 3 bars (cautious, confirms thesis)
- aggression=1: enter 100% immediately (all-in, high conviction)
- Intermediate: smooth ramp via `scale = min(1.0, aggression + (1-aggression) * bars_held/3)`
Reduces average entry cost and gives the model time to evaluate before full commitment.

**G10: Plan Termination as Learned Function**
Beyond hard stop/target thresholds, the plan head can output a `terminate` signal every bar: `should_exit = sigmoid(w_term @ h_s2 + plan_context)`. This captures complex exit signals the model learns: "microstructure shifted against me even though price hasn't hit my stop." The hard thresholds are safety nets; the learned termination handles nuanced exits. Implemented as the urgency branch output re-interpreted during active plan.

### NOVELS

**N13: Plan-Conditioned Q-Values**
Standard: Q(state, action). Plan head: Q(state, plan). The model evaluates "what's the expected return of deploying plan P in state S?" The plan encapsulates the ENTIRE trade lifecycle. Credit assignment becomes trivial: one plan → one trade → one reward signal at exit → clear gradient to plan head at entry bar.

**N14: Counter-Plan Evaluation**
At entry, compute Q-values for BOTH directions. Store the losing direction's Q-value. During the trade, if the market moves against the plan: compare actual P&L vs counter-plan expected value. If counter-plan would be winning by > plan's stop_loss, exit early. The model detects "I chose wrong" before the stop is hit — a smarter stop loss based on opportunity cost, not just absolute loss.

Implementation:
```cuda
float counter_q = ps[29];  // stored at entry
float current_pnl = (close - entry_price) / entry_price * sign;
// If I'm losing AND the opposite direction would be winning by more than my stop:
if (current_pnl < 0 && (-current_pnl) > counter_q * 0.5f) {
    force_exit = 1;  // cut: thesis is wrong, opportunity cost too high
}
```

**N15: Plan Replay Priority**
In the replay buffer, weight plan trades by informativeness:
- Plans where profit_target was hit exactly: high priority (model predicted well)
- Plans where stop_loss was hit immediately: high priority (model needs to learn)
- Plans that time-exited with small P&L: lower priority (inconclusive)
`priority = max(|plan_accuracy|, |loss_magnitude|)` where `plan_accuracy = actual_return / plan_profit_target`.

**N16: ISV-Modulated Plan Parameters at Runtime**
The plan is set at entry but ISV updates every step. Mid-trade adjustments:
- `effective_stop = plan_stop * max(regime_stability, 0.5)` — tighten stop during transitions
- `effective_target = plan_target * (0.5 + 0.5 * regime_stability)` — lower target expectations during uncertainty
- The model's direction stays committed, but risk management adapts. Like an experienced trader: "I'm still long but conditions deteriorated, tightening my levels."

---

## The 6 Plan Parameters

| Param | Range | Activation | Purpose |
|-------|-------|------------|---------|
| `target_bars` | [3, 25] | 3 + 22 * sigmoid | Maximum bars to hold before time exit |
| `profit_target` | [0.05%, 1.5%] | 0.0005 + 0.0145 * sigmoid | Unrealized P&L threshold for profit exit |
| `stop_loss` | [0.02%, 0.5%] | 0.0002 + 0.0048 * sigmoid | Unrealized P&L threshold for loss exit |
| `scale_aggression` | [0, 1] | sigmoid | Position ramp speed (0=cautious, 1=all-in) |
| `conviction` | [0, 1] | sigmoid | Position size fraction (Kelly proxy) |
| `asymmetry` | [0.5, 3.0] | 0.5 + 2.5 * sigmoid | profit_target / stop_loss ratio |

The model learns all 6 parameters per state. In a strong trending state with aligned microstructure: target_bars=15, profit=0.8%, stop=0.3%, scale=0.8 (fast ramp), conviction=0.9 (near full), asymmetry=2.5. In a choppy state: target_bars=4, profit=0.1%, stop=0.1%, scale=0.2 (slow ramp), conviction=0.3 (minimal or skip), asymmetry=1.0.

---

## CUDA Kernel: `trade_plan_forward`

```cuda
extern "C" __global__ void trade_plan_forward(
    const float* __restrict__ h_s2,        /* [B, SH2] */
    const float* __restrict__ w_plan_fc,   /* [AH, SH2] */
    const float* __restrict__ b_plan_fc,   /* [AH] */
    const float* __restrict__ w_plan_out,  /* [6, AH] */
    const float* __restrict__ b_plan_out,  /* [6] */
    float* __restrict__ plan_params,       /* [B, 6] output */
    int B, int SH2, int AH
) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i >= B) return;

    const float* h = h_s2 + (long long)i * SH2;
    float hidden[128]; /* AH <= 128 */

    for (int j = 0; j < AH; j++) {
        float val = b_plan_fc[j];
        for (int k = 0; k < SH2; k++)
            val += w_plan_fc[(long long)j * SH2 + k] * h[k];
        hidden[j] = fmaxf(val, 0.0f);
    }

    float* out = plan_params + i * 6;
    for (int p = 0; p < 6; p++) {
        float val = b_plan_out[p];
        for (int j = 0; j < AH; j++)
            val += w_plan_out[(long long)p * AH + j] * hidden[j];
        out[p] = val;
    }

    /* Activations */
    out[0] = 3.0f + 22.0f / (1.0f + expf(-out[0]));           /* target_bars [3, 25] */
    out[1] = 0.0005f + 0.0145f / (1.0f + expf(-out[1]));      /* profit_target [0.05%, 1.5%] */
    out[2] = 0.0002f + 0.0048f / (1.0f + expf(-out[2]));      /* stop_loss [0.02%, 0.5%] */
    out[3] = 1.0f / (1.0f + expf(-out[3]));                    /* scale_aggression [0, 1] */
    out[4] = 1.0f / (1.0f + expf(-out[4]));                    /* conviction [0, 1] */
    out[5] = 0.5f + 2.5f / (1.0f + expf(-out[5]));            /* asymmetry [0.5, 3.0] */
}
```

### `trade_plan_backward`

Standard MLP backward: d_loss → d_plan_out → d_hidden → d_w/d_b. The loss signal comes from the trade outcome at plan completion — the Q-value prediction error for this plan.

---

## Plan Enforcement in `experience_env_step`

### Plan Activation (Flat → Positioned transition)

```cuda
/* When model enters from flat, activate plan */
int was_flat = (fabsf(pre_trade_position) < 0.001f);
int now_positioned = (fabsf(position) > 0.001f);

if (was_flat && now_positioned) {
    /* Copy plan params to portfolio state */
    ps[23] = plan_params[i * 6 + 0];  /* target_bars */
    ps[24] = plan_params[i * 6 + 1];  /* profit_target */
    ps[25] = plan_params[i * 6 + 2];  /* stop_loss */
    ps[26] = plan_params[i * 6 + 3];  /* scale_aggression */
    ps[27] = plan_params[i * 6 + 4];  /* conviction */
    ps[28] = plan_params[i * 6 + 5];  /* asymmetry */

    /* Apply conviction to position size */
    position *= fmaxf(ps[27], 0.1f);  /* minimum 10% size */

    /* N14: Store counter-plan Q-value for later comparison */
    /* ps[29] = counter_direction_q; (passed from action selector) */
}
```

### Plan Execution (every bar while plan active)

```cuda
int has_plan = (ps[23] > 0.5f);

if (has_plan && fabsf(position) > 0.001f) {
    float pnl_pct = (raw_close - entry_price) / fmaxf(entry_price, 1.0f)
                     * ((position > 0.0f) ? 1.0f : -1.0f);

    /* N16: ISV-modulated thresholds */
    float stability = (isv_signals_ptr != NULL) ? isv_signals_ptr[11] : 1.0f;
    float eff_profit = ps[24] * (0.5f + 0.5f * stability);
    float eff_stop   = ps[25] * fmaxf(stability, 0.5f);

    /* G8: Asymmetric R/R — asymmetry scales profit target only.
     * asymmetry=2.5 means profit_target is 2.5× the base, stop stays as-is.
     * This gives 2.5:1 R/R ratio. Stop is independently learned. */
    eff_profit *= ps[28];

    /* Check exit conditions */
    int force_exit = 0;
    if (pnl_pct >= eff_profit) force_exit = 1;        /* profit target */
    if (pnl_pct <= -eff_stop) force_exit = 1;         /* stop loss */
    if (hold_time >= ps[23]) force_exit = 1;           /* time exit */
    if (stability < 0.3f) force_exit = 1;              /* regime emergency */

    /* G9: Scale schedule */
    if (hold_time < 3.0f && !force_exit) {
        float scale = ps[26] + (1.0f - ps[26]) * hold_time / 3.0f;
        position *= fminf(scale, 1.0f);
    }

    if (force_exit) {
        position = 0.0f;
        /* Clear plan */
        for (int s = 23; s <= 29; s++) ps[s] = 0.0f;
    }
}
```

### Direction Lock During Active Plan

In the action selection kernel, when a plan is active, the direction branch is locked:
```cuda
if (has_plan && cur_position != 0.0f) {
    /* Direction locked to current position sign */
    dir_idx = (cur_position > 0.001f) ? 2 : 0;  /* Long or Short */
    /* Magnitude follows scale schedule — not free */
    /* Order type: still free */
    /* Urgency: overridden by plan target_bars */
}
```

---

## Portfolio State Extension

Current layout: ps[0]-ps[27] used (ps[28]-ps[29] reserved). PORTFOLIO_STRIDE=23. **Must grow to 30** to accommodate 7 plan slots.

| Slot | Name | Description |
|------|------|-------------|
| ps[23] | plan_target_bars | 0 = no active plan. >0 = plan active. |
| ps[24] | plan_profit_target | Raw profit threshold (before ISV modulation) |
| ps[25] | plan_stop_loss | Raw stop threshold (before ISV modulation) |
| ps[26] | plan_scale_aggression | Position ramp speed |
| ps[27] | plan_conviction | Position size fraction |
| ps[28] | plan_asymmetry | Risk/reward ratio |
| ps[29] | counter_plan_q | N14: opposite direction Q-value at entry |

**PORTFOLIO_STRIDE: 23→30.** Must update:
- `#define PORTFOLIO_STRIDE 23` → `30` in experience_kernels.cu AND trade_stats_kernel.cu
- `i * 23` hardcoded in experience_action_select (line ~774) → `i * PORTFOLIO_STRIDE`
- Rust `gpu_experience_collector.rs` portfolio buffer allocation: `n_episodes * 23` → `n_episodes * 30`
- All `ps[N]` references where N >= 23 use new plan slots
- Portfolio reset block: zero ps[23]-ps[29] on hard reset

---

## New Weight Tensors

4 new tensors after ISV Phase 3 (adjust indices based on current NUM_WEIGHT_TENSORS):

| Index | Name | Shape | Size (AH=128, SH2=256) |
|-------|------|-------|------------------------|
| next | `w_plan_fc` | [AH, SH2] | 32768 |
| next+1 | `b_plan_fc` | [AH] | 128 |
| next+2 | `w_plan_out` | [6, AH] | 768 |
| next+3 | `b_plan_out` | [6] | 6 |

Total: 33670 params (~5.7% of existing ~580K). Larger than ISV because the plan head processes the full SH2=256 trunk representation.

The plan head weights are updated by the MAIN Adam optimizer (not separate SGD like risk branch) because plan outcomes directly affect the C51 Bellman loss.

---

## Forward Pass Integration

```
1. trunk forward (cuBLAS)              → h_s2 [B, SH2]
2. mamba2_step()                       → h_enriched [B, SH2]
3. isv_forward()                       → embedding, gate, gamma_mod
4. isv_feature_gate() (Phase 3)        → h_s2_gated
5. recursive_confidence_forward()      → predicted_error [B]
6. trade_plan_forward()                → plan_params [B, 6] ← NEW
7. risk_budget_forward()               → risk_budget [B]
8. branch heads (cuBLAS)               → Q values
9. branch_confidence_routing()         → gated Q
10. apply_risk_budget()                → scaled Q
11. action selection (plan-constrained)
12. env_step (plan activation + enforcement)
```

Step 6 is new. Step 11 uses plan constraints. Step 12 enforces plan exit conditions.

---

## Training Signal

The plan head learns from the same C51 distributional loss. When a trade completes:
1. Entry bar gets the full trade reward (via trade-level reward attribution)
2. The Q-value prediction for the entry state+plan is compared to actual outcome
3. C51 Bellman backup propagates the error through the trunk to the plan head weights

Plans that consistently hit profit targets → high Q → model enters more. Plans that consistently stop out → low Q → model avoids or adjusts parameters. The model learns: "in state S, plan {target=12, profit=0.5%, stop=0.2%, conviction=0.8} produces Sharpe X."

---

## What Plan Head Replaces

| Old | New | Why better |
|-----|-----|-----------|
| Adaptive hold (static 5 + ISV extension) | plan target_bars (learned per state) | Hold duration learned, not configured |
| No stop loss | plan stop_loss (learned per state) | Losers cut early, defined risk per trade |
| No profit target | plan profit_target (learned per state) | Winners locked in, no give-back |
| All-in position sizing | conviction × scale_aggression | Kelly-optimal sizing with pyramiding |
| 619K trades | conviction threshold filters entries | Only high-quality trades executed |
| Urgency branch (weak) | plan target_bars + scale | Direct temporal commitment |

The adaptive hold (Task 17) becomes the FALLBACK — if no plan is active (shouldn't happen), the adaptive hold still protects. The plan head is the primary mechanism.

---

## Implementation Order

1. **Plan head kernel** — `trade_plan_forward` + `trade_plan_backward`
2. **Portfolio state slots** — ps[23]-ps[29], verify PORTFOLIO_STRIDE
3. **Plan activation in env_step** — Flat→Positioned copies plan params
4. **Plan enforcement in env_step** — exit conditions, scale schedule, ISV modulation
5. **Direction lock in action_select** — constrain during active plan
6. **Weight tensors** — NUM_WEIGHT_TENSORS grows, compute_param_sizes, xavier init
7. **Counter-plan evaluation (N14)** — store and compare opposite Q
8. **Smoke test + compute-sanitizer**

---

## Phase 2: Advanced Planning — Learning from Every Trade

### PEARL: Counterfactual Plan Comparison (P11)
Every bar during a trade, the plan head computes a NEW plan from current state. Compare to entry plan. Feature in observation: `conviction_drift = current_conviction - entry_conviction`. Large negative drift = "I wouldn't enter this trade anymore" → strong exit signal. The model sees its thesis weakening in real-time.

Implementation: store entry conviction in ps[27] (already done). Current conviction from plan_params[4]. Observation feature: `out[portfolio_base + X] = plan_params[i*6+4] - ps[27]`.

### PEARL: Regime-Plan Alignment (P12)
The plan was created under a specific ISV regime state. If regime shifts mid-trade, the plan may be invalid. Feature: `regime_shift = |ISV_regime_now - ISV_regime_at_entry|`. Add ISV regime_stability at entry as ps[29] (overload counter_plan_q which we're not using). Feature: `out[X] = |isv_signals[11] - ps[29]|`.

### GEM: Temporal Plan Decay (G11)
Plan conviction decays over time. A 20-bar-old plan is less reliable than a 3-bar-old one. The model sees `effective_conviction = conviction * exp(-0.05 * hold_time)`. Encourages re-commitment or exit. Already computable from existing features (conviction × decay).

### GEM: Multi-Exit Strategy — Partial Profit Taking (G12)
Instead of all-or-nothing exits, the magnitude branch learns partial exits. When `pnl_vs_target > 0.5` (halfway to profit), the model can reduce from Full→Half (taking partial profit). When `pnl_vs_target > 1.0`, exit fully. The plan_conviction in observation enables this — the model sees "I'm 80% to target, conviction is fading → take half off."

No new kernel needed — the magnitude branch already handles Half/Quarter positions. The model just needs to LEARN this behavior from the plan-aware observation.

### NOVEL: Recursive Plan Revision (N17)
When readiness ≥ 0.5 and plan is active, the CURRENT plan output can REVISE the stored plan if the model's assessment changed significantly. Every bar: `if |current_plan - stored_plan| > threshold → update ps[23-28]`. The plan becomes a living document. The model learns WHEN to revise: "regime shifted → shorten target_bars from 15 to 5."

Implementation: in env_step, when plan is active AND readiness ≥ 0.7 (high confidence in revision):
```cuda
if (plan_active && readiness >= 0.7f) {
    float conv_now = plan_params_ptr[i*6+4];
    float conv_entry = ps[27];
    if (conv_now > conv_entry * 1.2f) {
        // Conviction increased → extend plan (confident)
        ps[23] = fmaxf(ps[23], plan_params_ptr[i*6+0]);  // don't shorten
    } else if (conv_now < conv_entry * 0.5f) {
        // Conviction halved → shorten plan (abort)
        ps[23] = fminf(ps[23], hold_time + 2.0f);  // exit in 2 bars
    }
}
```

### NOVEL: Hindsight Plan Labels — Learn from Every Trade (N18)
After each trade completes, compute the OPTIMAL plan:
- `optimal_target_bars = actual_hold_duration`
- `optimal_profit_target = max_unrealized_pnl_during_trade`
- `optimal_stop_loss = max_unrealized_drawdown_during_trade`
- `optimal_conviction = |actual_pnl| / expected_pnl` (how right was the thesis)

Store in the replay buffer alongside the trade. In the NEXT epoch, use as supervised targets for the plan head: `L_plan = MSE(plan_params, optimal_plan)` at 0.01× weight. The plan head learns from hindsight: "for this state, the optimal plan was hold=12 bars with 0.4% target."

Implementation: track `max_pnl_during_trade` and `max_dd_during_trade` in portfolio state (ps already has intra_trade_max_dd at ps[20]). At trade exit, write optimal plan to a hindsight buffer.

### NOVEL: Plan Ensemble — Confidence from Agreement (N19)
Run plan_forward 3× with different noise (dropout or input perturbation). 3 candidate plans. Ensemble agreement = confidence signal:
- All 3 agree on direction + similar target_bars → high conviction
- Divergent plans → low conviction → don't enter

Uses existing ensemble infrastructure (already 3 ensemble heads for Q-values). Apply same pattern to plans. Feature in observation: `plan_agreement = 1 - std(3_convictions) / mean(3_convictions)`.

---

## Risks and Mitigations

| Risk | Mitigation |
|------|-----------|
| Plan parameters collapse to constants (same plan every state) | Xavier init + diverse training data. ISV+regime+tick features give rich state representation for differentiation. |
| Stop loss too tight → stopped out on noise | Minimum stop_loss = 0.02% (activation floor). ISV modulation only tightens, never loosens below base. |
| Profit target too wide → never hit, always time-exit | Asymmetry parameter bounds target relative to stop. If asymmetry=1, target=stop. |
| Conviction always 0 → model never trades | Conviction floor at 0.1 (10% minimum size if entry decided by direction branch). |
| Plan conflicts with epsilon exploration | Plans only activate on GREEDY entries from flat. Once plan is active, epsilon exploration mid-trade is overridden by direction lock — this is INTENDED (plan commitment). Random actions can't break an active plan. |
| Portfolio state overflow (PORTFOLIO_STRIDE) | Verify current stride ≥ 23. Slots 16-22 = 7 new slots. |
| Plan head too large (33K params) | 5.7% of total. Same size as a branch head. Not excessive for the return it provides. |
| Counter-plan Q stale after many bars | Counter-plan comparison weakens with time. Only use for first 5 bars of plan, then rely on hard stop. |
