diff --git a/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md b/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md index 2d8d364d3..25348c206 100644 --- a/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md +++ b/docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md @@ -12,7 +12,7 @@ Take the system from "stable, gradient-flowing, but trades like a panicked toddl 1. **Phase 0** — Retune SP14 EGF (gate1=closed forever) so its anchors are ISV-driven, not hardcoded. 2. **Phase 1** — Honest numbers: unified sharpe kernel, cost-net sharpe (commission + spread + OFI-impact, dev/prod parity), drawdown reporting, 8 counterfactual baselines, `dd_pct` as foundational state input, `--holdout-quarters` + `--dev-quarters` CLI flags, consume abandoned walk-forward test slice. -3. **Phase 2** — 21-test behavioral suite on dev RTX 3050 Ti. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S. +3. **Phase 2** — 22-test behavioral suite on dev RTX 3050 Ti. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S. 4. **Phase 3** — 5 trader teachings (reward + action selection): r_quality/r_discipline split, explicit cost, quadratic DD penalty, regret signal, confidence-aware Hold floor. 5. **Phase 3.5** — 4 recovery mechanisms (3.5.1 already in Phase 1): asymmetric reward under DD, cooldown gate, plasticity injection, recovery curriculum in PER. 6. **Phase 4** — L40S walk-forward on Q1-Q7, Q8 final dev, sealed Q9 OOS — produces report card with deltas vs all 8 baselines. @@ -109,7 +109,7 @@ Val is temporally clean for parameter learning, but it's the selection set, not | Q2 | SP14 EGF cut vs retune | **Keep, retune to ISV-driven**. No hardcoded thresholds; Schmitt thresholds, variance gate refs, steepness anchors all become ISV-tracked. | | Q3 | Cost model dev/prod parity | **commission + spread + OFI-impact, everywhere**. Dev synthetic markets generate synthetic spread + OFI. Prod reads from MBP10 fxcache. Same kernel. | | Q4 | Counterfactual baselines | **All 8**: buyhold, hold-only, random_dir + same Kelly, naive momentum, aux-only, mag-Quarter-fixed, trail-only, naive reversion. | -| Q5 | Phase 2 test set | **All 21 tests**: 7 trading behavior + 10 state/arch grounding + 4 recovery dynamics. | +| Q5 | Phase 2 test set | **22 tests**: 7 trading behavior + 10 state/arch grounding + 5 recovery dynamics (added 2.22 plasticity_cooldown_interlock per critical-review fix). | | Q6 | OOS framework | **Train/dev/test split**: Q1-Q7 walk-forward train, Q8 final dev, Q9 sealed final test. | | Q7 | Phase 3.5.4 plasticity injection | **Land all 4 recovery mechanisms** including plasticity. No defer. | @@ -146,7 +146,50 @@ Phase 2A: behavioral test scaffolding ─┘ - `sp15-phase2a-test-scaffold` — synthetic market generators + harness (parallel to Phase 0+1) - **Merge model**: each phase merges back to `sp15-trader-discipline-recovery` atomically. SP15 branch merges to main only after Phase 4 production-track gate passes. -### 4.3 Cross-cutting discipline rule +### 4.3 ISV slot allocation map (resolves parallel-dispatch ISV-table conflict) + +Approach B parallel dispatch requires that Phase 0 / Phase 1 / Phase 2A sub-worktrees not collide on ISV slot indices. The spec pre-allocates disjoint slot ranges per phase. SP14 occupied slots 383-395 (`ISV_TOTAL_DIM = 396` post-merge `0275a25d9`). SP15 starts at 397. + +| Range | Owner | Purpose | Count | +|---|---|---|---| +| `[397..401)` | Phase 0.B (EGF retune) | `EGF_SCHMITT_HI`, `EGF_SCHMITT_LO`, `EGF_VAR_AUX_REF`, `EGF_VAR_Q_REF` | 4 | +| `[401..407)` | Phase 1.3 (drawdown) | `DD_CURRENT`, `DD_MAX`, `DD_RECOVERY_BARS`, `DD_PERSISTENCE`, `CALMAR`, `DD_PCT` | 6 | +| `[407..409)` | Phase 1.2 (cost kernel) | `OFI_IMPACT_LAMBDA` (ISV-tracked), `COST_PER_BAR_AVG` | 2 | +| `[409..417)` | Phase 1.4 (8 baselines) | one `BASELINE_SHARPE_NET_*` per baseline | 8 | +| `[417..420)` | Phase 3.1 (α split) | `ALPHA_SPLIT` (ISV-driven from grad ratio), `GRAD_NORM_QUALITY`, `GRAD_NORM_DISCIPLINE` | 3 | +| `[420..423)` | Phase 3.3 (DD penalty) | `LAMBDA_DD`, `DD_THRESHOLD`, `DD_PENALTY_GRAD_NORM` | 3 | +| `[423..426)` | Phase 3.4 (regret) | `REGRET_EMA`, `LAMBDA_REGRET`, `REGRET_GRAD_NORM` | 3 | +| `[426..430)` | Phase 3.5 (Hold floor) | `HOLD_FLOOR_ALPHA`, `HOLD_FLOOR_K`, `HOLD_FLOOR_EPS0`, `ENTROPY_DIST_REF` | 4 | +| `[430..433)` | Phase 3.5.2 (DD asymmetric reward) | `DD_ASYMMETRY_LAMBDA` (ISV-tracked from DD distribution), `R_GAIN_DD_BOOST`, `DD_DIST_VAR` | 3 | +| `[433..436)` | Phase 3.5.3 (cooldown) | `COOLDOWN_K_THRESHOLD` (ISV from running mean of per-trade PnL), `COOLDOWN_M_BARS`, `COOLDOWN_BARS_REMAINING` | 3 | +| `[436..439)` | Phase 3.5.4 (plasticity) | `PLASTICITY_FIRED_THIS_FOLD`, `PLASTICITY_PERSISTENCE_THRESHOLD`, `PLASTICITY_WARM_BARS_REMAINING` (force cooldown engagement post-fire) | 3 | +| `[439..441)` | Phase 3.5.5 (recovery curriculum) | `DD_TRAJECTORY_DECREASING` (per-step proxy for "in recovery"), `RECOVERY_OVERSAMPLE_WEIGHT` | 2 | +| **Total** | | **`ISV_TOTAL_DIM = 441` post-SP15** | 44 | + +Each phase's first commit lands `sp15_isv_slots.rs` extension covering its allocated range. Layout fingerprint seed extended atomically per phase. Sub-worktrees check the allocation map before assigning indices. + +### 4.4 Phase 2A ↔ Phase 1.2 ABI contract (resolves parallelism dependency) + +The synthetic-LOB triple `(price_t, spread_t, ofi_t)` is the canonical ABI between Phase 2A test markets and Phase 1.2 cost kernel. Phase 2A defines this contract first; Phase 1.2 kernel signature must conform. + +```rust +// Canonical ABI defined by Phase 2A scaffolding, consumed by Phase 1.2: +struct LobBar { + price: f32, // mid-price, units = MES futures dollar + spread: f32, // bid-ask spread, units = same as price (e.g., 0.25 = quarter-tick) + ofi: f32, // order flow imbalance signed magnitude, normalized to ~[-3, 3] +} +// Synthetic markets emit Vec; fxcache loader emits Vec; cost kernel reads &[LobBar] +``` + +Phase 2A lands the `LobBar` type + synthetic generators FIRST. Phase 1.2 cost kernel reads `&[LobBar]`. Phase 0 + Phase 1 (other subs) + Phase 2A scaffold can then dispatch in parallel because: +- Phase 0 doesn't touch LobBar (EGF kernels) +- Phase 1.1, 1.3-1.7 don't touch LobBar (sharpe kernel, DD reporting, baselines, dd_pct, CLI, test slice consumption) +- Phase 1.2 conforms to Phase 2A's pre-defined ABI + +This restores Approach B parallelism with explicit contract precedence. + +### 4.5 Cross-cutting discipline rule **No new pearl, controller, kernel, ISV slot, or reward term ships without:** 1. **Phase 2 behavioral test** that proves the intended trader behavior (the test commit and the implementation commit ship together). @@ -180,15 +223,15 @@ Either the thresholds are wrong (signal magnitudes are inherently smaller than e Every hardcoded anchor becomes ISV-driven: -| Anchor (current value) | New ISV slot | Producer | +| Anchor (current value) | ISV slot (per allocation map §4.3) | Producer | |---|---|---| -| Schmitt thresholds (±0.03 hardcoded) | `EGF_SCHMITT_HI_INDEX`, `EGF_SCHMITT_LO_INDEX` | rolling p99/p1 of α_raw over fold | -| Variance gate ref aux (0.01) | `EGF_VAR_AUX_REF_INDEX` | running mean of var_aux over fold | -| Variance gate ref Q (0.05) | `EGF_VAR_Q_REF_INDEX` | running mean of var_q over fold | -| Steepness normaliser (k_aux=20/(1+var_aux/0.01) hardcoded `0.01`) | reuses `EGF_VAR_AUX_REF_INDEX` | (above) | -| Steepness normaliser k_q (0.05 hardcoded) | reuses `EGF_VAR_Q_REF_INDEX` | (above) | +| Schmitt thresholds (±0.03 hardcoded) | `EGF_SCHMITT_HI` (slot 397), `EGF_SCHMITT_LO` (slot 398) | rolling p99/p1 of α_raw over fold | +| Variance gate ref aux (0.01) | `EGF_VAR_AUX_REF` (slot 399) | running mean of var_aux over fold | +| Variance gate ref Q (0.05) | `EGF_VAR_Q_REF` (slot 400) | running mean of var_q over fold | +| Steepness normaliser (k_aux=20/(1+var_aux/0.01) hardcoded `0.01`) | reuses `EGF_VAR_AUX_REF` (slot 399) | (above) | +| Steepness normaliser k_q (0.05 hardcoded) | reuses `EGF_VAR_Q_REF` (slot 400) | (above) | -Total: 4 new ISV slots + 2 new producer kernels. EGF kernels read these instead of constants. +Total: 4 new ISV slots (`[397..401)` per §4.3) + 2 new producer kernels. EGF kernels read these instead of constants. ### 5.4 Phase 0 anchor test (Phase 2.21 — `egf_gate_opens_under_disagreement`) @@ -208,16 +251,28 @@ Eliminates `sharpe_ema` (per-batch EMA with smoothing constant divergence) vs `s ### 6.2 Sub 1.2 — Cost-net sharpe -`cost_t = commission + spread_t × position_size + ofi_impact(spread_t, ofi_t)` +``` +cost_t = commission_per_rt × rt_indicator(t) + + half_spread_t × |position_size_t| × side_indicator(t) + + ofi_impact_t +``` -- `commission`: $1.50/RT default, `--commission` CLI overridable -- `spread_t`: real bid-ask from MBP10 fxcache (prod) OR synthetic spread from market generator (dev) -- `ofi_impact()`: linear-impact model `λ × |position_size| × |ofi_t|` where λ is ISV-tracked impact coefficient (initial: empirical fit from MES historical) -- `position_size`: contracts traded (1 for Quarter, 2 for Half, 4 for Full at base Kelly cap) +**Per-side semantics (resolves round-trip ambiguity)**: +- `commission_per_rt`: $1.50 default per round-trip, applied entirely on exit (not split). `--commission` CLI overridable. +- `rt_indicator(t)`: 1.0 on close-trade bars, 0 otherwise. +- `half_spread_t = spread_t / 2`: half-spread cost per side. **Charged BOTH at entry AND at exit.** `side_indicator(t)`: 1.0 on entry bars, 1.0 on exit bars, 0 otherwise. Net: a complete round-trip pays `commission_per_rt + half_spread_at_entry × position + half_spread_at_exit × position` — i.e., one full spread cost per RT plus one commission. +- `ofi_impact_t = λ × |position_size_t| × |ofi_t| × side_indicator(t)`: applied per-side (entry and exit). + - Initial `λ = 2e-4` (rough MES literature fit; impact ≈ 0.2 ticks per contract per σ-unit OFI). + - ISV-tracked at slot `OFI_IMPACT_LAMBDA` (slot 407 per §4.3) — refit per fold by minimizing prediction error of next-tick mid-price shift vs OFI quartile on fold's training window. +- `position_size`: contracts traded (1 for Quarter, 2 for Half, 4 for Full at base Kelly cap). -`sharpe_net = (mean_pnl − cost_per_bar_avg) / std_pnl_with_costs` +**Sharpe net**: +``` +sharpe_net = (mean_pnl − mean_cost) / std_pnl_with_costs +``` +where `mean_cost = sum(cost_t) / N_bars` is emitted to ISV slot `COST_PER_BAR_AVG` (slot 408). -The cost kernel reads `spread_t` and `ofi_t` as input arrays. Dev synthetic markets generate these as part of fixtures. This is the same kernel in both environments — `feedback_isv_for_adaptive_bounds` extends to costs. +**Dev/prod parity**: the cost kernel reads `spread_t`, `ofi_t` as input arrays. Phase 2A synthetic markets generate these via the §4.4 LobBar ABI. Prod fxcache produces the same LobBar type. One kernel, two environments — `feedback_isv_for_adaptive_bounds` extends to costs. ### 6.3 Sub 1.3 — Drawdown reporting @@ -229,22 +284,28 @@ New ISV slots: HEALTH_DIAG emits `max_dd_pct`, `current_dd`, `dd_recovery_bars`, `Calmar = mean_pnl / max(max_dd_pct, 1e-4)`. Denominator floor (1e-4) replaces the current Calmar=100 saturation artifact. -### 6.4 Sub 1.4 — 8 counterfactual baselines +### 6.4 Sub 1.4 — 8 counterfactual baselines (with shared trunk forward) -Each baseline is a pure CUDA kernel that replays the val/test window with a fixed action policy and computes sharpe_net + max_dd + trade_count. All 8 run in parallel kernels per fold per epoch: +Each baseline replays the val/test window with a different action policy and computes sharpe_net + max_dd + trade_count. **Critical implementation detail (resolves Important #9)**: baselines that need policy-trained heads (aux_only, mag_quarter_fixed, trail_only) MUST share the trunk forward pass with the main policy evaluation. Without trunk fusion, 8 separate full-policy kernel launches per fold per epoch would inflate eval cost ~4-8×. -| Baseline | Definition | -|---|---| -| `buyhold` | always Long, mag = Quarter (smallest), no exits | -| `hold_only` | always Hold, never enter | -| `random_dir_kelly` | random direction at random bars (seeded), mag pinned to policy's Kelly cap output | -| `naive_momentum` | direction = sign(last bar return), mag = Quarter | -| `aux_only` | direction = sign(aux head's next-bar prediction), mag = Quarter | -| `mag_quarter_fixed` | policy's direction, mag pinned to Quarter | -| `trail_only` | random entries (seeded), exits via policy's trail-stop logic | -| `naive_reversion` | direction = -sign(last bar return), mag = Quarter | +**Implementation**: a single fused trunk forward computes shared state representation `h_s2` for the val/test window. Then 8 lightweight per-baseline kernels read `h_s2` (where applicable) plus other inputs, output baseline action sequences, run cost-net sharpe kernel. -HEALTH_DIAG emits per-fold deltas: `vs_buyhold = sharpe_net − buyhold_sharpe_net`, etc. +| Baseline | Inputs | Trunk forward needed? | +|---|---|---| +| `buyhold` | price series only | No | +| `hold_only` | none (constant Hold) | No | +| `random_dir_kelly` | seeded RNG, policy Kelly cap output | trunk YES (for Kelly cap) | +| `naive_momentum` | last-bar return only | No | +| `aux_only` | trunk h_s2 + aux head only | trunk YES (subset head) | +| `mag_quarter_fixed` | trunk h_s2 + direction Q head only | trunk YES (subset head) | +| `trail_only` | seeded RNG, policy trail-distance ISV | trunk NO (just trail logic) | +| `naive_reversion` | last-bar return only | No | + +Of 8 baselines, 3 need the shared trunk (already computed once for main policy eval). Remaining 5 are constant or last-bar-only — pure kernels with negligible cost. + +**Realistic overhead estimate**: ~15-25% of full eval pass (not 5-10% as initially claimed). Acceptable for the validation rigor benefit. Documented honestly so we don't underestimate L40S timing. + +HEALTH_DIAG emits per-fold deltas: `vs_buyhold = sharpe_net − buyhold_sharpe_net`, etc. Each baseline's sharpe_net writes to its allocated ISV slot (`BASELINE_SHARPE_NET_*` slots 409-416 per §4.3). ### 6.5 Sub 1.5 — `dd_pct` as foundational state input @@ -291,7 +352,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050 - **Pre-commit hook** in `.claude/helpers/pre_commit_behavioral_suite.sh`: runs `cargo test -p ml --test behavioral_suite -- --nocapture`. Fail → block commit. - **CI integration**: `argo-train.sh` invokes the same hook before submitting workflow. Local fail = no L40S deploy. -### 7.3 21 tests by group +### 7.3 22 tests by group #### Group 1: Trading behavior (7 tests, lands as Phase 1 cost + market gen ships) @@ -320,7 +381,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050 | 2.20 | `state_input_grounding_test` | KL(uptrend-context, downtrend-context action dists) > 0.5 | | 2.21 | `egf_gate_opens` (Phase 0 anchor) | aux/Q deliberate disagreement → gate1 opens within 100 steps | -#### Group 3: Recovery dynamics (4 tests, anchors Phase 3.5 teachings) +#### Group 3: Recovery dynamics (5 tests, anchors Phase 3.5 teachings) | # | Test | Pass criterion | |---|---|---| @@ -328,6 +389,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050 | 2.10 | `recovery_after_streak` | inject 10R loss streak → recover ≥5R over next 200 bars (NOT collapse to all-Hold) | | 2.11 | `dd_state_aware_sizing` | KL(0% DD, 50% dd_budget action dists) > 0.1 — policy conditions sizing on DD | | 2.12 | `cooldown_engagement` | K=5 losses → cooldown fires within 1 bar; disengages cleanly after M=20 bars | +| 2.22 | `plasticity_cooldown_interlock` | trigger plasticity fire (synthetic persistent DD) → action selection forces Hold for full M_warm bars; zero trades during warm-up; no second plasticity trigger within fold | ### 7.4 Sequencing within Phase 2 @@ -354,7 +416,7 @@ Reward shape and action selection teach the trader role, not just optimize a num #### 3.1 — r_quality + r_discipline split -Architectural framing, no separate code beyond the convex combination: +Architectural framing: ``` r_total = α × r_quality_per_event + (1 − α) × r_discipline_per_bar @@ -362,7 +424,8 @@ r_total = α × r_quality_per_event + (1 − α) × r_discipline_per_bar - `r_quality_per_event` = SP12 per-trade event-driven reward (UNCHANGED) - `r_discipline_per_bar` = 3.4 regret signal (NEW) -- α is a fixed config constant (default 0.7), NOT learned, NOT controller-modulated. Per `feedback_no_feature_flags`: no `enable_*` toggle. +- **α is ISV-driven** at slot `ALPHA_SPLIT` (slot 417 per §4.3). Producer reads `GRAD_NORM_QUALITY` (slot 418, EMA of |∇r_quality|) and `GRAD_NORM_DISCIPLINE` (slot 419, EMA of |∇r_discipline|), computes `α = grad_norm_q / (grad_norm_q + grad_norm_d + ε)` so the two signals contribute equal gradient pressure regardless of fold-scale. +- Per `feedback_isv_for_adaptive_bounds` — α tracks the actual gradient distribution rather than being a hardcoded 0.7 guess. (Initial sentinel α = 0.5 until Welford warm-up completes; first-observation bootstrap per `pearl_first_observation_bootstrap`.) **Anchor tests**: 2.4 (cost_sensitivity), 2.6 (regime_silences). Both should be made simultaneously achievable by the split. @@ -405,15 +468,28 @@ Teaches "Hold when no edge: fine. Hold when edge exists: bad." #### 3.5 — Confidence-aware Hold floor -Action-selection level (NOT reward shaping): +Action-selection level (NOT reward shaping). **Function form: bounded sigmoid** (specified to avoid implementer ambiguity): ``` -hold_floor = monotonic_inv_func(entropy(dir_Q_distribution)) +hold_floor = α × σ(k × (entropy − ε₀)) ``` -When entropy is high (low confidence) → bias toward Hold by adding `+hold_floor` to the Hold-Q before argmax/Thompson selection. Hold becomes uncertainty expression, not distributional default. +Where: +- `entropy` = Shannon entropy of dir_Q_distribution (range `[0, log(K_dir)]`, K_dir=4 → max ≈ 1.386) +- `α` = scale factor in `HOLD_FLOOR_ALPHA` (slot 426, ISV-tracked from running max of |Q_dir| so the bias is comparable to Q magnitudes) +- `k` = sigmoid steepness in `HOLD_FLOOR_K` (slot 427, ISV-tracked from running variance of entropy — wider entropy distribution → gentler slope) +- `ε₀` = entropy threshold in `HOLD_FLOOR_EPS0` (slot 428, ISV-tracked at 75th percentile of running entropy distribution `ENTROPY_DIST_REF` slot 429 — biases Hold only when policy is in upper quartile of uncertainty) +- `σ` = standard logistic sigmoid -`monotonic_inv_func` parameters (slope, anchor) are ISV-driven from running entropy distribution. +Sigmoid (NOT linear, NOT exp) chosen because: +- Linear: doesn't saturate — at extreme entropy it can dominate Q values, breaking action selection at confident peaks too +- Exp: dominates too aggressively at full entropy, fails 2.2/2.3 (trend / mean-revert) +- Sigmoid: bounded `[0, α]`, smooth transition, parametrizable steepness — matches the qualitative "bias when uncertain, transparent when confident" behavior + +When `entropy ≪ ε₀` (high confidence) → `hold_floor ≈ 0` → no bias. +When `entropy ≫ ε₀` (low confidence) → `hold_floor ≈ α` → strong Hold bias. + +Hold-Q at action selection: `Q_hold_effective = Q_hold + hold_floor`. **Anchor tests**: 2.1 (flat_market_holds), 2.6 (regime_silences). Hold must dominate when dir entropy is high. @@ -436,12 +512,22 @@ Model can escape its own losses — break the spiral observed in train-dd4xl. #### 3.5.2 — Asymmetric reward under DD ``` -For gains: r_adjusted = r × (1 + λ × dd_pct) -For losses: r_adjusted = r (unchanged) +r_adjusted = r × (1 + λ × dd_pct) for gains (r > 0) +r_adjusted = r for losses (r ≤ 0) ``` -- `λ` ≈ 0.5 (ISV-tracked from observed DD distribution) -- Layered on top of SP12 asymmetric pos/neg caps. Composes cleanly because SP12 is event-level value cap; this is gain-side multiplier conditioned on DD state. +**Composition with SP12 caps (resolves Nit 12)**: multiplier applies BEFORE the SP12 pos-cap clamp. + +``` +intermediate = r_quality_pre_cap × (1 + λ × dd_pct) [this teaching's multiplier] +r_quality_post_cap = clamp(intermediate, NEG_CAP, POS_CAP) [SP12's cap stays final ceiling] +``` + +- For `dd_pct = 0` (at ATH): `(1 + 0) = 1` → no-op, SP12 cap unchanged. +- For `dd_pct > 0` AND `r_quality_pre_cap × (1 + λ × dd_pct) > POS_CAP`: the multiplier saturates the cap. Behaviorally correct — recovery trades hit ceiling sooner, encouraging frequent small recoveries over rare-large. +- SP12 NEG_CAP (loss side) still applies asymmetrically; this teaching doesn't touch loss multiplier. + +`λ` is ISV-tracked at `DD_ASYMMETRY_LAMBDA` (slot 430). Producer reads `DD_DIST_VAR` (slot 432, running variance of dd_pct over fold) — wider DD distribution → smaller λ (less aggressive multiplier when DD is volatile and unstable). Initial λ ≈ 0.5 sentinel. Teaches "a winning trade out of DD is worth more than the same trade at ATH." Matches actual trading P&L psychology — recovering from drawdown is harder than maintaining ATH, so recovery gains deserve more reward signal. @@ -450,31 +536,44 @@ Teaches "a winning trade out of DD is worth more than the same trade at ATH." Ma #### 3.5.3 — Cooldown gate After K consecutive losing trades, force Hold for M bars. Both K and M are ISV-driven: -- K: tracked from running variance of trade outcomes (cooldown trips when streak exceeds typical loss-cluster length) -- M: tracked from time-to-mean-reversion of vol_normalizer signal (cooldown lasts until regime stabilises) +- **K (loss-streak threshold)**: at slot `COOLDOWN_K_THRESHOLD` (slot 433). Producer reads running mean of per-trade PnL (NOT variance — variance is LOW during a streak when all losses cluster, which would delay cooldown precisely when needed). When running mean drops below a fold-percentile threshold (e.g., 25th percentile), K is reduced — cooldown trips sooner. Concretely: `K = max(2, floor(median_streak_length × clamp(mean_pnl_recent / mean_pnl_baseline, 0.5, 2.0)))`. This makes K small (cooldown sensitive) when recent trades are below baseline. +- **M (cooldown duration)**: at slot `COOLDOWN_M_BARS` (slot 434). Tracked from time-to-mean-reversion of vol_normalizer signal — cooldown lasts until regime stabilises. -**Cooldown counter is part of state** — model can reason about it, not work around it. New ISV slot `COOLDOWN_BARS_REMAINING_INDEX`. Producer decrements per bar; consumer (action selection) forces Hold while > 0. +**Cooldown counter as state**: ISV slot `COOLDOWN_BARS_REMAINING` (slot 435). Producer decrements per bar; consumer (action selection) forces Hold while > 0. Model SEES the counter on every forward — can reason about it, not work around it. **Anchor test**: 2.12 (cooldown_engagement). #### 3.5.4 — Plasticity injection during persistent drawdown -When `dd_persistence_bars > threshold` (Phase 1.3 ISV slot), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima. +When `DD_PERSISTENCE` (slot 404 per §4.3) exceeds `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima. -- Trigger fires at most once per fold (debounced via additional ISV slot `PLASTICITY_INJECTION_FIRED_INDEX`). -- Threshold is ISV-driven from running mean of `dd_persistence_bars` over training history. -- Reset target: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely. +- **Debounced**: fires at most once per fold via `PLASTICITY_FIRED_THIS_FOLD` (slot 436). +- **Threshold ISV-driven**: `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437) tracks running mean of `dd_persistence_bars` over training history. Set to mean + 2σ so only persistent (statistically unusual) drawdowns trigger. +- **Reset target**: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely. +- **Mandatory cooldown interlock (resolves Critical #3)**: when plasticity fires, the kernel ALSO writes `M_warm` to `PLASTICITY_WARM_BARS_REMAINING` (slot 438). This slot routes into the 3.5.3 cooldown consumer via OR-gate: `effective_cooldown = max(COOLDOWN_BARS_REMAINING, PLASTICITY_WARM_BARS_REMAINING)`. While `PLASTICITY_WARM_BARS_REMAINING > 0`, action selection forces Hold (same path as 3.5.3 cooldown). Prevents the random-tail-output → loss-streak → re-trigger loop. M_warm = 200 bars (initial sentinel; ISV-tracked post-fold from observed Adam re-equilibration time). -Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green. +Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green AND 2.22 (NEW interlock test) green. -**Anchor test**: 2.10 (recovery_after_streak) AND must NOT regress 2.16 (q_value_bounded_under_replay) — plasticity reset cannot create unbounded Q values. +**Anchor tests**: +- 2.10 (recovery_after_streak) — plasticity must improve recovery, not break it. +- 2.16 (q_value_bounded_under_replay) — plasticity reset cannot create unbounded Q values. +- 2.22 (NEW: plasticity_cooldown_interlock) — when plasticity fires, action selection MUST force Hold for full M_warm bars; no trades during warm-up period. -#### 3.5.5 — Recovery curriculum in PER +#### 3.5.5 — Recovery curriculum in PER (per-transition proxy) -When sampling from PER, **oversample episodes where policy started in drawdown and recovered to flat** (`dd > 0.05` at start, `dd < 0.01` at end). Provides positive recovery examples for credit assignment. +**Replaces episode-level metadata** (which doesn't exist in the transition-level PER) with a per-step proxy that's already computed. -- Oversample factor: ISV-driven from current-DD context (more DD now → more weight on recovery episodes). -- Currently PER oversamples loss states (high TD-error), which compounds the spiral. This counter-balances by giving "I survived a similar drawdown" examples more weight. +PER stores transitions `(s_t, a_t, r_t, s_{t+1})` with TD-error priority. Adding episode-level "started in DD, recovered to flat" tags requires structural PER changes (episode boundaries, episode-level indices) which are out of scope. + +**Per-transition proxy**: +- New per-step ISV signal `DD_TRAJECTORY_DECREASING` (slot 439): boolean (or smoothed scalar in `[0, 1]`) indicating `dd_pct(t) < dd_pct(t-1) AND dd_pct(t-1) > 0.02`. True when the transition is part of a recovery (DD shrinking from a non-trivial drawdown). +- This signal is COMPUTED PER-BAR by the existing dd_pct producer (sub 1.5) — no PER restructure, no episode tracking, no replay buffer schema change. +- When PER samples a batch, weight transitions by `1.0 + RECOVERY_OVERSAMPLE_WEIGHT × DD_TRAJECTORY_DECREASING_t` (slot 440). Transitions on a recovery path get amplified sampling weight. +- `RECOVERY_OVERSAMPLE_WEIGHT` is ISV-driven from current `dd_pct`: more DD now → higher weight (current trajectory needs more recovery examples). + +This still gives "transitions where recovery is happening" higher gradient signal — credit assignment for recovery patterns — without requiring episode metadata. + +Currently PER oversamples loss states (high TD-error), which compounds the spiral. This counter-balances by giving "I survived a similar drawdown" examples more weight via per-step trajectory signal. **Anchor test**: 2.10 (recovery_after_streak). @@ -499,7 +598,7 @@ Produce trustworthy "this trained system is/isn't a trader" signal. Phase 4 is t ### 10.2 Sub 4.1 — Pre-flight (~½ day) -- All 21 Phase 2 tests green on dev (RTX 3050 Ti). Pre-commit hook blocks if not. +- All 22 Phase 2 tests green on dev (RTX 3050 Ti). Pre-commit hook blocks if not. - Per-pearl audit doc updated, MEMORY.md index extended for any new pearls landed. - New worktree `sp15-final-validation` for the deployment artifacts. @@ -557,7 +656,7 @@ Deltas vs 8 baselines (Q9): vs naive_revert : +X.X (need > 0) Behavioral test status (final, on dev): - 21/21 passing + 22/22 passing Pearls landed: Architectural changes: dd_pct as state, cost-net unified, 8 baselines, Q9 sealed @@ -565,11 +664,18 @@ Architectural changes: dd_pct as state, cost-net unified, 8 baselines, Q9 sealed ### 10.6 Sub 4.5 — Production-track gate -**Pass criteria**: -1. Q9 `sharpe_net > 1.0` -2. Q9 `Calmar > 1.0` -3. Q9 beats ALL 8 baselines on net sharpe (delta > 0 for each) -4. All 21 Phase 2 tests still pass post-deployment +**Pass criteria** (thresholds are config-driven via `--gate-min-sharpe`, `--gate-min-calmar`, `--gate-baselines-must-beat`): + +| Threshold | Default | Rationale | +|---|---|---| +| `min_sharpe_net` | 1.0 | Floor below which deploying is worse than passive holding (buyhold typical sharpe). To be tightened post-Phase-1 once we observe what baseline numbers actually look like on Q9. | +| `min_calmar` | 1.0 | Calmar = annualized_return / max_dd. 1.0 means recovering one max-drawdown's worth of capital per year — a survival-bar threshold. | +| `must_beat_all_baselines` | true | Every baseline delta > 0. Most rigorous; can be weakened to "beats N of 8" if the bar proves too tight in practice. | +| Phase 2 status | all 22 green | No regression in behavioral tests | + +**Initial defaults are explicitly placeholders.** Phase 1 baseline runs will reveal actual baseline performance distribution. Phase 4.4 report card (first run) anchors realistic thresholds. If buyhold turns out to give sharpe_net = 0.5 on Q9, then "beats buyhold" is the floor and we tighten min_sharpe to e.g. 0.6. If buyhold gives 1.5, we tighten to 1.6+. + +The thresholds are NOT hardcoded as architectural assumptions — they're config-driven AND empirically anchored. **Fail mode → behavioral test mapping**: @@ -592,7 +698,7 @@ NOT a generic re-train. Each failure axis has a specific behavioral test to targ |---|---|---|---| | 0 | ~400 | 1-2 | EGF retune kernels, 4 ISV slots, test 2.21 | | 1 | ~700 | 2-3 | sharpe kernel, cost kernel, 8 baselines, dd_pct trunk concat, CLI flags | -| 2 | ~2000 | 3-5 | Synthetic markets, harness, 21 tests, pre-commit hook | +| 2 | ~2100 | 3-5 | Synthetic markets, harness, 22 tests, pre-commit hook | | 3 | ~800 | 3-5 | 5 teachings (reward + Hold floor) | | 3.5 | ~700 | 2-3 | 4 recovery mechanisms | | 4 | ~300 | 2-3 | argo-eval-final.sh, report card automation | @@ -618,11 +724,17 @@ Each baseline runs as a separate kernel against the val/test window. 8× the eva - Run baselines only once per fold per epoch (not per step) - Estimated overhead: ~5-10% of total eval cost -### 12.3 Q9 burn policy +### 12.3 Q9 burn policy is honor-system -"Each Q9 run generates audit file" relies on discipline, not enforcement. A motivated developer could run Q9 eval N times and only commit the best. Mitigation: -- Q9 run requires a tagged commit — `argo-eval-final.sh` rejects un-tagged or amended commits -- Audit log includes counter incrementing per architectural milestone — anomalous gaps surface gaming +**Explicit acknowledgment**: this is honor-system, not enforcement. No tooling makes Q9 contamination impossible — it requires developer honesty. + +A motivated developer could run Q9 eval N times and only commit the best. Even with tag requirements, lightweight tags are mutable; annotated tags can be deleted and recreated. The proposed mitigations reduce convenience of cheating but don't prevent it: + +- `argo-eval-final.sh` rejects un-tagged or amended commits (raises bar slightly) +- Each Q9 run writes a counter + commit-hash + result-summary to `trained-models-pvc` infrastructure (chain-hash). Retroactive deletion would leave detectable gaps in the chain. +- Per-fold audit doc commits go to main as part of every Q9 run; log of "I ran Q9" is git-history-visible. + +These are deterrents, not enforcement. Accepting the discipline-only nature of this is honest spec hygiene. The alternative (cryptographic infrastructure for unfakeable Q9 evaluation logs) is over-engineering for a single-developer project. ### 12.4 Synthetic LOB fidelity for Phase 2 @@ -656,19 +768,22 @@ Mitigation: --- -## 13. Anticipated memory pearls +## 13. Pearl candidates pending validation -To be authored alongside the implementation phases: +**Important**: Section 14's discipline rule says pearls emerge from validation, not pre-planning. The list below is **candidates for pearl authoring IF AND ONLY IF the technique demonstrably generalises** beyond this spec. Some, all, or none may be authored. Pre-naming creates risk of authoring-pressure regardless of generalisation; the candidate framing acknowledges this. -| Pearl | Origin | -|---|---| -| `pearl_dd_pct_foundational_state` | Phase 1.5 — DD-context as eval-time state input vs reward-only modulation | -| `pearl_unified_cost_kernel_dev_prod_parity` | Phase 1.2 — same cost kernel reads spread/OFI from synthetic (dev) or real (prod) LOB | -| `pearl_behavioral_test_gates_l40s_deploy` | Phase 2 — pre-commit hook + dev-runtime test suite is the L40S filter | -| `pearl_recovery_curriculum_in_per` | Phase 3.5.5 — PER weighting can encode "recovery exists" signal explicitly | -| `pearl_plasticity_injection_for_rl_local_minima` | Phase 3.5.4 — weight-reset mechanism for breaking spiral attractors (if 2.10 verifies) | -| `pearl_8_counterfactual_baselines_arch_specific` | Phase 1.4 — domain baselines that test specific subsystems we built (aux-only, mag-Q-fixed, trail-only) | -| `pearl_egf_isv_driven_anchors` | Phase 0.B — Schmitt thresholds and variance refs as ISV slots, not constants | +| Candidate | Authoring trigger | Origin | +|---|---|---| +| `pearl_dd_pct_foundational_state` | If 2.11 (dd_state_aware_sizing) shows policy reliably conditions on dd_pct | Phase 1.5 | +| `pearl_unified_cost_kernel_dev_prod_parity` | If Phase 4 Q9 net metrics match dev tests within tolerance (cost calibration verified) | Phase 1.2 | +| `pearl_behavioral_test_gates_l40s_deploy` | If pre-commit hook catches at least one regression that would have shipped | Phase 2 | +| `pearl_recovery_curriculum_per_step_proxy` | If 2.10 passes via 3.5.5 alone (without plasticity 3.5.4 dominating) | Phase 3.5.5 | +| `pearl_plasticity_injection_for_rl_local_minima` | If 3.5.4 fires AND 2.10 measurably improves AND no 2.16 regression | Phase 3.5.4 | +| `pearl_8_counterfactual_baselines_arch_specific` | If at least 2 of the architecture-specific baselines (aux-only, mag-Q-fixed, trail-only) reveal a real subsystem-not-earning-keep finding | Phase 1.4 | +| `pearl_egf_isv_driven_anchors` | If Phase 0.B retune makes gate1 actually open under target conditions (test 2.21 green AND non-trivial fire rate on real workloads) | Phase 0.B | +| `pearl_plasticity_cooldown_interlock` | If 2.22 verifies the OR-gate prevents re-trigger loops in real training | Phase 3.5.4 | + +Any candidate that doesn't trigger after Phase 4 — drop, don't author. The discipline rule cuts both ways: pearls only get written when the technique generalised. ---