gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(sp15): amend spec — 14 fixes from critical review

Browse Source 
Critical review (combined fresh-eyes subagent + self-critique) flagged
14 issues. All addressed in this amendment.

CRITICAL (blocks implementation):
- §4.3 NEW: ISV slot allocation map [397..441) per phase, prevents
   parallel-dispatch ISV-table conflict (was: undefined, would break
   Approach B)
- §4.4 NEW: Phase 2A ↔ Phase 1.2 ABI contract (LobBar struct as
   canonical contract, Phase 2A defines first; Phase 1.2 conforms).
   Restores parallelism dependency
- §9.2 (3.5.4): plasticity injection now MANDATES cooldown engagement
   via PLASTICITY_WARM_BARS_REMAINING slot 438. NEW anchor test 2.22
   plasticity_cooldown_interlock prevents random-tail-output retrigger
   loop
- §9.2 (3.5.5): recovery curriculum replaced episode-level metadata
   (didn't exist in transition-level PER) with per-step DD_TRAJECTORY_
   DECREASING signal — same goal, no PER restructure

IMPORTANT (rework prevention):
- §9.2 (3.5.3): cooldown K threshold from running mean of per-trade
   PnL (NOT variance — variance is LOW during streaks, would delay
   cooldown when needed)
- §8.2 (3.5): Hold floor function form specified as bounded sigmoid
   with α/k/ε₀ ISV-driven; was "monotonic_inv_func" (undefined family)
- §6.2: cost kernel per-side semantics explicit. Half-spread × side_
   indicator at entry AND exit; commission_per_rt on close. Resolves
   round-trip vs per-side ambiguity
- §6.2: OFI impact λ initial value 2e-4 + ISV refit methodology;
   was "empirical fit from MES historical" (hand-wavy)
- §6.4: 8 baselines now mandate shared trunk forward; honest 15-25%
   overhead estimate (was 5-10%, optimistic)
- §12.3: Q9 burn policy explicitly honor-system, not enforcement.
   Mitigations described as deterrents not enforcement
- §10.6: Phase 4.5 thresholds CLI-config-driven with rationale,
   anchored empirically post-Phase-1 (was hardcoded > 1.0)

NIT:
- §9.2 (3.5.2): multiplier vs SP12 cap interaction explicit (applied
   BEFORE cap)
- §8.2 (3.1): α=0.7 hardcoded → ISV-driven from grad ratio (was
   feedback_isv_for_adaptive_bounds violation)
- §13: pearls reframed as candidates pending validation per discipline
   rule; pre-naming pressure removed

Test count: 21 → 22 (added 2.22). Phase 2 LOC: 2000 → 2100.
ISV_TOTAL_DIM: 396 → 441 post-SP15.

All 14 issues addressed in spec text. Ready for user review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-06 08:51:35 +02:00
parent fb5f2394d0
commit f3cb0c78c2
1 changed files with 193 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

271
docs/superpowers/specs/2026-05-06-trader-discipline-and-recovery-design.md
View File

@@ -12,7 +12,7 @@ Take the system from "stable, gradient-flowing, but trades like a panicked toddl
1. **Phase 0** — Retune SP14 EGF (gate1=closed forever) so its anchors are ISV-driven, not hardcoded.
2. **Phase 1** — Honest numbers: unified sharpe kernel, cost-net sharpe (commission + spread + OFI-impact, dev/prod parity), drawdown reporting, 8 counterfactual baselines, `dd_pct` as foundational state input, `--holdout-quarters` + `--dev-quarters` CLI flags, consume abandoned walk-forward test slice.
3. **Phase 2** — 21-test behavioral suite on dev RTX 3050 Ti. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S.
3. **Phase 2** — 22-test behavioral suite on dev RTX 3050 Ti. Pre-commit hook blocks `argo-train.sh` if any test fails. CI gate before L40S.
4. **Phase 3** — 5 trader teachings (reward + action selection): r_quality/r_discipline split, explicit cost, quadratic DD penalty, regret signal, confidence-aware Hold floor.
5. **Phase 3.5** — 4 recovery mechanisms (3.5.1 already in Phase 1): asymmetric reward under DD, cooldown gate, plasticity injection, recovery curriculum in PER.
6. **Phase 4** — L40S walk-forward on Q1-Q7, Q8 final dev, sealed Q9 OOS — produces report card with deltas vs all 8 baselines.
@@ -109,7 +109,7 @@ Val is temporally clean for parameter learning, but it's the selection set, not
| Q2 | SP14 EGF cut vs retune | **Keep, retune to ISV-driven**. No hardcoded thresholds; Schmitt thresholds, variance gate refs, steepness anchors all become ISV-tracked. |
| Q3 | Cost model dev/prod parity | **commission + spread + OFI-impact, everywhere**. Dev synthetic markets generate synthetic spread + OFI. Prod reads from MBP10 fxcache. Same kernel. |
| Q4 | Counterfactual baselines | **All 8**: buyhold, hold-only, random_dir + same Kelly, naive momentum, aux-only, mag-Quarter-fixed, trail-only, naive reversion. |
| Q5 | Phase 2 test set | **All 21 tests**: 7 trading behavior + 10 state/arch grounding + 4 recovery dynamics. |
| Q5 | Phase 2 test set | **22 tests**: 7 trading behavior + 10 state/arch grounding + 5 recovery dynamics (added 2.22 plasticity_cooldown_interlock per critical-review fix). |
| Q6 | OOS framework | **Train/dev/test split**: Q1-Q7 walk-forward train, Q8 final dev, Q9 sealed final test. |
| Q7 | Phase 3.5.4 plasticity injection | **Land all 4 recovery mechanisms** including plasticity. No defer. |
@@ -146,7 +146,50 @@ Phase 2A: behavioral test scaffolding ─┘
- `sp15-phase2a-test-scaffold` — synthetic market generators + harness (parallel to Phase 0+1)
- **Merge model**: each phase merges back to `sp15-trader-discipline-recovery` atomically. SP15 branch merges to main only after Phase 4 production-track gate passes.
### 4.3 Cross-cutting discipline rule
### 4.3 ISV slot allocation map (resolves parallel-dispatch ISV-table conflict)
Approach B parallel dispatch requires that Phase 0 / Phase 1 / Phase 2A sub-worktrees not collide on ISV slot indices. The spec pre-allocates disjoint slot ranges per phase. SP14 occupied slots 383-395 (`ISV_TOTAL_DIM = 396` post-merge `0275a25d9`). SP15 starts at 397.
| Range | Owner | Purpose | Count |
|---|---|---|---|
| `[397..401)` | Phase 0.B (EGF retune) | `EGF_SCHMITT_HI`, `EGF_SCHMITT_LO`, `EGF_VAR_AUX_REF`, `EGF_VAR_Q_REF` | 4 |
| `[401..407)` | Phase 1.3 (drawdown) | `DD_CURRENT`, `DD_MAX`, `DD_RECOVERY_BARS`, `DD_PERSISTENCE`, `CALMAR`, `DD_PCT` | 6 |
| `[407..409)` | Phase 1.2 (cost kernel) | `OFI_IMPACT_LAMBDA` (ISV-tracked), `COST_PER_BAR_AVG` | 2 |
| `[409..417)` | Phase 1.4 (8 baselines) | one `BASELINE_SHARPE_NET_*` per baseline | 8 |
| `[417..420)` | Phase 3.1 (α split) | `ALPHA_SPLIT` (ISV-driven from grad ratio), `GRAD_NORM_QUALITY`, `GRAD_NORM_DISCIPLINE` | 3 |
| `[420..423)` | Phase 3.3 (DD penalty) | `LAMBDA_DD`, `DD_THRESHOLD`, `DD_PENALTY_GRAD_NORM` | 3 |
| `[423..426)` | Phase 3.4 (regret) | `REGRET_EMA`, `LAMBDA_REGRET`, `REGRET_GRAD_NORM` | 3 |
| `[426..430)` | Phase 3.5 (Hold floor) | `HOLD_FLOOR_ALPHA`, `HOLD_FLOOR_K`, `HOLD_FLOOR_EPS0`, `ENTROPY_DIST_REF` | 4 |
| `[430..433)` | Phase 3.5.2 (DD asymmetric reward) | `DD_ASYMMETRY_LAMBDA` (ISV-tracked from DD distribution), `R_GAIN_DD_BOOST`, `DD_DIST_VAR` | 3 |
| `[433..436)` | Phase 3.5.3 (cooldown) | `COOLDOWN_K_THRESHOLD` (ISV from running mean of per-trade PnL), `COOLDOWN_M_BARS`, `COOLDOWN_BARS_REMAINING` | 3 |
| `[436..439)` | Phase 3.5.4 (plasticity) | `PLASTICITY_FIRED_THIS_FOLD`, `PLASTICITY_PERSISTENCE_THRESHOLD`, `PLASTICITY_WARM_BARS_REMAINING` (force cooldown engagement post-fire) | 3 |
| `[439..441)` | Phase 3.5.5 (recovery curriculum) | `DD_TRAJECTORY_DECREASING` (per-step proxy for "in recovery"), `RECOVERY_OVERSAMPLE_WEIGHT` | 2 |
| **Total** | | **`ISV_TOTAL_DIM = 441` post-SP15** | 44 |
Each phase's first commit lands `sp15_isv_slots.rs` extension covering its allocated range. Layout fingerprint seed extended atomically per phase. Sub-worktrees check the allocation map before assigning indices.
### 4.4 Phase 2A ↔ Phase 1.2 ABI contract (resolves parallelism dependency)
The synthetic-LOB triple `(price_t, spread_t, ofi_t)` is the canonical ABI between Phase 2A test markets and Phase 1.2 cost kernel. Phase 2A defines this contract first; Phase 1.2 kernel signature must conform.
```rust
// Canonical ABI defined by Phase 2A scaffolding, consumed by Phase 1.2:
struct LobBar {
price: f32, // mid-price, units = MES futures dollar
spread: f32, // bid-ask spread, units = same as price (e.g., 0.25 = quarter-tick)
ofi: f32, // order flow imbalance signed magnitude, normalized to ~[-3, 3]
}
// Synthetic markets emit Vec<LobBar>; fxcache loader emits Vec<LobBar>; cost kernel reads &[LobBar]
```
Phase 2A lands the `LobBar` type + synthetic generators FIRST. Phase 1.2 cost kernel reads `&[LobBar]`. Phase 0 + Phase 1 (other subs) + Phase 2A scaffold can then dispatch in parallel because:
- Phase 0 doesn't touch LobBar (EGF kernels)
- Phase 1.1, 1.3-1.7 don't touch LobBar (sharpe kernel, DD reporting, baselines, dd_pct, CLI, test slice consumption)
- Phase 1.2 conforms to Phase 2A's pre-defined ABI
This restores Approach B parallelism with explicit contract precedence.
### 4.5 Cross-cutting discipline rule
**No new pearl, controller, kernel, ISV slot, or reward term ships without:**
1. **Phase 2 behavioral test** that proves the intended trader behavior (the test commit and the implementation commit ship together).
@@ -180,15 +223,15 @@ Either the thresholds are wrong (signal magnitudes are inherently smaller than e
Every hardcoded anchor becomes ISV-driven:
| Anchor (current value) | New ISV slot | Producer |
| Anchor (current value) | ISV slot (per allocation map §4.3) | Producer |
|---|---|---|
| Schmitt thresholds (±0.03 hardcoded) | `EGF_SCHMITT_HI_INDEX`, `EGF_SCHMITT_LO_INDEX` | rolling p99/p1 of α_raw over fold |
| Variance gate ref aux (0.01) | `EGF_VAR_AUX_REF_INDEX` | running mean of var_aux over fold |
| Variance gate ref Q (0.05) | `EGF_VAR_Q_REF_INDEX` | running mean of var_q over fold |
| Steepness normaliser (k_aux=20/(1+var_aux/0.01) hardcoded `0.01`) | reuses `EGF_VAR_AUX_REF_INDEX` | (above) |
| Steepness normaliser k_q (0.05 hardcoded) | reuses `EGF_VAR_Q_REF_INDEX` | (above) |
| Schmitt thresholds (±0.03 hardcoded) | `EGF_SCHMITT_HI` (slot 397), `EGF_SCHMITT_LO` (slot 398) | rolling p99/p1 of α_raw over fold |
| Variance gate ref aux (0.01) | `EGF_VAR_AUX_REF` (slot 399) | running mean of var_aux over fold |
| Variance gate ref Q (0.05) | `EGF_VAR_Q_REF` (slot 400) | running mean of var_q over fold |
| Steepness normaliser (k_aux=20/(1+var_aux/0.01) hardcoded `0.01`) | reuses `EGF_VAR_AUX_REF` (slot 399) | (above) |
| Steepness normaliser k_q (0.05 hardcoded) | reuses `EGF_VAR_Q_REF` (slot 400) | (above) |
Total: 4 new ISV slots + 2 new producer kernels. EGF kernels read these instead of constants.
Total: 4 new ISV slots (`[397..401)` per §4.3) + 2 new producer kernels. EGF kernels read these instead of constants.
### 5.4 Phase 0 anchor test (Phase 2.21 — `egf_gate_opens_under_disagreement`)
@@ -208,16 +251,28 @@ Eliminates `sharpe_ema` (per-batch EMA with smoothing constant divergence) vs `s
### 6.2 Sub 1.2 — Cost-net sharpe
`cost_t = commission + spread_t × position_size + ofi_impact(spread_t, ofi_t)`
```
cost_t = commission_per_rt × rt_indicator(t)
+ half_spread_t × |position_size_t| × side_indicator(t)
+ ofi_impact_t
```
- `commission`: $1.50/RT default, `--commission` CLI overridable
- `spread_t`: real bid-ask from MBP10 fxcache (prod) OR synthetic spread from market generator (dev)
- `ofi_impact()`: linear-impact model `λ × |position_size| × |ofi_t|` where λ is ISV-tracked impact coefficient (initial: empirical fit from MES historical)
- `position_size`: contracts traded (1 for Quarter, 2 for Half, 4 for Full at base Kelly cap)
**Per-side semantics (resolves round-trip ambiguity)**:
- `commission_per_rt`: $1.50 default per round-trip, applied entirely on exit (not split). `--commission` CLI overridable.
- `rt_indicator(t)`: 1.0 on close-trade bars, 0 otherwise.
- `half_spread_t = spread_t / 2`: half-spread cost per side. **Charged BOTH at entry AND at exit.** `side_indicator(t)`: 1.0 on entry bars, 1.0 on exit bars, 0 otherwise. Net: a complete round-trip pays `commission_per_rt + half_spread_at_entry × position + half_spread_at_exit × position` — i.e., one full spread cost per RT plus one commission.
- `ofi_impact_t = λ × |position_size_t| × |ofi_t| × side_indicator(t)`: applied per-side (entry and exit).
- Initial `λ = 2e-4` (rough MES literature fit; impact ≈ 0.2 ticks per contract per σ-unit OFI).
- ISV-tracked at slot `OFI_IMPACT_LAMBDA` (slot 407 per §4.3) — refit per fold by minimizing prediction error of next-tick mid-price shift vs OFI quartile on fold's training window.
- `position_size`: contracts traded (1 for Quarter, 2 for Half, 4 for Full at base Kelly cap).
`sharpe_net = (mean_pnl − cost_per_bar_avg) / std_pnl_with_costs`
**Sharpe net**:
```
sharpe_net = (mean_pnl − mean_cost) / std_pnl_with_costs
```
where `mean_cost = sum(cost_t) / N_bars` is emitted to ISV slot `COST_PER_BAR_AVG` (slot 408).
The cost kernel reads `spread_t` and `ofi_t` as input arrays. Dev synthetic markets generate these as part of fixtures. This is the same kernel in both environments — `feedback_isv_for_adaptive_bounds` extends to costs.
**Dev/prod parity**: the cost kernel reads `spread_t`, `ofi_t` as input arrays. Phase 2A synthetic markets generate these via the §4.4 LobBar ABI. Prod fxcache produces the same LobBar type. One kernel, two environments — `feedback_isv_for_adaptive_bounds` extends to costs.
### 6.3 Sub 1.3 — Drawdown reporting
@@ -229,22 +284,28 @@ New ISV slots:
HEALTH_DIAG emits `max_dd_pct`, `current_dd`, `dd_recovery_bars`, `Calmar = mean_pnl / max(max_dd_pct, 1e-4)`. Denominator floor (1e-4) replaces the current Calmar=100 saturation artifact.
### 6.4 Sub 1.4 — 8 counterfactual baselines
### 6.4 Sub 1.4 — 8 counterfactual baselines (with shared trunk forward)
Each baseline is a pure CUDA kernel that replays the val/test window with a fixed action policy and computes sharpe_net + max_dd + trade_count. All 8 run in parallel kernels per fold per epoch:
Each baseline replays the val/test window with a different action policy and computes sharpe_net + max_dd + trade_count. **Critical implementation detail (resolves Important #9)**: baselines that need policy-trained heads (aux_only, mag_quarter_fixed, trail_only) MUST share the trunk forward pass with the main policy evaluation. Without trunk fusion, 8 separate full-policy kernel launches per fold per epoch would inflate eval cost ~4-8×.
| Baseline | Definition |
|---|---|
| `buyhold` | always Long, mag = Quarter (smallest), no exits |
| `hold_only` | always Hold, never enter |
| `random_dir_kelly` | random direction at random bars (seeded), mag pinned to policy's Kelly cap output |
| `naive_momentum` | direction = sign(last bar return), mag = Quarter |
| `aux_only` | direction = sign(aux head's next-bar prediction), mag = Quarter |
| `mag_quarter_fixed` | policy's direction, mag pinned to Quarter |
| `trail_only` | random entries (seeded), exits via policy's trail-stop logic |
| `naive_reversion` | direction = -sign(last bar return), mag = Quarter |
**Implementation**: a single fused trunk forward computes shared state representation `h_s2` for the val/test window. Then 8 lightweight per-baseline kernels read `h_s2` (where applicable) plus other inputs, output baseline action sequences, run cost-net sharpe kernel.
HEALTH_DIAG emits per-fold deltas: `vs_buyhold = sharpe_net − buyhold_sharpe_net`, etc.
| Baseline | Inputs | Trunk forward needed? |
|---|---|---|
| `buyhold` | price series only | No |
| `hold_only` | none (constant Hold) | No |
| `random_dir_kelly` | seeded RNG, policy Kelly cap output | trunk YES (for Kelly cap) |
| `naive_momentum` | last-bar return only | No |
| `aux_only` | trunk h_s2 + aux head only | trunk YES (subset head) |
| `mag_quarter_fixed` | trunk h_s2 + direction Q head only | trunk YES (subset head) |
| `trail_only` | seeded RNG, policy trail-distance ISV | trunk NO (just trail logic) |
| `naive_reversion` | last-bar return only | No |
Of 8 baselines, 3 need the shared trunk (already computed once for main policy eval). Remaining 5 are constant or last-bar-only — pure kernels with negligible cost.
**Realistic overhead estimate**: ~15-25% of full eval pass (not 5-10% as initially claimed). Acceptable for the validation rigor benefit. Documented honestly so we don't underestimate L40S timing.
HEALTH_DIAG emits per-fold deltas: `vs_buyhold = sharpe_net − buyhold_sharpe_net`, etc. Each baseline's sharpe_net writes to its allocated ISV slot (`BASELINE_SHARPE_NET_*` slots 409-416 per §4.3).
### 6.5 Sub 1.5 — `dd_pct` as foundational state input
@@ -291,7 +352,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050
- **Pre-commit hook** in `.claude/helpers/pre_commit_behavioral_suite.sh`: runs `cargo test -p ml --test behavioral_suite -- --nocapture`. Fail → block commit.
- **CI integration**: `argo-train.sh` invokes the same hook before submitting workflow. Local fail = no L40S deploy.
### 7.3 21 tests by group
### 7.3 22 tests by group
#### Group 1: Trading behavior (7 tests, lands as Phase 1 cost + market gen ships)
@@ -320,7 +381,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050
| 2.20 | `state_input_grounding_test` | KL(uptrend-context, downtrend-context action dists) > 0.5 |
| 2.21 | `egf_gate_opens` (Phase 0 anchor) | aux/Q deliberate disagreement → gate1 opens within 100 steps |
#### Group 3: Recovery dynamics (4 tests, anchors Phase 3.5 teachings)
#### Group 3: Recovery dynamics (5 tests, anchors Phase 3.5 teachings)
| # | Test | Pass criterion |
|---|---|---|
@@ -328,6 +389,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050
| 2.10 | `recovery_after_streak` | inject 10R loss streak → recover ≥5R over next 200 bars (NOT collapse to all-Hold) |
| 2.11 | `dd_state_aware_sizing` | KL(0% DD, 50% dd_budget action dists) > 0.1 — policy conditions sizing on DD |
| 2.12 | `cooldown_engagement` | K=5 losses → cooldown fires within 1 bar; disengages cleanly after M=20 bars |
| 2.22 | `plasticity_cooldown_interlock` | trigger plasticity fire (synthetic persistent DD) → action selection forces Hold for full M_warm bars; zero trades during warm-up; no second plasticity trigger within fold |
### 7.4 Sequencing within Phase 2
@@ -354,7 +416,7 @@ Reward shape and action selection teach the trader role, not just optimize a num
#### 3.1 — r_quality + r_discipline split
Architectural framing, no separate code beyond the convex combination:
Architectural framing:
```
r_total = α × r_quality_per_event + (1 − α) × r_discipline_per_bar
@@ -362,7 +424,8 @@ r_total = α × r_quality_per_event + (1 − α) × r_discipline_per_bar
- `r_quality_per_event` = SP12 per-trade event-driven reward (UNCHANGED)
- `r_discipline_per_bar` = 3.4 regret signal (NEW)
- α is a fixed config constant (default 0.7), NOT learned, NOT controller-modulated. Per `feedback_no_feature_flags`: no `enable_*` toggle.
- **α is ISV-driven** at slot `ALPHA_SPLIT` (slot 417 per §4.3). Producer reads `GRAD_NORM_QUALITY` (slot 418, EMA of |∇r_quality|) and `GRAD_NORM_DISCIPLINE` (slot 419, EMA of |∇r_discipline|), computes `α = grad_norm_q / (grad_norm_q + grad_norm_d + ε)` so the two signals contribute equal gradient pressure regardless of fold-scale.
- Per `feedback_isv_for_adaptive_bounds` — α tracks the actual gradient distribution rather than being a hardcoded 0.7 guess. (Initial sentinel α = 0.5 until Welford warm-up completes; first-observation bootstrap per `pearl_first_observation_bootstrap`.)
**Anchor tests**: 2.4 (cost_sensitivity), 2.6 (regime_silences). Both should be made simultaneously achievable by the split.
@@ -405,15 +468,28 @@ Teaches "Hold when no edge: fine. Hold when edge exists: bad."
#### 3.5 — Confidence-aware Hold floor
Action-selection level (NOT reward shaping):
Action-selection level (NOT reward shaping). **Function form: bounded sigmoid** (specified to avoid implementer ambiguity):
```
hold_floor = monotonic_inv_func(entropy(dir_Q_distribution))
hold_floor = α × σ(k × (entropy − ε₀))
```
When entropy is high (low confidence) → bias toward Hold by adding `+hold_floor` to the Hold-Q before argmax/Thompson selection. Hold becomes uncertainty expression, not distributional default.
Where:
- `entropy` = Shannon entropy of dir_Q_distribution (range `[0, log(K_dir)]`, K_dir=4 → max ≈ 1.386)
- `α` = scale factor in `HOLD_FLOOR_ALPHA` (slot 426, ISV-tracked from running max of |Q_dir| so the bias is comparable to Q magnitudes)
- `k` = sigmoid steepness in `HOLD_FLOOR_K` (slot 427, ISV-tracked from running variance of entropy — wider entropy distribution → gentler slope)
- `ε₀` = entropy threshold in `HOLD_FLOOR_EPS0` (slot 428, ISV-tracked at 75th percentile of running entropy distribution `ENTROPY_DIST_REF` slot 429 — biases Hold only when policy is in upper quartile of uncertainty)
- `σ` = standard logistic sigmoid
`monotonic_inv_func` parameters (slope, anchor) are ISV-driven from running entropy distribution.
Sigmoid (NOT linear, NOT exp) chosen because:
- Linear: doesn't saturate — at extreme entropy it can dominate Q values, breaking action selection at confident peaks too
- Exp: dominates too aggressively at full entropy, fails 2.2/2.3 (trend / mean-revert)
- Sigmoid: bounded `[0, α]`, smooth transition, parametrizable steepness — matches the qualitative "bias when uncertain, transparent when confident" behavior
When `entropy ≪ ε₀` (high confidence) → `hold_floor ≈ 0` → no bias.
When `entropy ≫ ε₀` (low confidence) → `hold_floor ≈ α` → strong Hold bias.
Hold-Q at action selection: `Q_hold_effective = Q_hold + hold_floor`.
**Anchor tests**: 2.1 (flat_market_holds), 2.6 (regime_silences). Hold must dominate when dir entropy is high.
@@ -436,12 +512,22 @@ Model can escape its own losses — break the spiral observed in train-dd4xl.
#### 3.5.2 — Asymmetric reward under DD
```
For gains: r_adjusted = r × (1 + λ × dd_pct)
For losses: r_adjusted = r (unchanged)
r_adjusted = r × (1 + λ × dd_pct) for gains (r > 0)
r_adjusted = r for losses (r ≤ 0)
```
- `λ` ≈ 0.5 (ISV-tracked from observed DD distribution)
- Layered on top of SP12 asymmetric pos/neg caps. Composes cleanly because SP12 is event-level value cap; this is gain-side multiplier conditioned on DD state.
**Composition with SP12 caps (resolves Nit 12)**: multiplier applies BEFORE the SP12 pos-cap clamp.
```
intermediate = r_quality_pre_cap × (1 + λ × dd_pct) [this teaching's multiplier]
r_quality_post_cap = clamp(intermediate, NEG_CAP, POS_CAP) [SP12's cap stays final ceiling]
```
- For `dd_pct = 0` (at ATH): `(1 + 0) = 1` → no-op, SP12 cap unchanged.
- For `dd_pct > 0` AND `r_quality_pre_cap × (1 + λ × dd_pct) > POS_CAP`: the multiplier saturates the cap. Behaviorally correct — recovery trades hit ceiling sooner, encouraging frequent small recoveries over rare-large.
- SP12 NEG_CAP (loss side) still applies asymmetrically; this teaching doesn't touch loss multiplier.
`λ` is ISV-tracked at `DD_ASYMMETRY_LAMBDA` (slot 430). Producer reads `DD_DIST_VAR` (slot 432, running variance of dd_pct over fold) — wider DD distribution → smaller λ (less aggressive multiplier when DD is volatile and unstable). Initial λ ≈ 0.5 sentinel.
Teaches "a winning trade out of DD is worth more than the same trade at ATH." Matches actual trading P&L psychology — recovering from drawdown is harder than maintaining ATH, so recovery gains deserve more reward signal.
@@ -450,31 +536,44 @@ Teaches "a winning trade out of DD is worth more than the same trade at ATH." Ma
#### 3.5.3 — Cooldown gate
After K consecutive losing trades, force Hold for M bars. Both K and M are ISV-driven:
- K: tracked from running variance of trade outcomes (cooldown trips when streak exceeds typical loss-cluster length)
- M: tracked from time-to-mean-reversion of vol_normalizer signal (cooldown lasts until regime stabilises)
- **K (loss-streak threshold)**: at slot `COOLDOWN_K_THRESHOLD` (slot 433). Producer reads running mean of per-trade PnL (NOT variance — variance is LOW during a streak when all losses cluster, which would delay cooldown precisely when needed). When running mean drops below a fold-percentile threshold (e.g., 25th percentile), K is reduced — cooldown trips sooner. Concretely: `K = max(2, floor(median_streak_length × clamp(mean_pnl_recent / mean_pnl_baseline, 0.5, 2.0)))`. This makes K small (cooldown sensitive) when recent trades are below baseline.
- **M (cooldown duration)**: at slot `COOLDOWN_M_BARS` (slot 434). Tracked from time-to-mean-reversion of vol_normalizer signal — cooldown lasts until regime stabilises.
**Cooldown counter is part of state** — model can reason about it, not work around it. New ISV slot `COOLDOWN_BARS_REMAINING_INDEX`. Producer decrements per bar; consumer (action selection) forces Hold while > 0.
**Cooldown counter as state**: ISV slot `COOLDOWN_BARS_REMAINING` (slot 435). Producer decrements per bar; consumer (action selection) forces Hold while > 0. Model SEES the counter on every forward — can reason about it, not work around it.
**Anchor test**: 2.12 (cooldown_engagement).
#### 3.5.4 — Plasticity injection during persistent drawdown
When `dd_persistence_bars > threshold` (Phase 1.3 ISV slot), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima.
When `DD_PERSISTENCE` (slot 404 per §4.3) exceeds `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima.
- Trigger fires at most once per fold (debounced via additional ISV slot `PLASTICITY_INJECTION_FIRED_INDEX`).
- Threshold is ISV-driven from running mean of `dd_persistence_bars` over training history.
- Reset target: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely.
- **Debounced**: fires at most once per fold via `PLASTICITY_FIRED_THIS_FOLD` (slot 436).
- **Threshold ISV-driven**: `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437) tracks running mean of `dd_persistence_bars` over training history. Set to mean + 2σ so only persistent (statistically unusual) drawdowns trigger.
- **Reset target**: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely.
- **Mandatory cooldown interlock (resolves Critical #3)**: when plasticity fires, the kernel ALSO writes `M_warm` to `PLASTICITY_WARM_BARS_REMAINING` (slot 438). This slot routes into the 3.5.3 cooldown consumer via OR-gate: `effective_cooldown = max(COOLDOWN_BARS_REMAINING, PLASTICITY_WARM_BARS_REMAINING)`. While `PLASTICITY_WARM_BARS_REMAINING > 0`, action selection forces Hold (same path as 3.5.3 cooldown). Prevents the random-tail-output → loss-streak → re-trigger loop. M_warm = 200 bars (initial sentinel; ISV-tracked post-fold from observed Adam re-equilibration time).
Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green.
Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green AND 2.22 (NEW interlock test) green.
**Anchor test**: 2.10 (recovery_after_streak) AND must NOT regress 2.16 (q_value_bounded_under_replay) — plasticity reset cannot create unbounded Q values.
**Anchor tests**:
- 2.10 (recovery_after_streak) — plasticity must improve recovery, not break it.
- 2.16 (q_value_bounded_under_replay) — plasticity reset cannot create unbounded Q values.
- 2.22 (NEW: plasticity_cooldown_interlock) — when plasticity fires, action selection MUST force Hold for full M_warm bars; no trades during warm-up period.
#### 3.5.5 — Recovery curriculum in PER
#### 3.5.5 — Recovery curriculum in PER (per-transition proxy)
When sampling from PER, **oversample episodes where policy started in drawdown and recovered to flat** (`dd > 0.05` at start, `dd < 0.01` at end). Provides positive recovery examples for credit assignment.
**Replaces episode-level metadata** (which doesn't exist in the transition-level PER) with a per-step proxy that's already computed.
- Oversample factor: ISV-driven from current-DD context (more DD now → more weight on recovery episodes).
- Currently PER oversamples loss states (high TD-error), which compounds the spiral. This counter-balances by giving "I survived a similar drawdown" examples more weight.
PER stores transitions `(s_t, a_t, r_t, s_{t+1})` with TD-error priority. Adding episode-level "started in DD, recovered to flat" tags requires structural PER changes (episode boundaries, episode-level indices) which are out of scope.
**Per-transition proxy**:
- New per-step ISV signal `DD_TRAJECTORY_DECREASING` (slot 439): boolean (or smoothed scalar in `[0, 1]`) indicating `dd_pct(t) < dd_pct(t-1) AND dd_pct(t-1) > 0.02`. True when the transition is part of a recovery (DD shrinking from a non-trivial drawdown).
- This signal is COMPUTED PER-BAR by the existing dd_pct producer (sub 1.5) — no PER restructure, no episode tracking, no replay buffer schema change.
- When PER samples a batch, weight transitions by `1.0 + RECOVERY_OVERSAMPLE_WEIGHT × DD_TRAJECTORY_DECREASING_t` (slot 440). Transitions on a recovery path get amplified sampling weight.
- `RECOVERY_OVERSAMPLE_WEIGHT` is ISV-driven from current `dd_pct`: more DD now → higher weight (current trajectory needs more recovery examples).
This still gives "transitions where recovery is happening" higher gradient signal — credit assignment for recovery patterns — without requiring episode metadata.
Currently PER oversamples loss states (high TD-error), which compounds the spiral. This counter-balances by giving "I survived a similar drawdown" examples more weight via per-step trajectory signal.
**Anchor test**: 2.10 (recovery_after_streak).
@@ -499,7 +598,7 @@ Produce trustworthy "this trained system is/isn't a trader" signal. Phase 4 is t
### 10.2 Sub 4.1 — Pre-flight (~½ day)
- All 21 Phase 2 tests green on dev (RTX 3050 Ti). Pre-commit hook blocks if not.
- All 22 Phase 2 tests green on dev (RTX 3050 Ti). Pre-commit hook blocks if not.
- Per-pearl audit doc updated, MEMORY.md index extended for any new pearls landed.
- New worktree `sp15-final-validation` for the deployment artifacts.
@@ -557,7 +656,7 @@ Deltas vs 8 baselines (Q9):
vs naive_revert : +X.X (need > 0)
Behavioral test status (final, on dev):
21/21 passing
22/22 passing
Pearls landed: <list>
Architectural changes: dd_pct as state, cost-net unified, 8 baselines, Q9 sealed
@@ -565,11 +664,18 @@ Architectural changes: dd_pct as state, cost-net unified, 8 baselines, Q9 sealed
### 10.6 Sub 4.5 — Production-track gate
**Pass criteria**:
1. Q9 `sharpe_net > 1.0`
2. Q9 `Calmar > 1.0`
3. Q9 beats ALL 8 baselines on net sharpe (delta > 0 for each)
4. All 21 Phase 2 tests still pass post-deployment
**Pass criteria** (thresholds are config-driven via `--gate-min-sharpe`, `--gate-min-calmar`, `--gate-baselines-must-beat`):
| Threshold | Default | Rationale |
|---|---|---|
| `min_sharpe_net` | 1.0 | Floor below which deploying is worse than passive holding (buyhold typical sharpe). To be tightened post-Phase-1 once we observe what baseline numbers actually look like on Q9. |
| `min_calmar` | 1.0 | Calmar = annualized_return / max_dd. 1.0 means recovering one max-drawdown's worth of capital per year — a survival-bar threshold. |
| `must_beat_all_baselines` | true | Every baseline delta > 0. Most rigorous; can be weakened to "beats N of 8" if the bar proves too tight in practice. |
| Phase 2 status | all 22 green | No regression in behavioral tests |
**Initial defaults are explicitly placeholders.** Phase 1 baseline runs will reveal actual baseline performance distribution. Phase 4.4 report card (first run) anchors realistic thresholds. If buyhold turns out to give sharpe_net = 0.5 on Q9, then "beats buyhold" is the floor and we tighten min_sharpe to e.g. 0.6. If buyhold gives 1.5, we tighten to 1.6+.
The thresholds are NOT hardcoded as architectural assumptions — they're config-driven AND empirically anchored.
**Fail mode → behavioral test mapping**:
@@ -592,7 +698,7 @@ NOT a generic re-train. Each failure axis has a specific behavioral test to targ
|---|---|---|---|
| 0 | ~400 | 1-2 | EGF retune kernels, 4 ISV slots, test 2.21 |
| 1 | ~700 | 2-3 | sharpe kernel, cost kernel, 8 baselines, dd_pct trunk concat, CLI flags |
| 2 | ~2000 | 3-5 | Synthetic markets, harness, 21 tests, pre-commit hook |
| 2 | ~2100 | 3-5 | Synthetic markets, harness, 22 tests, pre-commit hook |
| 3 | ~800 | 3-5 | 5 teachings (reward + Hold floor) |
| 3.5 | ~700 | 2-3 | 4 recovery mechanisms |
| 4 | ~300 | 2-3 | argo-eval-final.sh, report card automation |
@@ -618,11 +724,17 @@ Each baseline runs as a separate kernel against the val/test window. 8× the eva
- Run baselines only once per fold per epoch (not per step)
- Estimated overhead: ~5-10% of total eval cost
### 12.3 Q9 burn policy
### 12.3 Q9 burn policy is honor-system
"Each Q9 run generates audit file" relies on discipline, not enforcement. A motivated developer could run Q9 eval N times and only commit the best. Mitigation:
- Q9 run requires a tagged commit — `argo-eval-final.sh` rejects un-tagged or amended commits
- Audit log includes counter incrementing per architectural milestone — anomalous gaps surface gaming
**Explicit acknowledgment**: this is honor-system, not enforcement. No tooling makes Q9 contamination impossible — it requires developer honesty.
A motivated developer could run Q9 eval N times and only commit the best. Even with tag requirements, lightweight tags are mutable; annotated tags can be deleted and recreated. The proposed mitigations reduce convenience of cheating but don't prevent it:
- `argo-eval-final.sh` rejects un-tagged or amended commits (raises bar slightly)
- Each Q9 run writes a counter + commit-hash + result-summary to `trained-models-pvc` infrastructure (chain-hash). Retroactive deletion would leave detectable gaps in the chain.
- Per-fold audit doc commits go to main as part of every Q9 run; log of "I ran Q9" is git-history-visible.
These are deterrents, not enforcement. Accepting the discipline-only nature of this is honest spec hygiene. The alternative (cryptographic infrastructure for unfakeable Q9 evaluation logs) is over-engineering for a single-developer project.
### 12.4 Synthetic LOB fidelity for Phase 2
@@ -656,19 +768,22 @@ Mitigation:
---
## 13. Anticipated memory pearls
## 13. Pearl candidates pending validation
To be authored alongside the implementation phases:
**Important**: Section 14's discipline rule says pearls emerge from validation, not pre-planning. The list below is **candidates for pearl authoring IF AND ONLY IF the technique demonstrably generalises** beyond this spec. Some, all, or none may be authored. Pre-naming creates risk of authoring-pressure regardless of generalisation; the candidate framing acknowledges this.
| Pearl | Origin |
|---|---|
| `pearl_dd_pct_foundational_state` | Phase 1.5 — DD-context as eval-time state input vs reward-only modulation |
| `pearl_unified_cost_kernel_dev_prod_parity` | Phase 1.2 — same cost kernel reads spread/OFI from synthetic (dev) or real (prod) LOB |
| `pearl_behavioral_test_gates_l40s_deploy` | Phase 2 — pre-commit hook + dev-runtime test suite is the L40S filter |
| `pearl_recovery_curriculum_in_per` | Phase 3.5.5 — PER weighting can encode "recovery exists" signal explicitly |
| `pearl_plasticity_injection_for_rl_local_minima` | Phase 3.5.4 — weight-reset mechanism for breaking spiral attractors (if 2.10 verifies) |
| `pearl_8_counterfactual_baselines_arch_specific` | Phase 1.4 — domain baselines that test specific subsystems we built (aux-only, mag-Q-fixed, trail-only) |
| `pearl_egf_isv_driven_anchors` | Phase 0.B — Schmitt thresholds and variance refs as ISV slots, not constants |
| Candidate | Authoring trigger | Origin |
|---|---|---|
| `pearl_dd_pct_foundational_state` | If 2.11 (dd_state_aware_sizing) shows policy reliably conditions on dd_pct | Phase 1.5 |
| `pearl_unified_cost_kernel_dev_prod_parity` | If Phase 4 Q9 net metrics match dev tests within tolerance (cost calibration verified) | Phase 1.2 |
| `pearl_behavioral_test_gates_l40s_deploy` | If pre-commit hook catches at least one regression that would have shipped | Phase 2 |
| `pearl_recovery_curriculum_per_step_proxy` | If 2.10 passes via 3.5.5 alone (without plasticity 3.5.4 dominating) | Phase 3.5.5 |
| `pearl_plasticity_injection_for_rl_local_minima` | If 3.5.4 fires AND 2.10 measurably improves AND no 2.16 regression | Phase 3.5.4 |
| `pearl_8_counterfactual_baselines_arch_specific` | If at least 2 of the architecture-specific baselines (aux-only, mag-Q-fixed, trail-only) reveal a real subsystem-not-earning-keep finding | Phase 1.4 |
| `pearl_egf_isv_driven_anchors` | If Phase 0.B retune makes gate1 actually open under target conditions (test 2.21 green AND non-trivial fire rate on real workloads) | Phase 0.B |
| `pearl_plasticity_cooldown_interlock` | If 2.22 verifies the OR-gate prevents re-trigger loops in real training | Phase 3.5.4 |
Any candidate that doesn't trigger after Phase 4 — drop, don't author. The discipline rule cuts both ways: pearls only get written when the technique generalised.
---