docs(sp15): amend spec v2 — 10 fixes from second critical review

Second critical review pass found 10 more issues, 3 critical.
All addressed:

CRITICAL:
- §8.2 (3.1): ALPHA_SPLIT cold-start was unspecified — formula
   produces 0/0=0, not the claimed 0.5 sentinel. Now: ISV slot
   initialized DIRECTLY to 0.5 in trainer constructor; formula
   takes over only after both grad-norm EMAs accumulate N_warm
   non-zero observations
- §9.2 (3.5.4): plasticity now performs TWO-STEP recovery:
   (1) Flat all positions at fire bar (close current losing trade),
   (2) engage warmup cooldown forcing Hold for M_warm bars.
   Without step 1, forced Hold preserved the losing position
   that drove drawdown for the entire 200-bar warmup
- §12.2: stale "5-10%" baseline cost estimate updated to "15-25%"
   matching §6.4 (was contradicting earlier amendment)

IMPORTANT:
- §9.2 (3.5.5): DD_TRAJECTORY_DECREASING threshold 0.02 hardcoded
   → ISV-driven via new slot DD_TRAJECTORY_FLOOR (slot 441,
   25th percentile of running dd_pct distribution)
- §8.2 (3.5): HOLD_FLOOR_ALPHA tracked from rolling 95th percentile
   of |Q_dir| (NOT running max — was outlier-ratchet vulnerable)
- §9.2 (3.5.3): MEDIAN_STREAK_LENGTH formerly undefined in cooldown
   K formula → ISV-driven via new slot 442, running median of
   observed loss-streak lengths via two-heap algorithm
- §9.2 (3.5.2): asymmetric reward × α split compound interaction
   explicitly stated as intentional with POS_CAP as binding ceiling

NIT:
- §7.4: "2C: Group 3 (4 tests)" → "(5 tests)" (was off-by-one
   after 2.22 added)
- §9.2 (3.5.4): Xavier → Kaiming-He init for advantage head reset
   (architecturally appropriate for ReLU-gated activation chain)

ISV_TOTAL_DIM: 441 → 443 post-SP15 (added DD_TRAJECTORY_FLOOR
and MEDIAN_STREAK_LENGTH at slots [441..443)). 46 SP15 slots
total. File: 799 → 811 lines.

Spec is now consistent end-to-end with no contradictions between
sections, no hardcoded values violating feedback_isv_for_adaptive_
bounds, and no underspecified load-bearing parameters.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-06 08:58:28 +02:00
parent f3cb0c78c2
commit 5417e27567

View File

@@ -164,7 +164,8 @@ Approach B parallel dispatch requires that Phase 0 / Phase 1 / Phase 2A sub-work
| `[433..436)` | Phase 3.5.3 (cooldown) | `COOLDOWN_K_THRESHOLD` (ISV from running mean of per-trade PnL), `COOLDOWN_M_BARS`, `COOLDOWN_BARS_REMAINING` | 3 |
| `[436..439)` | Phase 3.5.4 (plasticity) | `PLASTICITY_FIRED_THIS_FOLD`, `PLASTICITY_PERSISTENCE_THRESHOLD`, `PLASTICITY_WARM_BARS_REMAINING` (force cooldown engagement post-fire) | 3 |
| `[439..441)` | Phase 3.5.5 (recovery curriculum) | `DD_TRAJECTORY_DECREASING` (per-step proxy for "in recovery"), `RECOVERY_OVERSAMPLE_WEIGHT` | 2 |
| **Total** | | **`ISV_TOTAL_DIM = 441` post-SP15** | 44 |
| `[441..443)` | Phase 3.5 deferred ISV anchors (resolves second-review #5, #7) | `DD_TRAJECTORY_FLOOR` (25th percentile of `dd_pct` distribution — replaces hardcoded 0.02), `MEDIAN_STREAK_LENGTH` (running median of consecutive-loss streak lengths — used by 3.5.3 cooldown K formula) | 2 |
| **Total** | | **`ISV_TOTAL_DIM = 443` post-SP15** | 46 |
Each phase's first commit lands `sp15_isv_slots.rs` extension covering its allocated range. Layout fingerprint seed extended atomically per phase. Sub-worktrees check the allocation map before assigning indices.
@@ -396,7 +397,7 @@ Every behavior the policy must exhibit gets a deterministic test on dev RTX 3050
Three atomic commit batches:
- **2A: scaffolding** (synthetic gens + oracle + harness) — first
- **2B: Groups 1+2** (17 tests) — once Phase 0+1 ships; gates Phase 3 entry
- **2C: Group 3** (4 tests) — written alongside Phase 3.5 teachings; each test commit paired with its teaching commit per the discipline rule
- **2C: Group 3** (5 tests) — written alongside Phase 3.5 teachings; each test commit paired with its teaching commit per the discipline rule
### 7.5 Discipline gate enforcement
@@ -424,8 +425,10 @@ r_total = α × r_quality_per_event + (1 α) × r_discipline_per_bar
- `r_quality_per_event` = SP12 per-trade event-driven reward (UNCHANGED)
- `r_discipline_per_bar` = 3.4 regret signal (NEW)
- **α is ISV-driven** at slot `ALPHA_SPLIT` (slot 417 per §4.3). Producer reads `GRAD_NORM_QUALITY` (slot 418, EMA of |∇r_quality|) and `GRAD_NORM_DISCIPLINE` (slot 419, EMA of |∇r_discipline|), computes `α = grad_norm_q / (grad_norm_q + grad_norm_d + ε)` so the two signals contribute equal gradient pressure regardless of fold-scale.
- Per `feedback_isv_for_adaptive_bounds`α tracks the actual gradient distribution rather than being a hardcoded 0.7 guess. (Initial sentinel α = 0.5 until Welford warm-up completes; first-observation bootstrap per `pearl_first_observation_bootstrap`.)
- **α is ISV-driven** at slot `ALPHA_SPLIT` (slot 417 per §4.3). Producer reads `GRAD_NORM_QUALITY` (slot 418, EMA of |∇r_quality|) and `GRAD_NORM_DISCIPLINE` (slot 419, EMA of |∇r_discipline|).
- **Cold-start init (resolves second-review #2)**: `ALPHA_SPLIT` slot is initialized **directly to 0.5 in the trainer constructor** (NOT derived from formula at boot). The formula `α = grad_norm_q / (grad_norm_q + grad_norm_d + ε)` only takes over once BOTH grad-norm slots have accumulated `≥ N_warm` non-zero observations (Welford warm-up; `N_warm = 100` initial sentinel, ISV-tracked from per-fold convergence pattern). Until then, the slot remains at the constructor-written 0.5.
- The `pearl_first_observation_bootstrap` semantics (sentinel = 0; first observation replaces directly) apply to the grad-norm EMAs themselves — first observation of `|∇r_quality|` becomes `GRAD_NORM_QUALITY[0]`, etc. But the dependent slot `ALPHA_SPLIT` is NOT computed from these until both EMAs are warmed up.
- Per `feedback_isv_for_adaptive_bounds` — once warm, α tracks the actual gradient distribution rather than being a hardcoded 0.7 guess.
**Anchor tests**: 2.4 (cost_sensitivity), 2.6 (regime_silences). Both should be made simultaneously achievable by the split.
@@ -476,7 +479,7 @@ hold_floor = α × σ(k × (entropy ε₀))
Where:
- `entropy` = Shannon entropy of dir_Q_distribution (range `[0, log(K_dir)]`, K_dir=4 → max ≈ 1.386)
- `α` = scale factor in `HOLD_FLOOR_ALPHA` (slot 426, ISV-tracked from running max of |Q_dir| so the bias is comparable to Q magnitudes)
- `α` = scale factor in `HOLD_FLOOR_ALPHA` (slot 426, ISV-tracked from rolling 95th percentile of |Q_dir| over fold — NOT running max, which is one-way ratchet sensitive to outliers. 95th-percentile is robust to early-training noise spikes while still tracking actual Q magnitude scale)
- `k` = sigmoid steepness in `HOLD_FLOOR_K` (slot 427, ISV-tracked from running variance of entropy — wider entropy distribution → gentler slope)
- `ε₀` = entropy threshold in `HOLD_FLOOR_EPS0` (slot 428, ISV-tracked at 75th percentile of running entropy distribution `ENTROPY_DIST_REF` slot 429 — biases Hold only when policy is in upper quartile of uncertainty)
- `σ` = standard logistic sigmoid
@@ -526,6 +529,7 @@ r_quality_post_cap = clamp(intermediate, NEG_CAP, POS_CAP) [SP12's cap stays fi
- For `dd_pct = 0` (at ATH): `(1 + 0) = 1` → no-op, SP12 cap unchanged.
- For `dd_pct > 0` AND `r_quality_pre_cap × (1 + λ × dd_pct) > POS_CAP`: the multiplier saturates the cap. Behaviorally correct — recovery trades hit ceiling sooner, encouraging frequent small recoveries over rare-large.
- SP12 NEG_CAP (loss side) still applies asymmetrically; this teaching doesn't touch loss multiplier.
- **Compound interaction with α split (resolves second-review #8)**: `r_quality_post_cap` then feeds `r_total = α × r_quality_post_cap + (1 α) × r_discipline`. The compound multiplier × α weight of recovery-trade r_quality is **intentional and bounded by POS_CAP**. The POS_CAP IS the binding ceiling — implementers should NOT "fix" the apparent double-amplification by removing α-weighting or splitting the cap. The semantics: amplifier in DD → cap in DD → α-weighted contribution to total. Cap is the safety; α is the balance signal.
`λ` is ISV-tracked at `DD_ASYMMETRY_LAMBDA` (slot 430). Producer reads `DD_DIST_VAR` (slot 432, running variance of dd_pct over fold) — wider DD distribution → smaller λ (less aggressive multiplier when DD is volatile and unstable). Initial λ ≈ 0.5 sentinel.
@@ -536,7 +540,8 @@ Teaches "a winning trade out of DD is worth more than the same trade at ATH." Ma
#### 3.5.3 — Cooldown gate
After K consecutive losing trades, force Hold for M bars. Both K and M are ISV-driven:
- **K (loss-streak threshold)**: at slot `COOLDOWN_K_THRESHOLD` (slot 433). Producer reads running mean of per-trade PnL (NOT variance — variance is LOW during a streak when all losses cluster, which would delay cooldown precisely when needed). When running mean drops below a fold-percentile threshold (e.g., 25th percentile), K is reduced — cooldown trips sooner. Concretely: `K = max(2, floor(median_streak_length × clamp(mean_pnl_recent / mean_pnl_baseline, 0.5, 2.0)))`. This makes K small (cooldown sensitive) when recent trades are below baseline.
- **K (loss-streak threshold)**: at slot `COOLDOWN_K_THRESHOLD` (slot 433). Producer reads running mean of per-trade PnL (NOT variance — variance is LOW during a streak when all losses cluster, which would delay cooldown precisely when needed). When running mean drops below a fold-percentile threshold (e.g., 25th percentile), K is reduced — cooldown trips sooner. Concretely: `K = max(2, floor(MEDIAN_STREAK_LENGTH × clamp(mean_pnl_recent / mean_pnl_baseline, 0.5, 2.0)))`.
- `MEDIAN_STREAK_LENGTH` (slot 442) is **ISV-driven** — running median of observed consecutive-loss streak lengths within fold. Producer kernel maintains a streak counter (incremented on losing trade, reset on winning trade), updates median via two-heap algorithm or running quantile sketch on each trade-close event. Per `feedback_isv_for_adaptive_bounds`: K threshold derives from observed streak distribution, not a guessed constant.
- **M (cooldown duration)**: at slot `COOLDOWN_M_BARS` (slot 434). Tracked from time-to-mean-reversion of vol_normalizer signal — cooldown lasts until regime stabilises.
**Cooldown counter as state**: ISV slot `COOLDOWN_BARS_REMAINING` (slot 435). Producer decrements per bar; consumer (action selection) forces Hold while > 0. Model SEES the counter on every forward — can reason about it, not work around it.
@@ -545,12 +550,17 @@ After K consecutive losing trades, force Hold for M bars. Both K and M are ISV-d
#### 3.5.4 — Plasticity injection during persistent drawdown
When `DD_PERSISTENCE` (slot 404 per §4.3) exceeds `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437), reset last 10% of advantage-head weights to Xavier-initialised values to escape local minima.
When `DD_PERSISTENCE` (slot 404 per §4.3) exceeds `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437), reset last 10% of advantage-head weights (initialization scheme specified below) to escape local minima.
- **Debounced**: fires at most once per fold via `PLASTICITY_FIRED_THIS_FOLD` (slot 436).
- **Threshold ISV-driven**: `PLASTICITY_PERSISTENCE_THRESHOLD` (slot 437) tracks running mean of `dd_persistence_bars` over training history. Set to mean + 2σ so only persistent (statistically unusual) drawdowns trigger.
- **Reset target**: last 10% of advantage-head weights (NOT shared trunk, NOT direction Q-head fully). Affects only the magnitude/ordinal/urgency advantage tails where local minima are most likely.
- **Mandatory cooldown interlock (resolves Critical #3)**: when plasticity fires, the kernel ALSO writes `M_warm` to `PLASTICITY_WARM_BARS_REMAINING` (slot 438). This slot routes into the 3.5.3 cooldown consumer via OR-gate: `effective_cooldown = max(COOLDOWN_BARS_REMAINING, PLASTICITY_WARM_BARS_REMAINING)`. While `PLASTICITY_WARM_BARS_REMAINING > 0`, action selection forces Hold (same path as 3.5.3 cooldown). Prevents the random-tail-output → loss-streak → re-trigger loop. M_warm = 200 bars (initial sentinel; ISV-tracked post-fold from observed Adam re-equilibration time).
- **Reset initialization**: **Kaiming-He** (`mode=fan_in`, `gain` matched to activation). The trunk uses GRN blocks with GLU activations and the advantage heads chain through ReLU; Kaiming-He is calibrated for half-rectified activations and is the architecturally-appropriate default. Xavier (sometimes proposed) assumes symmetric linear activations and would underestimate variance for the actual activation chain. (Resolves second-review #10.)
- **Mandatory cooldown interlock (resolves Critical #3)**: when plasticity fires, the kernel performs a **two-step recovery sequence** (resolves second-review #3):
1. **Flat all positions** at the fire bar (action = Flat, force-overrides policy output for that bar). The position that drove persistent drawdown is closed; the policy is no longer holding a known-bad position while its advantage tail is randomised.
2. **Engage warmup cooldown** by writing `M_warm` to `PLASTICITY_WARM_BARS_REMAINING` (slot 438). This routes into the 3.5.3 cooldown consumer via OR-gate: `effective_cooldown = max(COOLDOWN_BARS_REMAINING, PLASTICITY_WARM_BARS_REMAINING)`. While `PLASTICITY_WARM_BARS_REMAINING > 0`, action selection forces Hold (now safe — no position to bleed since step 1 closed it).
- M_warm = 200 bars (initial sentinel; ISV-tracked post-fold from observed Adam re-equilibration time).
- Prevents the random-tail-output → loss-streak → re-trigger loop AND the open-losing-position-during-warmup leak. Symmetric: plasticity = "step away & restart" requires both closing current trade AND not re-entering during warmup, matching real-trader behavior.
Mimics real-trader "step away & restart" — the most aggressive recovery mechanism. Most research-y; gated strictly on 2.10 staying green AND 2.22 (NEW interlock test) green.
@@ -566,7 +576,8 @@ Mimics real-trader "step away & restart" — the most aggressive recovery mechan
PER stores transitions `(s_t, a_t, r_t, s_{t+1})` with TD-error priority. Adding episode-level "started in DD, recovered to flat" tags requires structural PER changes (episode boundaries, episode-level indices) which are out of scope.
**Per-transition proxy**:
- New per-step ISV signal `DD_TRAJECTORY_DECREASING` (slot 439): boolean (or smoothed scalar in `[0, 1]`) indicating `dd_pct(t) < dd_pct(t-1) AND dd_pct(t-1) > 0.02`. True when the transition is part of a recovery (DD shrinking from a non-trivial drawdown).
- New per-step ISV signal `DD_TRAJECTORY_DECREASING` (slot 439): boolean (or smoothed scalar in `[0, 1]`) indicating `dd_pct(t) < dd_pct(t-1) AND dd_pct(t-1) > DD_TRAJECTORY_FLOOR`. True when the transition is part of a recovery (DD shrinking from a non-trivial drawdown).
- `DD_TRAJECTORY_FLOOR` (slot 441) is **ISV-driven** — tracks 25th percentile of running `dd_pct` distribution. Per `feedback_isv_for_adaptive_bounds`: "non-trivial drawdown" scales with the fold's own DD regime, not a guessed `0.02` constant.
- This signal is COMPUTED PER-BAR by the existing dd_pct producer (sub 1.5) — no PER restructure, no episode tracking, no replay buffer schema change.
- When PER samples a batch, weight transitions by `1.0 + RECOVERY_OVERSAMPLE_WEIGHT × DD_TRAJECTORY_DECREASING_t` (slot 440). Transitions on a recovery path get amplified sampling weight.
- `RECOVERY_OVERSAMPLE_WEIGHT` is ISV-driven from current `dd_pct`: more DD now → higher weight (current trajectory needs more recovery examples).
@@ -719,10 +730,11 @@ No published results in trading RL for "reset weights mid-training to escape loc
### 12.2 8 baselines compute cost
Each baseline runs as a separate kernel against the val/test window. 8× the eval cost per fold per epoch. Mitigation:
- Baselines are pure forward (no backprop), much cheaper than training step
Each baseline runs against the val/test window. 8 baselines × N_bars per fold per epoch. Mitigation per §6.4:
- Baselines that need policy-trained heads (3 of 8: aux_only, mag_quarter_fixed, random_dir_kelly) share the trunk forward pass with main policy eval — no redundant trunk computation
- 5 of 8 are constant or last-bar-only (buyhold, hold_only, naive_momentum, trail_only, naive_reversion) — pure kernels with negligible cost
- Run baselines only once per fold per epoch (not per step)
- Estimated overhead: ~5-10% of total eval cost
- **Realistic overhead estimate: ~15-25% of total eval cost** (per §6.4 — revised from earlier 5-10% which underestimated trunk-fusion-required baselines)
### 12.3 Q9 burn policy is honor-system