diff --git a/docs/superpowers/specs/2026-05-24-sp20-trader-grade-trade-management-design.md b/docs/superpowers/specs/2026-05-24-sp20-trader-grade-trade-management-design.md new file mode 100644 index 000000000..55029deed --- /dev/null +++ b/docs/superpowers/specs/2026-05-24-sp20-trader-grade-trade-management-design.md @@ -0,0 +1,1223 @@ +# SP20 — Trader-Grade Trade Management: Foundational Production System + +> **Status:** Design spec v3 (post-critical-review). Post-7d7mp (commit +> `e87e0b077`) chain validation. The integrated RL trainer has reached +> its empirical ceiling under stateless step-wise decisions: WR=42.28%, +> payoff=1.21, late-half R/done=-$0.073, last-20k window crossed +$0.005. +> Break-even at current payoff requires WR=45.3%; the user goal is +> WR>55%. This SP transforms the agent from a stateless step picker +> into a **complete trader-grade trade-management system**. +> +> **Discipline:** Atomic intervention per +> `pearl_no_deferrals_for_complementary_fixes`. Greenfield mandate: +> every mechanic composes; no deferrals; production-ready foundation. +> **Hard requirement: every new mechanic exposes its full diagnostic +> surface in the JSONL.** No silent state. No opaque controllers. +> Ships as one PR / one cluster smoke / one post-mortem. +> +> **Foundational mandate (v3 revision):** No cheap workarounds. Where +> design has a choice between "easier but suboptimal" and "harder but +> correct," this spec picks the latter. Every design decision in §4 +> documents the rejected cheap alternative. + +## 0 — Foundational principles (non-negotiable, gate every phase) + +Three production-discipline mandates derived empirically from the pattern +of recent debugging. Each is a HARD gate per phase, not just at SP20 +ship time. Violations block phase merge. + +### 0.1 EVERY NUMERICAL CONSTANT IS ISV-RESIDENT + +Half of recent debugging cycles were spent re-tuning constants that +needed to be runtime-tunable. Hardcoded `#define`s in new kernels are +forbidden under this SP, with one tightly-scoped exception below. + +**Rule:** every magic number used by a kernel goes into an ISV slot, +seeded at trainer init via `rl_isv_write`. Re-tuning becomes a +re-seed, never a recompile. + +**Extension of user-stated "floors and clamp bounds" exemption:** +that exemption is OVERRIDDEN for this SP. Even floors and clamp +bounds are ISV-resident (seeded with their original values, but +tunable at runtime). Examples in this SP: `P_MIN` floor for π +sampling, `MIN_TRAIL` floor, `MAX_TRAIL` ceiling, `MIN_RATIO`, +`MAX_RATIO`, anti-mart bounds — ALL ISV slots. + +**Sole exception:** structural compile-time constants that determine +buffer dimensions (N_ACTIONS=11 dim, Q_N_ATOMS=21 dim, MAX_UNITS=4 +dim) remain CUDA `#define` because they control memory layout and +allocation. Their _bounds_ (e.g., MAX_UNITS as the unit-cap) are +still ISV-resident (slot 496 RL_PYRAMID_MAX_UNITS_INDEX) even +though the dimension is fixed. + +**Audit:** every new .cu file is grep'd for `#define` numeric +constants. Only structural-dim constants are permitted; everything +else must read from `isv` parameter. Phase ship gate: 0 violations. + +### 0.2 EVERY NEW KERNEL/SLOT/HEAD IS FULLY WIRED IN SAME COMMIT + +The other half of recent debugging discovered kernels that existed +but were never launched, slots that were written but never read, +actions that were defined but had no consumer (TrailTighten/Loosen +per [[pearl-dead-trail-stop-actions-a7-a8]]). + +**Rule:** every named thing has a producer AND a consumer in the +SAME commit as its definition. + +* Every kernel in `build.rs` KERNELS array → launch site in trainer +* Every ISV slot in `isv_slots.rs` → producer (`rl_isv_write` seed + OR controller kernel write) AND consumer (kernel read) +* Every `Action` enum entry → handler in `actions_to_market_targets` + OR an override-stack kernel +* Every new head module → forward kernel + backward kernel + loss + accumulation + Adam step + LR controller hookup + diag emission +* Every new per-batch buffer → allocator at trainer init + read + site(s) + reset/clear path for state-machine transitions + +**Audit:** phase ship gate runs a wiring verification: +``` +scripts/audit-wiring.sh + - For every entry in build.rs KERNELS: grep for `.launch(` site + - For every RL_*_INDEX in isv_slots.rs: grep for write site + read site + - For every Action enum value: grep for handler in actions_to_market_targets or override kernel + - For every "head" or "fn forward_" in src/heads/: grep for backward + loss + Adam launch +``` +Phase ship gate: 0 violations. + +### 0.3 DIAGNOSTICS BAKED IN AT BIRTH + +Diagnostic exposure is not "follow-up." Every observable state is in +the diag JSONL on the same commit that adds the state. + +**Rule:** every new mechanic emits its observable state to JSONL in +the SAME commit: +* Every new ISV slot → field in `isv_config` block +* Every new per-batch buffer → field in a dedicated diag block + (per-batch array OR aggregate stats; choose by buffer size — small + buffers go per-batch, large ones go p50/p99/min/max aggregated) +* Every override kernel → per-step fire-count + cumulative total +* Every head → per-step loss + LR + output statistics + (min/max/mean over the b_size dim) +* Every controller → input slot + output slot + internal EMA state + +**Audit:** phase ship gate runs a 100-step local smoke and parses +the diag JSONL to confirm every new field is present and non-trivial: +``` +scripts/audit-diag.sh + - Run local 100-step smoke (no cluster needed) + - Parse first 10 + middle 10 + last 10 rows of diag JSONL + - For each new field in spec: verify present + non-NaN + - For per-batch buffers: verify variance > 0 (i.e., not stuck) + - For counters: verify monotonically non-decreasing +``` +Phase ship gate: 0 missing fields. + +### 0.4 Per-phase ship-gate checklist (applied EVERY phase) + +Before any phase merges, all three audits MUST pass: + +- [ ] **ISV audit (§0.1):** 0 hardcoded numerical constants in new .cu +- [ ] **Wiring audit (§0.2):** 0 orphan kernels / orphan slots / dead actions / unwired heads +- [ ] **Diag audit (§0.3):** every new observable field present + non-trivial in local 100-step smoke + +These are STOP-the-line gates. If any audit fails, the phase doesn't +ship; the violation gets a follow-up commit before SP20 progresses. + +The audits replace the v3 "iterate within scope" softness — failures +are immediately actionable, not deferred. + +## 1 — Why this SP exists + +### 1.1 The empirical ceiling we hit + +Across runs rmgm5 → rdgzl → 8xwq8 → vj5f6 → wwcsz → rljzl → 7d7mp, +each architectural fix moved the needle: + +| Smoke | Key change | R/done (late half) | WR | +|---|---|---|---| +| rmgm5 | b_size=16 baseline | -$0.998 | 36% | +| rdgzl | adaptive WIN clamp | -$0.481 | 35% | +| 8xwq8 | + MARGIN adaptive | -$0.204 | 27% | +| vj5f6 | + C51 V_MAX ratchet | -$0.204 | 27% | +| wwcsz | + Q→π distill | -$0.282 | 39% | +| rljzl | + EWMA atom + adaptive RATIO + 200k | -$0.118 | 42% | +| **7d7mp** | + adaptive λ + reward_scale MIN | **-$0.073** | **42%** | + +### 1.2 Ceiling falsification first (Phase P-1, MANDATORY pre-SP20) + +Critical-review challenge: have we actually hit the ceiling, or just +the ceiling of single-seed/single-fold/200k-step training? + +**Phase P-1 is the falsification step that MUST precede SP20 implementation:** + +Run current 7d7mp architecture (commit `e87e0b077`) at: +* 1M steps (5× current duration) +* 5-fold walk-forward +* Single seed (seed=16962) AND second seed (different value, e.g. 42) +* Total cost: ~10 cluster runs, ~$50-150 + +**Acceptance for P-1 to clear SP20 to start:** +* If late-half WR < 45% across both seeds × 5-fold mean: SP20 IS justified +* If late-half WR ≥ 45%: SP20 is premature; ship architecture-stable run + to production, defer SP20 to next horizon + +Without P-1, SP20 risks 3-4 weeks of work on a problem already solvable +by more training. P-1 takes ~1 day. **Hard gate: P-1 must complete and +fail-its-ceiling before SP20 implementation begins.** + +### 1.3 The diagnosis: trade-management is the missing layer + +Behavioral analysis on 7d7mp: +* Action distribution late: ShortL=11%, ShortS=8%, Hold=9%, + FlatLn=12%, FlatSh=13%, LongS=14%, LongL=13%, TrailT=9%, TrailL=10% +* No-ops (Hold + TrailT + TrailL) = 28% of decisions +* Trail actions (a7/a8) verified DEAD per + [[pearl-dead-trail-stop-actions-a7-a8]] +* Per-action dominance proxy (wwcsz): ShortS profitable (+$0.78/close), + ShortL+LongS+LongL all losing — the market has asymmetric + opportunity the agent isn't sizing into + +Literature converges: **the gap between mediocre and elite traders is +in trade management, not in entry signal quality.** Both groups can +identify setups. Elites make more from winners and less from losers via: + +1. Vol-adjusted stops (cut losers fast, calibrated) +2. Trailing stops (lock gains as price moves) +3. Pyramiding (add to winners, never losers) +4. Partial profit-taking (scale OUT as you scale IN) +5. Forward-looking entry-quality forecasting (only A+ setups) +6. Anti-martingale sizing (bigger after wins, smaller after losses) +7. Position heat caps (survival) + +Our architecture has perfect entry plumbing but **zero of the seven +trade-management mechanics above.** SP20 wires them in with full +observability. + +### 1.4 Research synthesis (omnisearch + memory) + +#### 1.4.1 Trail-stop framework + +``` +Stop_t = Anchor_t ± k × σ_t +Long: Stop_t = max(Stop_{t-1}, P_t - k × σ_t) +Short: Stop_t = min(Stop_{t-1}, P_t + k × σ_t) +``` + +Per `pearl_trade_level_vol_for_stop_distance`: σ_t = `sqrt(realized_return_var)`. + +#### 1.4.2 Turtle pyramiding + +* Add 1 unit every 0.5N favorable move (N = ATR; our σ equivalent) +* Max 4 units per direction +* **Each unit gets its own 2N stop from its own entry** — critical + detail that SP20 v3 honors (was incorrect in v2) + +#### 1.4.3 Partial profit-taking (Van Tharp) + +* Take 1/3 off at +1R, 1/3 at +2R, ride 1/3 with trail +* R = initial risk + +#### 1.4.4 Forward-looking forecaster head (replaces survivor-biased checklist) + +Critical-review CRIT-1: a self-supervised label `is_next_trade_profitable` +suffers survivor bias — only trades the agent took get labels, so the +head learns tautological "trades I took were the right trades." + +**Foundational fix:** train a **forward-return-distribution head** with +a supervised target derived purely from market data (NOT from agent +behavior). Predict the distribution of forward returns at multiple +horizons: +* h₁ = 60 ticks (~short-term) +* h₂ = 300 ticks (~medium-term) +* h₃ = 1800 ticks (~long-term) + +For each horizon, the head outputs a categorical distribution over +return-bucket atoms (matches C51 pattern). Trained with cross-entropy +against the empirical forward-return distribution computed from +snapshot price changes. + +* No survivor bias (every snapshot has a forward-return label) +* No tautology (label is market-truth, not agent-truth) +* Useful beyond gating: gives the policy a calibrated future-return + forecast that any downstream head can consume +* Gating use: `entry_quality = P(forward_return > 0.5σ at h₂)` + exceeds threshold + +#### 1.4.5 Kelly / confidence-weighted sizing + +C51 distributional Q gives natural μ + σ per action; confidence gate +formula `conf = clamp(0, (μ - λ × σ) / σ_norm, 1)`. + +#### 1.4.6 Anti-martingale (per-batch) + +Critical-review CRIT-4: single global outcome EMA mixes 16 batches' +trade history, loses per-trade-arc fidelity. + +**Foundational fix:** per-batch outcome EMA buffer +`anti_mart_outcome_ema_d[B]: f32` — each batch tracks its own recent +done-event reward sign EMA. Sizing modulation is per-batch. + +#### 1.4.7 Multi-resolution market features (TIME-based, not event-count) + +Critical-review SIG-4: "1s/10s/600s in tick-event units" varies wildly +with market activity. + +**Foundational fix:** real-time scales using snapshot `ts_ns`. Define +horizons as wallclock seconds. Sliding-window aggregates over actual +time intervals. Requires snapshot timestamp field (`Mbp10RawInput.ts_ns` +or equivalent — verify existence; if missing, add to snapshot pipeline). + +#### 1.4.8 RL trade-management state design + +Per Coursera + arXiv 2411.07585: management state must include market +context + position context + trade lifecycle. Components 2+3 missing +from current state — Tier 1 + Tier 0 features fill the gap. + +## 2 — Architectural design (the complete trader system) + +### 2.1 The composite trader archetype + +This SP encodes a **patient sniper that rides trends, sizes into +winners with per-unit risk discipline, and scales out with conviction.** +Decision flow: + +1. **Read the wave at multiple time-scaled horizons** (multi-res features) +2. **Forecast forward-return distribution** (FRD head) +3. **Wait for setup** (FRD head + confidence gate) +4. **Enter with vol-scaled + anti-mart-modulated initial size** +5. **Each unit gets its own trail** (per-unit entry + per-unit trail_distance) +6. **Trail aggressively as winner unfolds** (a7 tightens) +7. **Loosen if mid-trade noise** (a8 widens) +8. **Pyramid on continuation** (LongLarge with pos>0 + favorable ≥ 0.5σ → ADD unit) +9. **Take partial profit at +1R, +2R** (HalfFlat actions close 1 unit at a time) +10. **Force-close most-at-risk unit on adverse trail breach** (per-unit stop) +11. **Cap total heat** (position_heat_check) +12. **Recover after losses, capitalize after wins** (per-batch anti-martingale) + +### 2.2 Five-tier integration (all required) + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ TIER 0 — MULTI-RESOLUTION TIME-SCALED MARKET READING │ +│ │ +│ 3 horizons × 4 features each → 12 multi-res dims per batch: │ +│ 1s wallclock: price_change, vol, oflow_imb, trade_burst │ +│ 10s wallclock: same 4 features │ +│ 600s wallclock: same 4 features │ +│ │ +│ Computed from real timestamps; sliding window per horizon. │ +│ Injected at ENCODER INPUT (per snapshot, replicated K) so │ +│ encoder's recurrent dynamics integrate them — NOT side-channel │ +│ per critical-review SIG-2. Foundational, not cheap. │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ TIER 1 — TRADE-ARC AWARENESS (EYES) │ +│ │ +│ 4 trade-context features per batch: │ +│ time_in_trade_norm │ +│ unrealized_R │ +│ pos_magnitude_norm │ +│ entry_distance_σ │ +│ │ +│ Also injected at ENCODER INPUT (replicated across K-window │ +│ since per-batch state is constant within forward window). │ +│ Encoder learns trade-phase-conditioned market representations. │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ TIER 2 — DEFENSIVE HANDS (per-unit trail + heat) │ +│ │ +│ Per-batch per-unit buffers (MAX_UNITS=4): │ +│ unit_entry_price_d[B × 4] │ +│ unit_entry_step_d[B × 4] │ +│ unit_lots_d[B × 4] │ +│ unit_trail_distance_d[B × 4] │ +│ unit_active_d[B × 4] (1 if unit holds lots, 0 if closed) │ +│ │ +│ rl_trail_mutate.cu — a7/a8 mutate ALL active units' │ +│ trail_distance (uniform tighten/ │ +│ loosen across units) │ +│ rl_trail_stop_check.cu — for each active unit: check if │ +│ price violates unit-specific stop. │ +│ If yes → mark unit for close via │ +│ partial-flat sized to that unit. │ +│ Most-at-risk unit closes first. │ +│ rl_position_heat_check — if total |pos| > heat_max → force │ +│ full flat │ +│ │ +│ Per-unit tracking is FOUNDATIONAL fix per critical-review │ +│ CRIT-2. Each unit has own entry, own trail, own stop. │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ TIER 3 — OFFENSIVE HANDS (per-unit pyramid + partial flat) │ +│ │ +│ 3a. Pyramiding (scale IN, per-unit): │ +│ LongLarge/ShortLarge on existing same-direction pos with │ +│ favorable move ≥ 0.5σ FROM LAST UNIT'S ENTRY → ADD new │ +│ unit (allocate next free unit slot). Each new unit gets │ +│ its OWN entry_price, entry_step, lot count, trail_distance.│ +│ Pyramid threshold uses LAST UNIT'S entry, not aggregate │ +│ (per Turtle: each add waits for fresh favorable confirm). │ +│ Cap at MAX_UNITS=4. │ +│ │ +│ 3b. Partial profit-taking (scale OUT, per-unit): │ +│ New actions a9=HalfFlatLong, a10=HalfFlatShort. │ +│ Closes ONE active unit (the OLDEST = first-in / most │ +│ profit per unit-of-time). Reduces pos by that unit's lots. │ +│ N_ACTIONS=9 → 11 (full refactor across all kernels). │ +│ Pyramid_units_count decrements; unit_active_d[oldest]=0. │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ TIER 4 — ENTRY DISCIPLINE & RISK CONTROL │ +│ │ +│ 4a. Forward-Return-Distribution (FRD) head — NEW head: │ +│ Outputs categorical distribution over forward-return │ +│ buckets at 3 horizons (60, 300, 1800 ticks). │ +│ Trained CE-loss against empirical forward returns │ +│ (purely supervised, NO survivor bias). │ +│ │ +│ 4b. FRD gate (override): │ +│ entry_quality_h2 = P(forward_return_h2 > 0.5σ). │ +│ If pos == 0 AND open-action AND quality < THR_FRD: │ +│ OVERRIDE → Hold. │ +│ │ +│ 4c. Confidence gate (Q-variance, complements FRD): │ +│ conf = clamp(0, (μ_Q - λ × σ_Q) / σ_norm, 1). │ +│ If pos == 0 AND open AND conf < THR_CONF: → Hold. │ +│ │ +│ 4d. Anti-martingale sizing (PER-BATCH): │ +│ per-batch outcome_ema_d[B]: EMA(reward sign on done). │ +│ size_eff = base × clamp(MIN, 1 + κ × outcome_ema[b], MAX) │ +│ Bigger after batch's own wins, smaller after batch's │ +│ own losses. │ +│ │ +│ 4e. Vol-adjusted defaults: │ +│ trail_init = k × sqrt(realized_return_var_ema) │ +│ pyramid_thr = 0.5 × sqrt(realized_return_var_ema) │ +│ heat_max_lots = MAX_UNITS × unit_size │ +│ R per unit = trail_init (initial stop distance) │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### 2.3 The override stack (correct ordering per critical-review CRIT-3) + +``` +1. rl_pi_action_kernel sample action from softmax(π) + P_MIN floor +2. rl_frd_gate pos==0 ∧ open ∧ entry_qualityheat_max → full flat +7. actions_to_market_targets pyramid semantics + partial-flat unit pick + + anti-mart per-batch sizing + execute +``` + +**Critical-review CRIT-3 fix:** trail-mutate (step 4) now runs BEFORE +trail-stop-check (step 5). When the policy chooses a7/a8, the mutation +takes effect, then stop check sees the new trail distance. No more +lost mutations. + +### 2.4 Why no deferrals — full coupling proof + +| Element shipped alone | What goes wrong | +|---|---| +| Multi-res only | Better signal, stateless decisions unchanged | +| Trade-arc only | Encoder sees phase, no new actions to exploit | +| Trail only | Mechanics work, π lacks state to use intelligently | +| Pyramid only | Scales size but no per-unit stops → trail useless | +| Partial-flat only | Bank profit but no pyramid to build it up | +| FRD only | Forecasts but no action filtering uses it | +| Confidence only | Same as FRD alone | +| Anti-mart only | Cross-trade modulation, intra-trade still binary | +| Heat cap only | Prevents catastrophe, no offensive capability | +| Vol-defaults only | Better calibration, no new actions/state | + +All compound with ≥2 others. Canonical `pearl_no_deferrals_for_complementary_fixes`. + +## 3 — Implementation phases (atomic single-PR after P-1 clears) + +### Phase P-1 — CEILING FALSIFICATION (intentionally skipped 2026-05-24) + +Per §1.2 design: run current 7d7mp architecture at 1M × 2 seeds × +5-fold to validate whether more training alone hits WR ≥ 45%. + +**Decision (2026-05-24):** P-1 SKIPPED per user direction — "we need +the SP first to see where we can go." SP20 is the architectural +launchpad for the broader trader system regardless of whether the +current arch could be pushed further with raw training. Even if P-1 +showed current arch hits 45%, the user's ultimate target is 55%+ +and the trade-management mechanics are foundational for the full +product. SP20 is therefore the next architectural cycle by choice, +not by necessity. + +Additionally, P-1 has technical caveats discovered during planning: +* The CLI binary's "eval phase" is NOT pure inference — Adam+PER + continue during eval (verified in alpha_rl_train.rs:885-890). So + "OOS WR after eval" actually means "WR after 50k more training on + unseen data" — not a clean OOS measurement. +* The 5-fold path in the loader has never been exercised; latent + bugs in fold-slicing could waste compute. +* Adding pure-inference eval mode is itself a ~100 LOC change that + would compete with SP20 for time. + +P-1 may be revisited as standalone validation work AFTER SP20 ships, +at which point pure-inference eval mode becomes its own focused SP. + +### Phase P0 — Multi-resolution time-scaled features + +**Critical-review SIG-4 fix:** real-time wallclock scales. + +* Verify `Mbp10RawInput.ts_ns` exists (or equivalent timestamp field + in snapshot struct). If missing: add it to snapshot pipeline as + prerequisite. **No event-count proxies.** +* Per-batch sidecar buffer `multires_features_d[B × 12]` (3 horizons + × 4 features). +* New kernel `rl_multires_features_update.cu`: + * For each (batch, horizon), maintain a circular buffer of + recent (ts, mid_price, bid_size, ask_size, trade_count) tuples + * On each new snapshot: append, evict entries older than horizon + * Compute window aggregates: + * `price_change = (current_mid - mid_at_window_start) / σ` + * `vol_realized = sqrt(Σ(r²) over window)` + * `oflow_imb = mean(bid_size - ask_size) / mean(total_size)` over window + * `trade_burst = trade_count_in_window / horizon_seconds` + * Storage cost: 3 horizons × ~10000 max entries (for 600s × event rate) × + 20 bytes ≈ 600KB per batch → 10MB for B=16. Acceptable on L40S. +* Horizons in seconds: ISV slots 508 (=1.0), 509 (=10.0), 510 (=600.0) + +### Phase P1 — Per-unit trade-state tracking + +**Critical-review CRIT-2 fix:** per-unit entry + trail. + +* New per-batch per-unit buffers (B × MAX_UNITS=4): + * `unit_entry_price_d[B × 4]: f32` + * `unit_entry_step_d[B × 4]: i32` + * `unit_lots_d[B × 4]: i32` (signed: +lots for long, -lots for short) + * `unit_initial_r_d[B × 4]: f32` (R = initial stop distance at unit's open) + * `unit_trail_distance_d[B × 4]: f32` + * `unit_active_d[B × 4]: u8` (1 = unit has lots, 0 = closed/empty) +* Per-batch aggregates (derived per step): + * `pyramid_units_count_d[B]: i32` = sum of unit_active per batch + * `trade_context_d[B × 4]: f32` (the 4 Tier-1 features, computed from + aggregate position + oldest active unit's entry) +* New kernel `rl_unit_state_update.cu`: + * On position-from-zero transition (new trade): allocate unit slot 0, + record entry_price/step/lots/r/trail; mark active. + * On pyramid add (handled by actions_to_market_targets — see P7): caller + sets which unit slot. + * On partial-flat (handled by actions_to_market_targets — see P7): + caller deactivates oldest unit. + * On position-to-zero (full close from trail/heat/manual): deactivate + all units, zero buffers. + * Per step: recompute aggregate features for trade_context_d using + OLDEST active unit as anchor (time_in_trade from oldest's step, R + from oldest's R, etc — matches Turtle's first-unit reference). + +### Phase P2 — Encoder input dim grow (NOT side-channel) + +**Critical-review SIG-2 fix:** features at encoder INPUT, not h_t. + +* Encoder input goes from `[B × K × FeatureDim]` to + `[B × K × (FeatureDim + 16)]` where 16 = 4 trade-arc + 12 multi-res. +* The 16 extra dims are PER-BATCH state, broadcast across K (constant + within a forward window — per-batch state doesn't vary by within-window + snapshot index). +* Encoder weights' first-layer input matrix grows by 16 columns; + initialize new columns at small random (Xavier-style) — they're + encoder input, encoder must learn to use them. +* Heads' first-layer dim unchanged (HIDDEN_DIM); they see the + encoder's integrated representation. +* This is a meaningful encoder refactor but FOUNDATIONAL — encoder + learns trade-aware market patterns through its recurrent dynamics. + +### Phase P3 — Forward-Return-Distribution (FRD) head + +**Critical-review CRIT-1 fix:** supervised forecaster, no survivor bias. + +* New head module `crates/ml-alpha/src/heads/frd.rs`: + * MLP: `[HIDDEN_DIM] → 64 → (3 × 21)` outputs 3 horizons × 21 + return-bucket logits per horizon + * Buckets uniform in vol units over span `[-range_σ, +range_σ]` + where `range_σ` reads from ISV slot 521 (seeded 3.0). 21 atoms + span yields Δ_atom = `2 × range_σ / 20`. Per §0.1: range + ISV-tunable; only the 21-atom count is structural compile-time. +* New kernels `rl_frd_fwd.cu` + `rl_frd_bwd.cu` following existing + head pattern. +* **Label generation (FULLY SUPERVISED, no survivor bias):** + * At each snapshot index `i`, compute forward returns at 3 horizons: + `r_h1[i] = (mid[i + h1_ticks] - mid[i]) / σ` + `r_h2[i] = (mid[i + h2_ticks] - mid[i]) / σ` + `r_h3[i] = (mid[i + h3_ticks] - mid[i]) / σ` + * Bucket: `bucket = argmin |r - bucket_center|` (or interpolated for + smoother gradient) + * One-hot target → cross-entropy loss per horizon +* Loss = `λ_frd × (CE(h1) + CE(h2) + CE(h3))`, λ_frd ISV-driven. +* Label computation happens in loader (where forward snapshots are + available — we already do this for the BCE aux head per horizon). +* Reuses existing label cache pattern for forward-horizon labels. +* h1 = 60 ticks, h2 = 300 ticks, h3 = 1800 ticks — ISV-driven slots + for tuning. + +### Phase P4 — N_ACTIONS=9 → 11 refactor (Half-Flat actions) + +* `crates/ml-alpha/src/rl/common.rs`: Action enum adds + `HalfFlatLong = 9`, `HalfFlatShort = 10` +* Every kernel hardcoding N_ACTIONS: + * `dqn_distributional_q.cu`: define 9 → 11 + * `argmax_expected_q.cu`: same + * `rl_action_kernel.cu`: same + * `rl_pi_action_kernel.cu`: same (P_MIN floor recalculated, see P12) + * `log_pi_at_action.cu`: same + * `rl_q_pi_distill_grad.cu`: same + * `bellman_target_projection.cu`: same (shared mem grows) +* Rust constant `N_ACTIONS` everywhere → 11 + +### Phase P5 — Trail-stop kernels (per-unit) + +* `rl_trail_mutate.cu`: + * 1 block per batch, MAX_UNITS=4 threads (one per unit slot) + * If `actions_d[b] == 7` AND `unit_active[b, u]`: + `unit_trail_distance[b, u] = max(MIN_TRAIL, distance × 0.9)` + * If `actions_d[b] == 8` AND `unit_active[b, u]`: + `unit_trail_distance[b, u] = min(MAX_TRAIL, distance × (1/0.9))` + (symmetric per critical-review MIN-4 — was asymmetric ×1.1 in v2) + * Mutate ALL active units uniformly per a7/a8 action. +* `rl_trail_stop_check.cu`: + * 1 block per batch, MAX_UNITS=4 threads (one per unit slot) + * For each active unit u: + `stop_long = unit_entry_price[b,u] - unit_trail_distance[b,u]` + `stop_short = unit_entry_price[b,u] + unit_trail_distance[b,u]` + * If unit is long AND `current_mid < stop_long`: mark unit for close + * If unit is short AND `current_mid > stop_short`: mark unit for close + * If any unit marked for close: pick the MOST-AT-RISK (largest + adverse-move-from-entry in σ units) unit; set + `actions_d[b]` to `a9 (HalfFlatLong)` if pos>0 else `a10 (HalfFlatShort)` + + write `close_unit_index[b] = u` (consumed by actions_to_market_targets) + * Stop-check therefore routes through partial-flat plumbing, closing + just the at-risk unit (not the whole position). +* Per `pearl_stop_checks_run_at_deadline_cadence`: stop check fires + every step; close routes through `apply_fill_to_pos` → realizes PnL. + +### Phase P6 — Position heat cap + +* `rl_position_heat_check.cu`: + * 1 block, b_size threads + * If `|aggregate_position| > MAX_UNITS × max_unit_size`: OVERRIDE + `actions_d[b]` to FlatFromLong/Short (FULL flat, not partial) + * Heat cap is the LAST defense; full flat is acceptable here + +### Phase P7 — Pyramiding semantics + partial-flat unit selection + +* `actions_to_market_targets.cu` extension: + * **Pyramid logic:** + * When action ∈ {LongSmall=5, LongLarge=6} AND existing position > 0: + * Find LAST active unit's entry_price (highest-index active slot) + * If `(current_mid - last_unit_entry_price) >= pyramid_threshold` + AND `pyramid_units_count[b] < MAX_UNITS`: + → ADD: side=0 (buy), size=action_size + → write to NEXT FREE unit slot: entry_price, entry_step, + lots, initial_r=current_trail_init, trail_distance=current_trail_init, active=1 + → increment pyramid_units_count + * Else: REPLACE (overwrite oldest unit's entry — this is the + "rotate to fresh trade" behavior; OR no-op if at MAX_UNITS) + * Symmetric for ShortSmall/Large + * **Partial-flat unit selection:** + * When action ∈ {HalfFlatLong=9, HalfFlatShort=10}: + * Read `close_unit_index[b]` (set by trail_stop_check) + OR if not set, use OLDEST active unit (= slot with smallest + active entry_step) + * Dispatch market order: side opposite to unit's direction, + size = |unit_lots[b, close_unit_index]| + * Mark `unit_active[b, close_unit_index] = 0` + * Decrement pyramid_units_count + * **Anti-martingale sizing:** for opening actions (when no add/replace + semantics fire), `size_eff = base_size × clamp(MIN, 1 + κ × outcome_ema_d[b], MAX)`, + floored at 1 lot. + +### Phase P8 — Confidence gate + +* `rl_confidence_gate.cu`: + * 1 block per batch, single thread + * Reads `q_logits[b × N_ACTIONS × Q_N_ATOMS]` + `atom_supports` + * For chosen action `a* = actions_d[b]`: + * Numerically-stable softmax over q_logits[b, a*, *] + * `μ = Σ probs · atom_supports` + * `σ² = Σ probs · (atom_supports - μ)²` + * `conf = clamp(0, (μ - λ × sqrt(σ²)) / σ_norm, 1)` + * If `position == 0 && a* ∈ {0,1,5,6} && conf < THR_CONF`: + OVERRIDE to Hold (a2) + +### Phase P9 — FRD gate + +* `rl_frd_gate.cu`: + * 1 block per batch, single thread + * Reads FRD head output: 3 horizons × 21 logits + * Compute `entry_quality_h2 = Σ_{bucket > +0.5σ} softmax(logits_h2)` + * If `position == 0 && a* ∈ {0,1,5,6}` AND: + * `a* ∈ {LongSmall, LongLarge}` AND `entry_quality_h2 < THR_FRD_LONG` OR + * `a* ∈ {ShortSmall, ShortLarge}` AND `entry_quality_h2_neg < THR_FRD_SHORT` + (where entry_quality_h2_neg = `Σ_{bucket < -0.5σ}`) + * OVERRIDE to Hold (a2) +* THR_FRD_LONG = THR_FRD_SHORT = 0.35 default (slightly above random + 35% — only take entries where forward bias is mildly favorable) + +### Phase P10 — Anti-martingale (per-batch, foundational CRIT-4 fix) + +* `rl_recent_outcome_update.cu`: + * 1 block, b_size threads + * For each batch with done_count > 0: + * `outcome_b = sign(reward[b])` + * `outcome_ema_d[b] = (1-α) × outcome_ema_d[b] + α × outcome_b` + * For batches with done_count == 0: no update (sparse-aware, + matches the discipline of other reward EMAs) +* Per-batch buffer `outcome_ema_d[B]: f32` (not a single ISV slot) +* Sizing modulation in actions_to_market_targets (see P7) + +### Phase P11 — Vol-adjusted defaults + +* Vol signal: `realized_return_var_ema` already exists in ISV. +* Initial trail per new unit: + `unit_trail_distance[b, u_new] = k_init × sqrt(realized_return_var_ema)` + where k_init from ISV slot 500. +* Pyramid threshold (recomputed per step, consumed by actions_to_market_targets): + `0.5 × sqrt(realized_return_var_ema)` +* Initial R per unit: `unit_initial_r[b, u_new] = trail_distance[b, u_new]` + (R = initial stop distance per Van Tharp) + +### Phase P12 — P_MIN floor adjustment for N=11 + +**Critical-review SIG-5 fix:** at N=9, max single-action prob = 0.84. +At N=11 with P_MIN=0.02, max = 0.80 (4pp drop in commitment ceiling). + +* Make P_MIN ISV-driven (was hardcoded #define): new ISV slot 513 + RL_PI_SAMPLE_P_MIN_INDEX +* Seed P_MIN to 0.015 → max prob = 1 - 10×0.015 = 0.85 (slight + uplift vs N=9 baseline) +* The kernel reads from ISV; runtime-tunable. + +### Phase P13 — Diagnostics — EXHAUSTIVE exposure + +Hard requirement: every new mechanic surfaces full state in diag JSONL. + +```json +"trade_context": { + "time_in_trade_norm": [b...], + "unrealized_R": [b...], + "pos_magnitude_norm": [b...], + "entry_distance_sigma": [b...] +}, +"multires_features": { + "scale_1s": {"price_change":[b], "vol":[b], "oflow_imb":[b], "trade_burst":[b]}, + "scale_10s": {...}, + "scale_600s": {...} +}, +"units": { // per-batch per-unit + "entry_price": [[u0,u1,u2,u3] per b...], + "entry_step": [[...] per b...], + "lots": [[...] per b...], + "trail_distance": [[...] per b...], + "active_mask": [[...] per b...], + "unit_count": [b...] // sum of active per batch +}, +"trail": { + "fired_count_step": n_per_unit_stop_fires_this_step, + "fired_count_total": cumulative, + "tightened_count_step": n_a7_fires_step, + "loosened_count_step": n_a8_fires_step, + "tightened_count_total": cumulative, + "loosened_count_total": cumulative +}, +"pyramid": { + "added_count_step": n_pyramid_adds_step, + "added_count_total": cumulative, + "units_distribution": [count_at_0, ..., count_at_4], // histogram over all batches + "max_units_reached": bool (any batch at MAX_UNITS this step) +}, +"partial_flat": { + "fired_count_step": n_half_flats_step, + "fired_count_total": cumulative, + "long_count_total": cumulative a9 fires, + "short_count_total": cumulative a10 fires, + "close_unit_index": [b...] // which unit slot was closed +}, +"frd": { + "entry_quality_h1": [b...], + "entry_quality_h2": [b...], + "entry_quality_h3": [b...], + "gated_count_step": n_blocked_by_frd_step, + "gated_count_total": cumulative, + "loss_h1": head loss per horizon, + "loss_h2": ..., + "loss_h3": ... +}, +"confidence_gate": { + "conf": [b...], + "gated_count_step": n_blocked_by_conf, + "gated_count_total": cumulative, + "mean_conf": window-mean +}, +"position_heat": { + "capped_count_step": n_heat_caps_step, + "capped_count_total": cumulative, + "heat_max_lots": current cap value +}, +"anti_martingale": { + "outcome_ema": [b...], // per-batch + "size_mult": [b...], // per-batch + "kappa": ISV value (single scalar) +} +``` + +Plus every new ISV slot in `isv_config` block. + +### Phase P14 — Local validation tests (28 tests, exhaustive interaction coverage) + +**Critical-review SIG-3 fix:** interaction edge cases added. + +| # | Test | Asserts | +|---|---|---| +| 1 | trail_one_way_long | trail_distance only ↓ on a7, only ↑ on a8 | +| 2 | trail_one_way_short | symmetric | +| 3 | trail_at_min_a7_noop | trail at MIN_TRAIL + a7 fires: stays at MIN | +| 4 | trail_at_max_a8_noop | trail at MAX_TRAIL + a8 fires: stays at MAX | +| 5 | trail_stop_fires_on_breach | per-unit stop check fires when violated | +| 6 | trail_stop_picks_most_at_risk | with 2 active units, both adverse, picks the one with largest adverse move | +| 7 | pyramid_only_in_profit | LongLarge pos=2 down 0.3σ: REPLACES; up 0.6σ from last unit: ADDS | +| 8 | pyramid_threshold_uses_last_unit | not aggregate entry — verify | +| 9 | pyramid_unit_cap_replaces_oldest | at MAX_UNITS: REPLACES oldest (rotates) — OR no-ops, decide and lock | +| 10 | partial_flat_closes_oldest_unit | a9/a10 default picks unit with smallest active entry_step | +| 11 | partial_flat_respects_trail_close_choice | when trail_stop_check pre-set close_unit_index, partial_flat uses it | +| 12 | partial_flat_zero_position_noop | a9/a10 with pos=0: no-op, no buffer mutation | +| 13 | partial_flat_single_unit | a9 with 1 active unit: closes it fully | +| 14 | confidence_gate_blocks_low_conf | low μ + high σ: open → Hold | +| 15 | confidence_gate_allows_high_conf | high μ + low σ: passes | +| 16 | confidence_gate_never_blocks_exit | pos != 0: exits + trail actions pass | +| 17 | frd_gate_blocks_low_quality | entry_quality_h2 < THR: open → Hold | +| 18 | frd_gate_long_short_directional | low entry_quality_h2 blocks Long, low entry_quality_h2_neg blocks Short | +| 19 | frd_gate_never_blocks_exit | pos != 0: exits + trail pass | +| 20 | both_gates_filter_same_action | FRD + confidence both block: action → Hold (no double-trigger needed) | +| 21 | anti_mart_per_batch_independent | batches' outcome_ema independent; per-batch size_mult differs | +| 22 | anti_mart_bigger_after_wins | outcome_ema=+0.8, κ=0.2: size_mult=1.16 | +| 23 | anti_mart_smaller_after_losses | outcome_ema=-0.8: size_mult=0.84 | +| 24 | anti_mart_bounded | extreme outcomes don't violate MIN/MAX | +| 25 | position_heat_force_full_flat | |pos|>cap: full FlatFromX (not partial) | +| 26 | trail_then_heat_race | trail fires AND heat fires same step: heat wins (full flat overrides partial) | +| 27 | trade_context_oldest_unit_anchor | with 3 active units, time_in_trade uses oldest unit's step | +| 28 | multires_features_real_time | scales computed from `ts_ns` deltas, not event counts; quiet/busy regimes give same time horizon | + +### Phase P15 — Cluster smoke + walk-forward 5-fold + +* Cluster smoke at 200k steps × 5 folds = 5 parallel argo runs +* Aggregate G8 gate via `mean - 0.5 × std` per + [[pearl-single-window-oos-is-not-oos]] +* Per-fold diag pulled, full chain validated + +## 4 — Design decisions (foundational picks documented) + +### 4.1 N_ACTIONS=9 → 11 (full refactor, not workaround) + +Rejected cheap alternative: amount-parameterized FlatFromX (single +action takes config'd close amount). Loses Q's per-action granularity; +Q can't distinguish "should fully close" from "should half close" as +distinct decisions. + +Chosen: full N_ACTIONS refactor with Half-Flat as first-class actions. +Touches ~8 kernels mechanically. + +### 4.2 Features at encoder INPUT (not side-channel) + +Rejected cheap alternative: side-channel concat at h_t (encoder weights +untouched). + +Critical-review SIG-2: side-channel means encoder never learns +trade-aware representations through its recurrent dynamics. + +Chosen: features at encoder input, broadcast across K. Encoder +weights' first-layer grows by 16 columns. Encoder learns market +patterns CONDITIONED on trade phase + multi-res context. + +### 4.3 FRD head (supervised forecaster, not survivor-biased classifier) + +Rejected cheap alternative (v2 spec): self-supervised checklist head +with label = "was next trade profitable?" — has survivor bias per +critical-review CRIT-1. + +Chosen: FRD head trained on forward-return DISTRIBUTION (no agent +involvement in label). Generally useful beyond gating — any +downstream consumer can use forward-return forecasts. + +### 4.4 Per-unit pyramid state (not aggregate-entry anchor) + +Rejected cheap alternative (v2 spec): aggregate entry as trail anchor. +Critical-review CRIT-2: late pyramid units get trails 3.5σ adverse +from their entries — pyramid INCREASES downside. + +Chosen: per-unit entry + per-unit trail + per-unit stop check. +Per-batch per-unit buffers add `B × MAX_UNITS × {f32×3 + i32×2 + u8}` +≈ 6.5KB for B=16 — trivial. + +### 4.5 Override stack order — trail-mutate BEFORE trail-stop-check + +Critical-review CRIT-3 fix. Mutation must take effect before check sees +the updated trail. + +### 4.6 Anti-martingale per-batch (not global) + +Critical-review CRIT-4 fix. Per-batch `outcome_ema_d[B]` preserves +each batch's own trade-streak fidelity. + +### 4.7 Multi-resolution scales TIME-based (not event-count) + +Critical-review SIG-4 fix. Requires `ts_ns` on snapshot. If +unavailable, prerequisite Phase P-0.5 adds it. + +### 4.8 FRD + Confidence gates both on entries — different failure modes + +* FRD gate: "does market data forecast favorable forward return?" + (supervised, market-truth) +* Confidence gate: "is Q evaluation high-conviction for this state-action?" + (RL-derived, value-based) + +Filter different signals. Both required for "best of N" entry quality. +Combined pass rate ≈ 0.6 × 0.85 ≈ 51% — selective but not extreme. + +Corrected per critical-review MIN-1: NOT "best of 100" pattern; SP20 +filters ~50% of entry opportunities, the bottom-half by joint +forecaster+conviction quality. + +### 4.9 Anti-martingale uses outcome SIGN, NOT magnitude + +Magnitude couples with reward-scale-controller drift. Sign at α +(slot 520 `RL_ANTI_MART_OUTCOME_EMA_ALPHA_INDEX`, seeded 0.05) +gives ~20-trade memory window per batch. α is ISV-tunable per §0.1. + +### 4.10 Pyramid threshold uses LAST UNIT'S entry as reference + +Each pyramid add waits for `current_mid - last_unit_entry ≥ 0.5σ` — +each add gets fresh favorable confirmation. Matches Turtle's +"add at every 0.5N from previous unit's entry" semantics. + +### 4.11 At-MAX-UNITS LongLarge behavior: ROTATE (replace oldest) + +When pyramid at max, additional LongLarge could no-op OR rotate. +Chosen: ROTATE — close oldest unit (via partial-flat semantics +internally), open new unit at current price. This lets the agent +"keep moving its window of bets forward" rather than getting stuck +holding stale entries. Test #9 verifies. + +### 4.12 Trail mutate symmetric — adjust rate ISV-driven + +Critical-review MIN-4 fix. Tighten and loosen multipliers reciprocal: +N tightens followed by N loosens returns to original distance. + +Per §0.1 audit: the rate itself (0.9) is now ISV-driven via slot 519 +`RL_TRAIL_ADJUST_RATE_INDEX`. Runtime tunable without recompile. +Tighten = `× rate`; loosen = `× (1/rate)`; symmetry preserved by +construction. + +### 4.13 Slot 495 RL_PYRAMID_THRESHOLD removed + +Critical-review MIN-5: consumer reads from +`realized_return_var_ema` directly with 0.5× multiplier inline. +Saves one slot, eliminates "no producer" inconsistency. + +### 4.14 Acceptance gates: multi-tier + +Critical-review SIG-1 + MIN-6: replace single hard gate with multi-tier: +* SOFT: WR > 43% (modest improvement validates direction) +* HARD: WR > 45% (significant improvement, near break-even WR) +* STRETCH: WR > 50% (mid-goal toward 55%) +* CELEBRATION: WR > 55% (user's ultimate goal) + +SP20 ships if HARD achieved (45%); STRETCH and CELEBRATION measured +for trajectory tracking. + +## 5 — ISV slot allocation + +| Slot | Name | Default seed | Producer / Consumer | +|---|---|---|---| +| 493 | RL_TRAIL_MIN_INDEX | `0.1 × sqrt(var_ema_floor)` | seed; rl_trail_mutate | +| 494 | RL_TRAIL_MAX_INDEX | `10.0 × sqrt(var_ema_floor)` | seed; rl_trail_mutate | +| 496 | RL_PYRAMID_MAX_UNITS_INDEX | 4.0 | seed; actions_to_market_targets + heat_check | +| 497 | RL_CONFIDENCE_LAMBDA_INDEX | 0.5 | seed; rl_confidence_gate | +| 498 | RL_CONFIDENCE_SIGMA_NORM_INDEX | 1.0 | seed; rl_confidence_gate | +| 499 | RL_ENTRY_CONFIDENCE_THRESHOLD_INDEX | 0.05 | seed; rl_confidence_gate | +| 500 | RL_TRAIL_K_INIT_INDEX | 2.0 (Turtle 2N) | seed; rl_unit_state_update on open/pyramid | +| 501 | RL_TRAIL_DISTANCE_DIAG_INDEX | per-batch mean trail | diag-only | +| 502 | RL_TRADE_TIME_NORM_FLOOR_INDEX | 30.0 | seed; trade_context normalization | +| 504 | RL_FRD_THRESHOLD_LONG_INDEX | 0.35 | seed; rl_frd_gate | +| 505 | RL_ANTI_MART_KAPPA_INDEX | 0.2 | seed; actions_to_market_targets | +| 506 | RL_ANTI_MART_MIN_MULT_INDEX | 0.5 | seed; actions_to_market_targets | +| 507 | RL_ANTI_MART_MAX_MULT_INDEX | 1.5 | seed; actions_to_market_targets | +| 508 | RL_MULTIRES_HORIZON_1_S_INDEX | 1.0 (seconds) | seed; rl_multires_features_update | +| 509 | RL_MULTIRES_HORIZON_2_S_INDEX | 10.0 | seed; same | +| 510 | RL_MULTIRES_HORIZON_3_S_INDEX | 600.0 | seed; same | +| 511 | RL_FRD_LAMBDA_INDEX | 0.5 | seed; FRD head backward | +| 512 | RL_FRD_LR_INDEX | 1e-3 | LR controller-driven | +| 513 | RL_PI_SAMPLE_P_MIN_INDEX | 0.015 | seed; rl_pi_action_kernel (was hardcoded 0.02) | +| 514 | RL_FRD_THRESHOLD_SHORT_INDEX | 0.35 | seed; rl_frd_gate (symmetric to 504) | +| 515 | RL_FRD_HORIZON_1_TICKS_INDEX | 60.0 | seed; FRD label gen + head | +| 516 | RL_FRD_HORIZON_2_TICKS_INDEX | 300.0 | seed; same | +| 517 | RL_FRD_HORIZON_3_TICKS_INDEX | 1800.0 | seed; same | +| 519 | RL_TRAIL_ADJUST_RATE_INDEX | 0.9 (×rate, loosen=1/0.9) | seed; rl_trail_mutate (was hardcoded in v3 draft per §0.1 audit) | +| 520 | RL_ANTI_MART_OUTCOME_EMA_ALPHA_INDEX | 0.05 | seed; rl_recent_outcome_update (was hardcoded per §0.1 audit) | +| 521 | RL_FRD_BUCKET_RANGE_SIGMA_INDEX | 3.0 (±3σ atom span) | seed; FRD head fwd + label gen (was hardcoded per §0.1 audit) | +| 522 | RL_PYRAMID_UNIT_SIZE_MAX_INDEX | 2.0 (= LongLarge size) | seed; heat cap calc + pyramid sizing (was implicit per §0.1 audit) | + +`RL_SLOTS_END`: 493 → 523 (28 new slots; 495 + 503 + 518 dropped per design +decisions §4.13, CRIT-4 fix, and consolidation). + +Per-batch buffers (NOT in ISV bus): +* `unit_entry_price_d[B × 4]: f32` +* `unit_entry_step_d[B × 4]: i32` +* `unit_lots_d[B × 4]: i32` +* `unit_initial_r_d[B × 4]: f32` +* `unit_trail_distance_d[B × 4]: f32` +* `unit_active_d[B × 4]: u8` +* `trade_context_d[B × 4]: f32` +* `multires_features_d[B × 12]: f32` +* `multires_window_state_d[B × 3 × MAX_WINDOW_ENTRIES × 5]: f32` (circular per horizon) +* `outcome_ema_d[B]: f32` (anti-martingale per-batch) +* `frd_head_output_d[B × 3 × 21]: f32` (forecast logits per horizon) +* `close_unit_index_d[B]: i32` (set by trail_stop_check, read by partial-flat) + +## 6 — Validation gates + +### 6.1 Local tests (28 tests, all in Phase P14) + +Enumerated above. Cover: +* Each mechanic in isolation (1-2, 4-5, 7-8, 14-15, 17, 21-23, 25) +* Edge cases at bounds (3, 4, 12, 13) +* Multi-unit interactions (6, 8, 10, 11, 26, 27) +* Override-stack races (20, 26) +* Reality checks (28) + +### 6.2 Cluster smoke G8 gates (5-fold walk-forward) + +| Gate | Criterion | Severity | +|---|---|---| +| G8-A | Trail fires meaningfully | trail_fired between 2% and 40% of dones | +| G8-B | Pyramid happens late | pyramid_added > 0 in last 50k window | +| G8-C | Partial-flat happens | partial_flat_count > 0 in last 50k window | +| G8-D | FRD gate active but not over-filtering | gated between 5% and 60% of entry-eligible steps | +| G8-E | Confidence gate active | gated between 1% and 50% | +| G8-F | Encoder grad-norm non-zero on new input dims (trade_context + multires) | check via separate forward+backward instrumented run | +| G8-G | Anti-martingale variance | per-batch size_mult std > 0.05 | +| G8-H | Per-unit tracking consistent | sum(unit_active × unit_lots) == position for every batch every step | +| G8-I | R/done improvement | walk-forward mean - 0.5σ > -$0.05 (significant improvement vs 7d7mp's -$0.073) | +| G8-J | WR HARD gate | walk-forward mean ≥ 45% | +| G8-K | WR STRETCH | walk-forward mean ≥ 50% (target, not blocking) | +| G8-L | WR CELEBRATION | walk-forward mean ≥ 55% (user's ultimate goal) | +| G8-M | No catastrophic regression | NaN/inf count = 0 in any fold | +| G8-N | Heat cap fires rarely | heat_capped between 0% and 3% | +| G8-O | FRD head learns | CE loss decreases monotonically over windowed mean | + +SHIP gate: G8-A through G8-J pass. G8-K and G8-L tracked for trajectory. + +## 7 — Memory rule compliance checklist + +### GPU/CPU contract +- `feedback_cpu_is_read_only` — all new kernels device-only +- `feedback_no_htod_htoh_only_mapped_pinned` — buffers `alloc_zeros` + DtoD +- `feedback_no_atomicadd` — per-batch + per-unit independent writes +- `feedback_no_nvrtc` — register all .cu in build.rs + +### Code quality +- `feedback_no_stubs` — every kernel has consumer, every state has reader +- `feedback_no_todo_fixme` — clean +- `feedback_no_hiding` — every arg used +- `feedback_no_feature_flags` — features unconditional + +### Architecture +- `feedback_isv_for_adaptive_bounds` — all bounds in ISV +- `feedback_wire_everything_up` — same-commit: kernels + modules + launches + diag + tests +- `pearl_no_deferrals_for_complementary_fixes` — all 5 tiers ship together +- `pearl_first_observation_bootstrap` — trade_context zero-init = "no trade"; + unit buffers zero-init = "no unit" +- `pearl_stop_checks_run_at_deadline_cadence` — trail check every step; + force-close via partial-flat → apply_fill_to_pos +- `pearl_trade_level_vol_for_stop_distance` — trail/pyramid scale with + sqrt(realized_return_var_ema) +- `pearl_single_window_oos_is_not_oos` — 5-fold walk-forward +- `pearl_extending_existing_code_audits_for_existing_violations` — + pre-extension audit of every file touched +- `pearl_audit_unboundedness_for_implicit_asymmetry` — trail + ×0.9 / ×(1/0.9) symmetric per §4.12 + +### Training discipline +- `feedback_default_to_l40s_pool` — smoke on ci-training-l40s × 5 parallel +- `feedback_stop_on_anomaly` — G8 NaN abort retained +- `feedback_push_before_deploy` — push before argo submit + +### Diagnostics discipline (NEW: hard requirement) +- Every new ISV slot exposed in `isv_config` +- Every new per-batch buffer exposed in dedicated diag block +- Every new override kernel exposes per-step fire count + cumulative +- Every new controller exposes its input + output + EMA state +- Every new head exposes its loss + LR + per-step output range + +## 8 — Risk register + +| Risk | Mitigation | +|---|---| +| Phase P-1 ceiling falsification shows current arch suffices | SP20 deferred; ship 1M-step P-1 result to production | +| N_ACTIONS=11 refactor breaks downstream kernels | 28 local tests cover dim consistency; pre-cluster local smoke at 1k steps validates | +| Encoder input dim grow destabilizes existing trained encoder | We're greenfield re-training anyway (heads' weights re-init for trade-arc); encoder also random-init from this commit | +| Per-unit tracking adds state mgmt bugs | Test #6, #8, #10, #11, #27 cover per-unit semantics; G8-H verifies invariant `sum(units) == position` | +| Multi-res window state buffer too large | At 600s × ~100 events/sec = 60k entries × 5 fields × 4 bytes = 1.2MB per batch per horizon; × 3 horizons × 16 batches ≈ 60MB. Fine for L40S 48GB. | +| FRD head label requires forward snapshots not always available | Use existing forward-horizon label cache pattern (BCE aux head already does this for h=10/100/1000); FRD reuses same loader infrastructure | +| FRD horizons differ from BCE aux horizons | Both can coexist; FRD has its own ISV slots (515-517); no conflict | +| FRD + confidence both block 80%+ of entries | Local test for combined rate < 70%; adjust thresholds before cluster smoke if violated | +| Pyramid rotate-at-max behavior loses good positions | Test #9 locks the behavior; if rotation hurts, revisit threshold values not the rotation logic | +| Anti-martingale per-batch outcome_ema starves at trade-sparse batches | Sparse-aware: EMA only updates on done events; batches with no trades retain prior EMA value (matches existing pos_max_ema discipline) | +| Trail mutate symmetric vs adapter expects ×1.1 | §4.12 documents the change; no consumer expects ×1.1 since current state is dead | +| Override stack ordering subtle bugs | §2.3 documents strict order; per-kernel test isolates each override | +| Cost of 5-fold walk-forward × multiple iterations | L40S autoscale; ~$25-75/session × 3-5 sessions = ~$100-400 total | + +## 9 — Memory pearls to write on completion + +* `pearl-ceiling-falsification-before-architecture-expansion` — P-1 discipline +* `pearl-trade-arc-features-at-encoder-input-not-side-channel` — design rationale +* `pearl-per-unit-pyramid-state-not-aggregate` — Turtle-correct anchor +* `pearl-trail-mutate-before-trail-stop-check` — override ordering +* `pearl-trail-stop-routes-through-partial-flat` — per-unit close mechanism +* `pearl-frd-head-supervised-not-survivor-biased-checklist` — forecaster pattern +* `pearl-per-batch-anti-martingale-outcome-ema` — sizing fidelity +* `pearl-multi-resolution-time-not-event-count` — temporal scale semantics +* `pearl-confidence-gate-and-frd-gate-complementary` — filter different failure modes +* `pearl-p-min-floor-adaptive-to-n-actions` — N_ACTIONS-aware commitment ceiling +* `pearl-half-flat-action-requires-action-space-extension` — partial-close design + +Per-finding pearls added as cluster evidence justifies. + +## 10 — Out of scope (true architectural alternatives, not deferred shortcuts) + +* **Per-unit asymmetric trail multipliers** — tighten at ×0.9, loosen at + ×1.111 per unit independently (different units could mutate differently). + Future SP if uniform-across-units shows issues. +* **Hierarchical option-critic** — formal entry vs management policies + with separate value functions. SP20's augmented-state approach is a + DIFFERENT architecture, not equivalent (clarified per critical-review MIN-2); + hierarchical is its own SP. +* **Multi-instrument extension** — single-instrument scope. +* **External event/news features** — outside MBP-10 feature set. +* **Per-action LR / Adam hyperparams for a9/a10** — new actions get + defaults same as a3/a4; per-action LR is its own SP. + +## 11 — Acceptance criteria (multi-tier per §4.14) + +SP20 ships when: + +1. **Foundational principles §0 audited per-phase AND globally:** + * §0.1 ISV audit: zero hardcoded numerical constants in any new .cu + * §0.2 Wiring audit: zero orphan kernels, zero orphan slots, zero + dead actions, zero unwired heads + * §0.3 Diag audit: every new observable field present + non-trivial + in local 100-step smoke +2. ~~Phase P-1 ceiling falsification~~ — intentionally skipped per §3 P-1 + user decision; not blocking +3. All §6.1 / Phase P14 local tests pass (28 tests, all mechanics + + interactions) +4. `SQLX_OFFLINE=true cargo check --workspace` clean +5. `SQLX_OFFLINE=true cargo test -p ml-alpha --lib` clean +6. Local 1k-step smoke (b_size=16) completes; diag JSONL contains + ALL new fields with non-trivial values for at least 1 step +7. Cluster 200k smoke × 5-fold walk-forward all Succeeded +8. G8 gates A-J pass with 5-fold `mean - 0.5σ` aggregation +9. G8-K (WR > 50%) measured; result drives next-SP scope + +If gate G8-I/J fail: full per-tier diag analysis (every new mechanic +has its own diag block per §0.3) identifies which sub-mechanic +underperformed; iterate within SP20 scope (re-tune ISV-seeded +thresholds — possible at runtime per §0.1) within ≤2 retry sessions +before declaring SP20 unsuccessful and pivoting to feature engineering. + +**§0 audit scripts (must exist + pass at every phase commit):** +* `scripts/audit-isv.sh` — greps new .cu for `#define` numerics + outside the §0.1 structural-dim exception list +* `scripts/audit-wiring.sh` — verifies every KERNELS entry has + launch site, every ISV slot has producer + consumer, every Action + has handler, every head has fwd+bwd+loss+adam +* `scripts/audit-diag.sh` — runs local 100-step smoke, parses + JSONL, verifies every spec'd field present + non-trivial + +## 12 — Implementation cost (honest per critical-review MIN-3) + +* **Files touched:** ~22-25 (8 kernels for N_ACTIONS refactor, ~12 new + kernels, encoder weight init handling, 3-4 Rust modules, build.rs, + loader for FRD labels) +* **New ISV slots:** 24 (493-517) +* **New per-batch buffers:** 12 (including multi-res window state) +* **New tests:** 28 +* **Estimated effort:** **3-4 weeks** careful work (single greenfield + commit). Includes: + * Week 1: Phases P-1 + P0 + P1 + P2 (foundational state + encoder refactor) + * Week 2: Phases P3 + P4 + P5 + P6 (FRD head + N_ACTIONS + trail + heat) + * Week 3: Phases P7 + P8 + P9 + P10 + P11 + P12 + P13 (pyramid/partial/gates/anti-mart/diag) + * Week 4: Phase P14 testing + 2-4 smoke iterations + post-mortem + pearls +* **Smoke cost:** 5-fold × ~$5-15 = ~$25-75 per session; expect 3-5 + sessions through iteration = ~$100-400 total + +## 13 — Linked pearls / SPs / research + +### Foundational rules +* `pearl-no-deferrals-for-complementary-fixes` — coupling discipline +* `pearl-trade-level-vol-for-stop-distance` — trail signal source +* `pearl-stop-checks-run-at-deadline-cadence` — stop integration +* `pearl-dead-trail-stop-actions-a7-a8` — the gap this SP fills +* `pearl-q-to-pi-distillation-breaks-v-vanishing-trap` — upstream chain +* `pearl-c51-atom-span-must-track-clamp-range` — upstream chain +* `pearl-adaptive-reward-clamp-from-positive-tail` — upstream chain +* `pearl-single-window-oos-is-not-oos` — multi-fold requirement +* `pearl-extending-existing-code-audits-for-existing-violations` — discipline +* `pearl-isv-for-adaptive-bounds` — ISV slot discipline +* `pearl-first-observation-bootstrap` — sentinel handling +* `pearl-audit-unboundedness-for-implicit-asymmetry` — symmetry discipline + +### Behavioral research sources +* Turtle Trading (Dennis / Faith): per-unit 2N stop, 0.5N add threshold +* Van Tharp R-multiples: partial profit-taking +1R, +2R +* Larry Williams / Ryan Jones: anti-martingale sizing +* Kelly (1956), fractional Kelly empirical (Wikipedia, Alpha Theory) +* Distributional RL trade-management (Coursera + arXiv 2411.07585): + hierarchical entry/management policy + position-context state +* Generic vol-stop framework (Zeiierman, AlchemyMarkets, + TradingSetupsReview): Stop = Anchor ± k × σ + +### Predecessor architecture +* `docs/superpowers/plans/2026-05-23-integrated-rl-trainer-rebuild.md` + — base architectural rebuild this SP extends + +## 14 — Revision history + +* **v1** (initial draft): 3 tiers, side-channel features, aggregate-entry + pyramid trail, self-supervised checklist with survivor bias, single-gate + acceptance, ~1.5-2 weeks scope estimate +* **v2** (user expansion): 5 tiers, partial profit-taking added, anti-martingale + added, multi-res features added, checklist head with self-supervised label, + ~1.5-2.5 weeks scope estimate +* **v3** (THIS — post-critical-review): foundational fixes for all 4 + CRIT + 6 SIG + 6 MIN findings: + * CRIT-1: Forward-Return-Distribution head replaces survivor-biased checklist + * CRIT-2: Per-unit pyramid state (entry + trail + R per unit) + * CRIT-3: Override stack reordered (trail-mutate before trail-stop-check) + * CRIT-4: Per-batch anti-martingale outcome EMA (was global scalar) + * SIG-1: Multi-tier acceptance (soft/hard/stretch/celebration) + * SIG-2: Encoder INPUT injection (was side-channel) + * SIG-3: 28 tests (was 19), interaction edge cases added + * SIG-4: Real-time multi-res scales (was event-count) + * SIG-5: P_MIN ISV-tunable + lowered for N=11 + * SIG-6: Phase P-1 ceiling falsification gate (pre-SP20) + * MIN-1: "Best of 100" framing corrected (~50% pass rate documented) + * MIN-2: Hierarchical-RL non-equivalence acknowledged + * MIN-3: Cost estimate 3-4 weeks (was 1.5-2.5) + * MIN-4: Trail mutate symmetric (was ×0.9 / ×1.1 asymmetric) + * MIN-5: Slot 495 dropped (no producer) + * MIN-6: Multi-tier gates (no soft "iterate within scope" ambiguity) diff --git a/scripts/audit-diag.sh b/scripts/audit-diag.sh new file mode 100755 index 000000000..0ee1dce5c --- /dev/null +++ b/scripts/audit-diag.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# audit-diag.sh — enforce "every observable surfaces in diag JSONL" +# per SP20 §0.3. +# +# Runs a local 100-step smoke (small footprint, b_size=4, predecoded +# test data) and parses the produced diag JSONL. For each entry in +# scripts/audit-manifest/diag-fields.txt: verify the jq-syntax path +# resolves to a non-null, non-NaN value in at least one sampled row. +# +# Exit codes: +# 0 — zero missing fields +# 1 — one or more missing fields +# 2 — usage / smoke-run failure + +set -euo pipefail + +ROOT=$(git rev-parse --show-toplevel) +MANIFEST="$ROOT/scripts/audit-manifest/diag-fields.txt" +BINARY="$ROOT/target/release/examples/alpha_rl_train" +SMOKE_OUT_DIR="${TMPDIR:-/tmp}/audit-diag-smoke-$$" +DIAG_FILE="$SMOKE_OUT_DIR/diag.jsonl" + +if ! command -v jq >/dev/null 2>&1; then + echo "ERROR: jq required (apt install jq)" >&2 + exit 2 +fi + +if [[ ! -f "$MANIFEST" ]]; then + echo "ERROR: manifest not found: $MANIFEST" >&2 + exit 2 +fi + +mapfile -t fields < <(grep -vE '^\s*(#|$)' "$MANIFEST") + +if [[ ${#fields[@]} -eq 0 ]]; then + echo "audit-diag: manifest empty — no diag fields to audit. PASS." + exit 0 +fi + +# ── Build binary if missing/stale ───────────────────────────── +if [[ ! -x "$BINARY" || "$BINARY" -ot "$ROOT/crates/ml-alpha/src/trainer/integrated.rs" ]]; then + echo "audit-diag: building alpha_rl_train release binary..." + (cd "$ROOT" && SQLX_OFFLINE=true cargo build -p ml-alpha --example alpha_rl_train --release 2>&1 | tail -5) +fi + +# ── Local smoke: 100 steps, b_size=4, single fold, single seed ─── +mkdir -p "$SMOKE_OUT_DIR" +echo "audit-diag: running 100-step local smoke (b_size=4) → $SMOKE_OUT_DIR" + +# Inputs — use test-data MBP-10 baseline +TEST_DATA="${FOXHUNT_TEST_DATA:-$ROOT/test_data/futures-baseline}" +if [[ ! -d "$TEST_DATA/ES.FUT" ]]; then + echo "ERROR: test data missing: $TEST_DATA/ES.FUT" >&2 + echo "Set FOXHUNT_TEST_DATA env var to a directory containing ES.FUT/ subdirectory" >&2 + exit 2 +fi + +PREDECODED="$TEST_DATA/predecoded" +if [[ ! -d "$PREDECODED" ]]; then + PREDECODED="$TEST_DATA" +fi + +"$BINARY" \ + --mbp10-data-dir "$TEST_DATA/ES.FUT" \ + --trades-data-dir "$TEST_DATA/ES.FUT" \ + --predecoded-dir "$PREDECODED" \ + --output-dir "$SMOKE_OUT_DIR" \ + --n-steps 100 \ + --n-backtests 4 \ + --seq-len 32 \ + --per-capacity 256 \ + --seed 16962 \ + --instrument-mode front-month \ + --n-folds 1 \ + --fold-idx 0 \ + --n-eval-steps 0 \ + > "$SMOKE_OUT_DIR/stdout.log" 2> "$SMOKE_OUT_DIR/stderr.log" \ + || { echo "ERROR: local smoke failed; see $SMOKE_OUT_DIR/stderr.log"; tail -20 "$SMOKE_OUT_DIR/stderr.log"; exit 2; } + +if [[ ! -f "$DIAG_FILE" ]]; then + echo "ERROR: smoke completed but diag.jsonl not produced at $DIAG_FILE" >&2 + exit 2 +fi + +n_rows=$(wc -l < "$DIAG_FILE") +echo "audit-diag: smoke produced $n_rows diag rows" + +if [[ $n_rows -lt 10 ]]; then + echo "ERROR: too few diag rows ($n_rows); smoke may have crashed early" >&2 + exit 2 +fi + +# Sample 3 rows: first, middle, last +mid_row=$((n_rows / 2)) +last_row=$n_rows +sample_rows=(1 $mid_row $last_row) + +# ── Verify each manifest field present + non-trivial ───────── +violations=0 +echo +echo "audit-diag: validating ${#fields[@]} field path(s)..." +for field in "${fields[@]}"; do + found_any=0 + bad_any=0 + for r in "${sample_rows[@]}"; do + val=$(sed -n "${r}p" "$DIAG_FILE" | jq -c "$field // empty" 2>/dev/null || echo "") + if [[ "$val" == "" ]]; then + bad_any=1 + continue + fi + if [[ -n "$val" && "$val" != "null" ]]; then + # Check for NaN (jq emits null for NaN typically; double-check) + if [[ "$val" != *"NaN"* && "$val" != *"nan"* ]]; then + found_any=1 + fi + fi + done + if [[ $found_any -eq 0 ]]; then + echo " VIOLATION field $field: not present (or null/NaN) in sampled rows" + violations=$((violations + 1)) + fi +done + +echo +echo "audit-diag: $violations missing field(s)" + +if [[ $violations -gt 0 ]]; then + echo + echo "Per SP20 §0.3: every observable state MUST be in diag JSONL same commit as the mechanic." + echo "Add the field to the diag JSONL emission in crates/ml-alpha/examples/alpha_rl_train.rs" + exit 1 +fi + +echo "audit-diag: PASS (sampled rows: ${sample_rows[*]})" +# Cleanup smoke dir on success +rm -rf "$SMOKE_OUT_DIR" +exit 0 diff --git a/scripts/audit-isv.sh b/scripts/audit-isv.sh new file mode 100755 index 000000000..a2aefd7a3 --- /dev/null +++ b/scripts/audit-isv.sh @@ -0,0 +1,105 @@ +#!/usr/bin/env bash +# audit-isv.sh — enforce "every numerical constant is ISV-resident" +# per SP20 §0.1. +# +# Walks scripts/audit-manifest/kernels.txt and greps each listed .cu +# for `#define NAME VALUE` lines where VALUE is a numeric literal. +# Flags any define whose NAME is NOT in the structural-dim allowlist +# AND does NOT end in `_INDEX` (ISV slot pointer). +# +# Exit codes: +# 0 — zero violations +# 1 — one or more violations found +# 2 — usage / setup error +# +# Allowlist for structural-dim names (compile-time required for buffer +# layouts / atom support / shared memory): +ALLOWED_NAMES=( + "N_ACTIONS" + "Q_N_ATOMS" + "HIDDEN_DIM" + "REGIME_DIM" + "MAX_UNITS" + "BLOCK_DIM" + "GRID_DIM" + "BLOCK_SIZE" + "WARP_SIZE" + "BLOCK_X" "BLOCK_Y" "BLOCK_Z" + "MAX_WINDOW_ENTRIES" +) + +set -euo pipefail + +ROOT=$(git rev-parse --show-toplevel) +MANIFEST="$ROOT/scripts/audit-manifest/kernels.txt" +CUDA_DIR="$ROOT/crates/ml-alpha/cuda" + +if [[ ! -f "$MANIFEST" ]]; then + echo "ERROR: manifest not found: $MANIFEST" >&2 + exit 2 +fi + +# Read manifest into array, skipping blank lines and comments. +mapfile -t kernels < <(grep -vE '^\s*(#|$)' "$MANIFEST" | awk '{print $1}') + +if [[ ${#kernels[@]} -eq 0 ]]; then + echo "audit-isv: manifest empty — no kernels to audit. PASS." + exit 0 +fi + +# Build a regex of allowed names + `_INDEX` suffix pattern. +allowed_regex="" +for name in "${ALLOWED_NAMES[@]}"; do + allowed_regex="${allowed_regex}|^${name}$" +done +allowed_regex="(${allowed_regex#|}|_INDEX$)" + +violations=0 +total_defines=0 + +echo "audit-isv: auditing ${#kernels[@]} kernel(s) from manifest..." +echo + +for kernel in "${kernels[@]}"; do + cu_file="$CUDA_DIR/${kernel}.cu" + if [[ ! -f "$cu_file" ]]; then + echo "ERROR: kernel listed in manifest but file missing: $cu_file" >&2 + violations=$((violations + 1)) + continue + fi + + # Extract `#define NAME VALUE` where VALUE is a numeric literal + # (integer or float, optional `f` suffix, optional sign). + # Skip macros that contain function-like () bodies — those are + # parameterized templates, not constants. + while IFS= read -r line; do + # Strip leading whitespace + trimmed="${line#"${line%%[![:space:]]*}"}" + # Parse: #define NAME VALUE + if [[ "$trimmed" =~ ^\#define[[:space:]]+([A-Z_][A-Z0-9_]*)[[:space:]]+([+-]?[0-9]+(\.[0-9]+)?f?)$ ]]; then + name="${BASH_REMATCH[1]}" + value="${BASH_REMATCH[2]}" + total_defines=$((total_defines + 1)) + if [[ "$name" =~ $allowed_regex ]]; then + : # allowed + else + echo " VIOLATION $kernel: #define $name $value (should be ISV slot)" + violations=$((violations + 1)) + fi + fi + done < "$cu_file" +done + +echo +echo "audit-isv: scanned $total_defines numeric #defines, found $violations violation(s)." + +if [[ $violations -gt 0 ]]; then + echo + echo "Per SP20 §0.1: every numerical constant in new kernels must be ISV-resident." + echo "Add an RL_*_INDEX slot to crates/ml-alpha/src/rl/isv_slots.rs," + echo "seed via rl_isv_write at trainer init, and read from the slot in the kernel." + exit 1 +fi + +echo "audit-isv: PASS" +exit 0 diff --git a/scripts/audit-manifest/README.md b/scripts/audit-manifest/README.md new file mode 100644 index 000000000..358bb4bc8 --- /dev/null +++ b/scripts/audit-manifest/README.md @@ -0,0 +1,34 @@ +# Audit manifest + +Tracks names of newly-added artifacts so the three audit scripts +(`scripts/audit-isv.sh`, `scripts/audit-wiring.sh`, +`scripts/audit-diag.sh`) can verify ISV-residency, wiring, and +diagnostic exposure on every commit. + +These manifests are SP-agnostic — they hold the cumulative set of +"things introduced by the active development line" that downstream +ship gates audit. SP20 starts the convention; future SPs extend it. + +## Files + +* `kernels.txt` — one new .cu kernel basename per line (e.g. `rl_trail_mutate`) +* `slots.txt` — one new RL_*_INDEX constant name per line (e.g. `RL_TRAIL_ADJUST_RATE_INDEX`) +* `heads.txt` — one new head module name per line (e.g. `frd`) +* `actions.txt` — one new Action enum entry per line (e.g. `HalfFlatLong`) +* `diag-fields.txt` — one new diag JSONL field path per line, jq-syntax + (e.g. `.units.unit_count`, `.trail.fired_count_step`) + +## Workflow per phase commit + +1. Implement the phase (new kernel / slot / head / action / diag field) +2. Append the names to relevant manifest files +3. Run `scripts/audit-isv.sh` — must report 0 violations +4. Run `scripts/audit-wiring.sh` — must report 0 violations +5. Run `scripts/audit-diag.sh` — must report 0 missing fields +6. Commit (the manifest append + the code in the same commit) + +Failures are immediately actionable — no "iterate within scope" +softness. Per SP20 spec §0.4. + +## Lines starting with `#` are comments and ignored by the audit +## scripts. Blank lines also ignored. diff --git a/scripts/audit-manifest/actions.txt b/scripts/audit-manifest/actions.txt new file mode 100644 index 000000000..1b143841e --- /dev/null +++ b/scripts/audit-manifest/actions.txt @@ -0,0 +1,3 @@ +# New Action enum entries added by the active development line. +# Each phase commit appends here. +# Format: one variant name per line (matches src/rl/common.rs::Action enum). diff --git a/scripts/audit-manifest/diag-fields.txt b/scripts/audit-manifest/diag-fields.txt new file mode 100644 index 000000000..cf2fce3dd --- /dev/null +++ b/scripts/audit-manifest/diag-fields.txt @@ -0,0 +1,4 @@ +# New diag JSONL field paths added by the active development line. +# Each phase commit appends here. +# Format: one jq-syntax path per line (e.g. `.trail.fired_count_step`). +# Used by audit-diag.sh against a local-smoke JSONL output. diff --git a/scripts/audit-manifest/heads.txt b/scripts/audit-manifest/heads.txt new file mode 100644 index 000000000..8afb69b5b --- /dev/null +++ b/scripts/audit-manifest/heads.txt @@ -0,0 +1,3 @@ +# New head module names added by the active development line. +# Each phase commit appends here. +# Format: one module name per line (matches crates/ml-alpha/src/heads/.rs). diff --git a/scripts/audit-manifest/kernels.txt b/scripts/audit-manifest/kernels.txt new file mode 100644 index 000000000..42642d7df --- /dev/null +++ b/scripts/audit-manifest/kernels.txt @@ -0,0 +1,3 @@ +# New .cu kernels added by the active development line. +# Each phase commit appends here. +# Format: one basename per line, without `.cu` extension. diff --git a/scripts/audit-manifest/slots.txt b/scripts/audit-manifest/slots.txt new file mode 100644 index 000000000..245fa6568 --- /dev/null +++ b/scripts/audit-manifest/slots.txt @@ -0,0 +1,3 @@ +# New ISV slot constants added by the active development line. +# Each phase commit appends here. +# Format: one RL_*_INDEX symbol per line. diff --git a/scripts/audit-wiring.sh b/scripts/audit-wiring.sh new file mode 100755 index 000000000..67756bf7e --- /dev/null +++ b/scripts/audit-wiring.sh @@ -0,0 +1,187 @@ +#!/usr/bin/env bash +# audit-wiring.sh — enforce "every kernel/slot/head/action fully wired" +# per SP20 §0.2. +# +# Walks all four manifests: +# * kernels.txt → every entry has a registration in build.rs KERNELS +# AND a `.launch_builder(&self._fn)` site +# in src/trainer/integrated.rs +# * slots.txt → every entry has a write site (seed in rl_isv_write +# OR write in a .cu kernel) AND a read site +# (read in a .cu kernel) +# * heads.txt → every entry has src/heads/.rs AND forward +# call AND backward call in the trainer +# * actions.txt → every entry has a branch in +# cuda/actions_to_market_targets.cu OR a handler +# in an override kernel +# +# Exit codes: +# 0 — zero violations +# 1 — one or more violations +# 2 — usage error +# Note: NOT using `set -e` — many grep|head|wc pipelines naturally +# return non-zero (SIGPIPE on head consuming early, grep no-match). +# We handle exit codes explicitly via `|| true` and `|| echo 0` on +# the suspect pipelines, and bookkeep `violations` for the final +# exit code. +set -uo pipefail + +ROOT=$(git rev-parse --show-toplevel) +M="$ROOT/scripts/audit-manifest" +BUILD_RS="$ROOT/crates/ml-alpha/build.rs" +TRAINER="$ROOT/crates/ml-alpha/src/trainer/integrated.rs" +ISV_SLOTS="$ROOT/crates/ml-alpha/src/rl/isv_slots.rs" +COMMON_RS="$ROOT/crates/ml-alpha/src/rl/common.rs" +ACTIONS_KERNEL="$ROOT/crates/ml-alpha/cuda/actions_to_market_targets.cu" +HEADS_DIR="$ROOT/crates/ml-alpha/src/heads" +CUDA_DIR="$ROOT/crates/ml-alpha/cuda" + +violations=0 + +read_manifest() { + local file="$1" + if [[ -f "$file" ]]; then + grep -vE '^\s*(#|$)' "$file" | awk '{print $1}' + fi +} + +# ─── 1. KERNELS ───────────────────────────────────────────── +echo "audit-wiring: KERNELS" +while IFS= read -r kernel; do + [[ -z "$kernel" ]] && continue + cu_file="$CUDA_DIR/${kernel}.cu" + if [[ ! -f "$cu_file" ]]; then + echo " VIOLATION kernel $kernel: source file missing ($cu_file)" + violations=$((violations + 1)) + continue + fi + # build.rs registration + if ! grep -qE "\"${kernel}\"" "$BUILD_RS"; then + echo " VIOLATION kernel $kernel: not registered in build.rs KERNELS" + violations=$((violations + 1)) + fi + # trainer field + if ! grep -qE "${kernel}_fn[: ,]" "$TRAINER"; then + echo " VIOLATION kernel $kernel: no _fn field in trainer" + violations=$((violations + 1)) + fi + # trainer launch site + if ! grep -qE "launch_builder\(&self\.${kernel}_fn\)" "$TRAINER"; then + echo " VIOLATION kernel $kernel: no launch site in trainer" + violations=$((violations + 1)) + fi +done < <(read_manifest "$M/kernels.txt") + +# ─── 2. SLOTS ─────────────────────────────────────────────── +echo "audit-wiring: SLOTS" +while IFS= read -r slot; do + [[ -z "$slot" ]] && continue + # slot defined in isv_slots.rs? + if ! grep -qE "pub const ${slot}\s*:\s*usize" "$ISV_SLOTS"; then + echo " VIOLATION slot $slot: no `pub const` definition in isv_slots.rs" + violations=$((violations + 1)) + continue + fi + # Extract numeric slot index for cross-kernel grep + slot_idx=$(awk -v name="$slot" ' + $0 ~ "pub const " name "[[:space:]]*:[[:space:]]*usize" { + if (match($0, /=[[:space:]]*[0-9]+/)) { + v = substr($0, RSTART, RLENGTH) + gsub(/[^0-9]/, "", v) + print v; exit + } + }' "$ISV_SLOTS") + [[ -z "$slot_idx" ]] && slot_idx="-1" + # producer: seeded in rl_isv_write list OR written by a .cu kernel + producer_seed=0 + producer_kernel=0 + if grep -qE "isv_slots::${slot}" "$TRAINER"; then + producer_seed=1 + fi + # kernel writes via isv[INDEX] = ... or isv[] = ... + if grep -qrE "isv\[(${slot}|${slot_idx})\][[:space:]]*=" "$CUDA_DIR" 2>/dev/null; then + producer_kernel=1 + fi + if [[ $producer_seed -eq 0 && $producer_kernel -eq 0 ]]; then + echo " VIOLATION slot $slot (idx $slot_idx): no producer (not seeded in rl_isv_write, not written by any kernel)" + violations=$((violations + 1)) + fi + # consumer: read by some .cu (isv[INDEX] in non-assignment context) + # Heuristic: count distinct .cu files containing the slot reference + # in a context that's NOT immediately followed by `=` (assignment). + # Pattern `isv[X][^=]` matches isv[X] where next char isn't `=`. + read_only_files=$(grep -lrE "isv\[(${slot}|${slot_idx})\][^=]" "$CUDA_DIR" 2>/dev/null | wc -l) + if [[ $read_only_files -eq 0 ]]; then + echo " VIOLATION slot $slot (idx $slot_idx): no consumer (no kernel reads it in non-assignment context)" + violations=$((violations + 1)) + fi +done < <(read_manifest "$M/slots.txt") + +# ─── 3. HEADS ─────────────────────────────────────────────── +echo "audit-wiring: HEADS" +while IFS= read -r head; do + [[ -z "$head" ]] && continue + head_rs="$HEADS_DIR/${head}.rs" + if [[ ! -f "$head_rs" ]]; then + echo " VIOLATION head $head: module file missing ($head_rs)" + violations=$((violations + 1)) + continue + fi + if ! grep -qE "fn forward" "$head_rs"; then + echo " VIOLATION head $head: no fn forward in module" + violations=$((violations + 1)) + fi + if ! grep -qE "fn backward" "$head_rs"; then + echo " VIOLATION head $head: no fn backward in module" + violations=$((violations + 1)) + fi + # Adam step + LR controller — grep for the head's forward call site + # in the trainer (loose heuristic). + if ! grep -qE "${head}_head" "$TRAINER"; then + echo " VIOLATION head $head: no ${head}_head field or call in trainer" + violations=$((violations + 1)) + fi +done < <(read_manifest "$M/heads.txt") + +# ─── 4. ACTIONS ───────────────────────────────────────────── +echo "audit-wiring: ACTIONS" +while IFS= read -r action; do + [[ -z "$action" ]] && continue + # Action enum entry + if ! grep -qE "${action}\s*=\s*[0-9]+" "$COMMON_RS"; then + echo " VIOLATION action $action: not in Action enum in common.rs" + violations=$((violations + 1)) + continue + fi + # Get action index + action_idx=$(awk -v name="$action" ' + $0 ~ name "[[:space:]]*=[[:space:]]*[0-9]+" { + if (match($0, /=[[:space:]]*[0-9]+/)) { + v = substr($0, RSTART, RLENGTH) + gsub(/[^0-9]/, "", v) + print v; exit + } + }' "$COMMON_RS") + [[ -z "$action_idx" ]] && action_idx="-1" + # Handler in actions_to_market_targets.cu: branch `action == ` + if ! grep -qE "action[[:space:]]*==[[:space:]]*${action_idx}\b" "$ACTIONS_KERNEL"; then + # Could be handled in override kernel — check across all .cu files + handler_count=$(grep -lrE "(action|actions\[[^]]*\])[[:space:]]*==[[:space:]]*${action_idx}\b" "$CUDA_DIR" 2>/dev/null | wc -l) + if [[ "$handler_count" -eq 0 ]]; then + echo " VIOLATION action $action (idx $action_idx): no handler in actions_to_market_targets or override kernels" + violations=$((violations + 1)) + fi + fi +done < <(read_manifest "$M/actions.txt") + +echo +echo "audit-wiring: $violations violation(s)" + +if [[ $violations -gt 0 ]]; then + echo + echo "Per SP20 §0.2: every kernel/slot/head/action MUST have producer + consumer in same commit." + exit 1 +fi + +echo "audit-wiring: PASS" +exit 0