docs(sp21): T2.2 Phase 1.5 + Phase 2 continuation plan + amendment
Adds the continuation plan for SP21 T2.2 Phase 1.5 (entry_q tracking
in portfolio_state slot 6) + Phase 2 (wire enrichment to real
per-trade tape from gpu_backtest_evaluator). Self-contained plan
that a fresh session can pick up cold:
- State-at-session-start summary with all 8 prior commits
- Phase 1.5 design (storage slot, capture site, plumbing,
EvalTrade extension)
- Phase 2 design (training_loop wire-up, enrichment refactor,
E5 alternate-signal options with recommendation)
- Hard rules carried from prior session (no NULLs, no stubs,
atomic per feedback_no_partial_refactor)
- Verification plan (5 GPU oracle test suites)
- Open design question (E5 alternate signal) with recommendation
- Multi-phase continuation map (Phases 3-7 + Phase 8)
- Final cascade verification (smoke run criteria)
Also amends the parent SP21 plan
(2026-05-10-sp21-train-eval-coherence-isv-defrost.md) with the
T2.2 multi-phase scope section that documents the full per-trade-tape
expansion the user committed to (option 3 — full wiring across
multiple sessions instead of the original recommended option (b)
aggregate-stats refactor).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -111,13 +111,29 @@ This principle is the SP21 unifying invariant. Per `feedback_adaptive_not_tuned.
|
||||
| # | Finding | Site | Decision needed |
|
||||
|---|---|---|---|
|
||||
| T2.1 | `check_early_stopping(avg_q_value)` — separate stop path on Q-value magnitude | `crates/ml/src/trainers/dqn/trainer/metrics.rs:436` | Q-value magnitude is not a learning signal — high Q can mean high alpha OR catastrophic value-explosion. Three options: (a) delete entirely (subsumed by val-loss path); (b) repurpose as Q-explosion guard with **signal-driven** threshold from `ISV[Q_VALUE_VAR_EMA]` per `feedback_isv_for_adaptive_bounds`; (c) keep but rename so its purpose is clear. Recommendation: **(a) delete** — `feedback_no_legacy_aliases` says no deprecated wrappers. If we keep it as a Q-explosion guard, it MUST be signal-driven (no hardcoded threshold). |
|
||||
| T2.2 | `extract_eval_trades_from_metrics` fabricates per-trade tape from aggregate stats; downstream enrichment functions are themselves constants-soup (T1.3) | `crates/ml/src/trainers/dqn/trainer/enrichment.rs:99-138` (synth) + entire 381-line file | Function is mathematically a fiction — it generates synthetic trades that "average to" the input aggregates. Each downstream enrichment function has multiple hardcoded thresholds. | Two options: (a) emit real per-trade tape from `gpu_evaluator` (significant new code) AND signal-drive each downstream threshold; (b) refactor enrichment functions to consume aggregate stats directly without the fake-trade detour AND signal-drive each downstream threshold (this is the bigger half — consequential refactor since each enrichment fn must read from ISV). Recommendation: **(b)**. Scope: T2.2 + T1.3 are mathematically the same superset of changes — enrichment.rs gets rewritten as a set of ISV-aware functions. |
|
||||
| T2.2 | `extract_eval_trades_from_metrics` fabricates per-trade tape from aggregate stats; downstream enrichment functions are themselves constants-soup (T1.3); 5 of 8 enrichment outputs (E1/E4/E6/E7/E8) computed-and-discarded since SP3+ | `crates/ml/src/trainers/dqn/trainer/enrichment.rs:99-138` (synth) + entire 381-line file | Function is mathematically a fiction — generates synthetic trades that "average to" the input aggregates. 3 of 8 outputs consumed (E2/E3/E5); 5 of 8 (E1/E4/E6/E7/E8) require per-trade data that never made it past the fake-trade synthesizer. | **DECISION 2026-05-10 (user-directed)**: option (a) — full per-trade tape emission + wire all 8 enrichments to real consumers. Multi-session scope (~5-6 days). See **T2.2 multi-phase scope** below. |
|
||||
| T2.3 | `EarlyStopping::should_stop` test signature changes (T1.1b) | `crates/ml/src/trainers/dqn/early_stopping.rs:158, 183, 200, 216, 236, 250` (6 unit tests) | Refactoring `should_stop(val_loss, epoch)` → `should_stop(val_loss, min_delta, epoch)` breaks the test signatures. | Update all 6 tests to pass explicit `min_delta` per call. Per `feedback_no_partial_refactor`, this is part of the atomic T1.1b commit, not a separate task. |
|
||||
| T2.4 | `MIN_HOLD_TARGET=30.0f` zombie #define after Class A audit | `state_layout.cuh:317` | Comment at lines 177, 296-322 explicitly says this constant was REPLACED by ISV slot 460 (`MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX`). But the #define still exists. Either: (a) zombie code that should be deleted; (b) used as fallback when ISV at sentinel — comment line 320 hints at fallback usage. Audit needed. | Trace usages of `MIN_HOLD_TARGET` macro. If only used via the fallback path, rename to `MIN_HOLD_TARGET_FALLBACK` and document. If unused, DELETE per `feedback_no_legacy_aliases`. |
|
||||
| T2.5 | PER `per_alpha=0.6` and `per_beta_start=0.6` hardcoded | `config.rs:1479-1480` | PER (Prioritized Experience Replay) parameters control training-data sampling distribution. The original Schaul et al. paper uses these as fixed but other works adapt them. With `feedback_always_per` mandating PER ON, the *parameters* are still fixed. | Two options: (a) keep fixed as paper-canonical (but then the hyperparams should be derived from a research citation, not a magic number); (b) adapt from priority distribution variance EMA. Recommendation: **keep fixed for SP21**, but file the question for a separate SP if PER tuning is later identified as a leverage point. Document choice in `pearl_per_alpha_beta_paper_canonical`. |
|
||||
| ~~T2.6~~ | ~~Curiosity trainer beta=0.0 hardcoded — silently disabled?~~ | ~~`gpu_curiosity_trainer.rs:342, 401`~~ | **WITHDRAWN 2026-05-10 (false alarm)**: the `beta=0.0` is the cuBLAS `cublasSgemm_v2` β API parameter (`C = α·A·B + β·C`), not a curiosity reward weight. Actual curiosity coefficient is `ISV[CURIOSITY_PRESSURE_INDEX=346]`, produced by SP11 Fix 39's `reward_subsystem_controller_kernel`, already signal-driven per `pearl_blend_formulas_must_have_permanent_floor`. Confirmed firing in xmd6b logs: `curiosity_p: 0.000 → 0.109 → 0.406` across epochs 0-2 (Pearl-A bootstrap from sentinel, then live). | **No fix needed.** System working as designed. |
|
||||
|
||||
**Success criterion** (T2.1): file no longer contains `check_early_stopping(avg_q_value, ...)`; tests that exercised it migrated or deleted. (T2.2): `extract_eval_trades_from_metrics` removed; enrichment functions take `&WindowMetrics` directly; behavior preserved for E2/gamma/ensemble code paths.
|
||||
**Success criterion** (T2.1): file no longer contains `check_early_stopping(avg_q_value, ...)`; tests that exercised it migrated or deleted. (T2.2): `extract_eval_trades_from_metrics` removed; enrichment functions consume real per-trade tape from `gpu_backtest_evaluator`; all 8 outputs wired to real consumers; all thresholds signal-driven from ISV.
|
||||
|
||||
#### T2.2 multi-phase scope (multi-session)
|
||||
|
||||
Phase decomposition for the per-trade-tape-driven enrichment refactor:
|
||||
|
||||
- **Phase 1**: Per-trade tape emission from `gpu_backtest_evaluator`. New buffers `[MAX_TRADES_PER_WINDOW]` × 7 fields (pnl, predicted_q at entry, direction, magnitude, holding_bars, ensemble_var at entry, bar_index). New CUDA kernel writes at trade-close events. Host readback path. Mapped-pinned for streaming. Sentinel/cold-start handling.
|
||||
- **Phase 2**: Replace `extract_eval_trades_from_metrics` with real tape consumer. Drop the fabrication.
|
||||
- **Phase 3**: Wire E1 (q_correction). New ISV slot `Q_BIAS_CORRECTION_INDEX`. Apply as additive Q-target offset at C51 loss + Bellman target.
|
||||
- **Phase 4**: Wire E4 (per-branch LR scaling). Apply via existing per-group Adam LR infrastructure (`pearl_per_group_adam_hyperparams`).
|
||||
- **Phase 5**: Wire E6 (winner indices) → PER priority bumps. Use existing PER infra to mark indices high-priority.
|
||||
- **Phase 6**: Wire E7 (hindsight) → replay buffer synthetic experience injection.
|
||||
- **Phase 7**: Wire E8 (curriculum weights) → segment sampling weights at episode start.
|
||||
- **Phase 8**: Signal-drive ALL enrichment thresholds (replace hardcoded `2.0`, `0.5`, `1.5×`, etc. with ISV-driven from `val_sharpe_var_ema`, ensemble disagreement EMA, etc.).
|
||||
- **Phase 9**: GPU oracle tests for each new code path.
|
||||
|
||||
Each phase is its own atomic commit (per `feedback_no_partial_refactor` — phase-internal contract migrations are atomic; cross-phase wiring is sequential). Phases 1-2 are mandatory together (can't have consumer without producer); Phases 3-7 can land independently in any order after Phase 2.
|
||||
|
||||
**Pearl compliance**: T2.1 honors `feedback_no_legacy_aliases`. T2.2 honors `feedback_no_stubs` (fake-trade synthesis is a stub-shaped pattern — produces shaped output without real signal).
|
||||
|
||||
|
||||
@@ -0,0 +1,397 @@
|
||||
# SP21 T2.2 Phase 1.5 + Phase 2 — Enrichment real-tape wireup
|
||||
|
||||
**Author**: jgrusewski + assistant pair
|
||||
**Date**: 2026-05-11 (continuation of 2026-05-10 SP21 work)
|
||||
**Branch**: `feat/sp18-combined` (off worktree branch `sp20-aux-h-fixed`)
|
||||
**Predecessor plan**: `docs/plans/2026-05-10-sp21-train-eval-coherence-isv-defrost.md`
|
||||
|
||||
## State at session start
|
||||
|
||||
8 SP21 commits landed in the prior session:
|
||||
|
||||
```
|
||||
6df44e4c6 docs(sp21): plan + revert sp20-aux-h-fixed experiment
|
||||
1790a31b6 feat(sp21): T3.1+T3.2 ISV defrost — wr_ema + hold_pct_ema
|
||||
4d4cd996d feat(sp21): T3.3 ISV defrost — hold_reward_ema
|
||||
74c7a8011 feat(sp21): T1.1a+T1.1b+T2.3 — signal-driven early-stopping
|
||||
c90553953 feat(sp21): T1.2+T1.4 — enrichment real metrics + backtracking signal-driven
|
||||
4ab1c132e feat(sp21): T2.1+T2.4 — Q-value early-stop + MIN_HOLD zombies deleted
|
||||
5d01190e1 feat(sp21): T2.2 Phase 1 Step A — per-trade tape kernel ABI
|
||||
7d538d930 feat(sp21): T2.2 Phase 1 Step B — per-trade tape buffers + readback
|
||||
```
|
||||
|
||||
Audit-doc detail: `docs/dqn-wire-up-audit.md` (all 2026-05-10 entries).
|
||||
|
||||
### What's done
|
||||
|
||||
- **Tier 1 closed**: val-loss-driven early-stopping with signal-driven
|
||||
`min_delta` from `ISV[VAL_SHARPE_VAR_EMA_INDEX=351]`.
|
||||
- **Tier 3 closed**: wr_ema/hold_pct_ema/hold_reward_ema all signal-driven
|
||||
(binary-majority → fractional refactor); 25/25 GPU oracle tests pass.
|
||||
- **Tier 2 partial**: T2.1 (Q-value early-stop deleted), T2.4 (MIN_HOLD
|
||||
zombies removed), T2.5 (PER hyperparams documented as deferred).
|
||||
- **T2.2 Phase 1**: per-trade tape infrastructure — both backtest kernels
|
||||
emit per-trade records at close events (5 fields: bar_index, pnl,
|
||||
holding_bars, direction, magnitude). `GpuBacktestEvaluator` owns the
|
||||
SoA buffers + `pub fn read_per_trade_tape() -> Vec<EvalTrade>`.
|
||||
|
||||
### What's pending
|
||||
|
||||
T2.2 **Phase 1.5** (entry_q capture) + **Phase 2** (wire enrichment to
|
||||
real tape) — atomic single commit.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1.5 — entry_q tracking
|
||||
|
||||
Captures predicted Q at trade open so the per-trade tape carries the
|
||||
6th field needed by E1 (q_correction = `predicted_q − pnl` bias).
|
||||
|
||||
### Storage
|
||||
|
||||
`portfolio_state[ps + 6]` → `entry_q`. Slot 6 is currently unused
|
||||
(zeroed at init, see `backtest_env_kernel.cu:57`). **Verify** before
|
||||
coding that `backtest_state_gather` doesn't already write to slot 6.
|
||||
Using slot 6 means **no `PORTFOLIO_STATE_SIZE` bump** — gather/forward
|
||||
kernels untouched.
|
||||
|
||||
### Capture site
|
||||
|
||||
In `backtest_env_step` and `backtest_env_step_batch`, AFTER the
|
||||
`unified_env_step_core` call returns, detect "new trade opened":
|
||||
|
||||
```cuda
|
||||
const int prev_pos_active = (fabsf(prev_position) > 0.001f);
|
||||
const int now_pos_active = (fabsf(position) > 0.001f);
|
||||
const int is_open = !prev_pos_active && now_pos_active;
|
||||
const int is_reverse = (prev_position * position < 0.0f);
|
||||
|
||||
if (is_open || is_reverse) {
|
||||
/* Read q_values_per_window[w * num_actions + actions[w]] and
|
||||
* write to portfolio_state[ps + 6]. New entry_q replaces the
|
||||
* snapshot from before unified_env_step_core (which closed the
|
||||
* old trade). */
|
||||
const float entry_q_new = q_values_per_window[
|
||||
(long long)w * num_actions + actions[w]];
|
||||
portfolio_state[ps + 6] = entry_q_new; /* via shmem_pf */
|
||||
}
|
||||
```
|
||||
|
||||
### Plumbing
|
||||
|
||||
New kernel arg on both backtest kernels:
|
||||
|
||||
```cuda
|
||||
const float* __restrict__ q_values_per_window /* [n_windows * num_actions] */
|
||||
int num_actions /* size of action space */
|
||||
```
|
||||
|
||||
Pass real device pointer from launcher. The Q-values are computed by
|
||||
`experience_action_select` immediately before env_step. In the chunked
|
||||
batch path they're in `chunked_q_values`; in the single-step path
|
||||
`q_values_buf` (or whichever buffer the action_select kernel writes).
|
||||
**Locate the actual buffer name** — grep for `expected_q` or
|
||||
`backtest_action_select` in `gpu_backtest_evaluator.rs`.
|
||||
|
||||
### Per-trade emission
|
||||
|
||||
Snapshot `pre_entry_q` BEFORE the call (line ~265 of
|
||||
`backtest_env_kernel.cu` after the existing `pre_entry_price` snapshot):
|
||||
|
||||
```cuda
|
||||
const float pre_entry_price = entry_price;
|
||||
const int pre_hold_bars = (int)hold_time;
|
||||
const float pre_entry_q = portfolio_state[ps + 6]; /* via shmem_pf[local_tid * STATE_SIZE + 6] */
|
||||
```
|
||||
|
||||
In the existing close-detection block, write the new buffer:
|
||||
|
||||
```cuda
|
||||
per_trade_predicted_q_out[idx] = pre_entry_q;
|
||||
```
|
||||
|
||||
### Buffer additions
|
||||
|
||||
`GpuBacktestEvaluator` gains a 6th SoA buffer:
|
||||
|
||||
```rust
|
||||
per_trade_predicted_q_buf: CudaSlice<f32>, // [n_windows × MAX_TRADES_PER_WINDOW]
|
||||
```
|
||||
|
||||
Allocated alongside the existing 5 buffers. Constructor zero-init.
|
||||
Reset at fold boundary (already handled by reset_evaluation_state if
|
||||
we just zero the count; predicted_q values past `count[w]` are never
|
||||
read host-side).
|
||||
|
||||
### EvalTrade struct extension
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct EvalTrade {
|
||||
pub bar_index: u32,
|
||||
pub pnl: f32,
|
||||
pub predicted_q: f32, // SP21 Phase 1.5 (2026-05-11): entry_q at trade open
|
||||
pub holding_bars: u32,
|
||||
pub direction: u8,
|
||||
pub magnitude: u8,
|
||||
}
|
||||
```
|
||||
|
||||
`read_per_trade_tape` reads the 6th SoA buffer, populates `predicted_q`
|
||||
in each `EvalTrade`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — wire enrichment to real tape
|
||||
|
||||
### training_loop.rs changes
|
||||
|
||||
`crates/ml/src/trainers/dqn/trainer/training_loop.rs:1510-1568` —
|
||||
replace the fake-trade synthesizer call with `read_per_trade_tape()`:
|
||||
|
||||
```rust
|
||||
// BEFORE (fake synthesizer):
|
||||
let eval_trades = super::enrichment::extract_eval_trades_from_metrics(
|
||||
val_sharpe_for_backtrack as f32,
|
||||
real_trade_count, real_total_pnl, real_win_rate,
|
||||
log_output.avg_q_value as f32,
|
||||
val_bars,
|
||||
);
|
||||
|
||||
// AFTER (real per-trade tape):
|
||||
let eval_trades = match self.gpu_evaluator.as_ref() {
|
||||
Some(ev) => ev.read_per_trade_tape().unwrap_or_default(),
|
||||
None => Vec::new(),
|
||||
};
|
||||
```
|
||||
|
||||
Cold-start (no evaluator yet): empty Vec → enrichment runs with no
|
||||
trades → returns defaults (current epsilon, gamma, agreement_threshold
|
||||
unchanged). Honest "no signal yet" — matches the pattern already used
|
||||
in T1.2 for `last_val_metrics: None`.
|
||||
|
||||
### enrichment.rs changes
|
||||
|
||||
1. **Drop `ensemble_var` field** from `EvalTrade`. The val backtest uses
|
||||
a single deterministic forward — no ensemble, no variance to capture.
|
||||
Refactor E5 (see below).
|
||||
|
||||
2. **Make `EvalTrade` a re-export** of
|
||||
`cuda_pipeline::gpu_backtest_evaluator::EvalTrade` (single source of
|
||||
truth):
|
||||
|
||||
```rust
|
||||
pub(crate) use crate::cuda_pipeline::gpu_backtest_evaluator::EvalTrade;
|
||||
```
|
||||
|
||||
Or duplicate-with-conversion if module visibility prevents re-export.
|
||||
|
||||
3. **Refactor E5 (`compute_agreement_threshold`)** to use trade-pnl
|
||||
quartile-spread Sharpe instead of `ensemble_var`. Concrete formula:
|
||||
|
||||
```rust
|
||||
/// E5 (SP21 Phase 2, 2026-05-11): agreement threshold from quartile-
|
||||
/// spread Sharpe. High spread between top-25% pnl Sharpe and bottom-
|
||||
/// 25% pnl Sharpe = high "trades when policy is right" vs "trades
|
||||
/// when wrong" separation = lower agreement threshold (trust agreement
|
||||
/// more). No ensemble_var needed — fully self-contained on per-trade
|
||||
/// pnl tape from gpu_backtest_evaluator.
|
||||
fn compute_agreement_threshold(
|
||||
trades: &[EvalTrade],
|
||||
current: f32,
|
||||
) -> f32 {
|
||||
if trades.len() < 4 { return current; }
|
||||
let mut pnls: Vec<f32> = trades.iter().map(|t| t.pnl).collect();
|
||||
pnls.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
|
||||
let q1_end = pnls.len() / 4;
|
||||
let q4_start = 3 * pnls.len() / 4;
|
||||
let q1 = &pnls[..q1_end];
|
||||
let q4 = &pnls[q4_start..];
|
||||
let sharpe_of = |slice: &[f32]| -> f32 {
|
||||
let mean = slice.iter().sum::<f32>() / slice.len() as f32;
|
||||
let var = slice.iter().map(|x| (x - mean).powi(2)).sum::<f32>()
|
||||
/ slice.len() as f32;
|
||||
if var > 1e-12 { mean / var.sqrt() } else { 0.0 }
|
||||
};
|
||||
let q1_sharpe = sharpe_of(q1);
|
||||
let q4_sharpe = sharpe_of(q4);
|
||||
let spread = q4_sharpe - q1_sharpe;
|
||||
/* Larger spread → tighten threshold (× 0.9). Smaller spread →
|
||||
* loosen (× 1.1). Spread > 1.5σ qualifies as "tight". */
|
||||
let new = if spread > 1.5 { current * 0.9 }
|
||||
else if spread < 0.5 { current * 1.1 }
|
||||
else { current };
|
||||
new.clamp(0.01, 10.0)
|
||||
}
|
||||
```
|
||||
|
||||
Note: hardcoded `1.5`/`0.5` thresholds here — flag for follow-up
|
||||
signal-drive (Phase 8 of T2.2 multi-phase scope).
|
||||
|
||||
4. **Delete `extract_eval_trades_from_metrics`** function entirely
|
||||
(the fake-trade synthesizer). Per `feedback_no_legacy_aliases`.
|
||||
|
||||
5. **Delete `HindsightExperience` struct** if unused after refactor.
|
||||
Check E7 (`compute_hindsight_labels`) — keeps if E7 still emits.
|
||||
|
||||
### Verify all 8 enrichment functions consume real per-trade data
|
||||
|
||||
| Fn | Uses | After Phase 1.5+2 |
|
||||
|---|---|---|
|
||||
| E1 `compute_q_correction` | predicted_q − pnl | ✓ real (Phase 1.5 supplies predicted_q) |
|
||||
| E2 `compute_adaptive_epsilon` | val_sharpe (aggregate) | ✓ real (already aggregate-only) |
|
||||
| E3 `compute_dynamic_gamma` | holding_bars, pnl | ✓ real |
|
||||
| E4 `compute_branch_lr_scale` | direction, magnitude, pnl, holding_bars | ✓ real |
|
||||
| E5 `compute_agreement_threshold` | (refactored) pnl quartile spread | ✓ real |
|
||||
| E6 `compute_winner_indices` | pnl, bar_index | ✓ real |
|
||||
| E7 `compute_hindsight_labels` | direction, pnl, bar_index | ✓ real |
|
||||
| E8 `compute_curriculum_weights` | pnl | ✓ real |
|
||||
|
||||
All 8 land on real data after this commit.
|
||||
|
||||
---
|
||||
|
||||
## Hard rules (carried from prior session)
|
||||
|
||||
- **NO NULL launcher passes.** Real buffers from the start.
|
||||
- **NO `// TODO` markers** (per `feedback_no_todo_fixme`).
|
||||
- **NO stub return values** / placeholder zeros for fields meant to
|
||||
carry signal. Refactor or delete the consumer if computation
|
||||
isn't honest.
|
||||
- **Atomic commits** per `feedback_no_partial_refactor`: contract
|
||||
changes ship with all consumers migrated together.
|
||||
- **If a kernel signature changes, both call sites migrate in the
|
||||
same commit.**
|
||||
- **Pre-commit hook runs `cargo check` + GPU hot-path leak check**;
|
||||
commits must pass both. (Note: the hook complains about pre-existing
|
||||
`unwrap()` and the line-78 `return 0.0` in training_loop.rs — those
|
||||
are pre-existing warnings, not introduced.)
|
||||
- **NEVER use `--no-verify`.** Per CLAUDE.md.
|
||||
|
||||
---
|
||||
|
||||
## Verification plan
|
||||
|
||||
Before committing:
|
||||
|
||||
1. `SQLX_OFFLINE=true cargo check -p ml --tests` — must pass clean.
|
||||
2. `SQLX_OFFLINE=true cargo test -p ml --test sp20_aggregate_inputs_test
|
||||
--features cuda -- --ignored --nocapture` — should still be 12/12
|
||||
(aggregator unaffected by these changes).
|
||||
3. `SQLX_OFFLINE=true cargo test -p ml --test sp20_phase1_4_wireup_test
|
||||
--features cuda -- --ignored --nocapture` — should still be 2/2.
|
||||
4. `SQLX_OFFLINE=true cargo test -p ml --test sp20_emas_compute_test
|
||||
--features cuda -- --ignored --nocapture` — should still be 4/4.
|
||||
5. `SQLX_OFFLINE=true cargo test -p ml --test sp20_controllers_compute_test
|
||||
--features cuda -- --ignored --nocapture` — should still be 7/7.
|
||||
|
||||
Optional (skip if blocking; document as Phase 2.5 follow-up):
|
||||
|
||||
6. NEW smoke test exercising `GpuBacktestEvaluator`'s per-trade tape
|
||||
end-to-end. Synthetic eval window → run kernel → read tape →
|
||||
assert non-empty `Vec<EvalTrade>` with valid `predicted_q` values.
|
||||
Test scaffolding for the eval kernel is heavy; the existing
|
||||
`sp20_aggregate_inputs` GPU oracle tests have a similar shape and
|
||||
can be adapted as a reference. Place in
|
||||
`crates/ml/tests/gpu_backtest_per_trade_tape.rs`.
|
||||
|
||||
---
|
||||
|
||||
## Files affected this session
|
||||
|
||||
- `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu`
|
||||
- Both kernels: new `q_values_per_window` + `num_actions` args.
|
||||
- Snapshot `pre_entry_q` BEFORE `unified_env_step_core` call.
|
||||
- On close: emit `per_trade_predicted_q_out[idx] = pre_entry_q`.
|
||||
- On open / reverse: write
|
||||
`portfolio_state[ps + 6] = q_values_per_window[w * num_actions +
|
||||
actions[w]]` (via shmem path).
|
||||
|
||||
- `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs`
|
||||
- New struct field `per_trade_predicted_q_buf: CudaSlice<f32>`.
|
||||
- Constructor: alloc 6th buffer.
|
||||
- Both launcher sites: pass q_values device pointer + num_actions.
|
||||
- `EvalTrade`: add `pub predicted_q: f32` field.
|
||||
- `read_per_trade_tape`: read 6th buffer + populate `predicted_q`.
|
||||
|
||||
- `crates/ml/src/trainers/dqn/trainer/enrichment.rs`
|
||||
- `EvalTrade` re-exports / matches gpu_backtest_evaluator's struct
|
||||
(drop `ensemble_var`).
|
||||
- `compute_agreement_threshold`: refactor to quartile-spread Sharpe.
|
||||
- Delete `extract_eval_trades_from_metrics`.
|
||||
- Delete `HindsightExperience` if unused.
|
||||
|
||||
- `crates/ml/src/trainers/dqn/trainer/training_loop.rs:1510-1568`
|
||||
- Replace `extract_eval_trades_from_metrics(...)` call with
|
||||
`evaluator.read_per_trade_tape()?`.
|
||||
- Drop the `extract_*`-related plumbing (`val_bars`,
|
||||
`real_trade_count` etc.) since real tape supplies these directly.
|
||||
|
||||
- `docs/dqn-wire-up-audit.md`
|
||||
- New section dated 2026-05-11 documenting Phase 1.5 + Phase 2
|
||||
closure. Per Invariant 7, every CUDA-pipeline change requires
|
||||
audit-doc update — pre-commit hook enforces this.
|
||||
|
||||
---
|
||||
|
||||
## Open question to resolve before coding
|
||||
|
||||
**E5 alternate signal — quartile-spread vs ISV ensemble disagreement?**
|
||||
|
||||
(a) **Quartile-spread Sharpe** (recommended): top-25% pnl Sharpe minus
|
||||
bottom-25% pnl Sharpe. Fully self-contained on per-trade pnl tape.
|
||||
No new infrastructure, no cross-coupling val/training data.
|
||||
|
||||
(b) **ISV ensemble disagreement EMA**: read from training-side ISV slot
|
||||
(if exposed to val read path). Requires checking per-epoch ordering
|
||||
(does training set ensemble_var EMA before val runs?). Adds val ↔
|
||||
training coupling.
|
||||
|
||||
**Recommend (a)** unless the user has a specific reason to reach
|
||||
across to training-side ensemble state. (a) is also philosophically
|
||||
cleaner: enrichment runs against val data, so val should be its
|
||||
self-contained signal source.
|
||||
|
||||
---
|
||||
|
||||
## After this commit lands
|
||||
|
||||
T2.2 multi-phase scope continues with Phases 3-7 + Phase 8:
|
||||
|
||||
- Phase 3: Wire E1 (q_correction) → ISV slot consumer (apply as
|
||||
additive Q-target offset at C51 loss + Bellman target).
|
||||
- Phase 4: Wire E4 (per-branch LR scaling) via existing per-group
|
||||
Adam infrastructure (`pearl_per_group_adam_hyperparams`).
|
||||
- Phase 5: Wire E6 (winner indices) → PER priority bumps.
|
||||
- Phase 6: Wire E7 (hindsight) → replay buffer synthetic experience
|
||||
injection.
|
||||
- Phase 7: Wire E8 (curriculum weights) → segment sampling weights.
|
||||
- Phase 8: Signal-drive remaining hardcoded thresholds in enrichment
|
||||
(the `1.5` / `0.5` constants in the new E5 quartile-spread, the
|
||||
`2.0` / `0.5` / `-0.5` in E2 epsilon adapter, etc.) using
|
||||
appropriate ISV variance EMAs.
|
||||
|
||||
Each is its own atomic commit (per `feedback_no_partial_refactor` —
|
||||
phase-internal contract migrations are atomic; cross-phase wiring is
|
||||
sequential).
|
||||
|
||||
---
|
||||
|
||||
## After all phases land
|
||||
|
||||
Dispatch a smoke training run on the L40S to validate the full SP21
|
||||
cascade end-to-end. Watch:
|
||||
|
||||
- HEALTH_DIAG `sp20_isv` line shows variance > 0 across epochs for
|
||||
wr_ema, hold_pct_ema, hold_reward_ema (Tier 3 verification).
|
||||
- `🛑 Early stopping triggered` fires within `patience` epochs of
|
||||
val_Sharpe peak (Tier 1 verification).
|
||||
- Enrichment receives real `Vec<EvalTrade>` instead of fake aggregates
|
||||
(T2.2 verification).
|
||||
- HEALTH_DIAG controller readouts (`c51_budget`, `cql_budget`,
|
||||
`q_var`, `iqn_budget`, `lb_active`) show inter-branch variation
|
||||
(Tier 4 cascade verification).
|
||||
- `evaluate_baseline` (or modern eval entry point) reproduces the
|
||||
val_Sharpe / val_PF observed in training (within 5% relative
|
||||
tolerance) on the same checkpoint + window (success criterion #6
|
||||
from the SP21 plan).
|
||||
Reference in New Issue
Block a user