diff --git a/docs/superpowers/specs/2026-04-19-ofi-momentum-dense-reward-design.md b/docs/superpowers/specs/2026-04-19-ofi-momentum-dense-reward-design.md index 7d82c9f54..3093eaad3 100644 --- a/docs/superpowers/specs/2026-04-19-ofi-momentum-dense-reward-design.md +++ b/docs/superpowers/specs/2026-04-19-ofi-momentum-dense-reward-design.md @@ -52,7 +52,18 @@ concat_ofi_features reads batch_states[b * SD + 66] for order branch (was 42) **Location:** `experience_kernels.cu` env_step_batch, replacing lines 1715-1721. -**Formula (revised — no open_price, uses close-to-close):** +**Targets buffer expansion (target_dim 4→6):** +``` +Current: tgt[0]=preproc_close, tgt[1]=preproc_next, tgt[2]=raw_close, tgt[3]=raw_next +New: tgt[0]=preproc_close, tgt[1]=preproc_next, tgt[2]=raw_close, tgt[3]=raw_next, + tgt[4]=raw_open, tgt[5]=mid_price_open +``` +- `raw_open`: OHLCV open price (already parsed in data_loading.rs:650) +- `mid_price_open`: MBP-10 best_bid/best_ask midpoint at bar formation — the true entry price + +Requires fxcache recomputation (one-time Argo job: `precompute_features` binary). + +**Formula:** ```c if (!segment_complete && fabsf(position) > 0.001f && micro_reward_scale > 0.0f) { /* Read current OFI from batch_states (wired in Phase A) */ @@ -67,16 +78,22 @@ if (!segment_complete && fabsf(position) > 0.001f && micro_reward_scale > 0.0f) float flow_momentum = sign_pos * 0.25f * (delta_depth + delta_vpin + delta_queue + delta_tarr); - /* Price confirmation: close-to-close return (no lookahead) */ - float prev_close = ps[PREV_CLOSE_SLOT]; /* stored from previous bar */ - float bar_return = (prev_close > 0.0f) ? (raw_close - prev_close) / fmaxf(prev_close, 1.0f) : 0.0f; - float price_confirm = sign_pos * bar_return / fmaxf(atr_pct, 1e-6f); + /* Price confirmation: intra-bar directional move from MBP-10 data */ + float raw_open = tgt[4]; /* bar open price from OHLCV */ + float mid_open = tgt[5]; /* mid-price at bar formation from MBP-10 */ + float bar_move = (raw_close - raw_open) / fmaxf(atr_abs, 1e-6f); + float price_confirm = sign_pos * bar_move; + + /* Spread cost awareness: distance from mid to open = half-spread at entry */ + float spread_at_entry = fabsf(raw_open - mid_open); /* Informed flow intensity */ float vol_informed = ofi_cur[6] * ofi_cur[7]; /* VPIN × Kyle's_lambda */ - /* Composite quality */ - float quality = flow_momentum * (1.0f + vol_informed) + 0.5f * price_confirm; + /* Composite quality — spread-aware */ + float quality = flow_momentum * (1.0f + vol_informed) + + 0.5f * price_confirm + - 0.1f * spread_at_entry / fmaxf(atr_abs, 1e-6f); /* penalize wide spreads */ reward = micro_reward_scale * tanhf(quality / 3.0f); } @@ -85,11 +102,8 @@ if (market_dim >= 50) { const float* ofi_cur = batch_states + (long long)i * state_dim + 66; for (int k = 0; k < 8; k++) ps[30 + k] = ofi_cur[k]; } -ps[PREV_CLOSE_SLOT] = raw_close; /* store close for next bar's price_confirm */ ``` -**PREV_CLOSE_SLOT:** Use slot 6 (currently reserved). Define `#define PREV_CLOSE_SLOT 6`. - **Note:** `batch_states` must be passed to `env_step_batch` as a new kernel parameter. Currently the kernel receives `features` (42-dim market features) but NOT the full `batch_states` (96-dim with OFI). Add `const float* __restrict__ batch_states` and `int state_dim` parameters. ### Phase D: Rank Normalization Fix @@ -237,7 +251,7 @@ No code change needed — PopArt handles this correctly by design. | File | Change | |------|--------| -| `experience_kernels.cu` | PORTFOLIO_STRIDE=38, micro-reward body, OFI delta storage, batch_states param | +| `experience_kernels.cu` | PORTFOLIO_STRIDE=38, micro-reward body with tgt[4]/tgt[5], OFI delta storage, batch_states param | | `gpu_experience_collector.rs` | Wire ofi_gpu into state_gather, pass batch_states to env_step | | `gpu_dqn_trainer.rs` | Fix concat_ofi indices 42→66, OFI embed MLP, wider Mamba2 params, d_h_history GEMMs | | `mamba2_temporal_kernel.cu` | Update history kernel to copy SH2+8 dims with ofi_embed param | @@ -248,6 +262,10 @@ No code change needed — PopArt handles this correctly by design. | `reward_shaping_kernel.cu` | Lower is_trade threshold to 1e-5 | | `dqn-production.toml` | micro_reward_scale = 0.1 | | `constructor.rs` | State layout docs, OFI delta indices 74-81 | +| `fxcache.rs` | target_dim 4→6, add raw_open + mid_price_open to targets | +| `data_loading.rs` | Populate tgt[4]=raw_open, tgt[5]=mid_price_open from OHLCV + MBP-10 | +| `precompute_features` binary | Rebuild fxcache with 6-slot targets | +| `train_baseline_rl.rs` / `evaluate_baseline.rs` | Update TARGET_DIM constant from 4 to 6 | ### Config Changes