Files
foxhunt/docs/dqn-named-dims.md
jgrusewski 8bc6f1ccd3 fix(dqn): mag_concat_qdir SH2+b0 vs SH2+3 off-by-one — buffer + weight migration
compute-sanitizer pinpointed mag_concat_qdir at experience_kernels.cu:3590
writing 260 floats/state (SH2 + b0_size where b0=4) into a buffer
allocated for 259 floats/state (SH2 + 3, legacy 3-direction layout).
Off-by-one corrupted the next row's first column on every write and
overran past the buffer end on the final row, surfacing as
CUDA_ERROR_ILLEGAL_ADDRESS in downstream kernels (denoise_bias_grad_p1,
cublasLt h_v matmul) on L40S production batch sizes.

The `+3` constant was overloaded:
- direction-conditioned (mag_concat, w_b1fc, w_gate_1) — incorrectly
  hardcoded SH2+3 instead of SH2+branch_0_size when the kernel migrated
  to 4-direction (S/H/L/F).
- OFI-conditioned (ord_concat, urg_concat, w_b2fc, w_b3fc, w_gate_2,
  w_gate_3) — correctly SH2+3 for 3 OFI features per branch
  (concat_ofi_features).

Migrated all direction-conditioned consumers in lockstep
(feedback_no_partial_refactor):

- gpu_dqn_trainer.rs: w_b1fc, w_gate_1 use shared_h2+branch_0_size
- gpu_dqn_trainer.rs: split mag_concat_dim (SH2+b0) from
  ofi_concat_dim (SH2+3) for buffer alloc
- gpu_dqn_trainer.rs: accumulate_d_h_s2_from_concat takes src_stride
  param so mag callers pass SH2+b0, ord/urg callers pass SH2+3
- batched_forward.rs: split mag_concat_dim (SH2+b0) from ofi_concat_dim
  (SH2+3); strided_scatter dst_stride and fc_k now diverge between mag
  (d==1) and ord/urg (d==2,3); add separate (SH2+3) GEMM cache shape
- batched_backward.rs: d==1 (magnitude) dX/dW dims use SH2+b0;
  d==2/3 (order/urgency) keep SH2+3; add separate (SH2+3) GEMM cache
  shape for OFI branches
- gradient_budget.rs: smoke test now allocates b1 with SH2+b0
  and b2/b3 with SH2+3 (was buggy SH2+3 for all three)
- value_decoder.rs: doc updated
- docs/dqn-named-dims.md: new "Branch FC input strides" section
  documenting the direction- vs OFI-conditioning invariant

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 22:19:41 +02:00

7.3 KiB
Raw Blame History

DQN Named Dimensions Registry

Invariant 8 enforcement. Every semantic index has a named constant. Raw indices ([0], [6], [23]) appear ONLY at definition sites. Consumer code always uses the named constant.

State vector offsets

Defined in crates/ml/src/cuda_pipeline/state_layout.cuh and mirrored in Rust via ml-core::state_layout.

Constant Value Meaning
SL_STATE_DIM 112 Total state vector length (D.6 Plan 2 Task 6A: was 104)
SL_STATE_DIM_PADDED 128 Padded for cuBLAS K-tile alignment
SL_MARKET_DIM 42 Market-feature block width
SL_OFI_DIM 32 OFI block width
SL_MTF_DIM 16 Multi-timeframe block width
SL_PORTFOLIO_BASE_DIM 8 Portfolio-base block width
SL_PORTFOLIO_PLAN_DIM 7 Plan-ISV block width (D.6 Plan 2 Task 6A: was 6)
SL_MARKET_START 0
SL_OFI_START 42
SL_MTF_START 74
SL_PORTFOLIO_START 90
SL_PLAN_ISV_START 98

Portfolio state slots (ps[0..PS_STRIDE=38))

Defined in crates/ml/src/cuda_pipeline/state_layout.cuh (PS_* constants). Consumer code uses the named constants; raw ps[N] access outside definition sites is a lint violation (Invariant 8). Slots 3-6 and 22 also carry legacy named aliases in experience_kernels.cu (DSR_A_SLOT etc.) retained for in-file use; PS_* names are the canonical cross-file form.

Index Constant Meaning
0 PS_POSITION Current contract position (signed)
1 PS_CASH Cash balance
2 PS_PORTFOLIO_VALUE Mark-to-market total (cash + pos×price)
3 PS_DSR_A EMA of realised trade returns (DSR)
4 PS_DSR_B EMA of squared trade returns (DSR)
5 PS_DSR_TRADE_COUNT Completed trades counter (DSR warmup)
6 PS_PREV_CLOSE Prev bar raw_close for DSR per-bar returns
7 PS_PEAK_EQUITY High-water mark (init to initial_capital)
8 PS_FLAT_COUNTER Consecutive flat steps
9 PS_PREV_EQUITY Equity at previous step
10 PS_HOLD_TIME Consecutive steps with position
11 PS_REALIZED_PNL Cumulative realised PnL
12 PS_ENTRY_PRICE Price when trade was entered
13 PS_TRADE_START_PNL Realised PnL snapshot at trade entry
14 PS_KELLY_WIN_COUNT Kelly: number of profitable trade exits
15 PS_KELLY_LOSS_COUNT Kelly: number of losing trade exits
16 PS_KELLY_SUM_WINS Kelly: cumulative profit from winners
17 PS_KELLY_SUM_LOSSES Kelly: cumulative |loss| from losers
18 PS_KELLY_SUM_RETURNS Kelly: cumulative net returns (for mu)
19 PS_KELLY_SUM_SQ_RETURNS Kelly: cumulative squared returns (sigma^2)
20 PS_INTRA_TRADE_MAX_DD Worst intra-trade drawdown (v8 reward)
21 PS_INTRA_TRADE_MAX_PNL Best intra-trade unrealised PnL (hindsight)
22 PS_PREV_MID Prev bar MBP-10 mid-price
23 PS_PLAN_TARGET_BARS plan: max hold bars (>0.5 = plan active)
24 PS_PLAN_PROFIT_TARGET plan: raw profit threshold
25 PS_PLAN_STOP_LOSS plan: raw stop threshold
26 PS_PLAN_SCALE_AGGRESSION plan: position ramp speed
27 PS_PLAN_CONVICTION plan: conviction at entry [0,1]
28 PS_PLAN_ASYMMETRY plan: profit/stop asymmetry
29 PS_PLAN_ENTRY_REGIME plan: regime_stability at entry
30-37 PS_OFI_PREV_BASE + k Prev-bar OFI scratch for delta computation
38 PS_STRIDE Total portfolio state stride (=PORTFOLIO_STRIDE)

Plan_isv dimensions (plan_isv[0..7))

D.6 Plan 2 Task 6A landed: 7th dimension PLAN_ISV_REMAINING_FRACTION added. These constants live alongside SL_PLAN_ISV_START in state_layout.cuh.

Index Constant Meaning
0 PLAN_ISV_PROGRESS hold_time / target_bars (clamped 2.0)
1 PLAN_ISV_PNL_VS_TARGET unrealized / (target × equity)
2 PLAN_ISV_PNL_VS_STOP -unrealized / (stop × equity)
3 PLAN_ISV_ENTRY_CONVICTION conviction at entry
4 PLAN_ISV_CONVICTION_DRIFT current conviction entry conviction
5 PLAN_ISV_REGIME_SHIFT |isv[11] entry_stability|
6 PLAN_ISV_REMAINING_FRACTION max(0, min(1, (target_bars hold_time) / target_bars)) when plan active, else 0 — temporal pressure signal

Plan_params output (plan_params[0..6))

From the trade_plan MLP output.

Index Constant Meaning
0 PLAN_PARAM_TARGET_BARS
1 PLAN_PARAM_PROFIT_TARGET
2 PLAN_PARAM_STOP_LOSS
3 PLAN_PARAM_SCALE_AGGRESSION
4 PLAN_PARAM_CONVICTION
5 PLAN_PARAM_ASYMMETRY

Branch indices (4-branch factored action)

Index Constant Meaning
0 BRANCH_DIR Direction branch (4 actions)
1 BRANCH_MAG Magnitude branch (3 actions)
2 BRANCH_ORD Order-type branch (3 actions)
3 BRANCH_URG Urgency branch (3 actions)

Direction action sub-indices

Index Constant Meaning
0 DIR_SHORT Open short (or close long + open short)
1 DIR_HOLD Keep current position (no-op on position)
2 DIR_LONG Open long (or close short + open long)
3 DIR_FLAT Close any position to zero

Magnitude action sub-indices

Index Constant Meaning
0 MAG_QUARTER 0.25× max_position
1 MAG_HALF 0.50× max_position
2 MAG_FULL 1.00× max_position

Branch FC input strides (direction- vs OFI-conditioned)

The four branch FC heads (w_b{0,1,2,3}fc) and their gate twins (w_gate_{0,1,2,3}) have different input strides depending on what each branch concatenates onto the trunk activation:

Branch (d) Constant role Input stride Conditioning
0 (Direction) trunk only shared_h2 none
1 (Magnitude) mag_concat shared_h2 + branch_0_size direction-conditioned (concat of h_s2 + per-direction Q values, 4 in production)
2 (Order) ord_concat shared_h2 + 3 OFI-conditioned (concat of vsn_masked + 3 OFI features)
3 (Urgency) urg_concat shared_h2 + 3 OFI-conditioned (concat of vsn_masked + 3 OFI features)

Invariant (off-by-one trap). The +3 constant is overloaded: in the OFI branches it's a literal (3 features per branch from concat_ofi_features), but in the magnitude branch it must derive from branch_0_size (since 4-direction S/H/L/F was added). Buffers, weight tensors, GEMM cache shapes, accumulator strides, and dX backward dims all share this contract — see commit fixing mag_concat_qdir OOB (compute-sanitizer caught 1679 errors). Direction-conditioned sites: w_b1fc, w_gate_1, mag_concat_buf, d_mag_concat_buf. OFI-conditioned sites: w_b{2,3}fc, w_gate_{2,3}, {ord,urg}_concat_buf, d_{ord,urg}_concat_buf.

Commit history

  • Task 4A (ps[0..PS_STRIDE) constants): commit 144c85b85
  • Task 4B (plan_isv[0..6) constants): commit 741cb48d5
  • Task 4C (plan_params[0..6) constants): commit 0ac83479e
  • Task 4D (BRANCH_* constants): commit 11755632b
  • Task 4E (DIR_* direction sub-indices): commit c64fde0be
  • Task 4F (MAG_* magnitude sub-indices): commit 537e18b84
  • Task 4 gap-fix (3 review issues): experience_kernels.cu lines 1148-1152 and 1194 missed by sed sweep (ps_base_plan+N syntax), replaced with PS_PLAN_TARGET_BARS/PS_POSITION/DIR_LONG/ DIR_SHORT/DIR_FLAT; ml-core state_layout.rs Rust mirror added for all Task 4 constants (PS_/PLAN_ISV_/PLAN_PARAM_/BRANCH_/DIR_/MAG_, 46 constants total).