gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
1305d6531b18307e9d1228c9a0f9505cefbd94de
foxhunt/docs/dqn-named-dims.md
jgrusewski 8bc6f1ccd3 fix(dqn): mag_concat_qdir SH2+b0 vs SH2+3 off-by-one — buffer + weight migration 
compute-sanitizer pinpointed mag_concat_qdir at experience_kernels.cu:3590
writing 260 floats/state (SH2 + b0_size where b0=4) into a buffer
allocated for 259 floats/state (SH2 + 3, legacy 3-direction layout).
Off-by-one corrupted the next row's first column on every write and
overran past the buffer end on the final row, surfacing as
CUDA_ERROR_ILLEGAL_ADDRESS in downstream kernels (denoise_bias_grad_p1,
cublasLt h_v matmul) on L40S production batch sizes.

The `+3` constant was overloaded:
- direction-conditioned (mag_concat, w_b1fc, w_gate_1) — incorrectly
  hardcoded SH2+3 instead of SH2+branch_0_size when the kernel migrated
  to 4-direction (S/H/L/F).
- OFI-conditioned (ord_concat, urg_concat, w_b2fc, w_b3fc, w_gate_2,
  w_gate_3) — correctly SH2+3 for 3 OFI features per branch
  (concat_ofi_features).

Migrated all direction-conditioned consumers in lockstep
(feedback_no_partial_refactor):

- gpu_dqn_trainer.rs: w_b1fc, w_gate_1 use shared_h2+branch_0_size
- gpu_dqn_trainer.rs: split mag_concat_dim (SH2+b0) from
  ofi_concat_dim (SH2+3) for buffer alloc
- gpu_dqn_trainer.rs: accumulate_d_h_s2_from_concat takes src_stride
  param so mag callers pass SH2+b0, ord/urg callers pass SH2+3
- batched_forward.rs: split mag_concat_dim (SH2+b0) from ofi_concat_dim
  (SH2+3); strided_scatter dst_stride and fc_k now diverge between mag
  (d==1) and ord/urg (d==2,3); add separate (SH2+3) GEMM cache shape
- batched_backward.rs: d==1 (magnitude) dX/dW dims use SH2+b0;
  d==2/3 (order/urgency) keep SH2+3; add separate (SH2+3) GEMM cache
  shape for OFI branches
- gradient_budget.rs: smoke test now allocates b1 with SH2+b0
  and b2/b3 with SH2+3 (was buggy SH2+3 for all three)
- value_decoder.rs: doc updated
- docs/dqn-named-dims.md: new "Branch FC input strides" section
  documenting the direction- vs OFI-conditioning invariant

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 22:19:41 +02:00

155 lines
7.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# DQN Named Dimensions Registry
**Invariant 8 enforcement.** Every semantic index has a named constant. Raw indices (`[0]`, `[6]`, `[23]`) appear ONLY at definition sites. Consumer code always uses the named constant.
## State vector offsets
Defined in `crates/ml/src/cuda_pipeline/state_layout.cuh` and mirrored in Rust via `ml-core::state_layout`.
| Constant | Value | Meaning |
|---|---|---|
| `SL_STATE_DIM` | 112 | Total state vector length (D.6 Plan 2 Task 6A: was 104) |
| `SL_STATE_DIM_PADDED` | 128 | Padded for cuBLAS K-tile alignment |
| `SL_MARKET_DIM` | 42 | Market-feature block width |
| `SL_OFI_DIM` | 32 | OFI block width |
| `SL_MTF_DIM` | 16 | Multi-timeframe block width |
| `SL_PORTFOLIO_BASE_DIM` | 8 | Portfolio-base block width |
| `SL_PORTFOLIO_PLAN_DIM` | 7 | Plan-ISV block width (D.6 Plan 2 Task 6A: was 6) |
| `SL_MARKET_START` | 0 | |
| `SL_OFI_START` | 42 | |
| `SL_MTF_START` | 74 | |
| `SL_PORTFOLIO_START` | 90 | |
| `SL_PLAN_ISV_START` | 98 | |
## Portfolio state slots (ps[0..PS_STRIDE=38))
Defined in `crates/ml/src/cuda_pipeline/state_layout.cuh` (PS_* constants).
Consumer code uses the named constants; raw `ps[N]` access outside definition
sites is a lint violation (Invariant 8). Slots 3-6 and 22 also carry legacy
named aliases in `experience_kernels.cu` (DSR_A_SLOT etc.) retained for
in-file use; PS_* names are the canonical cross-file form.
| Index | Constant | Meaning |
|---|---|---|
| 0 | `PS_POSITION` | Current contract position (signed) |
| 1 | `PS_CASH` | Cash balance |
| 2 | `PS_PORTFOLIO_VALUE` | Mark-to-market total (cash + pos×price) |
| 3 | `PS_DSR_A` | EMA of realised trade returns (DSR) |
| 4 | `PS_DSR_B` | EMA of squared trade returns (DSR) |
| 5 | `PS_DSR_TRADE_COUNT` | Completed trades counter (DSR warmup) |
| 6 | `PS_PREV_CLOSE` | Prev bar raw_close for DSR per-bar returns |
| 7 | `PS_PEAK_EQUITY` | High-water mark (init to initial_capital) |
| 8 | `PS_FLAT_COUNTER` | Consecutive flat steps |
| 9 | `PS_PREV_EQUITY` | Equity at previous step |
| 10 | `PS_HOLD_TIME` | Consecutive steps with position |
| 11 | `PS_REALIZED_PNL` | Cumulative realised PnL |
| 12 | `PS_ENTRY_PRICE` | Price when trade was entered |
| 13 | `PS_TRADE_START_PNL` | Realised PnL snapshot at trade entry |
| 14 | `PS_KELLY_WIN_COUNT` | Kelly: number of profitable trade exits |
| 15 | `PS_KELLY_LOSS_COUNT` | Kelly: number of losing trade exits |
| 16 | `PS_KELLY_SUM_WINS` | Kelly: cumulative profit from winners |
| 17 | `PS_KELLY_SUM_LOSSES` | Kelly: cumulative \|loss\| from losers |
| 18 | `PS_KELLY_SUM_RETURNS` | Kelly: cumulative net returns (for mu) |
| 19 | `PS_KELLY_SUM_SQ_RETURNS` | Kelly: cumulative squared returns (sigma^2) |
| 20 | `PS_INTRA_TRADE_MAX_DD` | Worst intra-trade drawdown (v8 reward) |
| 21 | `PS_INTRA_TRADE_MAX_PNL` | Best intra-trade unrealised PnL (hindsight) |
| 22 | `PS_PREV_MID` | Prev bar MBP-10 mid-price |
| 23 | `PS_PLAN_TARGET_BARS` | plan: max hold bars (>0.5 = plan active) |
| 24 | `PS_PLAN_PROFIT_TARGET` | plan: raw profit threshold |
| 25 | `PS_PLAN_STOP_LOSS` | plan: raw stop threshold |
| 26 | `PS_PLAN_SCALE_AGGRESSION` | plan: position ramp speed |
| 27 | `PS_PLAN_CONVICTION` | plan: conviction at entry [0,1] |
| 28 | `PS_PLAN_ASYMMETRY` | plan: profit/stop asymmetry |
| 29 | `PS_PLAN_ENTRY_REGIME` | plan: regime_stability at entry |
| 30-37 | `PS_OFI_PREV_BASE + k` | Prev-bar OFI scratch for delta computation |
| 38 | `PS_STRIDE` | Total portfolio state stride (=PORTFOLIO_STRIDE) |
## Plan_isv dimensions (plan_isv[0..7))
D.6 Plan 2 Task 6A landed: 7th dimension `PLAN_ISV_REMAINING_FRACTION` added. These constants live alongside `SL_PLAN_ISV_START` in `state_layout.cuh`.
| Index | Constant | Meaning |
|---|---|---|
| 0 | `PLAN_ISV_PROGRESS` | hold_time / target_bars (clamped 2.0) |
| 1 | `PLAN_ISV_PNL_VS_TARGET` | unrealized / (target × equity) |
| 2 | `PLAN_ISV_PNL_VS_STOP` | -unrealized / (stop × equity) |
| 3 | `PLAN_ISV_ENTRY_CONVICTION` | conviction at entry |
| 4 | `PLAN_ISV_CONVICTION_DRIFT` | current conviction − entry conviction |
| 5 | `PLAN_ISV_REGIME_SHIFT` | \|isv[11] − entry_stability\| |
| 6 | `PLAN_ISV_REMAINING_FRACTION` | max(0, min(1, (target_bars − hold_time) / target_bars)) when plan active, else 0 — temporal pressure signal |
## Plan_params output (plan_params[0..6))
From the trade_plan MLP output.
| Index | Constant | Meaning |
|---|---|---|
| 0 | `PLAN_PARAM_TARGET_BARS` | |
| 1 | `PLAN_PARAM_PROFIT_TARGET` | |
| 2 | `PLAN_PARAM_STOP_LOSS` | |
| 3 | `PLAN_PARAM_SCALE_AGGRESSION` | |
| 4 | `PLAN_PARAM_CONVICTION` | |
| 5 | `PLAN_PARAM_ASYMMETRY` | |
## Branch indices (4-branch factored action)
| Index | Constant | Meaning |
|---|---|---|
| 0 | `BRANCH_DIR` | Direction branch (4 actions) |
| 1 | `BRANCH_MAG` | Magnitude branch (3 actions) |
| 2 | `BRANCH_ORD` | Order-type branch (3 actions) |
| 3 | `BRANCH_URG` | Urgency branch (3 actions) |
## Direction action sub-indices
| Index | Constant | Meaning |
|---|---|---|
| 0 | `DIR_SHORT` | Open short (or close long + open short) |
| 1 | `DIR_HOLD` | Keep current position (no-op on position) |
| 2 | `DIR_LONG` | Open long (or close short + open long) |
| 3 | `DIR_FLAT` | Close any position to zero |
## Magnitude action sub-indices
| Index | Constant | Meaning |
|---|---|---|
| 0 | `MAG_QUARTER` | 0.25× max_position |
| 1 | `MAG_HALF` | 0.50× max_position |
| 2 | `MAG_FULL` | 1.00× max_position |
## Branch FC input strides (direction- vs OFI-conditioned)
The four branch FC heads (`w_b{0,1,2,3}fc`) and their gate twins
(`w_gate_{0,1,2,3}`) have different input strides depending on what each
branch concatenates onto the trunk activation:
| Branch (d) | Constant role | Input stride | Conditioning |
|---|---|---|---|
| 0 (Direction) | trunk only | `shared_h2` | none |
| 1 (Magnitude) | mag_concat | `shared_h2 + branch_0_size` | direction-conditioned (concat of `h_s2` + per-direction Q values, 4 in production) |
| 2 (Order) | ord_concat | `shared_h2 + 3` | OFI-conditioned (concat of `vsn_masked` + 3 OFI features) |
| 3 (Urgency) | urg_concat | `shared_h2 + 3` | OFI-conditioned (concat of `vsn_masked` + 3 OFI features) |
**Invariant (off-by-one trap).** The `+3` constant is overloaded: in the OFI
branches it's a literal (3 features per branch from `concat_ofi_features`), but
in the magnitude branch it must derive from `branch_0_size` (since 4-direction
S/H/L/F was added). Buffers, weight tensors, GEMM cache shapes, accumulator
strides, and dX backward dims all share this contract — see commit fixing
mag_concat_qdir OOB (compute-sanitizer caught 1679 errors). Direction-conditioned
sites: `w_b1fc`, `w_gate_1`, `mag_concat_buf`, `d_mag_concat_buf`. OFI-conditioned
sites: `w_b{2,3}fc`, `w_gate_{2,3}`, `{ord,urg}_concat_buf`,
`d_{ord,urg}_concat_buf`.
## Commit history
- Task 4A (ps[0..PS_STRIDE) constants): commit 144c85b85
- Task 4B (plan_isv[0..6) constants): commit 741cb48d5
- Task 4C (plan_params[0..6) constants): commit 0ac83479e
- Task 4D (BRANCH_* constants): commit 11755632b
- Task 4E (DIR_* direction sub-indices): commit c64fde0be
- Task 4F (MAG_* magnitude sub-indices): commit 537e18b84
- Task 4 gap-fix (3 review issues): experience_kernels.cu lines 1148-1152 and 1194 missed by
sed sweep (ps_base_plan+N syntax), replaced with PS_PLAN_TARGET_BARS/PS_POSITION/DIR_LONG/
DIR_SHORT/DIR_FLAT; ml-core state_layout.rs Rust mirror added for all Task 4 constants
(PS_*/PLAN_ISV_*/PLAN_PARAM_*/BRANCH_*/DIR_*/MAG_*, 46 constants total).
Reference in New Issue View Git Blame Copy Permalink