# Trade Physics Refactor + DQN Static Analysis

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Eliminate ALL train/eval mismatches by making `trade_physics.cuh` the single source of truth, then statically audit every CUDA kernel launch site for arg/buffer/type mismatches.

**Architecture:** Upgrade the shared header to handle training's richer logic (CUSUM spread, trailing stop), then rewrite both kernels to call shared functions. Follow with parallel static analysis of all 37+ kernel launch sites.

**Tech Stack:** CUDA C (.cu/.cuh), Rust (cudarc), build.rs precompilation

---

## Current Mismatches (Root Causes)

| Function | Training kernel | Backtest kernel | Shared header | Status |
|----------|----------------|-----------------|---------------|--------|
| Action decode | Inline (line 613-625) | Uses `decode_exposure_index()` | ✓ Exists | Training doesn't use shared |
| Position map | `exposure_idx_to_fraction()` hardcoded 0.25 | Uses `compute_target_position()` | ✓ Exists | Training uses old function |
| Tx cost | CUSUM spread + additive impact | Static sqrt spread | Simplified version | **MISMATCH** |
| Hold enforcement | Inline (line 826-846) + action aliasing fix | Uses `enforce_hold()` | ✓ Exists | Training has extras |
| Trailing stop | Present (line 799-813) | Missing | Not in header | **MISMATCH** |
| Capital floor | Inline (line 1001-1012) | Uses `check_capital_floor()` | ✓ Exists | Training doesn't use shared |
| Margin cap | Uses `apply_margin_cap()` | Uses `apply_margin_cap()` | ✓ Exists | Both use shared ✓ |
| Hold time | Inline | Uses `update_hold_time()` | ✓ Exists | Training doesn't use shared |

---

### Task 1: Upgrade `compute_tx_cost` to support CUSUM spread

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/trade_physics.cuh`

The current `compute_tx_cost` uses static `sqrt(delta/max_pos)` spread scaling. Training uses CUSUM-based spread from market features. Upgrade the shared function to accept an optional `spread_scale_override`:

- [ ] **Step 1: Add `spread_scale_override` parameter to `compute_tx_cost`**

```c
__device__ __forceinline__ float compute_tx_cost(
    float delta,
    float close,
    float tx_cost_bps,
    float spread_cost,
    float max_position,
    int   order_type_idx,
    float spread_scale_override  /* <= 0 = compute from sqrt(delta/max_pos) */
) {
    float abs_delta = fabsf(delta);
    float spread_scale;
    if (spread_scale_override > 0.0f) {
        spread_scale = spread_scale_override;  /* CUSUM-based from market features */
    } else {
        spread_scale = sqrtf(abs_delta / fmaxf(max_position, 0.01f));
        if (spread_scale < 1.0f) spread_scale = 1.0f;
    }
    float impact_scale = 1.0f + sqrtf(abs_delta / fmaxf(max_position, 0.01f));
    float order_premium = (order_type_idx == 0) ? 0.0f
                        : (order_type_idx == 1) ? 0.0002f
                        : -0.0005f;
    return abs_delta * close * (tx_cost_bps * 0.0001f * spread_scale * impact_scale + order_premium)
         + abs_delta * spread_cost * 0.5f;
}
```

Key: `impact_scale = 1 + sqrt(delta/max_pos)` now matches training (was hardcoded 1.0 in backtest). Both callers get the same Almgren-Chriss impact model. The only difference is spread_scale source (CUSUM vs sqrt).

- [ ] **Step 2: Update backtest_env_kernel.cu to pass `spread_scale_override = -1.0f`**

The backtest doesn't have CUSUM features, so it passes -1 to use the sqrt fallback.

- [ ] **Step 3: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml`

---

### Task 2: Replace training kernel's inline logic with shared functions

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu`

- [ ] **Step 1: Delete `exposure_idx_to_fraction` (line 121-127)**

Replace its call at line 627 with:
```c
float target_position = compute_target_position(exposure_idx, b0_size, max_position);
```

This eliminates the hardcoded 0.25 step size.

- [ ] **Step 2: Replace inline decode (lines 613-625) with shared function**

Replace:
```c
int exposure_idx;
if (b1_size > 0 && b2_size > 0) { ... }
```
With:
```c
int exposure_idx = decode_exposure_index(action_idx, b0_size, b1_size, b2_size);
```

- [ ] **Step 3: Replace inline hold enforcement (lines 826-846) with shared function**

The training kernel's hold enforcement has extra logic (action aliasing fix). Keep the aliasing fix but use `enforce_hold()` for the core hold decision:

```c
int is_last_bar = (bar_idx >= total_bars - 1) ? 1 : 0;
float held_position = enforce_hold(ps[0], position, hold_time, min_hold_bars, is_last_bar);
int hold_violation = (held_position != position);  /* enforce_hold overrode */
if (hold_violation) {
    position = held_position;
    cash = ps[1];
    /* ... keep action aliasing fix ... */
}
```

- [ ] **Step 4: Replace inline capital floor (lines 1001-1012) with shared function**

Replace:
```c
float capital_floor = peak_equity * 0.75f;
if (new_portfolio_value < capital_floor) { ... }
```
With:
```c
if (check_capital_floor(new_portfolio_value, peak_equity)) { ... }
```

- [ ] **Step 5: Replace inline tx_cost (lines 708-728) with shared function**

The training kernel computes CUSUM-based `spread_scale` at lines 698-703. Pass it to the shared function:

```c
float cusum_spread = cusum_raw / 0.5f;
cusum_spread = fmaxf(0.5f, fminf(2.0f, cusum_spread));
int order_type_idx = decode_order_type(action_idx, b1_size, b2_size);
tx_cost = compute_tx_cost(delta, raw_close, tx_cost_multiplier, 0.0f, max_position,
                          order_type_idx, cusum_spread);
```

Note: training uses `raw_close` (not `close`) and `tx_cost_multiplier` (not `tx_cost_bps`). These are the same concept — the shared function parameter is named `tx_cost_bps` but both are multipliers. No semantic mismatch.

- [ ] **Step 6: Replace inline order_type decode (line 720) with shared function**

Already handled by using `decode_order_type(action_idx, b1_size, b2_size)` in Step 5.

- [ ] **Step 7: Verify build**

Run: `SQLX_OFFLINE=true cargo check -p ml`

- [ ] **Step 8: Run smoke test**

Run: `FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib --profile release-test -- smoke_tests::training_stability::test_gpu_collector_auto_initializes --ignored --nocapture`

Expected: Sharpe > 0, trades > 0, no SIGSEGV.

---

### Task 3: Add trailing stop to shared header and backtest

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/trade_physics.cuh`
- Modify: `crates/ml/src/cuda_pipeline/backtest_env_kernel.cu`

The training kernel has a trailing stop (lines 799-813) that exits positions when profit retreats from peak. The backtest does NOT have this → train/eval mismatch.

- [ ] **Step 1: Add `check_trailing_stop` to `trade_physics.cuh`**

```c
/* Returns 1 if trailing stop triggers (profit retreated beyond threshold) */
__device__ __forceinline__ int check_trailing_stop(
    float hold_time,
    int   min_hold_bars,
    float peak_equity,
    float prev_equity,
    float unrealized_trade_pnl,
    float trail_distance    /* e.g. 0.005 = 0.5% base */
) {
    if (hold_time < (float)min_hold_bars || peak_equity <= 1.0f) return 0;
    float peak_return = (peak_equity - prev_equity) / fmaxf(prev_equity, 1.0f);
    if (peak_return > trail_distance) {
        float trail_floor = peak_return - trail_distance;
        if (unrealized_trade_pnl < trail_floor) return 1;
    }
    return 0;
}
```

- [ ] **Step 2: Wire trailing stop into backtest_env_kernel.cu**

After mark-to-market, before the post-trade floor check:
```c
float unrealized_trade_pnl = (fabsf(position) > 0.001f && entry_price > 0.0f)
    ? (close - entry_price) / entry_price : 0.0f;
if (check_trailing_stop(hold_time, min_hold_bars, max_equity, value,
                        unrealized_trade_pnl, 0.005f)) {
    /* Force flat — trailing stop triggered */
    position = 0.0f;
    cash = new_value;
    entry_price = 0.0f;
}
```

- [ ] **Step 3: Make training kernel use shared `check_trailing_stop`**

Replace inline trailing stop (lines 799-813) with the shared function call.

- [ ] **Step 4: Run hyperopt test**

Run: `FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib --profile release-test -- hyperopt::campaign::tests::test_local_hyperopt --ignored --nocapture`

Expected: 2 trials complete, max_dd < 30%, no SIGSEGV.

---

### Task 4: Fix remaining display bugs

**Files:**
- Modify: `crates/ml/src/hyperopt/adapters/dqn.rs`

- [ ] **Step 1: Fix win_rate in TRIAL_SUMMARY (line ~3287)**

Already identified: `best_win_rate` needs `* 100.0`. Verify the fix at line 3277 is applied.

- [ ] **Step 2: Search for any other win_rate display without *100**

```bash
grep -n 'win_rate' crates/ml/src/hyperopt/adapters/dqn.rs | grep -v '100'
```

Fix all instances.

---

### Task 5: Static analysis — kernel launch arg audit

**Files (read-only audit):**
- `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` (all `launch_builder` calls)
- `crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs` (all `launch_builder` calls)
- `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs` (all `launch_builder` calls)
- `crates/ml/src/cuda_pipeline/gpu_attention.rs` (forward + backward launches)
- `crates/ml/src/cuda_pipeline/gpu_iqn_head.rs` (IQN kernel launches)
- `crates/ml/src/cuda_pipeline/gpu_iql_trainer.rs` (IQL kernel launches)
- `crates/ml/src/cuda_pipeline/gpu_her.rs` (HER kernel launches)
- All `.cu` kernel source files

For EACH kernel launch (`launch_builder`):
1. Count args in Rust `.arg()` chain
2. Count params in the corresponding `extern "C" __global__` signature
3. Verify types match (f32 vs i32 vs u64/pointer)
4. Verify buffer sizes match kernel's max access index
5. Verify shared memory bytes >= kernel's shared memory usage

- [ ] **Step 1: Create audit checklist**

Generate a table of every kernel launch site with arg counts and match status.

- [ ] **Step 2: Flag all mismatches**

Any kernel where Rust arg count ≠ CUDA param count is a potential SIGSEGV.

- [ ] **Step 3: Fix any mismatches found**

---

### Task 6: Static analysis — buffer size audit

For EACH `alloc_zeros` / `clone_htod` buffer:
1. What size is it allocated at?
2. What's the maximum index the kernel accesses?
3. Is the max index < allocated size?

Focus on buffers that depend on `batch_size`, `num_atoms`, `n_windows`, `chunk_size`.

- [ ] **Step 1: Audit `gpu_backtest_evaluator.rs` buffers**
- [ ] **Step 2: Audit `gpu_dqn_trainer.rs` buffers**
- [ ] **Step 3: Audit `gpu_experience_collector.rs` buffers**
- [ ] **Step 4: Fix any underallocations found**

---

### Task 7: Hive deep analysis — DQN logic and math audit

**Goal:** Orchestrate a parallel hive of specialized agents to deep-audit the DQN model's logic and math. Each agent focuses on one domain. Results are synthesized into a master findings report.

**Orchestration:** Use Zen + parallel subagents. Each agent reads the relevant code and reports findings independently. No code changes in this task — output is a prioritized bug/risk list.

#### Agent 1: Reward function math audit
**Scope:** `experience_kernels.cu` lines 730-990 (reward shaping)
- Verify reward v6 formula: `mark-to-market portfolio return per bar`
- Check loss_aversion asymmetry: is it applied correctly?
- Verify DSR (Differential Sharpe Ratio) computation
- Check for division-by-zero in reward normalization
- Verify idle penalty scaling
- Check drawdown penalty threshold logic

#### Agent 2: Portfolio simulation correctness
**Scope:** `experience_kernels.cu` env_step + `backtest_env_kernel.cu`
- Verify mark-to-market P&L: `position * (next_price - current_price)`
- Check cash accounting: `cash -= delta * price` (buying costs cash, selling adds)
- Verify equity = cash + unrealized — no double counting
- Check trade reversal P&L: when going S100→L100, is the short P&L booked correctly before the long opens?
- Verify hold_time tracking matches between training and backtest
- Check that position signs are consistent (positive=long, negative=short)

#### Agent 3: C51 distributional RL math
**Scope:** `c51_loss_kernel.cu`, `mse_loss_kernel.cu`, `c51_grad_kernel.cu`, `mse_grad_kernel.cu`
- Verify Bellman projection: `T_z = r + gamma * z_j` clamped to [v_min, v_max]
- Verify log-softmax numerical stability (max subtraction before exp)
- Verify cross-entropy loss: `-sum(projected * log_probs)`
- Verify MSE loss through distributional expectation: `E[Q] = sum(softmax(logits) * support)`
- Verify gradient: `d_logit = is_weight * (exp(log_prob) - projected)`
- Check n-step return: `gamma^n` used correctly for multi-step Bellman
- Verify label smoothing: `projected = (1-eps)*projected + eps/num_atoms`

#### Agent 4: Annualization and financial metrics
**Scope:** `backtest_metrics_kernel.cu`, `financials.rs`
- Verify Sharpe: `(mean / std) * sqrt(bars_per_year)` — is annualization correct for 1-min bars?
- Verify Sortino: uses downside deviation only (negative returns)
- Verify max drawdown: sequential scan from equity curve
- Verify Calmar: `annualized_return / max_drawdown` — does annualization match Sharpe?
- Verify VaR/CVaR: 5th percentile of sorted returns
- Check for consistent use of `bars_per_day=390` across all calculations
- Verify trade counting uses exposure changes (not factored action changes)

#### Agent 5: CUDA memory safety audit
**Scope:** All `.rs` files in `cuda_pipeline/`
- Every `launch_builder` arg count vs kernel param count
- Every `alloc_zeros` size vs maximum kernel access index
- Every `shared_mem_bytes` vs kernel's `__shared__` usage
- Every `memcpy_dtod_async` size vs source/destination buffer sizes
- Every `CudaSlice` reinterpret cast (`as *const CudaSlice<u16>`) — is the element count correct?
- Every `device_ptr()` call — is the guard held long enough?
- OnceLock kernel compilation — can stale PTX be loaded with wrong context?

#### Agent 6: Hyperopt objective function audit
**Scope:** `hyperopt/adapters/dqn.rs` — `extract_objective`, `evaluate_gpu`, `train_with_params`
- Verify composite objective weights sum to reasonable total
- Check tanh normalization divisors match expected metric ranges
- Verify CVaR penalty threshold is correct for 1-min bars
- Check that all metrics flow correctly from GPU kernel → Rust aggregation → objective
- Verify no metric is used as both ratio (0-1) and percentage (0-100) inconsistently
- Check that `total_trades` counts exposure changes, not factored action flips
- Verify backtest window sizing: stride, overlap, max_window_bars

- [ ] **Step 1: Launch all 6 agents in parallel**

Each agent reads the specified source files and produces:
- A numbered list of findings (bugs, risks, inconsistencies)
- Severity: CRITICAL (wrong results), HIGH (potential crash), MEDIUM (correctness risk), LOW (style)
- For each finding: exact file, line number, and what's wrong

- [ ] **Step 2: Synthesize findings into master report**

Merge all 6 agents' findings into a single prioritized list. Group by severity. Create tasks for CRITICAL and HIGH findings.

- [ ] **Step 3: Fix CRITICAL findings immediately**

Any finding that produces wrong training results or crashes must be fixed before H100 deployment.

---

### Task 8: Research opportunities to improve DQN logic

**Goal:** Each hive agent (from Task 7) also produces an **improvement recommendations** section alongside its bug findings. These are NOT bug fixes — they're research-backed suggestions to improve training quality, convergence speed, or production profitability.

#### Agent 1 additions: Reward shaping improvements
- Is reward v6 (pure mark-to-market) optimal? Compare with alternatives: risk-adjusted return per bar, log-return, excess return over risk-free
- Should the idle penalty scale with market volatility (penalize inaction more in trending markets)?
- Could reward clipping improve stability? What range?
- Is loss_aversion=1.5 calibrated for ES futures, or just a guess?

#### Agent 2 additions: Portfolio simulation improvements
- Should position sizing use fractional Kelly from the start (not just after 20 trades)?
- Could the trailing stop be adaptive (tighter in low-vol, wider in high-vol)?
- Should the capital floor be dynamic (tighter when losing streak detected)?
- Is the current margin model (6% of notional) realistic for CME ES? Check actual CME SPAN margins.

#### Agent 3 additions: Distributional RL improvements
- Is C51 with 101 atoms optimal, or would IQN alone be better? Compare convergence speed.
- Is the MSE→C51 warmup schedule (c51_warmup_epochs) optimal? Could curriculum learning help?
- Would QR-DQN (fixed quantiles) outperform C51 (fixed support) for fat-tailed financial returns?
- Could Munchausen DQN (KL-regularized) improve exploration in the financial action space?

#### Agent 4 additions: Metrics and objective improvements
- Should the objective use risk-parity weighting (equalize contribution of Sharpe/Sortino/Calmar/Omega)?
- Is the CVaR penalty threshold correct? Should it be per-window or global?
- Could walk-forward cross-validation (purged) reduce overfitting to specific market regimes?
- Should the objective include a turnover penalty (penalize high trade frequency)?

#### Agent 5 additions: CUDA performance improvements
- Which kernels are occupancy-limited? Could register reduction help?
- Are there unnecessary GPU→CPU transfers in the hot path?
- Could the backtest evaluator benefit from CUDA Graph per-chunk (not full loop)?
- Is the cuBLAS workspace sized optimally for H100 tensor cores?

#### Agent 6 additions: Hyperopt improvements
- Is 22D still too many dimensions for PSO? Which params have the most sensitivity?
- Could Bayesian optimization (TPE/GP) outperform PSO for this space?
- Should hyperopt use early stopping per trial (kill bad trials at epoch 5 instead of running all 50)?
- Could multi-fidelity optimization (ASHA/Hyperband) be more efficient?

- [ ] **Step 1: Each agent produces 3-5 prioritized improvement suggestions**

For each suggestion: expected impact (High/Medium/Low), implementation effort (days), and evidence/citation.

- [ ] **Step 2: Synthesize into a ranked improvement roadmap**

Order by impact/effort ratio. The top 3 improvements become the next sprint's tasks.

---

## Validation

After all tasks:

```bash
# Unit tests
SQLX_OFFLINE=true cargo test -p ml --lib -- hyperopt::adapters::dqn::tests --nocapture

# Smoke test (training path)
FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib --profile release-test -- smoke_tests::training_stability::test_gpu_collector_auto_initializes --ignored --nocapture

# Integration test (hyperopt path)
FOXHUNT_TEST_DATA=test_data/futures-baseline SQLX_OFFLINE=true cargo test -p ml --lib --profile release-test -- hyperopt::campaign::tests::test_local_hyperopt --ignored --nocapture
```

Success criteria:
- 0 test failures
- max_dd < 30% (circuit breaker + margin cap working)
- No SIGSEGV
- Sharpe > 0 on smoke test
- Both hyperopt trials complete with finite metrics