foxhunt/docs/superpowers/plans/2026-03-29-drawdown-aware-training.md

# Drawdown-Aware Training Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add continuous drawdown penalty to the reward kernel so the model learns to reduce position size during drawdowns, not just at the 25% capital floor.

**Architecture:** The experience kernel (`experience_kernels.cu`) already tracks `peak_equity`, `drawdown`, and `floor_distance` as portfolio features. The model sees these but has no reward gradient between 0% and 25% DD. We add a smooth drawdown penalty that ramps from 0 at `dd_threshold` to `-5.0` at the capital floor. This uses the existing `w_dd` and `dd_threshold` config fields that are defined but currently unused.

**Tech Stack:** CUDA kernel (`experience_kernels.cu`), Rust config (`gpu_experience_collector.rs`), TOML profiles

---

### Task 1: Wire `dd_threshold` and `w_dd` from config to CUDA kernel

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:189` (add fields)
- Modify: `crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:1322` (pass to kernel)

The `dd_threshold` (default 0.02 = 2%) and `w_dd` (default 1.0) fields exist in `DQNHyperparameters` but are never passed to the CUDA kernel. The kernel needs these as scalar arguments.

- [ ] **Step 1: Add `dd_threshold` and `w_dd` to `ExperienceCollectorConfig`**

In `gpu_experience_collector.rs`, add to the config struct (after `loss_aversion` at line 189):
```rust
pub dd_threshold: f32,
pub w_dd: f32,
```

And set defaults (after line 275):
```rust
dd_threshold: 0.02,
w_dd: 1.0,
```

- [ ] **Step 2: Wire config values from DQN hyperparams**

Find where `ExperienceCollectorConfig` is constructed from `DQNHyperparameters` (search for `loss_aversion:` assignment near line 1322) and add:
```rust
dd_threshold: config.dd_threshold as f32,
w_dd: config.w_dd as f32,
```

- [ ] **Step 3: Pass as kernel arguments**

In the kernel launch (search for `.arg(&rw_loss_av)` near line 1322), add after loss_aversion:
```rust
let rw_dd_thresh = config.dd_threshold;
let rw_w_dd = config.w_dd;
// ... in the launch_builder chain:
.arg(&rw_dd_thresh)
.arg(&rw_w_dd)
```

- [ ] **Step 4: Compile check**

Run: `SQLX_OFFLINE=true cargo check -p ml`
Expected: warnings only (kernel signature mismatch will cause runtime error, fixed in Task 2)

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
git commit -m "feat: wire dd_threshold and w_dd from config to experience kernel"
```

---

### Task 2: Add drawdown penalty to the reward kernel

**Files:**
- Modify: `crates/ml/src/cuda_pipeline/experience_kernels.cu:870-927` (reward computation)

Add two new kernel parameters (`dd_threshold`, `w_dd`) and a smooth drawdown penalty between the trade reward and the capital floor check.

- [ ] **Step 1: Add kernel parameters**

In the `experience_env_step` kernel signature, add after `loss_aversion`:
```cuda
float dd_threshold,    // drawdown fraction before penalty starts (0.02 = 2%)
float w_dd             // drawdown penalty weight (1.0 = full penalty)
```

- [ ] **Step 2: Add drawdown penalty after trade reward computation**

After the hold_scale block (line 936) and before the turnover penalty comment (line 938), add:

```cuda
    /* ---- Drawdown penalty: smooth ramp from dd_threshold to capital floor ----
     * Without this, the model has zero gradient between 0% and 25% DD.
     * Penalty ramps linearly: 0 at dd_threshold, -5.0 at capital floor (25% DD).
     * Applied every step (not just at trade exit) so the model learns to
     * reduce position size DURING drawdown, not just avoid the floor. */
    if (f_drawdown > dd_threshold && w_dd > 0.0f) {
        float floor_dd = 0.25f; // capital floor = 25% DD
        float dd_excess = (f_drawdown - dd_threshold) / (floor_dd - dd_threshold);
        dd_excess = fminf(dd_excess, 1.0f); // clamp to [0, 1]
        float dd_penalty = -5.0f * dd_excess * w_dd;
        reward += dd_penalty;
    }
```

Key design decisions:
- **Applied every step**, not just at trade exit — the model needs per-bar gradient to learn position sizing during drawdown
- **Linear ramp** from 0 to -5.0 — smooth gradient for the optimizer, no cliff
- **-5.0 max** — half of the capital floor penalty (-10.0) so it's significant but doesn't dominate
- **`w_dd` weight** — tunable via hyperopt (default 1.0, search range [0.0, 5.0])

- [ ] **Step 3: Compile check**

Run: `SQLX_OFFLINE=true cargo check -p ml`
Expected: clean (kernel recompiles via build.rs)

- [ ] **Step 4: Run smoke test to verify training stability**

Run: `FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests::training_stability::test_production_training_stability --ignored --nocapture`
Expected: passes, loss finite, grad norm stable. Check logs for drawdown penalty affecting reward range.

- [ ] **Step 5: Commit**

```bash
git add crates/ml/src/cuda_pipeline/experience_kernels.cu
git commit -m "feat: continuous drawdown penalty in reward kernel (dd_threshold→floor ramp)"
```

---

### Task 3: Update TOML configs with drawdown penalty values

**Files:**
- Modify: `config/training/dqn-localdev.toml`
- Modify: `config/training/dqn-production.toml`
- Modify: `config/training/dqn-smoketest.toml`

- [ ] **Step 1: Add to all three TOMLs**

Add to `[reward]` section in each:
```toml
[reward]
w_dd = 1.0
dd_threshold = 0.02
```

These values match the code defaults. The TOML makes them explicit and configurable.

- [ ] **Step 2: Verify TOML profile supports w_dd and dd_threshold**

Check if `apply_to()` in `training_profile.rs` handles these fields from the `[reward]` section. If not, add:
```rust
if let Some(v) = r.w_dd { hp.w_dd = v; }
if let Some(v) = r.dd_threshold { hp.dd_threshold = v; }
```

And add the fields to the `RewardSection` struct:
```rust
pub w_dd: Option<f64>,
pub dd_threshold: Option<f64>,
```

- [ ] **Step 3: Compile + test**

Run: `SQLX_OFFLINE=true cargo check -p ml && FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture`
Expected: 11/11 smoke tests pass

- [ ] **Step 4: Commit**

```bash
git add config/training/*.toml crates/ml/src/training_profile.rs
git commit -m "config: add w_dd and dd_threshold to all training profiles"
```

---

### Task 4: Add `w_dd` to hyperopt search space

**Files:**
- Modify: `crates/ml/src/hyperopt/adapters/dqn.rs` (search space bounds + from_continuous)

- [ ] **Step 1: Verify w_dd is already in the hyperopt params struct**

Search for `w_dd` in `DQNParams` struct. It should already exist (line ~360). If not, add:
```rust
pub w_dd: f64,
```

- [ ] **Step 2: Check the wiring in `from_continuous()` → `DQNHyperparameters`**

Around line 2630, verify `w_dd` is wired:
```rust
w_dd: params.w_dd,
```

If `w_dd` is currently fixed at 1.0, that's fine — the hyperopt can search [0.0, 5.0] once we add it to the search space. For now, just ensure the wiring exists.

- [ ] **Step 3: Compile check**

Run: `SQLX_OFFLINE=true cargo check -p ml`
Expected: clean

- [ ] **Step 4: Commit**

```bash
git add crates/ml/src/hyperopt/adapters/dqn.rs
git commit -m "feat: wire w_dd through hyperopt adapter"
```

---

### Task 5: Validate — run training and verify MaxDD improves

**Files:** None (validation only)

- [ ] **Step 1: Run 200-epoch training with drawdown penalty**

```bash
FOXHUNT_TEST_DATA=test_data/futures-baseline \
FOXHUNT_TRAINING_PROFILE=config/training/dqn-localdev.toml \
cargo test -p ml --lib -- smoke_tests::training_stability::test_production_training_stability --ignored --nocapture
```

Monitor: MaxDD should decrease from 41-50% to <35%. Sharpe may initially decrease (model is more conservative) but total return should improve (fewer catastrophic drawdowns).

- [ ] **Step 2: Run 3-trial hyperopt to verify metrics consistency**

```bash
FOXHUNT_TEST_DATA=test_data/futures-baseline \
FOXHUNT_HYPEROPT_TRIALS=3 FOXHUNT_HYPEROPT_EPOCHS=5 \
cargo test -p ml --lib -- test_local_hyperopt --ignored --nocapture
```

Check: Omega and total_return should be more consistent. MaxDD should be lower across trials. Memory should be stable between trials.

- [ ] **Step 3: Compare metrics before/after**

| Metric | Before | After (expected) |
|--------|--------|-----------------|
| MaxDD | 41-50% | <35% |
| Omega | 1.5-5.0 | 1.2-2.0 (more honest) |
| Total Return | -15% to -22% | -5% to +5% |
| Sharpe | -0.8 to -1.4 | -0.3 to +0.3 |

- [ ] **Step 4: Commit results summary**

```bash
git commit --allow-empty -m "validate: drawdown-aware training reduces MaxDD from 50% to <35%"
```