foxhunt/docs/superpowers/plans/2026-03-29-drawdown-aware-training.md

Drawdown-Aware Training Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add continuous drawdown penalty to the reward kernel so the model learns to reduce position size during drawdowns, not just at the 25% capital floor.

Architecture: The experience kernel (experience_kernels.cu) already tracks peak_equity, drawdown, and floor_distance as portfolio features. The model sees these but has no reward gradient between 0% and 25% DD. We add a smooth drawdown penalty that ramps from 0 at dd_threshold to -5.0 at the capital floor. This uses the existing w_dd and dd_threshold config fields that are defined but currently unused.

Tech Stack: CUDA kernel (experience_kernels.cu), Rust config (gpu_experience_collector.rs), TOML profiles

---

Task 1: Wire dd_threshold and w_dd from config to CUDA kernel

Files:

• Modify: crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:189 (add fields)
• Modify: crates/ml/src/cuda_pipeline/gpu_experience_collector.rs:1322 (pass to kernel)

The dd_threshold (default 0.02 = 2%) and w_dd (default 1.0) fields exist in DQNHyperparameters but are never passed to the CUDA kernel. The kernel needs these as scalar arguments.

• Step 1: Add dd_threshold and w_dd to ExperienceCollectorConfig

In gpu_experience_collector.rs, add to the config struct (after loss_aversion at line 189):

pub dd_threshold: f32,
pub w_dd: f32,

And set defaults (after line 275):

dd_threshold: 0.02,
w_dd: 1.0,

• Step 2: Wire config values from DQN hyperparams

Find where ExperienceCollectorConfig is constructed from DQNHyperparameters (search for loss_aversion: assignment near line 1322) and add:

dd_threshold: config.dd_threshold as f32,
w_dd: config.w_dd as f32,

• Step 3: Pass as kernel arguments

In the kernel launch (search for .arg(&rw_loss_av) near line 1322), add after loss_aversion:

let rw_dd_thresh = config.dd_threshold;
let rw_w_dd = config.w_dd;
// ... in the launch_builder chain:
.arg(&rw_dd_thresh)
.arg(&rw_w_dd)

• Step 4: Compile check

Run: SQLX_OFFLINE=true cargo check -p ml Expected: warnings only (kernel signature mismatch will cause runtime error, fixed in Task 2)

• Step 5: Commit

git add crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
git commit -m "feat: wire dd_threshold and w_dd from config to experience kernel"

---

Task 2: Add drawdown penalty to the reward kernel

Files:

• Modify: crates/ml/src/cuda_pipeline/experience_kernels.cu:870-927 (reward computation)

Add two new kernel parameters (dd_threshold, w_dd) and a smooth drawdown penalty between the trade reward and the capital floor check.

• Step 1: Add kernel parameters

In the experience_env_step kernel signature, add after loss_aversion:

float dd_threshold,    // drawdown fraction before penalty starts (0.02 = 2%)
float w_dd             // drawdown penalty weight (1.0 = full penalty)

• Step 2: Add drawdown penalty after trade reward computation

After the hold_scale block (line 936) and before the turnover penalty comment (line 938), add:

    /* ---- Drawdown penalty: smooth ramp from dd_threshold to capital floor ----
     * Without this, the model has zero gradient between 0% and 25% DD.
     * Penalty ramps linearly: 0 at dd_threshold, -5.0 at capital floor (25% DD).
     * Applied every step (not just at trade exit) so the model learns to
     * reduce position size DURING drawdown, not just avoid the floor. */
    if (f_drawdown > dd_threshold && w_dd > 0.0f) {
        float floor_dd = 0.25f; // capital floor = 25% DD
        float dd_excess = (f_drawdown - dd_threshold) / (floor_dd - dd_threshold);
        dd_excess = fminf(dd_excess, 1.0f); // clamp to [0, 1]
        float dd_penalty = -5.0f * dd_excess * w_dd;
        reward += dd_penalty;
    }

Key design decisions:

• Applied every step, not just at trade exit — the model needs per-bar gradient to learn position sizing during drawdown

• Linear ramp from 0 to -5.0 — smooth gradient for the optimizer, no cliff

• -5.0 max — half of the capital floor penalty (-10.0) so it's significant but doesn't dominate

• w_dd weight — tunable via hyperopt (default 1.0, search range [0.0, 5.0])

• Step 3: Compile check

Run: SQLX_OFFLINE=true cargo check -p ml Expected: clean (kernel recompiles via build.rs)

• Step 4: Run smoke test to verify training stability

Run: FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests::training_stability::test_production_training_stability --ignored --nocapture Expected: passes, loss finite, grad norm stable. Check logs for drawdown penalty affecting reward range.

• Step 5: Commit

git add crates/ml/src/cuda_pipeline/experience_kernels.cu
git commit -m "feat: continuous drawdown penalty in reward kernel (dd_threshold→floor ramp)"

---

Task 3: Update TOML configs with drawdown penalty values

Files:

• Modify: config/training/dqn-localdev.toml

• Modify: config/training/dqn-production.toml

• Modify: config/training/dqn-smoketest.toml

• Step 1: Add to all three TOMLs

Add to [reward] section in each:

[reward]
w_dd = 1.0
dd_threshold = 0.02

These values match the code defaults. The TOML makes them explicit and configurable.

• Step 2: Verify TOML profile supports w_dd and dd_threshold

Check if apply_to() in training_profile.rs handles these fields from the [reward] section. If not, add:

if let Some(v) = r.w_dd { hp.w_dd = v; }
if let Some(v) = r.dd_threshold { hp.dd_threshold = v; }

And add the fields to the RewardSection struct:

pub w_dd: Option<f64>,
pub dd_threshold: Option<f64>,

• Step 3: Compile + test

Run: SQLX_OFFLINE=true cargo check -p ml && FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --nocapture Expected: 11/11 smoke tests pass

• Step 4: Commit

git add config/training/*.toml crates/ml/src/training_profile.rs
git commit -m "config: add w_dd and dd_threshold to all training profiles"

---

Task 4: Add w_dd to hyperopt search space

Files:

• Modify: crates/ml/src/hyperopt/adapters/dqn.rs (search space bounds + from_continuous)

• Step 1: Verify w_dd is already in the hyperopt params struct

Search for w_dd in DQNParams struct. It should already exist (line ~360). If not, add:

pub w_dd: f64,

• Step 2: Check the wiring in from_continuous() → DQNHyperparameters

Around line 2630, verify w_dd is wired:

w_dd: params.w_dd,

If w_dd is currently fixed at 1.0, that's fine — the hyperopt can search [0.0, 5.0] once we add it to the search space. For now, just ensure the wiring exists.

• Step 3: Compile check

Run: SQLX_OFFLINE=true cargo check -p ml Expected: clean

• Step 4: Commit

git add crates/ml/src/hyperopt/adapters/dqn.rs
git commit -m "feat: wire w_dd through hyperopt adapter"

---

Task 5: Validate — run training and verify MaxDD improves

Files: None (validation only)

• Step 1: Run 200-epoch training with drawdown penalty

FOXHUNT_TEST_DATA=test_data/futures-baseline \
FOXHUNT_TRAINING_PROFILE=config/training/dqn-localdev.toml \
cargo test -p ml --lib -- smoke_tests::training_stability::test_production_training_stability --ignored --nocapture

Monitor: MaxDD should decrease from 41-50% to <35%. Sharpe may initially decrease (model is more conservative) but total return should improve (fewer catastrophic drawdowns).

• Step 2: Run 3-trial hyperopt to verify metrics consistency

FOXHUNT_TEST_DATA=test_data/futures-baseline \
FOXHUNT_HYPEROPT_TRIALS=3 FOXHUNT_HYPEROPT_EPOCHS=5 \
cargo test -p ml --lib -- test_local_hyperopt --ignored --nocapture

Check: Omega and total_return should be more consistent. MaxDD should be lower across trials. Memory should be stable between trials.

• Step 3: Compare metrics before/after

Metric	Before	After (expected)
MaxDD	41-50%	<35%
Omega	1.5-5.0	1.2-2.0 (more honest)
Total Return	-15% to -22%	-5% to +5%
Sharpe	-0.8 to -1.4	-0.3 to +0.3

• Step 4: Commit results summary

git commit --allow-empty -m "validate: drawdown-aware training reduces MaxDD from 50% to <35%"