foxhunt/docs/superpowers/plans/2026-03-21-training-profiles.md

# Training Profile Configuration System

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Replace hardcoded training hyperparameters with TOML configuration profiles, extending the existing GPU profile pattern to cover all trainers, hyperopt, and walk-forward settings.

**Architecture:** Per-model-type TOML profiles loaded via 3-tier resolution (env var > filesystem > embedded defaults). All TOML fields are also exposed as CLI args. A `TrainingProfile` loader in `ml` crate applies profile values to existing `DQNHyperparameters` / PPO structs with `Option<T>` fields so sparse profiles work.

**Tech Stack:** Rust, serde + toml, clap (existing CLI framework), include_str! for embedded defaults

**Spec:** `docs/superpowers/specs/2026-03-21-training-profiles-design.md`

---

## Files to Create

| File | Responsibility |
|------|---------------|
| `config/training/dqn-production.toml` | Full production DQN config (all Rainbow features) |
| `config/training/dqn-smoketest.toml` | CI smoke test DQN (small network, 1 epoch) |
| `config/training/dqn-hyperopt.toml` | Hyperopt search space + fixed flags |
| `config/training/ppo-production.toml` | PPO production config |
| `config/training/ppo-smoketest.toml` | PPO smoke test |
| `config/training/supervised-production.toml` | Shared supervised model config |
| `config/training/supervised-smoketest.toml` | Supervised smoke test |
| `config/training/walk-forward.toml` | Walk-forward window defaults |
| `crates/ml/src/training_profile.rs` | Profile loader + section structs + apply_to() |

## Files to Delete

| File | Reason |
|------|--------|
| `config/ml/training.toml` | Replaced by per-model profiles in `config/training/` |

## Files to Modify

| File | Changes |
|------|---------|
| `crates/ml/src/lib.rs` | Add `pub mod training_profile;` |
| `crates/ml/examples/train_baseline_rl.rs` | Add `--training-profile` CLI arg, load + apply profile |
| `crates/ml/examples/train_baseline_supervised.rs` | Add `--training-profile` CLI arg |
| `crates/ml/src/trainers/dqn/smoke_tests/helpers.rs` | Load `dqn-smoketest.toml` instead of hardcoding |
| `infra/k8s/training/populate-test-data-job.yaml` | Copy `config/training/*.toml` to PVC |
| `infra/k8s/training/job-template.yaml` | Add `TRAINING_PROFILE` env var |

---

### Task 1: Create the profile loader module

**Files:**
- Create: `crates/ml/src/training_profile.rs`

- [ ] **Step 1: Define the TOML section structs**

All fields `Option<T>` — unset means "keep default":

```rust
use serde::Deserialize;
use std::sync::Arc;
use std::path::Path;

#[derive(Debug, Clone, Deserialize, Default)]
pub struct DqnTrainingProfile {
    pub training: Option<TrainingSection>,
    pub exploration: Option<ExplorationSection>,
    pub replay_buffer: Option<ReplayBufferSection>,
    pub distributional: Option<DistributionalSection>,
    pub branching: Option<BranchingSection>,
    pub advanced: Option<AdvancedSection>,
    pub risk: Option<RiskSection>,
    pub early_stopping: Option<EarlyStoppingSection>,
    pub experience: Option<ExperienceSection>,
    pub walk_forward: Option<WalkForwardSection>,
}

#[derive(Debug, Clone, Deserialize, Default)]
pub struct TrainingSection {
    pub epochs: Option<usize>,
    pub batch_size: Option<usize>,
    pub learning_rate: Option<f64>,
    pub gamma: Option<f64>,
    pub tau: Option<f64>,
    pub warmup_steps: Option<usize>,
    pub gradient_clip_norm: Option<f64>,
    pub weight_decay: Option<f64>,
    pub max_steps_per_epoch: Option<usize>,
    pub hidden_dim_base: Option<usize>,
}

// ... all other sections with Option<T> fields
```

- [ ] **Step 2: Implement the 3-tier loader**

Follow the same pattern as `crates/ml-core/src/gpu/profile.rs`:
```rust
const DQN_PRODUCTION_TOML: &str = include_str!("../../config/training/dqn-production.toml");
const DQN_SMOKETEST_TOML: &str = include_str!("../../config/training/dqn-smoketest.toml");
// ... etc

impl DqnTrainingProfile {
    pub fn load(profile_name: &str) -> Self {
        // 1. $FOXHUNT_TRAINING_PROFILE env var
        // 2. config/training/<profile>.toml filesystem
        // 3. include_str! embedded defaults
    }

    pub fn apply_to(&self, hp: &mut crate::trainers::dqn::DQNHyperparameters) {
        // For each Some(v) field, set the corresponding hp field
        if let Some(ref t) = self.training {
            if let Some(v) = t.epochs { hp.epochs = v; }
            if let Some(v) = t.batch_size { hp.batch_size = v; }
            // ... etc
        }
    }
}
```

- [ ] **Step 3: Add unit tests**

```rust
#[cfg(test)]
mod tests {
    #[test]
    fn test_load_dqn_production() {
        let p = DqnTrainingProfile::load("dqn-production");
        assert!(p.training.is_some());
        let t = p.training.unwrap();
        assert_eq!(t.epochs, Some(100));
    }

    #[test]
    fn test_sparse_profile_keeps_defaults() {
        // smoketest only sets a few fields
        let p = DqnTrainingProfile::load("dqn-smoketest");
        assert!(p.distributional.is_none()); // not set in smoketest
    }

    #[test]
    fn test_apply_to_overwrites_only_some_fields() {
        let mut hp = DQNHyperparameters::default();
        let original_gamma = hp.gamma;
        let p = DqnTrainingProfile::load("dqn-smoketest");
        p.apply_to(&mut hp);
        assert_eq!(hp.epochs, 1); // overridden by smoketest
        assert_eq!(hp.gamma, original_gamma); // not in smoketest, kept default
    }
}
```

- [ ] **Step 4: Register module**

Add `pub mod training_profile;` to `crates/ml/src/lib.rs`.

- [ ] **Step 5: Verify**

Run: `SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile`

- [ ] **Step 6: Commit**

```bash
git add crates/ml/src/training_profile.rs crates/ml/src/lib.rs
git commit -m "feat: training profile loader with 3-tier TOML resolution"
```

---

### Task 2: Create the TOML profile files

**Files:**
- Create: `config/training/dqn-production.toml`
- Create: `config/training/dqn-smoketest.toml`
- Create: `config/training/dqn-hyperopt.toml`
- Create: `config/training/ppo-production.toml`
- Create: `config/training/ppo-smoketest.toml`
- Create: `config/training/supervised-production.toml`
- Create: `config/training/supervised-smoketest.toml`
- Create: `config/training/walk-forward.toml`

- [ ] **Step 1: Create `dqn-production.toml`**

Extract current hardcoded defaults from `DQNHyperparameters::conservative()` and `smoke_params()` in helpers.rs. Every field that exists in DQNHyperparameters gets a TOML entry.

- [ ] **Step 2: Create `dqn-smoketest.toml`**

Sparse profile — only the overrides currently in `smoke_params()`:
```toml
[training]
epochs = 1
batch_size = 16
warmup_steps = 0
hidden_dim_base = 32
max_steps_per_epoch = 50

[replay_buffer]
buffer_size = 1024
min_replay_size = 32

[early_stopping]
enabled = false

[experience]
gpu_n_episodes = 2
gpu_timesteps_per_episode = 50
```

- [ ] **Step 3: Create remaining profiles**

PPO production/smoketest, supervised production/smoketest, walk-forward, hyperopt — all following the same pattern. Extract values from existing code defaults.

- [ ] **Step 4: Verify profiles parse**

Run: `SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile`

- [ ] **Step 5: Commit**

```bash
git add config/training/
git commit -m "feat: training profile TOML files for DQN, PPO, supervised, walk-forward"
```

---

### Task 3: Wire profiles into train_baseline_rl CLI

**Files:**
- Modify: `crates/ml/examples/train_baseline_rl.rs`

- [ ] **Step 1: Add `--training-profile` CLI arg**

```rust
#[arg(long, default_value = "dqn-production")]
training_profile: String,
```

- [ ] **Step 2: Load profile and apply to hyperparams**

After CLI arg parsing, before trainer creation:
```rust
let profile = ml::training_profile::DqnTrainingProfile::load(&args.training_profile);
profile.apply_to(&mut hyperparams);
// CLI args override profile (existing hp_f64/hp_usize/hp_bool logic stays)
```

- [ ] **Step 3: Add CLI args for ALL profile fields not yet exposed**

Check every field in `DqnTrainingProfile` sections. For any field that doesn't already have a `#[arg]` in the Args struct, add one. This ensures full CLI coverage.

- [ ] **Step 4: Verify**

Run: `SQLX_OFFLINE=true cargo check -p ml --example train_baseline_rl`
Run: `SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- --help` (verify all args appear)

- [ ] **Step 5: Commit**

```bash
git add crates/ml/examples/train_baseline_rl.rs
git commit -m "feat: --training-profile CLI arg for train_baseline_rl"
```

---

### Task 4: Wire profiles into train_baseline_supervised CLI

**Files:**
- Modify: `crates/ml/examples/train_baseline_supervised.rs`

- [ ] **Step 1: Add `--training-profile` CLI arg**

Same pattern as Task 3. Default to `supervised-production`.

- [ ] **Step 2: Load and apply profile**

- [ ] **Step 3: Verify + commit**

---

### Task 5: Migrate smoke test helpers to use TOML profiles

**Files:**
- Modify: `crates/ml/src/trainers/dqn/smoke_tests/helpers.rs`

- [ ] **Step 1: Replace hardcoded `smoke_params()` with profile load**

```rust
pub(super) fn smoke_params() -> DQNHyperparameters {
    let mut p = DQNHyperparameters::conservative();
    let profile = crate::training_profile::DqnTrainingProfile::load("dqn-smoketest");
    profile.apply_to(&mut p);
    // Keep production features that aren't in the TOML
    p.enable_gpu_experience_collector = true;
    p
}
```

- [ ] **Step 2: Verify all smoke tests still pass**

Run: `SQLX_OFFLINE=true cargo test -p ml --lib -- smoke_tests`
Run: `SQLX_OFFLINE=true FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --test-threads=1`

- [ ] **Step 3: Commit**

```bash
git add crates/ml/src/trainers/dqn/smoke_tests/helpers.rs
git commit -m "refactor: smoke test params loaded from dqn-smoketest.toml profile"
```

---

### Task 6: Update K8s job template + PVC populate

**Files:**
- Modify: `infra/k8s/training/job-template.yaml`
- Modify: `infra/k8s/training/populate-test-data-job.yaml`

- [ ] **Step 1: Add TRAINING_PROFILE env var to job template**

```yaml
- name: TRAINING_PROFILE
  value: "dqn-production"
```

And pass as `--training-profile=$TRAINING_PROFILE` in the args.

- [ ] **Step 2: Copy training profiles to PVC in populate job**

Add to the populate job script:
```bash
# Training profiles
mkdir -p /test-data/config/training
cp /cargo-target/src/config/training/*.toml /test-data/config/training/
echo "Training profiles: $(ls /test-data/config/training/ | wc -l) files copied"
```

- [ ] **Step 3: Commit**

```bash
git add infra/k8s/training/
git commit -m "infra: add TRAINING_PROFILE to K8s job + copy profiles to PVC"
```

---

### Task 7: Profile and validate

- [ ] **Step 1: Run tests**

```bash
SQLX_OFFLINE=true cargo test -p ml --lib
SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile
```

- [ ] **Step 2: Run train_baseline_rl with profile**

```bash
SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- \
  --training-profile dqn-smoketest \
  --data-dir test_data/futures-baseline --symbol ES.FUT \
  --epochs 2 --max-steps-per-epoch 20 \
  --train-months 3 --val-months 1 --test-months 1 --step-months 3
```

- [ ] **Step 3: Verify --help shows all args**

```bash
SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- --help
```

- [ ] **Step 4: Commit + push**

```bash
git add -A
git commit -m "feat: training profile system — TOML configs for all trainers"
git push origin main
```