foxhunt/docs/superpowers/plans/2026-03-21-training-profiles.md

Training Profile Configuration System

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace hardcoded training hyperparameters with TOML configuration profiles, extending the existing GPU profile pattern to cover all trainers, hyperopt, and walk-forward settings.

Architecture: Per-model-type TOML profiles loaded via 3-tier resolution (env var > filesystem > embedded defaults). All TOML fields are also exposed as CLI args. A TrainingProfile loader in ml crate applies profile values to existing DQNHyperparameters / PPO structs with Option<T> fields so sparse profiles work.

Tech Stack: Rust, serde + toml, clap (existing CLI framework), include_str! for embedded defaults

Spec: docs/superpowers/specs/2026-03-21-training-profiles-design.md

---

Files to Create

File	Responsibility
config/training/dqn-production.toml	Full production DQN config (all Rainbow features)
config/training/dqn-smoketest.toml	CI smoke test DQN (small network, 1 epoch)
config/training/dqn-hyperopt.toml	Hyperopt search space + fixed flags
config/training/ppo-production.toml	PPO production config
config/training/ppo-smoketest.toml	PPO smoke test
config/training/supervised-production.toml	Shared supervised model config
config/training/supervised-smoketest.toml	Supervised smoke test
config/training/walk-forward.toml	Walk-forward window defaults
crates/ml/src/training_profile.rs	Profile loader + section structs + apply_to()

Files to Delete

File	Reason
config/ml/training.toml	Replaced by per-model profiles in config/training/

Files to Modify

File	Changes
crates/ml/src/lib.rs	Add pub mod training_profile;
crates/ml/examples/train_baseline_rl.rs	Add --training-profile CLI arg, load + apply profile
crates/ml/examples/train_baseline_supervised.rs	Add --training-profile CLI arg
crates/ml/src/trainers/dqn/smoke_tests/helpers.rs	Load dqn-smoketest.toml instead of hardcoding
infra/k8s/training/populate-test-data-job.yaml	Copy config/training/*.toml to PVC
infra/k8s/training/job-template.yaml	Add TRAINING_PROFILE env var

---

Task 1: Create the profile loader module

Files:

• Create: crates/ml/src/training_profile.rs

• Step 1: Define the TOML section structs

All fields Option<T> — unset means "keep default":

use serde::Deserialize;
use std::sync::Arc;
use std::path::Path;

#[derive(Debug, Clone, Deserialize, Default)]
pub struct DqnTrainingProfile {
    pub training: Option<TrainingSection>,
    pub exploration: Option<ExplorationSection>,
    pub replay_buffer: Option<ReplayBufferSection>,
    pub distributional: Option<DistributionalSection>,
    pub branching: Option<BranchingSection>,
    pub advanced: Option<AdvancedSection>,
    pub risk: Option<RiskSection>,
    pub early_stopping: Option<EarlyStoppingSection>,
    pub experience: Option<ExperienceSection>,
    pub walk_forward: Option<WalkForwardSection>,
}

#[derive(Debug, Clone, Deserialize, Default)]
pub struct TrainingSection {
    pub epochs: Option<usize>,
    pub batch_size: Option<usize>,
    pub learning_rate: Option<f64>,
    pub gamma: Option<f64>,
    pub tau: Option<f64>,
    pub warmup_steps: Option<usize>,
    pub gradient_clip_norm: Option<f64>,
    pub weight_decay: Option<f64>,
    pub max_steps_per_epoch: Option<usize>,
    pub hidden_dim_base: Option<usize>,
}

// ... all other sections with Option<T> fields

• Step 2: Implement the 3-tier loader

Follow the same pattern as crates/ml-core/src/gpu/profile.rs:

const DQN_PRODUCTION_TOML: &str = include_str!("../../config/training/dqn-production.toml");
const DQN_SMOKETEST_TOML: &str = include_str!("../../config/training/dqn-smoketest.toml");
// ... etc

impl DqnTrainingProfile {
    pub fn load(profile_name: &str) -> Self {
        // 1. $FOXHUNT_TRAINING_PROFILE env var
        // 2. config/training/<profile>.toml filesystem
        // 3. include_str! embedded defaults
    }

    pub fn apply_to(&self, hp: &mut crate::trainers::dqn::DQNHyperparameters) {
        // For each Some(v) field, set the corresponding hp field
        if let Some(ref t) = self.training {
            if let Some(v) = t.epochs { hp.epochs = v; }
            if let Some(v) = t.batch_size { hp.batch_size = v; }
            // ... etc
        }
    }
}

• Step 3: Add unit tests

#[cfg(test)]
mod tests {
    #[test]
    fn test_load_dqn_production() {
        let p = DqnTrainingProfile::load("dqn-production");
        assert!(p.training.is_some());
        let t = p.training.unwrap();
        assert_eq!(t.epochs, Some(100));
    }

    #[test]
    fn test_sparse_profile_keeps_defaults() {
        // smoketest only sets a few fields
        let p = DqnTrainingProfile::load("dqn-smoketest");
        assert!(p.distributional.is_none()); // not set in smoketest
    }

    #[test]
    fn test_apply_to_overwrites_only_some_fields() {
        let mut hp = DQNHyperparameters::default();
        let original_gamma = hp.gamma;
        let p = DqnTrainingProfile::load("dqn-smoketest");
        p.apply_to(&mut hp);
        assert_eq!(hp.epochs, 1); // overridden by smoketest
        assert_eq!(hp.gamma, original_gamma); // not in smoketest, kept default
    }
}

• Step 4: Register module

Add pub mod training_profile; to crates/ml/src/lib.rs.

• Step 5: Verify

Run: SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile

• Step 6: Commit

git add crates/ml/src/training_profile.rs crates/ml/src/lib.rs
git commit -m "feat: training profile loader with 3-tier TOML resolution"

---

Task 2: Create the TOML profile files

Files:

• Create: config/training/dqn-production.toml

• Create: config/training/dqn-smoketest.toml

• Create: config/training/dqn-hyperopt.toml

• Create: config/training/ppo-production.toml

• Create: config/training/ppo-smoketest.toml

• Create: config/training/supervised-production.toml

• Create: config/training/supervised-smoketest.toml

• Create: config/training/walk-forward.toml

• Step 1: Create dqn-production.toml

Extract current hardcoded defaults from DQNHyperparameters::conservative() and smoke_params() in helpers.rs. Every field that exists in DQNHyperparameters gets a TOML entry.

• Step 2: Create dqn-smoketest.toml

Sparse profile — only the overrides currently in smoke_params():

[training]
epochs = 1
batch_size = 16
warmup_steps = 0
hidden_dim_base = 32
max_steps_per_epoch = 50

[replay_buffer]
buffer_size = 1024
min_replay_size = 32

[early_stopping]
enabled = false

[experience]
gpu_n_episodes = 2
gpu_timesteps_per_episode = 50

• Step 3: Create remaining profiles

PPO production/smoketest, supervised production/smoketest, walk-forward, hyperopt — all following the same pattern. Extract values from existing code defaults.

• Step 4: Verify profiles parse

Run: SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile

• Step 5: Commit

git add config/training/
git commit -m "feat: training profile TOML files for DQN, PPO, supervised, walk-forward"

---

Task 3: Wire profiles into train_baseline_rl CLI

Files:

• Modify: crates/ml/examples/train_baseline_rl.rs

• Step 1: Add --training-profile CLI arg

#[arg(long, default_value = "dqn-production")]
training_profile: String,

• Step 2: Load profile and apply to hyperparams

After CLI arg parsing, before trainer creation:

let profile = ml::training_profile::DqnTrainingProfile::load(&args.training_profile);
profile.apply_to(&mut hyperparams);
// CLI args override profile (existing hp_f64/hp_usize/hp_bool logic stays)

• Step 3: Add CLI args for ALL profile fields not yet exposed

Check every field in DqnTrainingProfile sections. For any field that doesn't already have a #[arg] in the Args struct, add one. This ensures full CLI coverage.

• Step 4: Verify

Run: SQLX_OFFLINE=true cargo check -p ml --example train_baseline_rl Run: SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- --help (verify all args appear)

• Step 5: Commit

git add crates/ml/examples/train_baseline_rl.rs
git commit -m "feat: --training-profile CLI arg for train_baseline_rl"

---

Task 4: Wire profiles into train_baseline_supervised CLI

Files:

• Modify: crates/ml/examples/train_baseline_supervised.rs

• Step 1: Add --training-profile CLI arg

Same pattern as Task 3. Default to supervised-production.

• Step 2: Load and apply profile

• Step 3: Verify + commit

---

Task 5: Migrate smoke test helpers to use TOML profiles

Files:

• Modify: crates/ml/src/trainers/dqn/smoke_tests/helpers.rs

• Step 1: Replace hardcoded smoke_params() with profile load

pub(super) fn smoke_params() -> DQNHyperparameters {
    let mut p = DQNHyperparameters::conservative();
    let profile = crate::training_profile::DqnTrainingProfile::load("dqn-smoketest");
    profile.apply_to(&mut p);
    // Keep production features that aren't in the TOML
    p.enable_gpu_experience_collector = true;
    p
}

• Step 2: Verify all smoke tests still pass

Run: SQLX_OFFLINE=true cargo test -p ml --lib -- smoke_tests Run: SQLX_OFFLINE=true FOXHUNT_TEST_DATA=test_data/futures-baseline cargo test -p ml --lib -- smoke_tests --ignored --test-threads=1

• Step 3: Commit

git add crates/ml/src/trainers/dqn/smoke_tests/helpers.rs
git commit -m "refactor: smoke test params loaded from dqn-smoketest.toml profile"

---

Task 6: Update K8s job template + PVC populate

Files:

• Modify: infra/k8s/training/job-template.yaml

• Modify: infra/k8s/training/populate-test-data-job.yaml

• Step 1: Add TRAINING_PROFILE env var to job template

- name: TRAINING_PROFILE
  value: "dqn-production"

And pass as --training-profile=$TRAINING_PROFILE in the args.

• Step 2: Copy training profiles to PVC in populate job

Add to the populate job script:

# Training profiles
mkdir -p /test-data/config/training
cp /cargo-target/src/config/training/*.toml /test-data/config/training/
echo "Training profiles: $(ls /test-data/config/training/ | wc -l) files copied"

• Step 3: Commit

git add infra/k8s/training/
git commit -m "infra: add TRAINING_PROFILE to K8s job + copy profiles to PVC"

---

Task 7: Profile and validate

• Step 1: Run tests

SQLX_OFFLINE=true cargo test -p ml --lib
SQLX_OFFLINE=true cargo test -p ml --lib -- training_profile

• Step 2: Run train_baseline_rl with profile

SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- \
  --training-profile dqn-smoketest \
  --data-dir test_data/futures-baseline --symbol ES.FUT \
  --epochs 2 --max-steps-per-epoch 20 \
  --train-months 3 --val-months 1 --test-months 1 --step-months 3

• Step 3: Verify --help shows all args

SQLX_OFFLINE=true cargo run --release --example train_baseline_rl -p ml -- --help

• Step 4: Commit + push

git add -A
git commit -m "feat: training profile system — TOML configs for all trainers"
git push origin main