foxhunt/docs/superpowers/specs/2026-03-21-training-profiles-design.md

Training Profile Configuration System

Goal

Replace hardcoded hyperparameters across DQN, PPO, and supervised trainers with TOML configuration profiles following the same pattern as the existing GPU profiles (config/gpu/*.toml). All training behavior is driven by config, not code.

Architecture

Per-model-type TOML profiles with 3-tier loading (env var > filesystem > embedded defaults). CLI args always override TOML values. GPU profile hardware constraints (batch_size, num_atoms) take precedence over training profile values.

Directory Structure

config/training/
├── dqn-production.toml
├── dqn-smoketest.toml
├── dqn-hyperopt.toml
├── ppo-production.toml
├── ppo-smoketest.toml
├── supervised-production.toml
├── supervised-smoketest.toml
└── walk-forward.toml

Loading Priority

1. $FOXHUNT_TRAINING_PROFILE env var (exact file path, highest priority)
2. config/training/<profile>.toml relative to workspace root
3. Embedded compile-time defaults via include_str!

CLI args always override TOML. TOML overrides struct defaults.

TOML Schema

DQN Profile (dqn-production.toml)

[training]
epochs = 100
batch_size = 1024
learning_rate = 0.0003
gamma = 0.99
tau = 0.005
warmup_steps = 1000
gradient_clip_norm = 1.0
weight_decay = 0.0001
max_steps_per_epoch = 2000

[exploration]
epsilon_start = 1.0
epsilon_end = 0.01
epsilon_decay = 0.995

[replay_buffer]
buffer_size = 500000
min_replay_size = 1000
use_per = true
per_alpha = 0.6
per_beta_start = 0.4

[distributional]
use_distributional = true
num_atoms = 51
v_min = -25.0
v_max = 25.0

[branching]
use_branching = true
branch_0_size = 5
branch_1_size = 3
branch_2_size = 3

[advanced]
use_double_dqn = true
use_dueling = true
use_noisy_nets = true
noisy_sigma_init = 0.5
use_cql = true
cql_alpha = 0.1
gradient_accumulation_steps = 1

[risk]
enable_kelly_sizing = true
kelly_fractional = 0.5
kelly_max_fraction = 0.25
enable_action_masking = true
max_position_absolute = 2.0
enable_circuit_breaker = true

[early_stopping]
enabled = true
patience = 20
min_epochs_before_stopping = 10
min_loss_improvement_pct = 0.1

[experience]
initial_capital = 100000.0
tx_cost_multiplier = 0.0001
hold_reward = 0.001

DQN Smoke Test Profile (dqn-smoketest.toml)

[training]
epochs = 1
batch_size = 16
learning_rate = 0.001
warmup_steps = 0
max_steps_per_epoch = 50

[replay_buffer]
buffer_size = 1024
min_replay_size = 32

[early_stopping]
enabled = false

[experience]
gpu_n_episodes = 2
gpu_timesteps_per_episode = 50

Walk-Forward Profile (walk-forward.toml)

[walk_forward]
train_months = 12
val_months = 3
test_months = 3
step_months = 3
expanding_window = true

Hyperopt Profile (dqn-hyperopt.toml)

[search_space]
learning_rate = [0.00001, 0.01]
gamma = [0.95, 0.999]
tau = [0.001, 0.05]
batch_size = [32, 512]
buffer_size = [10000, 500000]
per_alpha = [0.4, 0.8]
per_beta_start = [0.2, 0.6]
epsilon_decay = [0.99, 0.999]
cql_alpha = [0.01, 0.5]
gradient_accumulation_steps = [1, 4]

[fixed]
use_branching = true
use_double_dqn = true
use_per = true
use_distributional = true

[pso]
swarm_size = 20
max_iterations = 50
inertia = 0.7
cognitive = 1.5
social = 1.5

Rust Implementation

Profile Loader (crates/ml/src/training_profile.rs)

use serde::Deserialize;

#[derive(Debug, Clone, Deserialize, Default)]
pub struct DqnTrainingProfile {
    pub training: Option<TrainingSection>,
    pub exploration: Option<ExplorationSection>,
    pub replay_buffer: Option<ReplayBufferSection>,
    pub distributional: Option<DistributionalSection>,
    pub branching: Option<BranchingSection>,
    pub advanced: Option<AdvancedSection>,
    pub risk: Option<RiskSection>,
    pub early_stopping: Option<EarlyStoppingSection>,
    pub experience: Option<ExperienceSection>,
}

impl DqnTrainingProfile {
    pub fn load(profile_name: &str) -> Self { ... }
    pub fn apply_to(&self, hp: &mut DQNHyperparameters) { ... }
}

Every field in every section is Option<T> — unset fields keep the struct default. apply_to() applies only the Some values, leaving the rest untouched. This enables sparse profiles (smoketest only overrides 8 fields, production sets 40+).

Integration Points

1. train_baseline_rl.rs: Add --training-profile CLI arg. Load profile, apply to hyperparams, then apply CLI overrides on top.

2. Smoke test helpers: smoke_params() loads dqn-smoketest.toml instead of hardcoding values.

3. Hyperopt adapters: Load dqn-hyperopt.toml for search space bounds instead of hardcoded arrays.

4. K8s job template: Add TRAINING_PROFILE env var alongside TRAINING_BINARY.

Merge Priority (highest wins)

CLI args > Training profile TOML > GPU profile TOML > Struct defaults

GPU profile batch_size and num_atoms cap the training profile values (hardware constraint).

Scope

In scope

• DQN training profiles (production, smoketest, hyperopt)
• PPO training profiles (production, smoketest)
• Supervised training profile (shared across 8 models)
• Walk-forward profile
• Profile loader with 3-tier resolution
• CLI --training-profile arg
• Smoke test helper migration to TOML
• K8s job template update

Also in scope

• PVC deployment: Training profile TOML files uploaded to the training-data PVC so K8s jobs can read them. Update populate-test-data-job.yaml to copy config/training/*.toml to /data/config/training/.
• Full CLI coverage: Every field in the TOML profiles must also be settable via CLI arg on train_baseline_rl and train_baseline_supervised. No config-only or CLI-only fields — all values accessible from both paths.
• fxt CLI integration: fxt train subcommand accepts --training-profile to select profile by name.

Out of scope

• Per-symbol profiles (ES.FUT vs NQ.FUT — same config, different data)
• Runtime profile switching (profiles are loaded once at startup)
• Web UI for profile editing
• Profile versioning / migration