foxhunt/ml/hyperparams/README.md

ML Hyperparameter Configurations

This directory contains canonical hyperparameter configurations for ML models based on hyperopt optimization results.

Files

ppo_best.toml

• Source: Hyperopt Trial #1 (Pod bpxgh10c5ocus5)
• Date: 2025-11-01
• Objective: 2.4023 (best of 63 trials)
• Duration: 14.3 minutes
• Cost: $0.06

Key Parameters:

• Policy LR: 1.0e-6 (ultra-conservative)
• Value LR: 0.001 (aggressive, 1000x higher)
• Clip Epsilon: 0.1126
• Entropy Coef: 0.006142

Status: ✅ Production-ready

dqn_best.toml

• Source: Conservative defaults (awaiting hyperopt)
• Date: 2025-11-02
• Status: ⏳ Pending hyperopt deployment

Current Parameters:

• Learning Rate: 0.0001
• Batch Size: 128
• Gamma: 0.99
• Epsilon Decay: 0.995

Note: These are conservative defaults. Update after DQN hyperopt completes with action-dependent rewards.

Usage

In Code (Recommended)

Use the conservative() method for testing and development:

use ml::trainers::ppo::PpoHyperparameters;
use ml::trainers::dqn::DQNHyperparameters;

// PPO
let ppo_params = PpoHyperparameters::conservative();

// DQN
let dqn_params = DQNHyperparameters::conservative();

Loading from TOML (Production)

For production deployments, load from TOML files:

use std::fs;
use toml;

// Load PPO hyperparameters
let ppo_config = fs::read_to_string("ml/hyperparams/ppo_best.toml")?;
let ppo_params: PpoHyperparameters = toml::from_str(&ppo_config)?;

// Load DQN hyperparameters
let dqn_config = fs::read_to_string("ml/hyperparams/dqn_best.toml")?;
let dqn_params: DQNHyperparameters = toml::from_str(&dqn_config)?;

Why No Default Implementation?

The Default trait has been removed from PpoHyperparameters and DQNHyperparameters to prevent accidental use of suboptimal hyperparameters in production.

Previous Issue (PPO)

Using Default::default() caused loss stagnation in production:

• Pod 0hczpx9nj1ub88: Loss stuck at 1.158-1.159 for 200+ epochs
• Root cause: Single learning rate (0.001) was 1000x too high for policy network
• Cost: ~$0.10 wasted compute

Solution

1. Development/Testing: Use ::conservative() method
2. Production: Load from TOML files (this directory)
3. Optimization: Run hyperopt to find optimal parameters

Hyperopt History

PPO Hyperopt

• Date: 2025-11-01
• Pod: bpxgh10c5ocus5 (EUR-IS-1, RTX A4000)
• Trials: 63 (target: 50)
• Duration: 14.3 minutes (99.8% faster than estimate)
• Cost: $0.06 (98.7% cheaper than estimate)
• Best Trial: #1 (objective: 2.4023)

DQN Hyperopt

• Status: ⏳ Pending
• Fix Applied: Action-dependent rewards (2025-11-02)
• Expected: 50 trials, ~25 minutes, $0.12
• Note: Previous hyperopt produced identical objectives (bug fixed)

Related Documentation

• PPO: PPO_PARAMETERS_QUICK_REF.md (root directory)
• DQN: DQN_ACTION_DEPENDENT_REWARDS_FIX_SUMMARY.md (root directory)
• General: CLAUDE.md (system architecture, hyperopt results)

Updating Hyperparameters

After running hyperopt:

1. Identify best trial (highest objective for PPO, lowest for DQN)
2. Extract hyperparameters from trial results
3. Update corresponding .toml file
4. Document source (pod ID, trial number, objective score)
5. Update this README with new metadata

Testing

All trainer tests use ::conservative() method:

# Test PPO trainer
cargo test -p ml --lib trainers::ppo::tests

# Test DQN trainer
cargo test -p ml --lib trainers::dqn::tests

# All trainer tests
cargo test -p ml --lib trainers

Status: ✅ 42/42 trainer tests passing

---

Last Updated: 2025-11-02 Maintainer: ML Training Team