# ML Hyperparameter Configurations

This directory contains canonical hyperparameter configurations for ML models based on hyperopt optimization results.

## Files

### `ppo_best.toml`
- **Source**: Hyperopt Trial #1 (Pod bpxgh10c5ocus5)
- **Date**: 2025-11-01
- **Objective**: 2.4023 (best of 63 trials)
- **Duration**: 14.3 minutes
- **Cost**: $0.06

**Key Parameters**:
- Policy LR: 1.0e-6 (ultra-conservative)
- Value LR: 0.001 (aggressive, 1000x higher)
- Clip Epsilon: 0.1126
- Entropy Coef: 0.006142

**Status**: ✅ Production-ready

### `dqn_best.toml`
- **Source**: Conservative defaults (awaiting hyperopt)
- **Date**: 2025-11-02
- **Status**: ⏳ Pending hyperopt deployment

**Current Parameters**:
- Learning Rate: 0.0001
- Batch Size: 128
- Gamma: 0.99
- Epsilon Decay: 0.995

**Note**: These are conservative defaults. Update after DQN hyperopt completes with action-dependent rewards.

## Usage

### In Code (Recommended)

Use the `conservative()` method for testing and development:

```rust
use ml::trainers::ppo::PpoHyperparameters;
use ml::trainers::dqn::DQNHyperparameters;

// PPO
let ppo_params = PpoHyperparameters::conservative();

// DQN
let dqn_params = DQNHyperparameters::conservative();
```

### Loading from TOML (Production)

For production deployments, load from TOML files:

```rust
use std::fs;
use toml;

// Load PPO hyperparameters
let ppo_config = fs::read_to_string("ml/hyperparams/ppo_best.toml")?;
let ppo_params: PpoHyperparameters = toml::from_str(&ppo_config)?;

// Load DQN hyperparameters
let dqn_config = fs::read_to_string("ml/hyperparams/dqn_best.toml")?;
let dqn_params: DQNHyperparameters = toml::from_str(&dqn_config)?;
```

## Why No Default Implementation?

The `Default` trait has been **removed** from `PpoHyperparameters` and `DQNHyperparameters` to prevent accidental use of suboptimal hyperparameters in production.

### Previous Issue (PPO)

Using `Default::default()` caused loss stagnation in production:
- Pod 0hczpx9nj1ub88: Loss stuck at 1.158-1.159 for 200+ epochs
- Root cause: Single learning rate (0.001) was 1000x too high for policy network
- Cost: ~$0.10 wasted compute

### Solution

1. **Development/Testing**: Use `::conservative()` method
2. **Production**: Load from TOML files (this directory)
3. **Optimization**: Run hyperopt to find optimal parameters

## Hyperopt History

### PPO Hyperopt
- **Date**: 2025-11-01
- **Pod**: bpxgh10c5ocus5 (EUR-IS-1, RTX A4000)
- **Trials**: 63 (target: 50)
- **Duration**: 14.3 minutes (99.8% faster than estimate)
- **Cost**: $0.06 (98.7% cheaper than estimate)
- **Best Trial**: #1 (objective: 2.4023)

### DQN Hyperopt
- **Status**: ⏳ Pending
- **Fix Applied**: Action-dependent rewards (2025-11-02)
- **Expected**: 50 trials, ~25 minutes, $0.12
- **Note**: Previous hyperopt produced identical objectives (bug fixed)

## Related Documentation

- **PPO**: `PPO_PARAMETERS_QUICK_REF.md` (root directory)
- **DQN**: `DQN_ACTION_DEPENDENT_REWARDS_FIX_SUMMARY.md` (root directory)
- **General**: `CLAUDE.md` (system architecture, hyperopt results)

## Updating Hyperparameters

After running hyperopt:

1. Identify best trial (highest objective for PPO, lowest for DQN)
2. Extract hyperparameters from trial results
3. Update corresponding `.toml` file
4. Document source (pod ID, trial number, objective score)
5. Update this README with new metadata

## Testing

All trainer tests use `::conservative()` method:

```bash
# Test PPO trainer
cargo test -p ml --lib trainers::ppo::tests

# Test DQN trainer
cargo test -p ml --lib trainers::dqn::tests

# All trainer tests
cargo test -p ml --lib trainers
```

**Status**: ✅ 42/42 trainer tests passing

---

**Last Updated**: 2025-11-02
**Maintainer**: ML Training Team