foxhunt/AGENT_177_QUICK_REFERENCE.md

# Agent 177: PPO Checkpoint Loading - Quick Reference

## TL;DR

✅ **PPO checkpoint loading is now production-ready**

- Real checkpoint loading (Agent 170 validated)
- 4/4 integration tests passing
- CUDA GPU support enabled
- Zero-downtime hot-swap

---

## Quick Start

### Load Single PPO Model

```rust
use ml::ensemble::EnsembleCoordinator;

let coordinator = EnsembleCoordinator::new();

coordinator.load_ppo_checkpoint(
    "PPO_epoch420",
    "ml/trained_models/production/ppo/ppo_actor_epoch_420.safetensors",
    "ml/trained_models/production/ppo/ppo_critic_epoch_420.safetensors",
    0.33,
).await?;
```

### Make Prediction

```rust
use ml::Features;

let features = Features::new(
    vec![0.5, 0.6, 0.7, 0.8, 0.9],
    vec!["price_momentum", "volume", "volatility", "spread", "rsi"]
        .iter().map(|s| s.to_string()).collect(),
);

let decision = coordinator.predict(&features).await?;
```

---

## Available Checkpoints

```
ml/trained_models/production/ppo/
├── ppo_actor_epoch_420.safetensors   ⭐ Primary (best)
├── ppo_critic_epoch_420.safetensors
├── ppo_actor_epoch_130.safetensors   🔄 Fallback
└── ppo_critic_epoch_130.safetensors
```

---

## Common Patterns

### Multi-Model Ensemble

```rust
// Load PPO (33% weight)
coordinator.load_ppo_checkpoint("PPO", actor, critic, 0.33).await?;

// Register DQN (33% weight)
coordinator.register_model("DQN".to_string(), 0.33).await?;

// Register TFT (34% weight)
coordinator.register_model("TFT".to_string(), 0.34).await?;

// Get ensemble prediction
let decision = coordinator.predict(&features).await?;
```

### Hot-Swap Model

```rust
// Same model_id triggers hot-swap
coordinator.load_ppo_checkpoint("PPO_active", actor1, critic1, 0.5).await?;
// ... later ...
coordinator.load_ppo_checkpoint("PPO_active", actor2, critic2, 0.5).await?;
// Zero downtime!
```

---

## Test Validation

```bash
# Run integration tests
cargo test -p ml --test integration_ppo_ensemble --release

# Expected: 4/4 passing
✅ test_ppo_checkpoint_loading_in_ensemble
✅ test_ppo_ensemble_with_multiple_models
✅ test_ppo_hot_swap
✅ test_ppo_checkpoint_path_validation
```

---

## Performance

| Operation | Latency |
|-----------|---------|
| Checkpoint loading | ~100-500ms (one-time) |
| PPO inference | <100μs |
| Ensemble aggregation | ~5-10μs |
| Hot-swap | <100ms (0ms downtime) |

**Memory**: ~150MB per PPO checkpoint

**GPU**: RTX 3050 Ti (auto-detected) or CPU fallback

---

## API Reference

### `EnsembleCoordinator::load_ppo_checkpoint()`

```rust
pub async fn load_ppo_checkpoint(
    &self,
    model_id: &str,              // Unique identifier
    actor_checkpoint: &str,       // Path to actor safetensors
    critic_checkpoint: &str,      // Path to critic safetensors
    weight: f64,                  // Ensemble weight (0.0-1.0)
) -> MLResult<()>
```

### `EnsembleCoordinator::predict()`

```rust
pub async fn predict(
    &self,
    features: &Features,
) -> MLResult<EnsembleDecision>
```

**Returns**: `EnsembleDecision` with:
- `action`: Buy/Sell/Hold
- `confidence`: 0.0-1.0
- `signal`: -1.0 to 1.0
- `disagreement_rate`: 0.0-1.0
- `model_votes`: HashMap of individual votes

---

## Configuration

### PPO Config (in code)

```rust
PPOConfig {
    state_dim: 16,
    num_actions: 3,
    policy_hidden_dims: vec![256, 128],
    value_hidden_dims: vec![256, 128],
    policy_learning_rate: 0.0003,
    value_learning_rate: 0.001,
    clip_epsilon: 0.2,
    value_loss_coeff: 0.5,
    entropy_coeff: 0.01,
    gae_config: GAEConfig {
        gamma: 0.99,
        lambda: 0.95,
        normalize_advantages: true,
    },
    batch_size: 64,
    mini_batch_size: 32,
    num_epochs: 10,
    max_grad_norm: 0.5,
}
```

---

## Troubleshooting

### Issue: Checkpoint not found
```
Error: Failed to load PPO checkpoint: Checkpoint not found
```
**Solution**: Verify checkpoint paths exist:
```bash
ls -lh ml/trained_models/production/ppo/
```

### Issue: CUDA out of memory
```
Error: CUDA out of memory
```
**Solution**: Reduce batch size or use CPU:
```rust
let device = candle_core::Device::Cpu;
```

### Issue: Model not registered
```
Error: Model not found in ensemble
```
**Solution**: Ensure `load_ppo_checkpoint()` completed successfully

---

## Next Steps

1. **Agent 178**: Integrate into paper trading executor
2. **Agent 179**: Add DQN checkpoint loading
3. **Agent 180**: Add TFT checkpoint loading

---

## Related Documents

- `AGENT_177_SUMMARY.md` - Comprehensive implementation details
- `AGENT_177_INTEGRATION_COMPLETE.md` - Full validation report
- `AGENT_170_SUMMARY.md` - PPO checkpoint loading foundation

---

**Status**: ✅ Production Ready  
**Tests**: 4/4 passing (100%)  
**Build**: ✅ Success