Files
foxhunt/ml/src/hyperopt
jgrusewski 24e72b370e fix(ml/ppo): use real policy actions and cap trajectory length to prevent OOM
Three critical PPO fixes:

1. Add PPO::act_with_log_prob() returning (action, log_prob, value).
   The existing act() discarded the policy log-probability, making
   PPO importance sampling use wrong ratios during training.

2. Cap trajectory length with --max-steps-per-epoch in train_baseline
   PPO path. DQN already had this limit; PPO iterated all features
   (~500K per fold), causing OOM on 4GB GPU.

3. Replace random actions and fake log_prob/value in hyperopt PPO
   adapter with real agent.act_with_log_prob() calls. Trajectories
   now reflect actual policy behavior for meaningful hyperopt.

Also adds warmup offset alignment to PPO trajectory collection
(matching the DQN fix) and fixes .unwrap() in test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 00:52:02 +01:00
..