Three critical PPO fixes:
1. Add PPO::act_with_log_prob() returning (action, log_prob, value).
The existing act() discarded the policy log-probability, making
PPO importance sampling use wrong ratios during training.
2. Cap trajectory length with --max-steps-per-epoch in train_baseline
PPO path. DQN already had this limit; PPO iterated all features
(~500K per fold), causing OOM on 4GB GPU.
3. Replace random actions and fake log_prob/value in hyperopt PPO
adapter with real agent.act_with_log_prob() calls. Trajectories
now reflect actual policy behavior for meaningful hyperopt.
Also adds warmup offset alignment to PPO trajectory collection
(matching the DQN fix) and fixes .unwrap() in test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>