gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: pass reward-shaping params from hyperopt to walk-forward DQN

Browse Source 
Hyperopt found hold_penalty_weight=1.818, curiosity_weight=0.402, etc.
but walk-forward training used tiny defaults (0.01, 0.1), causing the
agent to learn "holding is optimal" → 0 trades across all 50 epochs.

Now passes: hold_penalty_weight, max_position_absolute, huber_delta,
entropy_coefficient, curiosity_weight, weight_decay, kelly_fractional,
kelly_max_fraction from hyperopt JSON to DQNHyperparameters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-04 06:24:08 +01:00
parent 1d25d96f7e
commit d3c834eeb7
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
crates/ml/examples/train_baseline_rl.rs
View File

@@ -387,6 +387,15 @@ fn train_dqn_fold(
use_qr_dqn: hp_bool(hp, "use_qr_dqn").unwrap_or(false),
num_quantiles: hp_usize(hp, "num_quantiles").unwrap_or(64),
noisy_epsilon_floor: hp_f64(hp, "noisy_epsilon_floor").unwrap_or(0.05).into(),
// Reward-shaping & environment params — critical for trade generation
hold_penalty_weight: hp_f64(hp, "hold_penalty_weight").unwrap_or(0.01),
max_position_absolute: hp_f64(hp, "max_position_absolute").unwrap_or(2.0),
huber_delta: hp_f64(hp, "huber_delta").unwrap_or(10.0),
entropy_coefficient: hp_f64(hp, "entropy_coefficient").map(Some).unwrap_or(Some(0.01)),
curiosity_weight: hp_f64(hp, "curiosity_weight").unwrap_or(0.1),
weight_decay: hp_f64(hp, "weight_decay").unwrap_or(1e-4),
kelly_fractional: hp_f64(hp, "kelly_fractional").unwrap_or(0.5),
kelly_max_fraction: hp_f64(hp, "kelly_max_fraction").unwrap_or(0.25),
..DQNHyperparameters::default()
};