feat(rl): surfer-philosophy reward shaping — entry cost + hold bonus
Three ISV-driven reward shaping components discourage churning and reward patience: 1. Entry cost ($15 default): subtracted when opening a new position. Agent must expect profit > cost to justify entry. Slightly above ES 1-tick spread ($12.50) so marginal trades are net-negative. 2. Short-hold penalty (0.5× for holds < 20 steps): multiplicative penalty at trade close for quick flips. "Don't bail on the first bump" — halves the reward for sub-5-second holds. 3. Hold bonus ($0.50/step × sqrt(hold_time)): per-step reward for staying in a profitable position. "Ride the wave" — incentivizes patience when the trade is working. Runs BEFORE reward_scale so all costs/bonuses are in raw USD terms. ISV slots 532-535. RL_SLOTS_END → 536. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -87,6 +87,7 @@ const KERNELS: &[&str] = &[
|
||||
"rl_multires_features_update", // P0: per-batch multi-resolution streaming features (3 horizons × 4 features)
|
||||
"rl_encoder_context_broadcast", // P2: broadcast per-batch context (16 dims) into encoder input [B,K,56] at cols 40-55
|
||||
"rl_gate_threshold_controller", // adaptive gate thresholds from trade frequency (dones EMA)
|
||||
"rl_reward_shaping", // surfer-philosophy: entry cost + short-hold penalty + hold bonus
|
||||
];
|
||||
|
||||
// Cache bust v31 — five new reduce / derive kernels populate the input
|
||||
|
||||
Reference in New Issue
Block a user