feat(rl): surfer-philosophy reward shaping — entry cost + hold bonus

Three ISV-driven reward shaping components discourage churning and
reward patience:

1. Entry cost ($15 default): subtracted when opening a new position.
   Agent must expect profit > cost to justify entry. Slightly above
   ES 1-tick spread ($12.50) so marginal trades are net-negative.

2. Short-hold penalty (0.5× for holds < 20 steps): multiplicative
   penalty at trade close for quick flips. "Don't bail on the first
   bump" — halves the reward for sub-5-second holds.

3. Hold bonus ($0.50/step × sqrt(hold_time)): per-step reward for
   staying in a profitable position. "Ride the wave" — incentivizes
   patience when the trade is working.

Runs BEFORE reward_scale so all costs/bonuses are in raw USD terms.
ISV slots 532-535. RL_SLOTS_END → 536.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-25 12:12:49 +02:00
parent a1d35cd6e6
commit 60c714c8d1
4 changed files with 142 additions and 2 deletions

View File

@@ -87,6 +87,7 @@ const KERNELS: &[&str] = &[
"rl_multires_features_update", // P0: per-batch multi-resolution streaming features (3 horizons × 4 features)
"rl_encoder_context_broadcast", // P2: broadcast per-batch context (16 dims) into encoder input [B,K,56] at cols 40-55
"rl_gate_threshold_controller", // adaptive gate thresholds from trade frequency (dones EMA)
"rl_reward_shaping", // surfer-philosophy: entry cost + short-hold penalty + hold bonus
];
// Cache bust v31 — five new reduce / derive kernels populate the input