gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(cuda): asymmetric reward shaping — reward fast loss-cutting

Browse Source 
The short-hold penalty was backwards: it penalized ALL quick exits
including losing trades. Quick-exiting a loser is GOOD (surfer bails
early on bad waves), not bad.

Fix: short-hold penalty now only fires on WINNING quick exits.
New: quick-exit bonus on LOSING trades — reduces loss magnitude by
up to 50% for immediate exits. Surfer philosophy: bail early = small
wipeout, hold too long = big wipeout.

| Scenario            | Before     | After              |
|---------------------|------------|--------------------|
| Quick exit winner   | Penalized  | Penalized          |
| Quick exit loser    | Penalized  | 50% loss reduction |
| Long hold winner    | Bonus      | Bonus              |
| Long hold loser     | No bonus   | No bonus           |

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-27 23:05:22 +02:00
parent de378c5f62
commit ff8cacb0e9
1 changed files with 12 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
crates/ml-alpha/cuda/rl_fused_reward_pipeline.cu
View File

@@ -220,11 +220,21 @@ extern "C" __global__ void rl_fused_reward_pipeline(
r -= entry_cost;
}
// 2. Short-hold penalty: trade close with hold time below minimum.
if (done > 0.5f && (float)hold_time < min_hold) {
// 2. Short-hold penalty: ONLY on winning quick exits.
// Quick exit on a LOSER is GOOD (bail early on bad wave) — no penalty.
// Quick exit on a WINNER is BAD (bailed on a good wave) — penalize.
if (done > 0.5f && r > 0.0f && (float)hold_time < min_hold) {
r *= penalty;
}
// 2b. Quick-exit bonus on LOSING trades: reward fast loss-cutting.
// Reduces the loss magnitude proportional to how quickly the model
// exited. Surfer philosophy: bail early on closing-out waves.
if (done > 0.5f && r < 0.0f && (float)hold_time < min_hold) {
const float cut_bonus = 1.0f - 0.5f * ((min_hold - (float)hold_time) / min_hold);
r *= cut_bonus; // reduces loss by up to 50% for immediate exits
}
// 3. Long-ride bonus: profitable close amplified by hold time.
if (done > 0.5f && r > 0.0f && hold_time > 0) {
const float ride_mult = 1.0f + hold_bonus * sqrtf((float)hold_time);