fix(cuda): disable hard stop-loss — dones override creates state mismatch

Setting dones[b]=1 in rl_drawdown_stop without actually closing the
lobsim position created a lie: Q trained on false trade-closes while
the position stayed open. Result: done_count=0 from step 99 onward,
990 trades total in 2000 steps (should be ~80k), training collapsed.

The drawdown penalty (per-step min(0, unrealized_r) × rate) is kept —
it creates exit gradient without corrupting the lobsim state. Hard
stop-loss needs to override the ACTION (to FlatL/FlatS) BEFORE the
lobsim step, not set dones after. Tracked for future implementation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-27 22:17:12 +02:00
parent 35d21cb42f
commit fab9c0e324

View File

@@ -39,10 +39,11 @@ extern "C" __global__ void rl_drawdown_stop(
rewards[b] += unrealized_r * penalty_rate;
}
// Hard stop-loss: force-close when drawdown exceeds threshold.
// unrealized_r is normalized by initial_r (trade-level vol), so
// threshold is in units of "initial risk multiples."
if (unrealized_r < -stop_thresh && dones[b] < 0.5f) {
dones[b] = 1.0f;
}
// Hard stop-loss DISABLED: setting dones=1 here creates a state
// mismatch — the lobsim position stays open while Q sees a false
// close. The done signal must come from actual position changes
// in the lobsim, not synthetic overrides. To implement hard
// stop-loss properly, the action must be overridden to FlatL/FlatS
// BEFORE the lobsim step, not after.
(void) stop_thresh;
}