fix(cuda): disable hard stop-loss — dones override creates state mismatch
Setting dones[b]=1 in rl_drawdown_stop without actually closing the lobsim position created a lie: Q trained on false trade-closes while the position stayed open. Result: done_count=0 from step 99 onward, 990 trades total in 2000 steps (should be ~80k), training collapsed. The drawdown penalty (per-step min(0, unrealized_r) × rate) is kept — it creates exit gradient without corrupting the lobsim state. Hard stop-loss needs to override the ACTION (to FlatL/FlatS) BEFORE the lobsim step, not set dones after. Tracked for future implementation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -39,10 +39,11 @@ extern "C" __global__ void rl_drawdown_stop(
|
||||
rewards[b] += unrealized_r * penalty_rate;
|
||||
}
|
||||
|
||||
// Hard stop-loss: force-close when drawdown exceeds threshold.
|
||||
// unrealized_r is normalized by initial_r (trade-level vol), so
|
||||
// threshold is in units of "initial risk multiples."
|
||||
if (unrealized_r < -stop_thresh && dones[b] < 0.5f) {
|
||||
dones[b] = 1.0f;
|
||||
}
|
||||
// Hard stop-loss DISABLED: setting dones=1 here creates a state
|
||||
// mismatch — the lobsim position stays open while Q sees a false
|
||||
// close. The done signal must come from actual position changes
|
||||
// in the lobsim, not synthetic overrides. To implement hard
|
||||
// stop-loss properly, the action must be overridden to FlatL/FlatS
|
||||
// BEFORE the lobsim step, not after.
|
||||
(void) stop_thresh;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user