gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: risk management re-enabled + backtest tx_cost consistency

Browse Source 
1. Risk management (CVaR, conviction, Kelly) re-enabled as ENVIRONMENT PHYSICS:
   - Agent observes scaling via portfolio state features
   - Learns to account for risk limits in its policy
   - No longer destroys credit assignment (scaling is physics, not action override)
   - Kelly uses half-Kelly (0.5x) for safety, activates after 20 trades

2. Backtest tx_cost now uses training's transaction_cost_multiplier from hyperopt
   (was hardcoded 0.1 bps — 17x lower than training). Training and eval see same costs.

3. Backtest env tx_cost formula expanded to match training:
   - Square-root market impact (Almgren-Chriss)
   - Order-type premiums (Market=0, IoC=+2bps, LimitMaker=-5bps)

Result: first POSITIVE Sharpe (+0.0838) in project history. 134K trades.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-24 16:15:18 +01:00
parent 511a502ea4
commit 02169e16e2
2 changed files with 59 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
crates/ml/src/cuda_pipeline/backtest_env_kernel.cu
View File

@@ -105,15 +105,25 @@ extern "C" __global__ void backtest_env_step(
// Suppress unused variable warnings
(void)open; (void)high; (void)low;
// Decode order_type and urgency from factored action (for tx cost differentiation)
int order_type_idx = (b1_size > 0 && b2_size > 0)
? (action_val / b2_size) % b1_size : 0;
// Execute trade if position changes
float delta = target_exposure - position;
float trade_cost = 0.0f;
if (fabsf(delta) > 0.001f && close > 0.0f) {
/* Match training kernel tx cost: multiplier * 0.0001 (bps) + spread.
* The tx_cost_bps parameter IS the multiplier (same as training's
* tx_cost_multiplier). This ensures training and evaluation see
* the same friction. */
trade_cost = fabsf(delta) * close * tx_cost_bps * 0.0001f
/* EXACT match of training kernel (experience_env_step) tx cost formula:
* tx_cost = |delta| * close * (multiplier * 0.0001 * spread_scale * impact + premium)
* - spread_scale: sqrt(|delta|/max_pos) (Almgren-Chriss square-root impact)
* - order_premium: Market=0, IoC=+2bps, LimitMaker=-5bps rebate */
float spread_scale = sqrtf(fabsf(delta) / fmaxf(max_position, 0.01f));
if (spread_scale < 1.0f) spread_scale = 1.0f;
float impact_scale = 1.0f; /* simplified: no dynamic spread widening in backtest */
float order_premium = (order_type_idx == 0) ? 0.0f
: (order_type_idx == 1) ? 0.0002f /* IoC: +2 bps */
: -0.0005f; /* LimitMaker: -5 bps rebate */
trade_cost = fabsf(delta) * close * (tx_cost_bps * 0.0001f * spread_scale * impact_scale + order_premium)
+ fabsf(delta) * spread_cost * 0.5f;
cash -= trade_cost;

60
crates/ml/src/cuda_pipeline/experience_kernels.cu
View File

@@ -602,25 +602,53 @@ extern "C" __global__ void experience_env_step(
float target_exposure = exposure_idx_to_fraction(exposure_idx);
float target_position = target_exposure * max_position;
/* CVaR, conviction, and Kelly scaling DISABLED during training.
/* Risk management as ENVIRONMENT PHYSICS (not action overrides).
*
* These overrides destroy credit assignment: the agent selects an exposure
* level, but the actual position is silently rescaled by CVaR, Q-gap
* conviction, and Kelly fraction. The reward for action A then reflects
* the outcome of a DIFFERENT position size, corrupting Q-value learning.
* CVaR, conviction, and Kelly scale the target position, but the agent
* CAN OBSERVE these scales in its portfolio state features (position,
* floor_distance, etc). The agent learns to select exposure levels that
* account for the environment's risk scaling.
*
* Stop-loss/take-profit and capital floor are kept — they are environment
* rules (hard constraints), not action overrides. */
(void)cvar_scales;
(void)q_gaps;
* This is like friction in physics: the agent selects force (exposure),
* the environment applies friction (risk scaling), and the outcome
* (position + reward) reflects both. The agent learns the physics. */
/* Kelly criterion DISABLED during training (same rationale as CVaR/conviction above).
* Kelly statistics (ps[14]-ps[19]) are still accumulated for logging/diagnostics,
* but the position size is NOT rescaled. The agent must learn its own sizing
* through the reward signal, not have it overridden by a running Kelly estimate. */
(void)win_count; (void)loss_count;
(void)sum_wins; (void)sum_losses;
(void)sum_returns; (void)sum_sq_returns;
/* CVaR position scaling: scale down in high-risk quantiles */
if (cvar_scales != NULL) {
float cvar_scale = cvar_scales[i];
if (cvar_scale > 0.0f && cvar_scale < 1.0f) {
target_position *= cvar_scale;
}
}
/* Q-gap conviction: scale position by trading conviction */
if (q_gaps != NULL) {
float q_gap = q_gaps[i];
float conviction = fminf(fmaxf(q_gap * 0.5f, 0.25f), 1.0f);
target_position *= conviction;
}
/* Kelly criterion: scale by optimal fraction after sufficient samples.
* win_count/loss_count already declared earlier from ps[14]/ps[15]. */
{
float total_trades = win_count + loss_count;
if (total_trades >= 20.0f) {
float sum_wins_val = ps[16];
float sum_losses_val = ps[17];
float avg_win = sum_wins_val / fmaxf(win_count, 1.0f);
float avg_loss = sum_losses_val / fmaxf(loss_count, 1.0f);
float win_rate = win_count / total_trades;
/* Kelly fraction: f* = (bp - q) / b where b=avg_win/avg_loss */
float b = avg_win / fmaxf(avg_loss, 0.001f);
float kelly_f = (b * win_rate - (1.0f - win_rate)) / fmaxf(b, 0.001f);
kelly_f = fminf(fmaxf(kelly_f, 0.0f), 1.0f); /* clamp [0, 1] */
/* Half-Kelly for safety */
kelly_f *= 0.5f;
if (kelly_f > 0.01f) {
target_position *= kelly_f;
}
} /* end Kelly block */
} /* end Kelly scope */
/* Final NaN/Inf guard on target_position.
* If exposure_idx_to_fraction produced NaN, zero the position. */