spec: Phase 3 cost-driven hold timing — replaces min_hold_bars
Gem: per-trade implementation shortfall cost (already exists) Pearl: continuous inventory penalty (holding_cost_rate * |position| * dt) Novel: graduated churn penalty for rapid flips (not a hard gate) Removes: min_hold_bars, enforce_hold(), adaptive hold extension Adds: holding_cost_rate, churn_threshold_bars, churn_penalty_scale ISV shifts from hold enforcement to risk tolerance modulation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -566,3 +566,129 @@ fork-join events become fixed graph edges. As long as:
|
||||
|
||||
Each phase is additive — Phase 1 child graphs don't change, they just gain
|
||||
internal parallelism. The parent graph composition is unchanged.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Cost-Driven Hold Timing (replaces min_hold_bars)
|
||||
|
||||
### Problem
|
||||
|
||||
`min_hold_bars=50` is a hardcoded gate that blocks exits before 50 bars.
|
||||
`enforce_hold()` in `trade_physics.cuh` overrides the model's action. This
|
||||
prevents the temporal attention from learning hold timing — the model never
|
||||
sees the consequences of early exits because they're blocked.
|
||||
|
||||
### Design: Continuous Cost Signal
|
||||
|
||||
Remove `min_hold_bars` and `enforce_hold()` entirely. Replace with three
|
||||
cost components in the reward function:
|
||||
|
||||
```
|
||||
reward = base_reward
|
||||
- round_trip_cost * |trade_occurred| # per-trade cost
|
||||
- holding_cost * |position| * dt # continuous inventory penalty
|
||||
+ conviction * plan_scaling # existing plan head
|
||||
```
|
||||
|
||||
#### Component 1: Per-trade cost (Gem — Implementation Shortfall)
|
||||
|
||||
Each trade entry/exit incurs a fixed cost equal to the round-trip transaction
|
||||
cost (commission + spread). This is already implemented in `experience_env_step`
|
||||
via `compute_tx_cost()`. No change needed — just ensure it's always active.
|
||||
|
||||
The key: the cost is VISIBLE in the reward. The model learns "each trade
|
||||
costs X bps" and naturally reduces frequency to maximize net reward.
|
||||
|
||||
#### Component 2: Holding cost (Pearl — Continuous Inventory Penalty)
|
||||
|
||||
Add a per-bar cost proportional to position size:
|
||||
|
||||
```cuda
|
||||
// In experience_env_step, after position update:
|
||||
float holding_cost_rate = 0.0001f; // 0.01 bps per bar per unit position
|
||||
float inventory_penalty = holding_cost_rate * fabsf(position);
|
||||
reward -= inventory_penalty;
|
||||
```
|
||||
|
||||
This teaches:
|
||||
- **Small positions held long**: low cost per bar → profitable if directional
|
||||
- **Large positions**: high cost per bar → exit quickly unless conviction high
|
||||
- **Flat**: zero cost → model learns flat is the safe default
|
||||
|
||||
The rate (0.0001) is a hyperparameter that balances hold duration vs turnover.
|
||||
Too high → model never enters. Too low → model ignores holding cost.
|
||||
The temporal attention learns the optimal hold time for each rate.
|
||||
|
||||
Calibration: `holding_cost_rate ≈ round_trip_cost / target_hold_bars`.
|
||||
For 0.52 bps round-trip and 100-bar target hold: `0.52/100 = 0.0052 bps/bar`.
|
||||
In reward units: `0.0052 / 10000 = 0.00000052` per bar per unit.
|
||||
|
||||
#### Component 3: Churn penalty (Novel — TFJ-DRL dual signal)
|
||||
|
||||
Add a supervised penalty for position flips within a short window:
|
||||
|
||||
```cuda
|
||||
// Track bars_since_last_trade in portfolio state
|
||||
float bars_since = ps[10]; // hold_time
|
||||
float churn_threshold = 10.0f; // minimum bars between trades
|
||||
if (trade_occurred && bars_since < churn_threshold) {
|
||||
float churn_penalty = 0.001f * (churn_threshold - bars_since) / churn_threshold;
|
||||
reward -= churn_penalty;
|
||||
}
|
||||
```
|
||||
|
||||
This is NOT a hard gate (the model CAN exit early), but early exits get a
|
||||
graduated penalty. The penalty is zero after `churn_threshold` bars. The
|
||||
temporal attention learns that waiting past the threshold is free but
|
||||
exiting before costs extra.
|
||||
|
||||
### What Gets Deleted
|
||||
|
||||
- `min_hold_bars` field from `DQNHyperparameters`
|
||||
- `min_hold_bars` field from `GpuBacktestConfig`
|
||||
- `enforce_hold()` function from `trade_physics.cuh`
|
||||
- `enforce_hold` call in `experience_env_step`
|
||||
- `enforce_hold` call in backtest evaluator
|
||||
- `isv_signals` adaptive hold extension (ISV now modulates gamma/gating, not hold)
|
||||
- All `min_hold_bars` references across config files
|
||||
|
||||
### What Gets Added
|
||||
|
||||
In `experience_env_step` (`experience_kernels.cu`):
|
||||
- `inventory_penalty = holding_cost_rate * |position|` subtracted from reward
|
||||
- `churn_penalty` for rapid position flips
|
||||
|
||||
In `DQNHyperparameters` (`config.rs`):
|
||||
- `holding_cost_rate: f64` (default: 0.00000052, derived from tx_cost/target_hold)
|
||||
- `churn_threshold_bars: usize` (default: 10)
|
||||
- `churn_penalty_scale: f64` (default: 0.001)
|
||||
|
||||
In `GpuBacktestConfig`:
|
||||
- Same three parameters propagated to validation backtest
|
||||
|
||||
### Temporal Attention Role
|
||||
|
||||
With cost-driven hold timing, the temporal attention (Mamba2 + ISV) learns:
|
||||
- **When to enter**: conviction high enough to overcome round-trip cost
|
||||
- **How long to hold**: until holding cost exceeds expected remaining profit
|
||||
- **When to exit**: cost of staying > cost of round-trip to re-enter later
|
||||
|
||||
ISV signals shift from "extend hold time" to "modulate risk tolerance":
|
||||
- High grad_norm → reduce position size (not extend hold)
|
||||
- Low regime_stability → widen stops (not force hold)
|
||||
- Negative reward_ema → cut losses (not block exit)
|
||||
|
||||
The plan head's `target_bars` becomes a PREDICTION, not an enforcement.
|
||||
The model predicts how long it will hold, and the cost signals teach it
|
||||
whether that prediction was optimal.
|
||||
|
||||
### Implementation Order
|
||||
|
||||
Phase 3 runs AFTER Phase 1 validates (child graphs working, <80s epochs).
|
||||
It's a reward engineering change, not a graph/GPU change:
|
||||
|
||||
1. Add `holding_cost_rate` + `churn_penalty` to experience_env_step
|
||||
2. Remove `enforce_hold()` and `min_hold_bars`
|
||||
3. Add config parameters
|
||||
4. Propagate to backtest evaluator
|
||||
5. Validate: train 50 epochs, compare trade frequency and val_Sharpe vs Phase 1
|
||||
|
||||
Reference in New Issue
Block a user