spec: Phase 3 cost-driven hold timing — replaces min_hold_bars

Gem: per-trade implementation shortfall cost (already exists)
Pearl: continuous inventory penalty (holding_cost_rate * |position| * dt)
Novel: graduated churn penalty for rapid flips (not a hard gate)

Removes: min_hold_bars, enforce_hold(), adaptive hold extension
Adds: holding_cost_rate, churn_threshold_bars, churn_penalty_scale
ISV shifts from hold enforcement to risk tolerance modulation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-18 00:27:40 +02:00
parent 74bdc58536
commit 874cff04a0

View File

@@ -566,3 +566,129 @@ fork-join events become fixed graph edges. As long as:
Each phase is additive — Phase 1 child graphs don't change, they just gain
internal parallelism. The parent graph composition is unchanged.
---
## Phase 3: Cost-Driven Hold Timing (replaces min_hold_bars)
### Problem
`min_hold_bars=50` is a hardcoded gate that blocks exits before 50 bars.
`enforce_hold()` in `trade_physics.cuh` overrides the model's action. This
prevents the temporal attention from learning hold timing — the model never
sees the consequences of early exits because they're blocked.
### Design: Continuous Cost Signal
Remove `min_hold_bars` and `enforce_hold()` entirely. Replace with three
cost components in the reward function:
```
reward = base_reward
- round_trip_cost * |trade_occurred| # per-trade cost
- holding_cost * |position| * dt # continuous inventory penalty
+ conviction * plan_scaling # existing plan head
```
#### Component 1: Per-trade cost (Gem — Implementation Shortfall)
Each trade entry/exit incurs a fixed cost equal to the round-trip transaction
cost (commission + spread). This is already implemented in `experience_env_step`
via `compute_tx_cost()`. No change needed — just ensure it's always active.
The key: the cost is VISIBLE in the reward. The model learns "each trade
costs X bps" and naturally reduces frequency to maximize net reward.
#### Component 2: Holding cost (Pearl — Continuous Inventory Penalty)
Add a per-bar cost proportional to position size:
```cuda
// In experience_env_step, after position update:
float holding_cost_rate = 0.0001f; // 0.01 bps per bar per unit position
float inventory_penalty = holding_cost_rate * fabsf(position);
reward -= inventory_penalty;
```
This teaches:
- **Small positions held long**: low cost per bar → profitable if directional
- **Large positions**: high cost per bar → exit quickly unless conviction high
- **Flat**: zero cost → model learns flat is the safe default
The rate (0.0001) is a hyperparameter that balances hold duration vs turnover.
Too high → model never enters. Too low → model ignores holding cost.
The temporal attention learns the optimal hold time for each rate.
Calibration: `holding_cost_rate ≈ round_trip_cost / target_hold_bars`.
For 0.52 bps round-trip and 100-bar target hold: `0.52/100 = 0.0052 bps/bar`.
In reward units: `0.0052 / 10000 = 0.00000052` per bar per unit.
#### Component 3: Churn penalty (Novel — TFJ-DRL dual signal)
Add a supervised penalty for position flips within a short window:
```cuda
// Track bars_since_last_trade in portfolio state
float bars_since = ps[10]; // hold_time
float churn_threshold = 10.0f; // minimum bars between trades
if (trade_occurred && bars_since < churn_threshold) {
float churn_penalty = 0.001f * (churn_threshold - bars_since) / churn_threshold;
reward -= churn_penalty;
}
```
This is NOT a hard gate (the model CAN exit early), but early exits get a
graduated penalty. The penalty is zero after `churn_threshold` bars. The
temporal attention learns that waiting past the threshold is free but
exiting before costs extra.
### What Gets Deleted
- `min_hold_bars` field from `DQNHyperparameters`
- `min_hold_bars` field from `GpuBacktestConfig`
- `enforce_hold()` function from `trade_physics.cuh`
- `enforce_hold` call in `experience_env_step`
- `enforce_hold` call in backtest evaluator
- `isv_signals` adaptive hold extension (ISV now modulates gamma/gating, not hold)
- All `min_hold_bars` references across config files
### What Gets Added
In `experience_env_step` (`experience_kernels.cu`):
- `inventory_penalty = holding_cost_rate * |position|` subtracted from reward
- `churn_penalty` for rapid position flips
In `DQNHyperparameters` (`config.rs`):
- `holding_cost_rate: f64` (default: 0.00000052, derived from tx_cost/target_hold)
- `churn_threshold_bars: usize` (default: 10)
- `churn_penalty_scale: f64` (default: 0.001)
In `GpuBacktestConfig`:
- Same three parameters propagated to validation backtest
### Temporal Attention Role
With cost-driven hold timing, the temporal attention (Mamba2 + ISV) learns:
- **When to enter**: conviction high enough to overcome round-trip cost
- **How long to hold**: until holding cost exceeds expected remaining profit
- **When to exit**: cost of staying > cost of round-trip to re-enter later
ISV signals shift from "extend hold time" to "modulate risk tolerance":
- High grad_norm → reduce position size (not extend hold)
- Low regime_stability → widen stops (not force hold)
- Negative reward_ema → cut losses (not block exit)
The plan head's `target_bars` becomes a PREDICTION, not an enforcement.
The model predicts how long it will hold, and the cost signals teach it
whether that prediction was optimal.
### Implementation Order
Phase 3 runs AFTER Phase 1 validates (child graphs working, <80s epochs).
It's a reward engineering change, not a graph/GPU change:
1. Add `holding_cost_rate` + `churn_penalty` to experience_env_step
2. Remove `enforce_hold()` and `min_hold_bars`
3. Add config parameters
4. Propagate to backtest evaluator
5. Validate: train 50 epochs, compare trade frequency and val_Sharpe vs Phase 1