From 874cff04a05e721dffa1a304a0cd29ea752136a4 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sat, 18 Apr 2026 00:27:40 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20Phase=203=20cost-driven=20hold=20timing?= =?UTF-8?q?=20=E2=80=94=20replaces=20min=5Fhold=5Fbars?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Gem: per-trade implementation shortfall cost (already exists) Pearl: continuous inventory penalty (holding_cost_rate * |position| * dt) Novel: graduated churn penalty for rapid flips (not a hard gate) Removes: min_hold_bars, enforce_hold(), adaptive hold extension Adds: holding_cost_rate, churn_threshold_bars, churn_penalty_scale ISV shifts from hold enforcement to risk tolerance modulation Co-Authored-By: Claude Opus 4.6 (1M context) --- ...17-unified-cublas-training-graph-design.md | 126 ++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/docs/superpowers/specs/2026-04-17-unified-cublas-training-graph-design.md b/docs/superpowers/specs/2026-04-17-unified-cublas-training-graph-design.md index 40ce799e2..8045b23a9 100644 --- a/docs/superpowers/specs/2026-04-17-unified-cublas-training-graph-design.md +++ b/docs/superpowers/specs/2026-04-17-unified-cublas-training-graph-design.md @@ -566,3 +566,129 @@ fork-join events become fixed graph edges. As long as: Each phase is additive — Phase 1 child graphs don't change, they just gain internal parallelism. The parent graph composition is unchanged. + +--- + +## Phase 3: Cost-Driven Hold Timing (replaces min_hold_bars) + +### Problem + +`min_hold_bars=50` is a hardcoded gate that blocks exits before 50 bars. +`enforce_hold()` in `trade_physics.cuh` overrides the model's action. This +prevents the temporal attention from learning hold timing — the model never +sees the consequences of early exits because they're blocked. + +### Design: Continuous Cost Signal + +Remove `min_hold_bars` and `enforce_hold()` entirely. Replace with three +cost components in the reward function: + +``` +reward = base_reward + - round_trip_cost * |trade_occurred| # per-trade cost + - holding_cost * |position| * dt # continuous inventory penalty + + conviction * plan_scaling # existing plan head +``` + +#### Component 1: Per-trade cost (Gem — Implementation Shortfall) + +Each trade entry/exit incurs a fixed cost equal to the round-trip transaction +cost (commission + spread). This is already implemented in `experience_env_step` +via `compute_tx_cost()`. No change needed — just ensure it's always active. + +The key: the cost is VISIBLE in the reward. The model learns "each trade +costs X bps" and naturally reduces frequency to maximize net reward. + +#### Component 2: Holding cost (Pearl — Continuous Inventory Penalty) + +Add a per-bar cost proportional to position size: + +```cuda +// In experience_env_step, after position update: +float holding_cost_rate = 0.0001f; // 0.01 bps per bar per unit position +float inventory_penalty = holding_cost_rate * fabsf(position); +reward -= inventory_penalty; +``` + +This teaches: +- **Small positions held long**: low cost per bar → profitable if directional +- **Large positions**: high cost per bar → exit quickly unless conviction high +- **Flat**: zero cost → model learns flat is the safe default + +The rate (0.0001) is a hyperparameter that balances hold duration vs turnover. +Too high → model never enters. Too low → model ignores holding cost. +The temporal attention learns the optimal hold time for each rate. + +Calibration: `holding_cost_rate ≈ round_trip_cost / target_hold_bars`. +For 0.52 bps round-trip and 100-bar target hold: `0.52/100 = 0.0052 bps/bar`. +In reward units: `0.0052 / 10000 = 0.00000052` per bar per unit. + +#### Component 3: Churn penalty (Novel — TFJ-DRL dual signal) + +Add a supervised penalty for position flips within a short window: + +```cuda +// Track bars_since_last_trade in portfolio state +float bars_since = ps[10]; // hold_time +float churn_threshold = 10.0f; // minimum bars between trades +if (trade_occurred && bars_since < churn_threshold) { + float churn_penalty = 0.001f * (churn_threshold - bars_since) / churn_threshold; + reward -= churn_penalty; +} +``` + +This is NOT a hard gate (the model CAN exit early), but early exits get a +graduated penalty. The penalty is zero after `churn_threshold` bars. The +temporal attention learns that waiting past the threshold is free but +exiting before costs extra. + +### What Gets Deleted + +- `min_hold_bars` field from `DQNHyperparameters` +- `min_hold_bars` field from `GpuBacktestConfig` +- `enforce_hold()` function from `trade_physics.cuh` +- `enforce_hold` call in `experience_env_step` +- `enforce_hold` call in backtest evaluator +- `isv_signals` adaptive hold extension (ISV now modulates gamma/gating, not hold) +- All `min_hold_bars` references across config files + +### What Gets Added + +In `experience_env_step` (`experience_kernels.cu`): +- `inventory_penalty = holding_cost_rate * |position|` subtracted from reward +- `churn_penalty` for rapid position flips + +In `DQNHyperparameters` (`config.rs`): +- `holding_cost_rate: f64` (default: 0.00000052, derived from tx_cost/target_hold) +- `churn_threshold_bars: usize` (default: 10) +- `churn_penalty_scale: f64` (default: 0.001) + +In `GpuBacktestConfig`: +- Same three parameters propagated to validation backtest + +### Temporal Attention Role + +With cost-driven hold timing, the temporal attention (Mamba2 + ISV) learns: +- **When to enter**: conviction high enough to overcome round-trip cost +- **How long to hold**: until holding cost exceeds expected remaining profit +- **When to exit**: cost of staying > cost of round-trip to re-enter later + +ISV signals shift from "extend hold time" to "modulate risk tolerance": +- High grad_norm → reduce position size (not extend hold) +- Low regime_stability → widen stops (not force hold) +- Negative reward_ema → cut losses (not block exit) + +The plan head's `target_bars` becomes a PREDICTION, not an enforcement. +The model predicts how long it will hold, and the cost signals teach it +whether that prediction was optimal. + +### Implementation Order + +Phase 3 runs AFTER Phase 1 validates (child graphs working, <80s epochs). +It's a reward engineering change, not a graph/GPU change: + +1. Add `holding_cost_rate` + `churn_penalty` to experience_env_step +2. Remove `enforce_hold()` and `min_hold_bars` +3. Add config parameters +4. Propagate to backtest evaluator +5. Validate: train 50 epochs, compare trade frequency and val_Sharpe vs Phase 1