feat(dqn-v2): D.4c conviction consistency bonus — reward stable pre-entry deliberation
Plan 3 Task 6c. Portfolio-state tail-append: - PS_PRE_ENTRY_CONVICTION_EMA = 41 (EMA mean of conviction_core during Flat) - PS_PRE_ENTRY_CONVICTION_VAR_EMA = 42 (EMA of squared deviations) - PS_STRIDE 41 -> 43 - All 6 hardcoded-stride sites migrated in lockstep Producer (experience_kernels.cu Flat branch): - Per-bar EMA update alpha=0.05 (matches Task 1 reward-ema convention) - Welford-style: delta = c - mean; var_ema = (1-alpha)*(var + alpha*delta^2) Consumer (experience_kernels.cu entering_trade block): - ratio = stddev/mean; stability = clamp(0, 1, 1 - ratio/0.2) - Fires only when ratio < 0.2 (stable pre-entry conviction) - bonus = shaping x vol_proxy x stability x conviction_core - All multiplicands in [0,1] except vol_proxy (<=0.01); max bonus ~ 0.01 - Mirrors B.2 novelty-bonus structure — one bounded shape replaced (novelty -> stability) - rc[5] += bonus; both EMA slots reset at entry, reversal, fold/episode boundary Per pearl_one_unbounded_signal_per_reward.md: exactly ONE unbounded multiplicand (vol_proxy); all others bounded. No `q_scale x |reward|` style blowout possible. No new ISV slot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
@@ -96,10 +96,12 @@ pub const PS_PLAN_CONVICTION: usize = 27; // plan: conviction at entry [0,
|
||||
pub const PS_PLAN_ASYMMETRY: usize = 28; // plan: profit/stop asymmetry
|
||||
pub const PS_PLAN_ENTRY_REGIME: usize = 29; // plan: regime_stability at entry
|
||||
pub const PS_OFI_PREV_BASE: usize = 30; // first OFI prev-bar slot (stride 8: [30..37])
|
||||
pub const PS_PEAK_PNL_BAR: usize = 38; // Plan 3 C.4: hold_time at which MAX_PNL was hit (temporal timing bonus)
|
||||
pub const PS_INTRA_TRADE_MIN_PNL: usize = 39; // Plan 3 D.4a: worst unrealised PnL during trade (symmetric to MAX_PNL at 21)
|
||||
pub const PS_REGIME_SHIFT_BAR: usize = 40; // Plan 3 D.4b: hold_time at first regime-shift detection (0 if no shift)
|
||||
pub const PS_STRIDE: usize = 41; // total portfolio state stride (== PORTFOLIO_STRIDE), grown 40→41 by Plan 3 D.4b
|
||||
pub const PS_PEAK_PNL_BAR: usize = 38; // Plan 3 C.4: hold_time at which MAX_PNL was hit (temporal timing bonus)
|
||||
pub const PS_INTRA_TRADE_MIN_PNL: usize = 39; // Plan 3 D.4a: worst unrealised PnL during trade (symmetric to MAX_PNL at 21)
|
||||
pub const PS_REGIME_SHIFT_BAR: usize = 40; // Plan 3 D.4b: hold_time at first regime-shift detection (0 if no shift)
|
||||
pub const PS_PRE_ENTRY_CONVICTION_EMA: usize = 41; // Plan 3 D.4c: EMA of conviction_core during Flat bars
|
||||
pub const PS_PRE_ENTRY_CONVICTION_VAR_EMA: usize = 42; // Plan 3 D.4c: EMA of (conviction_core − mean)² during Flat bars
|
||||
pub const PS_STRIDE: usize = 43; // total portfolio state stride (== PORTFOLIO_STRIDE), grown 41→43 by Plan 3 D.4c conviction consistency
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
// Mirrors of PLAN_ISV_* constants (plan_isv[0..PLAN_ISV_DIM=7)).
|
||||
|
||||
@@ -59,10 +59,10 @@
|
||||
*/
|
||||
|
||||
/* ------------------------------------------------------------------ */
|
||||
/* Portfolio stride for experience kernels (41 floats per episode). */
|
||||
/* Portfolio stride for experience kernels (43 floats per episode). */
|
||||
/* portfolio_sim_kernel uses its own stride of 8 — do NOT change it. */
|
||||
/* ------------------------------------------------------------------ */
|
||||
#define PORTFOLIO_STRIDE 41 /* was 40. Slots 30-37: prev-bar OFI; slot 38: PS_PEAK_PNL_BAR (Plan 3 C.4); slot 39: PS_INTRA_TRADE_MIN_PNL (Plan 3 D.4a); slot 40: PS_REGIME_SHIFT_BAR (Plan 3 D.4b) */
|
||||
#define PORTFOLIO_STRIDE 43 /* was 41. Slots 30-37: prev-bar OFI; slot 38: PS_PEAK_PNL_BAR (Plan 3 C.4); slot 39: PS_INTRA_TRADE_MIN_PNL (Plan 3 D.4a); slot 40: PS_REGIME_SHIFT_BAR (Plan 3 D.4b); slots 41-42: PS_PRE_ENTRY_CONVICTION_{EMA,VAR_EMA} (Plan 3 D.4c) */
|
||||
#define DSR_A_SLOT 3
|
||||
#define DSR_B_SLOT 4
|
||||
#define DSR_TRADE_COUNT_SLOT 5
|
||||
@@ -1852,10 +1852,12 @@ extern "C" __global__ void experience_env_step(
|
||||
ps[PS_PLAN_ASYMMETRY] = pp[PLAN_PARAM_ASYMMETRY]; /* asymmetry */
|
||||
/* Store entry regime_stability for drift detection (P12). */
|
||||
ps[PS_PLAN_ENTRY_REGIME] = (isv_signals_ptr != NULL) ? isv_signals_ptr[11] : 1.0f;
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for new trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for new trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
/* Apply conviction to position size — scale by readiness for smooth ramp */
|
||||
position *= fmaxf(ps[PS_PLAN_CONVICTION] * readiness, 0.1f);
|
||||
}
|
||||
@@ -2012,17 +2014,21 @@ extern "C" __global__ void experience_env_step(
|
||||
float intra_trade_max_dd = (ps[PS_INTRA_TRADE_MAX_DD]);
|
||||
if (entering_trade) {
|
||||
intra_trade_max_dd = 0.0f;
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for new trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for new trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
}
|
||||
if (reversing_trade) {
|
||||
intra_trade_max_dd = 0.0f;
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for reversed trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_INTRA_TRADE_MAX_PNL] = 0.0f; /* reset max P&L for reversed trade */
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: reset peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: reset MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f; /* Plan 3 D.4c: next Flat epoch starts fresh */
|
||||
}
|
||||
if (fabsf(position) > 0.001f && entry_price > 0.0f) {
|
||||
float unrealized = (position > 0.0f)
|
||||
@@ -2395,6 +2401,31 @@ extern "C" __global__ void experience_env_step(
|
||||
/* Fallback: original holding cost when micro_reward_scale=0 or no OFI */
|
||||
reward = -shaping_scale * holding_cost_rate * fabsf(position);
|
||||
} else if (!segment_complete) {
|
||||
/* Plan 3 D.4c: Welford-style EMA of conviction_core during Flat bars.
|
||||
* Updates running mean and squared-deviation EMAs that the consumer
|
||||
* (entry-bonus block below) uses to detect stable pre-entry conviction.
|
||||
*
|
||||
* α = 0.05 matches the Plan 3 Task 1 reward-component EMA convention —
|
||||
* not a tuned constant per task; same α_base used for all Plan 3 EMAs.
|
||||
*
|
||||
* Per-bar update:
|
||||
* delta = conviction_core − prev_mean
|
||||
* new_mean = prev_mean + α × delta
|
||||
* new_var = (1 − α) × (prev_var + α × delta²) // EMA-style, not exact Welford
|
||||
*
|
||||
* Runs only when conviction_ptr is available; otherwise skip cleanly
|
||||
* (cold-start or test-fixture safety). Reset to 0 at trade entry,
|
||||
* trade reversal, and fold/episode reset (see lifecycle blocks). */
|
||||
if (conviction_ptr != NULL) {
|
||||
const float alpha_c = 0.05f;
|
||||
const float prev_m = ps[PS_PRE_ENTRY_CONVICTION_EMA];
|
||||
const float prev_v = ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA];
|
||||
const float delta = conviction_core - prev_m;
|
||||
const float new_m = prev_m + alpha_c * delta;
|
||||
const float new_v = (1.0f - alpha_c) * (prev_v + alpha_c * delta * delta);
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = new_m;
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = new_v;
|
||||
}
|
||||
/* ── Flat opportunity cost — ISV-driven self-adapting via conviction ──
|
||||
* (val-Flat-collapse fix #3 revision, 2026-04-24)
|
||||
*
|
||||
@@ -2505,6 +2536,62 @@ extern "C" __global__ void experience_env_step(
|
||||
}
|
||||
}
|
||||
|
||||
/* ── D.4c (Plan 3 Task 6c): Conviction consistency bonus ──
|
||||
*
|
||||
* Fires on Flat→Positioned entry when the network's pre-entry conviction
|
||||
* has been STABLE across the Flat run (stddev/mean < 0.2). Rewards
|
||||
* deliberated entries vs noise-spike entries.
|
||||
*
|
||||
* Mirrors B.2 novelty-bonus structure exactly:
|
||||
* bonus = shaping × vol_proxy × stability × conviction_core
|
||||
*
|
||||
* All multiplicands are in [0,1] except vol_proxy (bounded ≤ 0.01).
|
||||
* Max bonus ≈ 0.01 — same order as B.2. Per
|
||||
* pearl_one_unbounded_signal_per_reward.md: exactly ONE technically
|
||||
* unbounded multiplicand (vol_proxy); all others bounded.
|
||||
*
|
||||
* stability mapping:
|
||||
* ratio = stddev / mean (sentinel 2.0 when mean ≈ 0 → disables bonus)
|
||||
* ratio < 0.2 ⇒ stable ⇒ stability = clamp(0, 1, 1 − ratio/0.2)
|
||||
* ratio ≥ 0.2 ⇒ skip
|
||||
*
|
||||
* The 0.2 threshold is the single tuned constant retained for this first
|
||||
* cut. Reviewer note: a follow-up could derive it from
|
||||
* ISV[Q_DIR_ABS_REF] for full ISV-driven shaping; deferred to keep this
|
||||
* task surgical. The `ratio/0.2` normalisation inside `1 − ratio/0.2`
|
||||
* makes stability=1 mean "perfectly stable" (ratio=0) and stability→0
|
||||
* as ratio→0.2 (margin), so credit is proportional to how stable the
|
||||
* pre-entry epoch was rather than a flat threshold pass/fail.
|
||||
*
|
||||
* Attribution: rc[5] += bonus (co-exists with B.2/C.4/D.4a/D.4b at
|
||||
* different (i,t) slots; the same ISV[68] REWARD_BONUS_EMA tracks the
|
||||
* net rc[5] signal). */
|
||||
if (entering_trade && isv_signals_ptr != NULL
|
||||
&& features != NULL && bar_idx < total_bars && market_dim > 9) {
|
||||
const float mean_c = ps[PS_PRE_ENTRY_CONVICTION_EMA];
|
||||
const float var_c = ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA];
|
||||
const float stddev_c = sqrtf(fmaxf(0.0f, var_c));
|
||||
const float ratio = (mean_c > 1e-6f) ? (stddev_c / mean_c) : 2.0f;
|
||||
if (ratio < 0.2f) {
|
||||
const float stability = fmaxf(0.0f, fminf(1.0f, 1.0f - ratio / 0.2f));
|
||||
const float atr_norm_c = features[(long long)bar_idx * market_dim + 9];
|
||||
const float log_atr_c = atr_norm_c * 16.0f - 7.0f;
|
||||
const float atr_pct_c = expf(log_atr_c) / fmaxf(raw_close, 1.0f);
|
||||
float vol_proxy_c = fmaxf(atr_pct_c, 0.0001f);
|
||||
if (vol_proxy_c > 0.01f) vol_proxy_c = 0.01f;
|
||||
const float bonus = shaping_scale * vol_proxy_c * stability * conviction_core;
|
||||
reward += bonus;
|
||||
reward_components_per_sample[out_off * 6 + 5] += bonus;
|
||||
}
|
||||
}
|
||||
/* Defensive reset of D.4c EMA slots at entry — redundant with the
|
||||
* lifecycle resets at the entering_trade handlers above (lines ~1858 and
|
||||
* ~2018), but cheap and harmless. The next Flat epoch starts fresh. */
|
||||
if (entering_trade) {
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f;
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f;
|
||||
}
|
||||
|
||||
/* ---- Drawdown penalty (from trade_physics.cuh) ──────────────────────
|
||||
* Phase 3: gated by shaping_scale. The drawdown penalty is *behavioral*
|
||||
* (steers the policy away from large equity drops); the *physics* of
|
||||
@@ -2848,6 +2935,8 @@ extern "C" __global__ void experience_env_step(
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: peak-bar alongside MAX_PNL */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: MIN_PNL alongside MAX_PNL */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f; /* Plan 3 D.4c: hard-reset Flat-epoch EMA */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f; /* Plan 3 D.4c: hard-reset Flat-epoch EMA */
|
||||
ps[PS_PLAN_TARGET_BARS] = 0.0f; ps[PS_PLAN_PROFIT_TARGET] = 0.0f; ps[PS_PLAN_STOP_LOSS] = 0.0f;
|
||||
ps[PS_PLAN_SCALE_AGGRESSION] = 0.0f; ps[PS_PLAN_CONVICTION] = 0.0f; ps[PS_PLAN_ASYMMETRY] = 0.0f;
|
||||
ps[PS_PLAN_ENTRY_REGIME] = 0.0f; /* plan slots zeroed */
|
||||
@@ -2865,6 +2954,8 @@ extern "C" __global__ void experience_env_step(
|
||||
ps[PS_PEAK_PNL_BAR] = 0.0f; /* Plan 3 C.4: peak-bar reset for next trade */
|
||||
ps[PS_INTRA_TRADE_MIN_PNL] = 0.0f; /* Plan 3 D.4a: MIN_PNL reset for next trade */
|
||||
ps[PS_REGIME_SHIFT_BAR] = 0.0f; /* Plan 3 D.4b: reset at trade lifecycle boundary */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_EMA] = 0.0f; /* Plan 3 D.4c: soft-reset Flat-epoch EMA */
|
||||
ps[PS_PRE_ENTRY_CONVICTION_VAR_EMA] = 0.0f; /* Plan 3 D.4c: soft-reset Flat-epoch EMA */
|
||||
ps[PS_PLAN_TARGET_BARS] = 0.0f; ps[PS_PLAN_PROFIT_TARGET] = 0.0f; ps[PS_PLAN_STOP_LOSS] = 0.0f;
|
||||
ps[PS_PLAN_SCALE_AGGRESSION] = 0.0f; ps[PS_PLAN_CONVICTION] = 0.0f; ps[PS_PLAN_ASYMMETRY] = 0.0f;
|
||||
ps[PS_PLAN_ENTRY_REGIME] = 0.0f; /* plan slots zeroed on trade complete */
|
||||
|
||||
@@ -10656,7 +10656,7 @@ impl GpuDqnTrainer {
|
||||
) -> Result<(), MLError> {
|
||||
let isv_ptr = self.isv_signals_dev_ptr;
|
||||
let isv_out = isv_ptr; // kernel writes to the same ISV bus (offset 44)
|
||||
let ps_stride: i32 = 41; // PS_STRIDE from state_layout.cuh (grown 40→41 by Plan 3 D.4b)
|
||||
let ps_stride: i32 = 43; // PS_STRIDE from state_layout.cuh (grown 41→43 by Plan 3 D.4c)
|
||||
unsafe {
|
||||
self.stream
|
||||
.launch_builder(&self.kelly_cap_update_kernel)
|
||||
|
||||
@@ -41,10 +41,10 @@ const MAX_EPISODES_LIMIT: usize = 0x8000;
|
||||
/// Absolute upper bound for validation — reject configs above this.
|
||||
const MAX_TIMESTEPS_LIMIT: usize = 1000;
|
||||
const PORTFOLIO_STATE_SIZE: usize = 8;
|
||||
/// Portfolio stride for DQN experience kernels (41 floats per episode).
|
||||
/// Portfolio stride for DQN experience kernels (43 floats per episode).
|
||||
/// Matches PORTFOLIO_STRIDE in experience_kernels.cu and trade_stats_kernel.cu.
|
||||
/// Do NOT change PORTFOLIO_STATE_SIZE above — it's for the PPO/legacy path.
|
||||
const PORTFOLIO_STRIDE: usize = 41; // was 40. Slots 30-37: prev-bar OFI; slot 38: PS_PEAK_PNL_BAR (Plan 3 C.4); slot 39: PS_INTRA_TRADE_MIN_PNL (Plan 3 D.4a); slot 40: PS_REGIME_SHIFT_BAR (Plan 3 D.4b)
|
||||
const PORTFOLIO_STRIDE: usize = 43; // was 41. Slots 30-37: prev-bar OFI; slot 38: PS_PEAK_PNL_BAR (Plan 3 C.4); slot 39: PS_INTRA_TRADE_MIN_PNL (Plan 3 D.4a); slot 40: PS_REGIME_SHIFT_BAR (Plan 3 D.4b); slots 41-42: PS_PRE_ENTRY_CONVICTION_{EMA,VAR_EMA} (Plan 3 D.4c)
|
||||
|
||||
/// Number of floats in the trade_stats_reduce output buffer.
|
||||
/// Layout: [win_count, loss_count, sum_wins, sum_losses, sum_returns, sum_sq_returns]
|
||||
@@ -568,7 +568,7 @@ pub struct GpuExperienceCollector {
|
||||
trade_stats_buf: CudaSlice<f32>,
|
||||
|
||||
// Per-episode state buffers [alloc_episodes, ...]
|
||||
portfolio_states: CudaSlice<f32>, // [alloc_episodes * PORTFOLIO_STRIDE] (41 floats per episode)
|
||||
portfolio_states: CudaSlice<f32>, // [alloc_episodes * PORTFOLIO_STRIDE] (43 floats per episode)
|
||||
episode_starts_buf: CudaSlice<i32>,// [alloc_episodes]
|
||||
|
||||
// Output buffers [alloc_episodes * alloc_timesteps, ...]
|
||||
|
||||
@@ -71,10 +71,12 @@
|
||||
#define PS_PLAN_ENTRY_REGIME 29 // plan: regime_stability at entry
|
||||
// Slots 30-37: prev-bar OFI scratch for delta computation (see experience_kernels.cu)
|
||||
#define PS_OFI_PREV_BASE 30 // first OFI prev-bar slot (stride 8: [30..37])
|
||||
#define PS_PEAK_PNL_BAR 38 // Plan 3 C.4: hold_time at which MAX_PNL was hit (temporal timing bonus)
|
||||
#define PS_INTRA_TRADE_MIN_PNL 39 // Plan 3 D.4a: worst unrealised PnL during trade (symmetric to MAX_PNL at 21)
|
||||
#define PS_REGIME_SHIFT_BAR 40 // Plan 3 D.4b: hold_time at first regime-shift detection (0 if no shift)
|
||||
#define PS_STRIDE 41 // total portfolio state stride (== PORTFOLIO_STRIDE), grown 40→41 by Plan 3 D.4b
|
||||
#define PS_PEAK_PNL_BAR 38 // Plan 3 C.4: hold_time at which MAX_PNL was hit (temporal timing bonus)
|
||||
#define PS_INTRA_TRADE_MIN_PNL 39 // Plan 3 D.4a: worst unrealised PnL during trade (symmetric to MAX_PNL at 21)
|
||||
#define PS_REGIME_SHIFT_BAR 40 // Plan 3 D.4b: hold_time at first regime-shift detection (0 if no shift)
|
||||
#define PS_PRE_ENTRY_CONVICTION_EMA 41 // Plan 3 D.4c: EMA of conviction_core during Flat bars
|
||||
#define PS_PRE_ENTRY_CONVICTION_VAR_EMA 42 // Plan 3 D.4c: EMA of (conviction_core − mean)² during Flat bars
|
||||
#define PS_STRIDE 43 // total portfolio state stride (== PORTFOLIO_STRIDE), grown 41→43 by Plan 3 D.4c conviction consistency
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────────
|
||||
// Plan-ISV slot indices (plan_isv[0..PLAN_ISV_DIM=7)).
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
/**
|
||||
* Trade stats reduction kernel.
|
||||
*
|
||||
* Reads portfolio_states[N, PORTFOLIO_STRIDE=41] and sums fields [14:19]
|
||||
* Reads portfolio_states[N, PORTFOLIO_STRIDE=43] and sums fields [14:19]
|
||||
* (win_count, loss_count, sum_wins, sum_losses, sum_returns, sum_sq_returns)
|
||||
* across all N episodes into a 6-float output buffer.
|
||||
* Uses shared-memory warp reduction (same pattern as monitoring_reduce).
|
||||
@@ -9,7 +9,7 @@
|
||||
* Launch config: grid=(1, 1, 1), block=(256, 1, 1).
|
||||
*/
|
||||
|
||||
#define PORTFOLIO_STRIDE 41 // was 40; grown by Plan 3 D.4b PS_REGIME_SHIFT_BAR append. Kelly slots [14..19] unchanged.
|
||||
#define PORTFOLIO_STRIDE 43 // was 41; grown by Plan 3 D.4c PS_PRE_ENTRY_CONVICTION_{EMA,VAR_EMA} append. Kelly slots [14..19] unchanged.
|
||||
#include "state_layout.cuh"
|
||||
|
||||
extern "C" __global__ void trade_stats_reduce(
|
||||
|
||||
@@ -64,6 +64,7 @@
|
||||
| C.4 Temporal timing bonus on trade exit (`experience_kernels.cu` segment_complete block) | `experience_env_step` kernel (inside `segment_complete && segment_hold_time > 0` block, AFTER Layer 2–9 credits land in `reward`); accumulates `+=` into `rc[5]` and adds `timing_bonus` to `reward`; consumed by `reward_component_ema` → ISV[68] (shares the bonus slot with B.2 — different (i,t) slots, so `+=` is idempotent) | Wired | Plan 3 Task 5 C.4 — bars_early = max(0, segment_hold_time − PS_PEAK_PNL_BAR); timing_bonus = shaping_scale × (bars_early / segment_hold_time) × \|final_pnl\| × conviction_core. Peak bar tracked in new portfolio-state slot PS_PEAK_PNL_BAR=38 (PS_STRIDE 38→39), snapshot alongside every MAX_PNL update, reset at every MAX_PNL reset site (entry/reverse/fold-reset). No tuned multiplier. | — |
|
||||
| D.4a Persistence credit on profitable drawdown recovery (`experience_kernels.cu` segment_complete block) | `experience_env_step` kernel (inside `segment_complete && segment_hold_time > 0` block, IMMEDIATELY AFTER C.4 timing bonus, gated on `reward > 0` AND `drawdown_depth > 1e-6`); accumulates `+=` into `rc[5]` and adds `persist_bonus` to `reward`; consumed by `reward_component_ema` → ISV[68] (shares the bonus slot with B.2 entry + C.4 exit — different (i,t) slots per trade, so `+=` is idempotent across all three) | Wired | Plan 3 Task 6a D.4a — drawdown_depth = max(0, −PS_INTRA_TRADE_MIN_PNL); persist_bonus = shaping_scale × conviction_core × drawdown_depth × tanh(reward / max(1e-4, drawdown_depth)). MIN_PNL tracked in new portfolio-state slot PS_INTRA_TRADE_MIN_PNL=39 (PS_STRIDE 39→40), updated per-bar in the same block as MAX_PNL (`fminf` against `pnl_pct`), reset at every MAX_PNL reset site (entry/reverse/fold hard-reset/trade-complete soft-reset). Self-scaling tanh: saturates +1 when reward ≫ drawdown, ≈0 when reward ≈ drawdown. No tuned multiplier. | — |
|
||||
| D.4b Regime-shift penalty for trades held past a regime flip (`experience_kernels.cu` segment_complete block) | `experience_env_step` kernel: detector inside the active-trade block (after MIN/MAX_PNL update) writes `PS_REGIME_SHIFT_BAR` on the FIRST bar where `\|isv[11] − PS_PLAN_ENTRY_REGIME\| > clamp(0.25 × \|clamp(sharpe, ±2)\|, 0.05, 0.5)`; consumer inside `segment_complete` block IMMEDIATELY AFTER D.4a accumulates `−=` into `rc[5]` and subtracts `penalty` from `reward`; consumed by `reward_component_ema` → ISV[68] (cancellation within rc[5] vs B.2/C.4/D.4a is intentional — that's what ISV[68] REWARD_BONUS_EMA tracks). Consumer also resets `PS_REGIME_SHIFT_BAR = 0` after use. | Wired | Plan 3 Task 6b D.4b — bars_late = max(0, saved_hold_time − PS_REGIME_SHIFT_BAR); penalty = shaping_scale × conviction_core × (bars_late / saved_hold_time) × ISV[Q_DIR_ABS_REF_INDEX=21] × \|reward\|. Shift-bar tracked in new portfolio-state slot PS_REGIME_SHIFT_BAR=40 (PS_STRIDE 40→41), reset at every MAX_PNL reset site (entry/reverse/fold hard-reset/trade-complete soft-reset) PLUS post-consumer (defensive). Self-scaling Q_DIR_ABS_REF coefficient (same B.1 pattern). Adaptive threshold tightens when \|sharpe\| is high. First-shift-only. No tuned multiplier. | — |
|
||||
| D.4c Conviction consistency bonus on Flat→Positioned entry (`experience_kernels.cu` Flat branch + entering_trade block) | `experience_env_step` kernel: producer inside the Flat branch (`!segment_complete && position≈0`, before opp-cost) updates two PS slots `PS_PRE_ENTRY_CONVICTION_EMA` (mean) and `PS_PRE_ENTRY_CONVICTION_VAR_EMA` (variance) via Welford-style EMA at α=0.05 on `conviction_core`. Consumer inside `entering_trade` block IMMEDIATELY AFTER B.2 novelty bonus computes `ratio = stddev/mean`; when `ratio < 0.2` accumulates `+=` into `rc[5]` and adds `bonus` to `reward`; consumed by `reward_component_ema` → ISV[68] (shares the bonus slot with B.2/C.4/D.4a/D.4b at different (i,t) slots; `+=` is idempotent). Both EMA slots reset at every trade-lifecycle reset site (entry/reverse/fold hard-reset/trade-complete soft-reset) PLUS post-consumer (defensive). | Wired | Plan 3 Task 6c D.4c — `bonus = shaping_scale × vol_proxy × stability × conviction_core` where `vol_proxy ∈ [0.0001, 0.01]`, `stability = clamp(0, 1, 1 − ratio/0.2)`, `conviction_core ∈ [0,1]`. Mirrors B.2 novelty-bonus structure exactly — ONE technically unbounded multiplicand (`vol_proxy`, bounded ≤ 0.01), all others bounded — per pearl_one_unbounded_signal_per_reward.md. Max bonus ≈ 0.01, same order as B.2. New portfolio-state slots `PS_PRE_ENTRY_CONVICTION_EMA=41` and `PS_PRE_ENTRY_CONVICTION_VAR_EMA=42`; `PS_STRIDE` 41→43; `PORTFOLIO_STRIDE` 41→43 in all consumers (`experience_kernels.cu`, `trade_stats_kernel.cu`, `gpu_experience_collector.rs`, `gpu_dqn_trainer.rs::launch_kelly_cap_update`, `ml-core::state_layout.rs`). The 0.2 stability threshold is the single tuned constant retained for this first cut (commented; reviewer may demote to ISV-driven follow-up). α=0.05 matches Plan 3 Task 1 reward-EMA convention. | — |
|
||||
|
||||
## CUDA Pipeline — Rust Wrappers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user