gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(magnitude): thread magnitude conviction into Kelly cap — eval Full reachable

Browse Source 
Smoke after var_scale floor (73fd49f4b) still showed eval Full = 0.000 with
intent = 0.911. Root cause: trade_physics.cuh::unified_env_step_core applied
Kelly cap using ONLY direction conviction (q_dir_gap / q_dir_abs_ref). At
smoke horizon with direction Q-values close (q_s≈q_h≈q_l≈q_f within ~0.07),
direction conviction was ~0.04 → safety = max(0.75, 0.04) = 0.75,
effective_kelly = max(0, 0.75) = 0.75, cap = 0.75 × max_pos × 0.75 =
0.5625 × max_pos → Half bucket pin (abs_pos < 0.75).

The magnitude branch was simultaneously highly confident (q_f / q_h ≈ 1.9,
mag conviction ≈ 0.5) — the policy strongly preferred Full. That signal
never reached the cap.

Thread per-sample magnitude conviction into the cap formula:
- new buffer magnitude_conviction_buf[N*L] (training) parallel to
  conviction_buf, and chunked_magnitude_conviction_buf[chunk_len * n_windows]
  (eval) parallel to chunked_conviction_buf
- experience_action_select writes (max(q_mag) − min(q_mag)) /
  fmaxf(ISV[16], 1e-6); ISV[16] = Q_ABS_REF (magnitude branch's |Q| EMA,
  recycled — no new ISV slot)
- scripted_policy_kernel writes 0.0 (no real magnitude preference; helper
  composer falls back to direction signal)
- unified_env_step_core takes a parallel magnitude_conviction parameter
- combined: policy_conviction = max(dir_conv, mag_conv); cap formula
  becomes safety = max(health_safety, policy_conviction); warmup_floor
  uses policy_conviction
- all three env_step kernel call sites migrated together
  (backtest_env_step, backtest_env_step_batch, experience_env_step) per
  feedback_no_partial_refactor — no mixed state

Per pearl_blend_formulas_must_have_permanent_floor: max() composition,
not multiplicative blend. Per feedback_isv_for_adaptive_bounds: signal
flows from existing ISV[16] = Q_ABS_REF, no tuned constants.

Smoke verification (20-epoch RTX 3050 Ti, magnitude_distribution test):
- pre-fix:  [EVAL_DIST] Q=0.586 H=0.414 F=0.000 (intent F=0.911)
- post-fix: [EVAL_DIST] Q=0.618 H=0.153 F=0.229 (intent F=0.911)

Intent unchanged (network already learned Full preference); the Kelly
cap no longer silences it. Both smoke assertions now pass:
  ef >= 0.05 (gate) and eh + ef >= 0.30 (Task 2.2 H10).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-27 23:30:31 +02:00
parent fa92cb8dbb
commit 94157a8a69
8 changed files with 231 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
crates/ml/src/cuda_pipeline/backtest_env_kernel.cu
View File

@@ -116,7 +116,13 @@ extern "C" __global__ void backtest_env_step(
/* Per-sample conviction [n_windows] for adaptive Kelly warmup floor.
* Written by experience_action_select (out_conviction). NULL → fallback
* 1.0 (trust policy face-value). Threaded into unified_env_step_core. */
const float* __restrict__ conviction_ptr
const float* __restrict__ conviction_ptr,
/* Per-sample magnitude-branch conviction [n_windows] — same shape as
* conviction_ptr, parallel signal. Written by experience_action_select
* (out_magnitude_conviction). Threaded into unified_env_step_core where
* the cap formula composes max(dir_conv, mag_conv). NULL → fallback 0.0
* (helper composes against direction conviction alone). */
const float* __restrict__ magnitude_conviction_ptr
) {
/* Both scales are no-ops in backtest today — ignored. The signature
* alignment is so the future unified_env_kernel can be a drop-in
@@ -218,10 +224,17 @@ extern "C" __global__ void backtest_env_step(
&trail_vol_scale, &trail_trend_scale
);
/* Per-sample conviction for Kelly warmup floor — matches training. */
/* Per-sample conviction for Kelly warmup floor — matches training.
* NULL conviction_ptr defaults to 1.0 (trust policy face-value, same as
* pre-mag-conviction behaviour). NULL magnitude_conviction_ptr defaults
* to 0.0 — the helper's max(dir_conv, mag_conv) then falls back to the
* direction signal alone, preserving prior single-conviction behaviour. */
float conviction_core = (conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, conviction_ptr[w]))
: 1.0f;
float mag_conviction_core = (magnitude_conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, magnitude_conviction_ptr[w]))
: 0.0f;
unified_env_step_core(
action_val, close,
@@ -230,7 +243,8 @@ extern "C" __global__ void backtest_env_step(
contract_multiplier, margin_pct,
health_k,
trail_vol_scale, trail_trend_scale,
conviction_core, /* adaptive Kelly warmup floor */
conviction_core, /* direction-branch Kelly warmup floor */
mag_conviction_core, /* magnitude-branch Kelly warmup floor */
&position, &cash, &entry_price, &hold_time,
&max_equity,
value, /* prev_equity (snapshot) */
@@ -364,7 +378,11 @@ extern "C" __global__ void backtest_env_step_batch(
/* Per-sample conviction [chunk_len * n_windows] — layout matches
* chunked_actions ([step_offset * n_windows + window]). NULL → fallback
* 1.0. Threaded into unified_env_step_core for adaptive Kelly warmup. */
const float* __restrict__ conviction_ptr
const float* __restrict__ conviction_ptr,
/* Per-sample magnitude-branch conviction — same shape and layout as
* conviction_ptr. NULL → fallback 0.0; the helper's max(dir_conv,
* mag_conv) composer falls back to the direction signal alone. */
const float* __restrict__ magnitude_conviction_ptr
) {
/* Phase 3 unified-env: shaping params retained for ABI but no longer
* applied to step_returns. See single-step variant header for rationale. */
@@ -462,10 +480,15 @@ extern "C" __global__ void backtest_env_step_batch(
);
/* Per-sample conviction for adaptive Kelly warmup floor. Layout
* matches chunked_actions: [s * n_windows + w]. */
* matches chunked_actions: [s * n_windows + w]. NULL magnitude_
* conviction_ptr defaults to 0.0 — the helper's max(dir, mag) composer
* falls back to the direction signal alone, preserving prior behaviour. */
float conviction_core = (conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, conviction_ptr[s * n_windows + w]))
: 1.0f;
float mag_conviction_core = (magnitude_conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, magnitude_conviction_ptr[s * n_windows + w]))
: 0.0f;
unified_env_step_core(
action_val, close,
@@ -474,7 +497,8 @@ extern "C" __global__ void backtest_env_step_batch(
contract_multiplier, margin_pct,
health_k_batch,
trail_vol_scale_b, trail_trend_scale_b,
conviction_core, /* adaptive Kelly warmup floor */
conviction_core, /* direction-branch Kelly warmup floor */
mag_conviction_core, /* magnitude-branch Kelly warmup floor */
&position, &cash, &entry_price, &hold_time,
&max_equity,
value, /* prev_equity (snapshot for step_return) */

72
crates/ml/src/cuda_pipeline/experience_kernels.cu
View File

@@ -742,6 +742,14 @@ extern "C" __global__ void experience_state_gather(
* = (max(q_dir) − min(q_dir)) / fmaxf(q_dir_abs_ref, 1e-6f).
* Consumed by env_step → unified_env_step_core → apply_kelly_cap
* as the adaptive warmup floor. NULL-tolerant.
* @param out_magnitude_conviction [N] output: per-sample magnitude-branch conviction ∈ [0, 1]
* = (max(q_mag) − min(q_mag)) / fmaxf(q_mag_abs_ref, 1e-6f).
* Mirrors the direction-conviction shape with the magnitude
* branch's ISV reference (ISV[16] = Q_ABS_REF, magnitude EMA).
* Consumed by env_step → unified_env_step_core where the
* Kelly-cap formula uses max(dir_conv, mag_conv) so a strong
* magnitude preference can unblock the cap when direction
* conviction is low (typical at smoke horizon). NULL-tolerant.
* @param epsilon exploration probability in [0, 1]
* @param N number of episodes
* @param b0_size direction branch size (4)
@@ -782,7 +790,8 @@ extern "C" __global__ void experience_action_select(
* env_step. NULL = skip. Diagnostic-only — lets operators see
* what the magnitude head WOULD pick if direction gating and
* post-policy physics didn't mask it. */
float* out_conviction /* [N] output: direction-branch conviction ∈ [0,1] for Kelly warmup. NULL=skip. */
float* out_conviction, /* [N] output: direction-branch conviction ∈ [0,1] for Kelly warmup. NULL=skip. */
float* out_magnitude_conviction /* [N] output: magnitude-branch conviction ∈ [0,1] for Kelly warmup. NULL=skip. */
) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return;
@@ -1179,6 +1188,51 @@ extern "C" __global__ void experience_action_select(
float conviction = q_range / denom;
out_conviction[i] = fminf(1.0f, fmaxf(0.0f, conviction));
}
/* Per-sample magnitude-branch conviction for adaptive Kelly warmup floor.
* mag_conviction = (max(q_mag) − min(q_mag)) / fmaxf(q_mag_abs_ref, 1e-6f)
* clamped to [0, 1]. Mirrors the direction-branch shape exactly with
* ISV slot 16 (Q_ABS_REF — magnitude branch's |Q| scale EMA, written by
* `q_mag_bin_means_reduce` on the stats cadence) as the normaliser.
*
* Why threading magnitude conviction matters (project_magnitude_eval_collapse):
* the existing direction conviction (ISV[21] = Q_DIR_ABS_REF) reflects the
* policy's confidence in WHERE to trade. The magnitude head independently
* learns a strong size preference (e.g. q_f / q_h ≈ 1.9 at smoke horizon
* with q_q ≈ 0.43, q_h ≈ 0.46, q_f ≈ 0.87 — a clear Full preference). The
* direction head can simultaneously be near-uniform (q_dir_gap / q_dir_abs
* _ref ≈ 0.04) when the policy hesitates between Hold and Long. The Kelly
* cap formula previously saw only direction conviction and pinned target
* positions to the Half bucket via safety = max(0.75, 0.04) = 0.75, never
* reaching the Full threshold abs_pos ≥ 0.75 × max_position. Threading
* magnitude conviction lets the cap composer take the max(dir_conv, mag_conv)
* — a confident magnitude pick (mag_conv ≈ 0.5) unblocks the cap to
* safety = max(0.75, 0.5) = 0.75 in the worst case but lifts to 1.0 when
* the policy concentrates Q-mass on a single magnitude bin.
*
* Same fallback semantics as direction conviction: when ISV[16] ≤ 1e-6 use
* q_range itself as denominator → conviction = 1.0 cold-start. Once the
* `q_mag_bin_means_reduce` stats cadence fires (every epoch) the adaptive
* normalisation takes over. Uses raw Q (no contrarian sign flip) — conviction
* is about magnitude-decision confidence regardless of direction sign. */
if (out_magnitude_conviction != NULL) {
float q_max_raw_m = q_b1[0];
float q_min_raw_m = q_b1[0];
for (int a = 1; a < b1_size; a++) {
float qv = q_b1[a];
q_max_raw_m = fmaxf(q_max_raw_m, qv);
q_min_raw_m = fminf(q_min_raw_m, qv);
}
float q_range_m = q_max_raw_m - q_min_raw_m;
float q_mag_abs_ref = (isv_signals_ptr != NULL)
? isv_signals_ptr[16]
: 0.0f;
float denom_m = (q_mag_abs_ref > 1e-6f)
? q_mag_abs_ref
: fmaxf(q_range_m, 1e-6f);
float mag_conviction = q_range_m / denom_m;
out_magnitude_conviction[i] = fminf(1.0f, fmaxf(0.0f, mag_conviction));
}
}
/* ================================================================== */
@@ -1331,6 +1385,12 @@ extern "C" __global__ void experience_env_step(
* 1.0 (trust the policy face-value; same outcome as the old static
* warmup_floor=0.5 at health=1). Threaded into unified_env_step_core. */
const float* __restrict__ conviction_ptr,
/* Per-sample magnitude-branch conviction ∈ [0, 1] — same shape as
* conviction_ptr, parallel signal. Written by experience_action_select
* (out_magnitude_conviction). Threaded into unified_env_step_core where
* the cap formula composes max(dir_conv, mag_conv). NULL → fallback 0.0
* (helper falls back to direction conviction alone). */
const float* __restrict__ magnitude_conviction_ptr,
/* C.2 Reward-component attribution (Plan 3 Task 1, spec §4.C.2).
* Per-sample per-component reward magnitudes written at every (i,t).
* Layout [N*L, 6] row-major; component indices:
@@ -1736,10 +1796,15 @@ extern "C" __global__ void experience_env_step(
* (trust policy face-value when action_select didn't write a value —
* e.g. future kernels that skip the conviction buffer). The action_select
* kernel clamps to [0, 1] already; defensive clamp here guards against
* direct device writes bypassing that path. */
* direct device writes bypassing that path. magnitude_conviction_ptr
* defaults to 0.0 — helper composes max(dir, mag) and degrades to direction
* conviction alone if the parallel buffer isn't wired. */
float conviction_core = (conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, conviction_ptr[i]))
: 1.0f;
float mag_conviction_core = (magnitude_conviction_ptr != NULL)
? fminf(1.0f, fmaxf(0.0f, magnitude_conviction_ptr[i]))
: 0.0f;
unified_env_step_core(
action_idx, raw_close,
@@ -1750,7 +1815,8 @@ extern "C" __global__ void experience_env_step(
contract_multiplier, margin_pct,
health_k_train,
trail_vol_scale, trail_trend_scale,
conviction_core, /* adaptive Kelly warmup floor */
conviction_core, /* direction-branch Kelly warmup floor */
mag_conviction_core, /* magnitude-branch Kelly warmup floor */
&position, &cash, &entry_price, &hold_time,
&peak_equity, /* helper updates peak_equity via fmaxf */
prev_equity, /* snapshot for step_return denominator */

25
crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs
View File

@@ -375,6 +375,13 @@ pub struct GpuBacktestEvaluator {
/// `experience_action_select` each chunk; consumed by `backtest_env_step_batch`
/// as the adaptive Kelly warmup-floor signal (via `unified_env_step_core`).
chunked_conviction_buf: Option<CudaSlice<f32>>,
/// Per-sample magnitude-branch conviction ∈ [0, 1], chunked layout matching
/// `chunked_conviction_buf`. Parallel signal threaded through the same
/// kernels for the Kelly cap's `max(dir_conv, mag_conv)` composer — lets
/// the cap relax for confident magnitude picks even when direction
/// conviction is low (typical at smoke horizon when q_dir values cluster
/// while the magnitude head sharply prefers Full).
chunked_magnitude_conviction_buf: Option<CudaSlice<f32>>,
/// Cached CUDA graph for `evaluate_dqn_graphed()` step loop.
///
@@ -676,6 +683,7 @@ impl GpuBacktestEvaluator {
q_gap_threshold: 0.0,
chunked_q_gaps_buf: None,
chunked_conviction_buf: None,
chunked_magnitude_conviction_buf: None,
dqn_graph: None,
chunked_q_values: None,
chunked_states_buf: None,
@@ -1366,6 +1374,9 @@ impl GpuBacktestEvaluator {
let ch_conviction = self.chunked_conviction_buf.as_ref().ok_or_else(|| {
MLError::ModelError("chunked_conviction_buf unexpectedly None".to_owned())
})?;
let ch_magnitude_conviction = self.chunked_magnitude_conviction_buf.as_ref().ok_or_else(|| {
MLError::ModelError("chunked_magnitude_conviction_buf unexpectedly None".to_owned())
})?;
let n = self.n_windows;
let state_dim = self.state_dim;
@@ -1480,7 +1491,8 @@ impl GpuBacktestEvaluator {
.arg(&self.isv_signals_ptr) // ISV signals for adaptive hold
.arg(&0i32) // F6: contrarian_active — always off during backtest
.arg(ch_intent) // out_intent_mag: pure-policy magnitude intent diagnostic
.arg(ch_conviction) // out_conviction: per-sample Kelly warmup signal
.arg(ch_conviction) // out_conviction: direction-branch Kelly warmup signal
.arg(ch_magnitude_conviction) // out_magnitude_conviction: magnitude-branch Kelly warmup signal
.launch(launch_cfg)
.map_err(|e| MLError::ModelError(format!(
"chunked action_select chunk_start={chunk_start}: {e}"
@@ -1589,7 +1601,8 @@ impl GpuBacktestEvaluator {
// Phase 3 unified-env scalars (NULL = no-op in val mode).
.arg(&0u64) // exploration_scale_ptr (NULL)
.arg(&0u64) // shaping_scale_ptr (NULL)
.arg(ch_conviction) // conviction_ptr: adaptive Kelly warmup signal
.arg(ch_conviction) // conviction_ptr: direction-branch Kelly warmup signal
.arg(ch_magnitude_conviction) // magnitude_conviction_ptr: magnitude-branch Kelly warmup signal
.launch(LaunchConfig {
grid_dim: (env_blocks.max(1), 1, 1),
block_dim: (256, 1, 1),
@@ -1925,6 +1938,12 @@ impl GpuBacktestEvaluator {
* chunked_actions/chunked_q_gaps — [chunk_len * n_windows]. */
let ch_conviction = self.stream.alloc_zeros::<f32>(cn)
.map_err(|e| MLError::ModelError(format!("alloc chunked conviction: {e}")))?;
/* Parallel magnitude-branch conviction — same shape, threaded into
* action_select + env_step_batch alongside the direction-branch
* conviction so the Kelly cap's max(dir, mag) composer can relax
* for confident magnitude picks. */
let ch_magnitude_conviction = self.stream.alloc_zeros::<f32>(cn)
.map_err(|e| MLError::ModelError(format!("alloc chunked magnitude conviction: {e}")))?;
info!(
"GpuBacktestEvaluator: action-select initialised — {} windows, \
@@ -1947,6 +1966,7 @@ impl GpuBacktestEvaluator {
self.picked_action_history_buf = Some(picked_action_history_buf);
self.scatter_intent_kernel = Some(scatter_intent);
self.chunked_conviction_buf = Some(ch_conviction);
self.chunked_magnitude_conviction_buf = Some(ch_magnitude_conviction);
Ok(())
}
@@ -2049,6 +2069,7 @@ impl GpuBacktestEvaluator {
.arg(&0u64) // exploration_scale_ptr (NULL)
.arg(&0u64) // shaping_scale_ptr (NULL)
.arg(&0u64) // conviction_ptr (NULL) — fallback 1.0 in-kernel
.arg(&0u64) // magnitude_conviction_ptr (NULL) — fallback 0.0 in-kernel; max(dir, mag) → dir alone
.launch(env_cfg)
.map_err(|e| {
MLError::ModelError(format!("backtest_env_step launch step {step}: {e}"))

23
crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
View File

@@ -535,6 +535,13 @@ pub struct GpuExperienceCollector {
/// by q_dir_abs_ref (ISV[21]). Drives adaptive Kelly warmup floor in
/// `unified_env_step_core` (replaces the old static 0.5 floor).
conviction_buf: CudaSlice<f32>,
/// Per-sample magnitude-branch conviction [N] for adaptive Kelly warmup floor.
/// Parallel to `conviction_buf` (direction-branch). Written by
/// `experience_action_select` (out_magnitude_conviction) on each step;
/// consumed by `experience_env_step` → `unified_env_step_core` where
/// the cap formula composes `max(dir_conv, mag_conv)` so a confident
/// magnitude pick can unblock the cap when direction conviction is low.
magnitude_conviction_buf: CudaSlice<f32>,
/// Kelly optimal fraction (per-epoch scalar from CPU).
/// Current timestep counter per episode: [N].
current_timesteps: CudaSlice<i32>,
@@ -1209,6 +1216,12 @@ impl GpuExperienceCollector {
let conviction_buf = stream
.alloc_zeros::<f32>(n)
.map_err(|e| MLError::ModelError(format!("alloc conviction_buf: {e}")))?;
/* Per-sample magnitude-branch conviction [N] — parallel to conviction_buf,
* threaded through the same kernels for the Kelly cap's max(dir, mag)
* composer. */
let magnitude_conviction_buf = stream
.alloc_zeros::<f32>(n)
.map_err(|e| MLError::ModelError(format!("alloc magnitude_conviction_buf: {e}")))?;
let current_timesteps = stream
.alloc_zeros::<i32>(n)
.map_err(|e| MLError::ModelError(format!("alloc current_timesteps: {e}")))?;
@@ -1557,6 +1570,7 @@ impl GpuExperienceCollector {
batch_actions,
q_gaps_buf,
conviction_buf,
magnitude_conviction_buf,
current_timesteps,
q_values,
q_select_buf,
@@ -3509,7 +3523,8 @@ impl GpuExperienceCollector {
.arg(&sd_i32)
.arg(&bar_idx_i32)
.arg(&mut self.batch_actions) // [N] factored action ∈ [0, 108)
.arg(&mut self.conviction_buf) // [N] conviction ∈ [0, 1]
.arg(&mut self.conviction_buf) // [N] direction conviction ∈ [0, 1]
.arg(&mut self.magnitude_conviction_buf) // [N] magnitude conviction ∈ [0, 1]
.launch(launch_cfg)
.map_err(|e| MLError::ModelError(format!(
"scripted_policy_select t={t}: {e}"
@@ -3552,7 +3567,8 @@ impl GpuExperienceCollector {
.arg(&self.isv_signals_dev_ptr) // ISV signals for adaptive hold (0=NULL=static)
.arg(&(self.contrarian_active_cache as i32)) // D7/N7: Q-negation flag
.arg(&0u64) // out_intent_mag: NULL — training path does not collect
.arg(&mut self.conviction_buf) // out_conviction: per-sample Kelly warmup signal
.arg(&mut self.conviction_buf) // out_conviction: per-sample direction-branch Kelly warmup signal
.arg(&mut self.magnitude_conviction_buf) // out_magnitude_conviction: per-sample magnitude-branch Kelly warmup signal
.launch(launch_cfg)
.map_err(|e| MLError::ModelError(format!(
"experience_action_select t={t}: {e}"
@@ -3685,7 +3701,8 @@ impl GpuExperienceCollector {
.arg(&cf_ratio) // D4/N4: adaptive counterfactual flip ratio
.arg(&self.exploration_scale_dev_ptr) // Phase 3 unified-env: gates saboteur/plan/CF flips
.arg(&self.shaping_scale_dev_ptr) // Phase 3 unified-env: gates additive reward shaping
.arg(&self.conviction_buf) // conviction_ptr: adaptive Kelly warmup signal
.arg(&self.conviction_buf) // conviction_ptr: direction-branch Kelly warmup signal
.arg(&self.magnitude_conviction_buf) // magnitude_conviction_ptr: magnitude-branch Kelly warmup signal
// C.2: per-component reward buffer [N*L * 6] — popart/cf/trail/micro/opp_cost/bonus
.arg(&mut self.reward_components_per_sample)
// B.2 Plan 3 Task 3: Flat→Positioned transition flag + ISV slot indices

2
crates/ml/src/cuda_pipeline/mod.rs
View File

@@ -1118,6 +1118,7 @@ mod tests {
let mut q_gaps_buf = stream.alloc_zeros::<f32>(batch).expect("alloc q_gaps");
let mut intent_buf = stream.alloc_zeros::<i32>(batch).expect("alloc intent");
let mut conv_buf = stream.alloc_zeros::<f32>(batch).expect("alloc conviction");
let mut mag_conv_buf = stream.alloc_zeros::<f32>(batch).expect("alloc magnitude conviction");
let blocks = (batch as u32).div_ceil(256);
let cfg = LaunchConfig {
@@ -1153,6 +1154,7 @@ mod tests {
.arg(&0_i32) // contrarian_active
.arg(&mut intent_buf)
.arg(&mut conv_buf)
.arg(&mut mag_conv_buf)
.launch(cfg)
.expect("launch experience_action_select");
}

10
crates/ml/src/cuda_pipeline/scripted_policy_kernel.cu
View File

@@ -44,7 +44,8 @@ extern "C" __global__ void scripted_policy_select(
int state_dim,
int bar_idx, /* current timestep, mixes into LCG seed */
int* __restrict__ actions_out, /* [N] factored action index ∈ [0, 108) */
float* __restrict__ conviction_out /* [N] conviction ∈ [0, 1] */
float* __restrict__ conviction_out, /* [N] direction conviction ∈ [0, 1] */
float* __restrict__ magnitude_conviction_out /* [N] magnitude conviction ∈ [0, 1] */
) {
const int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= n_samples) return;
@@ -108,4 +109,11 @@ extern "C" __global__ void scripted_policy_select(
const int urg = 1; /* Normal */
actions_out[i] = dir * 27 + mag * 9 + ord * 3 + urg;
conviction_out[i] = conviction;
/* The scripted policy hard-codes magnitude=Half and does not express
* a per-sample magnitude preference. Emit 0.0 for magnitude conviction
* so the env-step's max(dir_conv, mag_conv) Kelly composer falls back
* to the direction signal (the only signal this policy actually
* represents). The network's `experience_action_select` overwrites
* both buffers with real Q-spread normalised conviction once seeded. */
magnitude_conviction_out[i] = 0.0f;
}

77
crates/ml/src/cuda_pipeline/trade_physics.cuh
View File

@@ -548,11 +548,14 @@ __device__ __forceinline__ void compute_regime_trail_scales(
* 2. Hold action passthrough
* 3. Target position via compute_target_position_4branch
* 4. Margin cap (apply_margin_cap)
* 5. Kelly cap with health-coupled safety AND policy conviction
* (apply_kelly_cap). Conviction ∈ [0, 1] — per-sample, derived from
* direction Q-spread normalised by q_dir_abs_ref. Drives the cold-
* start warmup floor so Kelly cap adapts to policy conviction instead
* of a static 0.5 floor; once maturity → 1 it's a no-op.
* 5. Kelly cap with health-coupled safety AND per-branch policy conviction
* (apply_kelly_cap). Two conviction signals (direction + magnitude),
* each ∈ [0, 1] — per-sample, derived from each branch's Q-spread
* normalised by its ISV |Q|-scale reference (ISV[21] for direction,
* ISV[16] for magnitude). Composed with `max()` then composed with
* health_safety with `max()`. Drives the cold-start warmup floor so
* Kelly cap adapts to whichever branch expresses high confidence,
* instead of a static 0.5 floor; once maturity → 1 it's a no-op.
* 6. Trailing stop check — base 0.5%, regime-adaptive via vol_scale and
* trend_scale (caller resolves from ADX/CUSUM features; identical
* formulas in both training and val to preserve train/eval parity)
@@ -609,6 +612,17 @@ __device__ __forceinline__ void unified_env_step_core(
* inside kelly_position_cap; caller may pass out-of-range and the
* helper bounds it. */
float conviction,
/* ── Magnitude-branch conviction ∈ [0, 1] for adaptive Kelly warmup floor ──
* Per-sample magnitude Q-spread normalised by q_mag_abs_ref (ISV[16]).
* Composed with direction conviction inside this helper as
* policy_conviction = max(direction_conviction, magnitude_conviction)
* so a confident magnitude preference unblocks the Kelly cap when the
* direction head is near-uniform (typical at smoke-horizon: direction
* Q-values cluster while the magnitude head sharply prefers Full). Same
* shape/clamp/semantics as `conviction` — caller may pass out-of-range
* and the helper bounds it. NULL/zero from the caller is benign — the
* max() composer falls back to the direction signal alone. */
float magnitude_conviction,
/* ── Per-thread mutable state (in/out) ── */
float* position,
float* cash,
@@ -650,42 +664,53 @@ __device__ __forceinline__ void unified_env_step_core(
);
}
/* ── Step 5: Kelly cap (safety composed from two orthogonal signals
/* ── Step 5: Kelly cap (safety composed from three orthogonal signals
* + conviction-adaptive warmup floor; uses physics scale).
*
* safety_multiplier = max(health_safety, conviction)
* health_safety = 0.5 + 0.5 × health [training-stability floor]
* conviction ∈ [0, 1] [per-sample policy confidence]
* policy_conviction = max(direction_conviction, magnitude_conviction)
* safety_multiplier = max(health_safety, policy_conviction)
* health_safety = 0.5 + 0.5 × health [training-stability floor]
* direction_conviction ∈ [0, 1] [direction-branch confidence]
* magnitude_conviction ∈ [0, 1] [magnitude-branch confidence]
*
* Health measures training stability (is the optimiser making coherent
* progress?). Conviction measures per-state policy confidence (is the
* Q-spread clearly favouring the taken direction over pathology bins?).
* These are orthogonal: a policy can be confident on a given state even
* before training globally stabilises, and a stable training regime can
* still produce low-conviction per-state decisions. max() composes them
* conservatively — the cap uses whichever signal says "trust the policy
* more" at this sample. Bounded to [0.5, 1.0] by the health floor.
* progress?). Per-branch conviction measures the policy's confidence on
* each axis of the factored action: direction conviction reflects WHERE
* to trade, magnitude conviction reflects HOW MUCH. These are orthogonal —
* the policy can be near-uniform on one axis while sharply preferring a
* single bin on the other. The Kelly cap composes both per-branch signals
* via max(): if EITHER axis expresses high confidence, the policy is
* committed enough to its choice that the cap should relax. Health then
* composes via max() too as a training-stability floor. Bounded to
* [0.5, 1.0] by the health floor.
*
* Conviction flows per-sample from the action_select kernel's direction
* Q-spread. Both signals are adaptive / temporal. No static knobs.
* All signals flow per-sample from the action_select kernel's per-branch
* Q-spreads, normalised by ISV per-branch |Q|-scale EMAs (ISV[21] for
* direction, ISV[16] for magnitude). Adaptive / temporal — no static knobs.
*
* Per project_magnitude_eval_collapse_kelly_capped.md: at the smoke-test
* horizon, typical health=0.49 gives health_safety=0.745 which sits
* coincidentally on the Half/Full decoder boundary (abs_pos < 0.75).
* Letting conviction also drive safety unblocks Full realisation for
* confident policy actions at the smoke horizon, instead of requiring
* health graduation which 20-epoch smokes structurally can't reach. */
* horizon, typical health=0.49 gives health_safety=0.745. The direction
* branch is often near-uniform (q_dir_gap / q_dir_abs_ref ≈ 0.04 when
* Hold competes with Long), so direction_conviction alone leaves the cap
* formula at 0.5625 × max_pos — pinned to the Half bucket [0.375, 0.75).
* The magnitude branch simultaneously expresses sharp preferences
* (q_f / q_h ≈ 1.9 → mag_conviction ≈ 0.5). Threading mag_conviction
* unblocks Full realisation for confident magnitude picks even when the
* direction head hesitates, without requiring health graduation
* (structurally unreachable in 20-epoch smokes). */
if (!is_hold_action) {
float h = fminf(1.0f, fmaxf(0.0f, health_for_kelly));
float health_safety = 0.5f + 0.5f * h;
float conviction_clamped = fminf(1.0f, fmaxf(0.0f, conviction));
float safety = fmaxf(health_safety, conviction_clamped);
float dir_conv_clamped = fminf(1.0f, fmaxf(0.0f, conviction));
float mag_conv_clamped = fminf(1.0f, fmaxf(0.0f, magnitude_conviction));
float policy_conviction = fmaxf(dir_conv_clamped, mag_conv_clamped);
float safety = fmaxf(health_safety, policy_conviction);
target_position = apply_kelly_cap(
target_position,
*win_count, *loss_count,
*sum_wins, *sum_losses,
max_position_physics, safety,
conviction,
policy_conviction,
health_safety
);
}

27
docs/dqn-wire-up-audit.md
View File

@@ -2,6 +2,33 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
magnitude conviction threaded into Kelly cap (2026-04-27, follow-up): the
var_scale floor (above) addressed the **training** path. Smoke at the same
horizon then reproduced eval Full=0.000 with intent=0.911 because
`backtest_env_step_batch` doesn't apply var_scale (no Var[Q] shrink in val).
The remaining silencer is `unified_env_step_core`'s Kelly cap, which only
saw direction conviction. At smoke horizon (health≈0.5,
direction_conviction≈0.04 when Hold competes with Long), the formula
`safety = max(0.75, 0.04) = 0.75`, `effective_kelly = max(0, 0.75)`,
`cap = 0.75 × max_pos × 0.75 = 0.5625 × max_pos` lands in Half bucket.
The magnitude branch was simultaneously expressing strong preference
(q_f/q_h ≈ 1.9, mag_conviction ≈ 0.5) but never reached the cap.
Fix: per-sample magnitude conviction parallels direction conviction —
new buffers `magnitude_conviction_buf[N*L]` (training) and
`chunked_magnitude_conviction_buf[chunk_len * n_windows]` (eval).
`experience_action_select` writes `(max(q_mag) − min(q_mag)) /
fmaxf(ISV[16], 1e-6)` (ISV[16] = `Q_ABS_REF` magnitude branch reference,
recycled). `unified_env_step_core` composer:
`policy_conviction = max(dir_conv, mag_conv)`, `safety = max(health_safety,
policy_conviction)`. All three call sites (`backtest_env_step`,
`backtest_env_step_batch`, `experience_env_step`) migrated in lockstep
per `feedback_no_partial_refactor`. No tuned constants —
`max()` composition (per `pearl_blend_formulas_must_have_permanent_floor`),
ISV-driven scale (per `feedback_isv_for_adaptive_bounds`). Smoke
verification: pre-fix `[EVAL_DIST] Q=0.59 H=0.41 F=0.000`; post-fix
`[EVAL_DIST] Q=0.618 H=0.153 F=0.229`. Intent unchanged (the network
already learned Full preference); the cap no longer silences it.
var_scale permanent floor for eval magnitude collapse (2026-04-27): the
EVAL_DIST=Quarter 0.59 / Half 0.41 / Full 0.00 pathology persists even with
intent=0.911 for Full because the position-sizing pipeline composes four