From 9fb980da2bf71e1e847fbc2d199f61771e1f560c Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Fri, 8 May 2026 11:28:00 +0200 Subject: [PATCH] =?UTF-8?q?fixup(class-a-audit-batch-4b):=20invert=20MIN?= =?UTF-8?q?=5FHOLD=5FTEMPERATURE=20dir=5Facc=20=E2=86=92=20temp=20mapping?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Post-commit kernel-formula audit of `compute_min_hold_penalty` at trade_physics.cuh:567-577 showed the original `temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill` mapping was BACKWARDS for the WR-plateau scenario this 8-commit chain targets. Formula recap: soft_factor = deficit / (deficit + T) HIGH T → soft_factor → 0 → forgiving (no exit penalty) LOW T → soft_factor → 1 → sharp (max exit penalty) The deleted schedule's `START=50 → END=5` therefore meant "permissive early, strict late" (classic explore-then-exploit), not "permissive when committed, strict when uncertain" as the substituted mapping assumed. WR-plateau scenario: dir_acc ≈ 0.46-0.48 (model committed but wrong) — the model needs a permissive (HIGH T) exit ramp to bail out of bad committed directions, not maximum exit penalty (LOW T) locking it into them. Inverted the mapping using `confusion = 1 - skill`: - dir_acc ≈ 0.5 (random / plateau) → confusion=1 → temp=50 (permissive, plateau exit ramp) - dir_acc → 1.0 (saturated skill) → confusion=0 → temp=5 (strict, force informed commitment) This preserves the deleted schedule's epoch-0 anchor (T=50 cold-start) while reacting to actual realized skill instead of an epoch-time anchor. Files touched (atomic per `feedback_no_partial_refactor`): - min_hold_temperature_update_kernel.cu — formula + header doctring. - sp14_isv_slots.rs — slot-doc comment expanded with new mapping + fixup-rationale paragraph. - gpu_aux_trunk.rs — `MinHoldTemperatureUpdateOps` docstring. - gpu_dqn_trainer.rs — cubin docstring + ISV_TOTAL_DIM doc-comment. - tests/sp14_oracle_tests.rs — Tests 3 + 4 flipped (high dir_acc now expects temp=TEMP_MIN; low dir_acc now expects TEMP_MAX); Tests 1, 2, 5 numerically unchanged (sentinel guard fires before the mapping; midpoint dir_acc=0.75 has skill=confusion=0.5 so pre/post-fixup numerics match). - docs/dqn-wire-up-audit.md — Item 4 entry rewritten with the fixup rationale + mapping table updated. Verification: cargo check -p ml --tests --all-targets PASS cargo test sp14_audit_4b (lib) 4/4 PASS (slot layout locks unchanged) GPU oracle re-run deferred to next L40S smoke (kernel was rebuilt in-place so cubin contents change; the 5 oracles encode the new mapping and will exercise the new cubin). Per `pearl_first_observation_bootstrap.md` (sentinel→target replace; sentinel anchors unchanged) + `pearl_symmetric_clamp_audit.md` (bilateral clamp on target_temp preserved) + `feedback_no_partial_refactor.md`. Co-Authored-By: Claude Opus 4.7 (1M context) --- crates/ml/src/cuda_pipeline/gpu_aux_trunk.rs | 19 ++++- .../ml/src/cuda_pipeline/gpu_dqn_trainer.rs | 15 ++-- .../min_hold_temperature_update_kernel.cu | 71 ++++++++++++++----- crates/ml/src/cuda_pipeline/sp14_isv_slots.rs | 55 +++++++++----- crates/ml/tests/sp14_oracle_tests.rs | 67 ++++++++++------- docs/dqn-wire-up-audit.md | 21 +++--- 6 files changed, 173 insertions(+), 75 deletions(-) diff --git a/crates/ml/src/cuda_pipeline/gpu_aux_trunk.rs b/crates/ml/src/cuda_pipeline/gpu_aux_trunk.rs index 1243554aa..22b2a2859 100644 --- a/crates/ml/src/cuda_pipeline/gpu_aux_trunk.rs +++ b/crates/ml/src/cuda_pipeline/gpu_aux_trunk.rs @@ -973,16 +973,29 @@ impl DdSaturationFloorUpdateOps { /// reads `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` (sp13 fast EMA of /// binary directional accuracy), maps it to a [5, 50] temperature via /// -/// skill = clamp((short_ema − 0.5) / 0.5, 0, 1) -/// temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × skill +/// skill = clamp((short_ema − 0.5) / 0.5, 0, 1) +/// confusion = 1 − skill +/// temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × confusion /// /// and slow-EMA-blends into /// `ISV[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460]` with α=0.05. /// +/// Per `compute_min_hold_penalty` at trade_physics.cuh:575 +/// (`soft_factor = deficit / (deficit + T)`), HIGH T = forgiving +/// (saturates → 0, no exit penalty), LOW T = sharp (saturates → 1, +/// max exit penalty). The `confusion = 1 − skill` mapping therefore +/// gives the WR-plateau scenario its designed behavior: +/// - dir_acc ≈ 0.5 (random / committed-but-wrong) → confusion=1 → +/// temp=50 (permissive — exit ramp out of bad commitments) +/// - dir_acc → 1.0 (saturated skill) → confusion=0 → temp=5 +/// (strict — force the model to stay in informed positions) +/// /// Replaces the deleted epoch-driven anneal schedule /// `T_end + (T_start − T_end) × exp(-epoch / decay)` (formerly /// state_layout.cuh::MIN_HOLD_TEMPERATURE_{START, END, DECAY} + -/// training_loop.rs::min_hold_temperature_for_epoch). Decouples +/// training_loop.rs::min_hold_temperature_for_epoch). The original +/// schedule's `START=50 → END=5` was "permissive early, strict late" +/// (classic explore-then-exploit, time-anchored). Decouples /// temperature from epoch number — when WR plateaus, the old /// schedule pinned LOW temp (sharp) at end of training (max /// punishment exactly when most needed forgiveness); the signal-driven diff --git a/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs b/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs index daee3f979..3f090fafb 100644 --- a/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs +++ b/crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs @@ -2144,9 +2144,16 @@ pub(crate) static DD_SATURATION_FLOOR_UPDATE_CUBIN: &[u8] = include_bytes!(conca /// MIN_HOLD_TEMPERATURE producer cubin. Single-thread cold-path kernel /// that reads `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` (sp13 fast EMA of /// binary directional accuracy), maps it to a [5, 50] temperature via -/// `temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × skill` where -/// `skill = clamp((short_ema − 0.5)/0.5, 0, 1)`, and slow-EMA-blends -/// into `ISV[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460]`. Replaces the +/// `temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × confusion` where +/// `skill = clamp((short_ema − 0.5)/0.5, 0, 1)` and +/// `confusion = 1 − skill`, and slow-EMA-blends into +/// `ISV[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460]`. The `confusion` +/// inversion (vs. raw `skill`) gives the WR-plateau scenario its +/// designed exit ramp: dir_acc ≈ 0.5 → confusion=1 → temp HIGH +/// (permissive); dir_acc → 1.0 → confusion=0 → temp LOW (strict). +/// HIGH T = forgiving (deficit/(deficit+T) → 0); LOW T = sharp +/// (deficit/(deficit+T) → 1) per `compute_min_hold_penalty` at +/// trade_physics.cuh:575. Replaces the /// deleted epoch-driven anneal schedule `T_end + (T_start − T_end) × /// exp(-epoch / decay)` (formerly state_layout.cuh:: /// MIN_HOLD_TEMPERATURE_{START, END, DECAY} + @@ -2486,7 +2493,7 @@ const ISV_NETWORK_DIM: usize = 23; /// (shifted 112→116 in Plan 4 Task 6 Commit A). /// Written by the constructor; checked at checkpoint load. Fail-fast only — no migration /// path exists. See spec §4.A.2 and `LAYOUT_FINGERPRINT_CURRENT` for structural-hash rationale. -pub(crate) const ISV_TOTAL_DIM: usize = 461; // SP5 + Layer D D1+D2+D3 + SP7 + SP8 + SP9 + SP10 + SP11 + SP13 + SP13 v3 + SP14 (post-C.1) + SP15 + SP14-C aux trunk control plane + SP14-C Phase C.4b aux horizon + Class A P0-A adaptive reward caps + Class A P1-Producer adaptive Bayesian Kelly priors + Class A audit-fix Batch 4-A adaptive DD saturation floor + Class A audit-fix Batch 4-B adaptive plan_threshold floor + adaptive MIN_HOLD_TEMPERATURE. Bumped 459 → 461 by Class A audit-fix Batch 4-B (2026-05-08): added 2 slots [459..461) — PLAN_THRESHOLD_FLOOR_ADAPTIVE (459) + MIN_HOLD_TEMPERATURE_ADAPTIVE (460). PLAN_THRESHOLD_FLOOR_ADAPTIVE replaces the hardcoded `0.1f` floor in `plan_threshold_update_kernel.cu:44` (`fmaxf(0.1f, 0.5f * ema)`) — Design Y inline producer (no new kernel; same kernel writes both slot 49 and the slow-EMA-shadow floor at slot 459 in the same launch, then uses the freshly-written floor as the lower bound). Pearl-A bootstrap (sentinel 0.1 matches pre-fix hardcoded value for bit-identical cold-start) + α=0.005 slow EMA (per-fold cadence — the floor is a long-term shadow of `0.5 × readiness_ema` and must move much slower than the readiness EMA itself). Bounds [0.05, 0.50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds` (probability units). MIN_HOLD_TEMPERATURE_ADAPTIVE replaces the deleted epoch-driven anneal schedule `T_end + (T_start − T_end) × exp(-epoch / decay)` (formerly state_layout.cuh::MIN_HOLD_TEMPERATURE_{START=50, END=5, DECAY=20} + training_loop.rs::min_hold_temperature_for_epoch). Producer kernel `min_hold_temperature_update_kernel` reads `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` (sp13 fast EMA of binary directional accuracy), maps to a [5, 50] temperature via `temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × clamp((short_ema − 0.5)/0.5, 0, 1)`, and slow-EMA-blends into slot 460. Pearl-A bootstrap (sentinel 50.0 matches the deleted MIN_HOLD_TEMPERATURE_START=50 anchor for bit-identical cold-start under both regimes) + α=0.05 mid-cadence EMA. Bounds [5, 50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds` (matches the original schedule range). Decouples temperature from epoch number — when WR plateaus, the old schedule pinned LOW temp (sharp) at end of training (max punishment exactly when most needed forgiveness); the signal-driven version reacts to actual realized skill. Per `feedback_no_legacy_aliases.md` the legacy schedule constants + helper function are deleted atomically. Bumped 458 → 459 by Class A audit-fix Batch 4-A (2026-05-08): added 1 adaptive-DD-saturation-floor slot [458..459) — DD_SATURATION_FLOOR_ADAPTIVE (458). Replaces the hardcoded `0.25f` saturation floor in `trade_physics.cuh::apply_margin_cap` (the upper end of the linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac / saturation_floor)). Distinct semantic role from `SP15_DD_THRESHOLD_INDEX=421` (the SP15 quadratic DD-penalty *trigger* threshold, a *lower* bound). Producer kernel `dd_saturation_floor_update_kernel` aggregates per-env DD_MAX from the existing `dd_state_per_env[env*6 + 1]` tile (same source as the SP15 dd_state_kernel writes to), computes p75 estimator via Welford rule (mean + Z_75 × sigma) × safety_factor=1.5, and slow-EMA blends. Pearl-A bootstrap (sentinel 0.25 matches pre-fix hardcoded value for bit-identical cold-start) + α=0.01 slow EMA (DD distribution is a slow-moving fold-volatility property, shouldn't track per-epoch noise). Bounds [0.10, 0.50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`. Atomically deleted alongside this slot addition: legacy `compute_drawdown_penalty` device function in trade_physics.cuh + its single caller at experience_kernels.cu:3822 + the `w_dd` and `dd_threshold` kernel args (Item 2 Case A — SP15's quadratic asymmetric DD penalty in `compute_sp15_final_reward_kernel.cu:154` is the production-grade replacement; layering both creates double-counting per `feedback_no_partial_refactor`). Bumped 454 → 458 by Class A P1-Producer (2026-05-08): added 4 adaptive-Bayesian-prior slots [454..458) — KELLY_PRIOR_WINS (454) + KELLY_PRIOR_LOSSES (455) + KELLY_PRIOR_SUM_WINS (456) + KELLY_PRIOR_SUM_LOSSES (457). Per Class A audit, hardcoded `prior_wins=2.0f / prior_losses=2.0f / prior_sum_wins=0.01f / prior_sum_losses=0.01f` (kelly_cap_update_kernel.cu:39-42 + trade_physics.cuh::kelly_position_cap:304-307) was a static "I don't know" Bayesian prior frozen at sprint-1 defaults — the cold-start of every new fold ignored what prior folds had learned about the realized Kelly distribution. Producer kernel `kelly_bayesian_priors_update_kernel` aggregates realized Kelly stats from the existing `portfolio_state[n_envs, PS_STRIDE]` buffer (same source as kelly_cap_update consumes) and slow-EMA blends into the four slots. Pearl-A bootstrap (sentinels 2.0/2.0/0.01/0.01 match pre-P1-Producer hardcoded values for bit-identical cold-start) + α=0.005 slow EMA (per-fold cadence; the Bayesian prior is a *prior belief* and should change slowly across folds). Bounds counts∈[0.5, 100], sums∈[0.001, 1.0] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`. Bumped 452 → 454 by Class A P0-A (2026-05-08): added 2 adaptive-reward-cap slots [452..454) — REWARD_POS_CAP_ADAPTIVE (452) + REWARD_NEG_CAP_ADAPTIVE (453). Per Class A audit, hardcoded `REWARD_POS_CAP=+5.0f` / `REWARD_NEG_CAP=-10.0f` (state_layout.cuh:266-267) was structurally clipping the upper tail of realized alpha — controller signals (sharpe EMA, var_q, q_gap) all derive from the CAPPED buffer, so the controller cannot select for trades it cannot see. The state_layout comment explicitly deferred Phase 2 (ISV-driven) "IF Phase 1 validation reveals adaptive need"; 11 superprojects of WR-stuck-at-46-48% revealed adaptive need. Producer kernel `reward_cap_update_kernel` computes p99(winning_returns) × safety_factor=1.5 → POS cap and NEG = -2 × POS (preserves Kahneman 2:1 loss-aversion asymmetry per `pearl_audit_unboundedness_for_implicit_asymmetry` — moved from hardcoded scalar to producer-time multiplier, single source of truth). Pearl-A bootstrap (sentinels 5.0/-10.0) + Wiener-α=0.01 slow EMA. Bounds POS∈[1, 50], NEG∈[-100, -2] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`, NOT tuning. Bumped 450 → 452 by SP14 Layer C Phase C.4b (2026-05-08): added 2 aux-prediction-horizon slots [450..452) — AUX_PRED_HORIZON_BARS (450) + AVG_WIN_HOLD_TIME_BARS (451). Aux's original label was (p_{t+1} > p_t), pure HFT-scale microstructure noise — pivoted to (p_{t+H} > p_t) with H read from ISV[450], adaptively driven from observed avg winning hold time via Pearl-A bootstrap + Wiener-α EMA. Case B: existing `hold_at_exit_per_sample` + `trade_profitable_per_sample` per-sample buffers, no aggregate slot — added new aggregate slot 451 to expose the EMA target. Bumped 444 → 450 by SP14 Layer C Phase C.1 (2026-05-08): atomically deleted 9 α-machinery slots in [385..396) AND added 6 aux-trunk control-plane slots [444..450) — AUX_TRUNK_LR (444) + AUX_TRUNK_BETA1 (445) + AUX_TRUNK_BETA2 (446) + AUX_TRUNK_EPS (447) + AUX_TRUNK_GRAD_CLIP (448) + H_S2_AUX_RMS_EMA (449). Per `feedback_no_legacy_aliases` and `feedback_no_partial_refactor`. Deleted slots left as RESERVED gap (NOT compacted) to preserve checkpoint layout fingerprint compatibility — the surviving SP14 q_disagreement_* diagnostic slots (383, 384, 389) keep their original indices. Pre-C.1 base: 173 + 210 (164 SP5 slots @ 174..278/280..340, with 2-slot gap before cross-fold-persistent Kelly block; Layer D D1 PnL outputs at [286..290); Layer D D2 health composition outputs at [290..294); Layer D D3 training metrics EMA at [294..297); SP7 T1 loss-balance Wiener stats at [297..313); SP7 activation-flag fix per-(head,branch) flags at [313..321); SP8 Fix 36 train_active_frac canary @ [321..322) + LB_MAX_BUDGET per-(head,branch) at [322..330); SP9 Fix 37 Kelly warmup floor at [330..331) + Q_VAR_MAG_EMA at [331..332) + INTENT_EVAL_DIVERGENCE at [332..333) + 3 EMA targets at [333..336) + 3 eval_dist mag bins at [336..339); SP10 Fix 38 EVAL_THOMPSON_TEMP at [339..340); SP11 Fix 39 reward-subsystem controller at [340..367) — 6 component weights [340..346) + 4 controller scalar outputs [346..350) + 2 val-sharpe canaries [350..352) + 6 mag-ratio canaries [352..358) + saboteur engagement [358] + PnL magnitude EMA [359] + popart-component magnitude EMA [360, B1b fix-up] + per-component variance EMAs [361..367, B1b smoke-recovery z-score normalization]; SP13 directional-skill instrumentation at [372..380) — TARGET_DIR_ACC [372] + AUX_DIR_ACC_SHORT/LONG_EMA [373..375) + AUX_DIR_PREDICTION [375] + DIR_SKILL_BONUS_ALPHA/BETA [376..378) + LUCK_WIN_DISCOUNT [378] + SKILL_BONUS_CAP_RATIO [379]; SP13 v3 P0a.T3 Hold-pricing controller at [380..383) — HOLD_COST [380] + HOLD_RATE_TARGET [381] + HOLD_RATE_OBSERVED_EMA [382]; SP14 q_disagreement diagnostic at [383..385), [389] — Q_DISAGREEMENT_SHORT/LONG_EMA + Q_DISAGREEMENT_VARIANCE_EMA (slots [385..389) and [390..396) RESERVED gap from C.1 deletion); SP15 [397..444) per Phase 1; SP14-C [444..450) per Phase C.1. Intentional 5-slot boundary gap at [367..372)) +pub(crate) const ISV_TOTAL_DIM: usize = 461; // SP5 + Layer D D1+D2+D3 + SP7 + SP8 + SP9 + SP10 + SP11 + SP13 + SP13 v3 + SP14 (post-C.1) + SP15 + SP14-C aux trunk control plane + SP14-C Phase C.4b aux horizon + Class A P0-A adaptive reward caps + Class A P1-Producer adaptive Bayesian Kelly priors + Class A audit-fix Batch 4-A adaptive DD saturation floor + Class A audit-fix Batch 4-B adaptive plan_threshold floor + adaptive MIN_HOLD_TEMPERATURE. Bumped 459 → 461 by Class A audit-fix Batch 4-B (2026-05-08): added 2 slots [459..461) — PLAN_THRESHOLD_FLOOR_ADAPTIVE (459) + MIN_HOLD_TEMPERATURE_ADAPTIVE (460). PLAN_THRESHOLD_FLOOR_ADAPTIVE replaces the hardcoded `0.1f` floor in `plan_threshold_update_kernel.cu:44` (`fmaxf(0.1f, 0.5f * ema)`) — Design Y inline producer (no new kernel; same kernel writes both slot 49 and the slow-EMA-shadow floor at slot 459 in the same launch, then uses the freshly-written floor as the lower bound). Pearl-A bootstrap (sentinel 0.1 matches pre-fix hardcoded value for bit-identical cold-start) + α=0.005 slow EMA (per-fold cadence — the floor is a long-term shadow of `0.5 × readiness_ema` and must move much slower than the readiness EMA itself). Bounds [0.05, 0.50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds` (probability units). MIN_HOLD_TEMPERATURE_ADAPTIVE replaces the deleted epoch-driven anneal schedule `T_end + (T_start − T_end) × exp(-epoch / decay)` (formerly state_layout.cuh::MIN_HOLD_TEMPERATURE_{START=50, END=5, DECAY=20} + training_loop.rs::min_hold_temperature_for_epoch). Producer kernel `min_hold_temperature_update_kernel` reads `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` (sp13 fast EMA of binary directional accuracy), maps to a [5, 50] temperature via `temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × confusion` where `confusion = 1 − clamp((short_ema − 0.5)/0.5, 0, 1)` (so dir_acc ≈ 0.5 plateau → temp HIGH/permissive exit ramp; dir_acc → 1.0 → temp LOW/strict informed-commitment), and slow-EMA-blends into slot 460. HIGH T = forgiving (`deficit/(deficit+T)` saturation factor → 0); LOW T = sharp (→ 1) per `compute_min_hold_penalty` at trade_physics.cuh:575. Pearl-A bootstrap (sentinel 50.0 matches the deleted MIN_HOLD_TEMPERATURE_START=50 anchor for bit-identical cold-start under both regimes) + α=0.05 mid-cadence EMA. Bounds [5, 50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds` (matches the original schedule range). Decouples temperature from epoch number — when WR plateaus, the old schedule pinned LOW temp (sharp) at end of training (max punishment exactly when most needed forgiveness); the signal-driven version reacts to actual realized skill. Per `feedback_no_legacy_aliases.md` the legacy schedule constants + helper function are deleted atomically. Bumped 458 → 459 by Class A audit-fix Batch 4-A (2026-05-08): added 1 adaptive-DD-saturation-floor slot [458..459) — DD_SATURATION_FLOOR_ADAPTIVE (458). Replaces the hardcoded `0.25f` saturation floor in `trade_physics.cuh::apply_margin_cap` (the upper end of the linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac / saturation_floor)). Distinct semantic role from `SP15_DD_THRESHOLD_INDEX=421` (the SP15 quadratic DD-penalty *trigger* threshold, a *lower* bound). Producer kernel `dd_saturation_floor_update_kernel` aggregates per-env DD_MAX from the existing `dd_state_per_env[env*6 + 1]` tile (same source as the SP15 dd_state_kernel writes to), computes p75 estimator via Welford rule (mean + Z_75 × sigma) × safety_factor=1.5, and slow-EMA blends. Pearl-A bootstrap (sentinel 0.25 matches pre-fix hardcoded value for bit-identical cold-start) + α=0.01 slow EMA (DD distribution is a slow-moving fold-volatility property, shouldn't track per-epoch noise). Bounds [0.10, 0.50] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`. Atomically deleted alongside this slot addition: legacy `compute_drawdown_penalty` device function in trade_physics.cuh + its single caller at experience_kernels.cu:3822 + the `w_dd` and `dd_threshold` kernel args (Item 2 Case A — SP15's quadratic asymmetric DD penalty in `compute_sp15_final_reward_kernel.cu:154` is the production-grade replacement; layering both creates double-counting per `feedback_no_partial_refactor`). Bumped 454 → 458 by Class A P1-Producer (2026-05-08): added 4 adaptive-Bayesian-prior slots [454..458) — KELLY_PRIOR_WINS (454) + KELLY_PRIOR_LOSSES (455) + KELLY_PRIOR_SUM_WINS (456) + KELLY_PRIOR_SUM_LOSSES (457). Per Class A audit, hardcoded `prior_wins=2.0f / prior_losses=2.0f / prior_sum_wins=0.01f / prior_sum_losses=0.01f` (kelly_cap_update_kernel.cu:39-42 + trade_physics.cuh::kelly_position_cap:304-307) was a static "I don't know" Bayesian prior frozen at sprint-1 defaults — the cold-start of every new fold ignored what prior folds had learned about the realized Kelly distribution. Producer kernel `kelly_bayesian_priors_update_kernel` aggregates realized Kelly stats from the existing `portfolio_state[n_envs, PS_STRIDE]` buffer (same source as kelly_cap_update consumes) and slow-EMA blends into the four slots. Pearl-A bootstrap (sentinels 2.0/2.0/0.01/0.01 match pre-P1-Producer hardcoded values for bit-identical cold-start) + α=0.005 slow EMA (per-fold cadence; the Bayesian prior is a *prior belief* and should change slowly across folds). Bounds counts∈[0.5, 100], sums∈[0.001, 1.0] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`. Bumped 452 → 454 by Class A P0-A (2026-05-08): added 2 adaptive-reward-cap slots [452..454) — REWARD_POS_CAP_ADAPTIVE (452) + REWARD_NEG_CAP_ADAPTIVE (453). Per Class A audit, hardcoded `REWARD_POS_CAP=+5.0f` / `REWARD_NEG_CAP=-10.0f` (state_layout.cuh:266-267) was structurally clipping the upper tail of realized alpha — controller signals (sharpe EMA, var_q, q_gap) all derive from the CAPPED buffer, so the controller cannot select for trades it cannot see. The state_layout comment explicitly deferred Phase 2 (ISV-driven) "IF Phase 1 validation reveals adaptive need"; 11 superprojects of WR-stuck-at-46-48% revealed adaptive need. Producer kernel `reward_cap_update_kernel` computes p99(winning_returns) × safety_factor=1.5 → POS cap and NEG = -2 × POS (preserves Kahneman 2:1 loss-aversion asymmetry per `pearl_audit_unboundedness_for_implicit_asymmetry` — moved from hardcoded scalar to producer-time multiplier, single source of truth). Pearl-A bootstrap (sentinels 5.0/-10.0) + Wiener-α=0.01 slow EMA. Bounds POS∈[1, 50], NEG∈[-100, -2] are Category-1 dimensional safety floors per `feedback_isv_for_adaptive_bounds`, NOT tuning. Bumped 450 → 452 by SP14 Layer C Phase C.4b (2026-05-08): added 2 aux-prediction-horizon slots [450..452) — AUX_PRED_HORIZON_BARS (450) + AVG_WIN_HOLD_TIME_BARS (451). Aux's original label was (p_{t+1} > p_t), pure HFT-scale microstructure noise — pivoted to (p_{t+H} > p_t) with H read from ISV[450], adaptively driven from observed avg winning hold time via Pearl-A bootstrap + Wiener-α EMA. Case B: existing `hold_at_exit_per_sample` + `trade_profitable_per_sample` per-sample buffers, no aggregate slot — added new aggregate slot 451 to expose the EMA target. Bumped 444 → 450 by SP14 Layer C Phase C.1 (2026-05-08): atomically deleted 9 α-machinery slots in [385..396) AND added 6 aux-trunk control-plane slots [444..450) — AUX_TRUNK_LR (444) + AUX_TRUNK_BETA1 (445) + AUX_TRUNK_BETA2 (446) + AUX_TRUNK_EPS (447) + AUX_TRUNK_GRAD_CLIP (448) + H_S2_AUX_RMS_EMA (449). Per `feedback_no_legacy_aliases` and `feedback_no_partial_refactor`. Deleted slots left as RESERVED gap (NOT compacted) to preserve checkpoint layout fingerprint compatibility — the surviving SP14 q_disagreement_* diagnostic slots (383, 384, 389) keep their original indices. Pre-C.1 base: 173 + 210 (164 SP5 slots @ 174..278/280..340, with 2-slot gap before cross-fold-persistent Kelly block; Layer D D1 PnL outputs at [286..290); Layer D D2 health composition outputs at [290..294); Layer D D3 training metrics EMA at [294..297); SP7 T1 loss-balance Wiener stats at [297..313); SP7 activation-flag fix per-(head,branch) flags at [313..321); SP8 Fix 36 train_active_frac canary @ [321..322) + LB_MAX_BUDGET per-(head,branch) at [322..330); SP9 Fix 37 Kelly warmup floor at [330..331) + Q_VAR_MAG_EMA at [331..332) + INTENT_EVAL_DIVERGENCE at [332..333) + 3 EMA targets at [333..336) + 3 eval_dist mag bins at [336..339); SP10 Fix 38 EVAL_THOMPSON_TEMP at [339..340); SP11 Fix 39 reward-subsystem controller at [340..367) — 6 component weights [340..346) + 4 controller scalar outputs [346..350) + 2 val-sharpe canaries [350..352) + 6 mag-ratio canaries [352..358) + saboteur engagement [358] + PnL magnitude EMA [359] + popart-component magnitude EMA [360, B1b fix-up] + per-component variance EMAs [361..367, B1b smoke-recovery z-score normalization]; SP13 directional-skill instrumentation at [372..380) — TARGET_DIR_ACC [372] + AUX_DIR_ACC_SHORT/LONG_EMA [373..375) + AUX_DIR_PREDICTION [375] + DIR_SKILL_BONUS_ALPHA/BETA [376..378) + LUCK_WIN_DISCOUNT [378] + SKILL_BONUS_CAP_RATIO [379]; SP13 v3 P0a.T3 Hold-pricing controller at [380..383) — HOLD_COST [380] + HOLD_RATE_TARGET [381] + HOLD_RATE_OBSERVED_EMA [382]; SP14 q_disagreement diagnostic at [383..385), [389] — Q_DISAGREEMENT_SHORT/LONG_EMA + Q_DISAGREEMENT_VARIANCE_EMA (slots [385..389) and [390..396) RESERVED gap from C.1 deletion); SP15 [397..444) per Phase 1; SP14-C [444..450) per Phase C.1. Intentional 5-slot boundary gap at [367..372)) /// Legacy alias preserved for call sites that haven't been audited for the /// network-vs-total split. New code should pick `ISV_NETWORK_DIM` (for weight /// tensor sizing) or `ISV_TOTAL_DIM` (for the broadcast bus buffer). diff --git a/crates/ml/src/cuda_pipeline/min_hold_temperature_update_kernel.cu b/crates/ml/src/cuda_pipeline/min_hold_temperature_update_kernel.cu index cf82d1302..a68c9ee7a 100644 --- a/crates/ml/src/cuda_pipeline/min_hold_temperature_update_kernel.cu +++ b/crates/ml/src/cuda_pipeline/min_hold_temperature_update_kernel.cu @@ -1,6 +1,18 @@ /* ══════════════════════════════════════════════════════════════════════════ * Class A audit-fix Batch 4-B — adaptive MIN_HOLD_TEMPERATURE producer - * (2026-05-08, Item 4). + * (2026-05-08, Item 4). Fixup 2026-05-08: inverted the dir_acc → temp + * mapping after kernel-formula audit (`compute_min_hold_penalty` at + * trade_physics.cuh:575): HIGH T → soft_factor → 0 → forgiving exits; + * LOW T → soft_factor → 1 → sharp penalty. The original schedule + * `START=50 → END=5` therefore means "permissive early, strict late" + * (classic explore-then-exploit). The pre-fixup mapping + * `temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill` was BACKWARDS for + * the WR-plateau scenario this 8-commit chain targets: at plateau + * (dir_acc ≈ 0.46-0.48, model committed but wrong) skill = 0 → temp = 5 + * (sharp) → maximum exit penalty exactly when the model most needs to + * escape the bad committed direction. Inverted to `confusion = 1 - skill` + * so plateau → temp HIGH (permissive, exit ramp) and skilled → + * temp LOW (strict, force commitment). * * Replaces the deleted epoch-driven anneal schedule * `T_end + (T_start - T_end) × exp(-epoch / decay)` @@ -9,8 +21,9 @@ * ISV-driven signal-driven temperature derived from the model's * directional-accuracy skill. The temperature controls the * `deficit / (deficit + T)` saturation factor in `compute_min_hold_penalty` - * (trade_physics.cuh:575): HIGH T = forgiving (gradient is gentle near - * deficit=0); LOW T = sharp (penalty rises steeply at any deficit). + * (trade_physics.cuh:575): HIGH T = forgiving (penalty saturates near + * 0 — easy to exit early); LOW T = sharp (penalty rises steeply at any + * deficit — hold is forced). * * Why this matters (per Class A audit deferred-item batch 4-B): * @@ -20,19 +33,24 @@ * plateaus. Maximum punishment is delivered exactly when the model * most needs forgiveness to escape the plateau. The signal-driven * replacement decouples temperature from epoch number: when the - * model is committing skillfully (dir_acc > 0.5) → temp HIGH - * (permissive); when at random baseline (dir_acc ≈ 0.5) → temp LOW - * (sharp commitment pressure to escape the random-guess regime). + * model is at random baseline / WR plateau (dir_acc ≈ 0.5, + * confusion ≈ 1) → temp HIGH (permissive, exit ramp to escape bad + * committed direction); when committing skillfully (dir_acc → 1.0, + * confusion → 0) → temp LOW (strict, force commitment because the + * commitment is informed). * * Driving signal: dir_acc skill from * `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` — the SP13 fast EMA of * binary directional accuracy (sentinel 0.5 = random baseline). * - * skill = clamp((short_ema - 0.5) / 0.5, 0, 1) ∈ [0, 1] - * temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill + * skill = clamp((short_ema - 0.5) / 0.5, 0, 1) ∈ [0, 1] + * confusion = 1 - skill ∈ [0, 1] + * temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × confusion * - * - skill=0 (no directional skill, dir_acc ≤ 0.5) → temp=TEMP_MIN=5 - * - skill=1 (saturated dir_acc=1.0) → temp=TEMP_MAX=50 + * - confusion=1 (no directional skill, dir_acc ≤ 0.5) → temp=TEMP_MAX=50 + * (permissive — give the model an exit ramp out of the plateau) + * - confusion=0 (saturated dir_acc=1.0) → temp=TEMP_MIN=5 + * (strict — force commitment because it's informed) * * Cold-start fallback: if `isv[AUX_DIR_ACC_SHORT_EMA_INDEX]` is at * sentinel (within EPS of 0.5) — i.e. fold reset just fired and no @@ -52,7 +70,8 @@ * MIN_HOLD_TEMPERATURE_FALLBACK=50.0). Otherwise: * * skill = max(0, min(1, (short_ema - 0.5) / 0.5)) - * target_temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill + * confusion = 1 - skill + * target_temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × confusion * bilateral clamp to [TEMP_MIN, TEMP_MAX] * * Pearl-A bootstrap: if current is at sentinel (within EPS of 50.0), @@ -125,14 +144,34 @@ void min_hold_temperature_update( return; } - /* Skill mapping: clamp((short_ema - 0.5) / 0.5, 0, 1). - * - dir_acc=0.5 (random) → skill=0 → temp=temp_min=5 (sharp) - * - dir_acc=1.0 (saturated) → skill=1 → temp=temp_max=50 (forgiving) - * - dir_acc<0.5 (worse than random) → skill clamped to 0 → temp=5 */ + /* Confusion mapping: confusion = 1 - clamp((short_ema - 0.5)/0.5, 0, 1). + * Inverted from the original "skill → temp" mapping after + * trade_physics.cuh:575 audit revealed HIGH T = permissive + * (deficit/(deficit+T) saturates to 0 → no exit penalty), LOW T = + * strict (saturates to 1 → max exit penalty). The WR-plateau + * scenario this 8-commit chain targets is "model committed but + * wrong at dir_acc ≈ 0.46-0.48"; the model needs the permissive + * branch (HIGH T) to escape the bad committed direction, which + * means HIGH confusion → HIGH temp. + * + * - dir_acc=0.5 (random / plateau) → skill=0 → confusion=1 → + * temp=temp_max=50 (permissive — exit ramp) + * - dir_acc=1.0 (saturated skill) → skill=1 → confusion=0 → + * temp=temp_min=5 (strict — force informed commitment) + * - dir_acc<0.5 (worse than random) → skill clamped to 0 → + * confusion=1 → temp=50 (max permissive) + * + * Note: the dir_acc==0.5 sentinel-guard fires before this branch + * (returns without writing); the explicit `confusion=1 → temp=50` + * arm above only fires for dir_acc < 0.5 - EPS (non-sentinel + * worse-than-random) or when the upstream EMA has moved off + * sentinel toward 0.5+ε. This matches the deleted schedule's + * epoch-0 anchor (T=50). */ float skill = (dir_acc_short - sentinel_dir_acc) / sentinel_dir_acc; skill = fmaxf(0.0f, fminf(1.0f, skill)); + const float confusion = 1.0f - skill; - float target_temp = temp_min + (temp_max - temp_min) * skill; + float target_temp = temp_min + (temp_max - temp_min) * confusion; /* Bilateral clamp per `pearl_symmetric_clamp_audit.md`. Defends * against arithmetic round-off pushing target outside the diff --git a/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs b/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs index 062527893..3d4427745 100644 --- a/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs +++ b/crates/ml/src/cuda_pipeline/sp14_isv_slots.rs @@ -344,29 +344,48 @@ pub const SP14_AUDIT_4B_PLAN_FLOOR_SLOT_END: usize = 460; // temperature derived from the model's directional accuracy skill. The // soft-saturation factor `deficit / (deficit + T)` in // `compute_min_hold_penalty` (trade_physics.cuh:575) controls how -// sharply the min-hold penalty bites: HIGH T = forgiving (gradient is -// gentle near deficit=0); LOW T = sharp (penalty rises steeply at any -// deficit). +// sharply the min-hold penalty bites: HIGH T → soft_factor → 0 → +// forgiving (no exit penalty); LOW T → soft_factor → 1 → sharp +// (max exit penalty). The deleted schedule's `START=50 → END=5` +// therefore meant "permissive early, strict late" — classic +// explore-then-exploit. // // Driving signal: dir_acc skill — `clamp((short_ema - 0.5) / 0.5, 0, 1)` // from ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373] (sp13 directional-accuracy -// fast EMA, sentinel 0.5 = random-baseline). When the model is -// committing skillfully (dir_acc > 0.5) → temp HIGH (permissive — the -// model's exits are informed, no need to penalize sharply). When the -// model is at random baseline (dir_acc ≈ 0.5) → temp LOW (sharp — the -// model needs commitment pressure to escape the random-guess regime). +// fast EMA, sentinel 0.5 = random-baseline). The mapping uses +// `confusion = 1 - skill` so the model gets a permissive exit ramp +// when at the WR-plateau (committed-but-wrong) regime and a strict +// hold-enforcement when its commitment is informed: +// - dir_acc ≈ 0.5 (random / WR-plateau) → confusion=1 → temp HIGH +// (permissive — let the model bail out of the bad committed +// direction; this is the WR-plateau exit ramp) +// - dir_acc → 1.0 (saturated skill) → confusion=0 → temp LOW +// (strict — force the model to stay in informed positions) // -// Mapping: temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill, where: -// - skill=0 (no directional skill, dir_acc ≤ 0.5) → temp=TEMP_MIN=5 -// - skill=1 (saturated dir_acc=1.0) → temp=TEMP_MAX=50 +// Mapping: temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × confusion, where: +// - confusion=1 (dir_acc ≤ 0.5) → temp=TEMP_MAX=50 (permissive) +// - confusion=0 (dir_acc=1.0) → temp=TEMP_MIN=5 (strict) // -// Opposite of the deleted epoch-driven schedule which pinned LOW temp -// at end of training (T_end=5). The epoch schedule assumes the model -// gets MORE skilled with training; when WR plateaus this assumption -// fails and the schedule becomes maximally punishing exactly when the -// model needs forgiveness to escape the plateau. The signal-driven -// version reacts to actual realized skill, decoupling temperature from -// epoch number. +// This preserves the deleted schedule's MAX-at-cold-start anchor +// (T=50 at epoch 0) while decoupling the post-cold trajectory from +// the epoch number. Under the old schedule, end-of-training pinned +// LOW temp (T_end=5), assuming the model would be more skilled by +// then; when WR plateaus this assumption fails and the schedule +// becomes maximally punishing exactly when the model needs forgiveness +// to escape the plateau. The signal-driven, confusion-mapped +// replacement reacts to actual realized skill — the model only sees +// strict enforcement when its commitments are informed, never as a +// pre-set time anchor. +// +// FIXUP 2026-05-08: original mapping was the inverted form +// (`temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × skill`), which +// punished plateau exits with max sharpness. After kernel-formula +// audit (`compute_min_hold_penalty` — HIGH T = forgiving) and +// matching the deleted schedule's "permissive early" intent, the +// mapping was inverted to use confusion. Numerical fixed points: +// dir_acc=0.5 (sentinel) → guard returns; dir_acc=0.75 → temp=27.5 +// (midpoint, identical pre/post fixup); dir_acc=1.0 → temp=5; +// dir_acc<0.5 → temp=50. // // Producer kernel: `min_hold_temperature_update_kernel.cu` (per-epoch // boundary cadence). Consumer: `experience_kernels.cu` reads ISV[460] diff --git a/crates/ml/tests/sp14_oracle_tests.rs b/crates/ml/tests/sp14_oracle_tests.rs index c854694b3..14886bba4 100644 --- a/crates/ml/tests/sp14_oracle_tests.rs +++ b/crates/ml/tests/sp14_oracle_tests.rs @@ -1819,9 +1819,11 @@ mod sp14_audit_4b_min_hold_temperature_gpu { /// Test 1 — Pearl-A first-observation bootstrap. /// - /// Seed dir_acc EMA at 0.75 (skill = (0.75 − 0.5)/0.5 = 0.5 → temp = - /// 5 + 45 × 0.5 = 27.5). Pearl-A: temp at sentinel 50.0 → REPLACE - /// directly with 27.5. + /// Seed dir_acc EMA at 0.75 (skill = (0.75 − 0.5)/0.5 = 0.5; + /// confusion = 1 − skill = 0.5 → temp = 5 + 45 × 0.5 = 27.5). + /// Pearl-A: temp at sentinel 50.0 → REPLACE directly with 27.5. + /// Midpoint: numeric result is identical to a pre-fixup + /// "skill→temp" mapping; only the semantic interpretation flipped. #[test] #[ignore = "requires GPU"] fn min_hold_temp_pearl_a_bootstrap() { @@ -1850,7 +1852,9 @@ mod sp14_audit_4b_min_hold_temperature_gpu { let result = isv_buf.read_all(); let temp = result[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX]; - // skill = (0.75 − 0.5) / 0.5 = 0.5; target = 5 + 45 × 0.5 = 27.5. + // skill = (0.75 − 0.5) / 0.5 = 0.5; confusion = 1 − skill = 0.5; + // target = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × confusion + // = 5 + 45 × 0.5 = 27.5. let expected = 27.5_f32; assert!( (temp - expected).abs() < 1e-4, @@ -1895,20 +1899,24 @@ mod sp14_audit_4b_min_hold_temperature_gpu { ); } - /// Test 3 — High dir_acc skill → temp HIGH (permissive). + /// Test 3 — High dir_acc skill → temp LOW (strict, force informed + /// commitment). /// - /// Seed dir_acc=1.0 (perfect skill → skill=1.0 → temp=5+45=50). - /// Verifies the spec mapping "model commits skillfully → temp HIGH". - /// Pearl-A REPLACE on sentinel. + /// Seed dir_acc=1.0 (perfect skill → skill=1.0 → confusion=0 → + /// temp=TEMP_MIN=5). Verifies the inverted (post-fixup) semantic + /// "skilled commitment → temp LOW (strict)" — when the model is + /// committing skillfully there's no plateau-escape need, so the + /// kernel pins temp to the strict end (max exit penalty enforces + /// the informed commitment). Pearl-A REPLACE on sentinel. #[test] #[ignore = "requires GPU"] - fn min_hold_temp_high_dir_acc_yields_high_temp() { + fn min_hold_temp_high_dir_acc_yields_low_temp() { let stream = make_stream(); let kernel = load_kernel(&stream); const ISV_DIM: usize = 1024; let mut isv = vec![0.0_f32; ISV_DIM]; - // Saturated dir_acc → maximum skill → temp at MAX (permissive). + // Saturated dir_acc → maximum skill → confusion=0 → temp at MIN (strict). isv[AUX_DIR_ACC_SHORT_EMA_INDEX] = 1.0; isv[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX] = SENTINEL_MIN_HOLD_TEMPERATURE; @@ -1927,27 +1935,33 @@ mod sp14_audit_4b_min_hold_temperature_gpu { let result = isv_buf.read_all(); let temp = result[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX]; - // skill=1.0 → temp = TEMP_MAX = 50.0. + // skill=1.0 → confusion=0 → temp = TEMP_MIN = 5.0. assert!( - (temp - MIN_HOLD_TEMPERATURE_MAX).abs() < 1e-4, - "High dir_acc must drive temp HIGH (permissive); expected {}, got {temp}", - MIN_HOLD_TEMPERATURE_MAX + (temp - MIN_HOLD_TEMPERATURE_MIN).abs() < 1e-4, + "High dir_acc must drive temp LOW (strict — informed commitment); \ + expected {}, got {temp}", + MIN_HOLD_TEMPERATURE_MIN ); } - /// Test 4 — Low dir_acc (worse than random) → temp LOW (sharp). + /// Test 4 — Low dir_acc (worse than random / WR-plateau scenario) + /// → temp HIGH (permissive — exit ramp). /// /// Seed dir_acc=0.4 (worse than 0.5 random baseline). skill clamps - /// to 0 → temp = TEMP_MIN = 5 (sharp commitment pressure). + /// to 0 → confusion=1 → temp=TEMP_MAX=50 (permissive). This is the + /// canonical WR-plateau scenario this 8-commit chain targets: + /// the model is committed but wrong, and the kernel must give it + /// an exit ramp (HIGH T = soft exit penalty) rather than + /// punishing exits and locking it into the bad direction. #[test] #[ignore = "requires GPU"] - fn min_hold_temp_low_dir_acc_yields_low_temp() { + fn min_hold_temp_low_dir_acc_yields_high_temp() { let stream = make_stream(); let kernel = load_kernel(&stream); const ISV_DIM: usize = 1024; let mut isv = vec![0.0_f32; ISV_DIM]; - // dir_acc=0.4 (worse than random — skill clamps to 0). + // dir_acc=0.4 (worse than random — skill clamps to 0, confusion=1). isv[AUX_DIR_ACC_SHORT_EMA_INDEX] = 0.4; isv[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX] = SENTINEL_MIN_HOLD_TEMPERATURE; @@ -1966,20 +1980,23 @@ mod sp14_audit_4b_min_hold_temperature_gpu { let result = isv_buf.read_all(); let temp = result[MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX]; - // skill clamped to 0 → target = TEMP_MIN. Pearl-A REPLACE on - // sentinel → final temp = TEMP_MIN. + // skill clamped to 0 → confusion=1 → target = TEMP_MAX. Pearl-A + // REPLACE on sentinel → final temp = TEMP_MAX. assert!( - (temp - MIN_HOLD_TEMPERATURE_MIN).abs() < 1e-4, - "Low dir_acc must drive temp LOW (sharp); expected {}, got {temp}", - MIN_HOLD_TEMPERATURE_MIN + (temp - MIN_HOLD_TEMPERATURE_MAX).abs() < 1e-4, + "Low dir_acc must drive temp HIGH (permissive — plateau exit ramp); \ + expected {}, got {temp}", + MIN_HOLD_TEMPERATURE_MAX ); } /// Test 5 — Mid-cadence EMA blend after bootstrap. /// /// Seed temp at 30.0 (NOT sentinel — past bootstrap), dir_acc=0.75 - /// (target_temp = 27.5). Blended = (1 − 0.05) × 30.0 + 0.05 × 27.5 - /// = 28.5 + 1.375 = 29.875. + /// (skill=0.5, confusion=0.5, target_temp = 5+45×0.5 = 27.5). + /// Blended = (1 − 0.05) × 30.0 + 0.05 × 27.5 = 28.5 + 1.375 = 29.875. + /// Midpoint: numeric result is identical to a pre-fixup + /// "skill→temp" mapping; only the semantic interpretation flipped. #[test] #[ignore = "requires GPU"] fn min_hold_temp_mid_cadence_ema_after_bootstrap() { diff --git a/docs/dqn-wire-up-audit.md b/docs/dqn-wire-up-audit.md index 265300157..9e3c2b662 100644 --- a/docs/dqn-wire-up-audit.md +++ b/docs/dqn-wire-up-audit.md @@ -45,19 +45,20 @@ T(e) = T_end + (T_start - T_end) × exp(-e / decay) (formerly `state_layout.cuh::MIN_HOLD_TEMPERATURE_{START=50, END=5, DECAY=20}` + `training_loop.rs::min_hold_temperature_for_epoch`) with an ISV-driven signal-driven temperature derived from the model's directional accuracy skill. Producer kernel: `min_hold_temperature_update_kernel.cu` (NEW file, single-thread cold-path, per-epoch boundary launch). **Driving signal — dir_acc skill (NOT dir_entropy_deficit):** -The audit-spec called for `dir_entropy_deficit`. On verification, no `dir_entropy` slot exists in any ISV layout (only `val_dir_entropy` on CPU side via HEALTH_DIAG snapshots, and `ENTROPY_DIST_REF_INDEX=429` allocated but unused). The cleanest available signal is `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` — the SP13 fast EMA of binary directional accuracy (sentinel 0.5 = random baseline). This serves as a parallel proxy for "model commits" via dir_acc skill: +The audit-spec called for `dir_entropy_deficit`. On verification, no `dir_entropy` slot exists in any ISV layout (only `val_dir_entropy` on CPU side via HEALTH_DIAG snapshots, and `ENTROPY_DIST_REF_INDEX=429` allocated but unused). The cleanest available signal is `ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]` — the SP13 fast EMA of binary directional accuracy (sentinel 0.5 = random baseline). This serves as a parallel proxy for "model is uncertain" via dir_acc confusion: ``` -skill = clamp((short_ema − 0.5) / 0.5, 0, 1) ∈ [0, 1] -temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × skill +skill = clamp((short_ema − 0.5) / 0.5, 0, 1) ∈ [0, 1] +confusion = 1 − skill ∈ [0, 1] +temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × confusion ``` -- skill=0 (no skill, dir_acc ≤ 0.5) → temp=TEMP_MIN=5 (sharp commitment pressure) -- skill=1 (saturated dir_acc=1.0) → temp=TEMP_MAX=50 (permissive — informed exits, no need to penalize sharply) +- confusion=1 (no skill, dir_acc ≤ 0.5) → temp=TEMP_MAX=50 (permissive — exit ramp out of plateau / random-baseline commitments) +- confusion=0 (saturated dir_acc=1.0) → temp=TEMP_MIN=5 (strict — force the model to stay in informed positions) -This matches the audit-spec semantic ("model commits → temp HIGH"). Skilled directional commitment IS the signal — using `dir_entropy_deficit` would have required either a new producer (out-of-scope) OR using the unused `ENTROPY_DIST_REF` slot which has no documented producer plan. +The mapping uses `confusion` (not raw `skill`) because the consumer formula `compute_min_hold_penalty` (`soft_factor = deficit / (deficit + T)` at trade_physics.cuh:575) makes HIGH T = forgiving (saturation factor → 0, no exit penalty) and LOW T = sharp (saturation factor → 1, max exit penalty). The deleted schedule's `START=50 → END=5` therefore meant "permissive early, strict late" (classic explore-then-exploit). Mapping `skill → temp` directly would be backwards: at the WR-plateau scenario this 8-commit chain targets (dir_acc ≈ 0.46-0.48, model committed but wrong), `skill=0` → `temp=5` would lock the model into bad committed directions exactly when an exit ramp is needed. The `confusion` inversion preserves the deleted schedule's epoch-0 anchor (T=50 cold-start) and gives the plateau scenario its designed exit ramp. **Slot allocation:** ISV[460..461) — `MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460` (sentinel 50.0, bounds [5, 50], α=0.05). Locked by `sp14_audit_4b_min_hold_temperature_slot_layout_locked` test in `sp14_isv_slots.rs`. -**Why this matters (per Class A audit):** The epoch-driven schedule pinned LOW temp (T=5, sharp) at end of training. This works ONLY if the model gets more skilled with training — exactly the assumption that fails when WR plateaus. Maximum punishment is delivered exactly when the model most needs forgiveness to escape the plateau. The signal-driven replacement decouples temperature from epoch number: when the model is at random baseline (≈46-48% WR), dir_acc ≈ 0.5 → skill ≈ 0 → temp=5 (sharp), forcing commitment. When the model breaks through to dir_acc > 0.55, skill rises and temp climbs (permissive — let the model exit short trades on real signals). +**Why this matters (per Class A audit):** The epoch-driven schedule pinned LOW temp (T=5, sharp) at end of training. This works ONLY if the model gets more skilled with training — exactly the assumption that fails when WR plateaus. Maximum punishment is delivered exactly when the model most needs forgiveness to escape the plateau. The signal-driven replacement decouples temperature from epoch number and inverts the dir_acc → temp coupling: when the model is at random baseline (≈46-48% WR), dir_acc ≈ 0.5 → confusion ≈ 1 → temp=50 (permissive, exit ramp out of bad committed direction). When the model breaks through to dir_acc → 1.0, confusion → 0 and temp drops toward 5 (strict — force commitment because it's now informed). This is "permissive when uncertain, strict when informed" — the opposite of the broken epoch-anchored schedule. **DELETED atomically:** Per `feedback_no_legacy_aliases.md` + `feedback_no_partial_refactor.md`: 1. `state_layout.cuh::MIN_HOLD_TEMPERATURE_{START, END, DECAY}` #defines. @@ -80,7 +81,7 @@ This matches the audit-spec semantic ("model commits → temp HIGH"). Skilled di - `crates/ml/build.rs` — kernel added to compile list. - `crates/ml/src/trainers/dqn/state_reset_registry.rs` — `sp14_audit_4b_min_hold_temperature_adaptive` registry entry. - `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — DELETED `min_hold_temperature_for_epoch` body, DELETED both call sites, ADDED `read_min_hold_temperature_from_isv` helper, ADDED launcher invocation per-epoch, ADDED FoldReset dispatch arm. -- `crates/ml/tests/sp14_oracle_tests.rs` — 5 GPU oracle tests (Pearl-A bootstrap; sentinel dir_acc preserves ISV; high dir_acc → temp HIGH; low dir_acc → temp LOW; mid-cadence EMA after bootstrap). +- `crates/ml/tests/sp14_oracle_tests.rs` — 5 GPU oracle tests (Pearl-A bootstrap; sentinel dir_acc preserves ISV; high dir_acc → temp LOW [strict, informed commitment]; low dir_acc → temp HIGH [permissive, plateau exit ramp]; mid-cadence EMA after bootstrap). ### Verification @@ -100,11 +101,13 @@ ISV_TOTAL_DIM: 459 → 461. The audit-spec was correct on Item 3 (plan_threshold floor) — line 44, formula `fmaxf(0.1f, 0.5f * ema)`, `0.1f` is the absolute floor and `0.5f * ema` is the relative one. Picked Design Y (inline) per the audit's recommendation. The audit-spec was **partially wrong on Item 4** in two ways: -1. **Driving signal:** `dir_entropy_deficit` does not exist as an ISV signal. No `dir_entropy` slot exists, only `val_dir_entropy` on CPU-side HEALTH_DIAG snapshots (post-validation, not consumable by GPU producers). `ENTROPY_DIST_REF_INDEX=429` is allocated but unused. Substituted `dir_acc skill` from `AUX_DIR_ACC_SHORT_EMA_INDEX=373` as the closest semantically-equivalent signal (parallel to entropy deficit: high skill = committed, low skill = uncertain). Mapping preserves the spec's intent ("model commits → temp HIGH"). +1. **Driving signal:** `dir_entropy_deficit` does not exist as an ISV signal. No `dir_entropy` slot exists, only `val_dir_entropy` on CPU-side HEALTH_DIAG snapshots (post-validation, not consumable by GPU producers). `ENTROPY_DIST_REF_INDEX=429` is allocated but unused. Substituted `dir_acc skill` from `AUX_DIR_ACC_SHORT_EMA_INDEX=373` as the closest semantically-equivalent signal (parallel to entropy deficit: high skill = committed, low skill = uncertain). 2. **Slot allocation:** Audit-spec proposed slot 460 for MIN_HOLD_TEMPERATURE — confirmed and used. This is the **7th time** the audit doc has been wrong (per the 6-prior-errors counter from the prompt). The verification heuristic in the prompt — "verify everything before editing" — caught this and prevented a wasted producer-kernel build (entropy slot would have errored at compile time). +**Mapping-direction fixup (2026-05-08):** The initial wiring of this fix used `temp = TEMP_MIN + (TEMP_MAX − TEMP_MIN) × skill` based on the audit-spec phrase "model commits → temp HIGH". On post-commit review the kernel formula `compute_min_hold_penalty` was re-read at trade_physics.cuh:567-577: `soft_factor = deficit / (deficit + T)`. HIGH T → soft_factor → 0 → forgiving (no exit penalty); LOW T → soft_factor → 1 → sharp (max exit penalty). The deleted schedule's `START=50 → END=5` therefore meant "permissive early, strict late" (classic explore-then-exploit), not "permissive when committed, strict when uncertain". The original `skill → temp` mapping was backwards for the WR-plateau scenario this 8-commit chain targets: at plateau (dir_acc ≈ 0.46-0.48, committed but wrong), `skill=0 → temp=5` would have locked the model into bad committed directions with maximum exit penalty — exactly when an exit ramp is needed. Inverted to `confusion = 1 − skill` so plateau → temp HIGH (permissive exit ramp) and saturated skill → temp LOW (strict, force informed commitment). Tests 3 and 4 flipped accordingly; tests 1, 2, 5 (sentinel guard, midpoint, midpoint EMA) are numerically unchanged because their dir_acc inputs sit at 0.5/0.75 fixed points where `skill ≈ confusion` or the guard fires before the mapping. + Reading directly from `compute_min_hold_penalty` in `trade_physics.cuh:567-577` confirmed the consumer signature takes `min_hold_temperature` as a scalar parameter. Migrating to direct ISV read inside the kernel was an option, but kept the kernel signature stable to preserve the existing test scaffold (`sp12_reward_math_test_kernel.cu` at line 36 also uses the scalar arg). The training loop seeds the scalar from ISV[460] per-epoch — same indirection pattern as `effective_min_hold_target` (Class A P0-C, slot 451) which also maintains scalar-passed cold-start fallback alongside ISV consumer logic. ## 2026-05-08 — Class A audit-fix Batch 4-A: DD saturation floor adaptive + legacy DD path Case A