feat(sp18 v2 P3.T1-T5): adaptive HOLD_REWARD_POS/NEG_CAP producer kernel

Lifts the Phase 2 caps from sentinel-driven cold-start (5.0/-10.0
fixed) to p99(|step_ret|) over Long/Short trade closes × 1.5 safety
factor with Wiener-optimal alpha blend. The Phase 2 consumer
(compute_sp18_hold_opportunity_cost) sees producer-driven slots
[483]/[484] from epoch 1 onward; sentinel branch in the consumer
remains as the cold-start fallback path (zero-trade-close epoch ⇒
producer early-returns ⇒ slots stay at sentinel ⇒ consumer falls
through to the +5/-10 macro defaults — bit-identical to pre-Phase-3).

Mirrors SP14 P0-A reward_cap_update_kernel structural template with
three differences: (1) filter (is_close && step_ret != 0) — both
winners AND losers (the consumer needs the magnitude scale of *all*
Long/Short closes); (2) Welford-derived Wiener-α (slots [487..493))
replaces fixed α=0.01, with floor at WELFORD_ALPHA_MIN=0.4 per
pearl_wiener_alpha_floor_for_nonstationary (the policy-realised
distribution is intrinsically non-stationary as the policy adapts);
(3) bounds [0.5, 50.0] (vs. position-side [1.0, 50.0]).

Atomic single-commit per feedback_no_partial_refactor:
- crates/ml/src/cuda_pipeline/hold_reward_cap_update_kernel.cu (NEW)
- crates/ml/build.rs cubin manifest entry
- HoldRewardCapUpdateOps in gpu_aux_trunk.rs (new struct + impl)
- HOLD_REWARD_CAP_UPDATE_CUBIN static + struct field +
  launch_hold_reward_cap_update method + constructor instantiation +
  field-init in gpu_dqn_trainer.rs (5 sites)
- Per-epoch boundary launch in training_loop.rs right AFTER
  launch_reward_cap_update (shared step_ret/trade_close source buffers,
  independent ISV slot pairs)
- HEALTH_DIAG[N]: hold_reward_cap [pos={:.4} neg={:.4} fire_rate={:.4}]
- 3 GPU oracle tests (T5 producer-drives-slots, Pearl-A REPLACE,
  no-closes preserves-isv) — all pass on local RTX 3050 Ti
- Phase 3 close-out sections in docs/sp18-wireup-audit.md and
  docs/dqn-wire-up-audit.md

Pearls applied: feedback_no_atomicadd, pearl_first_observation_bootstrap,
pearl_wiener_optimal_adaptive_alpha, pearl_wiener_alpha_floor_for_nonstationary,
pearl_no_host_branches_in_captured_graph, pearl_symmetric_clamp_audit,
pearl_audit_unboundedness_for_implicit_asymmetry (NEG = -2 × POS at
producer time, single source of truth), feedback_isv_for_adaptive_bounds,
pearl_fused_per_group_statistics_oracle.

Validation: cargo check --workspace clean; 3 GPU oracle tests pass on
local RTX 3050 Ti; scripts/audit_sp18_consumers.sh --check exits 0
(no fingerprint drift in tracked sections).

Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
Phase 4-5 (B-leg target-net forward + q_next replacement) follows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-09 11:05:03 +02:00
parent 1f4cc0f207
commit 5ea5aa9b8e
8 changed files with 1455 additions and 0 deletions

View File

@@ -894,6 +894,38 @@ fn main() {
// Pure observability: pre-fix baseline for the B-DD9 ratio
// gate. Plan: SP18 Phase 0 Task 0.2.
"td_error_mag_ema_kernel.cu",
// SP18 v2 Phase 3 (2026-05-09): D-leg adaptive HOLD_REWARD_POS/
// NEG_CAP producer kernel. Single-block 256-thread kernel that
// sweeps the per-sample `step_ret_per_sample` +
// `trade_close_per_sample` buffers (populated by
// `unified_env_step_core` in `experience_kernels.cu` line ~2855),
// filters by `(is_close && step_ret != 0)` to extract the
// Long/Short trade-close magnitude distribution (DD3=b), computes
// a Welford `mean + Z_99 × sigma` p99 estimator with `max(p99,
// max_observed)` conservative takeover × HOLD_REWARD_CAP_SAFETY_
// FACTOR=1.5, applies Wiener-optimal α blend (per
// `pearl_wiener_optimal_adaptive_alpha`; floor at WELFORD_ALPHA_
// MIN=0.4 per `pearl_wiener_alpha_floor_for_nonstationary` since
// the policy-realised distribution is non-stationary), and
// writes both `ISV[HOLD_REWARD_POS_CAP_INDEX=483]` (POS cap) and
// `ISV[HOLD_REWARD_NEG_CAP_INDEX=484]` (NEG = 2 × POS, DD5=b
// mirrored asymmetry — same Kahneman 2:1 ratio as the position-
// side `reward_cap_update_kernel` for ISV[452]/[453], single
// source of truth at producer time per
// `pearl_audit_unboundedness_for_implicit_asymmetry`). Welford
// accumulators in slots [487..493). Pearl-A first-observation
// bootstrap on POS (sentinel 5.0/-10.0 matches the SP14 P0-A
// pattern for bit-identical cold-start). POS bounds [0.5, 50.0]
// are Category-1 dimensional safety floors per
// `feedback_isv_for_adaptive_bounds`; NEG ∈ [100, 1] derived.
// Block-tree-reduce in shmem (no atomicAdd per
// `feedback_no_atomicadd.md`); `CudaFunction` pre-loaded at
// construction per `pearl_no_host_branches_in_captured_graph`.
// Loaded by `gpu_aux_trunk::HoldRewardCapUpdateOps`. Per-epoch
// boundary launch right AFTER `launch_reward_cap_update` so the
// two cap producers (position-side + Hold-side) share the same
// step_ret/trade_close source buffers. Plan: SP18 Phase 3.
"hold_reward_cap_update_kernel.cu",
// SP18 v2 Phase 2 (2026-05-09): D-leg structural Hold
// opportunity-cost device-fn GPU oracle test wrapper. Single
// thread × single block; wraps `compute_sp18_hold_opportunity_cost`