feat(sp16-p2): adaptive Hold cost scale via ISV[461]

Browse Source

Per train-multi-seed-pfh9n post-mortem: observed_hold_rate climbed 0.25 → 0.52
across training while cost penalty (~0.006) was 100× smaller than per-bar
reward magnitudes (popart=0.97, cf=0.65). Hold action was effectively free,
allowing Q(Hold) to dominate via structural low-variance bias.

Fix: scale Hold cost adaptively. ISV[HOLD_COST_SCALE_INDEX=461] tracks:
  scale = clamp(1.0 + 24.0 × max(0, observed - target) / max(target, 0.01), 1.0, 25.0)

Effective cost at 100% overrun (observed=2× target): 0.006 × 25 = 0.15,
competitive with per-bar reward magnitudes (~0.01-0.1).
At/below target: scale = 1.0 (no extra penalty).

Pearl-A bootstrap + Welford slow EMA (α=0.05). Mirrors T1's
MIN_HOLD_TEMPERATURE pattern (same input signals: ISV[382] observed,
ISV[381] target).

Producer: hold_cost_scale_update_kernel.cu — single-thread cold-path,
per-epoch boundary, AFTER MIN_HOLD_TEMPERATURE in training_loop.rs.

Consumer migration (atomic per feedback_no_partial_refactor):
3 sites in experience_kernels.cu — segment_complete branch (line ~3089),
per-bar positioned-Hold branch (line ~3553), per-bar flat-Hold branch
(line ~3617). Cold-start fallback: scale=1.0 when slot ≤ 0 or
out-of-bounds (bit-identical pre-Phase-2 cost magnitude).

ISV_TOTAL_DIM: 461 → 462.

Behavioral tests (5/5 PASS on RTX 3050):
- sp16_phase2_hold_cost_scale_climbs_with_overrun
- sp16_phase2_hold_cost_scale_at_target_is_one
- sp16_phase2_hold_cost_scale_under_target_is_one
- sp16_phase2_hold_cost_scale_bounds_clamp
- sp16_phase2_hold_cost_scale_pearl_a_bootstrap

Regression: SP14 oracle suite 30/30 PASS, SP15 phase 1 oracle suite
36/36 PASS.

Instrumentation: HEALTH_DIAG[N]: hold_cost_scale_diag obs/tgt/norm/scale.

Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

...

This commit is contained in:

jgrusewski

2026-05-08 15:56:46 +02:00

parent 2c469af2f7

commit 1a3bcf97b8

11 changed files with 1027 additions and 8 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

120

crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs

File diff suppressed because one or more lines are too long