ffa6fda868bda05b5aa786f23ef2c9d9be7f4109
4827 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
ffa6fda868 |
feat(sp17): centered A in barrier/ib_gradient_direction (Commit D)
User design call DD11: migrate the two aux-CQL gradient kernels in
c51_loss_kernel.cu to the SP17 mean-zero advantage contract. The
pre-SP17 comment "skip advantage-mean centering — small epsilon vs
correctness simplicity" at barrier_gradient_direction:1212 is REMOVED;
its empirical-without-verification rationale doesn't hold under the
SP17 contract that compute_expected_q + c51_loss + c51_grad +
mag_concat_qdir + Thompson + quantile_q_select already enforce.
barrier_gradient_direction:
- Per-thread `a_mean_per_atom[NUM_ATOMS_MAX]` reduction over b0_size
direction actions.
- Forward Q-value computation reads `v_row[z] + (adv_a[z] - mean[z])`.
- Backward gradient recompute uses the same centered logits in the
per-action softmax probability accumulation. The Jacobian's symmetric
-1/b0_size per-atom offset cancels in the barrier's relative-push
gradient direction (max up, 2nd down — both targets see same offset).
- Stale "skip centering" comment deleted; replaced with SP17 explanation.
ib_gradient_direction:
- Same per-atom mean reduction at function entry.
- Forward Q computation + backward dq_dlogit recompute both flow through
centered probabilities. Variance var_q is invariant under common
per-atom shifts (math: shift cancels in (Q(a) - mean_q)^2), so var_q
numerics are bit-equivalent — but the gradient flows through the
centered probability `p(z|a)` for consistency with c51_grad backward.
NUM_ATOMS_MAX=128 ceiling guard added to both kernels (mirror of
experience_kernels.cu); early-exit `if (num_atoms > NUM_ATOMS_MAX) return`
matches the pattern already used by these kernels for unrelated
zero-op guards.
GPU oracle test (RTX 3050 Ti, 6/6 PASS):
barrier_gradient_direction_uses_centered_advantage — A=0, V=0 ⇒
centered logits all zero ⇒ uniform softmax ⇒ E[Q]=0 across actions ⇒
q_gap=0 ⇒ barrier fires at min_req=0.05 ⇒ asserts total |grad| > 1e-6.
Regression detector: any centering breakage produces non-finite or
zero gradients ⇒ test fails loudly. ib_gradient_direction shares the
identical per-atom mean reduction pattern so the same test covers
both kernels structurally.
Verification:
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 6/6 PASS
⚠ INTERIM STATE: c51_loss_batched + c51_grad_kernel already-centered
sites still need Commit E annotation pass to mark the existing Jacobian
+ per-d=1 magnitude-std as SP17-compliant.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
3107edb8f7 |
feat(sp17): Thompson V-wire-in (Commit C)
User design call DD10 option (b): Thompson direction selector now reads
softmax(V[z] + (A[a, z] - mean_a A[*, z])) instead of pre-SP17
softmax(A[a, z]). Without V wired in, action selection responded to a
DIFFERENT distribution than compute_expected_q's E[Q] (the Bellman-target
ranking) — the very pathology SP17 is fixing at the action layer.
Architectural change closes the contract gap atomically across:
Kernel signature (`experience_action_select`):
- Added `const float* __restrict__ v_logits_dir` after `b_logits_dir`.
NULL is invalid (no fallback) — kernel hard-requires V.
- New per-thread `a_mean_per_atom_dir[THOMPSON_MAX_ATOMS]` reduction
computes mean A across the 4 direction actions per atom z (sample-local
register, no atomicAdd).
- Pass 1 (E[Q] for temperature blend + conviction): builds per-action
`combined_logits_d[z] = V[z] + (A[a,z] - mean_a A[*,z])` and feeds
`softmax_c51_inline` instead of raw `b_logits_dir + d * n_atoms`.
- Pass 2 (Thompson sample): same combined-logits rebuild per direction.
- `softmax_c51_inline` device helper UNCHANGED — keeping centering at
caller maintains a narrower contract; thompson_test_kernel and other
potential callers stay unaffected.
- THOMPSON_MAX_ATOMS=128 ceiling preserved; __trap() on overflow.
QValueProvider trait extension:
- `compute_q_and_b_logits_to` now takes `v_logits_out_ptr: u64` and
DtoD-copies `on_v_logits_buf` per sub-iteration (atomic per
feedback_no_partial_refactor — every consumer migrates in lockstep).
Trainer + evaluator wire-up:
- `gpu_dqn_trainer.rs::on_v_logits_buf_ptr() -> u64` (new pub fn, mirror
of existing `on_b_logits_buf_ptr`).
- `gpu_backtest_evaluator.rs::chunked_v_logits_buf` field allocated
[chunk_n * NA + 32*3] (cuBLAS tail-safety pad).
- Both call sites (collector + evaluator) pass v_logits arg in launch.
GPU oracle test (RTX 3050 Ti, 5/5 PASS):
thompson_direction_select_reads_v_logits — runs production cubin
twice on identical A logits with V=[0,0,0] vs V=[10,0,0]; asserts
q_gap_v_dominant < 50% of q_gap_v_zero. If V is being IGNORED
(regression), both runs produce IDENTICAL q_gaps and the test fails
with a clear message. Uses MappedF32Buffer / MappedI32Buffer per
feedback_no_htod_htoh_only_mapped_pinned.
Note on the plan's "raw argmax = Hold but centered argmax = Long" test:
Mathematical analysis shows softmax-with-constant-shift preserves
action ordering (mean subtraction adds the same per-atom constant to
every action's logits), so the plan's specific assertion isn't
algebraically constructable with simple A/V. The replacement test
(V-dependence of q_gap) is more sensitive — it fails on the actual
regression case (V ignored ⇒ identical q_gaps) the plan was trying
to detect.
Verification:
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 5/5 PASS
⚠ INTERIM STATE: aux-CQL barrier_gradient_direction +
ib_gradient_direction still read raw advantage. Commit D closes them;
Commit E annotates the pre-SP17 c51_loss/c51_grad already-centered sites.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4802879494 |
feat(sp17): centered A in mag_concat_qdir (Commit B)
Migrates the magnitude-branch's direction-Q conditioning input to the
SP17 mean-zero contract. mag_concat_qdir builds a per-(sample, action)
softmax over V[z] + A_dir[a, z] across THREE inner passes (max,
sum_exp, E[Q]); all three now read `dir_logits_b[..] - a_mean_per_atom[z]`.
Without this migration the magnitude trunk-input would see raw E[Q_dir]
while c51_loss/c51_grad backward consumes centered A — mixed contract
ruled out by `feedback_no_partial_refactor`.
The Plan-4 q_rms adaptive scale (ISV[96]) is preserved unchanged;
centering changes the per-action E[Q_dir] values that feed it but the
scale formula itself is structurally orthogonal. Backward into the
direction logits flows through c51_grad (already mean-zero) so no bw
change is needed in this kernel.
GPU oracle test: B=1, SH2=2, NA=3, B0=4 with the same synthetic where
mean_a = [0.75, 0, 0.75] is non-zero. Asserts:
(a) h_s2[0..SH2] copied verbatim into concat prefix
(b) tail slots = centered_E[Q_dir] × (1.0 / q_rms) to ε=1e-5
(c) tail[0] < 0 / tail[3] > 0 sign asymmetry (regression detector)
Verification (RTX 3050 Ti):
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 4/4 PASS
⚠ INTERIM STATE: Thompson direction-select + aux-CQL barrier/ib still
read raw advantage. Commits C-D close them; Commit E annotates the two
already-centered c51_loss/c51_grad pre-SP17 sites.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9b6e94854f |
feat(sp17): centered A in quantile_q_select (Commit A)
Migrates `quantile_q_select` (uncertainty-driven action selection from
C51 atom CDFs) to the SP17 mean-zero advantage contract. The kernel
builds a per-(sample, branch, action) softmax over `V[z] + A[a, z]`
across THREE inner passes (max, sum_exp, CDF); all three now read
`adv_a[z] - a_mean_per_atom[z]`.
Plan flagged this as a hidden 6th consumer. Task 1.4/1.5 as authored
covered only c51_loss + c51_grad + mag_concat_qdir + Thompson; the
post-Task-1.2 audit found `quantile_q_select` reads raw advantage at
lines 5704/5709/5720 across all 4 branches and was missed entirely.
Sample-local register reduction (`float a_mean_per_atom[NUM_ATOMS_MAX]`,
NA_MAX=128 mirroring compute_expected_q); no atomicAdd, no shared mem,
no cross-thread sync per `feedback_no_atomicadd`. Device-side __trap()
on num_atoms overflow per `feedback_no_quickfixes`.
GPU oracle test: N=1, NA=3, B0=4 with V=[0,0,0] and A_raw chosen so the
per-atom mean is [0.75, 0, 0.75]. Test runs the production cubin with
iqn_readiness=0 and util_ema=1.0 and asserts each per-action q90 matches
the CPU oracle to ε=1e-5. Two structural-asymmetry assertions fail
loudly if centering regresses (action 0 q90 ≤ 0, action 3 q90 ≥ 0).
Mapped-pinned per `feedback_no_htod_htoh_only_mapped_pinned`.
Verification (RTX 3050 Ti):
cargo check --workspace → clean
cargo test -p ml --test sp17_dueling_oracle_tests --features cuda
-- --ignored → 3/3 PASS
⚠ INTERIM STATE: mag_concat_qdir + Thompson + aux-CQL barrier/ib still
read raw advantage. Commits B-D close them; Commit E annotates the two
already-centered c51_loss/c51_grad pre-SP17 sites. No L40S dispatch
escapes feat/sp17-dueling until every consumer is migrated per
`feedback_no_partial_refactor`.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
eabcf8d529 |
feat(sp17): mean-zero identifiability in compute_expected_q
Inserts per-atom mean-A reduction inside the per-branch outer loop: Q[a, z] = V[z] + (A[a, z] - mean_a A[*, z]) The post-centering atom-level identifiability holds: Σ_a A_centered[a, z] = 0 for every (sample, branch, atom). Per Wang et al. 2016 Dueling Networks: mean-zero is smoother and more stable than max-zero. Atom-level subtraction (not expectation-level) preserves distributional shape per action. Sample-local register reduction; no atomicAdd, no shared memory, no cross-thread sync. NUM_ATOMS_MAX=128 mirrors existing THOMPSON_MAX_ATOMS; device __trap() on overflow. ⚠ INTERIM STATE: c51_loss_kernel + c51_grad_kernel + mag_concat_qdir still read RAW (un-centered) advantage logits. Phase 1 Tasks 1.3-1.5 migrate them in lockstep within this branch BEFORE any L40S dispatch. A partially-migrated state is forbidden per feedback_no_partial_refactor. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7b13324bcb |
test(sp17): CPU oracle for mean-zero centered Q computation
Locks the mean-zero identifiability math contract independent of GPU implementation. Subsequent Phase 1 GPU oracle tests compare kernel output against this contract bit-for-bit. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
509035dea6 |
docs(sp17): copy plan file to feat/sp17-dueling branch
Plan was authored on sister worktree (sp15-phase1-honest-numbers) by the SP17 planner agent; copying it to feat/sp17-dueling so future task references in commit messages and audit-doc entries resolve from this branch. No content change vs sister-worktree copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
6178f68559 |
fixup(sp17-pp.2): lock-test producer constants for symmetry
Per code-quality review of
|
||
|
jgrusewski
|
cc4746b48d |
plan(sp17): HEALTH_DIAG v_a_means baseline (PRE-CENTERING)
Adds per-epoch readout of the V/A breakdown so we can observe the pre-SP17 baseline distribution before the identifiability projection lands. Also instruments the kill criterion for Phase 0: HEALTH_DIAG[N]: v_a_means [v=X a_dir=Y a_mag=... a_ord=... a_urg=...] If train-multi-seed-b5gmp's Q(Flat) over-attribution is V-driven, V should be elevated (~0.4+) while a_dir is small. If V is balanced and A is doing the work, dueling cannot help and we abort SP17. Block-tree-reduce kernel \`v_a_means_diag_kernel\` (no atomicAdd per \`feedback_no_atomicadd\`); MappedF32Buffer per \`feedback_no_htod_htoh_only_mapped_pinned\`. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a225926e5f |
plan(sp17): allocate ISV slots [474..483) for dueling-Q diagnostics
9 slots: A_var_ema × 4 branches (per-branch advantage variance EMA) + V_share × 4 branches (V/(V+A) magnitude share) + adaptive advantage_clip_bound. All Pearl-A first-observation bootstrap with fold-reset registry entries + dispatch arms in trainer/training_loop.rs::reset_named_state. Bump ISV_TOTAL_DIM 474 → 483 + layout fingerprint seed. Per `feedback_isv_for_adaptive_bounds`: advantage_clip_bound is producer-tracked from p99(|A_centered|) × 1.5 safety factor, never hardcoded. Bounds [0.1, 100.0] are Category-1 dimensional safety floors. No consumer change in this commit (additive infrastructure). Phase 1 mean-centering producer (atomic across compute_expected_q + c51_loss + c51_grad + mag_concat_qdir) lands in the next 4 commits. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
25ff1d4195 |
fix(sp16): floor Wiener-α at 0.4 for non-stationary control loops
Wiener-optimal α minimizes MSE for stationary stochastic signals.
The min_hold_temp / hold_cost_scale loops track a target driven by
the adapting policy itself — non-stationary by construction.
Empirical evidence: train-multi-seed-b5gmp sha
|
||
|
jgrusewski
|
641aa0dfde |
fix(sp16-t3): Wiener-optimal adaptive α per pearl — hold_cost_scale + min_hold_temperature
Per train-multi-seed-hjzss validation: SP16 T1+T2 chain was structurally
landed but BEHAVIORALLY INERT in 5-epoch smoke (bit-identical to pfh9n
baseline through epoch 3). Root cause: hardcoded `alpha = 0.05f` in both
producer kernels violates feedback_isv_for_adaptive_bounds AND prevents
convergence in short runs (~60 epochs needed from cold start).
Fix per pearl_wiener_optimal_adaptive_alpha:
α = diff_var / (diff_var + sample_var + ε)
Where sample_var = running variance of target signal (Welford accumulator)
and diff_var = running variance of consecutive one-step differences.
Cold-start: target jumps 1.0 → 6.4 → 7.0 → high diff_var → α ≈ 0.6+
→ near-bootstrap responsiveness in epochs 1-3
Steady-state: signal stabilizes → diff_var drops → α decays naturally
→ smoothing emerges without hardcoded constant
Adds 12 new ISV slots (6 per producer):
- HCS_TARGET_MEAN/M2, HCS_DIFF_MEAN/M2, HCS_PREV_TARGET, HCS_SAMPLE_COUNT
- MHT_TARGET_MEAN/M2, MHT_DIFF_MEAN/M2, MHT_PREV_TARGET, MHT_SAMPLE_COUNT
ISV_TOTAL_DIM 462 → 474.
Both kernels migrated atomically. Pearl-A bootstrap preserved (sentinel
on prev_blended triggers REPLACE; cold-start α=1.0 when N<3 samples).
Defensive bounds [WELFORD_ALPHA_MIN=0.01, WELFORD_ALPHA_MAX=0.95] on the
Wiener-derived α to guard against denormal/underflow corner cases.
HEALTH_DIAG[N] emit extended with `alpha=...` and `sample_count=...` for
direct trajectory observation in validation smoke.
Behavioral tests verify:
- α high during signal jumps (>0.3 at epoch 3 post-cold-start)
- α low in steady state (mean tail α<0.4 under converging signal)
- Pearl-A bootstrap fires on first observation (Welford state advances
regardless of REPLACE branch)
- α stays within [WELFORD_ALPHA_MIN, WELFORD_ALPHA_MAX] over 50 epochs
(post-cold-start; cold-start α=1.0 by design)
- No 0.05f hardcoded literal remains in blend math (regression-locked
via host-only string scan)
5 GPU + host tests pass: sp16_phase3_alpha_high_during_signal_jump,
alpha_low_in_steady_state, pearl_a_bootstrap_first_obs,
alpha_naturally_bounded, no_hardcoded_alpha. sp14 + sp15 oracle suites
unchanged (34 GPU tests + 4 host tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
6dce4ac28b |
docs(audit): note ensure-binary cache key bug for cross-arch resubmits
Discovered during SP16 validation: ensure-binary caches by SHA only,
so cross-pool resubmissions (L40S → H100) cache-hit the wrong-arch
binary and fail with CUDA_ERROR_NO_BINARY_FOR_GPU on rmsnorm.
Build.rs rerun-if-env-changed fix (
|
||
|
jgrusewski
|
1a3bcf97b8 |
feat(sp16-p2): adaptive Hold cost scale via ISV[461]
Per train-multi-seed-pfh9n post-mortem: observed_hold_rate climbed 0.25 → 0.52 across training while cost penalty (~0.006) was 100× smaller than per-bar reward magnitudes (popart=0.97, cf=0.65). Hold action was effectively free, allowing Q(Hold) to dominate via structural low-variance bias. Fix: scale Hold cost adaptively. ISV[HOLD_COST_SCALE_INDEX=461] tracks: scale = clamp(1.0 + 24.0 × max(0, observed - target) / max(target, 0.01), 1.0, 25.0) Effective cost at 100% overrun (observed=2× target): 0.006 × 25 = 0.15, competitive with per-bar reward magnitudes (~0.01-0.1). At/below target: scale = 1.0 (no extra penalty). Pearl-A bootstrap + Welford slow EMA (α=0.05). Mirrors T1's MIN_HOLD_TEMPERATURE pattern (same input signals: ISV[382] observed, ISV[381] target). Producer: hold_cost_scale_update_kernel.cu — single-thread cold-path, per-epoch boundary, AFTER MIN_HOLD_TEMPERATURE in training_loop.rs. Consumer migration (atomic per feedback_no_partial_refactor): 3 sites in experience_kernels.cu — segment_complete branch (line ~3089), per-bar positioned-Hold branch (line ~3553), per-bar flat-Hold branch (line ~3617). Cold-start fallback: scale=1.0 when slot ≤ 0 or out-of-bounds (bit-identical pre-Phase-2 cost magnitude). ISV_TOTAL_DIM: 461 → 462. Behavioral tests (5/5 PASS on RTX 3050): - sp16_phase2_hold_cost_scale_climbs_with_overrun - sp16_phase2_hold_cost_scale_at_target_is_one - sp16_phase2_hold_cost_scale_under_target_is_one - sp16_phase2_hold_cost_scale_bounds_clamp - sp16_phase2_hold_cost_scale_pearl_a_bootstrap Regression: SP14 oracle suite 30/30 PASS, SP15 phase 1 oracle suite 36/36 PASS. Instrumentation: HEALTH_DIAG[N]: hold_cost_scale_diag obs/tgt/norm/scale. Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2c469af2f7 |
fixup(sp16-p1): correct stale build.rs comment + remove tautological test assertion
Code-quality review of
|
||
|
jgrusewski
|
0426ce8887 |
fix(sp16-p1): MIN_HOLD_TEMPERATURE signal swap to hold-rate overrun + slot 330 non-bug closure
Per train-multi-seed-pfh9n post-mortem follow-up: slot 460 stuck at 50 in Fold 1 was NOT a launch-lifecycle bug. Producer fires per-epoch but kernel had early-return guard on AUX_DIR_ACC_SHORT_EMA (slot 373) at sentinel 0.5. Slot 373 reset on fold boundary; aux dir-acc EMA either didn't fire or settled within ε of 0.5 → kernel kept early-returning. Fix: drop slot 373 dependency entirely. Drive temperature from observed hold-rate vs target overrun: overrun = max(0, observed_hold_rate - target_hold_rate) overrun_norm = clamp(overrun / max(target, 0.01), 0, 1) new_temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × overrun_norm blended_temp = Welford EMA α=0.05 with Pearl-A bootstrap When over-holding: temp HIGH → exit ramp permissive (matches design intent). When at/under target: temp LOW → exit penalty strict. Survives fold reset: hold-rate measurement starts fresh with real data immediately, no chained-input-sentinel masking. Slot 330 (KELLY_WARMUP_FLOOR) investigated and confirmed NON-BUG: producer behaves correctly per pearl_kelly_cap_signal_driven_floors cross-fold- persistence. floor=0 post-warmup is correct steady state. Behavioral tests: - sp16_phase1_min_hold_temp_climbs_with_hold_overrun - sp16_phase1_min_hold_temp_strict_when_at_target - sp16_phase1_min_hold_temp_strict_when_under_target - sp16_phase1_min_hold_temp_no_longer_reads_slot_373 Instrumentation: HEALTH_DIAG[N]: min_hold_temp_diag obs/target/overrun/temp. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fb24614f07 |
feat(sp16-p0): Q-by-action HEALTH_DIAG diagnostic for Hold-bias observation
Per train-multi-seed-pfh9n post-mortem: structural-Q-bias hypothesis (Adam's m/sqrt(v) prefers low-variance Hold over noisy direction Q-targets) needs direct per-action Q observations to verify or refute. WR plateaued at ~0.46 across both folds while Hold% climbed 15% → 52% and Q-mean climbed 0.19 → 0.41 — but per-action Q was unobservable. Adds HEALTH_DIAG emit each epoch: HEALTH_DIAG[N]: q_by_action [hold=X long=Y short=Z flat=W] Reads via host-side averaging of `q_out_buf [B, total_actions]` direction-branch columns [0..b0=4]. No new kernel needed — q_out_buf is already populated row-major by compute_expected_q. Modeled directly on the existing Task 0.3 magnitude-bucket diagnostic (same dtoh-and- average cold path). Action-index ordering canonical, see state_layout.cuh:123-126: DIR_SHORT=0, DIR_HOLD=1, DIR_LONG=2, DIR_FLAT=3. Emit slot order is [hold, long, short, flat] (Hold first because the hypothesis is about Hold's ascent dominating direction Q-magnitudes). New surface: - gpu_dqn_trainer.rs: q_dir_means_cached field + update_q_dir_means_cached method + q_direction_action_means accessor + free static helper compute_q_dir_means_from_host_buf (factored for testability). - fused_training.rs: FusedTrainingCtx wrappers parallel to q_mag pair. - training_loop.rs: emit block adjacent to q_var_per_branch. Behavioral test: sp16_phase0_q_by_action_diagnostic_reads_four_action_means seeds known per-column constants, asserts canonical-dir-idx → emit-slot mapping at 1e-3 tolerance, and includes sentinel-leak guard (non-direction columns at 99.0 fail any wrong index→slot mapping with margin >12). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
98a81981ac |
fix(build-infra): cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP across 6 crates
Per pearl_build_rs_rerun_if_env_changed, every std::env::var() must be paired with cargo:rerun-if-env-changed. Task #321 fixed crates/ml/build.rs (commit |
||
|
jgrusewski
|
0371d6a768 |
docs(no-cpu-strict-sites-2-9): mark sites #2 and #9 CLOSED in audit grid
Per Class A audit grid no-CPU-compute sites 2+9 verification (audit doc lines 3825/3832): the code-vs-grid disagreement was the GRID being stale, not the code. Both sites were already resolved in 2026-05-01 close-out follow-up commits but the grid table verdict columns were never updated. Site 2 (compute_adaptive_tau): ALREADY-CLOSED Helper deleted in `a385c1d2b` (chore(sp4): delete compute_adaptive_tau orphan helper per feedback_wire_everything_up). Verified zero hits across `crates/`, `services/`, `bin/` for `compute_adaptive_tau` and `q_div_ema`. Grid row updated: DEFER → CLOSED, with closure SHA recorded. Site 9 (calibrate_homeostatic_targets): ALREADY-CLOSED Migrated to GPU in `6d0ac7beb` (fix(sp4): GPU-port calibrate_homeostatic_targets per feedback_no_cpu_compute_strict). Verified `calibrate_homeostatic_kernel.cu` exists at `crates/ml/src/cuda_pipeline/calibrate_homeostatic_kernel.cu`, Rust launcher at `gpu_dqn_trainer.rs:7958-7994` calls `launch_calibrate_homeostatic` (single-block, 6 threads, mapped-pinned EMA writeback with `__threadfence_system()`); host-side EMA loop body is gone. Grid row updated: NEW DISCOVERY → CLOSED, with closure SHA recorded. Sweep summary section updated: migrations completed 3→4, exceptions 5→4, deferrals 2→0. Final-grep claim of "1 match remaining" amended to "0 matches remaining post-close-out". The free-text "Sweep close-out" subsection at lines 3863-3871 was already correct and complete; only the GRID columns and summary counts were stale. Audit-doc errors found: 2 sites flagged as open (DEFER + NEW DISCOVERY) when both were already closed in commits dated 2026-05-01. The narrative close-out write-up beneath the grid was correct — only the grid table verdict columns and summary counts had not been kept in sync. No code changes; doc-only fix. Per feedback_trust_code_not_docs: code wins. Grid now matches reality. Cumulative WR-plateau fix series (commit M): - Class C, Class A batches 4a/4b, Class B #1, Class B #2 already landed - no-CPU-strict sites 2+9 audit-doc cleanup (this commit; NO_OP for code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7088999f9a |
fix(class-b-2): PER priority double-exponent corruption
Per Class B audit, per_update_pa was applying alpha twice: - priorities[idx] = |td|^α + ε ← α applied once - priorities_pa[idx] = priorities^α ← α applied AGAIN, gives |td|^(α²) Sum-tree sampler reads priorities_pa (verified at gpu_replay_buffer.rs:778 for per_prefix_scan + line 799 for per_sample), so effective sampling exponent was α² = 0.36 instead of α = 0.6 (default). High-TD events were sampled only ~2.5× more than low-TD events instead of ~10× — PER's prioritization power was attenuated by ~4×, washing out the signal from sparse trade-close events (3% of buffer). Fix (Option B — preserves variable semantics): the contract documented at gpu_replay_buffer.rs:1285 is `priorities_pa[i] = priorities[i]^alpha`. Store raw |td|+ε in `priorities` (one source of truth) and compute α once for `priorities_pa`. Same pattern applied to pow_alpha_diverse_f32 in replay_buffer_kernels.cu (the health<0.8 fallback path) which had the identical double-α bug. per_insert_pa NOT changed — it was already correct (priorities=effective, priorities_pa=effective^α, no double application). priority_update_f32 NOT changed — single-buffer kernel, verified UNUSED (no Rust caller); harmless idle code, deletion deferred. Behavioral test: gpu_replay_buffer::tests:: test_per_priority_single_exponent_no_double_alpha (GPU-gated #[ignore]) inserts 16 slots, fires update_priorities_gpu with TD errors spanning [0.001, 100.0], asserts priorities_pa ratio matches (100/0.001)^0.6 ≈ 1995× within 5%. Pre-fix would yield (100/0.001)^(0.6²) ≈ 100× — the 20× miss makes regression instant-detect. Cumulative WR-plateau fix series (commit 11): - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
658fec4939 |
fix(class-b-1): clear PER replay buffer at fold boundary (months-long WR plateau)
Per Class B audit, the PER ring buffer was never cleared from the
single source of truth at `DQN::reset_for_fold`. fold N+1 was
training on a 50/50 mix of stale fold N-1 experiences + fresh
fold N — averaging behavior locked learning to a "policy-agnostic"
middle ground producing 46-48% WR across SP3-SP15.
Fix: extend `DQN::reset_for_fold` to call `clear_replay_buffer()`
internally. Single source of truth — every fold reset clears the
buffer atomically. Removed the now-redundant explicit
`self.clear_replay_buffer().await?` call at the top of
`DQNTrainer::reset_for_fold` per `feedback_no_legacy_aliases`.
Tradeoff: loses cross-fold experience transfer (fold N+1 starts
from scratch on fold N data only). User-approved Option (a) per
Class B audit 2026-05-08.
Signature changes (atomic per `feedback_no_partial_refactor`):
- `DQN::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNAgentType::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNTrainer::reset_for_fold` (async): unchanged signature; the
agent.reset_for_fold call now propagates the Result.
`GpuReplayBuffer::clear` extended: it already zeroed
`priorities_pa` + `prefix_sum` but left `priorities` unzeroed.
Sampler reads `priorities_pa` (not `priorities`) so this was not
a sampling-contamination vector, but is leftover state that
`priorities_slice()` consumers would inherit. Fixed for
completeness alongside the fold-boundary-clear lift.
Behavioral test added: `test_clear_purges_priorities_and_blocks_sampling`
inserts 64 transitions, samples once to populate prefix_sum +
sample buffers, steps the β counter, then clears and asserts:
- len() == 0, is_empty() == true, can_sample(1) == false
- write_cursor() == 0, current_step == 0
- priorities[..n] all zero (DtoH readback)
- priorities_pa[..n] all zero (DtoH readback)
No mocks; exercises actual GPU memset_zeros + DtoH path.
GPU-gated `#[ignore]` — runs at deploy time.
Cumulative WR-plateau fix series (commit 10):
- Class C bug 1 + P0-B (
|
||
|
jgrusewski
|
9fb980da2b |
fixup(class-a-audit-batch-4b): invert MIN_HOLD_TEMPERATURE dir_acc → temp mapping
Post-commit kernel-formula audit of `compute_min_hold_penalty` at
trade_physics.cuh:567-577 showed the original `temp = TEMP_MIN +
(TEMP_MAX - TEMP_MIN) × skill` mapping was BACKWARDS for the
WR-plateau scenario this 8-commit chain targets.
Formula recap:
soft_factor = deficit / (deficit + T)
HIGH T → soft_factor → 0 → forgiving (no exit penalty)
LOW T → soft_factor → 1 → sharp (max exit penalty)
The deleted schedule's `START=50 → END=5` therefore meant
"permissive early, strict late" (classic explore-then-exploit), not
"permissive when committed, strict when uncertain" as the
substituted mapping assumed.
WR-plateau scenario: dir_acc ≈ 0.46-0.48 (model committed but
wrong) — the model needs a permissive (HIGH T) exit ramp to bail
out of bad committed directions, not maximum exit penalty (LOW T)
locking it into them.
Inverted the mapping using `confusion = 1 - skill`:
- dir_acc ≈ 0.5 (random / plateau) → confusion=1 → temp=50
(permissive, plateau exit ramp)
- dir_acc → 1.0 (saturated skill) → confusion=0 → temp=5
(strict, force informed commitment)
This preserves the deleted schedule's epoch-0 anchor (T=50
cold-start) while reacting to actual realized skill instead of an
epoch-time anchor.
Files touched (atomic per `feedback_no_partial_refactor`):
- min_hold_temperature_update_kernel.cu — formula + header doctring.
- sp14_isv_slots.rs — slot-doc comment expanded with new mapping
+ fixup-rationale paragraph.
- gpu_aux_trunk.rs — `MinHoldTemperatureUpdateOps` docstring.
- gpu_dqn_trainer.rs — cubin docstring + ISV_TOTAL_DIM doc-comment.
- tests/sp14_oracle_tests.rs — Tests 3 + 4 flipped (high dir_acc
now expects temp=TEMP_MIN; low dir_acc now expects TEMP_MAX);
Tests 1, 2, 5 numerically unchanged (sentinel guard fires before
the mapping; midpoint dir_acc=0.75 has skill=confusion=0.5 so
pre/post-fixup numerics match).
- docs/dqn-wire-up-audit.md — Item 4 entry rewritten with the
fixup rationale + mapping table updated.
Verification:
cargo check -p ml --tests --all-targets PASS
cargo test sp14_audit_4b (lib) 4/4 PASS (slot layout
locks unchanged)
GPU oracle re-run deferred to next L40S smoke (kernel was rebuilt
in-place so cubin contents change; the 5 oracles encode the new
mapping and will exercise the new cubin).
Per `pearl_first_observation_bootstrap.md` (sentinel→target
replace; sentinel anchors unchanged) +
`pearl_symmetric_clamp_audit.md` (bilateral clamp on target_temp
preserved) + `feedback_no_partial_refactor.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0b9ea77dc4 |
fix(class-a-audit-batch-4b): plan_threshold floor adaptive + MIN_HOLD_TEMPERATURE ISV-driven
Per Class A audit-fix Batch 4-B (final 2 of 4 deferred items from
P1-wiring/P1-producer). Completes the 8-commit WR-plateau intervention
chain. Validation deferred to next L40S smoke.
Item 3: plan_threshold adaptive floor (Design Y - inline producer)
- NEW slot PLAN_THRESHOLD_FLOOR_ADAPTIVE_INDEX=459
- Writes slow-EMA shadow of `0.5 * readiness_ema` from inside the
existing update kernel (no new file, no new launch).
- Pearl-A first-observation bootstrap (sentinel 0.1 matches pre-fix
hardcoded value for bit-identical cold-start) + Welford alpha=0.005
slow EMA.
- Bilateral clamp [0.05, 0.50] (probability units) per
pearl_symmetric_clamp_audit.
- Consumer reads isv[459] as the floor in the same launch's final
fmaxf; cold-start sentinel REPLACES with threshold_target so the
pre-fix `fmaxf(0.1, 0.5*ema)` semantic is preserved bit-identical
for any readiness EMA above 0.20.
Item 4: MIN_HOLD_TEMPERATURE -> ISV-driven (driving signal: dir_acc skill)
- NEW slot MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460
- NEW kernel min_hold_temperature_update_kernel.cu (single-thread
cold-path, per-epoch boundary launch).
- Driving signal: dir_acc skill = clamp((short_ema - 0.5)/0.5, 0, 1)
from ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]. When committing skillfully
(high dir_acc) -> temp HIGH (permissive). When at random baseline
(~0.5) -> temp LOW (sharp commitment pressure). Substituted for
the audit-spec's `dir_entropy_deficit` because no dir_entropy ISV
slot exists - dir_acc skill is the closest semantically-equivalent
signal that preserves the spec intent.
- Pearl-A bootstrap (sentinel 50.0 matches the deleted
MIN_HOLD_TEMPERATURE_START=50 anchor) + alpha=0.05 mid-cadence EMA.
- Bounds [5, 50] (matches the deleted schedule range).
- Decouples temperature from epoch number - the old schedule pinned
LOW temp (sharp) at end of training, exactly when a WR-plateaued
model needed forgiveness to escape.
- DELETED: state_layout.cuh::MIN_HOLD_TEMPERATURE_{START, END, DECAY}
#defines + training_loop.rs::min_hold_temperature_for_epoch helper
function (kept docstring tombstone explaining the deletion). Both
call sites migrated to the new ISV reader. Per
feedback_no_legacy_aliases + feedback_no_partial_refactor.
ISV_TOTAL_DIM: 459 -> 461.
Cumulative WR-plateau fix series (final commit, #8):
-
|
||
|
jgrusewski
|
7e9a8f6ef1 |
fix(class-a-audit-batch-4a): DD saturation floor adaptive + legacy DD path Case A
Per Class A audit-fix Batch 4-A (deferred from P1-wiring/P1-producer due to
audit-doc errors). Fixes 2 of 4 deferred items; Batch 4-B handles plan_threshold
floor + MIN_HOLD_TEMPERATURE in a separate commit.
Item 1: DD saturation floor (the upper end of the DD ramp at trade_physics.cuh:154
in apply_margin_cap, NOT line 548 as the audit doc claimed — that line is a
magnitude action constant; the actual saturation floor lives in apply_margin_cap)
- NEW slot DD_SATURATION_FLOOR_ADAPTIVE_INDEX=458
- Producer dd_saturation_floor_update_kernel.cu — p75(per-env DD_MAX) × 1.5
via Welford `mean + Z_75 × sigma` estimator with `max(p75, mean)` robustness
guard, mirrors P0-A REWARD_POS_CAP producer pattern (Pearl-A bootstrap +
Welford α=0.01)
- Cold-start fallback: 0.25f (DD_SATURATION_FLOOR_DEFAULT in state_layout.cuh)
- Bounds: [0.10, 0.50] (Category-1 dimensional safety)
- Distinct from SP15_DD_THRESHOLD_INDEX=421 (the SP15 quadratic DD-penalty
*trigger* threshold, a *lower* bound; this slot is the *upper* end of the
linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac/floor))
- Threaded `isv_signals_ptr` into `apply_margin_cap` with NULL-tolerant
cold-start fallback to DD_SATURATION_FLOOR_DEFAULT
- 4 oracle tests (Pearl-A bootstrap, no-DD guard, bounds clamp, Welford EMA)
Item 2: Legacy compute_drawdown_penalty path → Case A (DELETED)
- Decision rationale: SP15's quadratic asymmetric DD penalty
(compute_sp15_final_reward_kernel.cu:154 via sp15_dd_penalty helper) runs
unconditionally as a post-modifier on the SP11-composed reward with
ISV-driven λ_dd (slot 420) and DD threshold (slot 421). Layering the legacy
linear-ramp penalty inside the SP11 composer on top of the SP15 quadratic
creates double-counting of DD shaping — exactly the code-smell the Class A
audit was designed to eliminate. Per `feedback_no_legacy_aliases.md` and
`feedback_no_partial_refactor.md`.
- Atomic deletion across:
- `compute_drawdown_penalty` device function (trade_physics.cuh)
- Single call site at experience_kernels.cu:3822
- `dd_threshold` and `w_dd` kernel arguments
- `w_dd` Rust config field (gpu_experience_collector.rs +
trainers/dqn/config.rs DQNHyperparameters)
- `w_dd` profile section + dispatch (training_profile.rs RewardSection,
OptimizableParameterRanges, FixedRewardParameters, ParamLookup
dispatch, profile→hyperparam mapping, test assertion)
- `w_dd *= rki` risk-intensity multiplier (config.rs)
- `w_dd` TOML keys (dqn-hyperopt.toml × 2, dqn-localdev.toml,
dqn-production.toml, dqn-smoketest.toml)
- Stale doc comments on hyperopt/adapters/dqn.rs + config.rs
risk_intensity field
- `config.dd_threshold` SURVIVES (still consumed by `launch_sp15_dd_state`
as the dd_budget for DD_PCT scaling). Documented in field comment.
ISV_TOTAL_DIM: 458 → 459 (Item 1 adds 1 slot; Item 2 is pure deletion)
Cumulative WR-plateau fix series (this is commit 7):
- Class C bug 1 + P0-B (
|
||
|
jgrusewski
|
87d597d5d7 |
fix(class-a-p1-producer): adaptive Bayesian Kelly priors per-fold-end
Replace 4 hardcoded Bayesian priors in Kelly cap calculations
(prior_wins=2.0, prior_losses=2.0, prior_sum_wins=0.01,
prior_sum_losses=0.01) with ISV-driven slow-EMA values fed by a new
producer kernel that aggregates realized PS_KELLY_* fields across envs
from the same portfolio_state buffer kelly_cap_update_kernel reads
from at the same per-epoch boundary.
Slots [454..458):
- KELLY_PRIOR_WINS, KELLY_PRIOR_LOSSES,
KELLY_PRIOR_SUM_WINS, KELLY_PRIOR_SUM_LOSSES
ISV_TOTAL_DIM bumps 454 -> 458.
Producer:
- kelly_bayesian_priors_update_kernel.cu (per-fold-end / per-epoch
boundary, block-tree-reduce in shmem, no atomicAdd). Pearl-A
first-observation bootstrap (sentinels 2.0/2.0/0.01/0.01 match
pre-P1-Producer hardcoded values for bit-identical cold-start) +
alpha=0.005 slow EMA. Bounds counts in [0.5, 100], sums in [0.001,
1.0] per feedback_isv_for_adaptive_bounds. Launched RIGHT BEFORE
launch_kelly_cap_update so that kernel sees the freshly-blended
priors.
Consumers migrated atomically (same commit):
- kelly_cap_update_kernel.cu:39-42 -> ISV[454..458) with cold-start
fallback via kelly_prior_or_default helper (range guard).
- trade_physics.cuh::kelly_position_cap:304-307 -> NULL-tolerant
isv_signals_ptr threaded through apply_kelly_cap (single caller in
unified_env_step_core line 898 already had the bus pointer; mirrors
the existing kelly_f_smooth / kelly_warmup_floor_sp9 patterns).
State reset:
- 4 FoldReset registry entries (sp14_p1_kelly_prior_*) +
4 dispatch arms in training_loop.rs::reset_named_state.
Oracle tests (sp14_oracle_tests.rs, GPU-gated #[ignore]):
- Pearl-A bootstrap (sentinel -> REPLACE with aggregated targets)
- No realized trades -> ISV preserved bit-exactly
- Bounds clamp on extreme aggregates
- Slow EMA blend after bootstrap
CPU tests passing:
- sp14_p1_kelly_prior_slot_layout_locked
- all_sp14_p1_slots_fit_within_isv_total_dim
- every_fold_and_soft_reset_entry_has_dispatch_arm (C.10 lesson)
- layout_fingerprint_bumps_after_sp14_wire
DEFERRED — Item 2 (MIN_HOLD_TEMPERATURE EMA): the audit-spec said
"MIN_HOLD_TEMPERATURE = 0.5f hardcoded somewhere" but the actual code
has MIN_HOLD_TEMPERATURE_{START=50.0f, END=5.0f, DECAY=20.0f} as a
PER-EPOCH ANNEALING SCHEDULE driven by min_hold_temperature_for_epoch
in training_loop.rs:68-73. The kernel takes T as a runtime scalar
specifically to enable Phase 2 ISV-driven lift "without recompiling
cubin" per the SP12 v3 design comment. The audit's claim of "0.5f
hardcoded" does not match reality; the right Phase 2 signal is a
separate spec decision and should not be guessed at per
feedback_no_quickfixes. Reporting back per the prompt's hard rule #8.
Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor +
pearl_controller_anchors_isv_driven.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
657972a4b5 |
fix(class-a-p0a-downstream): DD penalty + MIN_HOLD_PENALTY_MAX scale to POS_CAP_ADAPTIVE
Per Class A audit P0-A downstream batch — both constants were tuned for fixed REWARD_POS_CAP=5.0f. P0-A made POS_CAP adaptive via isv[452]; this commit propagates the ratio to keep the Kahneman 2:1 asymmetry coherent across the reward shaping chain. Items: 1. DD penalty -5.0f → -1.0f * isv[REWARD_POS_CAP_ADAPTIVE_INDEX] (helper signature change in trade_physics.cuh::compute_drawdown_penalty adds dd_penalty_scale parameter; sole call site in experience_kernels.cu:3775 resolves the ISV value with the same defensive guard as sp15_apply_sp12_cap) 2. MIN_HOLD_PENALTY_MAX 3.0f → 0.6f * isv[REWARD_POS_CAP_ADAPTIVE_INDEX] (existing 60% ratio from state_layout.cuh comment line 252-253 preserved; resolved at the call site mirroring the effective_min_hold_target precedent for slot 451) Cold-start fallbacks preserved: - DD penalty: REWARD_POS_CAP=5.0f when ISV at sentinel/out-of-range - MIN_HOLD_PENALTY_MAX: kernel-passed 3.0f from gpu_experience_collector.rs:399 (bit-identical pre-P0-A behavior) Defensive guard at both consumer sites: ISV must be in [REWARD_POS_CAP_MIN_BOUND=1.0, REWARD_POS_CAP_MAX_BOUND=50.0] AND not within 1e-6f of SENTINEL_REWARD_POS_CAP=5.0f. Mirrors the existing sp15_apply_sp12_cap and segment-complete cap fallback patterns. Note: the SP15 quadratic DD penalty path (compute_sp15_final_reward_ kernel.cu::sp15_dd_penalty) is already fully ISV-driven via slots 420 (λ_dd) and 421 (dd_threshold) — only the legacy compute_drawdown_ penalty (linear ramp, slot-free) had the hardcoded -5.0f. The audit recommendation suggested ratio = 1.0 for MIN_HOLD_PENALTY_MAX assuming the value was 5.0f; actual is 3.0f and the existing tuning comment locks the ratio at 60% — pure wiring uses 0.6. Cumulative WR-plateau fix series: - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
c4b6d6ef29 |
fix(class-a-p1-wiring): var_floor q_gap-only adaptive (1 of 4 wireable)
Per Class A audit P1 wiring batch — only 1 of 4 items was wireable as spec'd. The other 3 are deferred per the spec's "DEFER not BLOCK" rule because their target ISV slots either don't exist or are unit-incompatible with the consumer site. Items: 1. trade_physics.cuh:548 (DD floor_dd 0.25f) — DEFERRED. Slot 421 (DD_THRESHOLD_INDEX) holds the DD trigger threshold (~0.05), NOT the penalty saturation floor (~0.25). Different parameters; substituting would break the ramp formula `(dd-thr)/(floor-thr)`. Needs a new slot for the saturation floor. 2. gpu_experience_collector.rs:343-344 (dd_threshold + w_dd config) — DEFERRED. No W_DD slot exists in any sp*_isv_slots.rs. The legacy compute_drawdown_penalty path (experience_kernels.cu:3769) uses config scalars; the SP15 reward axis (compute_sp15_final_reward_kernel) already reads slot 421 directly. Wiring needs a new W_DD slot. 3. plan_threshold_update_kernel.cu:44 (plan threshold floor) — DEFERRED. Unit mismatch: ema and plan_threshold are probabilities ∈[0,1]; Q_DIR_ABS_REF (slot 21) is a Q-value magnitude EMA (5–50). Scaling a probability by a Q-magnitude is dimensionally meaningless. 4. experience_kernels.cu:2471 (var_floor formula) — DONE. Drop the hardcoded 0.25f lower bound. Was `fminf(fmaxf(q_gap*0.5f, 0.25f), 1.0f)`; now `fminf(q_gap*0.5f, 1.0f)`. NULL-q_gaps fallback uses ε=1e-6f for numerical safety (vs prior 0.25f). var_scale itself remains (0,1] from `1/(1+sqrt(var_q))` so the multiplicative chain stays bounded. Cumulative WR-plateau fix series: - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
394de7d434 |
feat(class-a-p0a): REWARD_POS/NEG_CAP → ISV-driven adaptive caps from realized return distribution
Per Class A audit ranking, the highest-suspected-impact fix for the months-long WR-stuck-at-46-48% plateau across 11 superprojects. Hardcoded REWARD_POS_CAP=+5.0f / REWARD_NEG_CAP=-10.0f (state_layout.cuh:266-267) was structurally clipping the upper tail of realized alpha. Controller signals (sharpe EMA, var_q, q_gap) all derive from this CAPPED buffer, so the controller cannot select for trades it cannot see. Selectivity gradient evaporates — small wins clip to +5 alongside large wins also clipping to +5. The state_layout comment lines 261-264 explicitly deferred Phase 2 (ISV-driven) "IF Phase 1 validation reveals adaptive need". Phase 1 has been running 11 SPs without budging WR — adaptive need revealed. Architecture: - 2 new ISV slots [452..454): REWARD_POS_CAP_ADAPTIVE, REWARD_NEG_CAP_ADAPTIVE. - Producer kernel reward_cap_update_kernel.cu: block-tree-reduce Welford `mean + Z_99 × sigma` p99 estimator over winning realized returns + conservative `max(p99, max_win)` takeover, × 1.5 safety factor → POS cap. NEG cap = -2 × POS cap (preserves Kahneman 2:1 asymmetry per pearl_audit_unboundedness_for_implicit_asymmetry — asymmetry stays, but moved from hardcoded scalar to producer-time multiplier; single source of truth, no consumer applies the 2× ratio itself). - Pearl-A first-observation bootstrap from sentinel (5.0 / -10.0, matching pre-P0-A hardcoded values for bit-identical cold-start). Welford EMA α=0.01 thereafter (slow blend — reward distribution is the foundation of training and shouldn't move fast). - Bounds: POS in [1, 50], NEG in [-100, -2] (Category-1 dimensional safety per feedback_isv_for_adaptive_bounds, NOT tuning). - 3 consumer sites migrated atomically per feedback_no_partial_refactor: experience_kernels.cu:3112-3114 (segment_complete cap), compute_sp15_final_reward_kernel.cu:163 (Stage 4 helper invocation), sp15_reward_axis_helpers.cuh:211 (sp15_apply_sp12_cap device fn signature change to take isv ptr). - Cold-start fallback: when ISV slot at sentinel OR outside [REWARD_POS_CAP_MIN_BOUND=1, REWARD_POS_CAP_MAX_BOUND=50], consumers fall back to original macros (still defined in state_layout.cuh). - 2 new device-ptr accessors on the experience collector (step_ret_per_sample_dev_ptr, trade_close_per_sample_dev_ptr) — reuses existing per-sample buffers; no new buffer allocated. - Per-epoch boundary launch (cold path) in training_loop.rs alongside launch_aux_horizon_chain. - Reset registry entries + dispatch arms in reset_named_state per the C.10 lesson (missing dispatch causes runtime crash). - Layout fingerprint seed updated: ISV_TOTAL_DIM 452→454 + AUX_PRED_HORIZON_BARS=450 + AVG_WIN_HOLD_TIME_BARS=451 (previously missing from seed) + REWARD_POS_CAP_ADAPTIVE=452 + REWARD_NEG_CAP_ADAPTIVE=453. Per feedback_isv_for_adaptive_bounds: every adaptive bound in ISV. Verification: - cargo check -p ml --tests --all-targets: clean (19 pre-existing warnings, 0 new). - sp14_isv_slots tests: 8/8 pass (4 layout + 4 fits-within). - sp14_oracle_tests with --features cuda --ignored: 8/8 pass (4 existing q_disagreement/dir_concat + 4 new P0-A tests covering Pearl-A bootstrap, no-winning-trades preservation, bounds clamping to [1,50], Welford α=0.01 EMA blend). - sp15_phase1_oracle_tests with --features cuda --ignored: 36/36 pass (no regression from sp15_apply_sp12_cap signature change). Cumulative WR-plateau fix series: - Class C bug 1 ( |
||
|
jgrusewski
|
316db416bb |
fix(class-a-p0c): MIN_HOLD_TARGET → ISV[AVG_WIN_HOLD_TIME_BARS_INDEX=451] (adaptive)
Per Class A audit: MIN_HOLD_TARGET=30.0f hardcoded was creating a
deterministic gradient pushing trades toward 30-bar holds regardless of
edge expiry. User's trading frequency is between HFT-MFT and varies by
regime; a 30-bar fixed target kills MFT-frequency alpha when the
optimal hold for current data is shorter (or longer).
The producer slot ISV[AVG_WIN_HOLD_TIME_BARS_INDEX=451] already exists
from SP14 Layer C Phase C.4b (commit
|
||
|
jgrusewski
|
8f218cab24 |
fix(q-side): replay buffer intent→realized + Kelly warmup floor wiring (WR-plateau root causes)
Two independent bugs surfaced by Class C (frame-shift) + Class A (hardcoded bounds) audits, both implicated in the months-long WR-stuck-at-46-48% plateau across 11 superprojects. ## Bug 1: Replay buffer Sutton's deadly triad experience_kernels.cu:2275 was writing the original INTENT action to out_actions, but the reward in the replay buffer was computed from the REALIZED position (post-enforcement: Kelly cap, capital floor, trail-stop, broker cap can clamp Long→Flat). Replay buffer stored (s, intent, r_realized, s'). Q(s, Long) was therefore trained against r(s, Flat) whenever env clamped the intent. This explains the train_active_frac=0.40 vs val_active_frac=0.05 gap: train measures intent (40% Long/Short), eval measures realized (5% Long/Short). The 8× gap is env physics draining intent. Fix: after unified_env_step_core resolves actual_dir_core/actual_mag_core, overwrite out_actions[out_off] with the realized action (same encoding as backtest_env_kernel.cu:323-330, which has been doing it correctly all along). Order/urgency preserved from intent. ## Bug 2: Kelly cap update kernel ignored existing ISV warmup floor kelly_cap_update_kernel.cu:53 hardcoded the kelly_f floor at 0.0f. Cold path (per-epoch boundary). Per project_magnitude_eval_collapse_kelly_capped, this collapses kelly_cap to 0 → max position pinned to Quarter for cold start. The val-mag pathology. The warmup floor producer (ISV[KELLY_WARMUP_FLOOR_INDEX=330], SP9 Fix 37) was already populated and consumed by the per-step path at trade_physics.cuh:377-384, but this cold-path kernel never read it. Partial wiring. Fix: replace fmaxf(kelly_f, 0.0f) with fmaxf(kelly_f, isv[330]). One-line change. ## Predicted effect - train_active_frac and val_active_frac should converge (Bug 1 inflated train by counting overridden intents) - Magnitude distribution should escape Quarter-only (Bug 2 was pinning it) - WR ceiling at 46-48% may finally move (Bug 1 broke Bellman consistency; Bug 2 prevented edge realization) Falsification: 5-epoch L40S smoke. If unmoved by ep5, the plateau is deeper still (Class A P0-A REWARD_POS_CAP/NEG_CAP next). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
976f9b9807 |
fix(sp14-c.10): missing reset_named_state dispatch arm for sp14_q_disagreement_variance_ema
Phase C.1's atomic α deletion preserved Q_DISAGREEMENT_VARIANCE_EMA_INDEX=389
(HEALTH_DIAG diagnostic) and its state_reset_registry entry, but the
corresponding dispatch arm in reset_named_state was missing — even though
the inline comment at training_loop.rs:8024 falsely claimed it existed.
train-rqd8r (commit
|
||
|
jgrusewski
|
10e647c141 |
test(sp14-c9): synthetic smoke for aux trunk gradient chain + C.8/C.9 audit close-out
C.8 (ISV-driven aux trunk Adam β1/β2/ε/LR/grad-clip) was already complete in C.5a
commit
|
||
|
jgrusewski
|
0e61de408f |
feat(sp14-c6): h_s2_aux_rms_ema producer — ISV[449] per-collector-step
Single-block 256-thread CUDA kernel computing RMS(h_s2_aux [B, SH2]) and EMA-blending the step observation into ISV[H_S2_AUX_RMS_EMA_INDEX=449] directly. Pearl-A first-observation bootstrap embedded in kernel body (sentinel 0.0 → replace); fixed α=0.05 EMA blend thereafter. ISV slot 449 is outside the SP4/SP5 wiener buffer linear span so the scratch+apply_pearls_ad_kernel path is not available — self-contained Pearl-A logic mirrors the avg_win_hold_time_update_kernel precedent (slot 451). No atomicAdd; shmem block-tree-reduce only. Launched after aux_trunk_forward in the collector per-step hot path. - h_s2_aux_rms_ema_kernel.cu — new CUDA kernel (81 lines) - build.rs — cubin manifest entry - gpu_dqn_trainer.rs — H_S2_AUX_RMS_EMA_CUBIN static - gpu_aux_trunk.rs — HS2AuxRmsEmaOps struct + launch() - gpu_experience_collector.rs — field + constructor + hot-path launch - aux_trunk_oracle_tests.rs — h_s2_aux_rms_ema_pearl_a_bootstrap test - dqn-wire-up-audit.md — Phase C.6 audit entry cargo check -p ml --tests: clean (only pre-existing warnings) Oracle test: 1 new test added (requires GPU to run) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
jgrusewski
|
b26b189925 |
feat(sp14-c.5b): atomic contract migration — h_s2 → h_s2_aux + revert zero-fills
Atomic flip of the aux heads' input from Q's GRN trunk output `save_h_s2` to the SEPARATE aux trunk's output `h_s2_aux`. The aux trunk now trains its own w1/w2/w3/b1/b2/b3 from CE loss (next-bar + regime); Q's encoder is structurally protected by `aux_trunk_backward`'s missing `dx_in` output param (encoder boundary stop-grad enforced at the kernel-set level). Reverts the C.0 stop-grad band-aid commits (`872bd7392`, `411a30473`): the zero-fills in `aux_next_bar_backward` + `aux_regime_backward` Step 3 are replaced with the genuine SAXPY-back-to-input gradient (`dh_s2_aux[b,j] = sum_k sh_dh_pre[k] * w1[k,j]`). The leak that motivated stop-grad is now blocked structurally rather than by data zero-fill — aux gradient flows through the aux trunk's own params, never into Q's encoder. Wired in this commit (atomic, ~330 LOC): - 4 kernel signatures renamed `h_s2 → h_s2_aux` / `dh_s2_out → dh_s2_aux_out` (`aux_next_bar_forward`, `aux_regime_forward`, `aux_next_bar_backward`, `aux_regime_backward`); Rust wrappers in `gpu_aux_heads.rs` follow - Trainer fwd: insert `aux_trunk_forward_ops.launch(...)` in `aux_heads_forward` Step 0, populating `h_s2_aux` from `save_h_s1` (encoder layer-1 output, dim=shared_h1=256). Both head fwds redirect input pointer from `save_h_s2` to `h_s2_aux` - Trainer bwd: SAXPY both `aux_dh_s2_*_buf` into `dh_s2_aux_accum` (pre-zeroed each step via graph-safe `cuMemsetD32Async`); then `aux_trunk_backward_ops.launch(...)` propagates through w3/w2/w1 + b3/b2/b1; then `launch_aux_trunk_adam_update` applies global L2-norm clip + per-tensor Adam updates over 6 grad tensors - Collector fwd: insert `exp_aux_trunk_forward_ops.launch(...)` after `forward_online_f32`, reading `exp_h_s1_f32` and writing `exp_h_s2_aux`; redirect `exp_aux_heads_fwd.forward_next_bar` input from `exp_h_s2_f32` to `exp_h_s2_aux` - Pre-capture host-write of ISV-driven LR + grad-clip + step counter into mapped-pinned buffers in `launch_cublas_backward_to` (BEFORE `aux_heads_backward`); same `&mut self` pattern as `step_ofi_embed_adam` Verification: - `cargo check -p ml --tests` clean (1m02s, only pre-existing warnings) - `aux_trunk_oracle_tests` + `sp14_oracle_tests` 12/12 pass: - aux_trunk gradient check: max_rel_err=1.33e-2 (tol=2e-2) — matches C.4 baseline - aux_trunk_backward_does_not_write_dx: kernel source clean of dx_in/dx_in_out - aux_sign_label_lookahead_mask: 60/100 masked, 40/100 valid - 9 other oracle tests pass bit-identically Plan: docs/superpowers/plans/2026-05-07-sp14-layer-c-separate-aux-trunk.md §C.5b Audit: docs/dqn-wire-up-audit.md "SP14 Layer C Phase C.5b" section |
||
|
jgrusewski
|
1edd71a2c1 |
feat(sp14-c.5a-fixup): missing scratch buffers + collector ptr scaffolding (dead code)
Phase C.5a-fixup — completes C.5a's additive infrastructure so C.5b can be
a true atomic contract flip with zero scaffolding work mixed in:
- Allocate dh_aux1_pre_scratch [B_max, AUX_TRUNK_H1=256] +
dh_aux2_pre_scratch [B_max, AUX_TRUNK_H2=128] (missed in C.5a — both
required by aux_trunk_backward.launch per gpu_aux_trunk.rs:266-267).
- Collector struct gains 6× u64 aux_trunk_{w1,b1,w2,b2,w3,b3}_ptr fields,
exp_aux_trunk_forward_ops: AuxTrunkForwardOps field (constructed in
ctor on collector's stream), and set_trainer_aux_trunk_param_ptrs
setter — mirrors the existing set_trainer_params_ptr zero-copy pattern.
- Trainer gains aux_trunk_param_ptrs() -> (u64×6) accessor returning
raw_ptr() for all 6 aux trunk parameter tensors.
- training_loop wires the new setter at both existing
set_trainer_params_ptr call sites (initial fused-ctx init + fold-boundary
re-init).
NO contract change: wire sites still call save_h_s2. The setter is called
and the 6 aux trunk param ptrs are populated, but no collector-side launch
reads them yet — C.5b atomically inserts aux_trunk_forward.launch(...)
post-forward_online_f32 and switches the aux head input pointer.
Graph-capture audit: aux_heads_backward IS INSIDE the captured `forward`
child graph (call chain: submit_forward_ops_main → launch_cublas_backward
→ launch_cublas_backward_to → aux_heads_backward; capture begins at
fused_training.rs:2964 / capture_child_graph). The existing function body
is fully device-side (zero host writes) — capture-safe by construction.
C.5b's new aux_trunk Adam launch is also fully device-side and will sit
inside the same captured region. The host writes for aux_trunk_t_pinned
must use the existing GPU-side increment_step_counters kernel chain
(submit_counters_ops, line 22799) — NOT host-side aux_trunk_adam_step
+= 1 inside capture. ISV-driven LR/clip writes happen pre-capture
(cold-path); the captured graph reads via aux_trunk_lr_dev_ptr /
aux_trunk_grad_clip_dev_ptr. This avoids the &self → &mut self ripple on
aux_heads_backward (gap 4 in C.5b implementer's blocker report). Full
wiring strategy + alternative (pre-capture host-write) documented in
docs/dqn-wire-up-audit.md C.5a-fixup section.
Verification:
- cargo check -p ml --tests --all-targets: clean (no new warnings).
- cargo test -p ml --test aux_trunk_oracle_tests --test sp14_oracle_tests
--release -- --ignored --nocapture: 12/12 pass (8 aux_trunk + 4 sp14;
bit-identical to C.5a baseline — pure scaffolding, no regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
c90de98594 |
feat(sp14-c.5a): allocate aux trunk fwd/bwd buffers + Adam launcher (dead code)
Phase C.5a — additive infrastructure for the aux trunk wire-up. Allocates saved-fwd buffers (h_s2_aux, h_aux1, h_aux2), gradient buffers (6× aux_trunk_*_grad), accumulator (dh_s2_aux_accum), and dedicated Adam launcher (launch_aux_trunk_adam_update) reading β1/β2/ε/LR/grad-clip from ISV[444..449). No contract change. No call sites for the new launcher yet. C.5b atomically wires these in. Phase C.5 was split (authorized 2026-05-08) after the original implementer flagged ~600-800 LOC scope across 4 files with correctness windows. C.5a is purely additive; C.5b is the genuine ~300 LOC atomic migration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
3b71d21834 |
feat(sp14-c): aux prediction horizon ISV-driven (multi-bar pivot)
Aux's original label was (p_{t+1} > p_t) — pure HFT-scale microstructure
noise that's unlearnable at our HFT-MFT trading frequency. Migrated to
(p_{t+H} > p_t) where H is read from ISV[AUX_PRED_HORIZON_BARS_INDEX=450].
Adaptive producer drives H from observed avg winning hold time:
- Pearl-A first-observation bootstrap: replace sentinel H=60 directly
on first valid observation
- Steady-state Wiener-α EMA blend, slow (α=0.01) for stable horizon
(no target-variance EMA available, fallback per
pearl_wiener_optimal_adaptive_alpha)
- "No winning trades yet" guard keeps sentinel until first valid observation
Lookahead truncation: labels at t where t+H >= total_bars are masked
(sentinel -1, loss-reduce skips). The existing aux_next_bar_loss_reduce
in aux_heads_kernel.cu already supports the -1 mask convention via the
B_valid count — no new valid_mask parameter needed.
Step 5b finding: Case B — existing per-sample buffers
(hold_at_exit_per_sample, trade_profitable_per_sample) populated by
unified_env_step_core, but no aggregate ISV slot. Added new aggregator
slot AVG_WIN_HOLD_TIME_BARS_INDEX=451 + new producer kernel
avg_win_hold_time_update_kernel.cu (block-tree-reduce, no atomicAdd).
ISV_TOTAL_DIM bumped 450 → 452.
ATOMIC migration per feedback_no_partial_refactor: both label kernels
(aux_sign_label_kernel.cu trajectory + aux_sign_label_per_step_kernel.cu
per-rollout-step) migrated together to the new
(targets, bar_indices, isv, isv_h_idx, out_labels, total, total_bars)
signature. The lookahead host-passed scalar argument is removed; H is
read from ISV inside the kernel (broadcast value, single read per
thread, on-device clamp [1, 240]).
Producer chain (per-epoch boundary): new
GpuDqnTrainer::launch_aux_horizon_chain orchestrates
avg_win_hold_time_update → aux_horizon_update sequentially alongside
launch_kelly_cap_update at the existing epoch-boundary slot in
training_loop.rs.
Trunk math (C.2/C.3/C.4) unchanged — separate aux trunk is label-
agnostic. Validation in C.10 will use H=60 cold-start; the adaptive
producer drives H from real winning-trade observations.
Tests (8 oracle, 5 new + 3 preserved):
- aux_trunk_forward_matches_numpy_reference (C.3) ✓
- aux_trunk_backward_gradient_check (C.4) ✓
- aux_trunk_backward_does_not_write_dx (C.4) ✓
- aux_sign_label_h_bar_horizon (NEW) ✓
- aux_sign_label_lookahead_mask (NEW) ✓
- aux_horizon_pearl_a_bootstrap (NEW) ✓
- aux_horizon_converges_to_steady_target (NEW) ✓
- aux_horizon_holds_sentinel_with_no_winning_trades (NEW) ✓
8/8 pass on RTX 3050 Ti.
Phase C.4b of SP14 Layer C separate-aux-trunk refactor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
5d584dc751 |
feat(sp14-c): aux trunk backward kernel + gradient check + stop-grad invariant test
Backward propagates dh_s2_aux through w3/w2/w1 with block-tree-reduce
(no atomicAdd per feedback_no_atomicadd). Critical: kernel set does NOT
write dx_in — encoder gradient remains Q-shaped only. Stop-grad
invariant verified via parameter-list structural enforcement (kernels
literally cannot reference an `dx_in_out` pointer they don't accept) +
kernel source inspection that strips comments and asserts no `dx_in`
write pattern.
Three kernels in aux_trunk_backward_kernel.cu:
- aux_trunk_bwd_dh_pre: per-sample, computes dh_aux2_pre [B, H2] +
dh_aux1_pre [B, H1] using ELU' from POST-activation form
(`(y > 0) ? 1 : (1 + y)` mirrors aux_elu_bwd_from_post in
aux_heads_kernel.cu).
- aux_trunk_bwd_dW_reduce: generic outer-product reduce
`dW[k, j] = sum_b A[b, k] * B[b, j]`. One block per output
element, shmem-tree reduce over batch. Used 3× (dW3, dW2, dW1).
- aux_trunk_bwd_db_reduce: generic batch-reduce `db[j] = sum_b
B[b, j]`. One block per output element. Used 3× (db3, db2, db1).
Memory-efficient: no per-sample partials (avoids B×163,072 floats for
production topology). Per-element reduction means O(P) blocks each
doing O(B) work in shmem.
Rust wrapper AuxTrunkBackwardOps in gpu_aux_trunk.rs orchestrates seven
launches in fixed sequence (capture-friendly, no host branches per
pearl_no_host_branches_in_captured_graph). All three CudaFunction
handles pre-loaded once at construction. Field added to GpuDqnTrainer
alongside aux_trunk_forward_ops; constructor mirrors C.3 pattern.
Tests (all pass on RTX 3050 Ti, sub-ULP forward, 1.33e-2 max rel-err
backward gradient at smallest sampled gradient):
- aux_trunk_forward_matches_numpy_reference (C.3 — preserved).
- aux_trunk_backward_gradient_check (NEW): central-difference
numerical gradient at 16 sampled dW3 indices vs analytic from
backward kernel. Loss = 0.5 * ||h_s2_aux||^2 so dh_s2_aux =
h_s2_aux. EPS=1e-3, B=4, ENC=H1=H2=AUX=32 (33 forwards in ~2s).
REL_TOL = 2e-2 (f32 finite-difference noise floor for
small-gradient tail; production topology is dimension-independent
given runtime args).
- aux_trunk_backward_does_not_write_dx (NEW): reads kernel source,
strips C-style comments (so design-discussion text mentioning
`dx_in` doesn't false-positive), asserts no `dx_in` / `dx_in_out`
symbol survives in code. Complements the structural enforcement
(kernel signatures don't accept `dx_in_out` pointer).
Phase C.4 of SP14 Layer C separate-aux-trunk refactor. Module is
additive — wire-up into collector backward chain + Adam updates lands
in Phase C.5 (atomic).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
cb6bca4629 |
feat(sp14-c): aux trunk forward kernel + Rust wrapper + oracle test
3-layer MLP forward (Linear→ELU→Linear→ELU→Linear). Pre-loaded CudaFunction for graph-capture safety per pearl_no_host_branches_in_captured_graph. Oracle test verifies bit-for-bit match against numpy reference within 1e-4 tol. Saves h_aux1 and h_aux2 to global memory for backward. Phase C.3 of SP14 Layer C separate-aux-trunk refactor (plan: docs/superpowers/plans/2026-05-07-sp14-layer-c-separate-aux-trunk.md). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
4926fb7c65 |
feat(sp14-c): allocate aux trunk params (~132K) + Adam m/v state
3-layer MLP: encoder_out_dim → 256 → 128 → AUX_HIDDEN_DIM. Kaiming-He weights (Box-Muller from LCG-uniform), zero biases, separate Adam m/v buffers (12 state tensors). Allocated in trainer constructor — collector borrows via raw_ptr at wire-up time per existing OFI-embed / q-attn ownership pattern (no parallel param mirror needed). Not yet wired to forward/backward — pure allocation per Phase C.2 design. Topology dimensions resolved against actual codebase: - encoder_out_dim = config.shared_h1 (= SH1 = 256 in production) - AUX_HIDDEN_DIM = config.shared_h2 (= SH2 = 256, matches existing aux head's input dim per aux_heads_kernel.cu:118 `h_s2 [B, SH2]`) Total params: 65,536 + 256 + 32,768 + 128 + 32,768 + 256 = 131,712. Audit doc updated per Invariant 7. Phase C.2 of SP14 Layer C separate-aux-trunk refactor (plan: docs/superpowers/plans/2026-05-07-sp14-layer-c-separate-aux-trunk.md). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
4f372a49a8 |
refactor(sp14-c): atomic α machinery deletion + aux trunk ISV slot allocation
Phase C.1 of SP14 Layer C separate-aux-trunk refactor. Single atomic
commit per feedback_no_partial_refactor and feedback_no_legacy_aliases —
no DEPRECATED stage.
Deleted (per C.0 audit
|
||
|
jgrusewski
|
a7a162d1ff |
docs(sp14-c-preflight): catalogue α-machinery launch sites pending atomic deletion
Phase C.0 of SP14 Layer C separate-aux-trunk refactor. Pure audit doc entry — no source code change. Establishes "before" state of α machinery (Coupling B) launch sites + ISV slots, ahead of atomic deletion in Phase C.7. Files slated for deletion: alpha_grad_compute_kernel.cu, sp14_scale_wire_col_kernel.cu. ISV slots: VAR_AUX/VAR_Q/VAR_ALPHA/ ALPHA_RAW/ALPHA_SMOOTHED/GATE1_OPEN_STATE/GATE2_OPEN_STATE (7 total). Preserved: q_disagreement_* slots [383..390) + producer (diagnostic-only); dir_concat_qaux_kernel (Coupling A: forward feature wire, survives unchanged). Plan: docs/superpowers/plans/2026-05-07-sp14-layer-c-separate-aux-trunk.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
0ec791734d |
fix(sp15-cubin-preload): pre-load 6 SP15 launchers' CudaFunction handles in experience collector
Bug audit Pattern 2 — 6 SP15 launchers in gpu_dqn_trainer.rs were doing load_cubin + load_function PER CALL inside the launcher body, called from gpu_experience_collector.rs's per-rollout-step body (~thousands of times per epoch x 4096 envs x 1000 timesteps). Same architectural bug class as commits |
||
|
jgrusewski
|
fa1f299147 |
fix(sp7-cadence): migrate SP5 Pearl 2 budget + SP7 loss-balance controller from process_epoch_boundary to per-step submit_aux_ops
Bug audit finding #2 (post train-d2b2s diagnostic — same Pattern 1 class as SP14 B.11 commit |
||
|
jgrusewski
|
411a304731 |
fix(aux-head-regime): stop-gradient on aux_regime_backward dh_s2 — completes today's stop-gradient pair
Mirrors commit |
||
|
jgrusewski
|
872bd73927 |
fix(aux-head): stop-gradient on aux's h_s2 input — fixes aux-loss-rises-during-training pathology
Root cause from train-v8ztm 9-epoch HEALTH_DIAG aux next_bar_mse trajectory: - Ep 0: 0.352 (learnable signal — below random baseline ln(2)≈0.693) - Ep 9: 0.717 (above random baseline — aux is now WORSE than random) - aux_dir_acc_long stuck at 0.19 (anti-correlated with truth) Aux head's backward gradient was flowing back to shared trunk activation h_s2 via dh_s2_out write at aux_heads_kernel.cu:599-613. Q-loss gradient on h_s2 dominates (larger magnitude, structurally different objective: cumulative discounted reward vs next-bar direction). h_s2 evolves to support Q's task; aux's CE loss climbs as h_s2 features become anti-aligned with direction prediction. Fix: stop-gradient. Aux reads h_s2 via forward, trains its own w1/b1/w2/b2 from CE loss, but does NOT propagate to h_s2. Q-loss is the sole shaping force on h_s2. Aux must adapt to whatever h_s2 happens to be — if the representation has direction signal, aux's params will extract it; if not, aux can't learn (separate-trunk Option 2 deferred for that case). This was the SEVENTH fix in today's chain (after 6 SP14 EGF cadence/ gate/saturation fixes). The EGF was a scaffold over a broken aux head; fixing aux first is the architectural prerequisite for EGF to route useful signal. Verification: cargo check clean; sp14_oracle_tests 7/7 pass. Validation: aux next_bar_mse should now DECREASE during training in the next L40S smoke (vs the rising-from-0.35-to-0.72 pattern in v8ztm). Deferred follow-up: aux_regime_backward has the same architecture (propagates dh_s2 to trunk). Same fix is a candidate once next_bar result validates the approach. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
c260dca8bd |
fix(sp14-β): remove training-time EGF launches from submit_aux_ops — collector path is canonical
Atomic step 5 of β migration. SP14 EGF producer chain now fires
ONLY from the experience collector (commit
|
||
|
jgrusewski
|
c691bd381a |
feat(sp14-β): wire collector-native SP14 producer chain after aux forward
Step 4 of β migration: 3 SP14/SP13-EGF producers fire per-rollout-step
in collector using collector-owned kernel handles + collector stream.
Reads rollout-time q_values (post-expected-Q, pre-IQR/ensemble/noise)
+ exp_aux_nb_softmax. Writes to shared ISV.
Producer order preserved (matches trainer submit_aux_ops chain):
1. SP13 dir-acc reduce → 2 fixed-α EMAs → aux_pred to ISV[375]
2. SP14 q_disagreement_update (reads aux softmax + q_values)
3. SP14 alpha_grad_compute (pure ISV state machine)
Same kernel gate (commit
|
||
|
jgrusewski
|
296ba92282 |
feat(sp14-β): wire collector-side aux-head forward + label producer per-rollout-step
Step 3 of β migration: collector now runs aux_next_bar_forward on rollout state every step. Label producer (new thin variant aux_sign_label_per_step_kernel) derives sign(price[t+1] - price[t]) per env using bar = episode_starts[ep] + t. Aux predictions feed the EGF kernel chain (step 4), NOT the Q-head's input (rollout Q-head still sees raw h_s2, dir_qaux_concat_ptr remains 0u64). Placement: AFTER captured forward graph, BEFORE expected_q kernel. Same-stream serial ordering reads exp_h_s2_f32 populated by forward_online_f32 inside the captured graph. Cold-start gated on trainer_params_ptr != 0 to skip the test-scaffold path where the trainer hasn't wired its params yet. Files added: aux_sign_label_per_step_kernel.cu (66 lines). Files modified: build.rs (+8 lines, register cubin), gpu_experience_collector.rs (+106 lines: struct field, cubin static, load in new(), per-step launch block). Compile clean; sp14_oracle_tests 2/2 non-GPU pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
88eb7aa241 |
feat(sp14-β): allocate rollout-sized aux buffers + AuxHeadsForwardOps in collector
Step 2 of β migration: 5 new buffers sized to alloc_episodes (vs trainer's batch_size). AuxHeadsForwardOps instance is collector- owned and stream-bound to collector stream via AuxHeadsForwardOps::new(&stream). Param tensors shared with trainer via existing f32_weight_ptrs_from_base path. Buffer sizing: exp_aux_nb_hidden_buf [alloc_episodes × 32], exp_aux_nb_logits/softmax_buf [alloc_episodes × 2], exp_aux_nb_label_buf [alloc_episodes] i32, exp_aux_dir_acc_buf [6] mapped-pinned (post-B1.1a 6-float layout matches trainer). Compile clean; sp14_oracle_tests 2/2 non-GPU pass (7 GPU tests ignored on RTX 3050 Ti host). No aux forward yet — buffers allocated, ready for wire. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |