gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(sp18-v2): ISV-adaptive shrink-and-perturb at fold transition

Browse Source 
Replaces the hardcoded `α=0.8 / σ=0.01` constants in `reset_for_fold`'s
`shrink_and_perturb` call with ISV-driven adaptive bounds per
`feedback_isv_for_adaptive_bounds` and `feedback_adaptive_not_tuned`.

The driving signal is the relative weight drift `||current - best||₂ /
||best||₂` already computed each epoch by the SP18 v2 weight-drift
diagnostic kernel (commit aab13a83f). Extended that kernel atomically
to also write α / σ into ISV[505] / ISV[506] from the same two block-
tree-reductions — single producer, single launch, single source of
truth. No new kernel; reuses `||best||₂` + `relative_drift` already
computed.

Added 2 ISV slots [505..507):

  [505] SHRINK_ALPHA_ADAPTIVE — drift-driven preservation factor.
        α = 1 - clamp(rel_drift, 1-MAX, 1-MIN). Small drift → α near
        MAX (preserve hard); large drift → α near MIN (shrink hard).

  [506] SHRINK_SIGMA_ADAPTIVE — scale-aware noise. σ tracks
        ||best||_RMS / RMS_REFERENCE so noise stays scale-relative as
        weight magnitudes evolve across folds.

Pearl-A first-observation bootstrap: sentinels 0.8 / 0.01 match prior
hardcoded values exactly. Cold-start path (first fold, no
`best_params_snapshot`) leaves slots at sentinels — bit-identical to
pre-fix behavior.

Bounds [SHRINK_ALPHA_MIN=0.5, SHRINK_ALPHA_MAX=0.95] +
[SHRINK_SIGMA_MIN=1e-4, SHRINK_SIGMA_MAX=0.1] are Category-1
dimensional safety floors per `feedback_isv_for_adaptive_bounds`.
Bilateral clamp per `pearl_symmetric_clamp_audit`.

Atomic in same commit per `feedback_no_partial_refactor` and
`feedback_wire_everything_up`:

  * 2 ISV slot constants + sentinels + bounds + lock test in
    `sp14_isv_slots.rs`.
  * `ISV_TOTAL_DIM` 505 → 507 + layout fingerprint seed bump in
    `gpu_dqn_trainer.rs`.
  * `state_layout.cuh` mirror for kernel-side reads.
  * `weight_drift_diag_kernel.cu` extended: 4-element output buffer
    `[l2_diff, rel, α_target, σ_target]` + ISV writes via
    `__threadfence_system()`.
  * `launch_weight_drift_diag` + `read_shrink_perturb_adaptive`
    plumbing; mapped-pinned 4-float buffer.
  * `reset_for_fold` reads ISV[505]/ISV[506] via `read_isv_signal_at`
    + defensive host-side clamp; replaces former hardcoded constants.
  * 2 fold-reset registry entries with dispatch arms (sentinel 0.8 /
    0.01 — Pearl-A bootstrap matches prior hardcoded for bit-
    identical cold-start).
  * Per-epoch HEALTH_DIAG `weight_drift` line extended with
    `alpha_adaptive` / `sigma_adaptive` for observability.
  * 8 behavioral tests (CPU-only oracle pinning the math contract).
  * `docs/dqn-wire-up-audit.md` — Invariant 7 audit entry.

`shrink_and_perturb` signature unchanged — kept as 2-arg
`(alpha, sigma)` so the 4 existing call sites compile without ripple
(3 in `training_loop.rs`, 1 in `fused_training.rs`). Only the
fold-transition site at `fused_training.rs::reset_for_fold` is
migrated to ISV reads in this commit; the 3 health-driven /
backtracking sites in `training_loop.rs` continue to use
`hyperparams.shrink_perturb_alpha/sigma` (different driving signal —
those are unhealthy-streak rescue interventions, not fold-transition
plasticity refresh; surfaced as a follow-up concern in
DONE_WITH_CONCERNS report).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-09 02:09:25 +02:00
parent aab13a83f2
commit 0b737ec30a
9 changed files with 776 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
docs/dqn-wire-up-audit.md
View File

@@ -2,6 +2,45 @@
**Status:** Populated during Plan 1 Task 6 (A.5 orphan audit). Updated on every commit per Invariant 7.
## 2026-05-09 — SP18 v2 ISV-adaptive shrink-and-perturb at fold transition
Replaces the hardcoded `α=0.8 / σ=0.01` constants in `fused_training.rs::reset_for_fold`'s `shrink_and_perturb` call with ISV-driven adaptive bounds per `feedback_isv_for_adaptive_bounds` and `feedback_adaptive_not_tuned`. The driving signal is the relative weight drift `||current − best||₂ / ||best||₂` already computed each epoch by the SP18 v2 weight-drift diagnostic kernel (commit `aab13a83f`).
**Files added/modified:**
- `crates/ml/src/cuda_pipeline/sp14_isv_slots.rs` — 2 new ISV slot constants `SHRINK_ALPHA_ADAPTIVE_INDEX=505` + `SHRINK_SIGMA_ADAPTIVE_INDEX=506`; sentinels `SENTINEL_SHRINK_ALPHA=0.8` + `SENTINEL_SHRINK_SIGMA=0.01` (match prior hardcoded values for bit-identical cold-start); Category-1 dimensional safety bounds `[SHRINK_ALPHA_MIN=0.5, SHRINK_ALPHA_MAX=0.95]` + `[SHRINK_SIGMA_MIN=1e-4, SHRINK_SIGMA_MAX=0.1]`; producer reference scale `SHRINK_SIGMA_RMS_REFERENCE=0.5` (Category-1 dimensional anchor — typical mean RMS of healthy DQN weights). Lock test `sp18_shrink_perturb_slot_layout_locked`.
- `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — `ISV_TOTAL_DIM` 505 → 507 with full SP18 close-out commentary; `layout_fingerprint_seed` extended with `SLOT_505_SHRINK_ALPHA_ADAPTIVE` + `SLOT_506_SHRINK_SIGMA_ADAPTIVE`.
- `crates/ml/src/cuda_pipeline/state_layout.cuh` — mirror `#define`s for the 2 new slots, sentinels, bounds, and `SHRINK_SIGMA_RMS_REFERENCE`.
- `crates/ml/src/cuda_pipeline/weight_drift_diag_kernel.cu` — extended from 2-element output `[l2_diff, rel]` to 4-element `[l2_diff, rel, α_target, σ_target]`. Added 5th `isv` argument; same single-block × 256-thread launch + two block-tree-reductions. Thread-0 cold-path arithmetic computes `α_target = 1 − clamp(rel, 1−ALPHA_MAX, 1−ALPHA_MIN)` and `σ_target = clamp(SENTINEL_SIGMA × (best_rms / RMS_REF), SIGMA_MIN, SIGMA_MAX)` with bilateral clamp per `pearl_symmetric_clamp_audit`, writes to both the mapped-pinned output buffer AND `isv[505] / isv[506]` with `__threadfence_system()`. No new kernel — this is the sole producer of both diagnostic and controller outputs (single-launch atomicity per the spec's "Watch for: Drift diagnostic timing — have the drift kernel ALSO emit the adaptive α/σ as a 4-element output — fewer kernels, same atomicity").
- `crates/ml/src/trainers/dqn/fused_training.rs` — `sp18_weight_drift_diag_buf` resized 2 → 4 floats; `launch_weight_drift_diag` plumbs `isv_signals_dev_ptr` as a new arg + cold-start path writes `[0.0, 0.0, SENTINEL_SHRINK_ALPHA, SENTINEL_SHRINK_SIGMA]` to the host-pinned slice (skip kernel; ISV slots remain at FoldReset sentinels — bit-identical to prior hardcoded behavior). New convenience reader `read_shrink_perturb_adaptive`. `reset_for_fold` migrated: replaces hardcoded `let sp_alpha = 0.8; let sp_sigma = 0.01;` with ISV reads via `read_isv_signal_at(SHRINK_ALPHA_ADAPTIVE_INDEX)` / `read_isv_signal_at(SHRINK_SIGMA_ADAPTIVE_INDEX)` + defensive host-side clamp at the same MIN/MAX bounds the kernel enforces (Category-1 dimensional safety). 4-arg `shrink_and_perturb(alpha, sigma)` signature unchanged — no ripple to the 3 other call sites in `training_loop.rs` (those continue to use `hyperparams.shrink_perturb_alpha/sigma` — different driving signal: unhealthy-streak rescue vs fold-transition plasticity refresh).
- `crates/ml/src/trainers/dqn/state_reset_registry.rs` — 2 new `RegistryEntry` records (`sp18_shrink_alpha_adaptive`, `sp18_shrink_sigma_adaptive`) with `FoldReset` category. Sentinels match prior hardcoded values for cold-start bit-identical equivalence. `sp18_fold_reset_entries_present` lock test bumped to assert 24 SP18 entries (was 22).
- `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — 2 dispatch arms in `reset_named_state` writing the sentinels at fold boundary (so the cold-start fallback in `launch_weight_drift_diag` reads the sentinel rather than stale state). Per-epoch HEALTH_DIAG `weight_drift` line extended: `[norm_l2={} relative={} branch_max={} alpha_adaptive={} sigma_adaptive={}]` — surfaces the per-epoch α/σ targets the next fold-transition `shrink_and_perturb` will use.
- `crates/ml/tests/sp18_shrink_perturb_adaptive_test.rs` (new) — 8 CPU-only oracle tests pinning the math contract: small drift → α near MAX, large drift → α clamped MIN, medium drift → linear interpolation, σ scales with `||best||_RMS`, σ floor/ceiling clamps, typical operating point ≡ legacy constants, cold-start = sentinel = legacy.
**Atomicity envelope** (per `feedback_no_partial_refactor` and `feedback_wire_everything_up`): ISV slots + producer + consumer + HEALTH_DIAG + tests all land in the same commit. The producer kernel's outputs are wired to two consumers in this commit — the existing diagnostic readback (`read_weight_drift_diag` returns `[l2, rel]` from `out[0..2]`) and the new fold-transition controller (`reset_for_fold` reads `ISV[505] / ISV[506]`).
**Cold-start handling** (per `pearl_first_observation_bootstrap`): on the first fold or any fold where `best_params_snapshot` is `None` (no Sharpe improvement yet), the host launcher writes zeros + sentinels to the mapped-pinned buffer and skips the kernel. ISV slots stay at FoldReset sentinels (`0.8 / 0.01`) — `reset_for_fold`'s `shrink_and_perturb(0.8, 0.01)` call is bit-identical to the prior hardcoded behavior.
**Producer cadence**: HEALTH_DIAG (per-epoch). The drift kernel runs at `read_weight_drift_diag()` inside the per-epoch HEALTH_DIAG block, which sync's the stream — host visibility of `ISV[505]/ISV[506]` is guaranteed before the next fold-transition. `reset_for_fold` reads the values written by the LAST epoch's drift launch — i.e., drift OBSERVED at end-of-fold N, applied to fold N+1's perturbation (per the spec).
**Concerns surfaced** (DONE_WITH_CONCERNS):
1. **3 other `shrink_and_perturb` call sites unchanged**: `training_loop.rs:1096` (Phase 3 boundary), `training_loop.rs:3549` (D3/N3 health-triggered), `training_loop.rs:3609` (D6/N6 ensemble-collapse-triggered), and `training_loop.rs:7320` (PerturbationStrategy::ShrinkPerturbBranches with hardcoded `0.9, 0.1`). These all read from `self.hyperparams.shrink_perturb_alpha/sigma` (or hardcoded `0.9, 0.1` for backtracking). They're a different driving-signal regime — health-driven rescue interventions, not fold-transition plasticity refresh. Migrating those to ISV-adaptive is a separate concern (different signal: health collapse + ensemble disagreement, not weight drift) — surfaced for follow-up but explicitly out of scope for this commit per `feedback_no_partial_refactor` (the contract change is the fold-transition site only; the others have their own driving signal).
2. **Cold-start drift = 0.0 → α = 1.0?**: Per the spec's "Watch for" section, when drift = 0 and we're not in cold-start, `α_target = 1.0 − clamp(0, 0.05, 0.5) = 1.0 − 0.05 = 0.95` — clamped at MAX, not 1.0. The bilateral clamp's lower bound `1 − ALPHA_MAX = 0.05` ensures we never get α=1.0 (full preservation, no perturbation). The cold-start case (no `best_params_snapshot`) is handled via the host-side bypass writing sentinels — drift never appears as 0.0 from the kernel side because the kernel doesn't run in cold-start.
**Verification**:
- `SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check --workspace` — clean.
- `cargo test -p ml --lib sp18` — 4 SP18 lock tests pass: `all_sp18_slots_fit_within_isv_total_dim`, `sp18_combined_slot_layout_locked`, `sp18_shrink_perturb_slot_layout_locked`, `sp18_fold_reset_entries_present`.
- `cargo test -p ml --lib every_fold_and_soft_reset_entry_has_dispatch_arm` — pass (24 SP18 dispatch arms verified).
- `cargo test -p ml --test sp18_shrink_perturb_adaptive_test` — 8/8 pass (CPU oracle math contract).
- `cargo test -p ml --test sp18_fold_transition_test` — 3/3 pass (existing fold-transition restore-best test still validates).
**HEALTH_DIAG line format** (extended):
```
HEALTH_DIAG[N]: weight_drift [norm_l2=0.123456 relative=0.234567 branch_max=0.234567 alpha_adaptive=0.7654 sigma_adaptive=0.012345]
```
## 2026-05-08 — SP18 v2 Phase 0 Task 0.2: B-leg V_SHARE trajectory + TD-error magnitude diagnostic
Phase 0 observability: kernel + Rust launcher + V_SHARE history ring buffer + per-epoch HEALTH_DIAG emit + GPU oracle tests. All in the same atomic commit per `feedback_no_partial_refactor`.