gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(sp9): targets as Invariant-1 anchors + divergence sentinel handling

Browse Source 
Two targeted fixes to the SP9 Kelly cold-start warmup floor (Fix 37,
commit `48a8b9ee7`) for quirks observed in smoke-test-wrwkz on a
5-epoch L40S smoke. Atomic per `feedback_no_partial_refactor`.

**Quirk 1 — EMA target saturation:** ep1 HEALTH_DIAG showed
`stat_count_tgt=419 div_tgt=105002 temp_tgt=1.0 conf [stat=1.000
bhv=0.333 tmp=1.000] floor=0.0`. Pearl A sentinel-bootstrap on the
3 EMA target updaters (`kelly_*_target_ema_kernel.cu`) caused the
first observation to replace the sentinel directly, so
`target = first_obs`, `current/target = 1.0`, `confidence = 1.0`,
`floor = base × (1 − 1.0) = 0` — defeating the cold-start mechanism.

The principled fix per `feedback_isv_for_adaptive_bounds.md`:
"sufficient" is defined in absolute terms via Invariant-1 numerical
anchors. The 3 target ISV slots become constructor-written constants
(written ONCE in trainer constructor, identical pattern to existing
`config.cql_alpha → ISV[CQL_ALPHA_INDEX=48]` from Plan 1 Task 12).

  - ISV[KELLY_SAMPLE_COUNT_TARGET_INDEX=333] = 100.0 (trades)
  - ISV[KELLY_DIVERGENCE_TARGET_INDEX=334]   = 2.0   (ratio)
  - ISV[KELLY_TEMPORAL_TARGET_INDEX=335]     = 5.0   (epochs)

**Quirk 2 — divergence ratio explosion:** ep1 showed
`divergence=70005`. Algebraically `intent_f / max(eval_f, 1e-6) =
0.07 / 1e-6 = 70 000` because `ISV[EVAL_DIST_F_INDEX=338]` was still
at Pearl A sentinel 0 before the first val window populated it.
Algebraically the warmup-floor kernel still derived
`behavioral_conf = 0` (correct semantic), but the synthetic 70 000
in HEALTH_DIAG masked the real signal flow.

Fix: explicit sentinel detection in
`intent_eval_divergence_compute_kernel.cu`:

  if (eval_f < SENTINEL_THRESHOLD=1e-5)
    divergence = SENTINEL_DIVERGENCE=1e6  // forces behavioral_conf=0
  else
    divergence = intent_f / eval_f

Same `behavioral_conf = 0` outcome but with explicit provenance —
HEALTH_DIAG now reads `divergence=1e6` until eval_f matures.

**Atomic structure:**

- DELETE 3 EMA target updater kernels (`.cu` files)
- DELETE 3 entries from `crates/ml/build.rs::kernels_with_common`
- DELETE 3 cubin static byte arrays + 3 struct fields + 3 cubin
  loaders + 3 fields in trainer construction tuple in
  `gpu_dqn_trainer.rs`
- DELETE 3 launches in `launch_sp9_kelly_warmup_floor` (chain
  shrinks 5 → 2: q_var_mag_ema + main warmup-floor)
- DELETE 3 scratch slots; SP5_SCRATCH_TOTAL 268 → 265;
  SCRATCH_SP9_EVAL_DIST_BASE slides 265 → 262
- ADD constructor-write of 3 Invariant-1 anchors (100.0, 2.0, 5.0)
  immediately before layout-fingerprint write
- UPDATE 3 dispatch arms in `reset_named_state` to rewrite the
  Invariant-1 anchors at fold boundary (NOT sentinel 0) — these
  are constants, not stateful EMAs
- UPDATE 3 registry descriptions to reflect Invariant-1 anchor
  semantic + smoke-test-wrwkz evidence
- UPDATE `intent_eval_divergence_compute_kernel.cu` with
  sentinel-detect branch
- ISV slot layout UNCHANGED (3 target slots @ ISV[333..336)
  remain); Wiener-buffer linear span (`SP5_PRODUCER_COUNT=165`)
  UNCHANGED — 3 wiener triples for deleted producers become
  reserved-unused like Pearl 6's [525..543) carve-out
- audit doc Fix 37.1 entry with full provenance + smoke evidence

**Verification:**

- `SQLX_OFFLINE=true cargo check -p ml` — clean (only pre-existing
  18 warnings; no new errors or warnings).
- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset` — 4/4 pass
  including `every_fold_and_soft_reset_entry_has_dispatch_arm`.
- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — 9/9
  pass including `sp9_slots_contiguous_above_sp8_block`.

**Expected smoke signature post-fix:**

- floor: starts ≈ base_floor × 1.0, decays as confidence accumulates
- divergence: 1e6 until first val window, small (≤ 5) afterward
- conf: stat=count/100, bhv=0 until eval_dist matures, tmp=ep/5
- combined_conf reaches 1.0 only when at least one axis genuinely
  matures (typically temporal at epoch 5 in fold 0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-05-03 21:36:11 +02:00
parent 48a8b9ee7c
commit 8a25b330fc
10 changed files with 334 additions and 267 deletions
Download Patch File Download Diff File Expand all files Collapse all files

167
docs/dqn-wire-up-audit.md
View File

@@ -4679,3 +4679,170 @@ fmaxf chain. Per-fold persistence vs cross-fold persistence is the
existing carve-out — Pearl 6's KELLY_F_SMOOTH stays cross-fold; SP9's
warmup_floor is per-fold so each fold's cold-start protection re-engages
cleanly.
## Fix 37.1 — SP9 Kelly cold-start follow-up: targets as Invariant-1 anchors + divergence sentinel handling (2026-05-03)
**Symptom:** smoke-test-wrwkz (5-epoch L40S smoke on Fix 37 commit
`48a8b9ee7`) ep1 HEALTH_DIAG line:
```
sp9_kelly_warmup [floor=0.0000 divergence=70005.93 q_var_mag_ema=6.85e-2
conf [stat=1.000 bhv=0.333 tmp=1.000]
targets [stat_count_tgt=419 div_tgt=105002 temp_tgt=1.0]]
```
`floor=0` and `conf=1` from epoch 1 onward — the cold-start mechanism is
effectively never engaged.
**Root cause (two distinct quirks):**
*Quirk 1 — EMA target saturation:* The 3 target ISV slots
(`KELLY_SAMPLE_COUNT_TARGET_INDEX=333`, `KELLY_DIVERGENCE_TARGET_INDEX=334`,
`KELLY_TEMPORAL_TARGET_INDEX=335`) were originally Fix 37 routed through
`kelly_*_target_ema_kernel.cu` + `apply_pearls_ad_kernel`. Pearl A
sentinel-bootstrap on the EMA path causes the FIRST observation to
replace the sentinel directly (per
`pearl_first_observation_bootstrap.md`). So `target = first_obs`, ratio
`current/target = 1.0`, confidence `clamp(ratio, 0, 1) = 1.0`,
`floor = base × (1 − 1.0) = 0`. The cold-start mechanism is defeated
because the EMA target tracks the running observation rather than a
fixed threshold.
The principled fix per `feedback_isv_for_adaptive_bounds.md`:
"sufficient" is defined in absolute terms via Invariant-1 numerical
anchors. The 3 target slots become constructor-written constants
(written ONCE in the trainer constructor, similar to existing
`config.cql_alpha → ISV[CQL_ALPHA_INDEX=48]`), and the target-relative
ratio `current_observation / fixed_anchor` then yields a self-normalised
confidence in [0, 1] that rises monotonically as samples /
divergence-stability / fold-time accumulate.
- `ISV[KELLY_SAMPLE_COUNT_TARGET_INDEX=333] = 100.0f` — minimum trade
samples for statistical confidence to release the cap
- `ISV[KELLY_DIVERGENCE_TARGET_INDEX=334] = 2.0f` — divergence ratio
above which intent-vs-eval is considered diverged enough to keep the
cap closed
- `ISV[KELLY_TEMPORAL_TARGET_INDEX=335] = 5.0f` — minimum epochs in
fold for temporal confidence to release the cap (safety net per
Risk 3 in spec)
*Quirk 2 — divergence ratio explosion:* The
`intent_eval_divergence_compute_kernel.cu` previously computed
`divergence = max(intent_f, EPS_DIV) / max(eval_f, EPS_DIV)`. Before the
first val-window populates `ISV[EVAL_DIST_F_INDEX=338]`, the slot is at
Pearl A sentinel 0. With `intent_f ≈ 0.07` and `eval_f` at sentinel,
flooring at `EPS_DIV=1e-6` produced a synthetic
`divergence ≈ 0.07 / 1e-6 ≈ 70 000` (matched in smoke-test-wrwkz
exactly). Algebraically the warmup-floor kernel still derived
`behavioral_conf = clamp(1 − 70 000/2, 0, 1) = 0` (correct semantic
"no behavioral signal yet"), but the synthetic 70 000 in HEALTH_DIAG
masked the actual signal-flow and was easily misread as a runaway.
The principled fix: explicitly detect the sentinel via threshold
`SENTINEL_THRESHOLD = 1e-5f` (well below any realisable `eval_f` after
the Pearl A first-observation replacement) and emit
`SENTINEL_DIVERGENCE = 1e6f`. Same `behavioral_conf = 0` outcome but
with explicit provenance — HEALTH_DIAG now reads `divergence=1e6` until
the first val window populates `eval_f`, at which point the real ratio
takes over.
**Fix structure (atomic commit per `feedback_no_partial_refactor`):**
- DELETE 3 EMA target updater kernels (`.cu` files):
- `kelly_sample_count_target_ema_kernel.cu`
- `kelly_divergence_target_ema_kernel.cu`
- `kelly_temporal_target_ema_kernel.cu`
- DELETE 3 entries from `crates/ml/build.rs::kernels_with_common`
- DELETE 3 cubin static byte arrays + 3 struct fields + 3 cubin loaders
+ 3 fields in trainer construction tuple in `gpu_dqn_trainer.rs`
- DELETE 3 launches in `launch_sp9_kelly_warmup_floor` (chain shrinks
from 5 → 2 launches: q_var_mag_ema + main warmup-floor kernel)
- DELETE 3 scratch slots (`SCRATCH_SP9_SAMPLE_COUNT_TARGET=262`,
`SCRATCH_SP9_DIVERGENCE_TARGET=263`, `SCRATCH_SP9_TEMPORAL_TARGET=264`);
`SP5_SCRATCH_TOTAL` 268 → 265; `SCRATCH_SP9_EVAL_DIST_BASE` slides
265 → 262 (3 floats reclaimed)
- ADD constructor-write of 3 Invariant-1 anchors (100.0, 2.0, 5.0) at
`gpu_dqn_trainer.rs` constructor-time, immediately before the layout-
fingerprint write — same pattern as existing
`*sig_ptr.add(CQL_ALPHA_INDEX) = config.cql_alpha`
- UPDATE 3 dispatch arms in `reset_named_state` to rewrite the same
Invariant-1 anchors at fold boundary (NOT sentinel 0) — these are
constants, not stateful EMAs; they must never reach 0 between folds
- UPDATE 3 registry descriptions in `state_reset_registry.rs` to reflect
the Invariant-1 anchor semantic + the smoke-test-wrwkz evidence
- UPDATE `intent_eval_divergence_compute_kernel.cu` with
`SENTINEL_THRESHOLD=1e-5f` / `SENTINEL_DIVERGENCE=1e6f` sentinel-detect
branch
- ISV slot layout UNCHANGED — the 3 target slots @ ISV[333..336)
remain. Wiener-buffer linear span (`SP5_PRODUCER_COUNT=165`) UNCHANGED
— the 3 wiener triples for the deleted producers become "reserved-
unused" similar to Pearl 6's [525..543) carve-out (no
`launch_apply_pearls` consumer fires for those slots, so the wiener
state stays zero-initialised)
**Files touched:**
- `crates/ml/src/cuda_pipeline/kelly_sample_count_target_ema_kernel.cu` (DELETED)
- `crates/ml/src/cuda_pipeline/kelly_divergence_target_ema_kernel.cu` (DELETED)
- `crates/ml/src/cuda_pipeline/kelly_temporal_target_ema_kernel.cu` (DELETED)
- `crates/ml/build.rs` — 3 entries removed; comment updated to "Four
producer kernels"
- `crates/ml/src/cuda_pipeline/intent_eval_divergence_compute_kernel.cu` —
sentinel-detect branch for uninitialised `eval_f`
- `crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs` — 3 cubin statics +
3 struct fields + 3 cubin loaders + 3 tuple entries + 3 launches
removed; constructor adds 3 Invariant-1 anchor writes;
`SP5_SCRATCH_TOTAL` 268 → 265; `SCRATCH_SP9_EVAL_DIST_BASE` 265 → 262;
3 obsolete `SCRATCH_SP9_*_TARGET` constants removed
- `crates/ml/src/trainers/dqn/state_reset_registry.rs` — 3 descriptions
updated to reflect Invariant-1 anchor semantic
- `crates/ml/src/trainers/dqn/trainer/training_loop.rs` — 3 dispatch
arms updated to write 100.0 / 2.0 / 5.0 (NOT 0.0)
- `docs/dqn-wire-up-audit.md` (this entry)
**Verification:**
- `SQLX_OFFLINE=true cargo check -p ml` — clean (only pre-existing 18
warnings; no new errors or warnings).
- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset` — all 4 tests
pass (contract test `every_fold_and_soft_reset_entry_has_dispatch_arm`
still covers all 9 SP9 entries; their dispatch arms now write
Invariant-1 anchors instead of 0).
- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — all 9
tests pass (slot layout unchanged; producer count unchanged).
**Expected smoke-test signature post-fix (5-epoch L40S):**
- `floor`: starts at base_floor × 1.0 in epoch 1 (Pearl D Wiener-α
smoothing of base_floor × (1 − combined_confidence) where confidence
is genuinely low), decays as confidence accumulates over epochs
- `divergence`: `1e6` (SENTINEL_DIVERGENCE) until the first val window
populates `ISV[EVAL_DIST_F_INDEX=338]`; small (≤ 5) afterward
- `conf [stat=X bhv=Y tmp=Z]`: `stat = sample_count/100` rises with
trade count; `bhv` stays 0 until eval_dist matures; `tmp = epoch_idx/5`
rises linearly with fold-time
- combined_conf reaches 1.0 only when at least one axis genuinely
matures — typically temporal axis at epoch 5 in fold 0
**Per-pearl provenance:**
- `feedback_isv_for_adaptive_bounds.md` — Invariant-1 numerical anchors
are the survivable case for hardcoded constants (100.0, 2.0, 5.0
define what "sufficient" means in absolute terms).
- `pearl_controller_anchors_isv_driven.md` — caveat: not every
anchor / target needs to be EMA-tracked. Cold-start exit thresholds
are the canonical case where a fixed threshold is the principled
design (matching test for "regime change") — the EMA anti-pattern
(target = current_obs) was a misapplication.
- `pearl_first_observation_bootstrap.md` — sentinel-detect branch in
divergence kernel makes "before first observation" explicit; same
semantic as Pearl A but with cleaner provenance in HEALTH_DIAG.
- `feedback_no_partial_refactor.md` — atomic commit (kernel deletions
+ Rust deletions + constructor-writes + dispatch-arm updates + audit
entry, all in one commit).
- `feedback_no_stubs.md` — Invariant-1 anchors are real values defining
"sufficient" in absolute terms (100 trades / 2.0 ratio / 5 epochs),
not stubs.
- `feedback_no_cpu_compute_strict.md` — constructor-writes are GPU-side
ISV writes via mapped-pinned `sig_ptr`; same path as all existing
constructor-time ISV initialization (CQL_ALPHA, GAMMA_DIR, etc.).