3c318953a7d4df68f674872b01c2adcd9c7a5d54
4848 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
3c318953a7 |
feat(sp18 v2 P1.T3-T6): atomic delete SP13/SP16 hold_cost_scale chain
Deletes the entire reactive Hold-cost controller chain from SP13 P0a / SP16
P2 / SP16 T3 atomically — replaced by the SP18 D-leg structural opportunity-
cost reward landing in Phase 2.
Per DD7(c): observation chain preserved (slots 380, 382). Slot 380 becomes
constructor-init-only diagnostic at HOLD_COST_BASE=0.005, never updated.
18 consumer sites deleted (8 from plan checklist + 10 from A1-A10 audit
expansion):
Production code:
- 3 reward subtraction sites (replaced with Phase 2 placeholder comments)
- hold_cost_scale_update_kernel.cu (file deleted)
- HoldCostScaleUpdateOps struct in gpu_aux_trunk.rs (~120 lines)
- launch_hold_cost_scale_update in gpu_dqn_trainer.rs (forwarder)
- Host controller block in training_loop.rs (~60 lines)
- Slot constants [461..468) in sp14_isv_slots.rs
- HOLD_COST_CONTROLLER_GAIN/FLOOR/CEIL constants in sp13_isv_slots.rs
- state_layout.cuh mirror constants for [461..468)
- build.rs cubin manifest entry
- 7 fold-reset registry entries + 7 dispatch arms
Tests:
- sp14_oracle_tests.rs lines 2173-2926 (11 tests + helpers, ~750 lines)
— these include_bytes! the deleted cubin and cannot survive
- lock_sp18_v2_pp4_retired_chain test (deleted, no inverse-contract
replacement — locking a deletion is pointless ceremony per user call)
Layout fingerprint: range [461..468) is RESERVED gap (SP14-C.1 pattern;
ISV_TOTAL_DIM stays at 507 for checkpoint compatibility). Layout fingerprint
seed updated atomically: HOLD_COST_SCALE + HCS_* slots replaced with
RESERVED_GAP_461_TO_468=SP18_P1_RETIRED marker.
INTERIM STATE: 3 reward sites are now missing the per-bar Hold cost. This
is forbidden to L40S-dispatch until Phase 2 lands the
compute_sp18_hold_opportunity_cost device fn that replaces the deleted
subtraction. The audit doc records this hold.
Validation: cargo check --workspace clean; cargo test -p ml --lib passes
(state_reset_registry::every_fold_and_soft_reset_entry_has_dispatch_arm,
sp16_t3_wiener_welford_slot_layout_locked, sp18_combined_slot_layout_locked,
sp18_shrink_perturb_slot_layout_locked, sp18_fold_reset_entries_present —
all OK). Audit fingerprint: per-bar Hold-cost subtraction sites count
13 → 0 (smoking gun for complete deletion).
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
Audit: docs/sp18-wireup-audit.md (full A1-A10 expansion + post-deletion
fingerprint)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
b43413e4d3 |
feat(sp18 v2 P1.T1+T2): pre-dispatch consumer-audit script + pre-commit hook
Phase 1 Tasks 1.1 + 1.2 of docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md.
Per the plan's audit-first design: surface every live reference to the
SP13/SP16 Hold-cost-scale chain (D-leg) and the TD(λ) `q_next = rewards`
self-bootstrap (B-leg) BEFORE atomic deletion. This catches the SP17-style
"missed quantile_q_select consumer" failure mode where a sweeping refactor
left a dangling reference that compiled but broke at runtime.
What lands
- `scripts/audit_sp18_consumers.sh` — 17-section grep that walks every
consumer pattern from the plan's locked checklist (D-leg slot 380,
461, [462..468) HCS_*, hold_cost_scale_update kernel, hold_rate_observer
kernel retained chain, build.rs cubin manifest, state_layout.cuh
mirror constants, state_reset_registry entries, plus B-leg
td_lambda_kernel launch sites, q_next origin, rewards_out consumers,
PER priority sites, replay buffer schema, c51_loss target-Q origin,
target_params_buf consumers, PopArt slot 63 references). Three modes:
default (full grep output), `--fingerprint` (per-section per-file hit
count for diff-able snapshots), `--check` (diff fingerprint vs locked
snapshot in docs/sp18-wireup-audit.md, exit 1 on drift).
- `docs/sp18-wireup-audit.md` — Phase 1 Task 1.1 outcome with two
sections of import:
1. The plan's locked checklist (8 entries the plan author explicitly
identified as deletion targets).
2. The 10 ADDITIONAL CONSUMERS surfaced by the audit (A1-A10) that
the plan-author missed. Per the task input's halt-on-drift
directive, Phase 1 Tasks 1.3-1.5 (atomic deletion) are HALTED
pending human review of the expanded scope. The audit doc is the
spec-amendment record.
3. A locked fingerprint snapshot the pre-commit hook uses for drift
detection on subsequent commits.
B-leg verification confirms B-DD4 (no PER migration) + B-DD1 (target_
params_buf reusable) + the single q_next bootstrap site at
gpu_experience_collector.rs:4143.
- `scripts/pre-commit-hook.sh` — Invariant 7 list extended with the new
audit doc; new `check_sp18_consumer_audit` step runs the audit script
in `--check` mode whenever a commit touches the chain files
(experience_kernels.cu, hold_*_kernel.cu, state_layout.cuh, sp1[3-8]_
isv_slots.rs, gpu_dqn_trainer.rs, gpu_aux_trunk.rs, gpu_experience_
collector.rs, state_reset_registry.rs, training_loop.rs, build.rs,
sp1[3-8]_oracle_tests.rs). Drift triggers a hook failure with a
pointer to the regeneration command. This is a generalisation of
Invariant 7 and addresses Open Q-B from the spec (audit-as-pre-commit
hook for SP-chain consumer drift).
Findings (HALT trigger)
The audit found 10 consumers NOT in the plan's locked 8-site checklist:
A1 — gpu_aux_trunk.rs:1240-1323 HoldCostScaleUpdateOps struct + impl
(84 lines; the plan said launcher was in gpu_dqn_trainer but the
real struct/impl lives in gpu_aux_trunk; trainer just has a thin
forwarding method)
A2 — sp14_oracle_tests.rs:2174-2920 11 GPU oracle tests (~750 lines)
directly exercising the deleted kernel via include_bytes!
(sp16_phase2_hold_cost_scale_climbs_with_overrun + 10 others)
A3 — training_loop.rs:8971-9043 7 dispatch arms in reset_named_state
(slot 461 + HCS_* slots 462-467); contract test
every_fold_and_soft_reset_entry_has_dispatch_arm requires
atomic deletion alongside registry entries
A4 — gpu_dqn_trainer.rs:23200-23216 constructor block writing
HOLD_COST_BASE to slot 380 (RETAINED per DD7c, comment requires
update)
A5 — gpu_dqn_trainer.rs:617, 2245-2256 doc-block prose
A6 — sp13_isv_slots.rs:75-77 HOLD_COST_CONTROLLER_GAIN/FLOOR/CEIL
constants (HOLD_COST_BASE retained for A4)
A7 — gpu_experience_collector.rs:5579-5585 stale comment doc-ref
A8 — sp5_isv_slots.rs:325-327 comment doc-ref
A9 — state_reset_registry.rs:1138-1271 7 already-RETIRED entries
promote to FULL DELETION
A10 — state_reset_registry.rs:2256-2308 lock_sp18_v2_pp4_retired_chain
contract test asserts retired entries STILL EXIST; must be
deleted/rewritten when entries are removed
None of these are architectural surprises — they're straight extensions
of the atomic-deletion scope. But per the task input's halt-on-drift
directive ("If your audit surfaces ANY additional consumer, halt and
report — do NOT proceed with deletion ad-hoc"), Phase 1 Tasks 1.3-1.5
HALT pending human sign-off on the expanded scope.
Path forward (next phase)
Either expand the atomic-deletion commit scope to cover all 18 sites
(8 plan + 10 audit-surfaced) — recommended per `feedback_no_partial_
refactor`; the audit doc serves as the spec-amendment record — or
human-review the audit doc and explicitly approve/reject each A* entry.
The audit doc + script + hook are the pre-condition for either path
and ship together as Tasks 1.1 + 1.2 of Phase 1. Tasks 1.3-1.6 await
human go-ahead.
Branch is at INTERIM STATE: NOT runnable for L40S smoke between this
commit and the post-Phase-1.6 close-out. The interim is purely-additive
(audit script + hook + doc); the actual deletion that creates the
"3 reward sites missing Hold cost" interim from the plan's Task 1.5
has NOT yet been performed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0b737ec30a |
feat(sp18-v2): ISV-adaptive shrink-and-perturb at fold transition
Replaces the hardcoded `α=0.8 / σ=0.01` constants in `reset_for_fold`'s
`shrink_and_perturb` call with ISV-driven adaptive bounds per
`feedback_isv_for_adaptive_bounds` and `feedback_adaptive_not_tuned`.
The driving signal is the relative weight drift `||current - best||₂ /
||best||₂` already computed each epoch by the SP18 v2 weight-drift
diagnostic kernel (commit
|
||
|
jgrusewski
|
aab13a83f2 |
feat(sp18-v2): weight-drift HEALTH_DIAG diagnostic
Add per-epoch HEALTH_DIAG line measuring how far current online params
have drifted from the best-Sharpe checkpoint. The fold-transition
restore-best fix preserves peak weights across folds, but within-fold
edge-decay still happens (Adam keeps stepping after peak Sharpe). The
drift diagnostic surfaces the trajectory so operators can correlate
Sharpe-peak decay with parameter movement.
HEALTH_DIAG line:
HEALTH_DIAG[N]: weight_drift [norm_l2={:.6} relative={:.6} branch_max={:.6}]
Where:
- norm_l2 = ||params_flat - best_params||₂ (absolute L2)
- relative = norm_l2 / max(||best_params||₂, EPS) (relative drift)
- branch_max — currently mirrors `relative` (single-scalar form per spec).
Per-branch breakdown is a deferred follow-up: branch heads are
protected from S&P (skip_start..skip_end), so trunk drift dominates
the L2 in any case.
Cold-start: when best_params_snapshot is None (first fold or any fold
without a Sharpe improvement yet), launcher emits [0.0, 0.0] directly
(kernel skipped, mapped-pinned host slice written from CPU).
Distinguishable from "snapshot matches current" via best_epoch elsewhere.
Kernel (weight_drift_diag_kernel.cu): single block × 256 threads. Two
block tree-reductions sharing one shmem tile sequentially:
Pass 1: ||params - best||₂² over (params - best)
Pass 2: ||best||₂² over best
Thread 0 finalizes sqrt + EPS-floored ratio + writes via
__threadfence_system() for PCIe-visible coherence.
Per feedback_no_atomicadd — block tree-reduce only.
Per feedback_no_htod_htoh_only_mapped_pinned — output is
MappedF32Buffer<2>; cold-start bypass uses host_slice_mut.
Wire-up (atomic):
- crates/ml/build.rs: cubin manifest entry
- crates/ml/src/cuda_pipeline/weight_drift_diag_kernel.cu (NEW)
- crates/ml/src/trainers/dqn/fused_training.rs: field + constructor
+ launch_weight_drift_diag() + read_weight_drift_diag()
- crates/ml/src/trainers/dqn/trainer/mod.rs:
DQNTrainer::read_weight_drift_diag() public wrapper (CPU-only path
emits zeros)
- crates/ml/src/trainers/dqn/trainer/training_loop.rs: per-epoch
HEALTH_DIAG emit adjacent to SP17 dueling block
GPU oracle test (crates/ml/tests/sp18_weight_drift_test.rs, NEW):
4 cases, all #[ignore = "requires GPU"]:
1. weight_drift_matches_cpu_oracle_4096 — N=4096, ε=1e-5
2. weight_drift_is_zero_when_params_equals_best
3. weight_drift_eps_floor_when_best_is_zero (catches NaN/Inf edge)
4. weight_drift_n_zero_emits_zeros (degenerate guard)
All 4 pass on local RTX 3050 Ti.
Pre-commit Invariant 7: docs/dqn-wire-up-audit.md updated with kernel
algorithm, wire-up table, cross-pearl invariants, and oracle-test
inventory.
Per:
- feedback_no_atomicadd (block tree-reduce in drift kernel)
- feedback_no_htod_htoh_only_mapped_pinned (MappedF32Buffer<2>)
- feedback_wire_everything_up (kernel + launcher + emit + test atomic)
- pearl_no_host_branches_in_captured_graph (cold-path, outside graph)
- feedback_no_partial_refactor (single atomic commit)
|
||
|
jgrusewski
|
3e9cefdbd9 |
fix(sp18-v2): restore-best precedes shrink-and-perturb at fold transition
`train_walk_forward` called `reset_for_fold().await?` directly at the fold boundary, which applies `shrink_and_perturb(α=0.8, σ=0.01)` to whatever `params_flat` holds. At end-of-fold, `params_flat` carries the LATEST-EPOCH decayed weights, NOT the best-Sharpe snapshot saved mid-fold. Fold N+1 then started from `0.8 × decayed + noise` and carried the within-fold edge-decay forward. Insert `restore_best_gpu_params()` BEFORE `reset_for_fold` so S&P operates on the best-Sharpe snapshot. Cold-start guard swallows the "no snapshot saved" error on the first fold (or any fold without a Sharpe improvement) — matches pre-fix behavior on cold start, no regression. State interaction with `reset_for_fold`: restore-best writes `params_flat` ONLY (online weights); reset_for_fold then runs S&P on the restored online, hard-syncs target ← restored online, and resets Adam state on every optimizer. Non-conflicting. `best_params_snapshot` is constructor-init `None` on FusedTrainingCtx and is NEVER cleared at fold boundary. The trainer's `self.best_sharpe` IS reset to NEG_INFINITY in `reset_for_fold` so the first improvement in fold N+1 will overwrite the snapshot. Until then, the snapshot holds whichever fold most recently saved a peak — intended training-scoped behavior. Strict within-fold semantics flagged as a follow-up consideration. Behavioral test (CPU oracle) pins the math contract: - shrink_and_perturb(α, σ) on X = α × X + (1−α) × N(0, σ) - with-fix vs without-fix differ by α × (W_best − W_curr) - cold-start (best == current) is bit-identical between orderings GPU-level kernel coverage stays in compile_training_kernels smoke and the L40S 30-epoch validation gating SP18 Phase 1. Pre-commit Invariant 7: docs/dqn-wire-up-audit.md updated with full rationale, state-interaction analysis, lifecycle notes, and the preserved cross-pearl invariants. Per: - feedback_no_partial_refactor (call-ordering + test + audit-doc atomic) - feedback_no_legacy_aliases (reuses existing API unchanged) - feedback_wire_everything_up (restore_best_gpu_params gains second cold-path consumer) - pearl_no_host_branches_in_captured_graph (runs outside graph capture) |
||
|
jgrusewski
|
662bf7eb36 |
plan(sp18 v2): Phase 0 Task 0.5 — pearl candidate draft
Drafts `pearl_diagnostic_decomposition_before_reward_intervention` per
the plan's Task 0.5 deliverable. Captures the discipline pattern
applied in Tasks 0.1–0.2:
Before changing reward weights or Bellman target arithmetic to fight
a Q-attractor, instrument a per-action decomposition of the existing
reward components AND a trajectory observable on the Q-target
distribution. The instrumentation is observability-only and lands
atomically. One epoch of dispatch surfaces whether the suspected
pathology is real BEFORE any production-path code changes.
Pearl is DRAFT — pending the Task 0.3 (reviewer L40S 1-epoch
dispatch) + Task 0.4 (KILL CRITERION evaluation) outcomes. Promotion
to user memory + cross-reference to MEMORY.md is gated on both legs
returning PROCEED.
Cross-references: pearl_canary_input_freshness_launch_order,
pearl_first_observation_bootstrap, pearl_wiener_alpha_floor_for_
nonstationary, feedback_no_partial_refactor, feedback_wire_everything_up.
No production code changes — single doc-only commit.
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
§ Phase 0 Task 0.5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
27f8e332da |
plan(sp18 v2): Phase 0 Task 0.2 — B-leg V_SHARE trajectory + TD-error magnitude diagnostic
Adds the SP18 v2 Phase 0 B-leg observability scaffold per the plan's
Task 0.2 spec:
- New kernel `td_error_mag_ema_kernel.cu` (single-block × 256 threads):
reads `td_errors_buf [B]` (already populated by C51/MSE loss for PER
priority recomputation, post-train-step), block tree-reduces
`mean(|td_errors[b]|)` (no atomicAdd), and EMA-blends into
`ISV[TD_ERROR_MAG_EMA_INDEX=493]` via Pearl-A first-observation
bootstrap (sentinel 0.0 → REPLACE) + fixed α=`WELFORD_ALPHA_MIN=0.4`
per `pearl_wiener_alpha_floor_for_nonstationary`. The TDB_* Welford
accumulators in slots [498..504) are RESERVED for the Phase 4
q_next_target Wiener-α chain — not used in Phase 0.
- Cubin manifest entry in `crates/ml/build.rs` + `TD_ERROR_MAG_EMA_
CUBIN` re-export in `gpu_dqn_trainer.rs`.
- `sp18_td_error_mag_ema_kernel` field on `GpuDqnTrainer` + cubin load
on the trainer's stream + `launch_sp18_td_error_mag_ema_update()`
cold-path launcher + `read_sp18_td_error_mag_ema()` convenience
wrapper.
- New `sp18_v_share_history: [[f32; 4]; 5]` field on `DQNTrainer` —
fixed-size ring buffer of the last 5 epochs of per-branch V_SHARE
EMA readings (slots [478..482) per SP17 Phase 3.2). Initialised to
`[[NaN; 4]; 5]`; epochs 0–3 emit `nan` as the slope and skip the
ISV write; epoch 4 onward computes `(EMA[now] - EMA[now-4]) / 4`
per branch and writes the dir-branch slope to
`ISV[V_SHARE_TREND_DIAG_INDEX=496]`.
- Two new HEALTH_DIAG lines in `training_loop.rs` at the per-epoch
boundary (right after the SP18 reward_decomp line):
HEALTH_DIAG[N]: v_share_traj [dir_slope=X mag_slope=Y ord_slope=Z urg_slope=W]
HEALTH_DIAG[N]: td_error_pre [magnitude_ema=X]
V_SHARE slope is host-side computation against the ring buffer
(`(now - now_m4) / 4` per branch). TD-error magnitude is post-blend
ISV slot 493 read (producer fires inside `read_sp18_td_error_mag_
ema()`). Pre-fix baseline for the B-DD9 ratio gate
(`avg(|TD-error|) ratio post-fix / pre-fix ∈ [0.5, 5.0]`).
- New GPU oracle tests in `crates/ml/tests/sp18_hold_reward_oracle_
tests.rs`:
* `td_error_mag_ema_pearl_a_bootstrap` — synthetic td_errors with
closed-form mean(|td|)=1.125; pre-populate slot at sentinel;
assert post-launch slot equals the mean (Pearl-A direct-replace).
* `td_error_mag_ema_blend_post_bootstrap` — synthetic td_errors
with mean(|td|)=0.5; pre-populate slot at non-sentinel 1.0;
assert blend equals `(1 - 0.4) × 1.0 + 0.4 × 0.5 = 0.8`.
Pure observability — no production-path consumer in this commit. No
reward changes, no Bellman target changes, no kernel modifications to
the action-selection or training paths. Per `feedback_no_partial_
refactor` the kernel + cubin manifest + buffer + launcher + ring
buffer + HEALTH_DIAG emit + GPU oracle tests all land atomically.
Verification:
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check --workspace clean.
All 4 GPU oracle tests pass on RTX 3050 Ti (2.09s):
reward_decomp_per_action_gpu_oracle, reward_decomp_empty_bin,
td_error_mag_ema_pearl_a_bootstrap, td_error_mag_ema_blend_post_bootstrap.
Existing slot lock + state_reset_registry tests still pass.
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
§ Phase 0 Task 0.2.
Audit: docs/dqn-wire-up-audit.md § "SP18 v2 Phase 0 Task 0.2".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
15b50ac38f |
plan(sp18 v2): Phase 0 Task 0.1 — D-leg per-action reward decomposition diagnostic
Adds the SP18 v2 Phase 0 D-leg observability scaffold per the plan's
"Phase 0 — diagnostic emit (NO functional change)" task:
- New kernel `reward_decomp_diag_kernel.cu`: block tree-reduce
(4 blocks × 256 threads, one block per direction-axis bin) reading
`reward_components_per_sample [N×6]` + `actions_out [N]` and emitting
5 per-bin stats (mean r_micro / mean r_opp_cost / mean r_popart /
mean |reward| / fire_rate) into a 20-float row-major output. Bin
order: Hold(0)→Long(1)→Short(2)→Flat(3); col order: micro→opp→
popart→abs→fire. Empty-bin guard emits 0.0 (NOT NaN) per the
consumer-side KILL CRITERION arithmetic contract.
- Cubin manifest entry in `crates/ml/build.rs` + `REWARD_DECOMP_DIAG_
CUBIN` re-export in `gpu_dqn_trainer.rs`.
- 20-float `MappedF32Buffer sp18_reward_decomp_diag_buf` field on
`GpuDqnTrainer` + accessor pair (`sp18_reward_decomp_diag_dev_ptr`
for the writer-side launcher; `read_sp18_reward_decomp_diag` for the
HEALTH_DIAG reader). Buffer is constructor-zeroed so cold-start
HEALTH_DIAG emits a deterministic zero block.
- `sp18_reward_decomp_diag_kernel` field on `GpuExperienceCollector` +
cubin load on the collector's stream + `launch_sp18_reward_decomp_
diag(n, b1, b2, b3, out_dev_ptr)` launcher. Wired in
`training_loop.rs` at the per-step boundary, BEFORE
`launch_reward_component_ema_inplace` (which `memset_zeros` the
source buffer after consuming it) per `pearl_canary_input_freshness_
launch_order`.
- New per-epoch HEALTH_DIAG line emit at the existing per-epoch
boundary (after the SP17 dueling line):
HEALTH_DIAG[N]: reward_decomp [hold(micro=X opp=Y popart=Z abs=W
fire=F) long(...) short(...) flat(...)]
Reads the mapped-pinned 20-float diag buffer directly via the
collector→trainer host_ptr — no DtoH copy.
- New `crates/ml/tests/sp18_hold_reward_oracle_tests.rs`:
* `reward_decomp_per_action_cpu_oracle` (CPU oracle pinning the
per-bin reduction math against a 4-sample synthetic batch).
* `reward_decomp_per_action_gpu_oracle` (GPU oracle, ignored unless
`--ignored`; asserts kernel matches CPU oracle bit-for-bit within
1e-6 f32 budget).
* `reward_decomp_empty_bin_emits_zero_not_nan` (empty-bin contract
guard).
Pure observability — no production-path consumer in this commit. No
reward changes, no Bellman target changes, no kernel modifications to
the action-selection or training paths. Per
`feedback_no_partial_refactor` the kernel + cubin manifest + buffer +
launcher + production wire-up + HEALTH_DIAG emit + GPU oracle test all
land atomically.
Verification:
SQLX_OFFLINE=true CUDA_COMPUTE_CAP=86 cargo check --workspace clean.
CPU oracle test passes; GPU oracle + empty-bin guard both pass on
RTX 3050 Ti (2.13s).
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
§ Phase 0 Task 0.1.
Audit: docs/dqn-wire-up-audit.md § "SP18 v2 Phase 0 Task 0.1".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
52100ae7a1 |
docs(sp18 v2): PP.5 H100 collection-time profiling — deferred-to-Phase-4 marker
PP.5 is the B-DD6 headroom-check: profile current
gpu_experience_collector::collect_experience end-to-end ms/cycle on
H100 to confirm budget for the new target-net forward pass (B-leg
Phase 4 estimated ~30% slowdown). Plan explicitly marks this as
reviewer-only ("subagent CANNOT run this step (reviewer-only). Hands
off to reviewer at PP.5.").
Pre-Phase subagent dispatch context lacks H100/Argo deploy access, so
the actual nsys profile is deferred to Phase 4 reviewer pickup. NO
production code modified — this is observability infrastructure for
the Phase 4 atomic refactor's regression check.
Pre-Phase exit criteria still satisfied (22 ISV slots locked, registry
entries with dispatch arms, retirement markers, audit doc current);
only the H100 measurement is deferred per the plan's reviewer-only
gating.
Phase 4 pickup instructions documented in audit doc (4-step procedure:
push → argo-train profile → read nsys → fallback rules + human-review
gate).
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
c66e15f6d0 |
plan(sp18 v2): mark SP16 P2/T3 HOLD_COST_SCALE chain RETIRED (PP.4)
Per DD7=c, the SP18 D-leg structural Hold opportunity-cost reward (compute_sp18_hold_opportunity_cost in trade_physics.cuh + ISV[483..493) cap producer chain) replaces the reactive per-bar Hold-cost chain (SP13 P0a + SP16 P2 + SP16 T3). This Pre-Phase commit is slot-level documentation only — Phase 1 of SP18 deletes the producer kernel + 3 consumer sites + host controller atomically. Retired slot descriptions (sentinel 0.0 preserved for checkpoint compat): 461 sp16_phase2_hold_cost_scale_adaptive 462 sp16_t3_hcs_target_mean (Welford accumulator) 463 sp16_t3_hcs_target_m2 464 sp16_t3_hcs_diff_mean 465 sp16_t3_hcs_diff_m2 466 sp16_t3_hcs_prev_target 467 sp16_t3_hcs_sample_count Preserved per DD7=c "keep the observation chain": 380..383 SP13 P0a hold-pricing observer (per-step writes still active) 460 MIN_HOLD_TEMPERATURE_ADAPTIVE (separate producer chain) 468..474 MHT_* Welford accumulators (MIN_HOLD_TEMPERATURE) Pattern: SP14-C.1 RESERVED-gap precedent — retired slots stay in the layout_fingerprint_seed + state_reset_registry (sentinel 0.0 rewritten each fold) so checkpoints with the old slot names continue to load. Legacy consumers fall back to scale=1.0 (bit-identical pre-Phase-2 behavior). New `sp16_t2_t3_slots_marked_retired` lock test asserts the "SP18 v2: RETIRED" marker is on each retired slot's description, AND that the preserved MHT_* slots are NOT marked retired. Audit doc updated per Invariant 7. Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2b91f587b7 |
plan(sp18 v2): 22 fold-reset registry entries + dispatch arms (PP.3)
Adds FoldReset registry coverage for all 22 SP18 ISV slots [483..505)
landed in PP.2 (10 D-leg + 12 B-leg), in lockstep with matching match
arms in `reset_named_state` per the existing
`every_fold_and_soft_reset_entry_has_dispatch_arm` contract test (which
catches the "add registry entry, forget dispatch arm → runtime panic
at fold boundary" pattern surfaced twice historically — SP5 #281, SP7
T7 commit
|
||
|
jgrusewski
|
16100912c3 |
plan(sp18 v2): allocate 22 ISV slots [483..505) for D-leg + B-leg
Adds combined SP18 D-leg Hold-reward adaptive-cap slots [483..493) + B-leg TD(λ) Q(s') bootstrap diagnostics + PopArt reset flag at [493..505) per spec DD6 (D-leg 10 slots) + B-DD13 (B-leg 12 slots). D-leg [483..493) — 10 slots: 483 HOLD_REWARD_POS_CAP (sentinel 5.0, bounds [0.5, 50]) 484 HOLD_REWARD_NEG_CAP (sentinel -10.0) 485 HOLD_REWARD_DECOMP_DIAG (sentinel 0.0) 486 HOLD_OPP_COST_FIRE_RATE_EMA (sentinel 0.0) 487..493 HRC_* Welford accumulators (mirrors SP16 T3 HCS_* pattern) B-leg [493..505) — 12 slots: 493 TD_ERROR_MAG_EMA (HEALTH_DIAG B-DD9 ratio gate input) 494 Q_NEXT_TARGET_P99 (target-Q bootstrap bound check) 495 Q_NEXT_MINUS_REWARD_P99 (sanity: should be O(1) post-fix) 496 V_SHARE_TREND_DIAG (B-leg synergy probe) 497 POPART_RESET_FLAG (sentinel 1.0 — one-shot, B-DD11) 498..504 TDB_* Welford accumulators (mirrors HRC_* pattern) 504 RESERVED (B-leg follow-up) Pearl-A first-observation bootstrap sentinels match position-side SP14 P0-A REWARD_POS_CAP_ADAPTIVE pattern (POS=5.0, NEG=-10.0). Slot 497 POPART_RESET_FLAG sentinel = 1.0 per B-DD11 — host writes 1.0 once at first SP18 epoch, kernel zeroes after consuming, gating the per-fold PopArt slot 63 EMA reset at the SP18 deployment boundary. ISV_TOTAL_DIM bumped 483 → 505; layout_fingerprint_seed updated with all 22 new slot names; state_layout.cuh C-side mirror in lockstep (continues SP14-P0A/P1/audit-fix-4A/4B mirror precedent — SP16/SP17 slots intentionally not mirrored per existing pattern, only SP18 gets fresh mirror entries). `feedback_no_partial_refactor`: both legs share an ISV section + a single fingerprint bump; SP13 [380..383) and SP16 [461..474) slots remain ALLOCATED but RETIRED in PP.4 (sentinel 0.0, no producer launch — RESERVED-gap pattern from SP14-C.1 preserves checkpoint compatibility). Audit doc updated per Invariant 7 with Pre-Phase PP.2 entry. Spec: docs/superpowers/specs/2026-05-08-sp18-reward-shape-hold-attractor-design.md Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
44e1f58b69 |
docs(sp18): copy spec + plan to feat/sp18-combined branch
Mirrors the SP17 PP.1 plan-copy pattern. Plan and spec source-of-truth live on the sister `feat/sp17-dueling-q-network` worktree; this commit copies them onto the SP18 branch so subsequent commits reference local paths. Spec: docs/superpowers/specs/2026-05-08-sp18-reward-shape-hold-attractor-design.md Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
0c57a5a31f |
docs(sp17-3.3): Phase 3 close-out — canonical HEALTH_DIAG line + pearl
Phase 3.3 finalises the SP17 dueling-Q diagnostic chain. Canonical HEALTH_DIAG line (already emitted by Phase 3.2 commit |
||
|
jgrusewski
|
b6b17d46bb |
feat(sp17-3.2): V_share + advantage_clip_bound producers + extended emit
Phase 3.2 lands the remaining two SP17 dueling-Q diagnostic producers
atomically with kernel + launcher + Rust wrapper + extended HEALTH_DIAG
emit + GPU oracle tests per `feedback_wire_everything_up`.
V_share[d] = |E[V]| / (|E[V]| + |E[A_centered, picked]|)
where picked = argmax_a Σ_z A_raw[i, a, z] (max-Q semantic — tractable
per-batch without depending on actions_history_buf which is collector-
time state stale relative to the cuBLAS forward at HEALTH_DIAG cadence).
Pearl-A bootstrap (sentinel 0.5) + α=WELFORD_ALPHA_MIN=0.4 + bilateral
[0, 1] clamp per `pearl_symmetric_clamp_audit`. 4 blocks × 256 threads.
advantage_clip_bound = p99(|A_centered|) × ADVANTAGE_CLIP_SAFETY_FACTOR=1.5
via sp4_histogram_p99 (block tree-reduce + per-warp tile binning, NO
atomicAdd per `pearl_fused_per_group_statistics_oracle`). EMA α=0.01
slow per-fold + bilateral clamp [0.1, 100.0] per
`pearl_symmetric_clamp_audit`. Pearl-A bootstrap (sentinel 1.0).
Single block × 256 threads + flat |A_centered| scratch buffer
(mapped-pinned, sized to B × Σ_d b_d × NA).
Observability-only — the actual clipping wire-up is Phase 5 follow-up.
The Phase 1 mean-zero contract (commits eabcf8d52..6f53d676f) makes
A_centered a meaningful signal; this commit observes it.
Extended HEALTH_DIAG line:
HEALTH_DIAG[N]: dueling [v_share=(d=X m=Y o=Z u=W)]
[a_var=(d=A m=B o=C u=D)] [clip=K]
GPU oracle tests on RTX 3050 Ti (all pass, 13/13 SP17 tests):
- v_share_per_branch_matches_closed_form: synthetic V=2.0 + linear A
per branch; closed-form V_share = 2/(2 + |K_d × (n_d-1)/2|);
ε=1e-4. Pearl-A bootstrap REPLACES on first launch.
- advantage_clip_bound_tracks_p99_safety: synthetic A with action-
dominant + per-(i,z) jitter (the jitter is REQUIRED — pathologically
lockstep values undercount in sp4_histogram_p99's non-atomic warp
tile binning per the kernel's documented "1/(256×32) loss for
uniformly distributed signals" qualifier; concentrated values violate
the assumption. Real |A_centered| in production is continuous, so
this is a test-data-only effect.) ε=0.20 (jitter + linear histogram
quantization).
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Phase 3.2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
1e70cd5e59 |
feat(sp17-3.1): A_var_ema per-branch producer + HEALTH_DIAG emit
Phase 3 of SP17 dueling-Q identifiability — first of three diagnostic
producers landed atomically with kernel + launcher + Rust wrapper +
HEALTH_DIAG emit + GPU oracle test per `feedback_wire_everything_up`.
Per branch d ∈ {dir, mag, ord, urg}:
Var_d = (1/(B × n_d × NA)) Σ_{i, a, z} (A[i, a, z] − mean_a A[*, z])²
Block tree-reduce (no atomicAdd, `feedback_no_atomicadd`); 4 blocks ×
256 threads. Pearl-A first-observation bootstrap (sentinel 0.0 →
REPLACE on first launch); steady-state α = WELFORD_ALPHA_MIN=0.4 per
`pearl_wiener_alpha_floor_for_nonstationary` — the structural-control
floor preserves catch-up bandwidth without storing 24 Welford
accumulator slots for a cold-path-cadence diagnostic.
Cold-path emit: single launch per HEALTH_DIAG cadence (epoch boundary)
right after `v_a_means`. New line:
HEALTH_DIAG[N]: dueling [a_var=(d=X m=Y o=Z u=W)]
The line will be extended with V_share + advantage_clip_bound in
Phase 3.2, then finalised in Phase 3.3.
GPU oracle test on RTX 3050 Ti: synthetic A constructed so each branch
d has a closed-form Var(A_centered); kernel readback matches expected
value within ε=1e-4 (f32 rounding budget for ~8×n×51 accumulator
length).
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Phase 3.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
079a06e485 |
test(sp17): asymmetric A → deterministic argmax under MIN_TEMP=0.5
Verifies that when one direction's centered advantage dominates
strongly enough, action selection picks it deterministically across
all N=1000 Philox-keyed runs.
Plan specifies "Thompson temp = 0.0 → pure argmax E[Q]", but the
kernel floors thompson_temp at MIN_TEMP=0.5 per
pearl_blend_formulas_must_have_permanent_floor — pure τ=0 isn't
ISV-accessible. With τ=0.5 the blend is q_eff = 0.5·E[Q] +
0.5·q_sample; deterministic argmax across all τ=0.5 draws requires
the E[Q] gap to exceed atom_span/2.
Construction: NA=3 atoms [-0.1, 0, +0.1] (atom_span=0.2). A_dir
puts +300 at z=2 for Long, -100 at z=2 for others (per-atom mean
zero ⇒ centering preserves shape). Long centered E[Q] ≈ +0.1, others
≈ -0.05 (gap 0.15 > 0.1). q_sample[Long] = +0.1 with prob ≈ 1
(softmax(300) ≈ delta at z=2); q_sample[other] ∈ {-0.1, 0} (zero
prob for +0.1). q_eff[Long] = 0.1; q_eff[other] ≤ -0.025. Long
strictly wins all draws.
If the centering breaks (Long no longer dominant under centered E[Q])
or τ blends a non-Long sample over Long, this test fires.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
f31a6b7ff0 |
test(sp17): symmetric A ⇒ identical per-direction E[Q]
Verifies that under uniform A (== 0 across all action × atom slots),
every direction has identical centered E[Q] regardless of V.
The plan's original wording probes this through Thompson selector
("uniform action distribution"), but the kernel's first-wins-strict-
`>` argmax over four i.i.d. Thompson samples produces a structurally
non-uniform distribution under symmetric A even with correct centering
(closed-form earlier-bias predicts ≈[44%, 26%, 18%, 11%] across
Short/Hold/Long/Flat from the tie statistics). The Thompson distribution
is V-dependent through tie statistics — NOT a centering regression.
Restated as the structural pre-Thompson property: with A=0 and
V arbitrary, centered logit = V + 0 is identical across all directions
⇒ per-direction E[Q] identical to ε=1e-5. The Thompson selector reads
these centered logits; if A=0 produced non-zero per-direction E[Q]
spread, *that* would be the centering regression — exactly what this
test catches.
Probed via compute_expected_q (reads back per-action E[Q] directly,
no Thompson noise as red herring). V-non-uniform sanity check confirms
the kernel reads V (non-zero E[Q] when V ≠ 0).
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
c2dd6917e0 |
test(sp17): A-centered invariance under V shift
Verifies the architectural contract: a uniform additive shift to V across atoms cannot leak into the post-centering distribution. The plan specifies reading A_centered directly, but A_centered is a register-local quantity inside compute_expected_q; this test restates the property as the equivalent behavioral assertion that adding a uniform constant to V leaves every per-action E[Q] identical (softmax translation invariance). A regression that accidentally reduced over (V + A) instead of A alone would shift the per-atom mean by V_SHIFT and corrupt the centered logits; the per-action E[Q] would diverge by O(1), failing the ε=1e-4 assertion. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
10fafe8e60 |
test(sp17): V-invariance behavioral test
Verifies the architectural contract: V depends only on state, not on the specific advantage tensor. Two A tensors with same per-atom mean produce different post-centering shapes ⇒ different per-action E[Q] (centering is shape-sensitive), but the V-only diagnostic readout is identical across the two runs (V cannot leak in via the per-atom mean reduction). This is the structural property that makes dueling work — without it, the model conflates "state value" with "action value" and gradient updates corrupt V via raw-A noise. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
6f53d676fb |
docs(sp17): annotate c51_loss/c51_grad SP17 compliance (Commit E)
Audit-only commit per user design call DD9. The plan's Task 1.4
incorrectly claimed c51_loss_batched + c51_grad_kernel needed migration;
both kernels have been mean-zero centered since commit
|
||
|
jgrusewski
|
ffa6fda868 |
feat(sp17): centered A in barrier/ib_gradient_direction (Commit D)
User design call DD11: migrate the two aux-CQL gradient kernels in
c51_loss_kernel.cu to the SP17 mean-zero advantage contract. The
pre-SP17 comment "skip advantage-mean centering — small epsilon vs
correctness simplicity" at barrier_gradient_direction:1212 is REMOVED;
its empirical-without-verification rationale doesn't hold under the
SP17 contract that compute_expected_q + c51_loss + c51_grad +
mag_concat_qdir + Thompson + quantile_q_select already enforce.
barrier_gradient_direction:
- Per-thread `a_mean_per_atom[NUM_ATOMS_MAX]` reduction over b0_size
direction actions.
- Forward Q-value computation reads `v_row[z] + (adv_a[z] - mean[z])`.
- Backward gradient recompute uses the same centered logits in the
per-action softmax probability accumulation. The Jacobian's symmetric
-1/b0_size per-atom offset cancels in the barrier's relative-push
gradient direction (max up, 2nd down — both targets see same offset).
- Stale "skip centering" comment deleted; replaced with SP17 explanation.
ib_gradient_direction:
- Same per-atom mean reduction at function entry.
- Forward Q computation + backward dq_dlogit recompute both flow through
centered probabilities. Variance var_q is invariant under common
per-atom shifts (math: shift cancels in (Q(a) - mean_q)^2), so var_q
numerics are bit-equivalent — but the gradient flows through the
centered probability `p(z|a)` for consistency with c51_grad backward.
NUM_ATOMS_MAX=128 ceiling guard added to both kernels (mirror of
experience_kernels.cu); early-exit `if (num_atoms > NUM_ATOMS_MAX) return`
matches the pattern already used by these kernels for unrelated
zero-op guards.
GPU oracle test (RTX 3050 Ti, 6/6 PASS):
barrier_gradient_direction_uses_centered_advantage — A=0, V=0 ⇒
centered logits all zero ⇒ uniform softmax ⇒ E[Q]=0 across actions ⇒
q_gap=0 ⇒ barrier fires at min_req=0.05 ⇒ asserts total |grad| > 1e-6.
Regression detector: any centering breakage produces non-finite or
zero gradients ⇒ test fails loudly. ib_gradient_direction shares the
identical per-atom mean reduction pattern so the same test covers
both kernels structurally.
Verification:
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 6/6 PASS
⚠ INTERIM STATE: c51_loss_batched + c51_grad_kernel already-centered
sites still need Commit E annotation pass to mark the existing Jacobian
+ per-d=1 magnitude-std as SP17-compliant.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
3107edb8f7 |
feat(sp17): Thompson V-wire-in (Commit C)
User design call DD10 option (b): Thompson direction selector now reads
softmax(V[z] + (A[a, z] - mean_a A[*, z])) instead of pre-SP17
softmax(A[a, z]). Without V wired in, action selection responded to a
DIFFERENT distribution than compute_expected_q's E[Q] (the Bellman-target
ranking) — the very pathology SP17 is fixing at the action layer.
Architectural change closes the contract gap atomically across:
Kernel signature (`experience_action_select`):
- Added `const float* __restrict__ v_logits_dir` after `b_logits_dir`.
NULL is invalid (no fallback) — kernel hard-requires V.
- New per-thread `a_mean_per_atom_dir[THOMPSON_MAX_ATOMS]` reduction
computes mean A across the 4 direction actions per atom z (sample-local
register, no atomicAdd).
- Pass 1 (E[Q] for temperature blend + conviction): builds per-action
`combined_logits_d[z] = V[z] + (A[a,z] - mean_a A[*,z])` and feeds
`softmax_c51_inline` instead of raw `b_logits_dir + d * n_atoms`.
- Pass 2 (Thompson sample): same combined-logits rebuild per direction.
- `softmax_c51_inline` device helper UNCHANGED — keeping centering at
caller maintains a narrower contract; thompson_test_kernel and other
potential callers stay unaffected.
- THOMPSON_MAX_ATOMS=128 ceiling preserved; __trap() on overflow.
QValueProvider trait extension:
- `compute_q_and_b_logits_to` now takes `v_logits_out_ptr: u64` and
DtoD-copies `on_v_logits_buf` per sub-iteration (atomic per
feedback_no_partial_refactor — every consumer migrates in lockstep).
Trainer + evaluator wire-up:
- `gpu_dqn_trainer.rs::on_v_logits_buf_ptr() -> u64` (new pub fn, mirror
of existing `on_b_logits_buf_ptr`).
- `gpu_backtest_evaluator.rs::chunked_v_logits_buf` field allocated
[chunk_n * NA + 32*3] (cuBLAS tail-safety pad).
- Both call sites (collector + evaluator) pass v_logits arg in launch.
GPU oracle test (RTX 3050 Ti, 5/5 PASS):
thompson_direction_select_reads_v_logits — runs production cubin
twice on identical A logits with V=[0,0,0] vs V=[10,0,0]; asserts
q_gap_v_dominant < 50% of q_gap_v_zero. If V is being IGNORED
(regression), both runs produce IDENTICAL q_gaps and the test fails
with a clear message. Uses MappedF32Buffer / MappedI32Buffer per
feedback_no_htod_htoh_only_mapped_pinned.
Note on the plan's "raw argmax = Hold but centered argmax = Long" test:
Mathematical analysis shows softmax-with-constant-shift preserves
action ordering (mean subtraction adds the same per-atom constant to
every action's logits), so the plan's specific assertion isn't
algebraically constructable with simple A/V. The replacement test
(V-dependence of q_gap) is more sensitive — it fails on the actual
regression case (V ignored ⇒ identical q_gaps) the plan was trying
to detect.
Verification:
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 5/5 PASS
⚠ INTERIM STATE: aux-CQL barrier_gradient_direction +
ib_gradient_direction still read raw advantage. Commit D closes them;
Commit E annotates the pre-SP17 c51_loss/c51_grad already-centered sites.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4802879494 |
feat(sp17): centered A in mag_concat_qdir (Commit B)
Migrates the magnitude-branch's direction-Q conditioning input to the
SP17 mean-zero contract. mag_concat_qdir builds a per-(sample, action)
softmax over V[z] + A_dir[a, z] across THREE inner passes (max,
sum_exp, E[Q]); all three now read `dir_logits_b[..] - a_mean_per_atom[z]`.
Without this migration the magnitude trunk-input would see raw E[Q_dir]
while c51_loss/c51_grad backward consumes centered A — mixed contract
ruled out by `feedback_no_partial_refactor`.
The Plan-4 q_rms adaptive scale (ISV[96]) is preserved unchanged;
centering changes the per-action E[Q_dir] values that feed it but the
scale formula itself is structurally orthogonal. Backward into the
direction logits flows through c51_grad (already mean-zero) so no bw
change is needed in this kernel.
GPU oracle test: B=1, SH2=2, NA=3, B0=4 with the same synthetic where
mean_a = [0.75, 0, 0.75] is non-zero. Asserts:
(a) h_s2[0..SH2] copied verbatim into concat prefix
(b) tail slots = centered_E[Q_dir] × (1.0 / q_rms) to ε=1e-5
(c) tail[0] < 0 / tail[3] > 0 sign asymmetry (regression detector)
Verification (RTX 3050 Ti):
cargo check --workspace → clean
cargo test sp17_dueling_oracle_tests --features cuda
-- --ignored → 4/4 PASS
⚠ INTERIM STATE: Thompson direction-select + aux-CQL barrier/ib still
read raw advantage. Commits C-D close them; Commit E annotates the two
already-centered c51_loss/c51_grad pre-SP17 sites.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9b6e94854f |
feat(sp17): centered A in quantile_q_select (Commit A)
Migrates `quantile_q_select` (uncertainty-driven action selection from
C51 atom CDFs) to the SP17 mean-zero advantage contract. The kernel
builds a per-(sample, branch, action) softmax over `V[z] + A[a, z]`
across THREE inner passes (max, sum_exp, CDF); all three now read
`adv_a[z] - a_mean_per_atom[z]`.
Plan flagged this as a hidden 6th consumer. Task 1.4/1.5 as authored
covered only c51_loss + c51_grad + mag_concat_qdir + Thompson; the
post-Task-1.2 audit found `quantile_q_select` reads raw advantage at
lines 5704/5709/5720 across all 4 branches and was missed entirely.
Sample-local register reduction (`float a_mean_per_atom[NUM_ATOMS_MAX]`,
NA_MAX=128 mirroring compute_expected_q); no atomicAdd, no shared mem,
no cross-thread sync per `feedback_no_atomicadd`. Device-side __trap()
on num_atoms overflow per `feedback_no_quickfixes`.
GPU oracle test: N=1, NA=3, B0=4 with V=[0,0,0] and A_raw chosen so the
per-atom mean is [0.75, 0, 0.75]. Test runs the production cubin with
iqn_readiness=0 and util_ema=1.0 and asserts each per-action q90 matches
the CPU oracle to ε=1e-5. Two structural-asymmetry assertions fail
loudly if centering regresses (action 0 q90 ≤ 0, action 3 q90 ≥ 0).
Mapped-pinned per `feedback_no_htod_htoh_only_mapped_pinned`.
Verification (RTX 3050 Ti):
cargo check --workspace → clean
cargo test -p ml --test sp17_dueling_oracle_tests --features cuda
-- --ignored → 3/3 PASS
⚠ INTERIM STATE: mag_concat_qdir + Thompson + aux-CQL barrier/ib still
read raw advantage. Commits B-D close them; Commit E annotates the two
already-centered c51_loss/c51_grad pre-SP17 sites. No L40S dispatch
escapes feat/sp17-dueling until every consumer is migrated per
`feedback_no_partial_refactor`.
Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
eabcf8d529 |
feat(sp17): mean-zero identifiability in compute_expected_q
Inserts per-atom mean-A reduction inside the per-branch outer loop: Q[a, z] = V[z] + (A[a, z] - mean_a A[*, z]) The post-centering atom-level identifiability holds: Σ_a A_centered[a, z] = 0 for every (sample, branch, atom). Per Wang et al. 2016 Dueling Networks: mean-zero is smoother and more stable than max-zero. Atom-level subtraction (not expectation-level) preserves distributional shape per action. Sample-local register reduction; no atomicAdd, no shared memory, no cross-thread sync. NUM_ATOMS_MAX=128 mirrors existing THOMPSON_MAX_ATOMS; device __trap() on overflow. ⚠ INTERIM STATE: c51_loss_kernel + c51_grad_kernel + mag_concat_qdir still read RAW (un-centered) advantage logits. Phase 1 Tasks 1.3-1.5 migrate them in lockstep within this branch BEFORE any L40S dispatch. A partially-migrated state is forbidden per feedback_no_partial_refactor. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7b13324bcb |
test(sp17): CPU oracle for mean-zero centered Q computation
Locks the mean-zero identifiability math contract independent of GPU implementation. Subsequent Phase 1 GPU oracle tests compare kernel output against this contract bit-for-bit. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
509035dea6 |
docs(sp17): copy plan file to feat/sp17-dueling branch
Plan was authored on sister worktree (sp15-phase1-honest-numbers) by the SP17 planner agent; copying it to feat/sp17-dueling so future task references in commit messages and audit-doc entries resolve from this branch. No content change vs sister-worktree copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
6178f68559 |
fixup(sp17-pp.2): lock-test producer constants for symmetry
Per code-quality review of
|
||
|
jgrusewski
|
cc4746b48d |
plan(sp17): HEALTH_DIAG v_a_means baseline (PRE-CENTERING)
Adds per-epoch readout of the V/A breakdown so we can observe the pre-SP17 baseline distribution before the identifiability projection lands. Also instruments the kill criterion for Phase 0: HEALTH_DIAG[N]: v_a_means [v=X a_dir=Y a_mag=... a_ord=... a_urg=...] If train-multi-seed-b5gmp's Q(Flat) over-attribution is V-driven, V should be elevated (~0.4+) while a_dir is small. If V is balanced and A is doing the work, dueling cannot help and we abort SP17. Block-tree-reduce kernel \`v_a_means_diag_kernel\` (no atomicAdd per \`feedback_no_atomicadd\`); MappedF32Buffer per \`feedback_no_htod_htoh_only_mapped_pinned\`. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a225926e5f |
plan(sp17): allocate ISV slots [474..483) for dueling-Q diagnostics
9 slots: A_var_ema × 4 branches (per-branch advantage variance EMA) + V_share × 4 branches (V/(V+A) magnitude share) + adaptive advantage_clip_bound. All Pearl-A first-observation bootstrap with fold-reset registry entries + dispatch arms in trainer/training_loop.rs::reset_named_state. Bump ISV_TOTAL_DIM 474 → 483 + layout fingerprint seed. Per `feedback_isv_for_adaptive_bounds`: advantage_clip_bound is producer-tracked from p99(|A_centered|) × 1.5 safety factor, never hardcoded. Bounds [0.1, 100.0] are Category-1 dimensional safety floors. No consumer change in this commit (additive infrastructure). Phase 1 mean-centering producer (atomic across compute_expected_q + c51_loss + c51_grad + mag_concat_qdir) lands in the next 4 commits. Plan: docs/superpowers/plans/2026-05-08-sp17-dueling-q-network.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
25ff1d4195 |
fix(sp16): floor Wiener-α at 0.4 for non-stationary control loops
Wiener-optimal α minimizes MSE for stationary stochastic signals.
The min_hold_temp / hold_cost_scale loops track a target driven by
the adapting policy itself — non-stationary by construction.
Empirical evidence: train-multi-seed-b5gmp sha
|
||
|
jgrusewski
|
641aa0dfde |
fix(sp16-t3): Wiener-optimal adaptive α per pearl — hold_cost_scale + min_hold_temperature
Per train-multi-seed-hjzss validation: SP16 T1+T2 chain was structurally
landed but BEHAVIORALLY INERT in 5-epoch smoke (bit-identical to pfh9n
baseline through epoch 3). Root cause: hardcoded `alpha = 0.05f` in both
producer kernels violates feedback_isv_for_adaptive_bounds AND prevents
convergence in short runs (~60 epochs needed from cold start).
Fix per pearl_wiener_optimal_adaptive_alpha:
α = diff_var / (diff_var + sample_var + ε)
Where sample_var = running variance of target signal (Welford accumulator)
and diff_var = running variance of consecutive one-step differences.
Cold-start: target jumps 1.0 → 6.4 → 7.0 → high diff_var → α ≈ 0.6+
→ near-bootstrap responsiveness in epochs 1-3
Steady-state: signal stabilizes → diff_var drops → α decays naturally
→ smoothing emerges without hardcoded constant
Adds 12 new ISV slots (6 per producer):
- HCS_TARGET_MEAN/M2, HCS_DIFF_MEAN/M2, HCS_PREV_TARGET, HCS_SAMPLE_COUNT
- MHT_TARGET_MEAN/M2, MHT_DIFF_MEAN/M2, MHT_PREV_TARGET, MHT_SAMPLE_COUNT
ISV_TOTAL_DIM 462 → 474.
Both kernels migrated atomically. Pearl-A bootstrap preserved (sentinel
on prev_blended triggers REPLACE; cold-start α=1.0 when N<3 samples).
Defensive bounds [WELFORD_ALPHA_MIN=0.01, WELFORD_ALPHA_MAX=0.95] on the
Wiener-derived α to guard against denormal/underflow corner cases.
HEALTH_DIAG[N] emit extended with `alpha=...` and `sample_count=...` for
direct trajectory observation in validation smoke.
Behavioral tests verify:
- α high during signal jumps (>0.3 at epoch 3 post-cold-start)
- α low in steady state (mean tail α<0.4 under converging signal)
- Pearl-A bootstrap fires on first observation (Welford state advances
regardless of REPLACE branch)
- α stays within [WELFORD_ALPHA_MIN, WELFORD_ALPHA_MAX] over 50 epochs
(post-cold-start; cold-start α=1.0 by design)
- No 0.05f hardcoded literal remains in blend math (regression-locked
via host-only string scan)
5 GPU + host tests pass: sp16_phase3_alpha_high_during_signal_jump,
alpha_low_in_steady_state, pearl_a_bootstrap_first_obs,
alpha_naturally_bounded, no_hardcoded_alpha. sp14 + sp15 oracle suites
unchanged (34 GPU tests + 4 host tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
6dce4ac28b |
docs(audit): note ensure-binary cache key bug for cross-arch resubmits
Discovered during SP16 validation: ensure-binary caches by SHA only,
so cross-pool resubmissions (L40S → H100) cache-hit the wrong-arch
binary and fail with CUDA_ERROR_NO_BINARY_FOR_GPU on rmsnorm.
Build.rs rerun-if-env-changed fix (
|
||
|
jgrusewski
|
1a3bcf97b8 |
feat(sp16-p2): adaptive Hold cost scale via ISV[461]
Per train-multi-seed-pfh9n post-mortem: observed_hold_rate climbed 0.25 → 0.52 across training while cost penalty (~0.006) was 100× smaller than per-bar reward magnitudes (popart=0.97, cf=0.65). Hold action was effectively free, allowing Q(Hold) to dominate via structural low-variance bias. Fix: scale Hold cost adaptively. ISV[HOLD_COST_SCALE_INDEX=461] tracks: scale = clamp(1.0 + 24.0 × max(0, observed - target) / max(target, 0.01), 1.0, 25.0) Effective cost at 100% overrun (observed=2× target): 0.006 × 25 = 0.15, competitive with per-bar reward magnitudes (~0.01-0.1). At/below target: scale = 1.0 (no extra penalty). Pearl-A bootstrap + Welford slow EMA (α=0.05). Mirrors T1's MIN_HOLD_TEMPERATURE pattern (same input signals: ISV[382] observed, ISV[381] target). Producer: hold_cost_scale_update_kernel.cu — single-thread cold-path, per-epoch boundary, AFTER MIN_HOLD_TEMPERATURE in training_loop.rs. Consumer migration (atomic per feedback_no_partial_refactor): 3 sites in experience_kernels.cu — segment_complete branch (line ~3089), per-bar positioned-Hold branch (line ~3553), per-bar flat-Hold branch (line ~3617). Cold-start fallback: scale=1.0 when slot ≤ 0 or out-of-bounds (bit-identical pre-Phase-2 cost magnitude). ISV_TOTAL_DIM: 461 → 462. Behavioral tests (5/5 PASS on RTX 3050): - sp16_phase2_hold_cost_scale_climbs_with_overrun - sp16_phase2_hold_cost_scale_at_target_is_one - sp16_phase2_hold_cost_scale_under_target_is_one - sp16_phase2_hold_cost_scale_bounds_clamp - sp16_phase2_hold_cost_scale_pearl_a_bootstrap Regression: SP14 oracle suite 30/30 PASS, SP15 phase 1 oracle suite 36/36 PASS. Instrumentation: HEALTH_DIAG[N]: hold_cost_scale_diag obs/tgt/norm/scale. Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2c469af2f7 |
fixup(sp16-p1): correct stale build.rs comment + remove tautological test assertion
Code-quality review of
|
||
|
jgrusewski
|
0426ce8887 |
fix(sp16-p1): MIN_HOLD_TEMPERATURE signal swap to hold-rate overrun + slot 330 non-bug closure
Per train-multi-seed-pfh9n post-mortem follow-up: slot 460 stuck at 50 in Fold 1 was NOT a launch-lifecycle bug. Producer fires per-epoch but kernel had early-return guard on AUX_DIR_ACC_SHORT_EMA (slot 373) at sentinel 0.5. Slot 373 reset on fold boundary; aux dir-acc EMA either didn't fire or settled within ε of 0.5 → kernel kept early-returning. Fix: drop slot 373 dependency entirely. Drive temperature from observed hold-rate vs target overrun: overrun = max(0, observed_hold_rate - target_hold_rate) overrun_norm = clamp(overrun / max(target, 0.01), 0, 1) new_temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × overrun_norm blended_temp = Welford EMA α=0.05 with Pearl-A bootstrap When over-holding: temp HIGH → exit ramp permissive (matches design intent). When at/under target: temp LOW → exit penalty strict. Survives fold reset: hold-rate measurement starts fresh with real data immediately, no chained-input-sentinel masking. Slot 330 (KELLY_WARMUP_FLOOR) investigated and confirmed NON-BUG: producer behaves correctly per pearl_kelly_cap_signal_driven_floors cross-fold- persistence. floor=0 post-warmup is correct steady state. Behavioral tests: - sp16_phase1_min_hold_temp_climbs_with_hold_overrun - sp16_phase1_min_hold_temp_strict_when_at_target - sp16_phase1_min_hold_temp_strict_when_under_target - sp16_phase1_min_hold_temp_no_longer_reads_slot_373 Instrumentation: HEALTH_DIAG[N]: min_hold_temp_diag obs/target/overrun/temp. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
fb24614f07 |
feat(sp16-p0): Q-by-action HEALTH_DIAG diagnostic for Hold-bias observation
Per train-multi-seed-pfh9n post-mortem: structural-Q-bias hypothesis (Adam's m/sqrt(v) prefers low-variance Hold over noisy direction Q-targets) needs direct per-action Q observations to verify or refute. WR plateaued at ~0.46 across both folds while Hold% climbed 15% → 52% and Q-mean climbed 0.19 → 0.41 — but per-action Q was unobservable. Adds HEALTH_DIAG emit each epoch: HEALTH_DIAG[N]: q_by_action [hold=X long=Y short=Z flat=W] Reads via host-side averaging of `q_out_buf [B, total_actions]` direction-branch columns [0..b0=4]. No new kernel needed — q_out_buf is already populated row-major by compute_expected_q. Modeled directly on the existing Task 0.3 magnitude-bucket diagnostic (same dtoh-and- average cold path). Action-index ordering canonical, see state_layout.cuh:123-126: DIR_SHORT=0, DIR_HOLD=1, DIR_LONG=2, DIR_FLAT=3. Emit slot order is [hold, long, short, flat] (Hold first because the hypothesis is about Hold's ascent dominating direction Q-magnitudes). New surface: - gpu_dqn_trainer.rs: q_dir_means_cached field + update_q_dir_means_cached method + q_direction_action_means accessor + free static helper compute_q_dir_means_from_host_buf (factored for testability). - fused_training.rs: FusedTrainingCtx wrappers parallel to q_mag pair. - training_loop.rs: emit block adjacent to q_var_per_branch. Behavioral test: sp16_phase0_q_by_action_diagnostic_reads_four_action_means seeds known per-column constants, asserts canonical-dir-idx → emit-slot mapping at 1e-3 tolerance, and includes sentinel-leak guard (non-direction columns at 99.0 fail any wrong index→slot mapping with margin >12). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
98a81981ac |
fix(build-infra): cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP across 6 crates
Per pearl_build_rs_rerun_if_env_changed, every std::env::var() must be paired with cargo:rerun-if-env-changed. Task #321 fixed crates/ml/build.rs (commit |
||
|
jgrusewski
|
0371d6a768 |
docs(no-cpu-strict-sites-2-9): mark sites #2 and #9 CLOSED in audit grid
Per Class A audit grid no-CPU-compute sites 2+9 verification (audit doc lines 3825/3832): the code-vs-grid disagreement was the GRID being stale, not the code. Both sites were already resolved in 2026-05-01 close-out follow-up commits but the grid table verdict columns were never updated. Site 2 (compute_adaptive_tau): ALREADY-CLOSED Helper deleted in `a385c1d2b` (chore(sp4): delete compute_adaptive_tau orphan helper per feedback_wire_everything_up). Verified zero hits across `crates/`, `services/`, `bin/` for `compute_adaptive_tau` and `q_div_ema`. Grid row updated: DEFER → CLOSED, with closure SHA recorded. Site 9 (calibrate_homeostatic_targets): ALREADY-CLOSED Migrated to GPU in `6d0ac7beb` (fix(sp4): GPU-port calibrate_homeostatic_targets per feedback_no_cpu_compute_strict). Verified `calibrate_homeostatic_kernel.cu` exists at `crates/ml/src/cuda_pipeline/calibrate_homeostatic_kernel.cu`, Rust launcher at `gpu_dqn_trainer.rs:7958-7994` calls `launch_calibrate_homeostatic` (single-block, 6 threads, mapped-pinned EMA writeback with `__threadfence_system()`); host-side EMA loop body is gone. Grid row updated: NEW DISCOVERY → CLOSED, with closure SHA recorded. Sweep summary section updated: migrations completed 3→4, exceptions 5→4, deferrals 2→0. Final-grep claim of "1 match remaining" amended to "0 matches remaining post-close-out". The free-text "Sweep close-out" subsection at lines 3863-3871 was already correct and complete; only the GRID columns and summary counts were stale. Audit-doc errors found: 2 sites flagged as open (DEFER + NEW DISCOVERY) when both were already closed in commits dated 2026-05-01. The narrative close-out write-up beneath the grid was correct — only the grid table verdict columns and summary counts had not been kept in sync. No code changes; doc-only fix. Per feedback_trust_code_not_docs: code wins. Grid now matches reality. Cumulative WR-plateau fix series (commit M): - Class C, Class A batches 4a/4b, Class B #1, Class B #2 already landed - no-CPU-strict sites 2+9 audit-doc cleanup (this commit; NO_OP for code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7088999f9a |
fix(class-b-2): PER priority double-exponent corruption
Per Class B audit, per_update_pa was applying alpha twice: - priorities[idx] = |td|^α + ε ← α applied once - priorities_pa[idx] = priorities^α ← α applied AGAIN, gives |td|^(α²) Sum-tree sampler reads priorities_pa (verified at gpu_replay_buffer.rs:778 for per_prefix_scan + line 799 for per_sample), so effective sampling exponent was α² = 0.36 instead of α = 0.6 (default). High-TD events were sampled only ~2.5× more than low-TD events instead of ~10× — PER's prioritization power was attenuated by ~4×, washing out the signal from sparse trade-close events (3% of buffer). Fix (Option B — preserves variable semantics): the contract documented at gpu_replay_buffer.rs:1285 is `priorities_pa[i] = priorities[i]^alpha`. Store raw |td|+ε in `priorities` (one source of truth) and compute α once for `priorities_pa`. Same pattern applied to pow_alpha_diverse_f32 in replay_buffer_kernels.cu (the health<0.8 fallback path) which had the identical double-α bug. per_insert_pa NOT changed — it was already correct (priorities=effective, priorities_pa=effective^α, no double application). priority_update_f32 NOT changed — single-buffer kernel, verified UNUSED (no Rust caller); harmless idle code, deletion deferred. Behavioral test: gpu_replay_buffer::tests:: test_per_priority_single_exponent_no_double_alpha (GPU-gated #[ignore]) inserts 16 slots, fires update_priorities_gpu with TD errors spanning [0.001, 100.0], asserts priorities_pa ratio matches (100/0.001)^0.6 ≈ 1995× within 5%. Pre-fix would yield (100/0.001)^(0.6²) ≈ 100× — the 20× miss makes regression instant-detect. Cumulative WR-plateau fix series (commit 11): - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
658fec4939 |
fix(class-b-1): clear PER replay buffer at fold boundary (months-long WR plateau)
Per Class B audit, the PER ring buffer was never cleared from the
single source of truth at `DQN::reset_for_fold`. fold N+1 was
training on a 50/50 mix of stale fold N-1 experiences + fresh
fold N — averaging behavior locked learning to a "policy-agnostic"
middle ground producing 46-48% WR across SP3-SP15.
Fix: extend `DQN::reset_for_fold` to call `clear_replay_buffer()`
internally. Single source of truth — every fold reset clears the
buffer atomically. Removed the now-redundant explicit
`self.clear_replay_buffer().await?` call at the top of
`DQNTrainer::reset_for_fold` per `feedback_no_legacy_aliases`.
Tradeoff: loses cross-fold experience transfer (fold N+1 starts
from scratch on fold N data only). User-approved Option (a) per
Class B audit 2026-05-08.
Signature changes (atomic per `feedback_no_partial_refactor`):
- `DQN::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNAgentType::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNTrainer::reset_for_fold` (async): unchanged signature; the
agent.reset_for_fold call now propagates the Result.
`GpuReplayBuffer::clear` extended: it already zeroed
`priorities_pa` + `prefix_sum` but left `priorities` unzeroed.
Sampler reads `priorities_pa` (not `priorities`) so this was not
a sampling-contamination vector, but is leftover state that
`priorities_slice()` consumers would inherit. Fixed for
completeness alongside the fold-boundary-clear lift.
Behavioral test added: `test_clear_purges_priorities_and_blocks_sampling`
inserts 64 transitions, samples once to populate prefix_sum +
sample buffers, steps the β counter, then clears and asserts:
- len() == 0, is_empty() == true, can_sample(1) == false
- write_cursor() == 0, current_step == 0
- priorities[..n] all zero (DtoH readback)
- priorities_pa[..n] all zero (DtoH readback)
No mocks; exercises actual GPU memset_zeros + DtoH path.
GPU-gated `#[ignore]` — runs at deploy time.
Cumulative WR-plateau fix series (commit 10):
- Class C bug 1 + P0-B (
|
||
|
jgrusewski
|
9fb980da2b |
fixup(class-a-audit-batch-4b): invert MIN_HOLD_TEMPERATURE dir_acc → temp mapping
Post-commit kernel-formula audit of `compute_min_hold_penalty` at
trade_physics.cuh:567-577 showed the original `temp = TEMP_MIN +
(TEMP_MAX - TEMP_MIN) × skill` mapping was BACKWARDS for the
WR-plateau scenario this 8-commit chain targets.
Formula recap:
soft_factor = deficit / (deficit + T)
HIGH T → soft_factor → 0 → forgiving (no exit penalty)
LOW T → soft_factor → 1 → sharp (max exit penalty)
The deleted schedule's `START=50 → END=5` therefore meant
"permissive early, strict late" (classic explore-then-exploit), not
"permissive when committed, strict when uncertain" as the
substituted mapping assumed.
WR-plateau scenario: dir_acc ≈ 0.46-0.48 (model committed but
wrong) — the model needs a permissive (HIGH T) exit ramp to bail
out of bad committed directions, not maximum exit penalty (LOW T)
locking it into them.
Inverted the mapping using `confusion = 1 - skill`:
- dir_acc ≈ 0.5 (random / plateau) → confusion=1 → temp=50
(permissive, plateau exit ramp)
- dir_acc → 1.0 (saturated skill) → confusion=0 → temp=5
(strict, force informed commitment)
This preserves the deleted schedule's epoch-0 anchor (T=50
cold-start) while reacting to actual realized skill instead of an
epoch-time anchor.
Files touched (atomic per `feedback_no_partial_refactor`):
- min_hold_temperature_update_kernel.cu — formula + header doctring.
- sp14_isv_slots.rs — slot-doc comment expanded with new mapping
+ fixup-rationale paragraph.
- gpu_aux_trunk.rs — `MinHoldTemperatureUpdateOps` docstring.
- gpu_dqn_trainer.rs — cubin docstring + ISV_TOTAL_DIM doc-comment.
- tests/sp14_oracle_tests.rs — Tests 3 + 4 flipped (high dir_acc
now expects temp=TEMP_MIN; low dir_acc now expects TEMP_MAX);
Tests 1, 2, 5 numerically unchanged (sentinel guard fires before
the mapping; midpoint dir_acc=0.75 has skill=confusion=0.5 so
pre/post-fixup numerics match).
- docs/dqn-wire-up-audit.md — Item 4 entry rewritten with the
fixup rationale + mapping table updated.
Verification:
cargo check -p ml --tests --all-targets PASS
cargo test sp14_audit_4b (lib) 4/4 PASS (slot layout
locks unchanged)
GPU oracle re-run deferred to next L40S smoke (kernel was rebuilt
in-place so cubin contents change; the 5 oracles encode the new
mapping and will exercise the new cubin).
Per `pearl_first_observation_bootstrap.md` (sentinel→target
replace; sentinel anchors unchanged) +
`pearl_symmetric_clamp_audit.md` (bilateral clamp on target_temp
preserved) + `feedback_no_partial_refactor.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
0b9ea77dc4 |
fix(class-a-audit-batch-4b): plan_threshold floor adaptive + MIN_HOLD_TEMPERATURE ISV-driven
Per Class A audit-fix Batch 4-B (final 2 of 4 deferred items from
P1-wiring/P1-producer). Completes the 8-commit WR-plateau intervention
chain. Validation deferred to next L40S smoke.
Item 3: plan_threshold adaptive floor (Design Y - inline producer)
- NEW slot PLAN_THRESHOLD_FLOOR_ADAPTIVE_INDEX=459
- Writes slow-EMA shadow of `0.5 * readiness_ema` from inside the
existing update kernel (no new file, no new launch).
- Pearl-A first-observation bootstrap (sentinel 0.1 matches pre-fix
hardcoded value for bit-identical cold-start) + Welford alpha=0.005
slow EMA.
- Bilateral clamp [0.05, 0.50] (probability units) per
pearl_symmetric_clamp_audit.
- Consumer reads isv[459] as the floor in the same launch's final
fmaxf; cold-start sentinel REPLACES with threshold_target so the
pre-fix `fmaxf(0.1, 0.5*ema)` semantic is preserved bit-identical
for any readiness EMA above 0.20.
Item 4: MIN_HOLD_TEMPERATURE -> ISV-driven (driving signal: dir_acc skill)
- NEW slot MIN_HOLD_TEMPERATURE_ADAPTIVE_INDEX=460
- NEW kernel min_hold_temperature_update_kernel.cu (single-thread
cold-path, per-epoch boundary launch).
- Driving signal: dir_acc skill = clamp((short_ema - 0.5)/0.5, 0, 1)
from ISV[AUX_DIR_ACC_SHORT_EMA_INDEX=373]. When committing skillfully
(high dir_acc) -> temp HIGH (permissive). When at random baseline
(~0.5) -> temp LOW (sharp commitment pressure). Substituted for
the audit-spec's `dir_entropy_deficit` because no dir_entropy ISV
slot exists - dir_acc skill is the closest semantically-equivalent
signal that preserves the spec intent.
- Pearl-A bootstrap (sentinel 50.0 matches the deleted
MIN_HOLD_TEMPERATURE_START=50 anchor) + alpha=0.05 mid-cadence EMA.
- Bounds [5, 50] (matches the deleted schedule range).
- Decouples temperature from epoch number - the old schedule pinned
LOW temp (sharp) at end of training, exactly when a WR-plateaued
model needed forgiveness to escape.
- DELETED: state_layout.cuh::MIN_HOLD_TEMPERATURE_{START, END, DECAY}
#defines + training_loop.rs::min_hold_temperature_for_epoch helper
function (kept docstring tombstone explaining the deletion). Both
call sites migrated to the new ISV reader. Per
feedback_no_legacy_aliases + feedback_no_partial_refactor.
ISV_TOTAL_DIM: 459 -> 461.
Cumulative WR-plateau fix series (final commit, #8):
-
|
||
|
jgrusewski
|
7e9a8f6ef1 |
fix(class-a-audit-batch-4a): DD saturation floor adaptive + legacy DD path Case A
Per Class A audit-fix Batch 4-A (deferred from P1-wiring/P1-producer due to
audit-doc errors). Fixes 2 of 4 deferred items; Batch 4-B handles plan_threshold
floor + MIN_HOLD_TEMPERATURE in a separate commit.
Item 1: DD saturation floor (the upper end of the DD ramp at trade_physics.cuh:154
in apply_margin_cap, NOT line 548 as the audit doc claimed — that line is a
magnitude action constant; the actual saturation floor lives in apply_margin_cap)
- NEW slot DD_SATURATION_FLOOR_ADAPTIVE_INDEX=458
- Producer dd_saturation_floor_update_kernel.cu — p75(per-env DD_MAX) × 1.5
via Welford `mean + Z_75 × sigma` estimator with `max(p75, mean)` robustness
guard, mirrors P0-A REWARD_POS_CAP producer pattern (Pearl-A bootstrap +
Welford α=0.01)
- Cold-start fallback: 0.25f (DD_SATURATION_FLOOR_DEFAULT in state_layout.cuh)
- Bounds: [0.10, 0.50] (Category-1 dimensional safety)
- Distinct from SP15_DD_THRESHOLD_INDEX=421 (the SP15 quadratic DD-penalty
*trigger* threshold, a *lower* bound; this slot is the *upper* end of the
linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac/floor))
- Threaded `isv_signals_ptr` into `apply_margin_cap` with NULL-tolerant
cold-start fallback to DD_SATURATION_FLOOR_DEFAULT
- 4 oracle tests (Pearl-A bootstrap, no-DD guard, bounds clamp, Welford EMA)
Item 2: Legacy compute_drawdown_penalty path → Case A (DELETED)
- Decision rationale: SP15's quadratic asymmetric DD penalty
(compute_sp15_final_reward_kernel.cu:154 via sp15_dd_penalty helper) runs
unconditionally as a post-modifier on the SP11-composed reward with
ISV-driven λ_dd (slot 420) and DD threshold (slot 421). Layering the legacy
linear-ramp penalty inside the SP11 composer on top of the SP15 quadratic
creates double-counting of DD shaping — exactly the code-smell the Class A
audit was designed to eliminate. Per `feedback_no_legacy_aliases.md` and
`feedback_no_partial_refactor.md`.
- Atomic deletion across:
- `compute_drawdown_penalty` device function (trade_physics.cuh)
- Single call site at experience_kernels.cu:3822
- `dd_threshold` and `w_dd` kernel arguments
- `w_dd` Rust config field (gpu_experience_collector.rs +
trainers/dqn/config.rs DQNHyperparameters)
- `w_dd` profile section + dispatch (training_profile.rs RewardSection,
OptimizableParameterRanges, FixedRewardParameters, ParamLookup
dispatch, profile→hyperparam mapping, test assertion)
- `w_dd *= rki` risk-intensity multiplier (config.rs)
- `w_dd` TOML keys (dqn-hyperopt.toml × 2, dqn-localdev.toml,
dqn-production.toml, dqn-smoketest.toml)
- Stale doc comments on hyperopt/adapters/dqn.rs + config.rs
risk_intensity field
- `config.dd_threshold` SURVIVES (still consumed by `launch_sp15_dd_state`
as the dd_budget for DD_PCT scaling). Documented in field comment.
ISV_TOTAL_DIM: 458 → 459 (Item 1 adds 1 slot; Item 2 is pure deletion)
Cumulative WR-plateau fix series (this is commit 7):
- Class C bug 1 + P0-B (
|
||
|
jgrusewski
|
87d597d5d7 |
fix(class-a-p1-producer): adaptive Bayesian Kelly priors per-fold-end
Replace 4 hardcoded Bayesian priors in Kelly cap calculations
(prior_wins=2.0, prior_losses=2.0, prior_sum_wins=0.01,
prior_sum_losses=0.01) with ISV-driven slow-EMA values fed by a new
producer kernel that aggregates realized PS_KELLY_* fields across envs
from the same portfolio_state buffer kelly_cap_update_kernel reads
from at the same per-epoch boundary.
Slots [454..458):
- KELLY_PRIOR_WINS, KELLY_PRIOR_LOSSES,
KELLY_PRIOR_SUM_WINS, KELLY_PRIOR_SUM_LOSSES
ISV_TOTAL_DIM bumps 454 -> 458.
Producer:
- kelly_bayesian_priors_update_kernel.cu (per-fold-end / per-epoch
boundary, block-tree-reduce in shmem, no atomicAdd). Pearl-A
first-observation bootstrap (sentinels 2.0/2.0/0.01/0.01 match
pre-P1-Producer hardcoded values for bit-identical cold-start) +
alpha=0.005 slow EMA. Bounds counts in [0.5, 100], sums in [0.001,
1.0] per feedback_isv_for_adaptive_bounds. Launched RIGHT BEFORE
launch_kelly_cap_update so that kernel sees the freshly-blended
priors.
Consumers migrated atomically (same commit):
- kelly_cap_update_kernel.cu:39-42 -> ISV[454..458) with cold-start
fallback via kelly_prior_or_default helper (range guard).
- trade_physics.cuh::kelly_position_cap:304-307 -> NULL-tolerant
isv_signals_ptr threaded through apply_kelly_cap (single caller in
unified_env_step_core line 898 already had the bus pointer; mirrors
the existing kelly_f_smooth / kelly_warmup_floor_sp9 patterns).
State reset:
- 4 FoldReset registry entries (sp14_p1_kelly_prior_*) +
4 dispatch arms in training_loop.rs::reset_named_state.
Oracle tests (sp14_oracle_tests.rs, GPU-gated #[ignore]):
- Pearl-A bootstrap (sentinel -> REPLACE with aggregated targets)
- No realized trades -> ISV preserved bit-exactly
- Bounds clamp on extreme aggregates
- Slow EMA blend after bootstrap
CPU tests passing:
- sp14_p1_kelly_prior_slot_layout_locked
- all_sp14_p1_slots_fit_within_isv_total_dim
- every_fold_and_soft_reset_entry_has_dispatch_arm (C.10 lesson)
- layout_fingerprint_bumps_after_sp14_wire
DEFERRED — Item 2 (MIN_HOLD_TEMPERATURE EMA): the audit-spec said
"MIN_HOLD_TEMPERATURE = 0.5f hardcoded somewhere" but the actual code
has MIN_HOLD_TEMPERATURE_{START=50.0f, END=5.0f, DECAY=20.0f} as a
PER-EPOCH ANNEALING SCHEDULE driven by min_hold_temperature_for_epoch
in training_loop.rs:68-73. The kernel takes T as a runtime scalar
specifically to enable Phase 2 ISV-driven lift "without recompiling
cubin" per the SP12 v3 design comment. The audit's claim of "0.5f
hardcoded" does not match reality; the right Phase 2 signal is a
separate spec decision and should not be guessed at per
feedback_no_quickfixes. Reporting back per the prompt's hard rule #8.
Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor +
pearl_controller_anchors_isv_driven.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
657972a4b5 |
fix(class-a-p0a-downstream): DD penalty + MIN_HOLD_PENALTY_MAX scale to POS_CAP_ADAPTIVE
Per Class A audit P0-A downstream batch — both constants were tuned for fixed REWARD_POS_CAP=5.0f. P0-A made POS_CAP adaptive via isv[452]; this commit propagates the ratio to keep the Kahneman 2:1 asymmetry coherent across the reward shaping chain. Items: 1. DD penalty -5.0f → -1.0f * isv[REWARD_POS_CAP_ADAPTIVE_INDEX] (helper signature change in trade_physics.cuh::compute_drawdown_penalty adds dd_penalty_scale parameter; sole call site in experience_kernels.cu:3775 resolves the ISV value with the same defensive guard as sp15_apply_sp12_cap) 2. MIN_HOLD_PENALTY_MAX 3.0f → 0.6f * isv[REWARD_POS_CAP_ADAPTIVE_INDEX] (existing 60% ratio from state_layout.cuh comment line 252-253 preserved; resolved at the call site mirroring the effective_min_hold_target precedent for slot 451) Cold-start fallbacks preserved: - DD penalty: REWARD_POS_CAP=5.0f when ISV at sentinel/out-of-range - MIN_HOLD_PENALTY_MAX: kernel-passed 3.0f from gpu_experience_collector.rs:399 (bit-identical pre-P0-A behavior) Defensive guard at both consumer sites: ISV must be in [REWARD_POS_CAP_MIN_BOUND=1.0, REWARD_POS_CAP_MAX_BOUND=50.0] AND not within 1e-6f of SENTINEL_REWARD_POS_CAP=5.0f. Mirrors the existing sp15_apply_sp12_cap and segment-complete cap fallback patterns. Note: the SP15 quadratic DD penalty path (compute_sp15_final_reward_ kernel.cu::sp15_dd_penalty) is already fully ISV-driven via slots 420 (λ_dd) and 421 (dd_threshold) — only the legacy compute_drawdown_ penalty (linear ramp, slot-free) had the hardcoded -5.0f. The audit recommendation suggested ratio = 1.0 for MIN_HOLD_PENALTY_MAX assuming the value was 5.0f; actual is 3.0f and the existing tuning comment locks the ratio at 60% — pure wiring uses 0.6. Cumulative WR-plateau fix series: - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
c4b6d6ef29 |
fix(class-a-p1-wiring): var_floor q_gap-only adaptive (1 of 4 wireable)
Per Class A audit P1 wiring batch — only 1 of 4 items was wireable as spec'd. The other 3 are deferred per the spec's "DEFER not BLOCK" rule because their target ISV slots either don't exist or are unit-incompatible with the consumer site. Items: 1. trade_physics.cuh:548 (DD floor_dd 0.25f) — DEFERRED. Slot 421 (DD_THRESHOLD_INDEX) holds the DD trigger threshold (~0.05), NOT the penalty saturation floor (~0.25). Different parameters; substituting would break the ramp formula `(dd-thr)/(floor-thr)`. Needs a new slot for the saturation floor. 2. gpu_experience_collector.rs:343-344 (dd_threshold + w_dd config) — DEFERRED. No W_DD slot exists in any sp*_isv_slots.rs. The legacy compute_drawdown_penalty path (experience_kernels.cu:3769) uses config scalars; the SP15 reward axis (compute_sp15_final_reward_kernel) already reads slot 421 directly. Wiring needs a new W_DD slot. 3. plan_threshold_update_kernel.cu:44 (plan threshold floor) — DEFERRED. Unit mismatch: ema and plan_threshold are probabilities ∈[0,1]; Q_DIR_ABS_REF (slot 21) is a Q-value magnitude EMA (5–50). Scaling a probability by a Q-magnitude is dimensionally meaningless. 4. experience_kernels.cu:2471 (var_floor formula) — DONE. Drop the hardcoded 0.25f lower bound. Was `fminf(fmaxf(q_gap*0.5f, 0.25f), 1.0f)`; now `fminf(q_gap*0.5f, 1.0f)`. NULL-q_gaps fallback uses ε=1e-6f for numerical safety (vs prior 0.25f). var_scale itself remains (0,1] from `1/(1+sqrt(var_q))` so the multiplicative chain stays bounded. Cumulative WR-plateau fix series: - Class C bug 1 + P0-B ( |
||
|
jgrusewski
|
394de7d434 |
feat(class-a-p0a): REWARD_POS/NEG_CAP → ISV-driven adaptive caps from realized return distribution
Per Class A audit ranking, the highest-suspected-impact fix for the months-long WR-stuck-at-46-48% plateau across 11 superprojects. Hardcoded REWARD_POS_CAP=+5.0f / REWARD_NEG_CAP=-10.0f (state_layout.cuh:266-267) was structurally clipping the upper tail of realized alpha. Controller signals (sharpe EMA, var_q, q_gap) all derive from this CAPPED buffer, so the controller cannot select for trades it cannot see. Selectivity gradient evaporates — small wins clip to +5 alongside large wins also clipping to +5. The state_layout comment lines 261-264 explicitly deferred Phase 2 (ISV-driven) "IF Phase 1 validation reveals adaptive need". Phase 1 has been running 11 SPs without budging WR — adaptive need revealed. Architecture: - 2 new ISV slots [452..454): REWARD_POS_CAP_ADAPTIVE, REWARD_NEG_CAP_ADAPTIVE. - Producer kernel reward_cap_update_kernel.cu: block-tree-reduce Welford `mean + Z_99 × sigma` p99 estimator over winning realized returns + conservative `max(p99, max_win)` takeover, × 1.5 safety factor → POS cap. NEG cap = -2 × POS cap (preserves Kahneman 2:1 asymmetry per pearl_audit_unboundedness_for_implicit_asymmetry — asymmetry stays, but moved from hardcoded scalar to producer-time multiplier; single source of truth, no consumer applies the 2× ratio itself). - Pearl-A first-observation bootstrap from sentinel (5.0 / -10.0, matching pre-P0-A hardcoded values for bit-identical cold-start). Welford EMA α=0.01 thereafter (slow blend — reward distribution is the foundation of training and shouldn't move fast). - Bounds: POS in [1, 50], NEG in [-100, -2] (Category-1 dimensional safety per feedback_isv_for_adaptive_bounds, NOT tuning). - 3 consumer sites migrated atomically per feedback_no_partial_refactor: experience_kernels.cu:3112-3114 (segment_complete cap), compute_sp15_final_reward_kernel.cu:163 (Stage 4 helper invocation), sp15_reward_axis_helpers.cuh:211 (sp15_apply_sp12_cap device fn signature change to take isv ptr). - Cold-start fallback: when ISV slot at sentinel OR outside [REWARD_POS_CAP_MIN_BOUND=1, REWARD_POS_CAP_MAX_BOUND=50], consumers fall back to original macros (still defined in state_layout.cuh). - 2 new device-ptr accessors on the experience collector (step_ret_per_sample_dev_ptr, trade_close_per_sample_dev_ptr) — reuses existing per-sample buffers; no new buffer allocated. - Per-epoch boundary launch (cold path) in training_loop.rs alongside launch_aux_horizon_chain. - Reset registry entries + dispatch arms in reset_named_state per the C.10 lesson (missing dispatch causes runtime crash). - Layout fingerprint seed updated: ISV_TOTAL_DIM 452→454 + AUX_PRED_HORIZON_BARS=450 + AVG_WIN_HOLD_TIME_BARS=451 (previously missing from seed) + REWARD_POS_CAP_ADAPTIVE=452 + REWARD_NEG_CAP_ADAPTIVE=453. Per feedback_isv_for_adaptive_bounds: every adaptive bound in ISV. Verification: - cargo check -p ml --tests --all-targets: clean (19 pre-existing warnings, 0 new). - sp14_isv_slots tests: 8/8 pass (4 layout + 4 fits-within). - sp14_oracle_tests with --features cuda --ignored: 8/8 pass (4 existing q_disagreement/dir_concat + 4 new P0-A tests covering Pearl-A bootstrap, no-winning-trades preservation, bounds clamping to [1,50], Welford α=0.01 EMA blend). - sp15_phase1_oracle_tests with --features cuda --ignored: 36/36 pass (no regression from sp15_apply_sp12_cap signature change). Cumulative WR-plateau fix series: - Class C bug 1 ( |
||
|
jgrusewski
|
316db416bb |
fix(class-a-p0c): MIN_HOLD_TARGET → ISV[AVG_WIN_HOLD_TIME_BARS_INDEX=451] (adaptive)
Per Class A audit: MIN_HOLD_TARGET=30.0f hardcoded was creating a
deterministic gradient pushing trades toward 30-bar holds regardless of
edge expiry. User's trading frequency is between HFT-MFT and varies by
regime; a 30-bar fixed target kills MFT-frequency alpha when the
optimal hold for current data is shorter (or longer).
The producer slot ISV[AVG_WIN_HOLD_TIME_BARS_INDEX=451] already exists
from SP14 Layer C Phase C.4b (commit
|