0ca45ef61d67fe2ed3ebbdb9d7bd105b9f63f0fe
962 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
0ca45ef61d |
docs(sp13): v2 spec + plan after critical review
Spec v2 supersedes v1 with five major fixes from the critical review: - Drop alpha-vs-benchmark reward (mathematically tautological — Long trades had alpha = -costs always against always-long benchmark) - Split Phase 0 into 0a (Hold-only, clean test of user hypothesis) + 0b (aux amplification probe, only if 0a partial), avoiding the v1 confounded experiment - Bound direction-skill bonus relative to |alpha| (cap_ratio × |alpha|) to prevent reward gaming on small-alpha correct-direction losers; spec adds 8-quadrant worked-example matrix verifying the no-negative-EV invariant - Replace 5-epoch absolute-threshold gate with 10-epoch trajectory criterion to avoid false-negatives from aux head underconvergence - Aux_w controller adds dual-EMA stagnation detector (decays toward base when no improvement) — prevents permanent destabilization of Q-head in data-limited case Plan v2 mirrors spec changes: - Phase 0a (Hold elimination + dir_acc instrumentation, ~300 LOC, 1 atomic commit) - Phase 0b (aux_w controller replacement, conditional, ~80 LOC) - Layer B (aux head regression -> binary classification, ~120 LOC) - Layer C (skill bonus + luck discount with calibrated bounds, ~150 LOC) - Layer D (30-epoch validation + 3 new pearls) ISV slot allocation [372..380): drops slot 371 (BENCHMARK_PNL_CUMULATIVE), adds 374 (AUX_DIR_ACC_LONG_EMA — stagnation), 379 (SKILL_BONUS_CAP_RATIO). Plan integrates Explore-agent touch-list (50-70 sites) with corrected enum ordering: Short=0, Long=1, Flat=2 (preserves codebase Short-first convention, avoids 30+ stale-comment churn). Adds direction-bias signal handling at experience_kernels.cu:5159 (Hold's 0.5 softening gate vanishes -> [1, 1, 0]). Three new pearls planned for Layer D close-out: - pearl_redefine_success_for_predictive_skill - pearl_skill_bonus_must_be_alpha_bounded (calibration lesson) - pearl_reward_quadrant_audit_required (meta-pearl) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a1681abc46 |
Revert "exp(sp13): aux_w=1.0 override + directional accuracy metric for data investigation"
This reverts commit
|
||
|
jgrusewski
|
d2a27a0042 |
exp(sp13): aux_w=1.0 override + directional accuracy metric for data investigation
One-shot diagnostic to answer the SP13 root question: does the data have
predictable directional signal at the bar level?
Wires the existing `aux_next_bar_loss_reduce` kernel (which already had a
4-strip shmem reduction emitting `[dir_acc, pos_pred_frac, pos_label_frac]`
into a 3-float output) end-to-end:
- `gpu_aux_heads.rs`: launcher takes `dir_acc_out_ptr`, allocates 4×AUX_BLOCK
shmem to back the four parallel reductions.
- `gpu_dqn_trainer.rs`: adds `aux_nb_dir_acc_buf` (3 f32 device buffer),
threads it through the loss-reduce launch, exposes `read_aux_dir_acc()`
accessor for once-per-epoch DtoH readback.
- `training_loop.rs`: pins `aux_w = 1.0` (instead of the ISV-driven 0.05–0.3
clamp) so the supervised aux head dominates the loss; emits a new
`HEALTH_DIAG[ep]: aux_dir_acc accuracy=… pos_pred_frac=… pos_label_frac=…`
line per epoch.
Verdict thresholds:
* dir_acc > 55% by ep 5 ⇒ data has signal, DQN failing to use it
* dir_acc ≈ 50% throughout ⇒ data lacks signal at this timescale
* dir_acc 60–70% ⇒ strong signal we're not using
EXPERIMENT BRANCH — revert this commit after the investigation reads back the
5-epoch table from smoke logs. All five touch points are tagged
"SP13 data-investigation" / "EXPERIMENT" for clean revert.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
1c645264e6 |
test(sp12): GPU oracle tests for the 3 reward math changes
Adds the GPU oracle test scaffold deferred from commit
|
||
|
jgrusewski
|
17cfbb2503 |
fix(sp12): per-trade event-driven reward composition
Three architectural changes in one atomic commit per spec |
||
|
jgrusewski
|
ecab584c32 |
spec(sp12): per-trade event-driven reward composition (v3)
Three architectural changes in one unified design to push the model from
HFT noise extraction (62% trade rate, sharpe-gaming) toward MFT alpha-hunting:
1. Asymmetric bounded cap (-10/+5) — restores loss aversion erased by
SP11 symmetric cap. 2:1 ratio matches Kahneman/Tversky prospect theory.
Anchor: pearl_audit_unboundedness_for_implicit_asymmetry.
2. Min-hold soft penalty with temperature curriculum — patience requirement
at exit. Soft factor = deficit/(deficit+T), T anneals 50→5 over 50 epochs.
Forces commitment without paralysis in early training.
3. Zero per-bar shaping (gate micro/opp_cost on events) — eliminates
continuous-reward gradient that pulls toward continuous exposure.
Anchor: pearl_event_driven_reward_density_alignment.
Combined: reward fires only on trade events with prospect-theory loss
aversion + commitment requirement. Pure per-trade event-driven Q-learning
properly aligned with per-trade P&L objective.
~50 LOC across 3-4 files. No new ISV slots in Phase 1 (constants only).
Cost ~€1.30 (€0.30 smoke + €1.00 30-epoch validation).
Empirical motivation: train-multi-seed-pmbwn 50-epoch on commit
|
||
|
jgrusewski
|
6a259942e9 |
docs(sp11): close-out — pearl_symmetric_clamp_audit + audit doc
Documents the SP11 sharpe-degradation root cause investigation: - |
||
|
jgrusewski
|
b92dcc3dfc |
chore(sp11): remove reward-chain diagnostic instrumentation
Instrumentation from |
||
|
jgrusewski
|
348f6078b8 |
fix(sp11): plan_isv symmetric clamp — 4 sites mirror reward-cap bug
Implementer of
|
||
|
jgrusewski
|
35db310893 |
fix(sp11): symmetric reward cap — losses were unbounded
experience_kernels.cu:2788:
float capped_pnl = fminf(base_reward, 10.0f);
^^^^^^^^^^^^^ caps profits, NOT losses
Diagnostic instrumentation in smoke-test-k9drh on commit
|
||
|
jgrusewski
|
774d7552a0 |
diag(sp11): instrument reward chain to find the 5000x inflater
Smoke smoke-test-gwfn8 on
|
||
|
jgrusewski
|
7a19c51522 |
docs(sp11): B1b smoke-recovery close-out — audit + 3 memory pearls
Layer C close-out for SP11 B1b smoke-recovery work. Two atomic commits land the fix: - |
||
|
jgrusewski
|
fd24b53833 |
fix(sp11): B1b launch-order — reward_component_ema before mag-ratio canary
smoke-test-4rbv9 on
|
||
|
jgrusewski
|
b3b4d02789 |
fix(sp11): B1b smoke-recovery — z-score normalization for mag-ratio canary
Linear magnitude ratios in reward_component_mag_ratio_compute_kernel amplified popart's intrinsic O(100) magnitude over the other 5 components' O(0.1-2) magnitudes, causing controller to saturate w_pop toward MAX_WEIGHT regardless of actual signal quality. Replaced with z-score: z[c] = mag[c] / max(sqrt(var[c]), EPS_DIV). 6 new ISV slots [361..367) for per-component variance EMAs computed via Welford's online algorithm in extended popart_component_ema_kernel and reward_component_ema_kernel. Atomic per feedback_no_partial_refactor: slot allocation + state-reset registry + 2 producer kernels + canary signature + launcher Pearls A+D + tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
85069ba75e |
spec(sp11): amend B1b smoke-recovery — z-score normalization for mag-ratio canary
smoke-test-6wd2c on commit |
||
|
jgrusewski
|
61b2fa962b |
fix(sp11): B1b bug 3 — cf-component feedback loop in mag-ratio canary
Deep audit on |
||
|
jgrusewski
|
b435d25bec |
fix(sp11): B1b bug-hunt fix-up — stale rc[] init + cf_flip ordering
Bug-hunt review on
|
||
|
jgrusewski
|
5e16b67ca6 |
fix(sp11): B1b follow-up — add slot 360 for popart-component mag EMA
Per spec §4 amendment at
|
||
|
jgrusewski
|
034ba16801 |
feat(sp11): B1b — structural reward composition refactor (production flip)
Per spec §3.5.3 amended at
|
||
|
jgrusewski
|
d5e1214f25 |
fix(sp11): B1a — saboteur GPU multiplication + SimHash state_stride
Per feedback_cpu_is_read_only, saboteur effective scale now computed on-device: - saboteur_generate_params kernel signature gains isv ptr + saboteur_intensity_mult_slot parameter - Kernel reads `mult = fmaxf(isv[saboteur_intensity_mult_slot], SABOTEUR_MIN)` (sentinel-0 defense for cold-start before A2's controller first runs) and applies `effective_scale = base × mult` to perturbation generation - gpu_experience_collector.rs launcher updated; only one call site SimHash state_stride parameter added to lookup + update kernels — prepares for B1c replay-time curiosity wiring against trainer. states_buf which is STATE_DIM_PADDED=128-strided. Kernel inner loop reads `state[i × state_stride + d]` (was `i × 42 + d`). Proj-init kernel unchanged (writes projection, doesn't read states). Pre-requisite for B1c (curiosity wiring) and a small atomic step toward full SP11 production behavior. cargo check + build clean; 6/6 SP11 GPU tests + 14/14 contract tests still pass. |
||
|
jgrusewski
|
302992f63a |
fix(sp11): B0 — controller renorm Σ=1 → mean=1 (post-A2 spec amendment)
Per spec §3.4.3 amended at
|
||
|
jgrusewski
|
44fb4531a8 |
fix(sp11): A2 follow-up — delete dead launchers + expand XOR-fold rationale
Code-quality review on
|
||
|
jgrusewski
|
25eba79ad5 |
feat(sp11): A2 — controller kernel + SimHash novelty buffer
reward_subsystem_controller_kernel: 5 canaries → 10 outputs, true Z-score (delta_ema/sqrt(var_ema)), sigmoid blending, weight renormalization to Σ=1, saboteur post-clamp, curiosity permanent floor (0.2 × bound). Pearls A+D chained on outputs per spec §3.4.1. novelty_simhash_kernel: 42×16 random projection → 16-bit SimHash code, 1M-slot bucket count table for novelty signal `1/sqrt(1+count)`. Race- tolerated update per feedback_no_atomicadd (under-counts bias novelty UPWARD — safe direction). novelty_simhash_proj_init_kernel: Philox-seeded GPU init for the projection matrix (CPU is read-only per feedback_no_cpu_forwards). HEALTH_DIAG `sp11_reward` line emits 10 outputs + improvement_z each epoch. Reset registry: novelty hash table reset arm wired (closes the A0 deferral); projection matrix is frozen at trainer init for run lifetime, not reset. All 20 SP11 slots populate every step. No consumer reads them yet — training behavior unchanged from A1. 3 new GPU oracle tests pass on RTX 3050 Ti (controller midpoint, weight renorm, saboteur clamp). Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md §3.4 §3.5.2 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
91b48bc7a5 |
feat(sp11): A1 — three canary producer kernels (no behavior change)
Adds val_sharpe_delta + saboteur_engagement + reward_component_mag_ratio
GPU producers for the SP11 reward-as-controlled-subsystem chain. Each is
a single-block producer chained with apply_pearls_ad_kernel for Pearls
A+D smoothing per pearl_first_observation_bootstrap.md +
pearl_wiener_optimal_adaptive_alpha.md. All three write to slots in
[350..360) which no consumer reads yet — Layer A is additive; consumer
migration lands atomically in Layer B.
A1.1 — val_sharpe_delta_compute_kernel.cu
Two-pass: writes raw delta + (delta - prev_delta_ema)^2 to scratch.
Chained Pearls A+D (n_slots=2) → ISV[VAL_SHARPE_DELTA_EMA_INDEX=350,
VAL_SHARPE_VAR_EMA_INDEX=351]. Host writes val_sharpe to mapped-pinned
history[1]; rotation handled in training_loop.rs at val emit boundary
(a literal already-computed value — no host-side compute, no htod_copy).
A1.2 — saboteur_engagement_compute_kernel.cu
Per-bar |Δreward| > 0.01 × ISV[PNL_REWARD_MAGNITUDE_EMA_INDEX] check
with block tree-reduce (no atomicAdd per feedback_no_atomicadd). The
per-bar Δreward signal is produced by experience_env_step's saboteur
perturbation site as `traded × |reward| × max(|eff_spread − 1|,
|eff_slip − 1|)` — a structural proxy for the cost-differential the
saboteur imposed on bars where the model traded. Single kernel-side
emit (no parallel reward computation), per spec §3.3.1.
Chained Pearls A+D → ISV[SABOTEUR_ENGAGEMENT_RATE_INDEX=358].
A1.3 — reward_component_mag_ratio_compute_kernel.cu
Reads ISV[REWARD_POPART_EMA_INDEX..+6) (the SP4 reward-component
magnitude EMAs), normalises to ratios, and mirrors popart magnitude
into scratch[6] as a side-output. ONE non-pointer parameter
(popart_ema_base_slot) — no _unused param per feedback_no_stubs.
Two chained Pearls A+D launches:
n_slots=6 → ISV[REWARD_COMPONENT_MAG_RATIO_BASE..+6)
n_slots=1 → ISV[PNL_REWARD_MAGNITUDE_EMA_INDEX=359]
(slots non-contiguous: 352..358 then 359.)
Wire-up (per feedback_wire_everything_up):
- 3 cubin entries appended to crates/ml/build.rs
- 3 kernel handles + val_sharpe_history_pinned (MappedF32Buffer[2]) +
saboteur_delta_reward dev-ptr cache fields on GpuDqnTrainer
- 3 launchers (launch_sp11_*) + 1 setter (set_sp11_saboteur_delta_reward_buf)
- saboteur_delta_reward_per_sample buffer field on GpuExperienceCollector
- experience_env_step kernel signature extended with the new buffer arg;
every call site in the same commit per feedback_no_partial_refactor
- training_loop.rs init wires collector→trainer setter; val emit boundary
invokes launch_sp11_val_sharpe_delta_compute; per-epoch metrics block
invokes launch_sp11_mag_ratio_compute then
launch_sp11_saboteur_engagement_compute (mag_ratio first so the
signal-relative threshold base is populated before the saboteur reader)
- SP5_SCRATCH_TOTAL grown 266 → 276 (10 new scratch slots: 2+1+7)
- docs/isv-slots.md SP11 section updated to reflect A1 producers
3 GPU oracle tests in crates/ml/tests/sp11_producer_unit_tests.rs
pass on RTX 3050 Ti via MappedF32Buffer fixtures (zero htod_copy /
dtoh_sync_copy / alloc_zeros — feedback_no_htod_htoh_only_mapped_pinned
compliant).
Note on Step 8a path: the plan offered two routes for the saboteur
Δreward producer — in-kernel diff emission OR a small dedicated
reader-of-existing-buffers. The existing reward path emits ONE reward
(not both with/without), so the dedicated-reader alternative was
infeasible. The in-kernel emission landed as a small write site at the
END of experience_env_step (after total_reward_per_sample is finalised),
threading saboteur_eff_spread/saboteur_eff_slip from the perturbation
site forward to the END via stack vars. Single new kernel parameter,
single new GPU-only buffer, single existing call site updated.
Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md §5
Plan: docs/superpowers/plans/2026-05-04-sp11-reward-as-controlled-subsystem.md (Task A1)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
201b59dfbc |
fix(sp11): A0 sweep — eliminate remaining stale wiener-buffer refs
A0 follow-up (
|
||
|
jgrusewski
|
1e5a65912c |
fix(sp11): A0 follow-up — update stale wiener-buffer + isv-slots header
Code-quality review on
|
||
|
jgrusewski
|
bf3a32d63a |
feat(sp11): A0 — allocate 20 ISV slots [340..360) + 20 reset entries
Pure infrastructure. No producer kernels, no consumer reads. Existing training paths trace identically because no consumer reads slots [340..360) yet. Layout-fingerprint bumped to ISV_TOTAL_DIM=360. Spec: docs/superpowers/specs/2026-05-04-sp11-reward-as-controlled-subsystem.md |
||
|
jgrusewski
|
e0d3abd9d2 |
plan(sp11): patch all 5 violations + 2 gaps from review
feedback_no_htod_htoh_only_mapped_pinned (tests not exempt):
- All test fixtures converted from htod_copy/dtoh_sync_copy to
MappedF32Buffer with host_slice / host_slice_mut access.
- Novelty hash buffer + projection matrix changed from
cudarc::CudaSlice<f32> + alloc_zeros to MappedF32Buffer.
feedback_no_cpu_forwards (CPU is read-only):
- Projection matrix initialization changed from host-side StdRng +
host_slice_mut writes to a one-shot GPU init kernel
(novelty_simhash_proj_init_kernel) using Philox seeded from
config.seed. No host RNG, no host writes.
feedback_no_cpu_compute_strict (saboteur multiplication):
- B1 step 5 reverted from Rust-side `read_isv_slot * scale` to
GPU-side: pass base scale + ISV pointer + slot index to the
saboteur perturbation kernel; multiplication happens on-device.
feedback_trust_code_not_docs (grad-ratio terminology):
- Spec §3.3.1 "per-component grad EMA" was wrong — SP4 grad-balancer
is per-branch (4 slots), not per-reward-component. Renamed:
reward_component_grad_ratio_compute_kernel
→ reward_component_mag_ratio_compute_kernel
REWARD_COMPONENT_GRAD_RATIO_BASE
→ REWARD_COMPONENT_MAG_RATIO_BASE
Source: existing REWARD_POPART_EMA_INDEX..+6 (per-component reward
magnitude EMAs from SP4 reward_component_ema_kernel). Semantic
equivalent for the controller's exploit/diversify blend.
feedback_no_stubs (dead parameter):
- Removed `eps_div_idx_unused` from mag_ratio kernel signature.
Saboteur engagement: missing producer specified
- Spec §3.3.1's two-reward-arrays formulation replaced with single
`saboteur_delta_reward_buf` produced by the saboteur perturbation
kernel itself (single reward computation, diff emitted as side
output). Engagement kernel signature simplified to one input array.
PNL_REWARD_MAGNITUDE_EMA_INDEX (slot 359): producer wired
- mag-ratio kernel mirrors `isv[REWARD_POPART_EMA_INDEX]` to
scratch_out[6]; chained apply_pearls_ad targets slot 359.
Replay sample kernel location specified
- graph_utility_kernels.cu:71 (gather_f32_scalar). New sibling kernel
`gather_replay_reward_with_curiosity` defined; replaces the existing
scalar gather (no legacy alias per feedback_no_legacy_aliases).
novelty_simhash_lookup runs before, novelty_simhash_update after.
A2 controller test placeholders → full GPU oracle assertions
- Three controller tests (z=0 midpoint, weight renorm, saboteur clamp)
have full mapped-pinned fixtures with assertions on weight sum,
individual values, post-clamp bounds.
Plan now passes:
- feedback_no_htod_htoh_only_mapped_pinned (tests + production)
- feedback_no_cpu_forwards (CPU never writes/computes for GPU)
- feedback_no_cpu_compute_strict (all multiplications GPU-side)
- feedback_no_atomicadd (race-tolerated non-atomic, safety documented)
- feedback_no_partial_refactor (Layer B atomic; saboteur kernel sig
change touches all callers in the same commit)
- feedback_no_stubs (no dead parameters)
- feedback_trust_code_not_docs (corrected spec terminology)
- feedback_wire_everything_up (every new field has init + producer)
- feedback_no_legacy_aliases (old gather_f32_scalar replaced, deleted)
1447 lines, +268 from previous version.
|
||
|
jgrusewski
|
d8b44e0829 |
plan(sp11): implementation plan for reward-as-controlled-subsystem
Task-by-task plan for the SP11 spec at HEAD
|
||
|
jgrusewski
|
9395b983cc |
spec(sp11): fix all 13 review issues — straight-up implementation
Critical bugs fixed:
1. Z-score formula: was mean(Δ)/mean(|Δ|), bounded to [-1,+1] — sigmoid
never saturated, controller stuck in [0.27, 0.73]. Now true Z-score:
delta_ema / sqrt(delta_var_ema) with two-pass var. Renamed canary
slot 351 from VAL_SHARPE_STD_EMA to VAL_SHARPE_VAR_EMA.
2. Component weight renormalization added — Σweights = 1 enforced
post-floor so PopArt's reward-magnitude EMA stays uncontaminated.
3. SABOTEUR_MIN bound violation — engagement-floor (0.1) was silently
pulling output below 0.5 stated minimum. Added post-multiplication
clamp to [SABOTEUR_MIN, SABOTEUR_MAX].
4. ratios[] undeclared — now __shared__ float ratios[6] block-loaded
once from ISV.
5. §3.6 slot count off-by-five (15 → 20). All 20 slots get FoldReset
entries; cold-start window behavior specified explicitly (no NaN path).
Architectural fixes:
6. Q-overconfidence concern addressed in scope — controller's
REWARD_CF_WEIGHT_INDEX covers CQL conservatism. No SP11′ deferral;
if T10 fails, extend controller outputs (e.g., target-update τ).
7. Curiosity recompute-at-replay specified (§3.5.1) — replay buffer
stores base reward only; curiosity added per-tuple at replay time
against current visit-count and current ISV. Eliminates stale-signal
replay contamination that would worsen ep1-overfitting.
Smaller fixes:
8. Novelty signal specified — SimHash 42×16 projection → 16-bit code,
1M-slot per-bucket count table, 1/sqrt(1+count). Hash table lives
in state-reset registry (cleared at fold boundary).
9. Saboteur engagement operationally defined — per-bar
|reward_with_saboteur − reward_without_saboteur| > EPS_ENGAGEMENT,
block tree-reduce, EMA'd. EPS_ENGAGEMENT = 0.01 × PnL_EMA.
10. Duplicate sentence in §7 component-starvation paragraph removed.
11. Validation criteria strengthened (§9): aggregation across 9
trajectories specified; primary metric = median peak-epoch ≥ 10
(baseline median peak-epoch = 1); secondary = drop-from-peak ≤ 5%
by ep20; tertiary = ep20 mean ≥ baseline ep1 mean. §9.3 fix-forward
response codified — no rollback.
12. Phases split per project pattern — Layer A additive (3 commits:
slots, canaries, controller), Layer B atomic consumer migration
(1 commit, including replay-time curiosity), Layer C validation +
close-out. No falsification gate; smoke is validation only.
13. Pearls A+D applied to controller outputs (§3.4.1) — chained
apply_pearls_ad_kernel after controller writes scratch, smoothed
values land in ISV. Consumers read smoothed slots.
Constants surviving (Invariant-1 anchors only, all rate-not-regime):
EPS_DIV, WEIGHT_HARD_FLOOR, SABOTEUR_MIN/MAX, CURIOSITY_PERMANENT_FRACTION,
CURIOSITY_BOUND_FRACTION, WEIGHT_FLOOR_FRACTION, ENGAGEMENT_FLOOR.
Spec is now full straight-up implementation; smoke validates Layer A
infrastructure and Layer B consumers, T10 validates SP11 success metric.
~1550 LOC across 3 commits in Layer A + 1 atomic commit in Layer B +
close-out in Layer C.
|
||
|
jgrusewski
|
d49fbbe4e2 |
spec(sp11): reframe curiosity as feature + add permanent floor
User correction: curiosity is the *fix* for the ep1-peak overfitting pathology, not a hazard to defend against. Reframed §7 from "Risks" to "Design notes" — curiosity bound is a signal-relative scale, not a defensive cap. Fix the contradiction this exposed in the formula: previous `curiosity_pressure = stagnant_or_worse * curiosity_bound` went to zero when improving, which would cancel the always-on exploration the §7 narrative now relies on. Replace with permanent-floor pattern per pearl_blend_formulas_must_have_permanent_floor: curiosity_floor = 0.2 * curiosity_bound (CURIOSITY_PERMANENT_FRACTION) curiosity_dynamic = stagnant_or_worse * curiosity_bound curiosity_pressure = max(curiosity_dynamic, curiosity_floor) Now curiosity is always ≥ 20% of bound (anti-overfitting baseline) and rises toward the bound when stagnant (stagnation breaker). Updated unit-test guidance to assert pressure > 0 even at z=+10. CURIOSITY_PERMANENT_FRACTION=0.2 added to Invariant-1 fraction list. Saboteur-rising-with-improvement reframed as adversarial-load feature rather than over-stress risk. |
||
|
jgrusewski
|
a72a23e6b1 |
spec(sp11): reward as controlled subsystem — fully ISV-driven
Brainstorm spec for SP11. Resolves the policy-stagnation pathology
surfaced in T10 train-multi-seed-xkjkb seed-0 ep0-14: model finds a
stable fixed point at ep1 (peak val sharpe 80.61), then OVERFITS to
it across remaining epochs (decline 80.61 → 70.58). Q-values grow but
val performance declines because reward function has no improvement
pressure.
Architecture (every input ISV-driven):
- Z-score-driven adaptation (no hardcoded "improving" threshold):
improvement_z = val_sharpe_delta_ema / max(val_sharpe_std_ema, EPS)
- 10 ISV outputs: 6 component weights + curiosity_pressure +
saboteur_intensity_mult + adaptive weight_floor + curiosity_bound
- 5 ISV canaries: val_sharpe_delta + val_sharpe_std (Z-score noise
estimate) + 6 per-component grad ratios + saboteur engagement +
PnL magnitude EMA (signal-relative curiosity bound)
- 4 new producer kernels (controller + 3 canary computers)
- Audit + migrate hardcoded cf_weight=0.3 in mse_loss_kernel.cu:318
and c51_loss_kernel.cu:789, plus other shaping multipliers
- NEW reward dimension: curiosity bonus, bounded by PnL magnitude
Per pearl_controller_anchors_isv_driven: every threshold replaced with
sigmoid(z) — no constants encode "what counts as improving". Per
pearl_blend_formulas_must_have_permanent_floor: every weight has
adaptive floor preventing zero-out. Per pearl_engagement_rate_self_
correction: saboteur intensity self-corrects via engagement rate canary.
Per pearl_cold_start_exit_signal_or: improvement signal OR'd from
multiple canaries so single-signal-failure doesn't stall controller.
New pearl authored alongside spec: pearl_reward_as_controlled_subsystem
— meta-principle that every reward path degree of freedom is a unified
controller output. Subsumes controller-anchor pearl at the reward layer.
Scope: 20 ISV slots, 4 producer kernels, audit + migration of
hardcoded reward shaping constants, 1 atomic commit.
~1300-1700 LOC; ~2.5-3 hours subagent work.
Success metric: val_sharpe[ep20] > val_sharpe[ep1] (Fix 33-38 baseline
peaked at ep1; SP11 should shift peak later as model continues
learning).
|
||
|
jgrusewski
|
e580c1388c |
fix(log): epoch summary Return uses scientific notation, fixes overflow display
`total_return` from financials.rs:80-94 is log-space cumulative growth
across every per-bar step_return. With ~4M step_returns in a fold-
convergence run, even sub-bps positive bars compound to absurd
magnitudes (observed: 1.93e37%) when displayed as `{:+.2}%`. Math is
correct; display needs scientific notation.
Surfaced in T10 train-multi-seed-xkjkb seed-0 ep3 epoch summary while
SP10 chain validates structural fixes. Cosmetic-only change; no
training-path impact. Audit doc updated with Cosmetic 38.1 entry.
|
||
|
jgrusewski
|
a5457b0dc7 |
Revert "fix(log): epoch summary Return uses scientific notation, fixes overflow display"
This reverts commit
|
||
|
jgrusewski
|
368454788a |
fix(log): epoch summary Return uses scientific notation, fixes overflow display
`total_return` from financials.rs:80-94 is the log-space cumulative growth
across every per-bar step_return: `exp(sum(ln(max(1+r, 1e-10)))) - 1`.
With ~4M per-bar step returns in a fold-convergence run, even sub-bps
positive bars compound to absurd magnitudes (observed: 1.93e37%) when
displayed as `{:+.2}%`. The math is correct; the display is broken.
Switching to `{:+.3e}%` shows the same magnitude compactly across the
full dynamic range without hiding the number. Comment added explaining
the underlying compute's HFT-inappropriate semantic so a future reader
knows the value is correct-but-meaningless rather than buggy.
Surfaced in T10 train-multi-seed-xkjkb seed-0 ep3 epoch summary while
the SP10 chain validates structural fixes. Cosmetic-only change; no
training-path impact.
|
||
|
jgrusewski
|
920a2d0219 |
fix(sp10): unconditional Thompson selector + ISV-driven temperature (Fix 38)
T10 train-multi-seed-khr7c (commit
|
||
|
jgrusewski
|
6ae9bc0055 |
spec(sp10): unconditional Thompson selector + ISV-driven temperature
Brainstorm spec for SP10. Resolves the val-Flat-collapse pathology that persisted through Fix 33-37: the eval-time argmax in experience_action_ select picks Hold deterministically every val bar (dir_entropy=0, trade_count=1 in 214k bars) regardless of controller state. Architecture: - Delete `if (eval_mode)` argmax branch in direction-selector kernel - Use temperature-blended Thompson: q_eff = E[Q] + τ × (Thompson - E[Q]) - τ = clamp(intent_eval_divergence / divergence_target, 0.5, 2.0) - τ self-corrects: collapse → high τ; healthy → low τ; permanent 0.5 floor - Reuses SP9's intent_eval_divergence_compute_kernel (extended, not new) Per pearl_controller_anchors_isv_driven: τ is ISV-driven, no hardcoded constants beyond Invariant 1 numerical anchors (clamp range). Per pearl_blend_formulas_must_have_permanent_floor: MIN_TEMP=0.5 ensures eval ALWAYS has stochasticity. The pearl_thompson_for_distributional_action_selection was about Bellman TARGET argmax (selector/target symmetry). It does NOT prohibit Thompson at the rollout selector. SP10 amends the pearl to clarify. Scope: 1 ISV slot, kernel modification (no new kernel — extend SP9's producer), 1 consumer kernel rewrite, test update, pearl amendment, audit doc Fix 38. ~300-500 LOC, single atomic commit per feedback_no_partial_refactor. |
||
|
jgrusewski
|
8a25b330fc |
fix(sp9): targets as Invariant-1 anchors + divergence sentinel handling
Two targeted fixes to the SP9 Kelly cold-start warmup floor (Fix 37,
commit `48a8b9ee7`) for quirks observed in smoke-test-wrwkz on a
5-epoch L40S smoke. Atomic per `feedback_no_partial_refactor`.
**Quirk 1 — EMA target saturation:** ep1 HEALTH_DIAG showed
`stat_count_tgt=419 div_tgt=105002 temp_tgt=1.0 conf [stat=1.000
bhv=0.333 tmp=1.000] floor=0.0`. Pearl A sentinel-bootstrap on the
3 EMA target updaters (`kelly_*_target_ema_kernel.cu`) caused the
first observation to replace the sentinel directly, so
`target = first_obs`, `current/target = 1.0`, `confidence = 1.0`,
`floor = base × (1 − 1.0) = 0` — defeating the cold-start mechanism.
The principled fix per `feedback_isv_for_adaptive_bounds.md`:
"sufficient" is defined in absolute terms via Invariant-1 numerical
anchors. The 3 target ISV slots become constructor-written constants
(written ONCE in trainer constructor, identical pattern to existing
`config.cql_alpha → ISV[CQL_ALPHA_INDEX=48]` from Plan 1 Task 12).
- ISV[KELLY_SAMPLE_COUNT_TARGET_INDEX=333] = 100.0 (trades)
- ISV[KELLY_DIVERGENCE_TARGET_INDEX=334] = 2.0 (ratio)
- ISV[KELLY_TEMPORAL_TARGET_INDEX=335] = 5.0 (epochs)
**Quirk 2 — divergence ratio explosion:** ep1 showed
`divergence=70005`. Algebraically `intent_f / max(eval_f, 1e-6) =
0.07 / 1e-6 = 70 000` because `ISV[EVAL_DIST_F_INDEX=338]` was still
at Pearl A sentinel 0 before the first val window populated it.
Algebraically the warmup-floor kernel still derived
`behavioral_conf = 0` (correct semantic), but the synthetic 70 000
in HEALTH_DIAG masked the real signal flow.
Fix: explicit sentinel detection in
`intent_eval_divergence_compute_kernel.cu`:
if (eval_f < SENTINEL_THRESHOLD=1e-5)
divergence = SENTINEL_DIVERGENCE=1e6 // forces behavioral_conf=0
else
divergence = intent_f / eval_f
Same `behavioral_conf = 0` outcome but with explicit provenance —
HEALTH_DIAG now reads `divergence=1e6` until eval_f matures.
**Atomic structure:**
- DELETE 3 EMA target updater kernels (`.cu` files)
- DELETE 3 entries from `crates/ml/build.rs::kernels_with_common`
- DELETE 3 cubin static byte arrays + 3 struct fields + 3 cubin
loaders + 3 fields in trainer construction tuple in
`gpu_dqn_trainer.rs`
- DELETE 3 launches in `launch_sp9_kelly_warmup_floor` (chain
shrinks 5 → 2: q_var_mag_ema + main warmup-floor)
- DELETE 3 scratch slots; SP5_SCRATCH_TOTAL 268 → 265;
SCRATCH_SP9_EVAL_DIST_BASE slides 265 → 262
- ADD constructor-write of 3 Invariant-1 anchors (100.0, 2.0, 5.0)
immediately before layout-fingerprint write
- UPDATE 3 dispatch arms in `reset_named_state` to rewrite the
Invariant-1 anchors at fold boundary (NOT sentinel 0) — these
are constants, not stateful EMAs
- UPDATE 3 registry descriptions to reflect Invariant-1 anchor
semantic + smoke-test-wrwkz evidence
- UPDATE `intent_eval_divergence_compute_kernel.cu` with
sentinel-detect branch
- ISV slot layout UNCHANGED (3 target slots @ ISV[333..336)
remain); Wiener-buffer linear span (`SP5_PRODUCER_COUNT=165`)
UNCHANGED — 3 wiener triples for deleted producers become
reserved-unused like Pearl 6's [525..543) carve-out
- audit doc Fix 37.1 entry with full provenance + smoke evidence
**Verification:**
- `SQLX_OFFLINE=true cargo check -p ml` — clean (only pre-existing
18 warnings; no new errors or warnings).
- `SQLX_OFFLINE=true cargo test -p ml --lib state_reset` — 4/4 pass
including `every_fold_and_soft_reset_entry_has_dispatch_arm`.
- `SQLX_OFFLINE=true cargo test -p ml --lib sp5_isv_slots` — 9/9
pass including `sp9_slots_contiguous_above_sp8_block`.
**Expected smoke signature post-fix:**
- floor: starts ≈ base_floor × 1.0, decays as confidence accumulates
- divergence: 1e6 until first val window, small (≤ 5) afterward
- conf: stat=count/100, bhv=0 until eval_dist matures, tmp=ep/5
- combined_conf reaches 1.0 only when at least one axis genuinely
matures (typically temporal at epoch 5 in fold 0)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
48a8b9ee7c |
fix(sp9): Kelly cold-start ISV-driven warmup floor (Fix 37)
Per pearl_cold_start_exit_signal_or.md + pearl_controller_anchors_isv_driven.md:
the Kelly cap's `warmup_floor` and its release condition were both
regime-encoded constants. Single-axis statistical cold-start exit
(`total_trades >= 10` in trade_physics.cuh::kelly_position_cap) created
a chicken-and-egg deadlock: cap closed → no trades → no Kelly samples →
cap stays closed. T10 wsnc6 ep1-3 evidence (post-Fix 36):
intent_dist_f rising to 0.47 while eval_dist_f pinned to 0.00 with
kelly_f=0 across all epochs — train-vs-eval divergence as direct
symptom of the cold-start deadlock.
SP9 lifts the floor and its release condition fully onto ISV. The exit
condition is OR'd across statistical / behavioral / temporal axes per
the pearl — any single axis firing exits cold-start.
Atomic commit per feedback_no_partial_refactor:
- 9 new ISV slots @ [330..339):
- KELLY_WARMUP_FLOOR_INDEX (330) — adaptive floor consumed by
unified_env_step_core
- Q_VAR_MAG_EMA_INDEX (331) — historical EMA baseline for the
self-relative `base_floor` ratio (eliminates 0.5 hardcoded
half-Kelly)
- INTENT_EVAL_DIVERGENCE_INDEX (332) — behavioral-axis numerator
- 3 EMA targets (sample_count / divergence / temporal)
- 3 EVAL_DIST_{Q,H,F}_INDEX — replaces DtoH
read_eval_intent_magnitude_distribution()
- ISV_TOTAL_DIM 330 → 339; SP5_PRODUCER_COUNT linear span 156 → 165
- 9 new scratch slots @ [259..268); SP5_SCRATCH_TOTAL 259 → 268
- 7 new producer kernels (single-block cold-path, chained through
apply_pearls_ad_kernel for Pearl A bootstrap + Pearl D Wiener-α):
- eval_intent_dist_compute_kernel.cu — GPU reduction of
intent_mag_buf, replaces DtoH path per
feedback_no_cpu_compute_strict.md
- intent_eval_divergence_compute_kernel.cu
- q_var_mag_ema_compute_kernel.cu
- kelly_sample_count_target_ema_kernel.cu
- kelly_divergence_target_ema_kernel.cu
- kelly_temporal_target_ema_kernel.cu
- kelly_warmup_floor_compute_kernel.cu — main producer combining
all 8 ISV inputs
- consumer migration in trade_physics.cuh::unified_env_step_core:
health_safety_sp9 = fmaxf(health_safety_sp5,
isv[SP9_KELLY_WARMUP_FLOOR_INDEX])
threading through apply_kelly_cap's health_floor parameter; zero
sentinel → no-op via fmaxf so existing path's behavior is preserved
- DELETE read_eval_intent_magnitude_distribution() in
gpu_backtest_evaluator.rs (was DtoH read_all + host loop) per
feedback_no_htod_htoh_only_mapped_pinned.md; replaced by
intent_mag_dev_ptr_and_n() GPU accessor; metrics.rs:908 consumer
migrated from DtoH to launch + ISV mapped-pinned read
- 9 new state-reset registry FoldReset entries + 9 dispatch arms in
reset_named_state (Pearl A bootstrap on first observation)
- HEALTH_DIAG sp9_kelly_warmup line emits floor + divergence + 3
confidence axes + 3 EMA targets per feedback_no_hiding
- audit doc Fix 37 entry with full provenance, files touched, pearls
applied, verification
Constants: only KELLY_FLOOR_MIN_RATIO=0.25, KELLY_FLOOR_MAX_RATIO=1.0,
EPS_DIV=1e-6 — Invariant 1 numerical-stability anchors per
feedback_isv_for_adaptive_bounds.md. All thresholds, targets, and
the floor itself are signal-driven via ISV.
Pre-existing 18 cargo warnings unchanged. State-reset contract test
`every_fold_and_soft_reset_entry_has_dispatch_arm` passes for all 9
new entries; SP5 ISV slot layout test
`sp9_slots_contiguous_above_sp8_block` passes;
`SQLX_OFFLINE=true cargo check -p ml` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
38d2408604 |
spec(sp9): Kelly cold-start warmup floor — fully ISV-driven design
Brainstorm spec for SP9 (Fix 37). Resolves the val-Flat-collapse pathology
surfaced in T10 train-multi-seed-wsnc6 ep1-3: training intent_dist_f=0.47
but eval_dist_f=0.00 because Kelly cap pinned eval mag to Quarter, with
trade_count=1 in 214k bars (chicken-and-egg: cap closed → no trades → no
Kelly samples → cap stays closed).
Architecture per pearl_controller_anchors_isv_driven and
pearl_cold_start_exit_signal_or:
final_kelly_f = max(measured_kelly_f, warmup_floor)
warmup_floor = base_floor × (1 − combined_confidence)
base_floor = clamp(q_var_mag / q_var_mag_ema, MIN, MAX)
combined_confidence = max(statistical, behavioral, temporal)
All thresholds ISV-driven via Pearl D Wiener-α; only Invariant 1 numerical
anchors (clamp ranges, EPS) remain as constants.
Scope: 9 new ISV slots, 6 new GPU producer kernels including a
mandatory eval_dist GPU migration (eliminating DtoH readback at
gpu_backtest_evaluator.rs:1353 per feedback_no_cpu_compute_strict).
~1000-1200 LOC, single atomic commit per feedback_no_partial_refactor.
Pearls authored as part of brainstorm:
- pearl_controller_anchors_isv_driven (regime-encoded constants)
- pearl_cold_start_exit_signal_or (OR'd signals across statistical /
behavioral / temporal axes)
|
||
|
jgrusewski
|
e0dbae3c99 |
fix(sp7): ISV-driven MAX_BUDGET via GPU train_active_frac canary (Fix 36)
Per pearl_controller_anchors_isv_driven.md: Fix 35 (CQL formula direction
flip) is necessary but insufficient — `const float MAX_BUDGET = 1.0f;` at
line 113 of loss_balance_controller_kernel.cu is regime-encoded the same
way the original CQL target_ratio formula was. Under healthy training
the cap is fine; under val-Flat-collapse it lets the controller saturate
budgets to 1.0 while the model is regressing into Flat.
The pearl prescribes the structural fix: pick the canary that fires
under the failure mode → train_active_frac (Long+Short / total during
training rollout). Lift it onto ISV with Pearls A+D smoothing; route it
through a producer kernel that derives per-(head, branch) cap via linear
interpolation FLOOR + active_frac × (CEIL - FLOOR); read the cap from
ISV in the controller kernel (no more hardcoded 1.0).
Atomic commit per feedback_no_partial_refactor:
- new kernel: train_active_frac_compute_kernel.cu — single-thread
reduction over monitoring_summary[5..17), writes scratch[250]; chained
apply_pearls_ad → ISV[TRAIN_ACTIVE_FRAC_INDEX=321]
- new kernel: loss_balance_max_budget_compute_kernel.cu — 8 threads
(2 heads × 4 branches), reads ISV[TRAIN_ACTIVE_FRAC_INDEX], writes
scratch[251..259); chained apply_pearls_ad ×8 →
ISV[LB_MAX_BUDGET_{CQL,C51}_BASE..+4)
- consumer: loss_balance_controller_kernel.cu — replace
`const float MAX_BUDGET = 1.0f;` with per-(head, branch) ISV reads,
defensive Pearl A bootstrap clamp
- 9 new ISV slots @ [321..330); ISV_TOTAL_DIM 321 → 330;
SP5_PRODUCER_COUNT linear span 147 → 156
- 9 new scratch slots @ [250..259); SP5_SCRATCH_TOTAL 250 → 259
- delete CPU train_active_frac compute (training_loop.rs:3392-3396) per
feedback_no_cpu_compute_strict; delete host field
`last_train_active_frac: f32`; replace with accessor reading from ISV
- 3 new state-reset registry entries (FoldReset; Pearl A bootstrap on
first observation) + dispatch arms in reset_named_state
- audit doc: Fix 36 entry with full provenance, files touched, pearls
applied, verification
Pre-existing 18 cargo warnings unchanged. State-reset contract test
`every_fold_and_soft_reset_entry_has_dispatch_arm` passes for all 3 new
entries; SP5 ISV slot layout test
`sp8_max_budget_slots_contiguous_and_above_activation_block` passes;
`SQLX_OFFLINE=true cargo check -p ml` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
65c3083dec |
fix(sp7): flip CQL target_ratio direction — death spiral on magnitude
`target_ratio[CQL][b] = ANCHOR_CQL_RATIO * (1 - flatness[b])` had the sign inverted: it pushed CQL HIGH when Q was flat (the collapse state) and LOW when Q had variance. Per pearl_2_budget_kernel.cu, `flatness = var_q / σ²`, so flatness HIGH means Q has variance — exactly when overconfidence-risk-driven CQL pressure should kick in. Symptom (T10-v3 train-multi-seed-x7sl2 logs): Once IQN Fix 34 woke IQN-mag/ord/urg branches and produced real Q-targets, the controller's inverted formula sent cql_budget to MAX_BUDGET=1.0 saturation on every branch, the conservative pull kept Q flat, the model collapsed to single-Flat-action eval (val_active_frac=0, dir_entropy=0, trade_count=1, sharpe=0). Fix is one operational character: `(1 - flatness)` → `flatness`. C51 formula was already correctly aligned (`flatness * ANCHOR_C51_RATIO`) and unchanged. Per pearl_controller_anchors_isv_driven.md: the inverted formula was masked for months because the IQN-dead regime (Fix 34) suppressed cql_raw on mag/ord/urg, never letting the controller engage with the inverted target. Fixing the IQN regime exposed the dormant formula bug. ANCHOR_CQL_RATIO=2.0 and MAX_BUDGET=1.0 remain hardcoded; Fix 36 will make those ISV-driven via a GPU train_active_frac canary signal per the pearl's "pick the canary that fires under the pathology" rule. |
||
|
jgrusewski
|
9b5296b2f0 |
fix(iqn): set_cached_target_h_s2 per branch — fixes silent 3/4 IQN branch dropout
execute_training_pipeline consumes cached_target_h_s2_ptr via .take(), so
the SP6 Pearl 5 per-branch loop (4 sequential iqn calls per step) only
worked for branch 0. Branches 1, 2, 3 saw cached_ptr=None, returned a
"non-fatal" Err logged as a warning, and silently skipped apply_iqn_
trunk_gradient — only the direction branch's IQN quantile signal ever
reached the trunk, for months.
Surfaced in T10-retry train-multi-seed-ksjcm logs as repeated:
WARN: IQN parallel branch 1 step failed (non-fatal):
WARN: IQN parallel branch 2 step failed (non-fatal):
WARN: IQN parallel branch 3 step failed (non-fatal):
cached_target_h_s2_ptr is None.
Pre-existing since SP6 Pearl 5 (P4.T3-era multi-quantile IQN per-branch
τ schedules) — masked by the non-fatal warning classification.
Likely explains:
- cql_mag persistence at SP7 smoke (cql_mag=0.07 vs cql_dir=1.0):
CQL was the only learning signal mag had because IQN-mag was dead.
- SP7 c51 1000:1 controller suppression on mag/ord/urg may stabilize
at non-collapse values once IQN signal returns to those branches.
Fix is local: re-set cached ptr inside both 4-branch loops (parallel arm
near line 2123, sequential arm near line 2308). The ptr is the same
value (target_h_s2 is per-step, not per-branch), but the .take()
contract requires set-per-call. Defensive single-shot semantic preserved
— if a future bug skips the set, IQN errors loud rather than reusing
stale ptr.
Per feedback_no_partial_refactor: SP6 changed the call pattern 1→4 per
step but didn't migrate the cache contract. Per feedback_no_hiding: the
non-fatal classification hid the structural bug; escalation to hard
error deferred to next commit after post-fix T10 validates the warnings
disappear.
Audit doc updated with Fix 34 entry.
|
||
|
jgrusewski
|
07f5ed5d74 |
fix(sp7): GPU dispatch for cql/c51 budget — eliminate capture-time host-branch freeze
The SP7 controller wrote real budgets to ISV[BUDGET_CQL_BASE/C51_BASE] and
the activation flag correctly transitioned 0→1, but downstream SAXPY ops
still saw exactly 0.02/0.05 every step. Root cause: compute_adaptive_budgets
ran host-side Rust with `if cql_active >= 0.5 { real } else { bootstrap }`,
captured into the aux_child CUDA Graph at step 0 of each fold (when
cql_active is FoldReset sentinel 0.0). The bootstrap branch resolves to
literal 0.02/0.05 and freezes into kernel arg buffers; ~1000 graph replays
per fold use frozen scalars regardless of runtime ISV state.
Same disease as SP4 host-side EMA elimination 2026-05-01. Fix:
- New consume_lb_budget_kernel.cu (8 threads, 1 block): reads ISV slots
for activation flag + controller budget per (head, branch); applies
if/else dispatch on-device; writes trunk_mean + per-branch correction
to mapped-pinned scratch buf [10] = (cql_trunk + cql_corr×4 + c51_trunk
+ c51_corr×4). Same cubin houses dqn_{scale,saxpy}_f32_dev_ptr_kernel
variants reading alpha from a device pointer (no kernel-arg freeze).
- Buffer + launcher + cubin loading + 4 dev-ptr accessor methods
in gpu_dqn_trainer.rs.
- apply_c51_budget_scale / apply_cql_saxpy + per-branch variants take
alpha_dev_ptr: u64 instead of scalar f32. The "skip when ≈ 1.0"
optimization is dropped — the value lives on-device. Existing scalar
dqn_{saxpy,scale}_f32_kernel are NOT modified — they remain reused
by distill, VSN dW dilution, etc.
- compute_adaptive_budgets refactored to launch the dispatch kernel
inline (captured along with consumers; replay-time fresh values
every step) and stop returning host-resolved CQL/C51 scalars.
IQN/ENS keep their host-side resolution (out-of-scope per change
scope — no activation-flag dispatch).
- HEALTH_DIAG reads from lb_budget_effective_buf via new accessor
methods on FusedTrainingCtx; dead caches on GpuDqnTrainer
(last_{cql,c51}_budget_eff / _per_branch) deleted.
- Audit doc Fix 33. Memory pearl out-of-tree.
Bootstrap byte-equivalence preserved: kernel-arg constants
cql_bootstrap=0.02 / c51_bootstrap=0.05 remain byte-identical to the
prior host-side CQL_BOOTSTRAP_BUDGET / C51_BOOTSTRAP_BUDGET literals
and to loss_balance_controller_kernel.cu's COLD_START_FLOOR_* anchors.
Files: consume_lb_budget_kernel.cu (new), build.rs, gpu_dqn_trainer.rs,
fused_training.rs, training_loop.rs, dqn-wire-up-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
14af4854fb |
fix(merge): resolve sp5→main merge fixups — DevicePtrMut import + adam_step 7-arg signature
After sp5 merge with -X theirs, two compile errors needed manual fix: - gpu_her.rs lost the DevicePtrMut trait import - gpu_tlob.rs Fix 20 regression test used the 4-arg adam_step signature before sp5's Pearl 4 added β1/β2/ε params. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
6c3a91c878 |
fix(sp7): wire raw CQL norm kernel reading cql_grad_scratch directly
The SP7 controller's CQL reference signal was reading cql_sx (post-SAXPY, budget-scaled) which created a self-perpetuating deadlock at small budget values: cql_sx_norm = budget × raw_grad → small budget → small cql_sx → controller can't update → budget stays small. The earlier offset 6 → 3 attempt failed because grad_decomp_launch_cql was never effectively populating slot 3 — the snapshot pattern measures ‖grad_buf − snapshot‖, but apply_cql_gradient writes to cql_grad_scratch (separate buffer), not grad_buf, so the snapshot delta is always 0. This commit adds a real producer kernel cql_raw_norm_compute that reads cql_grad_scratch directly and computes ‖raw_cql‖ over mag/dir/trunk slices. Wired to fire AFTER apply_cql_gradient and BEFORE apply_cql_saxpy, populating grad_decomp_result_pinned[3..6] with the raw norm independent of cql_budget. SP7 launcher updated to read from offset 3 (now: real raw CQL norm, not the never-populated cql snapshot delta). HEALTH_DIAG label renamed cql_sx → cql_raw to reflect the new contract; component index in the cached grad_component_norms_* arrays switched 2 → 1. The historical grad_decomp_launch_cql() call is removed — keeping it would overwrite the slot with 0 after cql_raw_norm_compute fires. The paired grad_decomp_snapshot_cql snapshot is left in place to scope the diff to Path A; buffer cleanup (grad_snapshot_cql allocation + grad_decomp_launch_cql definition) belongs in a follow-up commit per feedback_no_partial_refactor. Files: cql_raw_norm_kernel.cu (new, 97 LOC), build.rs, gpu_dqn_trainer.rs (struct field + cubin static + load + launcher + SP7 read offset 6→3), loss_balance_controller_kernel.cu (docstring + arg comment), fused_training.rs (4 launch_cql_raw_norm call sites, 1 dead grad_decomp_launch_cql call removed), training_loop.rs (HEALTH_DIAG label + index), audit doc Fix 31 SP7 Path A entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
6cf6f26ab9 |
diag(sp7): emit grad_decomp_pinned + lb_active_per_branch HEALTH_DIAG
The SP7 smoke at
|
||
|
jgrusewski
|
237b3dbfb0 |
fix(sp7): bump ISV_TOTAL_DIM 294→321 to match SP5_SLOT_END
SP7 T1 added 24 ISV slots (LB_DIFF_VAR_CQL_BASE=297 ... LB_C51_ACTIVE through 321) without bumping ISV_TOTAL_DIM, leaving SP7's wiener-state and activation slots out-of-bounds of the allocated pinned buffer (294 * 4 = 1176 bytes; SP7 slots write to bytes 1188..1284). GPU direct-pointer writes silently corrupted memory in the next page; CPU read_isv_signal_at returned garbage in release builds. This explains the SP7 smoke's confusing zero-budget-everywhere observation: the activation flag was reading OOB memory, the consumer saw inconsistent values, and the controller's actual ISV state was never visible. Bumped ISV_TOTAL_DIM to 321 (max valid index 320, covering SP5_SLOT_END-1). Made pub(crate) so the new contract test can reference it from sp5_isv_slots.rs. Updated layout_fingerprint_seed()'s slot entries to include the previously missing D3 and SP7 slots (TRAINING_SHARPE_EMA=294 through LB_C51_ACTIVE_BASE=317) and updated ISV_TOTAL_DIM= literal to 321 in lockstep per feedback_no_partial_refactor. Added contract test all_sp5_slots_fit_within_isv_total_dim to permanently gate this class of bug at cargo test time. The test would have caught SP7 T1 instantly; future slot allocations cannot regress this. Files: gpu_dqn_trainer.rs (ISV_TOTAL_DIM + comment + fingerprint), sp5_isv_slots.rs (test), docs/dqn-wire-up-audit.md (Fix 31 sub-bullet). Cargo check workspace clean. Cargo test ml --lib 935 passed (934 baseline + 1 new contract test); 16 pre-existing GPU-hardware failures unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
jgrusewski
|
bbf15ad690 |
diag(sp7): HEALTH_DIAG emit Q_VAR_PER_BRANCH signal
The SP7 controller's flatness gate reads per-branch Q-variance from ISV[Q_VAR_PER_BRANCH_BASE=222..226) but that signal was not visible in HEALTH_DIAG. The existing "var_q" in the main HEALTH_DIAG line is realized step-return variance per magnitude bin from gpu_experience_collector — a trade-outcome metric, semantically distinct from per-branch Q-output variance. Added one emit line immediately after cql_budget_per_branch: HEALTH_DIAG[E]: q_var_per_branch [dir=X.XXXX mag=X.XXXX ord=X.XXXX urg=X.XXXX] Reads ISV[222..226) via read_isv_signal_at — the same slots written by q_branch_stats_kernel.cu (scratch slot 2 per branch) and routed into ISV by apply_pearls_ad_kernel in launch_sp5_pearl_1_atom. No new ISV slots, no kernel change, no StateResetRegistry entry. Class 2 signal (mag_concat_scale / q_rms): Option A infeasible — q_rms is a per-sample register variable in mag_concat_qdir with no existing ISV slot; h_s2_rms_ema at ISV[96] is the only available proxy. Option B (new ISV slot) blocked pending explicit controller OK. See audit doc for full stop-and-report rationale. Files touched: crates/ml/src/trainers/dqn/trainer/training_loop.rs (+28 LOC) docs/dqn-wire-up-audit.md (+13 LOC) [ISV slot decision: Option A reused existing Q_VAR_PER_BRANCH_BASE=222..226] Cargo check workspace clean. State-reset contract test passes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
jgrusewski
|
cb6a89714b |
sp7(controller): activation flag + Welford-α to escape bootstrap trap
Fixes the SP7 controller dormancy discovered in smoke-test-8556k:
per-branch budgets stuck at exactly bootstrap constants because the
kernel's cold_start_basis numerically equaled the consumer's bootstrap
fallback, AND the Wiener-α was clamped at ALPHA_FLOOR=1e-4 from step 1.
Architectural change:
- 8 new ISV slots LB_{CQL,C51}_ACTIVE_BASE per (head × branch).
Activation flag is monotonic per fold, FoldReset on boundary.
- Kernel only sets active=1 when grads are populated AND on subsequent
active-state computation; cold-start branch leaves active=0 (fresh
branch) or holds prior budget steady (transient grad gate).
- Consumer dispatches on activation: bootstrap when active<0.5,
controller verbatim when active>=0.5. No more spurious bootstrap
when controller writes legitimate small values.
- Welford-α hybrid (max of 1/max(1,epoch_idx_in_fold) and Wiener-α)
gives full update on first active step, falls off as 1/N until
Wiener takes over with meaningful variance estimates. EPOCH_IDX_INDEX
is the existing per-fold-reset counter (no new tuned constants).
State reset registry: 2 new sp7_lb_*_active FoldReset entries +
matching dispatch arms in reset_named_state. Contract test
(every_fold_and_soft_reset_entry_has_dispatch_arm) gates compile.
GPU unit test sp7_loss_balance_controller_activation_flag_transitions
exercises 3 transitions (cold start → both flags 0; active → both
flags 1 with controller-computed budget != bootstrap; transient grad-
gate → flags hold at 1, prior budget held verbatim). Passes on local
RTX 3050 Ti.
Audit doc: Fix 31 sub-bullet describing the activation-flag fix.
Memory pearl out-of-tree (controller will dispatch separately).
Touched:
crates/ml/src/cuda_pipeline/sp5_isv_slots.rs
crates/ml/src/cuda_pipeline/loss_balance_controller_kernel.cu
crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
crates/ml/src/trainers/dqn/fused_training.rs
crates/ml/src/trainers/dqn/state_reset_registry.rs
crates/ml/src/trainers/dqn/trainer/training_loop.rs
crates/ml/tests/sp5_producer_unit_tests.rs
docs/dqn-wire-up-audit.md
Cargo check workspace clean. Cargo test ml --lib clean (incl. contract
test + 6 sp5_isv_slots tests). 16 pre-existing failures unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|