Phase 1 of the foxhunt specialized agents/skills rollout is complete: 14
commits, 5 auditor agents, 7 workflow/maintenance skills, 2 helper scripts,
warn-only PostToolUse hook router. All 8 acceptance criteria from the spec
verified. Hook latency 11-13 ms per call (target <200 ms). Memory-write
invariant held: only pearl-distiller and memory-curator are authorized
writers, no agent-driven memory edits during rollout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
14-task plan covering: foundation hook router (Task 1), 5 auditor agents
(Tasks 2-5, 13), 5 workflow skills (Tasks 6-10), 2 maintenance skills
(Tasks 11-12), end-to-end acceptance check (Task 14). Tasks 2-5, 6-8, 9-10,
11-12 fan out in parallel. Task 13 (sp-critical-reviewer) composes the four
domain auditors and is built last.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Append a "Plan Accuracy Errata" section to the SP19+20 plan
documenting the 6 deviations from the original Phase 2 plan that
emerged during implementation. Each entry captures the gap, the
decision made, and the rationale, so future implementers see what
was actually built vs what was specified.
Gaps documented:
1. Task 2.0 (label-at-open infrastructure) was NEW — split out from
Task 2.2 to land buffer + alloc + reset + write site atomically
before Task 2.2's consumer ships. Sign convention mapping detail
captured (kernel emits {0, 1, -1}, SP20 spec uses {-1, 0, +1}).
2. Per-env alpha plumbing was not in Task 2.2 plan scope — built in
Task 2.2 (NOT deferred to Phase 4) to preserve the
`feedback_no_partial_refactor` contract atomicity. Touches 5
files in one commit.
3. Per-bar SP18 D-leg sites — KEEP for Phase 2 per `feedback_no_stubs`.
Plan's wording "DELETE [these helpers]" was overbroad; only the
trade-close-site call is deleted. The phantom
`compute_sp12_reward_with_cost` doesn't exist as a function (the
SP12 v3 reward is the inlined block).
4. `sp20_compute_event_reward` placement — new dedicated
`sp20_reward.cuh` header (NOT inside `experience_kernels.cu`,
NOT inside `trade_physics.cuh`). Mirrors the
compute_asymmetric_capped_pnl / compute_min_hold_penalty
header-only pattern; needed for GPU oracle test wrapper to share
the function bit-for-bit per `feedback_no_cpu_test_fallbacks`.
5. Task 2.3 was subsumed by Task 2.2 — the existing Path C chain
consumes the new `alpha` field automatically once `alpha_per_env`
is wired; no separate `sp20_emas_compute` producer call needed.
6. `min_hold_*` kernel-arg trio cleanup deferred to Task 2.4 — the 3
kernel args were deleted in Task 2.2 (per `feedback_no_hiding`)
but the upstream producer chain (`min_hold_temperature_update_kernel`,
ISV[460], `read_min_hold_temperature_from_isv`, `config.min_hold_*`)
deferred to Task 2.4 because it touches SP14 ISV slot registry +
StateResetRegistry + ISV layout fingerprint bump.
Implementation-level details remain in `docs/dqn-wire-up-audit.md`
Task 2.0 / 2.1 / 2.2 entries; this errata is the plan-level
"what was actually built vs what was specified".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Atomic Phase 1.4 commit per `feedback_no_partial_refactor`. Wires the
SP20 fused-producer chain (Stats → Aggregate → EMAs → Controllers) into
GpuExperienceCollector's per-rollout-step path with one new aggregation
kernel + Phase 1.2 EMA kernel-signature refactor + production wire-up
+ tests + audit/spec/plan amendments, all in one commit.
## Path C decision rationale
The Phase 1.2 EMA kernel originally took 9 scalar value-args. The Phase
1.4 wire-up site (per-rollout-step in GpuExperienceCollector) needs to
feed per-env GPU-resident trade signals (trade_close_per_sample,
step_ret_per_sample, hold_at_exit_per_sample, packed factored
actions_out) into the kernel — forbidden via host sync
(`feedback_cpu_is_read_only`) and via memcpy_dtoh/htod
(`feedback_no_htod_htoh_only_mapped_pinned`). Path C refactors the EMA
kernel to take a device struct pointer (`SP20EmaInputs* ema_inputs`) +
adds an aggregation kernel that writes the struct on the GPU. Phase 1.2
had zero production consumers yet, so the sig change + Phase 1.4 wire-up
land atomically.
## What this commit contains
1. **EMA kernel signature refactor** (Phase 1.2 → Path C):
`sp20_emas_compute_kernel.cu` swaps 9 scalar args for a single
`const SP20EmaInputs* __restrict__ ema_inputs` device pointer.
Math semantics bit-identical. Rust `EmaInputs` mirrored as
`#[repr(C)]` byte-for-byte; new `pack_inputs_into_f32_view` helper
for tests + collectors that fill the struct via a mapped-pinned
f32-aliased buffer.
2. **New aggregation kernel** (`sp20_aggregate_inputs_kernel.cu`):
Per-env arrays (sliced to current rollout step) + sp20_stats outputs
(p50/std) + aux_dir_acc_reduce_kernel output [0] → SP20EmaInputs
struct. Block tree-reduce (4 stripes × bdim × 4 bytes shmem); no
atomicAdd. Aggregation rules per spec §4.5:
- is_close = OR over envs
- is_win = (≥ 0.5 of closed envs were wins)
- trade_duration = round(mean(hold_at_exit) over closed envs)
- action_is_hold = strict majority (count*2 > n_envs) HOLD
- alpha = 0.0 (Phase 2 forward ref — reward kernel)
- per_bar_hold_reward = 0.0 (Phase 3.2 forward ref — Hold-cost dual)
- aux_logits_p50/std/dir_acc forwarded from upstream
3. **Production wire-up in GpuExperienceCollector**:
4 kernel handles + 4 mapped-pinned buffers (struct fields, alloc,
init); per-rollout-step launch sequence after env_step (step 5c):
`Stats → Aggregate → EMAs → Controllers`. All on the same stream;
stream-implicit producer→consumer ordering. Gated on
`isv_signals_dev_ptr != 0 && trainer_params_ptr != 0` (matches the
existing SP14-β EGF chain pattern).
4. **Fold-boundary reset**:
3 new RegistryEntry records (sp20_ema_inputs_buf,
sp20_emas_internal_buf, sp20_emas_obs_count_buf) + 13 new dispatch
arms in training_loop.rs (10 SP20 ISV slot resets + 3 buffer resets).
Closes the pre-existing `every_fold_and_soft_reset_entry_has_dispatch_arm`
regression (was failing on fresh check after the SP20 ISV slot
registrations landed in commit 4249ebc96).
5. **HEALTH_DIAG emit**:
Per-epoch `HEALTH_DIAG[N]: sp20_isv [loss_cap=… alpha_ema=…
wr_ema=… hold_cost_scale=… target_hold_pct=… hold_pct_ema=…
hold_reward_ema=… n_step=… aux_conf_threshold=… aux_gate_temp=…]`
right after the existing q_disagreement_diag emit.
6. **Tests** (5 test files, all green on RTX 3050 Ti sm_86):
- sp20_emas_compute_test.rs (4 GPU oracle): updated to use the
post-Path-C buffer-arg API (mapped-pinned f32-aliased struct).
- sp20_aggregate_inputs_test.rs (NEW, 6 GPU oracle): aggregation
rule coverage including the Phase 2 / Phase 3.2 placeholder
contract.
- sp20_phase1_4_wireup_test.rs (NEW, 2 GPU oracle): end-to-end
4-kernel chain integration test.
- sp20_stats_compute_test.rs (4 GPU oracle): unchanged, regression.
- sp20_controllers_compute_test.rs (7 GPU oracle): unchanged,
regression.
- 18 lib unit tests across the SP20 launchers.
7. **Phase 2 / Phase 3.2 forward references**:
The `alpha` (Phase 2) and `per_bar_hold_reward` (Phase 3.2) fields
are emitted as 0.0 placeholders and documented at:
- `sp20_aggregate_inputs_kernel.cu:46-52` (in-kernel docstring)
- `sp20_aggregate_inputs.rs:46-49` (launcher docstring)
- `dqn-wire-up-audit.md` "Phase 2 / Phase 3.2 forward references"
These are NOT stubs — Phase 2 / 3.2 will replace the kernel's 0.0
writes with real signals atomically with their respective producers.
The original Task 2.3 (host-side aggregation) is **subsumed** by
the GPU-side aggregation kernel.
## Hard rules
- `feedback_no_partial_refactor` — kernel sig change + aggregation
kernel + production wire-up + tests + audit/spec/plan amendments
in one atomic commit
- `feedback_no_atomicadd` — aggregation kernel uses block tree-reduce
- `feedback_no_cpu_compute_strict` — every aggregation lives on GPU
- `feedback_no_htod_htoh_only_mapped_pinned` — every Phase 1.4 buffer
is mapped-pinned with `cuMemHostAlloc(DEVICEMAP)` reachable via
`dev_ptr`; no memcpy_dtoh/htod
- `pearl_first_observation_bootstrap` — counter-based bootstrap
preserved; placeholder writes (0.0) keep the ALPHA / HOLD_REWARD
EMAs at 0.0 sentinel until Phase 2 / 3.2 wire real producers
- `pearl_no_host_branches_in_captured_graph` — every kernel is single-
block; no host branches; safe to capture in the per-step CUDA Graph
- `pearl_tests_must_prove_not_lock_observations` — integration test
asserts invariants (slots populate, bounds respected) rather than
locked observed values
## Verification
- `cargo check -p ml --features cuda` clean
- 18 lib unit tests for SP20 launchers pass
- 23 GPU oracle tests across 5 test files pass on RTX 3050 Ti (sm_86)
- `every_fold_and_soft_reset_entry_has_dispatch_arm` regression test
now passes (was failing pre-change)
- 14 baseline lib test failures unchanged (none introduced by this
commit)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Phase 1.1 sp20_stats_compute kernel (de922c6a4) assumed the SP14-C aux
head emits 3-class logits {short, hold, long} with baseline 1/3. This was
an error in the SP19+20 spec — production aux head emits K=2 logits
{down, up} per gpu_aux_heads.rs:61 (AUX_NEXT_BAR_K = 2, established by
SP13 B1.1a). Wiring K=2 production aux into the K=3 kernel = OOB reads +
corrupt stats.
Retargets the kernel + launcher + tests to K=2 atomically per
feedback_no_partial_refactor:
- Kernel: SP20_K_CLASSES 3→2; SP20_UNIFORM_K3 0.333…→SP20_UNIFORM_K2 0.5f;
Pass A reads 2 logits (was 3); aux_conf range [0, 1/2] (was [0, 2/3]).
- Launcher: AUX_K_CLASSES 3→2; renamed unit test
aux_k_classes_is_three → aux_k_classes_matches_production_aux_head.
- Tests: rewrote CPU oracle for K=2 input shape and 0.5 baseline; updated
expected p50/std ranges; redesigned heterogeneous-distribution test to
use ramped (not lockstep) clusters — K=2's saturated softmax in the hot
half collapses every row to bin 255, triggering the
pearl_sp4_histogram_warp_tile_undercount trap; ramped clusters distribute
bin indices across each warp's 32 lanes so the histogram path matches
the CPU oracle within bin_width tolerance.
Phase 1.2 (sp20_emas_compute) and Phase 1.3 (sp20_controllers_compute)
consume the scalar [p50, std] outputs and do NOT carry the K dimension.
Both regression suites verified passing unmodified:
- sp20_emas_compute_test: 4 GPU + 1 unit, all pass.
- sp20_controllers_compute_test: 7 GPU, all pass.
Verification (RTX 3050 Ti, sm_86):
- sp20_stats_compute_test: 4 GPU oracle + 4 launcher unit, all pass.
- cargo check -p ml --features cuda: clean (pre-existing warnings only).
Spec + plan amended at top with "AMENDED 2026-05-09" notes; audit doc
Phase 1.1 entry has a "K=2 fixup" subsection documenting the change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Q1 design decision (b): add explicit hold_reward_ema to center the per-bar
Hold reward, so Q(Hold) and Q(trade) targets are scale-comparable. The
marginal Q(trade) > Q(Hold) preference now comes from data variance in
each state (high-aux states pull Q(Hold) more negative), not from
structural scale asymmetry that depends on cost_scale magnitude.
Q2 decision: keep 4-quadrant fixed (no ramping partials). Already in spec.
Changes:
- §4.2 Hold opp-cost: dual emission documented — R_per_bar_centered
(= R_per_bar - hold_reward_ema) for Q-target/replay tuple,
R_per_bar uncentered for hold_baseline_buffer (Component 1 baseline)
- §4.5 Kernel 1: hold_reward_ema added (per-step on Hold-state bars only)
- §5 data flow: per-bar reward path shows the centered/uncentered split
- ISV slots: 9 → 10 (HOLD_REWARD_EMA_INDEX added)
- §8 footprint: Component 2 LoC 70 → 90 (+20 for dual emission)
- Total LoC estimate: 1620
Drafts `pearl_diagnostic_decomposition_before_reward_intervention` per
the plan's Task 0.5 deliverable. Captures the discipline pattern
applied in Tasks 0.1–0.2:
Before changing reward weights or Bellman target arithmetic to fight
a Q-attractor, instrument a per-action decomposition of the existing
reward components AND a trajectory observable on the Q-target
distribution. The instrumentation is observability-only and lands
atomically. One epoch of dispatch surfaces whether the suspected
pathology is real BEFORE any production-path code changes.
Pearl is DRAFT — pending the Task 0.3 (reviewer L40S 1-epoch
dispatch) + Task 0.4 (KILL CRITERION evaluation) outcomes. Promotion
to user memory + cross-reference to MEMORY.md is gated on both legs
returning PROCEED.
Cross-references: pearl_canary_input_freshness_launch_order,
pearl_first_observation_bootstrap, pearl_wiener_alpha_floor_for_
nonstationary, feedback_no_partial_refactor, feedback_wire_everything_up.
No production code changes — single doc-only commit.
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
§ Phase 0 Task 0.5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the SP17 PP.1 plan-copy pattern. Plan and spec source-of-truth
live on the sister `feat/sp17-dueling-q-network` worktree; this commit
copies them onto the SP18 branch so subsequent commits reference local
paths.
Spec: docs/superpowers/specs/2026-05-08-sp18-reward-shape-hold-attractor-design.md
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan was authored on sister worktree (sp15-phase1-honest-numbers) by
the SP17 planner agent; copying it to feat/sp17-dueling so future task
references in commit messages and audit-doc entries resolve from this
branch.
No content change vs sister-worktree copy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per train-multi-seed-hjzss validation: SP16 T1+T2 chain was structurally
landed but BEHAVIORALLY INERT in 5-epoch smoke (bit-identical to pfh9n
baseline through epoch 3). Root cause: hardcoded `alpha = 0.05f` in both
producer kernels violates feedback_isv_for_adaptive_bounds AND prevents
convergence in short runs (~60 epochs needed from cold start).
Fix per pearl_wiener_optimal_adaptive_alpha:
α = diff_var / (diff_var + sample_var + ε)
Where sample_var = running variance of target signal (Welford accumulator)
and diff_var = running variance of consecutive one-step differences.
Cold-start: target jumps 1.0 → 6.4 → 7.0 → high diff_var → α ≈ 0.6+
→ near-bootstrap responsiveness in epochs 1-3
Steady-state: signal stabilizes → diff_var drops → α decays naturally
→ smoothing emerges without hardcoded constant
Adds 12 new ISV slots (6 per producer):
- HCS_TARGET_MEAN/M2, HCS_DIFF_MEAN/M2, HCS_PREV_TARGET, HCS_SAMPLE_COUNT
- MHT_TARGET_MEAN/M2, MHT_DIFF_MEAN/M2, MHT_PREV_TARGET, MHT_SAMPLE_COUNT
ISV_TOTAL_DIM 462 → 474.
Both kernels migrated atomically. Pearl-A bootstrap preserved (sentinel
on prev_blended triggers REPLACE; cold-start α=1.0 when N<3 samples).
Defensive bounds [WELFORD_ALPHA_MIN=0.01, WELFORD_ALPHA_MAX=0.95] on the
Wiener-derived α to guard against denormal/underflow corner cases.
HEALTH_DIAG[N] emit extended with `alpha=...` and `sample_count=...` for
direct trajectory observation in validation smoke.
Behavioral tests verify:
- α high during signal jumps (>0.3 at epoch 3 post-cold-start)
- α low in steady state (mean tail α<0.4 under converging signal)
- Pearl-A bootstrap fires on first observation (Welford state advances
regardless of REPLACE branch)
- α stays within [WELFORD_ALPHA_MIN, WELFORD_ALPHA_MAX] over 50 epochs
(post-cold-start; cold-start α=1.0 by design)
- No 0.05f hardcoded literal remains in blend math (regression-locked
via host-only string scan)
5 GPU + host tests pass: sp16_phase3_alpha_high_during_signal_jump,
alpha_low_in_steady_state, pearl_a_bootstrap_first_obs,
alpha_naturally_bounded, no_hardcoded_alpha. sp14 + sp15 oracle suites
unchanged (34 GPU tests + 4 host tests).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per train-multi-seed-pfh9n post-mortem follow-up: slot 460 stuck at 50
in Fold 1 was NOT a launch-lifecycle bug. Producer fires per-epoch but
kernel had early-return guard on AUX_DIR_ACC_SHORT_EMA (slot 373) at
sentinel 0.5. Slot 373 reset on fold boundary; aux dir-acc EMA either
didn't fire or settled within ε of 0.5 → kernel kept early-returning.
Fix: drop slot 373 dependency entirely. Drive temperature from observed
hold-rate vs target overrun:
overrun = max(0, observed_hold_rate - target_hold_rate)
overrun_norm = clamp(overrun / max(target, 0.01), 0, 1)
new_temp = TEMP_MIN + (TEMP_MAX - TEMP_MIN) × overrun_norm
blended_temp = Welford EMA α=0.05 with Pearl-A bootstrap
When over-holding: temp HIGH → exit ramp permissive (matches design intent).
When at/under target: temp LOW → exit penalty strict.
Survives fold reset: hold-rate measurement starts fresh with real data
immediately, no chained-input-sentinel masking.
Slot 330 (KELLY_WARMUP_FLOOR) investigated and confirmed NON-BUG: producer
behaves correctly per pearl_kelly_cap_signal_driven_floors cross-fold-
persistence. floor=0 post-warmup is correct steady state.
Behavioral tests:
- sp16_phase1_min_hold_temp_climbs_with_hold_overrun
- sp16_phase1_min_hold_temp_strict_when_at_target
- sp16_phase1_min_hold_temp_strict_when_under_target
- sp16_phase1_min_hold_temp_no_longer_reads_slot_373
Instrumentation: HEALTH_DIAG[N]: min_hold_temp_diag obs/target/overrun/temp.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Atomic flip of the aux heads' input from Q's GRN trunk output `save_h_s2`
to the SEPARATE aux trunk's output `h_s2_aux`. The aux trunk now trains
its own w1/w2/w3/b1/b2/b3 from CE loss (next-bar + regime); Q's encoder
is structurally protected by `aux_trunk_backward`'s missing `dx_in`
output param (encoder boundary stop-grad enforced at the kernel-set
level).
Reverts the C.0 stop-grad band-aid commits (`872bd7392`, `411a30473`):
the zero-fills in `aux_next_bar_backward` + `aux_regime_backward` Step 3
are replaced with the genuine SAXPY-back-to-input gradient
(`dh_s2_aux[b,j] = sum_k sh_dh_pre[k] * w1[k,j]`). The leak that
motivated stop-grad is now blocked structurally rather than by data
zero-fill — aux gradient flows through the aux trunk's own params, never
into Q's encoder.
Wired in this commit (atomic, ~330 LOC):
- 4 kernel signatures renamed `h_s2 → h_s2_aux` / `dh_s2_out → dh_s2_aux_out`
(`aux_next_bar_forward`, `aux_regime_forward`, `aux_next_bar_backward`,
`aux_regime_backward`); Rust wrappers in `gpu_aux_heads.rs` follow
- Trainer fwd: insert `aux_trunk_forward_ops.launch(...)` in
`aux_heads_forward` Step 0, populating `h_s2_aux` from `save_h_s1`
(encoder layer-1 output, dim=shared_h1=256). Both head fwds redirect
input pointer from `save_h_s2` to `h_s2_aux`
- Trainer bwd: SAXPY both `aux_dh_s2_*_buf` into `dh_s2_aux_accum`
(pre-zeroed each step via graph-safe `cuMemsetD32Async`); then
`aux_trunk_backward_ops.launch(...)` propagates through w3/w2/w1 +
b3/b2/b1; then `launch_aux_trunk_adam_update` applies global L2-norm
clip + per-tensor Adam updates over 6 grad tensors
- Collector fwd: insert `exp_aux_trunk_forward_ops.launch(...)` after
`forward_online_f32`, reading `exp_h_s1_f32` and writing `exp_h_s2_aux`;
redirect `exp_aux_heads_fwd.forward_next_bar` input from
`exp_h_s2_f32` to `exp_h_s2_aux`
- Pre-capture host-write of ISV-driven LR + grad-clip + step counter
into mapped-pinned buffers in `launch_cublas_backward_to` (BEFORE
`aux_heads_backward`); same `&mut self` pattern as `step_ofi_embed_adam`
Verification:
- `cargo check -p ml --tests` clean (1m02s, only pre-existing warnings)
- `aux_trunk_oracle_tests` + `sp14_oracle_tests` 12/12 pass:
- aux_trunk gradient check: max_rel_err=1.33e-2 (tol=2e-2) — matches C.4 baseline
- aux_trunk_backward_does_not_write_dx: kernel source clean of dx_in/dx_in_out
- aux_sign_label_lookahead_mask: 60/100 masked, 40/100 valid
- 9 other oracle tests pass bit-identically
Plan: docs/superpowers/plans/2026-05-07-sp14-layer-c-separate-aux-trunk.md §C.5b
Audit: docs/dqn-wire-up-audit.md "SP14 Layer C Phase C.5b" section
Phase C.5a — additive infrastructure for the aux trunk wire-up.
Allocates saved-fwd buffers (h_s2_aux, h_aux1, h_aux2),
gradient buffers (6× aux_trunk_*_grad), accumulator
(dh_s2_aux_accum), and dedicated Adam launcher
(launch_aux_trunk_adam_update) reading β1/β2/ε/LR/grad-clip from
ISV[444..449).
No contract change. No call sites for the new launcher yet.
C.5b atomically wires these in.
Phase C.5 was split (authorized 2026-05-08) after the original
implementer flagged ~600-800 LOC scope across 4 files with
correctness windows. C.5a is purely additive; C.5b is the genuine
~300 LOC atomic migration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Aux's original label was (p_{t+1} > p_t) — pure HFT-scale microstructure
noise that's unlearnable at our HFT-MFT trading frequency. Migrated to
(p_{t+H} > p_t) where H is read from ISV[AUX_PRED_HORIZON_BARS_INDEX=450].
Adaptive producer drives H from observed avg winning hold time:
- Pearl-A first-observation bootstrap: replace sentinel H=60 directly
on first valid observation
- Steady-state Wiener-α EMA blend, slow (α=0.01) for stable horizon
(no target-variance EMA available, fallback per
pearl_wiener_optimal_adaptive_alpha)
- "No winning trades yet" guard keeps sentinel until first valid observation
Lookahead truncation: labels at t where t+H >= total_bars are masked
(sentinel -1, loss-reduce skips). The existing aux_next_bar_loss_reduce
in aux_heads_kernel.cu already supports the -1 mask convention via the
B_valid count — no new valid_mask parameter needed.
Step 5b finding: Case B — existing per-sample buffers
(hold_at_exit_per_sample, trade_profitable_per_sample) populated by
unified_env_step_core, but no aggregate ISV slot. Added new aggregator
slot AVG_WIN_HOLD_TIME_BARS_INDEX=451 + new producer kernel
avg_win_hold_time_update_kernel.cu (block-tree-reduce, no atomicAdd).
ISV_TOTAL_DIM bumped 450 → 452.
ATOMIC migration per feedback_no_partial_refactor: both label kernels
(aux_sign_label_kernel.cu trajectory + aux_sign_label_per_step_kernel.cu
per-rollout-step) migrated together to the new
(targets, bar_indices, isv, isv_h_idx, out_labels, total, total_bars)
signature. The lookahead host-passed scalar argument is removed; H is
read from ISV inside the kernel (broadcast value, single read per
thread, on-device clamp [1, 240]).
Producer chain (per-epoch boundary): new
GpuDqnTrainer::launch_aux_horizon_chain orchestrates
avg_win_hold_time_update → aux_horizon_update sequentially alongside
launch_kelly_cap_update at the existing epoch-boundary slot in
training_loop.rs.
Trunk math (C.2/C.3/C.4) unchanged — separate aux trunk is label-
agnostic. Validation in C.10 will use H=60 cold-start; the adaptive
producer drives H from real winning-trade observations.
Tests (8 oracle, 5 new + 3 preserved):
- aux_trunk_forward_matches_numpy_reference (C.3) ✓
- aux_trunk_backward_gradient_check (C.4) ✓
- aux_trunk_backward_does_not_write_dx (C.4) ✓
- aux_sign_label_h_bar_horizon (NEW) ✓
- aux_sign_label_lookahead_mask (NEW) ✓
- aux_horizon_pearl_a_bootstrap (NEW) ✓
- aux_horizon_converges_to_steady_target (NEW) ✓
- aux_horizon_holds_sentinel_with_no_winning_trades (NEW) ✓
8/8 pass on RTX 3050 Ti.
Phase C.4b of SP14 Layer C separate-aux-trunk refactor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Targeted fixes per user direction (option 3 of "fix critical only,
accept rest as execution-time gaps"):
1. build.rs cubin manifest is 1:1 source-to-cubin (verified: list of
.cu filenames, NOT (source, name) tuples). Fixed Phase 3.1 Step 5
to use single manifest entry; both kernels load from SAME module
via get_function() with their symbolic names.
2. egf_anchor_p1 helper in sp4_histogram_p99.cuh: replaced the
`return 0.0f` stub with the full ~80-line body. Pass 1 + Pass 2
byte-identical to sp4_histogram_p99 (mirrored verbatim from the
existing sibling). Pass 3 walks cumulative-from-bottom for p1
instead of cumulative-from-top for p99. Returns lower-edge of the
first bin reaching 1% threshold.
3. Added Task 4.2.5: `fxt evaluate` subcommand (PREREQUISITE for 4.3).
Verified that bin/fxt/src/main.rs Commands enum has no Evaluate
variant. Without this task, eval-final-template.yaml fails at
runtime. Full subcommand shown: bin/fxt/src/evaluate.rs with 5
flags (--checkpoint-dir, --quarter, --seeds, --output-dir,
--report-card-md), report card markdown emitter per spec §10.5.
5 IMPORTANT and 2 NIT issues from review remain documented as
execution-time friction; subagent-driven-development's spec-compliance
reviewer between tasks is expected to catch them per-task.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite of 2026-05-06-sp15-trader-discipline-and-recovery.md from v1
(commit 0178a53ab) which had 16 reviewer-flagged issues including
5 critical compile-breakers and 8 important plan-failure violations.
CRITICAL fixes:
- Real GATE1_OPEN_STATE_INDEX (slot 391) — was wrong GATE1_STATE_INDEX
- Real PS_PEAK_EQUITY/PS_PREV_EQUITY (state_layout.cuh slots 7+9) —
no invented CUMULATIVE_EQUITY_INDEX
- Real sp4_histogram_p99<BLOCK_SIZE> block-tree-reduce pattern, no
atomicAdd_block (was feedback_no_atomicadd violation)
- Real evaluate_dqn_graphed pattern (gpu_backtest_evaluator.rs:1143) —
no nonexistent evaluate_on_val_slice
- Real services/ml_training_service/src/main.rs path — was wrong
IMPORTANT fixes:
- Fork from current main 0178a53ab, not stale 5417e2756
- Phase 0.B has explicit case-(a)/case-(b) branches driven by 0.A
diagnostic conclusion
- Phase 2B uses TEMPLATE + 17-row differential table — every test fully
specified, no compressed bullets
- Phase 3 each teaching gets full TDD: write test → run-fail → kernel
(full code) → launcher → consumer → run-pass → commit (5+ steps)
- Phase 3.5 each mechanism same TDD pattern with full kernel code
- Plasticity 3.5.4 specifies TWO-STEP recovery (Flat first, then
cooldown) + Kaiming-He init (not Xavier — architecturally appropriate)
- Phase 4.3 includes FULL eval-final-template.yaml (not 4 bullets)
- 4 EGF constants replaced (not 3 — verified all four in
alpha_grad_compute_kernel.cu lines 142,143,144,147)
- Behavioral_suite test target ordering fixed (Cargo.toml entry lands
before tests run against it)
Stats: 4947 lines, ~270 [- [ ]] step checkboxes, 0 todo!() in test
bodies, every kernel shown in compilable form.
Self-review section maps every spec section to implementing task,
verifies all cross-references against verified codebase facts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second critical review pass found 10 more issues, 3 critical.
All addressed:
CRITICAL:
- §8.2 (3.1): ALPHA_SPLIT cold-start was unspecified — formula
produces 0/0=0, not the claimed 0.5 sentinel. Now: ISV slot
initialized DIRECTLY to 0.5 in trainer constructor; formula
takes over only after both grad-norm EMAs accumulate N_warm
non-zero observations
- §9.2 (3.5.4): plasticity now performs TWO-STEP recovery:
(1) Flat all positions at fire bar (close current losing trade),
(2) engage warmup cooldown forcing Hold for M_warm bars.
Without step 1, forced Hold preserved the losing position
that drove drawdown for the entire 200-bar warmup
- §12.2: stale "5-10%" baseline cost estimate updated to "15-25%"
matching §6.4 (was contradicting earlier amendment)
IMPORTANT:
- §9.2 (3.5.5): DD_TRAJECTORY_DECREASING threshold 0.02 hardcoded
→ ISV-driven via new slot DD_TRAJECTORY_FLOOR (slot 441,
25th percentile of running dd_pct distribution)
- §8.2 (3.5): HOLD_FLOOR_ALPHA tracked from rolling 95th percentile
of |Q_dir| (NOT running max — was outlier-ratchet vulnerable)
- §9.2 (3.5.3): MEDIAN_STREAK_LENGTH formerly undefined in cooldown
K formula → ISV-driven via new slot 442, running median of
observed loss-streak lengths via two-heap algorithm
- §9.2 (3.5.2): asymmetric reward × α split compound interaction
explicitly stated as intentional with POS_CAP as binding ceiling
NIT:
- §7.4: "2C: Group 3 (4 tests)" → "(5 tests)" (was off-by-one
after 2.22 added)
- §9.2 (3.5.4): Xavier → Kaiming-He init for advantage head reset
(architecturally appropriate for ReLU-gated activation chain)
ISV_TOTAL_DIM: 441 → 443 post-SP15 (added DD_TRAJECTORY_FLOOR
and MEDIAN_STREAK_LENGTH at slots [441..443)). 46 SP15 slots
total. File: 799 → 811 lines.
Spec is now consistent end-to-end with no contradictions between
sections, no hardcoded values violating feedback_isv_for_adaptive_
bounds, and no underspecified load-bearing parameters.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two-sub-project plan:
- Layer A (4 tasks, ~50 LOC): C51 atom-floor, set_aux_weight clamp lift,
stagnation warmup gate, smoke validation
- Layer B (15 tasks, ~1180 LOC): 13 ISV slots, 4 kernels (q_disagreement,
alpha_grad, gradient_hack_detect, dir_concat_qaux), forward wire,
backward gradient gating, orchestrator wire-up, HEALTH_DIAG, tests, smoke
Each task has bite-sized TDD steps (write failing test, run fail,
implement, run pass, commit) per the writing-plans skill conventions.
Phase 0 verification tasks (A.0, B.0) anchor against current code at
HEAD 40e737a18+ before edits.
19 total tasks across 2 layers. Each layer commits independently as
an atomic feature; intermediate per-task commits during Layer B keep
the wire safety-protected (forward concat lands before backward gating
in B.9 → B.10 sequence).
8 explicit kill criteria for Smoke A2-B per spec B.7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two further user-flagged corrections from second-critical review:
1. Direction Q-head emits K=4 actions, NOT K=3.
Authoritative source: state_layout.cuh:123-126
#define DIR_SHORT 0 // open/maintain short
#define DIR_HOLD 1 // keep current position (no-op)
#define DIR_LONG 2 // open/maintain long
#define DIR_FLAT 3 // close all to zero
The "default: 3 — Short/Flat/Long" comment at gpu_dqn_trainer.rs:2438
is stale pre-SP13. Production callsites all set branch_0_size: 4.
SP13 added DIR_HOLD as a separate fourth direction action (Hold-pricing).
The config default comment was never updated when DIR_HOLD landed.
This is exactly the feedback_trust_code_not_docs failure mode — a
single stale comment would have silently corrupted the q_disagreement
signal (Hold/Flat being indices 1/3 instead of just Flat=1).
Updates:
- B.1 action-space context: 4 actions with Hold AND Flat both
non-committal (Hold = keep position, Flat = exit to zero)
- B.2.3 q_disagreement mapping: K=4↔K=2 with both Hold and Flat
masked from disagreement signal (no new directional commitment
to evaluate); only Short and Long picks contribute to disagreement
- Edge case handling for all-Hold/all-Flat batches
2. Adaptive β rate limiter (was structural β=0.9).
Per feedback_isv_for_adaptive_bounds, β should be signal-driven
not hardcoded. v3 derives β from variance of α_grad_raw, mirroring
the k_aux/k_q variance-driven steepness pattern in B.2.5.
Formula:
β = clip(β_base + variance_alpha_raw / variance_ref_alpha,
[β_base, β_max])
β_base = 0.5 (light smoothing baseline; ~2-step half-life)
β_max = 0.95 (heavy smoothing; ~20-step half-life)
Stable α_grad_raw → β = β_base (preserves directional intent)
Volatile α_grad_raw → β → β_max (dampens jitter)
Adds 2 ISV slots:
ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX (Welford variance)
BETA_RATE_LIMITER_ADAPTIVE_INDEX (current β value)
ISV slot count: 11 → 13 (net +2 for variance + adaptive β).
LOC estimate: ~1150 → ~1180 (negligible delta; 3 Welford
variances now in alpha_grad_compute_kernel instead of 2).
HEALTH_DIAG pearl_egf_diag emit updated to expose all three
adaptive scalars (α, β, k_aux/k_q) plus all three driving
variances (var_alpha, var_aux, var_q) for full observability.
Verified against current code at HEAD d243a6f08:
- state_layout.cuh:123-126 (DIR_* enum truth source)
- gpu_dqn_trainer.rs:2438 (stale K=3 comment confirmed)
- branch_0_size: 4 in 5 production callsites
(smoke_tests, gpu_iqn_head, gpu_backtest_evaluator)
- DIR_HOLD usage in experience_kernels.cu:1298 + 14 other sites
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 corrections from code-anchored critical review at HEAD eaf4adcb9:
1. Direction Q-head emits K=3 (Short/Flat/Long), not K=4. Aux head
emits K=2 (down/up). Gate 2 q_disagreement now uses K=3↔K=2 mapping
with Flat masking. Verified against gpu_dqn_trainer.rs:2438.
2. Forward launch order constraint added: aux forward must complete
before direction Q-head forward (new serial dep). Cited
pearl_canary_input_freshness_launch_order.
3. Adaptive sigmoid k formula fixed: v1 had unreachable k_max=50
because formula caps k ≤ k_base. v2 uses max(..., k_min) with
k_max = k_base implicit.
4. α_grad rate limiter promoted from nice-to-have to v1. Schmitt
state-flip introduces sigmoid discontinuity. β=0.9 EMA smoothing
added; new ALPHA_GRAD_SMOOTHED_INDEX slot.
5. q_disagreement_baseline drop: v1 had adaptive baseline as long-EMA
(feedback loop risk). v2 uses structural 0.5 (analytic K=3-with-
Flat-masked random alignment). Drop BASELINE_INDEX slot.
6. Backward gradient scaling clarified: α_grad scales dL/dx (input
gradient flowing back to aux), NOT dL/dW (Q-head's weight grad).
Q-head learns to use the wire freely; gate only controls upstream
flow.
7. 4 hard rules added: feedback_no_hiding,
feedback_no_htod_htoh_only_mapped_pinned,
feedback_kill_runs_on_anomaly_quickly,
pearl_canary_input_freshness_launch_order.
8. Smoke A2 explicit kill criteria table added (8 triggers).
Net ISV slot count unchanged (11), composition shifted: dropped
BASELINE, added SMOOTHED. Total impl cost ~1150 LOC (was ~1060).
Verified against current code:
- TARGET_DIR_ACC_INDEX=372, AUX_DIR_ACC_SHORT_EMA_INDEX=373,
AUX_DIR_PREDICTION_INDEX=375 (sp13_isv_slots.rs)
- set_aux_weight clamp(0.05, 0.3) at gpu_dqn_trainer.rs:14722
(confirms Bug 3 from Smoke A diagnostic)
- mag_concat_qdir precedent at experience_kernels.cu:4560
(direction-conditioning pattern; SP14's wire is the analog)
- state_reset_registry pattern at lines 913-922 (canonical
template for new EMA fold-reset entries)
- branch_0_size = 3 in production config (the K=3 finding)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Designs the SP14 chain on top of SP13 Layer B (HEAD 6657e5626):
1. Sub-project A — stability fixes (3 small bugs found in Smoke A)
- C51 atom-probability floor (ISV-driven from SP4 atom_pos_p99)
- aux_w setter clamp lift [0.05, 0.3] → [0.15, 1.5]
- Stagnation warmup gate at fold boundary
2. Sub-project B — the architectural piece (THIS spec)
- Forward wire: aux_softmax_diff per-bar into direction Q-head input
concat (in_dim+1, fingerprint bump, zero-init new column)
- Earned Gradient Flow pearl — adaptive ISV-driven gradient gating:
* Gate 1 (aux competence) — Schmitt-trigger hysteresis
* Gate 2 (Q-head disagreement) — NEW signal, EMA per-step argmax
mismatch
* ISV-adaptive sigmoid steepness (variance-driven k_aux, k_q)
* Per-epoch warmup ramp
* Anti-gradient-hacking circuit breaker (mesa-opt defense)
- 11 new ISV slots, 3 new GPU kernels, ~1060 LOC total
- HEALTH_DIAG pearl_egf_diag observability line
3. Sub-project C — Adaptive LR (deferred until A+B effects measured)
Motivation from Smoke A diagnostic:
- aux_dir_acc reached 0.61 (signal extraction works)
- val_win_rate stuck 45-48% (no path to action selection)
- WR-flat-while-aux-varies = Q-head directional weights frozen
- 1109 GRAD_CLIP_OUTLIER events (chronic; not noise)
Three parallel diagnostic agents triangulated three interlocking root
causes:
- Slot 375 has zero readers (the wire was scoped but never built)
- C51 raw grad reaches 9.5e6, saturates SP7 budget controller
- aux_w controller muzzled by SP11-era [0.05, 0.3] clamp
The Earned Gradient Flow pearl is a new application of the codebase's
pearl pattern: ISV-driven adaptive controller, but applied to backward-
pass gradient flow instead of forward-pass features. The wire is one-
way (stop-gradient) by default; co-training is earned by both:
(a) aux head demonstrating label competence, AND
(b) Q-head showing it's actually fighting aux signal (informative
disagreement above baseline).
Stability additions hardened against:
- Oscillation around target (Schmitt hysteresis)
- Numerical sigmoid saturation (argument clipping ±30)
- Stale variance EMAs across folds (state-reset-registry)
- Discontinuous warmup transitions (linear ramp)
- Mesa-optimization (gradient-hacking circuit breaker)
- Cold-start sentinel-state spurious gate openings (Pearl-A bootstrap)
Awaiting user review before invoking superpowers:writing-plans for
sub-project B (and a separate small plan for sub-project A).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
While P0b smoke (train-sw4ws on bdc5cb8bb) runs, two prep items:
(A) scripts/argo-train.sh — add `ci-training` to the L40S sm_89 case
arm. Bare `ci-training` is an L40S pool alias in some clusters;
previously defaulted to sm_90 (Hopper), causing train-mnpf7 to
deploy with wrong-arch cubins (terminated + resubmitted manually).
Now both `*l40s*` and bare `ci-training` resolve correctly.
(C) docs/superpowers/plans/...sp13...md — Layer B section expanded
with concrete codebase locations discovered during P0a:
- aux_heads_kernel.cu, aux_heads_loss_ema_kernel.cu locations
- aux_nb_label_buf populated as column 0 of next_states (log_return)
- F1/F2 regression history note (don't alias the label buffer)
- aux_pred_to_isv_tanh_kernel.cu is a P0a placeholder per its own
header — Layer B should rewrite to read softmax logit-diff
- dir_acc kernel + oracle tests need softmax-read updates
- Layer B + P0b combined rationale post-P0a empirics
Saves the next implementer ~30 min of re-investigation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P0a.T3 v2 implementer's audit revealed `DirectionAction` enum doesn't exist; the
codebase uses an 8-variant fused `ExposureLevel` (ShortSmall/Half/Full, Hold,
LongSmall/Half/Full, Flat) with cross-crate consumers across 77 files and 32+
test files pinning the 8-variant invariant. Atomic Hold elimination would
cascade massively.
User insight (2026-05-04): Hold being FREE is the bug, not Hold itself. MFT
trading legitimately needs multi-bar holds; we want the model to use them
deliberately, not as a CQL-bias lazy default. Holding isn't free in the real
world — broker fees, margin interest, opportunity cost.
v3 reframes as Hold-pricing:
- 4-way action space stays; ExposureLevel::Hold stays; no cross-crate cascade
- 3 new ISV slots (380-382): HOLD_COST_INDEX, HOLD_RATE_TARGET_INDEX,
HOLD_RATE_OBSERVED_EMA_INDEX
- Hold-rate observer: small GPU kernel + Pearls A+D smoothing
- Hold-cost controller: 5-line deficit-driven formula
(excess > target → cost rises 1×→5× base; observed ≤ target → relax)
- Per-bar reward subtraction at action == DIR_HOLD site
- 2 GPU oracle tests for the controller
P0a.T3 cuts from ~250 LOC + 32-test cascade → ~120 LOC additive. T1+T2
already-staged work unchanged. T4/T5/Layer B/C/D structure preserved.
Tension with pearl_event_driven_reward_density_alignment acknowledged in
spec — per-bar Hold cost is exposure-NEGATIVE (away from Hold), models real
economic carry, ISV-bounded by controller. Inverse of the pearl's failure
mode. Faithful reward modeling, not artificial shaping.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three architectural changes in one unified design to push the model from
HFT noise extraction (62% trade rate, sharpe-gaming) toward MFT alpha-hunting:
1. Asymmetric bounded cap (-10/+5) — restores loss aversion erased by
SP11 symmetric cap. 2:1 ratio matches Kahneman/Tversky prospect theory.
Anchor: pearl_audit_unboundedness_for_implicit_asymmetry.
2. Min-hold soft penalty with temperature curriculum — patience requirement
at exit. Soft factor = deficit/(deficit+T), T anneals 50→5 over 50 epochs.
Forces commitment without paralysis in early training.
3. Zero per-bar shaping (gate micro/opp_cost on events) — eliminates
continuous-reward gradient that pulls toward continuous exposure.
Anchor: pearl_event_driven_reward_density_alignment.
Combined: reward fires only on trade events with prospect-theory loss
aversion + commitment requirement. Pure per-trade event-driven Q-learning
properly aligned with per-trade P&L objective.
~50 LOC across 3-4 files. No new ISV slots in Phase 1 (constants only).
Cost ~€1.30 (€0.30 smoke + €1.00 30-epoch validation).
Empirical motivation: train-multi-seed-pmbwn 50-epoch on commit 6a259942e
showed sharpe-gaming pattern (PnL -30% over 8 epochs while sharpe held).
SP11 cap fix unmasked the per-bar shaping bias plus erased loss-aversion
that the unbounded loss path was implicitly providing.
Continues on sp11-reward-as-controlled-subsystem branch — SP12 is
architectural continuation of SP11, not separate work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
smoke-test-6wd2c on commit 61b2fa962 (B1b + 4 bug fix-ups) revealed a 5th
pathology not covered by the previous fix-ups: the mag-ratio canary's
linear-magnitude-ratio formula amplifies whichever component is
intrinsically largest, regardless of whether that's a useful signal.
Popart (trade P&L on segment_complete) is O(100) per fire while the
other 5 components (cf/trail/micro/opp_cost/bonus) are O(0.1-2). Even
with the slot 360 fix preventing total-reward contamination, popart's
intrinsic magnitude makes popart_mag / Σ ≈ 0.93. The controller blend
'winner_weight = ratio' then amplifies popart further. Smoke trajectory:
w_pop=2.0 → 2.44 → 2.57, curiosity_b=30 → 120 → 199, sharpe_ema=10.7 →
2.4 → 0.75 (cascading collapse).
Resolution: z-score normalization. Each component's magnitude divided
by its own running standard deviation before computing the ratio:
popart_z = popart_mag_ema / max(sqrt(popart_var_ema), EPS_DIV) ≈ O(1)
cf_z = cf_mag_ema / max(sqrt(cf_var_ema), EPS_DIV) ≈ O(1)
...
ratio[c] = component_z / Σ component_z ≈ ~1/6 each when stable
Allocates 6 new ISV slots [361..367) for per-component variance EMAs.
Producers: extend popart_component_ema_kernel + reward_component_ema_kernel
to also emit variance via Welford's online algorithm (single-pass).
SP5_SLOT_END = 367, ISV_TOTAL_DIM = 367.
Carries forward main's slot 360 amendment (commit 52c0b7521 on main)
which the sp11 branch was missing, plus this z-score amendment.
Per-component gradient ratios (the original spec intent) don't fix this
either — in DQN there's no per-component gradient pathway; grad norm
scales with current_weight × magnitude, so it's the same bias. Z-score
normalization is the standard scale-invariant measure of significance
and matches what the SP11 controller is trying to express.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feedback_no_htod_htoh_only_mapped_pinned (tests not exempt):
- All test fixtures converted from htod_copy/dtoh_sync_copy to
MappedF32Buffer with host_slice / host_slice_mut access.
- Novelty hash buffer + projection matrix changed from
cudarc::CudaSlice<f32> + alloc_zeros to MappedF32Buffer.
feedback_no_cpu_forwards (CPU is read-only):
- Projection matrix initialization changed from host-side StdRng +
host_slice_mut writes to a one-shot GPU init kernel
(novelty_simhash_proj_init_kernel) using Philox seeded from
config.seed. No host RNG, no host writes.
feedback_no_cpu_compute_strict (saboteur multiplication):
- B1 step 5 reverted from Rust-side `read_isv_slot * scale` to
GPU-side: pass base scale + ISV pointer + slot index to the
saboteur perturbation kernel; multiplication happens on-device.
feedback_trust_code_not_docs (grad-ratio terminology):
- Spec §3.3.1 "per-component grad EMA" was wrong — SP4 grad-balancer
is per-branch (4 slots), not per-reward-component. Renamed:
reward_component_grad_ratio_compute_kernel
→ reward_component_mag_ratio_compute_kernel
REWARD_COMPONENT_GRAD_RATIO_BASE
→ REWARD_COMPONENT_MAG_RATIO_BASE
Source: existing REWARD_POPART_EMA_INDEX..+6 (per-component reward
magnitude EMAs from SP4 reward_component_ema_kernel). Semantic
equivalent for the controller's exploit/diversify blend.
feedback_no_stubs (dead parameter):
- Removed `eps_div_idx_unused` from mag_ratio kernel signature.
Saboteur engagement: missing producer specified
- Spec §3.3.1's two-reward-arrays formulation replaced with single
`saboteur_delta_reward_buf` produced by the saboteur perturbation
kernel itself (single reward computation, diff emitted as side
output). Engagement kernel signature simplified to one input array.
PNL_REWARD_MAGNITUDE_EMA_INDEX (slot 359): producer wired
- mag-ratio kernel mirrors `isv[REWARD_POPART_EMA_INDEX]` to
scratch_out[6]; chained apply_pearls_ad targets slot 359.
Replay sample kernel location specified
- graph_utility_kernels.cu:71 (gather_f32_scalar). New sibling kernel
`gather_replay_reward_with_curiosity` defined; replaces the existing
scalar gather (no legacy alias per feedback_no_legacy_aliases).
novelty_simhash_lookup runs before, novelty_simhash_update after.
A2 controller test placeholders → full GPU oracle assertions
- Three controller tests (z=0 midpoint, weight renorm, saboteur clamp)
have full mapped-pinned fixtures with assertions on weight sum,
individual values, post-clamp bounds.
Plan now passes:
- feedback_no_htod_htoh_only_mapped_pinned (tests + production)
- feedback_no_cpu_forwards (CPU never writes/computes for GPU)
- feedback_no_cpu_compute_strict (all multiplications GPU-side)
- feedback_no_atomicadd (race-tolerated non-atomic, safety documented)
- feedback_no_partial_refactor (Layer B atomic; saboteur kernel sig
change touches all callers in the same commit)
- feedback_no_stubs (no dead parameters)
- feedback_trust_code_not_docs (corrected spec terminology)
- feedback_wire_everything_up (every new field has init + producer)
- feedback_no_legacy_aliases (old gather_f32_scalar replaced, deleted)
1447 lines, +268 from previous version.
User correction: curiosity is the *fix* for the ep1-peak overfitting
pathology, not a hazard to defend against. Reframed §7 from "Risks"
to "Design notes" — curiosity bound is a signal-relative scale, not
a defensive cap.
Fix the contradiction this exposed in the formula: previous
`curiosity_pressure = stagnant_or_worse * curiosity_bound` went to
zero when improving, which would cancel the always-on exploration
the §7 narrative now relies on. Replace with permanent-floor pattern
per pearl_blend_formulas_must_have_permanent_floor:
curiosity_floor = 0.2 * curiosity_bound (CURIOSITY_PERMANENT_FRACTION)
curiosity_dynamic = stagnant_or_worse * curiosity_bound
curiosity_pressure = max(curiosity_dynamic, curiosity_floor)
Now curiosity is always ≥ 20% of bound (anti-overfitting baseline)
and rises toward the bound when stagnant (stagnation breaker).
Updated unit-test guidance to assert pressure > 0 even at z=+10.
CURIOSITY_PERMANENT_FRACTION=0.2 added to Invariant-1 fraction list.
Saboteur-rising-with-improvement reframed as adversarial-load feature
rather than over-stress risk.
Brainstorm spec for SP11. Resolves the policy-stagnation pathology
surfaced in T10 train-multi-seed-xkjkb seed-0 ep0-14: model finds a
stable fixed point at ep1 (peak val sharpe 80.61), then OVERFITS to
it across remaining epochs (decline 80.61 → 70.58). Q-values grow but
val performance declines because reward function has no improvement
pressure.
Architecture (every input ISV-driven):
- Z-score-driven adaptation (no hardcoded "improving" threshold):
improvement_z = val_sharpe_delta_ema / max(val_sharpe_std_ema, EPS)
- 10 ISV outputs: 6 component weights + curiosity_pressure +
saboteur_intensity_mult + adaptive weight_floor + curiosity_bound
- 5 ISV canaries: val_sharpe_delta + val_sharpe_std (Z-score noise
estimate) + 6 per-component grad ratios + saboteur engagement +
PnL magnitude EMA (signal-relative curiosity bound)
- 4 new producer kernels (controller + 3 canary computers)
- Audit + migrate hardcoded cf_weight=0.3 in mse_loss_kernel.cu:318
and c51_loss_kernel.cu:789, plus other shaping multipliers
- NEW reward dimension: curiosity bonus, bounded by PnL magnitude
Per pearl_controller_anchors_isv_driven: every threshold replaced with
sigmoid(z) — no constants encode "what counts as improving". Per
pearl_blend_formulas_must_have_permanent_floor: every weight has
adaptive floor preventing zero-out. Per pearl_engagement_rate_self_
correction: saboteur intensity self-corrects via engagement rate canary.
Per pearl_cold_start_exit_signal_or: improvement signal OR'd from
multiple canaries so single-signal-failure doesn't stall controller.
New pearl authored alongside spec: pearl_reward_as_controlled_subsystem
— meta-principle that every reward path degree of freedom is a unified
controller output. Subsumes controller-anchor pearl at the reward layer.
Scope: 20 ISV slots, 4 producer kernels, audit + migration of
hardcoded reward shaping constants, 1 atomic commit.
~1300-1700 LOC; ~2.5-3 hours subagent work.
Success metric: val_sharpe[ep20] > val_sharpe[ep1] (Fix 33-38 baseline
peaked at ep1; SP11 should shift peak later as model continues
learning).
Brainstorm spec for SP10. Resolves the val-Flat-collapse pathology that
persisted through Fix 33-37: the eval-time argmax in experience_action_
select picks Hold deterministically every val bar (dir_entropy=0,
trade_count=1 in 214k bars) regardless of controller state.
Architecture:
- Delete `if (eval_mode)` argmax branch in direction-selector kernel
- Use temperature-blended Thompson: q_eff = E[Q] + τ × (Thompson - E[Q])
- τ = clamp(intent_eval_divergence / divergence_target, 0.5, 2.0)
- τ self-corrects: collapse → high τ; healthy → low τ; permanent 0.5 floor
- Reuses SP9's intent_eval_divergence_compute_kernel (extended, not new)
Per pearl_controller_anchors_isv_driven: τ is ISV-driven, no
hardcoded constants beyond Invariant 1 numerical anchors (clamp range).
Per pearl_blend_formulas_must_have_permanent_floor: MIN_TEMP=0.5 ensures
eval ALWAYS has stochasticity.
The pearl_thompson_for_distributional_action_selection was about Bellman
TARGET argmax (selector/target symmetry). It does NOT prohibit Thompson
at the rollout selector. SP10 amends the pearl to clarify.
Scope: 1 ISV slot, kernel modification (no new kernel — extend SP9's
producer), 1 consumer kernel rewrite, test update, pearl amendment,
audit doc Fix 38. ~300-500 LOC, single atomic commit per
feedback_no_partial_refactor.
Brainstorm spec for SP9 (Fix 37). Resolves the val-Flat-collapse pathology
surfaced in T10 train-multi-seed-wsnc6 ep1-3: training intent_dist_f=0.47
but eval_dist_f=0.00 because Kelly cap pinned eval mag to Quarter, with
trade_count=1 in 214k bars (chicken-and-egg: cap closed → no trades → no
Kelly samples → cap stays closed).
Architecture per pearl_controller_anchors_isv_driven and
pearl_cold_start_exit_signal_or:
final_kelly_f = max(measured_kelly_f, warmup_floor)
warmup_floor = base_floor × (1 − combined_confidence)
base_floor = clamp(q_var_mag / q_var_mag_ema, MIN, MAX)
combined_confidence = max(statistical, behavioral, temporal)
All thresholds ISV-driven via Pearl D Wiener-α; only Invariant 1 numerical
anchors (clamp ranges, EPS) remain as constants.
Scope: 9 new ISV slots, 6 new GPU producer kernels including a
mandatory eval_dist GPU migration (eliminating DtoH readback at
gpu_backtest_evaluator.rs:1353 per feedback_no_cpu_compute_strict).
~1000-1200 LOC, single atomic commit per feedback_no_partial_refactor.
Pearls authored as part of brainstorm:
- pearl_controller_anchors_isv_driven (regime-encoded constants)
- pearl_cold_start_exit_signal_or (OR'd signals across statistical /
behavioral / temporal axes)
Critical re-review of v1 caught:
1. Task 6 retained Pearl 2 kernel signature with (void) no-op args
"to save 3-site cascade" — that's the partial-refactor anti-pattern
feedback_no_partial_refactor explicitly forbids. v2 does the full
contract change: signature shrinks 9→6 args, launcher migrates
atomically.
2. SCRATCH_PEARL_2_C51/_CQL/_ENS would become orphan constants —
feedback_wire_everything_up says delete or wire. v2 deletes them in
T6.
3. Audit-doc updates were separated into a single late commit — would
fail check_audit_doc_updates pre-commit gate on T1-T6. v2 folds Fix
31 stub into T1 and extends it commit-by-commit.
4. T5 referenced grad_decomp_*_result_dev_ptr field names that don't
exist — actual layout is one shared 27-float pinned buffer with
per-component byte offsets (iqn=0, cql_sx=24, c51=36). v2 uses
pointer arithmetic from grad_decomp_result_dev_ptr.
5. Tasks 1, 2, 4 had "if file shape X then Y, otherwise Z" placeholder
branches — writing-plans skill forbids these. v2 has concrete patches
based on reading the actual files.
Also corrected: T2 fixes the stale "regime_stability allocator"
description in state_reset_registry.rs alongside the new entries.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Outcome-driven controller replacing Pearl 2's hardcoded-0 CQL budget and
floor-pinned C51 budget with multiplicative ratio adaptation. Targets
`grad_split_bwd cql/iqn = 2.0` and `c51/iqn = 1.0` per slice (trunk, dir,
mag). Slow EMA (α=0.01) consistent with existing controller pearls; Pearl
A sentinel-bootstrap on fold boundary; Pearl D Wiener-optimal smoothing.
Replaces ghost docstring at fused_training.rs:3409 ("0.10×(1−regime)×health"
formula was never implemented). Pearl 2 keeps owning IQN budget (reference
denominator) and FLATNESS_BASE (consumed by NoisyNet σ pearl); stops
writing CQL/C51/ENS slots so the new controller is the sole driver.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three independent sub-projects (one per Pearl), each an atomic commit,
each running in its own git worktree (sp6-pearl-2/3/5). Pearl 2 expands
compute_adaptive_budgets() to per-branch [f32;4] arrays + trunk-mean
scalars, dispatches branch-correction SAXPY sub-launches. Pearl 3
replaces noise_sigma: f32 with noise_sigma_per_branch: [f32;4] in
ExperienceCollectorConfig and extends add_advantage_noise to per-branch
lookup. Pearl 5 adds 4-branch tau buffer triplets to GpuIqnHead, runs
4 IQN forward passes per step (Approach B), each contributing
iqn_trunk/4 to prevent 4x gradient accumulation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Critical self-review surfaced 15 issues; user directed fixes:
- "a no cpu path": KEEP host-EMA close-out (rule compliance) — but split
into separate Layer D atomic commit (was mis-scoped as Layer A)
- Pearl 4 kept: 3 concrete risks documented (constant-β proof break,
β2 memory reset destabilization, ε numerical envelope), structural
envelope bounds added, ALPHA_META halved, ε-only fall-back path defined
- Pearl 6 Kelly cross-fold persistence carve-out: separate slot range
280..286, NOT in SP4 fold-reset registry, Invariant 1 architectural exception
- Commit ordering Pearls 1 → 3 → 2 → 4 → 5 → 6 → 8 → 1-ext (resolves
Pearl 2 circular dependency on 1+3)
- Acceptance criteria: correctness gates (must pass) + performance gates
(loosened to "not catastrophically negative", within 2σ of pre-SP5)
- Pearl 7 timing: explicitly Layer C step 4, post-Layer-B + 3-seed validation
- Pearl 8 enumeration: 4 slots (TRAIL_DIST_PER_DIR per direction)
- Pearl 9 collapsed: 0 slots (Thompson achieved via Pearl 1's atom adaptation)
- Total slot count corrected: 110 (was 120-128 inconsistent)
Layer structure: A (additive, 8 commits) → B (atomic, 11 consumers) → C
(validation + Pearl 7 investigation) → D (host-EMA close-out, separate
atomic commit). Layer D split off from A's "close-out" because PnL
aggregation pipeline migration is its own architectural concern.
User final review pending before invoking writing-plans skill.
Per user direction:
Q1 (Layer A granularity): per-pearl commits (~9 commits, decision c)
Q2 (Pearl 7 timing): investigate-only in SP5; fix in follow-up
if Bin(2,0.5) persists post-Pearls-1-3 (decision a)
Q3 (Pearl 4 Adam β): include as designed; accept theoretical risk
with Pearls A+D + EPS_CLAMP_FLOOR mitigation;
Layer C smoke monitors for destabilization;
fall-back to ε-only if observed (decision a)
Spec section updated:
- Layer A description: per-pearl commit structure
- Pearl 7 framed as investigation-only with conditional follow-up
- Pearl 4 documents theoretical caveat + mitigation + fallback
User final review pending before invoking writing-plans skill.
