Mid-execution architecture revision: Mamba2 stays as a sequence
encoder; CfC becomes the layer on top (replacing the Phase 1d.3 MLP
stacker). Gate becomes 'stacked AUC >= Mamba2-only stacker AUC at
every horizon' — proves the CfC layer is additive, rather than CfC
alone beating Mamba2 alone.
Plan 1 kernels (cfc_step, heads, projection, BCE, AdamW, Graph A)
are unchanged. Only CfcTrunk's forward path gains a Mamba2 prefix
that consumes the snapshot stream and emits a 128-dim h_mamba which
CfC reads. The Mamba2 kernel (mamba2_alpha_kernel.cubin) is already
in the build.
Option B (parallel + fused dual-stream) is documented as the Plan 2
fallback if the stacked gate fails.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two documentation deliverables produced while T14 backtest runs:
1. Plan update (specs/2026-05-15-phase-e-4-a-temporal-foundation.md):
adds 'Execution Status' section reflecting actual T1-T14
progression. T5 deferred (real MBP-10 peek), T9 skipped (GRN
moved to E.4.B per integration notes), T10 partial (new C51
grad-input kernel landed but Mamba2 backward wiring deferred),
T14 in flight. Documents the 4 execution learnings:
research-first saved a week of duplicate kernel work; cheap
falsification experiments (Path 2, Path 3) avoided expensive
investments; C51 borrow was the largest single Sharpe-lift in
the session; GpuTensor/CudaSlice interop friction is the real
integration cost.
2. T10 patch sketch (specs/2026-05-15-t10-mamba2-backward-from-h-enriched.md):
ready-to-apply patch for ml-alpha::Mamba2Block adding a new
public method backward_from_h_enriched(cache, d_h_enriched).
Bypasses the W_out projection backward, accepts the
[B, hidden_dim] gradient from C51's grad-input kernel directly,
zero-initialises dw_out/db_out (AdamW step on zero grad is a
no-op with correct moment decay — effectively freezes W_out
params which is correct semantics since Phase E never uses
them). Includes the smoke binary wiring snippet that consumes
the new method via launch_alpha_c51_grad_input → Mamba2
backward → AdamW step. Application gated on T14 backtest
validation — if frozen Mamba2 already lifts Sharpe, T10
becomes optimisation rather than prerequisite.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Design doc (specs/): TFT-style architecture for Phase E execution
policy — sliding window → Mamba2 SSM → GRN trunk → MoE regime gate
→ C51 head → Thompson selector. Two core pillars added per user:
A) Full L1-L10 LOB depth input via hybrid MBP-10 peek
B) ISV-continual-learning: controllers fire at training AND
inference; Q-net weights frozen at inference but effective
policy adapts via ISV modulation
Plan doc (plans/): 14-task implementation plan for E.4.A foundation
(window buffer + L1-L10 depth + Mamba2 forward+backward + ISV-eval
controllers). Falsification gates: smoke R_mean improvement ≥ 50%,
backtest cost=0 Sharpe ≥ +8 (no regression vs C51-flat +10.41),
half-tick Sharpe ≥ -8 (closes 5pt+ of 10pt gap to Phase 1d.4
baseline -4.0).
TGGN (foxhunt Temporal Graph Gated Network) explicitly deferred to
Phase E.5+: existing CPU graph implementation + GPU adapter at
ml-supervised/src/tgnn/ — marginal benefit for single-instrument ES
futures vs the TFT-Mamba2 stack; revisit for multi-instrument
extension or production HFT inference layer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Findings:
- Production Mamba2 (gpu_dqn_trainer) is coupled to SH2=256 trunk +
ofi_embed + ISV[8] temporal routing — not portable to Phase E.
- ml-alpha::mamba2_block::Mamba2Block is from-scratch, fully
configurable (in_dim/hidden_dim/state_dim/seq_len), GPU-pure with
forward_train/backward/AdamW. Used in Phase 1d.2 to lift AUC 0.50
to 0.66. ml-alpha is already a workspace dep of ml.
- GRN skipped for E.4.A — Mamba2 output goes straight to C51 head.
Reintroduce GRN in E.4.B if Sharpe gates don't pass.
- Controller-at-inference: kernel has no training-mode branches;
Wiener state preserved across episodes/cost cells for natural
live-deployment simulation.
Revises Tasks 8-10 of the plan: use Mamba2Block API instead of
writing custom kernels. Only new CUDA needed: alpha_c51_grad_input
(C51 gradient w.r.t. input features, for Mamba2 backward chain).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Atomic Phase 1.4 commit per `feedback_no_partial_refactor`. Wires the
SP20 fused-producer chain (Stats → Aggregate → EMAs → Controllers) into
GpuExperienceCollector's per-rollout-step path with one new aggregation
kernel + Phase 1.2 EMA kernel-signature refactor + production wire-up
+ tests + audit/spec/plan amendments, all in one commit.
## Path C decision rationale
The Phase 1.2 EMA kernel originally took 9 scalar value-args. The Phase
1.4 wire-up site (per-rollout-step in GpuExperienceCollector) needs to
feed per-env GPU-resident trade signals (trade_close_per_sample,
step_ret_per_sample, hold_at_exit_per_sample, packed factored
actions_out) into the kernel — forbidden via host sync
(`feedback_cpu_is_read_only`) and via memcpy_dtoh/htod
(`feedback_no_htod_htoh_only_mapped_pinned`). Path C refactors the EMA
kernel to take a device struct pointer (`SP20EmaInputs* ema_inputs`) +
adds an aggregation kernel that writes the struct on the GPU. Phase 1.2
had zero production consumers yet, so the sig change + Phase 1.4 wire-up
land atomically.
## What this commit contains
1. **EMA kernel signature refactor** (Phase 1.2 → Path C):
`sp20_emas_compute_kernel.cu` swaps 9 scalar args for a single
`const SP20EmaInputs* __restrict__ ema_inputs` device pointer.
Math semantics bit-identical. Rust `EmaInputs` mirrored as
`#[repr(C)]` byte-for-byte; new `pack_inputs_into_f32_view` helper
for tests + collectors that fill the struct via a mapped-pinned
f32-aliased buffer.
2. **New aggregation kernel** (`sp20_aggregate_inputs_kernel.cu`):
Per-env arrays (sliced to current rollout step) + sp20_stats outputs
(p50/std) + aux_dir_acc_reduce_kernel output [0] → SP20EmaInputs
struct. Block tree-reduce (4 stripes × bdim × 4 bytes shmem); no
atomicAdd. Aggregation rules per spec §4.5:
- is_close = OR over envs
- is_win = (≥ 0.5 of closed envs were wins)
- trade_duration = round(mean(hold_at_exit) over closed envs)
- action_is_hold = strict majority (count*2 > n_envs) HOLD
- alpha = 0.0 (Phase 2 forward ref — reward kernel)
- per_bar_hold_reward = 0.0 (Phase 3.2 forward ref — Hold-cost dual)
- aux_logits_p50/std/dir_acc forwarded from upstream
3. **Production wire-up in GpuExperienceCollector**:
4 kernel handles + 4 mapped-pinned buffers (struct fields, alloc,
init); per-rollout-step launch sequence after env_step (step 5c):
`Stats → Aggregate → EMAs → Controllers`. All on the same stream;
stream-implicit producer→consumer ordering. Gated on
`isv_signals_dev_ptr != 0 && trainer_params_ptr != 0` (matches the
existing SP14-β EGF chain pattern).
4. **Fold-boundary reset**:
3 new RegistryEntry records (sp20_ema_inputs_buf,
sp20_emas_internal_buf, sp20_emas_obs_count_buf) + 13 new dispatch
arms in training_loop.rs (10 SP20 ISV slot resets + 3 buffer resets).
Closes the pre-existing `every_fold_and_soft_reset_entry_has_dispatch_arm`
regression (was failing on fresh check after the SP20 ISV slot
registrations landed in commit 4249ebc96).
5. **HEALTH_DIAG emit**:
Per-epoch `HEALTH_DIAG[N]: sp20_isv [loss_cap=… alpha_ema=…
wr_ema=… hold_cost_scale=… target_hold_pct=… hold_pct_ema=…
hold_reward_ema=… n_step=… aux_conf_threshold=… aux_gate_temp=…]`
right after the existing q_disagreement_diag emit.
6. **Tests** (5 test files, all green on RTX 3050 Ti sm_86):
- sp20_emas_compute_test.rs (4 GPU oracle): updated to use the
post-Path-C buffer-arg API (mapped-pinned f32-aliased struct).
- sp20_aggregate_inputs_test.rs (NEW, 6 GPU oracle): aggregation
rule coverage including the Phase 2 / Phase 3.2 placeholder
contract.
- sp20_phase1_4_wireup_test.rs (NEW, 2 GPU oracle): end-to-end
4-kernel chain integration test.
- sp20_stats_compute_test.rs (4 GPU oracle): unchanged, regression.
- sp20_controllers_compute_test.rs (7 GPU oracle): unchanged,
regression.
- 18 lib unit tests across the SP20 launchers.
7. **Phase 2 / Phase 3.2 forward references**:
The `alpha` (Phase 2) and `per_bar_hold_reward` (Phase 3.2) fields
are emitted as 0.0 placeholders and documented at:
- `sp20_aggregate_inputs_kernel.cu:46-52` (in-kernel docstring)
- `sp20_aggregate_inputs.rs:46-49` (launcher docstring)
- `dqn-wire-up-audit.md` "Phase 2 / Phase 3.2 forward references"
These are NOT stubs — Phase 2 / 3.2 will replace the kernel's 0.0
writes with real signals atomically with their respective producers.
The original Task 2.3 (host-side aggregation) is **subsumed** by
the GPU-side aggregation kernel.
## Hard rules
- `feedback_no_partial_refactor` — kernel sig change + aggregation
kernel + production wire-up + tests + audit/spec/plan amendments
in one atomic commit
- `feedback_no_atomicadd` — aggregation kernel uses block tree-reduce
- `feedback_no_cpu_compute_strict` — every aggregation lives on GPU
- `feedback_no_htod_htoh_only_mapped_pinned` — every Phase 1.4 buffer
is mapped-pinned with `cuMemHostAlloc(DEVICEMAP)` reachable via
`dev_ptr`; no memcpy_dtoh/htod
- `pearl_first_observation_bootstrap` — counter-based bootstrap
preserved; placeholder writes (0.0) keep the ALPHA / HOLD_REWARD
EMAs at 0.0 sentinel until Phase 2 / 3.2 wire real producers
- `pearl_no_host_branches_in_captured_graph` — every kernel is single-
block; no host branches; safe to capture in the per-step CUDA Graph
- `pearl_tests_must_prove_not_lock_observations` — integration test
asserts invariants (slots populate, bounds respected) rather than
locked observed values
## Verification
- `cargo check -p ml --features cuda` clean
- 18 lib unit tests for SP20 launchers pass
- 23 GPU oracle tests across 5 test files pass on RTX 3050 Ti (sm_86)
- `every_fold_and_soft_reset_entry_has_dispatch_arm` regression test
now passes (was failing pre-change)
- 14 baseline lib test failures unchanged (none introduced by this
commit)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Phase 1.1 sp20_stats_compute kernel (de922c6a4) assumed the SP14-C aux
head emits 3-class logits {short, hold, long} with baseline 1/3. This was
an error in the SP19+20 spec — production aux head emits K=2 logits
{down, up} per gpu_aux_heads.rs:61 (AUX_NEXT_BAR_K = 2, established by
SP13 B1.1a). Wiring K=2 production aux into the K=3 kernel = OOB reads +
corrupt stats.
Retargets the kernel + launcher + tests to K=2 atomically per
feedback_no_partial_refactor:
- Kernel: SP20_K_CLASSES 3→2; SP20_UNIFORM_K3 0.333…→SP20_UNIFORM_K2 0.5f;
Pass A reads 2 logits (was 3); aux_conf range [0, 1/2] (was [0, 2/3]).
- Launcher: AUX_K_CLASSES 3→2; renamed unit test
aux_k_classes_is_three → aux_k_classes_matches_production_aux_head.
- Tests: rewrote CPU oracle for K=2 input shape and 0.5 baseline; updated
expected p50/std ranges; redesigned heterogeneous-distribution test to
use ramped (not lockstep) clusters — K=2's saturated softmax in the hot
half collapses every row to bin 255, triggering the
pearl_sp4_histogram_warp_tile_undercount trap; ramped clusters distribute
bin indices across each warp's 32 lanes so the histogram path matches
the CPU oracle within bin_width tolerance.
Phase 1.2 (sp20_emas_compute) and Phase 1.3 (sp20_controllers_compute)
consume the scalar [p50, std] outputs and do NOT carry the K dimension.
Both regression suites verified passing unmodified:
- sp20_emas_compute_test: 4 GPU + 1 unit, all pass.
- sp20_controllers_compute_test: 7 GPU, all pass.
Verification (RTX 3050 Ti, sm_86):
- sp20_stats_compute_test: 4 GPU oracle + 4 launcher unit, all pass.
- cargo check -p ml --features cuda: clean (pre-existing warnings only).
Spec + plan amended at top with "AMENDED 2026-05-09" notes; audit doc
Phase 1.1 entry has a "K=2 fixup" subsection documenting the change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Q1 design decision (b): add explicit hold_reward_ema to center the per-bar
Hold reward, so Q(Hold) and Q(trade) targets are scale-comparable. The
marginal Q(trade) > Q(Hold) preference now comes from data variance in
each state (high-aux states pull Q(Hold) more negative), not from
structural scale asymmetry that depends on cost_scale magnitude.
Q2 decision: keep 4-quadrant fixed (no ramping partials). Already in spec.
Changes:
- §4.2 Hold opp-cost: dual emission documented — R_per_bar_centered
(= R_per_bar - hold_reward_ema) for Q-target/replay tuple,
R_per_bar uncentered for hold_baseline_buffer (Component 1 baseline)
- §4.5 Kernel 1: hold_reward_ema added (per-step on Hold-state bars only)
- §5 data flow: per-bar reward path shows the centered/uncentered split
- ISV slots: 9 → 10 (HOLD_REWARD_EMA_INDEX added)
- §8 footprint: Component 2 LoC 70 → 90 (+20 for dual emission)
- Total LoC estimate: 1620
Mirrors the SP17 PP.1 plan-copy pattern. Plan and spec source-of-truth
live on the sister `feat/sp17-dueling-q-network` worktree; this commit
copies them onto the SP18 branch so subsequent commits reference local
paths.
Spec: docs/superpowers/specs/2026-05-08-sp18-reward-shape-hold-attractor-design.md
Plan: docs/superpowers/plans/2026-05-08-sp18-reward-shape-hold-attractor.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second critical review pass found 10 more issues, 3 critical.
All addressed:
CRITICAL:
- §8.2 (3.1): ALPHA_SPLIT cold-start was unspecified — formula
produces 0/0=0, not the claimed 0.5 sentinel. Now: ISV slot
initialized DIRECTLY to 0.5 in trainer constructor; formula
takes over only after both grad-norm EMAs accumulate N_warm
non-zero observations
- §9.2 (3.5.4): plasticity now performs TWO-STEP recovery:
(1) Flat all positions at fire bar (close current losing trade),
(2) engage warmup cooldown forcing Hold for M_warm bars.
Without step 1, forced Hold preserved the losing position
that drove drawdown for the entire 200-bar warmup
- §12.2: stale "5-10%" baseline cost estimate updated to "15-25%"
matching §6.4 (was contradicting earlier amendment)
IMPORTANT:
- §9.2 (3.5.5): DD_TRAJECTORY_DECREASING threshold 0.02 hardcoded
→ ISV-driven via new slot DD_TRAJECTORY_FLOOR (slot 441,
25th percentile of running dd_pct distribution)
- §8.2 (3.5): HOLD_FLOOR_ALPHA tracked from rolling 95th percentile
of |Q_dir| (NOT running max — was outlier-ratchet vulnerable)
- §9.2 (3.5.3): MEDIAN_STREAK_LENGTH formerly undefined in cooldown
K formula → ISV-driven via new slot 442, running median of
observed loss-streak lengths via two-heap algorithm
- §9.2 (3.5.2): asymmetric reward × α split compound interaction
explicitly stated as intentional with POS_CAP as binding ceiling
NIT:
- §7.4: "2C: Group 3 (4 tests)" → "(5 tests)" (was off-by-one
after 2.22 added)
- §9.2 (3.5.4): Xavier → Kaiming-He init for advantage head reset
(architecturally appropriate for ReLU-gated activation chain)
ISV_TOTAL_DIM: 441 → 443 post-SP15 (added DD_TRAJECTORY_FLOOR
and MEDIAN_STREAK_LENGTH at slots [441..443)). 46 SP15 slots
total. File: 799 → 811 lines.
Spec is now consistent end-to-end with no contradictions between
sections, no hardcoded values violating feedback_isv_for_adaptive_
bounds, and no underspecified load-bearing parameters.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two further user-flagged corrections from second-critical review:
1. Direction Q-head emits K=4 actions, NOT K=3.
Authoritative source: state_layout.cuh:123-126
#define DIR_SHORT 0 // open/maintain short
#define DIR_HOLD 1 // keep current position (no-op)
#define DIR_LONG 2 // open/maintain long
#define DIR_FLAT 3 // close all to zero
The "default: 3 — Short/Flat/Long" comment at gpu_dqn_trainer.rs:2438
is stale pre-SP13. Production callsites all set branch_0_size: 4.
SP13 added DIR_HOLD as a separate fourth direction action (Hold-pricing).
The config default comment was never updated when DIR_HOLD landed.
This is exactly the feedback_trust_code_not_docs failure mode — a
single stale comment would have silently corrupted the q_disagreement
signal (Hold/Flat being indices 1/3 instead of just Flat=1).
Updates:
- B.1 action-space context: 4 actions with Hold AND Flat both
non-committal (Hold = keep position, Flat = exit to zero)
- B.2.3 q_disagreement mapping: K=4↔K=2 with both Hold and Flat
masked from disagreement signal (no new directional commitment
to evaluate); only Short and Long picks contribute to disagreement
- Edge case handling for all-Hold/all-Flat batches
2. Adaptive β rate limiter (was structural β=0.9).
Per feedback_isv_for_adaptive_bounds, β should be signal-driven
not hardcoded. v3 derives β from variance of α_grad_raw, mirroring
the k_aux/k_q variance-driven steepness pattern in B.2.5.
Formula:
β = clip(β_base + variance_alpha_raw / variance_ref_alpha,
[β_base, β_max])
β_base = 0.5 (light smoothing baseline; ~2-step half-life)
β_max = 0.95 (heavy smoothing; ~20-step half-life)
Stable α_grad_raw → β = β_base (preserves directional intent)
Volatile α_grad_raw → β → β_max (dampens jitter)
Adds 2 ISV slots:
ALPHA_GRAD_RAW_VARIANCE_EMA_INDEX (Welford variance)
BETA_RATE_LIMITER_ADAPTIVE_INDEX (current β value)
ISV slot count: 11 → 13 (net +2 for variance + adaptive β).
LOC estimate: ~1150 → ~1180 (negligible delta; 3 Welford
variances now in alpha_grad_compute_kernel instead of 2).
HEALTH_DIAG pearl_egf_diag emit updated to expose all three
adaptive scalars (α, β, k_aux/k_q) plus all three driving
variances (var_alpha, var_aux, var_q) for full observability.
Verified against current code at HEAD d243a6f08:
- state_layout.cuh:123-126 (DIR_* enum truth source)
- gpu_dqn_trainer.rs:2438 (stale K=3 comment confirmed)
- branch_0_size: 4 in 5 production callsites
(smoke_tests, gpu_iqn_head, gpu_backtest_evaluator)
- DIR_HOLD usage in experience_kernels.cu:1298 + 14 other sites
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 corrections from code-anchored critical review at HEAD eaf4adcb9:
1. Direction Q-head emits K=3 (Short/Flat/Long), not K=4. Aux head
emits K=2 (down/up). Gate 2 q_disagreement now uses K=3↔K=2 mapping
with Flat masking. Verified against gpu_dqn_trainer.rs:2438.
2. Forward launch order constraint added: aux forward must complete
before direction Q-head forward (new serial dep). Cited
pearl_canary_input_freshness_launch_order.
3. Adaptive sigmoid k formula fixed: v1 had unreachable k_max=50
because formula caps k ≤ k_base. v2 uses max(..., k_min) with
k_max = k_base implicit.
4. α_grad rate limiter promoted from nice-to-have to v1. Schmitt
state-flip introduces sigmoid discontinuity. β=0.9 EMA smoothing
added; new ALPHA_GRAD_SMOOTHED_INDEX slot.
5. q_disagreement_baseline drop: v1 had adaptive baseline as long-EMA
(feedback loop risk). v2 uses structural 0.5 (analytic K=3-with-
Flat-masked random alignment). Drop BASELINE_INDEX slot.
6. Backward gradient scaling clarified: α_grad scales dL/dx (input
gradient flowing back to aux), NOT dL/dW (Q-head's weight grad).
Q-head learns to use the wire freely; gate only controls upstream
flow.
7. 4 hard rules added: feedback_no_hiding,
feedback_no_htod_htoh_only_mapped_pinned,
feedback_kill_runs_on_anomaly_quickly,
pearl_canary_input_freshness_launch_order.
8. Smoke A2 explicit kill criteria table added (8 triggers).
Net ISV slot count unchanged (11), composition shifted: dropped
BASELINE, added SMOOTHED. Total impl cost ~1150 LOC (was ~1060).
Verified against current code:
- TARGET_DIR_ACC_INDEX=372, AUX_DIR_ACC_SHORT_EMA_INDEX=373,
AUX_DIR_PREDICTION_INDEX=375 (sp13_isv_slots.rs)
- set_aux_weight clamp(0.05, 0.3) at gpu_dqn_trainer.rs:14722
(confirms Bug 3 from Smoke A diagnostic)
- mag_concat_qdir precedent at experience_kernels.cu:4560
(direction-conditioning pattern; SP14's wire is the analog)
- state_reset_registry pattern at lines 913-922 (canonical
template for new EMA fold-reset entries)
- branch_0_size = 3 in production config (the K=3 finding)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Designs the SP14 chain on top of SP13 Layer B (HEAD 6657e5626):
1. Sub-project A — stability fixes (3 small bugs found in Smoke A)
- C51 atom-probability floor (ISV-driven from SP4 atom_pos_p99)
- aux_w setter clamp lift [0.05, 0.3] → [0.15, 1.5]
- Stagnation warmup gate at fold boundary
2. Sub-project B — the architectural piece (THIS spec)
- Forward wire: aux_softmax_diff per-bar into direction Q-head input
concat (in_dim+1, fingerprint bump, zero-init new column)
- Earned Gradient Flow pearl — adaptive ISV-driven gradient gating:
* Gate 1 (aux competence) — Schmitt-trigger hysteresis
* Gate 2 (Q-head disagreement) — NEW signal, EMA per-step argmax
mismatch
* ISV-adaptive sigmoid steepness (variance-driven k_aux, k_q)
* Per-epoch warmup ramp
* Anti-gradient-hacking circuit breaker (mesa-opt defense)
- 11 new ISV slots, 3 new GPU kernels, ~1060 LOC total
- HEALTH_DIAG pearl_egf_diag observability line
3. Sub-project C — Adaptive LR (deferred until A+B effects measured)
Motivation from Smoke A diagnostic:
- aux_dir_acc reached 0.61 (signal extraction works)
- val_win_rate stuck 45-48% (no path to action selection)
- WR-flat-while-aux-varies = Q-head directional weights frozen
- 1109 GRAD_CLIP_OUTLIER events (chronic; not noise)
Three parallel diagnostic agents triangulated three interlocking root
causes:
- Slot 375 has zero readers (the wire was scoped but never built)
- C51 raw grad reaches 9.5e6, saturates SP7 budget controller
- aux_w controller muzzled by SP11-era [0.05, 0.3] clamp
The Earned Gradient Flow pearl is a new application of the codebase's
pearl pattern: ISV-driven adaptive controller, but applied to backward-
pass gradient flow instead of forward-pass features. The wire is one-
way (stop-gradient) by default; co-training is earned by both:
(a) aux head demonstrating label competence, AND
(b) Q-head showing it's actually fighting aux signal (informative
disagreement above baseline).
Stability additions hardened against:
- Oscillation around target (Schmitt hysteresis)
- Numerical sigmoid saturation (argument clipping ±30)
- Stale variance EMAs across folds (state-reset-registry)
- Discontinuous warmup transitions (linear ramp)
- Mesa-optimization (gradient-hacking circuit breaker)
- Cold-start sentinel-state spurious gate openings (Pearl-A bootstrap)
Awaiting user review before invoking superpowers:writing-plans for
sub-project B (and a separate small plan for sub-project A).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P0a.T3 v2 implementer's audit revealed `DirectionAction` enum doesn't exist; the
codebase uses an 8-variant fused `ExposureLevel` (ShortSmall/Half/Full, Hold,
LongSmall/Half/Full, Flat) with cross-crate consumers across 77 files and 32+
test files pinning the 8-variant invariant. Atomic Hold elimination would
cascade massively.
User insight (2026-05-04): Hold being FREE is the bug, not Hold itself. MFT
trading legitimately needs multi-bar holds; we want the model to use them
deliberately, not as a CQL-bias lazy default. Holding isn't free in the real
world — broker fees, margin interest, opportunity cost.
v3 reframes as Hold-pricing:
- 4-way action space stays; ExposureLevel::Hold stays; no cross-crate cascade
- 3 new ISV slots (380-382): HOLD_COST_INDEX, HOLD_RATE_TARGET_INDEX,
HOLD_RATE_OBSERVED_EMA_INDEX
- Hold-rate observer: small GPU kernel + Pearls A+D smoothing
- Hold-cost controller: 5-line deficit-driven formula
(excess > target → cost rises 1×→5× base; observed ≤ target → relax)
- Per-bar reward subtraction at action == DIR_HOLD site
- 2 GPU oracle tests for the controller
P0a.T3 cuts from ~250 LOC + 32-test cascade → ~120 LOC additive. T1+T2
already-staged work unchanged. T4/T5/Layer B/C/D structure preserved.
Tension with pearl_event_driven_reward_density_alignment acknowledged in
spec — per-bar Hold cost is exposure-NEGATIVE (away from Hold), models real
economic carry, ISV-bounded by controller. Inverse of the pearl's failure
mode. Faithful reward modeling, not artificial shaping.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three architectural changes in one unified design to push the model from
HFT noise extraction (62% trade rate, sharpe-gaming) toward MFT alpha-hunting:
1. Asymmetric bounded cap (-10/+5) — restores loss aversion erased by
SP11 symmetric cap. 2:1 ratio matches Kahneman/Tversky prospect theory.
Anchor: pearl_audit_unboundedness_for_implicit_asymmetry.
2. Min-hold soft penalty with temperature curriculum — patience requirement
at exit. Soft factor = deficit/(deficit+T), T anneals 50→5 over 50 epochs.
Forces commitment without paralysis in early training.
3. Zero per-bar shaping (gate micro/opp_cost on events) — eliminates
continuous-reward gradient that pulls toward continuous exposure.
Anchor: pearl_event_driven_reward_density_alignment.
Combined: reward fires only on trade events with prospect-theory loss
aversion + commitment requirement. Pure per-trade event-driven Q-learning
properly aligned with per-trade P&L objective.
~50 LOC across 3-4 files. No new ISV slots in Phase 1 (constants only).
Cost ~€1.30 (€0.30 smoke + €1.00 30-epoch validation).
Empirical motivation: train-multi-seed-pmbwn 50-epoch on commit 6a259942e
showed sharpe-gaming pattern (PnL -30% over 8 epochs while sharpe held).
SP11 cap fix unmasked the per-bar shaping bias plus erased loss-aversion
that the unbounded loss path was implicitly providing.
Continues on sp11-reward-as-controlled-subsystem branch — SP12 is
architectural continuation of SP11, not separate work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
smoke-test-6wd2c on commit 61b2fa962 (B1b + 4 bug fix-ups) revealed a 5th
pathology not covered by the previous fix-ups: the mag-ratio canary's
linear-magnitude-ratio formula amplifies whichever component is
intrinsically largest, regardless of whether that's a useful signal.
Popart (trade P&L on segment_complete) is O(100) per fire while the
other 5 components (cf/trail/micro/opp_cost/bonus) are O(0.1-2). Even
with the slot 360 fix preventing total-reward contamination, popart's
intrinsic magnitude makes popart_mag / Σ ≈ 0.93. The controller blend
'winner_weight = ratio' then amplifies popart further. Smoke trajectory:
w_pop=2.0 → 2.44 → 2.57, curiosity_b=30 → 120 → 199, sharpe_ema=10.7 →
2.4 → 0.75 (cascading collapse).
Resolution: z-score normalization. Each component's magnitude divided
by its own running standard deviation before computing the ratio:
popart_z = popart_mag_ema / max(sqrt(popart_var_ema), EPS_DIV) ≈ O(1)
cf_z = cf_mag_ema / max(sqrt(cf_var_ema), EPS_DIV) ≈ O(1)
...
ratio[c] = component_z / Σ component_z ≈ ~1/6 each when stable
Allocates 6 new ISV slots [361..367) for per-component variance EMAs.
Producers: extend popart_component_ema_kernel + reward_component_ema_kernel
to also emit variance via Welford's online algorithm (single-pass).
SP5_SLOT_END = 367, ISV_TOTAL_DIM = 367.
Carries forward main's slot 360 amendment (commit 52c0b7521 on main)
which the sp11 branch was missing, plus this z-score amendment.
Per-component gradient ratios (the original spec intent) don't fix this
either — in DQN there's no per-component gradient pathway; grad norm
scales with current_weight × magnitude, so it's the same bias. Z-score
normalization is the standard scale-invariant measure of significance
and matches what the SP11 controller is trying to express.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User correction: curiosity is the *fix* for the ep1-peak overfitting
pathology, not a hazard to defend against. Reframed §7 from "Risks"
to "Design notes" — curiosity bound is a signal-relative scale, not
a defensive cap.
Fix the contradiction this exposed in the formula: previous
`curiosity_pressure = stagnant_or_worse * curiosity_bound` went to
zero when improving, which would cancel the always-on exploration
the §7 narrative now relies on. Replace with permanent-floor pattern
per pearl_blend_formulas_must_have_permanent_floor:
curiosity_floor = 0.2 * curiosity_bound (CURIOSITY_PERMANENT_FRACTION)
curiosity_dynamic = stagnant_or_worse * curiosity_bound
curiosity_pressure = max(curiosity_dynamic, curiosity_floor)
Now curiosity is always ≥ 20% of bound (anti-overfitting baseline)
and rises toward the bound when stagnant (stagnation breaker).
Updated unit-test guidance to assert pressure > 0 even at z=+10.
CURIOSITY_PERMANENT_FRACTION=0.2 added to Invariant-1 fraction list.
Saboteur-rising-with-improvement reframed as adversarial-load feature
rather than over-stress risk.
Brainstorm spec for SP11. Resolves the policy-stagnation pathology
surfaced in T10 train-multi-seed-xkjkb seed-0 ep0-14: model finds a
stable fixed point at ep1 (peak val sharpe 80.61), then OVERFITS to
it across remaining epochs (decline 80.61 → 70.58). Q-values grow but
val performance declines because reward function has no improvement
pressure.
Architecture (every input ISV-driven):
- Z-score-driven adaptation (no hardcoded "improving" threshold):
improvement_z = val_sharpe_delta_ema / max(val_sharpe_std_ema, EPS)
- 10 ISV outputs: 6 component weights + curiosity_pressure +
saboteur_intensity_mult + adaptive weight_floor + curiosity_bound
- 5 ISV canaries: val_sharpe_delta + val_sharpe_std (Z-score noise
estimate) + 6 per-component grad ratios + saboteur engagement +
PnL magnitude EMA (signal-relative curiosity bound)
- 4 new producer kernels (controller + 3 canary computers)
- Audit + migrate hardcoded cf_weight=0.3 in mse_loss_kernel.cu:318
and c51_loss_kernel.cu:789, plus other shaping multipliers
- NEW reward dimension: curiosity bonus, bounded by PnL magnitude
Per pearl_controller_anchors_isv_driven: every threshold replaced with
sigmoid(z) — no constants encode "what counts as improving". Per
pearl_blend_formulas_must_have_permanent_floor: every weight has
adaptive floor preventing zero-out. Per pearl_engagement_rate_self_
correction: saboteur intensity self-corrects via engagement rate canary.
Per pearl_cold_start_exit_signal_or: improvement signal OR'd from
multiple canaries so single-signal-failure doesn't stall controller.
New pearl authored alongside spec: pearl_reward_as_controlled_subsystem
— meta-principle that every reward path degree of freedom is a unified
controller output. Subsumes controller-anchor pearl at the reward layer.
Scope: 20 ISV slots, 4 producer kernels, audit + migration of
hardcoded reward shaping constants, 1 atomic commit.
~1300-1700 LOC; ~2.5-3 hours subagent work.
Success metric: val_sharpe[ep20] > val_sharpe[ep1] (Fix 33-38 baseline
peaked at ep1; SP11 should shift peak later as model continues
learning).
Brainstorm spec for SP10. Resolves the val-Flat-collapse pathology that
persisted through Fix 33-37: the eval-time argmax in experience_action_
select picks Hold deterministically every val bar (dir_entropy=0,
trade_count=1 in 214k bars) regardless of controller state.
Architecture:
- Delete `if (eval_mode)` argmax branch in direction-selector kernel
- Use temperature-blended Thompson: q_eff = E[Q] + τ × (Thompson - E[Q])
- τ = clamp(intent_eval_divergence / divergence_target, 0.5, 2.0)
- τ self-corrects: collapse → high τ; healthy → low τ; permanent 0.5 floor
- Reuses SP9's intent_eval_divergence_compute_kernel (extended, not new)
Per pearl_controller_anchors_isv_driven: τ is ISV-driven, no
hardcoded constants beyond Invariant 1 numerical anchors (clamp range).
Per pearl_blend_formulas_must_have_permanent_floor: MIN_TEMP=0.5 ensures
eval ALWAYS has stochasticity.
The pearl_thompson_for_distributional_action_selection was about Bellman
TARGET argmax (selector/target symmetry). It does NOT prohibit Thompson
at the rollout selector. SP10 amends the pearl to clarify.
Scope: 1 ISV slot, kernel modification (no new kernel — extend SP9's
producer), 1 consumer kernel rewrite, test update, pearl amendment,
audit doc Fix 38. ~300-500 LOC, single atomic commit per
feedback_no_partial_refactor.
Brainstorm spec for SP9 (Fix 37). Resolves the val-Flat-collapse pathology
surfaced in T10 train-multi-seed-wsnc6 ep1-3: training intent_dist_f=0.47
but eval_dist_f=0.00 because Kelly cap pinned eval mag to Quarter, with
trade_count=1 in 214k bars (chicken-and-egg: cap closed → no trades → no
Kelly samples → cap stays closed).
Architecture per pearl_controller_anchors_isv_driven and
pearl_cold_start_exit_signal_or:
final_kelly_f = max(measured_kelly_f, warmup_floor)
warmup_floor = base_floor × (1 − combined_confidence)
base_floor = clamp(q_var_mag / q_var_mag_ema, MIN, MAX)
combined_confidence = max(statistical, behavioral, temporal)
All thresholds ISV-driven via Pearl D Wiener-α; only Invariant 1 numerical
anchors (clamp ranges, EPS) remain as constants.
Scope: 9 new ISV slots, 6 new GPU producer kernels including a
mandatory eval_dist GPU migration (eliminating DtoH readback at
gpu_backtest_evaluator.rs:1353 per feedback_no_cpu_compute_strict).
~1000-1200 LOC, single atomic commit per feedback_no_partial_refactor.
Pearls authored as part of brainstorm:
- pearl_controller_anchors_isv_driven (regime-encoded constants)
- pearl_cold_start_exit_signal_or (OR'd signals across statistical /
behavioral / temporal axes)
Outcome-driven controller replacing Pearl 2's hardcoded-0 CQL budget and
floor-pinned C51 budget with multiplicative ratio adaptation. Targets
`grad_split_bwd cql/iqn = 2.0` and `c51/iqn = 1.0` per slice (trunk, dir,
mag). Slow EMA (α=0.01) consistent with existing controller pearls; Pearl
A sentinel-bootstrap on fold boundary; Pearl D Wiener-optimal smoothing.
Replaces ghost docstring at fused_training.rs:3409 ("0.10×(1−regime)×health"
formula was never implemented). Pearl 2 keeps owning IQN budget (reference
denominator) and FLATNESS_BASE (consumed by NoisyNet σ pearl); stops
writing CQL/C51/ENS slots so the new controller is the sole driver.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Critical self-review surfaced 15 issues; user directed fixes:
- "a no cpu path": KEEP host-EMA close-out (rule compliance) — but split
into separate Layer D atomic commit (was mis-scoped as Layer A)
- Pearl 4 kept: 3 concrete risks documented (constant-β proof break,
β2 memory reset destabilization, ε numerical envelope), structural
envelope bounds added, ALPHA_META halved, ε-only fall-back path defined
- Pearl 6 Kelly cross-fold persistence carve-out: separate slot range
280..286, NOT in SP4 fold-reset registry, Invariant 1 architectural exception
- Commit ordering Pearls 1 → 3 → 2 → 4 → 5 → 6 → 8 → 1-ext (resolves
Pearl 2 circular dependency on 1+3)
- Acceptance criteria: correctness gates (must pass) + performance gates
(loosened to "not catastrophically negative", within 2σ of pre-SP5)
- Pearl 7 timing: explicitly Layer C step 4, post-Layer-B + 3-seed validation
- Pearl 8 enumeration: 4 slots (TRAIL_DIST_PER_DIR per direction)
- Pearl 9 collapsed: 0 slots (Thompson achieved via Pearl 1's atom adaptation)
- Total slot count corrected: 110 (was 120-128 inconsistent)
Layer structure: A (additive, 8 commits) → B (atomic, 11 consumers) → C
(validation + Pearl 7 investigation) → D (host-EMA close-out, separate
atomic commit). Layer D split off from A's "close-out" because PnL
aggregation pipeline migration is its own architectural concern.
User final review pending before invoking writing-plans skill.
Per user direction:
Q1 (Layer A granularity): per-pearl commits (~9 commits, decision c)
Q2 (Pearl 7 timing): investigate-only in SP5; fix in follow-up
if Bin(2,0.5) persists post-Pearls-1-3 (decision a)
Q3 (Pearl 4 Adam β): include as designed; accept theoretical risk
with Pearls A+D + EPS_CLAMP_FLOOR mitigation;
Layer C smoke monitors for destabilization;
fall-back to ε-only if observed (decision a)
Spec section updated:
- Layer A description: per-pearl commit structure
- Pearl 7 framed as investigation-only with conditional follow-up
- Pearl 4 documents theoretical caveat + mitigation + fallback
User final review pending before invoking writing-plans skill.
SP5 design covers every known adaptive-parameter deferral in the DQN
training loop in a single coherent project. After SP5: zero hardcoded
multipliers, every adaptive value ISV-driven via Pearls A+D.
9 pearls + 1 sweep close-out + 1 validation milestone:
1-3. Per-branch atom span / loss budget / NoisyNet σ (52 slots)
4. Per-group Adam β1/β2/ε ISV-driven (24 slots)
5. Per-branch IQN τ schedule (20 slots)
6. Kelly cap signal-driven floors (6 slots)
7. dist_q/h/f Bin(2,0.5) audit + action_select fix (0-8 slots)
8. Trail stop signal-driven thresholds (6-8 slots)
9. Thompson direction-branch temperature (4 slots)
1-ext. Per-branch C51 num_atoms (4 slots)
Layer A close-out: 5 host-EMA host→GPU migrations
Validation: 3-seed × 50-epoch acceptance gate
Total: 120-128 new ISV slots, ~5000-7500 LOC, 11-13 producer kernels,
~12 consumer migrations.
Layer A (additive infrastructure, ~15 commits) → Layer B (atomic
consumer migration, single coordinated commit) → Layer C (validation +
cleanup). Mirrors SP4's layer pattern.
Triggering data: train-multi-seed-cv2mw 50-epoch L40S baseline
(terminated F0 ep10) revealed magnitude head Q-flatness, eval collapse,
and frozen action distributions. Plus all SP4 close-out + sweep
deferrals folded in per user direction "no deferrals — make a single
plan based on ALL findings".
Spec at:
docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md
User review pending before invoking writing-plans skill.
Layer C close-out C1 redesigned. Original plan claimed
grad_norm_slow_ema_pinned was orphan post-Mech-6 migration; verification
surfaced a SECOND live consumer (fold_warmup_factor_update, commit
4ef1d8ebb) that legitimately reads the slow EMA as cross-fold
steady-state baseline.
The actual defect: the EMA UPDATE at update_adaptive_clip:22720-22737
was host-side `(1-α)*prev + α*obs` arithmetic — exactly the pattern
feedback_no_cpu_compute_strict (saved 2026-05-01) strictly forbids.
Migrated:
- New `update_grad_norm_emas_kernel.cu` — single-thread fused fast+slow
EMA update kernel. Reads `grad_norm_buf[0]`, updates two mapped-pinned
EMA scalars via dev_ptr. `__threadfence_system()` ensures the
`fold_warmup_factor_kernel` consumer sees freshly-written values.
- `launch_update_grad_norm_emas` Rust launcher chained on the
producer's stream — graph-capture-compatible, no host sync.
- update_adaptive_clip's host-side `unsafe { ... }` block replaced
with the GPU launcher call (warn-and-continue on launch failure
mirroring the launch_h_s2_rms_ema / launch_fold_warmup_factor
per-step ISV producer pattern at training_loop.rs:3450/3464).
Preserved:
- grad_norm_slow_ema_pinned mapped-pinned buffer (cross-fold persistent;
legitimate consumer is fold_warmup_factor_update).
- Fixed-α design (FAST_ALPHA=0.1, SLOW_ALPHA=0.001) — Pearls A+D
adaptive α would defeat the cross-fold-baseline semantic the warmup
factor depends on.
- Cold-start sentinel (`prev ≤ 0.0` ⇒ assign obs directly) — same
formula as the deleted host code.
- Host-side update_adaptive_clip early-return guard — kernel only
launches when observed_grad_norm is finite and > 0.
- grad_norm_emas_step_count host counter — scalar control-flow
metadata for warmup-window gating, not compute.
Plan/spec docs updated to remove stale "orphan" claim. State-reset
registry doc-comment + field doc-comments updated to reflect GPU-only
update path. fold_warmup_factor_kernel docstring no longer describes
its grad-norm EMA inputs as host-side-fed.
Build clean, sp4 + state_reset_registry lib tests pass (11/11), 16/16
SP4 producer GPU tests pass on RTX 3050 Ti. No behavior change — pure
architectural fix.
Refs: SP4 Layer C C1 redesigned (was: retire). Original plan
docs/superpowers/plans/2026-04-30-sp4-signal-driven-magnitude-control.md
lines 2184-2221.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PEARL A — First-observation bootstrap: eliminates Xavier-derived
formulas (2.33 z-score, √2 std, √(2/K_in)) from the bootstrap section.
Sentinel ISV[X]=0 at fold reset; producer step 0 detects sentinel,
replaces directly with step_observation. Subsequent steps EMA-blend.
Consumer cold-start safety via .max(1.0) numerical floor only (Adam-ε
category, not magnitude). Truly zero magnitude constants in bootstrap.
PEARL B — Fused per-param-group statistics oracle: producer count
36 → 14. Per-group fused kernel reads (params, grads, adam_m, adam_v)
once and computes WEIGHT_BOUND, ADAM_M_BOUND, ADAM_V_BOUND, WD_RATE
in one multi-pass operation. Trunk's oracle adds Pass E for L1_LAMBDA
gradient-direction entropy. 4× memory bandwidth reduction. Cleaner
conceptual unit (per-param-group bounds = one oracle).
PEARL C — Engagement-rate self-correction: detects post-clamp
feedback-loop saturation. For in-kernel clamps, theoretical engagement
rate = 1% (top 1% by p99 definition). Producer-side rate-deficit EMA
detects sustained deviation; force-bumps bound to step_max when
detected. Resolves the "in-kernel feedback loop accepted" limitation
from first draft. Per-Adam-kernel block-shared-memory engagement
counter (no atomicAdd), block-wide reduce, host-side rate-deficit EMA.
PEARL D — Wiener-optimal adaptive α: replaces all hardcoded EMA rates
across 14 new producers AND 7 existing pre-SP4 producers. Per-step:
α* = diff_var / (diff_var + sample_var + ε_num). Theoretically optimal
under Wiener-filter analysis; subsumes Pearl A as t=0 edge case
(both vars=0 → α=1 → first-observation replacement). On stationary
signals: α→0 (smooth). On non-stationary: α→1 (track). Eliminates
the recursion problem (α controlled by signal stats, not another α).
ε_num = 1e-8 (Adam-ε numerical category). 3 floats of state per
producer. 36+ hardcoded α values eliminated codebase-wide.
Limitations section restructured: 6 of 8 first-draft limitations
RESOLVED by pearls (in-kernel feedback loop, smoke time-budget,
producer plumbing, stale-bound, 8 carved-out items, magnitude bootstrap
formulas). Remaining 6 limitations are genuinely irreducible (F0
launch-scheduling variance, Layer B atomic flip risk, novel-pearl
field-validation, equilibrium-formula non-stationarity, Pearl C
counter-state cost, Pearl D state cost).
SP4 now closes 100% of the magnitude/regularization/EMA-rate surface.
No hardcoded scalars remain in the entire bound/clamp/EMA chain.
Producer count: 14. ISV slot count: 36. Unit tests: 36.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds two new producer signals into SP4 scope, eliminating the earlier
"carved out" exception:
WEIGHT DECAY (per param-group, 7 new ISV slots):
λ = |w·g| / max(||w||², ε)
Derived from equilibrium analysis of d/dt(||w||²) = 2(w·g) - 2λ||w||².
The equilibrium gradient-projection-onto-weight-direction divided by
weight norm. Same theoretical-derivation category as Adam β values.
EMA half-life α=0.005 (~140 steps). Bootstrap 1.0.
L1 LAMBDA (trunk only, 1 new ISV slot — NOVEL PEARL):
λ = (mean(|g|) / mean(|w|)) × D
where D = (log K - H_observed) / log K is gradient-direction entropy deficit
and H_observed = -Σ p[i]·log p[i], p[i] = ||g[:,i]|| / Σ ||g[:,j]||
L1 regularization-strength derives from gradient-direction entropy
deficit across input features. When gradient is uniform across features
(D≈0): network hasn't differentiated, λ=0 (no pruning). When gradient
concentrates on few features (D≈1): network has identified what matters,
λ ramps up to prune the rest. Self-curriculum — L1 strength tracks the
emergence of feature differentiation.
This extends pearl_adaptive_moe_lambda (regularization strength = EMA-
tracked deficit of regularized quantity) to feature-redundancy domain.
Pearl-name candidate (post-validation): pearl_signal_driven_regularisation_strength.
Bootstrap λ=0 means cold-start = no L1 pruning; ramp-up only after
gradient differentiates. Worst-case behavior is "L1 disabled" — graceful.
Total ISV slot count: 28 → 36. Total producer kernels: 28 → 36.
Effort estimate: 3000-4000 → 3500-4500 LOC, 1-1.5 → 1.5-2 weeks.
Acknowledged limitations updated: removed item #8 (carve-out) since
no carve-outs remain. Added items for L1 pearl novelty (untested) and
weight decay equilibrium-formula non-stationarity. Both have graceful
worst-case behavior and explicit validation criteria (#10 and #11) to
detect anomalies.
SP4 now closes 100% of the magnitude/regularization surface — no
hardcoded scalars remain in the entire chain. AdamW config fields for
weight_decay and l1_lambda removed from HyperParams to prevent
accidental hardcoding regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second-pass self-review found a new critical issue: under "diagnostic =
clamp engagement", post-clamp diagnostics (Mech 9 weights, Mech 5 Adam
m/v slots 36-43, slots 44-45) NEVER fire — because post-clamp |v| ≤
bound by construction, so producer-side comparison always returns false.
Fix: split diagnostic implementation by clamp location.
- Buffer-based clamps (Mech 1, 2, 10): diagnostic stays in producer
(reads pre-clamp buffer, fires when step_max > bound).
- In-kernel clamps (Mech 6, 9): diagnostic lives INSIDE the Adam
kernel at the clamp step. Each Adam kernel takes a `diag_slot` arg
alongside `weight_clamp_max_abs`; on clamp engagement, writes
`nan_flags_buf[diag_slot] = 1`. Idempotent per-thread store (all
threads writing 1, race-free per existing convention). No atomicAdd.
Without this fix, slots 36-45 would be dead diagnostics under SP4 —
detecting nothing, providing no signal. With this fix, every diagnostic
slot fires on its corresponding clamp engagement regardless of where
the clamp lives.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review found multiple problems with the first draft. This revision
addresses all 8 critical issues:
1. P² parallelization claim was WRONG — P² is sequential.
Replaced with dynamic-range histogram (3-pass single-block kernel:
max-reduce → log-spaced bin → cumulative-from-top to find p99).
256 bins → ~0.4% quantile precision; numerical-precision derivation
in same theoretical-constant category as floating-point precision.
2. Bootstrap values had wrong magnitudes — used Xavier σ instead of
p99 of max-element. Recomputed: WEIGHT_BOUND[group] = 2.33 ×
√(2/K_in[group]); H_S2_BOUND = 2.33 × √2 ≈ 3.3. All bootstraps
now from theoretical p99 under Xavier-init, not std.
3. EMA half-life of 700 steps (α=0.001) didn't converge in 5-epoch
smoke. Revised α=0.005 (~140 steps) for weight/Adam producers —
reaches ~99% convergence within one fold's training. Smoke now
validates steady-state behavior, not just bootstrap.
4. Weight decay, L1 lambda, CLIP_MULTIPLIER were listed in scope but
undesignable in producer-consumer pattern. CARVED OUT explicitly:
weight decay + L1 λ → separate research-spec; CLIP_MULTIPLIER and
MIN_CLIP subsumed by SP4's GRAD_CLIP_BOUND slot.
5. F0 ≥ 45 acceptance criterion was uncertain. Revised to F0 ≥ 37.5
(matches the 1e30-effectively-unclamped diagnostic smoke). The
~8-point F0 variance from launch-scheduling-shift is independent
of clamp value; SP4 cannot guarantee F0=45 even with ideal design.
6. Per-param-group p99 plumbing concretized: each producer takes
(offset, length) launch args; main DQN params buffer sliced into
trunk/value/branch using existing param_sizes layout knowledge.
7. Pre/post-clamp feedback loop EXPLICIT: producer runs BEFORE
consumer for buffer-based clamps (h_s2, target_q, atom_pos —
in captured graph immediately before clamp). Producer runs AFTER
for in-kernel clamps (weights, Adam m/v — feedback loop accepted
with documented soft-anchor dynamics). No more hand-waving.
8. Unit test strategy: per-producer kernel test with synthetic
Gaussian input → known p99 ≈ 2.33 → assert |computed - 2.33|<5%.
28 tests total. Catches algorithm bugs before L40S smoke.
Effort estimate revised UP: 3500-4500 LOC (from 2-3000), 1.5-2.5 weeks
(from 1-2). 28 producer kernels + 28 unit tests is more plumbing than
first draft assumed.
Self-review limitations section now explicit about: in-kernel feedback
loop accepted, smoke time-budget marginally validates Adam EMAs, F0
may not return to 45, Layer B atomic-flip risk, plumbing density,
stale-by-one-step bound, histogram-precision is design choice not
tuning, carved-out items remain hardcoded post-SP4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comprehensive design replacing every hardcoded magnitude multiplier in
the SP3 mechanism stack (Mechs 1, 2, 5, 6, 9, 10) plus pre-SP3 mechs in
the same magnitude-control surface, per feedback_isv_for_adaptive_bounds
and feedback_adaptive_not_tuned.
Core principle: the BOUND lives in an ISV slot, computed by a producer
kernel as p99 EMA of observed signal magnitude. Consumer reads the slot
and clamps directly — no multiplier between ISV read and clamp. Cold-
start ε from theoretical-init bootstrap (Xavier, etc.) — same theoretical-
constant category as Adam β values, not tuning knobs.
Architecture:
- 28 new ISV slots (7 base bounds × per-param-group split where appropriate)
- Per-signal P² (Jain-Chlamtac) quantile producer kernels
- Diagnostic = clamp engagement (sticky flag from producer's max-comparison)
- Migration in 3 layers: additive infra → atomic consumer flip → smoke
Out of scope: theoretical/structural constants (Adam β, Xavier formula,
attention 1/√d, hidden_dim, num_atoms). EMA rates stay as documented
statistical-design parameters (half-life ≈ observation time-window).
Estimated: ~2-3000 LOC across 3 layers, 1-2 weeks, 1 L40S smoke.
Awaiting user spec review before writing implementation plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
9 substantive issues addressed inline:
1. ISV-driven design elevated from 'if applicable' to MANDATORY for
all dynamic bounds in SP1 fixes. Numerical-stability ε bounds are
the only carve-out (Invariant 1). Hardcoded tuning constants for
dynamic ranges explicitly rejected.
2. F0 Sharpe regression criterion changed from absolute (≥55) to
ratio-based (≥95% of latest baseline; floor 53.08 currently).
Prevents iterative erosion across multiple fix commits.
3. 'F1 trending positive' replaced with concrete monotone-improvement
test: Best Sharpe at last epoch ≥ Best Sharpe at first epoch of
the same fold.
4. Pass criterion distinguishes NaN-CLAMPED-TO-ZERO (failure) from
'Genuine grad collapse' (legitimate observation, permitted) per
the existing infrastructure from commit d1808df14.
5. Multi-source NaN scenarios explicitly supported — γ + β may
identify multiple kernels; SP1 fixes ALL within the same cycle.
6. F0-safety paper-review gate added BEFORE smoke validation. Audit
doc carries 'F0 risk' (low/medium/high) per proposed fix; high-
risk fixes get math-on-paper inspection before consuming L40S.
7. Audit doc structure now requires 'ISV bound option' column —
forces ISV-first thinking at audit stage, not as afterthought.
8. Anti-patterns expanded: micro-clamping (per-op clamping that hides
upstream causes); combining unrelated fixes (anti-pattern of the
rich-commit principle); hardcoded constants for dynamic bounds.
9. 48-slot allocation explicitly justified (24 used + 12 new + 12
headroom) AND marked reviewable by SP2 if right-size differs.
All 5 design sections preserved structurally; revisions integrated
inline. Per brainstorming skill: spec self-review fixes applied
without re-review cycle.
The original PAUSED state was motivated by measurement-artefact hunt
exit. The bug-hunt cycle is complete (commits a86fba2b1, b8788511c).
A new pathology has surfaced: ff00af68a's UCB count bonus activation
causes selector/target asymmetry → F0 Q-drift kill at epoch 2 →
F1+F2 cascade. Verified by paired DIAG smokes (smoke-test-qlz7t fail,
smoke-test-wmsht pass).
Thompson sampling on C51+IQN distributions eliminates the asymmetry
by construction (sample from learned distribution; no augment-then-
argmax step). Net code-surface decrease — replaces eps-greedy +
Boltzmann + UCB with one principled mechanism.
Plan C Phase 2 execution begins on branch plan-c-phase-2-thompson.
T1.6 implementer correctly identified that the gate input dim is
ml_core::state_layout::STATE_DIM=128, not the literal 42 the spec/plan
incorrectly stated. The 42-dim figure was the bar-feature subset; the
actual state vector is 128-dim (42 features + portfolio + MTF + OFI
padded to 128 for cuBLAS alignment).
Updated spec §3 architecture diagram, §4.1 gate subnetwork description
+ parameter count (3,272 → 8,776), and plan header architecture line.
Implementation in commit 28c707f6a is correct; this commit just makes
the spec match the implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per feedback_no_htod_htoh_only_mapped_pinned.md (newly recorded): every
CPU<->GPU path in this redesign uses mapped pinned memory exclusively.
No cudaMemcpy HtoD, no Vec-to-Vec defensive copies, including in test
code. CPU is strictly read-only on the production surface.
Plan changes:
- New Task 2.0 promotes MappedF32Buffer / MappedI32Buffer from
distributional_q_tests.rs local definitions to a shared
crates/ml/src/cuda_pipeline/mapped_pinned.rs module so all kernel
test wrappers (Test 0.F, upcoming MoE tests) share one
implementation. Adds write_from_slice helper for direct host_ptr
write (no memcpy).
- Task 2.1 test wrapper rewritten to allocate mapped pinned buffers
+ write to host_ptr + read GPU-written output via host_ptr. No more
memcpy_stod / memcpy_dtov in test code.
Spec: new section 6.4 codifies the mapped-pinned-only constraint and
references the shared module + reference implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end investigation (2026-04-27) confirmed RegimeConditionalDQN is
vestigial decoration — 3 heads constructed at training start but only
trending_head ever receives gradient updates. GpuDqnTrainer (the actual
production GPU trainer) has zero references to RegimeType/regime
routing; experience replay inserts go to trending_head.memory only;
ranging_head and volatile_head stay at random init for the entire
training run. Several support APIs (get_count_bonuses_branched, config,
get_state_dim) hardcode-delegate to trending_head, ignoring the regime
split entirely. Per `feedback_no_hiding.md` (wire up or delete) and the
user's preference to fix not delete: the design wires regime
conditioning properly via Mixture-of-Experts replacing the vestigial
3-head architecture.
Pearl introduced and saved as `pearl_learned_gate_subsumes_handcoded.md`:
when the network already sees the heuristic's inputs, a learned gate
strictly subsumes any hand-coded discretization. This is the
load-bearing rationale — ADX/CUSUM are already at state indices 40/41,
so threshold-based regime classification is a strict information
bottleneck the gate can recover and improve on.
Design summary:
- Architecture: shared GRN trunk -> K=8 small expert MLPs (256->64->256
bottleneck per expert, ~33k params each) -> learned gating network
(state[42]->64->8 softmax) -> mixed h_s2 -> existing branching heads
+ C51 + IQN dual head. Soft full mixture (no top-k hardcoding); gate
emerges peaky or flat from data. Anti-collapse load-balancing aux
loss with default lambda=0.01 (configurable hyperparameter, not a
kernel constant) prevents init-noise-dominated single-expert lock-in
without forcing uniform utilization. User-confirmed signal:
"collapses don't recover well in this codebase".
- 9 new ISV slots (118-126: per-expert utilization EMA + gate entropy
EMA), GPU-driven producer per
`pearl_cold_path_no_exception_to_gpu_drives.md`.
- 3 new small CUDA kernels (moe_mixture_forward/backward,
moe_load_balance_loss) + 1 ISV producer; everything else is cuBLAS-
reusable. CUDA Graph capture compatible.
- Atomic deletion (no fallback): regime_conditional.rs (~700 LOC),
RegimeType enum, classify_from_features, RegimeMetrics,
RegimeClassConfig, 4 DQNConfig regime threshold fields, per-regime
3-file checkpoint format. DQNAgentType becomes thin wrapper over
single DQN. Old checkpoints fail loudly with layout-fingerprint
mismatch.
- 5-layer testing strategy (unit kernels, smoke, gate-differentiation
validation, L40S production validation with explicit kill criteria,
architecture-hash backward-incompat).
Out of scope (explicit): top-k routing, per-expert action heads,
hierarchical MoE, regime-conditional CountBonus/NoisySigma broadcast,
expert warm-start from existing trending checkpoint, CVaR action
selection on mixed C51 distribution.
Precondition: the in-progress use_* flag cleanup + count_bonus
[f32; N] refactor lands as its own commit before MoE implementation
begins, per `feedback_no_partial_refactor.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Retracts the prior SUPERSEDED footer (commit 42ffd6aad). The technical
proposal still stands — Thompson sampling on C51+IQN distributions is
the canonical action selector for distributional RL (Bellemare 2017,
Dabney 2018). Phase 0 tests and the Aggregation Contract are sound
math/engineering regardless of the measurement-bug findings.
What changed is the URGENCY framing, not the validity. The val-Flat-
collapse / Short-collapse observations cited as motivating evidence
were partly distorted by three measurement bugs (a86fba2b1 + b8788511c)
in the diagnostic infrastructure. The "ship Thompson NOW because
val_dir_dist collapses to 80%+ Hold/Flat" narrative dissolves; the
"Thompson is the principled action selector for our distributional
model" narrative stands.
Sequencing: PAUSED pending evidence from a fresh L40S 30-epoch
baseline (train-f8h6q, 2026-04-27 12:20) on post-fix code. The
baseline is a bug-hunting expedition — kill on anomaly, diagnose,
fix, re-run per feedback_stop_on_anomaly.md. Once healthy baseline
established, Phase 2 ships as principled improvement with clean A/B
against the trustworthy post-fix metrics.
Plans B / C / D are PAUSED, not cancelled. Files remain in
docs/superpowers/plans/. Resumption gate: post-fix baseline run is
bug-free or all surfaced bugs are addressed.
