Critical self-review surfaced 15 issues; user directed fixes:
- "a no cpu path": KEEP host-EMA close-out (rule compliance) — but split
into separate Layer D atomic commit (was mis-scoped as Layer A)
- Pearl 4 kept: 3 concrete risks documented (constant-β proof break,
β2 memory reset destabilization, ε numerical envelope), structural
envelope bounds added, ALPHA_META halved, ε-only fall-back path defined
- Pearl 6 Kelly cross-fold persistence carve-out: separate slot range
280..286, NOT in SP4 fold-reset registry, Invariant 1 architectural exception
- Commit ordering Pearls 1 → 3 → 2 → 4 → 5 → 6 → 8 → 1-ext (resolves
Pearl 2 circular dependency on 1+3)
- Acceptance criteria: correctness gates (must pass) + performance gates
(loosened to "not catastrophically negative", within 2σ of pre-SP5)
- Pearl 7 timing: explicitly Layer C step 4, post-Layer-B + 3-seed validation
- Pearl 8 enumeration: 4 slots (TRAIL_DIST_PER_DIR per direction)
- Pearl 9 collapsed: 0 slots (Thompson achieved via Pearl 1's atom adaptation)
- Total slot count corrected: 110 (was 120-128 inconsistent)
Layer structure: A (additive, 8 commits) → B (atomic, 11 consumers) → C
(validation + Pearl 7 investigation) → D (host-EMA close-out, separate
atomic commit). Layer D split off from A's "close-out" because PnL
aggregation pipeline migration is its own architectural concern.
User final review pending before invoking writing-plans skill.
Per user direction:
Q1 (Layer A granularity): per-pearl commits (~9 commits, decision c)
Q2 (Pearl 7 timing): investigate-only in SP5; fix in follow-up
if Bin(2,0.5) persists post-Pearls-1-3 (decision a)
Q3 (Pearl 4 Adam β): include as designed; accept theoretical risk
with Pearls A+D + EPS_CLAMP_FLOOR mitigation;
Layer C smoke monitors for destabilization;
fall-back to ε-only if observed (decision a)
Spec section updated:
- Layer A description: per-pearl commit structure
- Pearl 7 framed as investigation-only with conditional follow-up
- Pearl 4 documents theoretical caveat + mitigation + fallback
User final review pending before invoking writing-plans skill.
SP5 design covers every known adaptive-parameter deferral in the DQN
training loop in a single coherent project. After SP5: zero hardcoded
multipliers, every adaptive value ISV-driven via Pearls A+D.
9 pearls + 1 sweep close-out + 1 validation milestone:
1-3. Per-branch atom span / loss budget / NoisyNet σ (52 slots)
4. Per-group Adam β1/β2/ε ISV-driven (24 slots)
5. Per-branch IQN τ schedule (20 slots)
6. Kelly cap signal-driven floors (6 slots)
7. dist_q/h/f Bin(2,0.5) audit + action_select fix (0-8 slots)
8. Trail stop signal-driven thresholds (6-8 slots)
9. Thompson direction-branch temperature (4 slots)
1-ext. Per-branch C51 num_atoms (4 slots)
Layer A close-out: 5 host-EMA host→GPU migrations
Validation: 3-seed × 50-epoch acceptance gate
Total: 120-128 new ISV slots, ~5000-7500 LOC, 11-13 producer kernels,
~12 consumer migrations.
Layer A (additive infrastructure, ~15 commits) → Layer B (atomic
consumer migration, single coordinated commit) → Layer C (validation +
cleanup). Mirrors SP4's layer pattern.
Triggering data: train-multi-seed-cv2mw 50-epoch L40S baseline
(terminated F0 ep10) revealed magnitude head Q-flatness, eval collapse,
and frozen action distributions. Plus all SP4 close-out + sweep
deferrals folded in per user direction "no deferrals — make a single
plan based on ALL findings".
Spec at:
docs/superpowers/specs/2026-05-01-sp5-magnitude-differentiation-and-eval-collapse-design.md
User review pending before invoking writing-plans skill.
Layer C close-out C1 redesigned. Original plan claimed
grad_norm_slow_ema_pinned was orphan post-Mech-6 migration; verification
surfaced a SECOND live consumer (fold_warmup_factor_update, commit
4ef1d8ebb) that legitimately reads the slow EMA as cross-fold
steady-state baseline.
The actual defect: the EMA UPDATE at update_adaptive_clip:22720-22737
was host-side `(1-α)*prev + α*obs` arithmetic — exactly the pattern
feedback_no_cpu_compute_strict (saved 2026-05-01) strictly forbids.
Migrated:
- New `update_grad_norm_emas_kernel.cu` — single-thread fused fast+slow
EMA update kernel. Reads `grad_norm_buf[0]`, updates two mapped-pinned
EMA scalars via dev_ptr. `__threadfence_system()` ensures the
`fold_warmup_factor_kernel` consumer sees freshly-written values.
- `launch_update_grad_norm_emas` Rust launcher chained on the
producer's stream — graph-capture-compatible, no host sync.
- update_adaptive_clip's host-side `unsafe { ... }` block replaced
with the GPU launcher call (warn-and-continue on launch failure
mirroring the launch_h_s2_rms_ema / launch_fold_warmup_factor
per-step ISV producer pattern at training_loop.rs:3450/3464).
Preserved:
- grad_norm_slow_ema_pinned mapped-pinned buffer (cross-fold persistent;
legitimate consumer is fold_warmup_factor_update).
- Fixed-α design (FAST_ALPHA=0.1, SLOW_ALPHA=0.001) — Pearls A+D
adaptive α would defeat the cross-fold-baseline semantic the warmup
factor depends on.
- Cold-start sentinel (`prev ≤ 0.0` ⇒ assign obs directly) — same
formula as the deleted host code.
- Host-side update_adaptive_clip early-return guard — kernel only
launches when observed_grad_norm is finite and > 0.
- grad_norm_emas_step_count host counter — scalar control-flow
metadata for warmup-window gating, not compute.
Plan/spec docs updated to remove stale "orphan" claim. State-reset
registry doc-comment + field doc-comments updated to reflect GPU-only
update path. fold_warmup_factor_kernel docstring no longer describes
its grad-norm EMA inputs as host-side-fed.
Build clean, sp4 + state_reset_registry lib tests pass (11/11), 16/16
SP4 producer GPU tests pass on RTX 3050 Ti. No behavior change — pure
architectural fix.
Refs: SP4 Layer C C1 redesigned (was: retire). Original plan
docs/superpowers/plans/2026-04-30-sp4-signal-driven-magnitude-control.md
lines 2184-2221.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PEARL A — First-observation bootstrap: eliminates Xavier-derived
formulas (2.33 z-score, √2 std, √(2/K_in)) from the bootstrap section.
Sentinel ISV[X]=0 at fold reset; producer step 0 detects sentinel,
replaces directly with step_observation. Subsequent steps EMA-blend.
Consumer cold-start safety via .max(1.0) numerical floor only (Adam-ε
category, not magnitude). Truly zero magnitude constants in bootstrap.
PEARL B — Fused per-param-group statistics oracle: producer count
36 → 14. Per-group fused kernel reads (params, grads, adam_m, adam_v)
once and computes WEIGHT_BOUND, ADAM_M_BOUND, ADAM_V_BOUND, WD_RATE
in one multi-pass operation. Trunk's oracle adds Pass E for L1_LAMBDA
gradient-direction entropy. 4× memory bandwidth reduction. Cleaner
conceptual unit (per-param-group bounds = one oracle).
PEARL C — Engagement-rate self-correction: detects post-clamp
feedback-loop saturation. For in-kernel clamps, theoretical engagement
rate = 1% (top 1% by p99 definition). Producer-side rate-deficit EMA
detects sustained deviation; force-bumps bound to step_max when
detected. Resolves the "in-kernel feedback loop accepted" limitation
from first draft. Per-Adam-kernel block-shared-memory engagement
counter (no atomicAdd), block-wide reduce, host-side rate-deficit EMA.
PEARL D — Wiener-optimal adaptive α: replaces all hardcoded EMA rates
across 14 new producers AND 7 existing pre-SP4 producers. Per-step:
α* = diff_var / (diff_var + sample_var + ε_num). Theoretically optimal
under Wiener-filter analysis; subsumes Pearl A as t=0 edge case
(both vars=0 → α=1 → first-observation replacement). On stationary
signals: α→0 (smooth). On non-stationary: α→1 (track). Eliminates
the recursion problem (α controlled by signal stats, not another α).
ε_num = 1e-8 (Adam-ε numerical category). 3 floats of state per
producer. 36+ hardcoded α values eliminated codebase-wide.
Limitations section restructured: 6 of 8 first-draft limitations
RESOLVED by pearls (in-kernel feedback loop, smoke time-budget,
producer plumbing, stale-bound, 8 carved-out items, magnitude bootstrap
formulas). Remaining 6 limitations are genuinely irreducible (F0
launch-scheduling variance, Layer B atomic flip risk, novel-pearl
field-validation, equilibrium-formula non-stationarity, Pearl C
counter-state cost, Pearl D state cost).
SP4 now closes 100% of the magnitude/regularization/EMA-rate surface.
No hardcoded scalars remain in the entire bound/clamp/EMA chain.
Producer count: 14. ISV slot count: 36. Unit tests: 36.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds two new producer signals into SP4 scope, eliminating the earlier
"carved out" exception:
WEIGHT DECAY (per param-group, 7 new ISV slots):
λ = |w·g| / max(||w||², ε)
Derived from equilibrium analysis of d/dt(||w||²) = 2(w·g) - 2λ||w||².
The equilibrium gradient-projection-onto-weight-direction divided by
weight norm. Same theoretical-derivation category as Adam β values.
EMA half-life α=0.005 (~140 steps). Bootstrap 1.0.
L1 LAMBDA (trunk only, 1 new ISV slot — NOVEL PEARL):
λ = (mean(|g|) / mean(|w|)) × D
where D = (log K - H_observed) / log K is gradient-direction entropy deficit
and H_observed = -Σ p[i]·log p[i], p[i] = ||g[:,i]|| / Σ ||g[:,j]||
L1 regularization-strength derives from gradient-direction entropy
deficit across input features. When gradient is uniform across features
(D≈0): network hasn't differentiated, λ=0 (no pruning). When gradient
concentrates on few features (D≈1): network has identified what matters,
λ ramps up to prune the rest. Self-curriculum — L1 strength tracks the
emergence of feature differentiation.
This extends pearl_adaptive_moe_lambda (regularization strength = EMA-
tracked deficit of regularized quantity) to feature-redundancy domain.
Pearl-name candidate (post-validation): pearl_signal_driven_regularisation_strength.
Bootstrap λ=0 means cold-start = no L1 pruning; ramp-up only after
gradient differentiates. Worst-case behavior is "L1 disabled" — graceful.
Total ISV slot count: 28 → 36. Total producer kernels: 28 → 36.
Effort estimate: 3000-4000 → 3500-4500 LOC, 1-1.5 → 1.5-2 weeks.
Acknowledged limitations updated: removed item #8 (carve-out) since
no carve-outs remain. Added items for L1 pearl novelty (untested) and
weight decay equilibrium-formula non-stationarity. Both have graceful
worst-case behavior and explicit validation criteria (#10 and #11) to
detect anomalies.
SP4 now closes 100% of the magnitude/regularization surface — no
hardcoded scalars remain in the entire chain. AdamW config fields for
weight_decay and l1_lambda removed from HyperParams to prevent
accidental hardcoding regression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second-pass self-review found a new critical issue: under "diagnostic =
clamp engagement", post-clamp diagnostics (Mech 9 weights, Mech 5 Adam
m/v slots 36-43, slots 44-45) NEVER fire — because post-clamp |v| ≤
bound by construction, so producer-side comparison always returns false.
Fix: split diagnostic implementation by clamp location.
- Buffer-based clamps (Mech 1, 2, 10): diagnostic stays in producer
(reads pre-clamp buffer, fires when step_max > bound).
- In-kernel clamps (Mech 6, 9): diagnostic lives INSIDE the Adam
kernel at the clamp step. Each Adam kernel takes a `diag_slot` arg
alongside `weight_clamp_max_abs`; on clamp engagement, writes
`nan_flags_buf[diag_slot] = 1`. Idempotent per-thread store (all
threads writing 1, race-free per existing convention). No atomicAdd.
Without this fix, slots 36-45 would be dead diagnostics under SP4 —
detecting nothing, providing no signal. With this fix, every diagnostic
slot fires on its corresponding clamp engagement regardless of where
the clamp lives.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review found multiple problems with the first draft. This revision
addresses all 8 critical issues:
1. P² parallelization claim was WRONG — P² is sequential.
Replaced with dynamic-range histogram (3-pass single-block kernel:
max-reduce → log-spaced bin → cumulative-from-top to find p99).
256 bins → ~0.4% quantile precision; numerical-precision derivation
in same theoretical-constant category as floating-point precision.
2. Bootstrap values had wrong magnitudes — used Xavier σ instead of
p99 of max-element. Recomputed: WEIGHT_BOUND[group] = 2.33 ×
√(2/K_in[group]); H_S2_BOUND = 2.33 × √2 ≈ 3.3. All bootstraps
now from theoretical p99 under Xavier-init, not std.
3. EMA half-life of 700 steps (α=0.001) didn't converge in 5-epoch
smoke. Revised α=0.005 (~140 steps) for weight/Adam producers —
reaches ~99% convergence within one fold's training. Smoke now
validates steady-state behavior, not just bootstrap.
4. Weight decay, L1 lambda, CLIP_MULTIPLIER were listed in scope but
undesignable in producer-consumer pattern. CARVED OUT explicitly:
weight decay + L1 λ → separate research-spec; CLIP_MULTIPLIER and
MIN_CLIP subsumed by SP4's GRAD_CLIP_BOUND slot.
5. F0 ≥ 45 acceptance criterion was uncertain. Revised to F0 ≥ 37.5
(matches the 1e30-effectively-unclamped diagnostic smoke). The
~8-point F0 variance from launch-scheduling-shift is independent
of clamp value; SP4 cannot guarantee F0=45 even with ideal design.
6. Per-param-group p99 plumbing concretized: each producer takes
(offset, length) launch args; main DQN params buffer sliced into
trunk/value/branch using existing param_sizes layout knowledge.
7. Pre/post-clamp feedback loop EXPLICIT: producer runs BEFORE
consumer for buffer-based clamps (h_s2, target_q, atom_pos —
in captured graph immediately before clamp). Producer runs AFTER
for in-kernel clamps (weights, Adam m/v — feedback loop accepted
with documented soft-anchor dynamics). No more hand-waving.
8. Unit test strategy: per-producer kernel test with synthetic
Gaussian input → known p99 ≈ 2.33 → assert |computed - 2.33|<5%.
28 tests total. Catches algorithm bugs before L40S smoke.
Effort estimate revised UP: 3500-4500 LOC (from 2-3000), 1.5-2.5 weeks
(from 1-2). 28 producer kernels + 28 unit tests is more plumbing than
first draft assumed.
Self-review limitations section now explicit about: in-kernel feedback
loop accepted, smoke time-budget marginally validates Adam EMAs, F0
may not return to 45, Layer B atomic-flip risk, plumbing density,
stale-by-one-step bound, histogram-precision is design choice not
tuning, carved-out items remain hardcoded post-SP4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comprehensive design replacing every hardcoded magnitude multiplier in
the SP3 mechanism stack (Mechs 1, 2, 5, 6, 9, 10) plus pre-SP3 mechs in
the same magnitude-control surface, per feedback_isv_for_adaptive_bounds
and feedback_adaptive_not_tuned.
Core principle: the BOUND lives in an ISV slot, computed by a producer
kernel as p99 EMA of observed signal magnitude. Consumer reads the slot
and clamps directly — no multiplier between ISV read and clamp. Cold-
start ε from theoretical-init bootstrap (Xavier, etc.) — same theoretical-
constant category as Adam β values, not tuning knobs.
Architecture:
- 28 new ISV slots (7 base bounds × per-param-group split where appropriate)
- Per-signal P² (Jain-Chlamtac) quantile producer kernels
- Diagnostic = clamp engagement (sticky flag from producer's max-comparison)
- Migration in 3 layers: additive infra → atomic consumer flip → smoke
Out of scope: theoretical/structural constants (Adam β, Xavier formula,
attention 1/√d, hidden_dim, num_atoms). EMA rates stay as documented
statistical-design parameters (half-life ≈ observation time-window).
Estimated: ~2-3000 LOC across 3 layers, 1-2 weeks, 1 L40S smoke.
Awaiting user spec review before writing implementation plan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implementation plan for SP1 (Sub-project 1 of 3) of the numerical
stability investigation. Follows the γ + β methodology from the spec
at docs/superpowers/specs/2026-04-29-numerical-stability-investigation-design.md.
8 tasks across 4 phases:
- Phase A (Task 1): γ read-only audit producing docs/dqn-backward-nan-audit.md
- Phase B (Tasks 2-5): always-landing β instrumentation expanding
nan_flags_buf 24→48 with 12 new backward-kernel NaN check slots
+ 12 reserved slots for future coverage
- Phase C (Task 6): surgical fix(es), content-driven by audit + smoke
topology, ISV-driven for any dynamic bound (mandatory)
- Phase D (Task 7): multi-fold L40S smoke validation against 7 pass
criteria (F0 ≥ 95% baseline, F1+F2 monotone improvement, zero
NaN-CLAMPED-TO-ZERO, all 48 NaN flag slots remain at zero)
- Closure (Task 8): audit doc closure, memory entry, SP2/SP3 handoff
Operating principles (mandatory per spec):
- No deferrals — anomalies discovered during investigation get fixed
within SP1, not punted to SP2/SP3
- Combined RELATED fixes ship as rich commits (per
feedback_no_partial_refactor)
- ISV-driven design for any dynamic bound
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
9 substantive issues addressed inline:
1. ISV-driven design elevated from 'if applicable' to MANDATORY for
all dynamic bounds in SP1 fixes. Numerical-stability ε bounds are
the only carve-out (Invariant 1). Hardcoded tuning constants for
dynamic ranges explicitly rejected.
2. F0 Sharpe regression criterion changed from absolute (≥55) to
ratio-based (≥95% of latest baseline; floor 53.08 currently).
Prevents iterative erosion across multiple fix commits.
3. 'F1 trending positive' replaced with concrete monotone-improvement
test: Best Sharpe at last epoch ≥ Best Sharpe at first epoch of
the same fold.
4. Pass criterion distinguishes NaN-CLAMPED-TO-ZERO (failure) from
'Genuine grad collapse' (legitimate observation, permitted) per
the existing infrastructure from commit d1808df14.
5. Multi-source NaN scenarios explicitly supported — γ + β may
identify multiple kernels; SP1 fixes ALL within the same cycle.
6. F0-safety paper-review gate added BEFORE smoke validation. Audit
doc carries 'F0 risk' (low/medium/high) per proposed fix; high-
risk fixes get math-on-paper inspection before consuming L40S.
7. Audit doc structure now requires 'ISV bound option' column —
forces ISV-first thinking at audit stage, not as afterthought.
8. Anti-patterns expanded: micro-clamping (per-op clamping that hides
upstream causes); combining unrelated fixes (anti-pattern of the
rich-commit principle); hardcoded constants for dynamic bounds.
9. 48-slot allocation explicitly justified (24 used + 12 new + 12
headroom) AND marked reviewable by SP2 if right-size differs.
All 5 design sections preserved structurally; revisions integrated
inline. Per brainstorming skill: spec self-review fixes applied
without re-review cycle.
Plan C as authored assumed:
- c51_probs_dir = post-softmax probabilities (didn't exist on collector;
only raw exp_b_logits is materialised)
- atom_values = single global linear support (production uses per-sample
per-direction adaptive [v_min, v_max, delta_z] + optional atom_positions)
- iqn_quantiles_dir = rollout-side IQN inference (doesn't exist; IQN is
training-only)
Amended kernel signature uses the production architecture:
- b_logits_dir [N, b0_size, n_atoms] — raw direction-branch logits
- per_sample_support [N, b0_size, 3] — adaptive per-direction support
- atom_positions [b0_size, n_atoms] — non-linear positions (NULL = linear)
- n_atoms
Direction-branch Thompson is now single-distribution over C51 (with adaptive
support), not joint C51+IQN. Single-distribution still provides the
principled posterior sample that fixes the UCB selector/target asymmetry —
the goal of Plan C is preserved, just sourced from the existing rollout-time
distribution instead of a non-existent IQN inference path.
New device-inline helpers: softmax_c51_inline, compute_atom_values_inline.
Plan + audit doc updated. Phase 0 standalone test kernel gained two new
entry points (direction_thompson_v2_test, argmax_eq_v2_test) matching the
amended production API; original Plan A entry points retained for tests
0.B-0.F. Tasks 3+4 (buffer wiring) unblocked — collector/evaluator already
have per_sample_support_buf and exp_b_logits in production.
The original PAUSED state was motivated by measurement-artefact hunt
exit. The bug-hunt cycle is complete (commits a86fba2b1, b8788511c).
A new pathology has surfaced: ff00af68a's UCB count bonus activation
causes selector/target asymmetry → F0 Q-drift kill at epoch 2 →
F1+F2 cascade. Verified by paired DIAG smokes (smoke-test-qlz7t fail,
smoke-test-wmsht pass).
Thompson sampling on C51+IQN distributions eliminates the asymmetry
by construction (sample from learned distribution; no augment-then-
argmax step). Net code-surface decrease — replaces eps-greedy +
Boltzmann + UCB with one principled mechanism.
Plan C Phase 2 execution begins on branch plan-c-phase-2-thompson.
T1.6 implementer correctly identified that the gate input dim is
ml_core::state_layout::STATE_DIM=128, not the literal 42 the spec/plan
incorrectly stated. The 42-dim figure was the bar-feature subset; the
actual state vector is 128-dim (42 features + portfolio + MTF + OFI
padded to 128 for cuBLAS alignment).
Updated spec §3 architecture diagram, §4.1 gate subnetwork description
+ parameter count (3,272 → 8,776), and plan header architecture line.
Implementation in commit 28c707f6a is correct; this commit just makes
the spec match the implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per feedback_no_htod_htoh_only_mapped_pinned.md (newly recorded): every
CPU<->GPU path in this redesign uses mapped pinned memory exclusively.
No cudaMemcpy HtoD, no Vec-to-Vec defensive copies, including in test
code. CPU is strictly read-only on the production surface.
Plan changes:
- New Task 2.0 promotes MappedF32Buffer / MappedI32Buffer from
distributional_q_tests.rs local definitions to a shared
crates/ml/src/cuda_pipeline/mapped_pinned.rs module so all kernel
test wrappers (Test 0.F, upcoming MoE tests) share one
implementation. Adds write_from_slice helper for direct host_ptr
write (no memcpy).
- Task 2.1 test wrapper rewritten to allocate mapped pinned buffers
+ write to host_ptr + read GPU-written output via host_ptr. No more
memcpy_stod / memcpy_dtov in test code.
Spec: new section 6.4 codifies the mapped-pinned-only constraint and
references the shared module + reference implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end investigation (2026-04-27) confirmed RegimeConditionalDQN is
vestigial decoration — 3 heads constructed at training start but only
trending_head ever receives gradient updates. GpuDqnTrainer (the actual
production GPU trainer) has zero references to RegimeType/regime
routing; experience replay inserts go to trending_head.memory only;
ranging_head and volatile_head stay at random init for the entire
training run. Several support APIs (get_count_bonuses_branched, config,
get_state_dim) hardcode-delegate to trending_head, ignoring the regime
split entirely. Per `feedback_no_hiding.md` (wire up or delete) and the
user's preference to fix not delete: the design wires regime
conditioning properly via Mixture-of-Experts replacing the vestigial
3-head architecture.
Pearl introduced and saved as `pearl_learned_gate_subsumes_handcoded.md`:
when the network already sees the heuristic's inputs, a learned gate
strictly subsumes any hand-coded discretization. This is the
load-bearing rationale — ADX/CUSUM are already at state indices 40/41,
so threshold-based regime classification is a strict information
bottleneck the gate can recover and improve on.
Design summary:
- Architecture: shared GRN trunk -> K=8 small expert MLPs (256->64->256
bottleneck per expert, ~33k params each) -> learned gating network
(state[42]->64->8 softmax) -> mixed h_s2 -> existing branching heads
+ C51 + IQN dual head. Soft full mixture (no top-k hardcoding); gate
emerges peaky or flat from data. Anti-collapse load-balancing aux
loss with default lambda=0.01 (configurable hyperparameter, not a
kernel constant) prevents init-noise-dominated single-expert lock-in
without forcing uniform utilization. User-confirmed signal:
"collapses don't recover well in this codebase".
- 9 new ISV slots (118-126: per-expert utilization EMA + gate entropy
EMA), GPU-driven producer per
`pearl_cold_path_no_exception_to_gpu_drives.md`.
- 3 new small CUDA kernels (moe_mixture_forward/backward,
moe_load_balance_loss) + 1 ISV producer; everything else is cuBLAS-
reusable. CUDA Graph capture compatible.
- Atomic deletion (no fallback): regime_conditional.rs (~700 LOC),
RegimeType enum, classify_from_features, RegimeMetrics,
RegimeClassConfig, 4 DQNConfig regime threshold fields, per-regime
3-file checkpoint format. DQNAgentType becomes thin wrapper over
single DQN. Old checkpoints fail loudly with layout-fingerprint
mismatch.
- 5-layer testing strategy (unit kernels, smoke, gate-differentiation
validation, L40S production validation with explicit kill criteria,
architecture-hash backward-incompat).
Out of scope (explicit): top-k routing, per-expert action heads,
hierarchical MoE, regime-conditional CountBonus/NoisySigma broadcast,
expert warm-start from existing trending checkpoint, CVaR action
selection on mixed C51 distribution.
Precondition: the in-progress use_* flag cleanup + count_bonus
[f32; N] refactor lands as its own commit before MoE implementation
begins, per `feedback_no_partial_refactor.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Retracts the prior SUPERSEDED footer (commit 42ffd6aad). The technical
proposal still stands — Thompson sampling on C51+IQN distributions is
the canonical action selector for distributional RL (Bellemare 2017,
Dabney 2018). Phase 0 tests and the Aggregation Contract are sound
math/engineering regardless of the measurement-bug findings.
What changed is the URGENCY framing, not the validity. The val-Flat-
collapse / Short-collapse observations cited as motivating evidence
were partly distorted by three measurement bugs (a86fba2b1 + b8788511c)
in the diagnostic infrastructure. The "ship Thompson NOW because
val_dir_dist collapses to 80%+ Hold/Flat" narrative dissolves; the
"Thompson is the principled action selector for our distributional
model" narrative stands.
Sequencing: PAUSED pending evidence from a fresh L40S 30-epoch
baseline (train-f8h6q, 2026-04-27 12:20) on post-fix code. The
baseline is a bug-hunting expedition — kill on anomaly, diagnose,
fix, re-run per feedback_stop_on_anomaly.md. Once healthy baseline
established, Phase 2 ships as principled improvement with clean A/B
against the trustworthy post-fix metrics.
Plans B / C / D are PAUSED, not cancelled. Files remain in
docs/superpowers/plans/. Resumption gate: post-fix baseline run is
bug-free or all surfaced bugs are addressed.
The val-Flat-collapse / Short-collapse / C51 expected-Q bias hypothesis
that motivated the 4-plan distributional-RL Thompson rollout was
largely a measurement artefact in the diagnostic infrastructure, not
a real policy pathology. Three layered bugs in actions_history_buf
init + reader + mag_stats attribution conspired to inflate val_dir_dist
Short, inflate active_frac, and pin wr_h/wr_f to zero. After fixing
all three (commits a86fba2b1 + b8788511c), val_dir_dist matches
val_picked_dir_dist within ~5pp — no collapse, diverse picks.
Status footer added to the spec documenting:
- what was actually wrong (3 measurement bugs)
- what the post-fix data shows (mild passivity bias from early
training, not a structural collapse)
- what is preserved (Phase 0 unit tests as latent infrastructure
for any future distributional Q-head; Aggregation Contract
pearl as a sound engineering invariant)
- what is cancelled (Plans B / C / D — Phase 1 audit, Phase 2
Thompson integration, Phase 3 long verification — superseded)
- future revival condition (if fresh L40S 30-epoch on post-fix
code shows val_picked_dir_dist itself collapsing toward Hold/Flat,
reopen)
Plan files remain in docs/superpowers/plans/ as historical record.
User correctly identified that CPU mirror function tests don't test
the production GPU code path. A bug shared between mirror and kernel
(translated identically wrong) would slip through. Mirror tests + a
single GPU bridge test were a weak compromise.
GPU-direct testing strategy:
- All Phase 0 kernel-correctness tests (0.A, 0.B, 0.C, 0.D, 0.F):
launch tiny test-only kernels with the SAME math the Phase 2
production kernel will use; assert properties of the output.
- Test 0.E (synthetic edge discovery): stays CPU. It tests an
ALGORITHMIC PROPERTY of Thompson exploration (does it discover
edge if edge exists?), not a kernel correctness property.
- All Phase 2 unit tests (2.A-2.D): GPU-direct against the
modified production kernel.
- Phase 0.F (real checkpoint extraction): unchanged — already GPU.
Local development uses RTX 3050 GPU (per memory user_dev_environment.md).
CI runs --ignored flag to skip GPU tests on CPU-only runners.
Time budget: Phase 0 was 1-2 days (CPU mirror); now 2-3 days
(GPU-direct, includes kernel wrapper setup half-day).
Other delta:
- Phase 0 deliverable file renamed: distributional_q.rs ->
distributional_q_tests.rs (no mirror functions, just tests +
kernel wrappers).
- Phase 2 unit tests rephrased to launch production kernel rather
than compare against CPU mirror.
- L1 verification gate runtime: seconds -> minutes (GPU launch
overhead per test).
The user's intuition was right: testing production directly is the
honest approach. Mirror was an optimization that traded correctness
for speed; with local GPU available the optimization isn't needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-review identified 8 major + 8 medium + 15 minor issues. All fixed:
MAJORS:
M1 Test 0.A: clarified — compare argmax(E[Q]), Boltzmann(E[Q]),
and Thompson sampling distributions; assertion is on relative
ordering across all three.
M2 IQN quantile count: replaced hardcoded `5` with N_IQN_QUANTILES
constant (defined per task #147 fixed-quantile design).
M3 "Converged checkpoint" definition: ≥60 epochs trained AND
val_sharpe stabilised (no >10% change over last 10 epochs).
Cites prior 60-epoch validation runs (task #80, train-7rgqd).
M4 R3 reframed: replaced "no issue" handwave with explicit
by-design tradeoff acknowledgment + cost analysis. Wasted
exploration is the cost of finding out whether edge exists.
M5 Test 0.D σ_long=0.05 justified: chosen to match expected order
of magnitude given typical |return| ~ 50bps; Phase 0.F
validates against real checkpoint.
M6 rng_ctr post-increment: clarified — matches existing pattern
at experience_kernels.cu:858 (no behaviour change).
M7 train_active_frac instrumentation: NEW Phase 2 deliverable —
existing HEALTH_DIAG only has val_active_frac, but L3 verifies
training-time active_frac. Spec now explicitly adds this
~10-line metrics.rs change as a Phase 2 deliverable.
M8 eps_dir cleanup code-level detail: explicit reference to
experience_kernels.cu lines 814-865; remove eps_dir from both
static EPS_FLOOR clamp AND adaptive boost block; verify
variable can be removed from kernel signature via grep.
MEDIUMS:
Med1 Current C51/IQN combination: clarified that compute_expected_q
blends per training schedule; Phase 2 replaces with explicit
0.5*E_C51 + 0.5*E_IQN equal weighting; Phase 0.F verifies.
Med2 Eval mode phrasing: "eval mode already sets eps=0 in existing
kernel" — no semantic override, factually correct.
Med3 Magnitude σ claim: clarified — magnitude branch likely has σ
bias in OPPOSITE direction (Full has larger σ; UCB would
prefer Full and worsen saturation). Empirical verification
deferred. Phase 0.F should also report per-magnitude σ.
Med4 Hierarchical sampling claim corrected: it's not about
balancing 50/50 (already 50/50). It's about decoupling
cluster-best decisions; clarified.
Med5 n_atoms vs N_IQN_QUANTILES: clarified — n_atoms variable per
config (currently 51); N_IQN_QUANTILES fixed at 5.
Med6 Conviction code: removed pseudo-code; references existing
implementation at experience_kernels.cu:1091; provides
implementation hint for E[Q] reuse.
Med7 Q-target propagation: clarified — uses full distribution
(C51 atom projection / IQN quantile regression), not just
E[Q]. Thompson modifies action selection only.
Med8 References: added Thompson 1933 (original), Bellemare 2017
(C51), Dabney 2018 (IQN) for theoretical foundations.
MINORS:
Min1 Date updated to 2026-04-27.
Min2-3 Argmax monotonic /2 simplified out — argmax(a+b) =
argmax((a+b)/2). Code clarity improved.
Min4 P(argmax picks Long) = 0 deterministic; reframed assertion.
Min5 Test 0.F structural assertions added: σ_C51[FLAT] < 0.01 ×
σ_C51[LONG]; same for IQN; E[Q_FLAT] > E[Q_LONG]; argmax
picks FLAT; Thompson P(LONG)+P(SHORT) ≥ 0.20.
Min6 -INFINITY → CUDART_INF_F (CUDA convention).
Min7 dir_idx scope: comment notes it's declared earlier in kernel.
Min8 action_select args: explicit — three buffers exist on GPU
but not currently passed; new params, no new buffers.
Min9 Phase 0 time math: 5 hours tests + 1 hour enumeration + 2
hours 0.F + (3 hours runtime if checkpoint training needed,
runs in parallel). Honest budget.
Min10 "Two-stream" → "5-Layer Gate" header.
Min11 Plan 5 reference uses full path consistently.
Min12 Plan B time budget: explicit 1 day if pass; 2-5 days if bug.
Min13 active_frac: clarified Long+Short combined, not per direction.
Min14 train_active_frac: now in Phase 2 deliverables (see M7).
Min15 "20 mechanisms" → 21, with sub-counts in section headers.
Spec now 615 lines, comprehensive coverage of:
- Pearl + theoretical foundation
- Problem statement (with bias-might-be-correct caveat)
- Architecture (Thompson at training, argmax at eval)
- Train vs eval distinct selectors with behavior-change disclosure
- Direction-only scope with magnitude σ-bias warning
- Conviction stays E[Q]-based (no Kelly cap jitter)
- Interaction matrix: 21 mechanisms in 3 categories
- 6 v2 enhancements documented + deferred
- Phase 0/1/2/3 with tests, exit gates, time budgets
- 5-layer verification + train_active_frac instrumentation
- 8 risks with mitigations + 5 stop conditions
- What v1 doesn't touch (referencing interaction matrix)
- References (Thompson 1933, Bellemare 2017, Dabney 2018, etc.)
- Aggregation contract (project-wide pearl, enforced)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-user direction: every existing mechanism in the DQN system MUST be
explicitly considered for interaction with Thompson, and Thompson itself
MUST be examined for system-specific improvements beyond vanilla.
INTERACTION MATRIX (3 categories, 20 mechanisms):
Category 1 — Compose with Thompson (no change required):
Counterfactual reward, B.2 novelty bonus, PopArt, Saboteur,
Curiosity, NoisyNets/VSN, Distillation, CQL, Polyak target EMA,
HER, PER, Replay warm-start, Multi-fold validation harness.
Category 2 — Trivially adapt to Thompson (one-line changes):
D7/N7 contrarian sign flip (negate the SAMPLE), cosine epsilon
schedule (still applies to mag/ord/urg), per-sample epsilon (IQL
expectile gap), adaptive Boltzmann tau (still applies to mag/ord/urg).
Category 3 — Take precedence over Thompson (hard constraints):
Plan-based action lock (Thompson sample discarded if plan active),
per-magnitude Kelly cap, trail stop, capital floor breach.
Critical insights from the audit:
1. NoisyNets is ALREADY a form of training-time Thompson at the
parameter level. Output-space Thompson stacks on top —
total exploration = parameter-space ⊗ output-space (multiplicative).
2. Curiosity is ORTHOGONAL to Thompson — Thompson explores actions
whose Q is uncertain; curiosity explores states whose dynamics
are uncertain. Both axes desirable; no conflict.
3. Plan lock takes precedence; same as currently with Boltzmann.
OUTSIDE-THE-BOX v2 ENHANCEMENTS (deferred to follow-up specs):
v2.1 Triple-source Thompson (C51 + IQN + Ensemble) — incorporate
the existing ensemble Q-head as 3rd uncertainty source.
v2.2 Persistent Thompson (anti-churn for HFT) — bias sampling
toward current direction, ISV-driven; reduces tx_cost from
Long/Short oscillation across bars.
v2.3 CVaR-aware eval (risk-adjusted deployment) — eval picks
argmax(E[Q] − λ·CVaR_α[Q]); risk-aware decision making for
production with real capital.
v2.4 Information-Directed Sampling (Russo & Van Roy 2014) — picks
action minimizing regret²/info_gain; more efficient than
vanilla Thompson when learning saturates.
v2.5 Hierarchical Thompson on (trade vs no-trade) → (which
direction) — addresses 50/50 structural advantage of no-trade.
v2.6 Composition with curiosity-driven exploration — explicit
coupling beyond reward-side composition.
Each v2 enhancement gets its own spec/plan when prioritised. Vanilla
Thompson is v1; ships first; verified independently.
ALSO FIXED (from earlier self-review):
- Pearl claim softened: only the C51 Hold/Flat bias is directly
attributed; other historic bugs had different mechanisms.
- TFT entry removed from contract table — TFT is a Variable
Selection Network (feature processor), not a Q-head. Replaced
with generic "Future Q-head additions" placeholder.
- Eval direction = argmax E[Q] explicitly flagged as a behavior
change from current Boltzmann-with-tau (val_dir_dist will be
more concentrated than current).
- Phase 0.E budget reduced (1-2 hours, not 1 day) — synthetic
bandit is ~50 lines of Rust, not full RL training loop.
- Phase 0 enumerates existing checkpoints before training new one.
- Architecture diagram parenthesis fixed.
- Conviction implementation note: compute E[Q] once, reuse for
conviction AND eval-mode argmax — no redundant computation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User correctly challenged the "band-aid removal" framing. All fixes
shipped during the val-Flat-collapse investigation addressed real bugs
at their respective layers and should be preserved:
- Kelly cap warm-branch (0c9d1ee39): post-decision physics layer.
Thompson-independent. KEEP.
- Train Return display + Sharpe annualization (non-tau parts of
7a3d88646): display/metric layer. Thompson-independent. KEEP.
- Direction Boltzmann tau-floor (tau part of 7a3d88646) +
adaptive eps_dir floor (d54b49efc): gates inside direction-branch
action selection. Phase 2 replaces direction-branch action
selection wholesale (eps-greedy + Boltzmann → Thompson), so these
direction-only code paths become structurally unreachable.
The latter two are NOT band-aids being removed because Thompson is
better. They are dead code being cleaned up because Thompson replaces
the surrounding mechanism. Magnitude/order/urgency branches keep their
existing eps-greedy + Boltzmann + tau-floor + EPS_FLOOR paths intact.
Reframed Phase 3 deliverable: "direction-branch dead-code cleanup"
with explicit rationale (per feedback_no_legacy_aliases.md and
feedback_no_partial_refactor.md). 0.5 day budget instead of 1.
Also clarified eval action selection: argmax of (E[Q_C51]+E[Q_IQN])/2
is correct. Bellman backup is a Q-learning UPDATE rule, not an
action-selection rule. Once Q is learned, optimal policy is greedy
argmax of learned Q. Online Bellman lookahead at eval would require
a forward model of market dynamics — not available, not standard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revises the C51-bias spec after deeper review surfaced 12 design gaps,
of which 4 were critical:
1. Train-only vs train+eval ambiguity — UCB at eval would conflate
"model recommends Long" with "model is uncertain about Long",
inflating reported edge. CRITICAL for trading where eval drives
real capital decisions.
2. Thompson sampling is more principled than UCB:
- parameter-free (no κ to tune)
- uses distribution directly without scalar reduction
- naturally explore-exploit balanced via distribution shape
3. c51_alpha is the wrong blend weight (it's C51-vs-MSE-warmup, not
C51-vs-IQN). Equal-weight average of C51 and IQN samples is the
structural choice — no tuned blend weight needed.
4. The bias might be CORRECT BEHAVIOUR — model rationally choosing
Flat when no edge has been discovered. Phase 0 must include a
synthetic-edge test (controlled MDP with KNOWN positive Long
expected value) to verify Thompson can discover edge if it exists.
Other gaps fixed:
- Eval at argmax E[Q] (not Boltzmann, not Thompson)
- Pearl wording broadened to cover ensembles + future methods
- Ensemble Q-head added to aggregation contract table
- Explicit caveat: NEVER extend Thompson to magnitude branch (would
worsen existing magnitude saturation)
- Phase 0.F uses CONVERGED checkpoint (≥30 epochs), not 2-epoch run
- L4 long smoke (30 epochs, ~1 hour) added — Thompson edge discovery
needs longer feedback loop than 5 epochs
- Phase 3 explicitly removes eps-floor + tau-floor band-aids
(Thompson replaces direction Boltzmann; band-aids become dead code)
- Conviction stays E[Q]-based, not sample-based (avoid Kelly cap
jitter from stochastic samples)
Architecture (Thompson only, no UCB):
TRAINING: dir_idx = argmax(0.5 × (sample_C51(d) + sample_IQN(d)))
magnitude/order/urgency: existing Boltzmann + ε-greedy
EVAL: dir_idx = argmax(0.5 × (E[Q_C51] + E[Q_IQN]))
magnitude/order/urgency: existing Boltzmann (eval mode)
Direction-branch ε-greedy + Boltzmann are REMOVED — Thompson is the
exploration mechanism. No new GPU buffers; existing C51 atoms + IQN
quantiles passed to action_select.
5-7 days active work across 4 sub-plans; each gets its own
writing-plans cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Designs the structural fix for the C51 expected-Q Hold/Flat bias exposed
by the Kelly val-Flat-collapse fix. The bias is a manifestation of a
deeper project-wide pearl:
"Distributional RL aggregation discards uncertainty;
action selection must restore it."
Any value head representing Q as a distribution (atoms, quantiles,
ensembles) MUST expose both E[Q] and σ(Q) to action selection. Boltzmann
on E[Q] alone produces structural bias toward low-variance actions
regardless of expected payoff — the C51+IQN Flat-attractor is one
instance of this lost-information pattern.
Fix: extract σ(Q) from BOTH C51 atoms (closed form) and IQN quantiles
(IQR/1.349), blend by loss-time weight, feed Q_eff = E[Q] + κ·σ
(κ=1.0 structural identity) to direction-branch Boltzmann ONLY.
4-phase implementation:
Phase 0 — TDD hypothesis verification (Rust mirror functions + 5 unit
tests including GPU integration on real checkpoint)
Phase 1 — Audit existing reward levers (B.2, CF, PopArt, Q-target)
via 6 unit tests; fix any bugs found
Phase 2 — UCB integration: new compute_q_with_uncertainty kernel,
modified action_select, Rust orchestration, project-wide
aggregation contract in dqn-wire-up-audit.md
Phase 3 — Verification per Plan 5 Task 5 multi-seed × multi-fold
5-layer verification gate; 8 risks with mitigations; 4 stop conditions
that halt execution and force redesign.
Existing band-aid fixes (Kelly cap, eps-floor, tau-floor) stay — they
address symptoms at different layers. UCB adds the missing aggregation
step that was the common root across all the symptoms.
Direction-branch only — magnitude/order/urgency don't have the
Flat-attractor (atom-mass collapse asymmetry).
5-7 days active work across 4 sub-plans; each gets its own
writing-plans cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A first dispatch of monolithic Task 2c was refused with a second scope
assessment that surfaced three architectural facts beyond what 2a/2b
had captured:
(d) attn_layer_norm_bwd_dx is a SIMPLIFIED element-wise approximation
(d_x = d_out * gamma / std), not the full LN Jacobian. Reusing
it in the GRN backward would propagate the approximation into
trunk gradients silently. Task 2c.1 must write a new full
Jacobian: (1/D) * rstd * (D * d_out_g - sum_d(d_out_g) -
normed * sum_d(d_out_g * normed)).
(e) IQN target-trunk migration is not a clean swap. iqn_compute_
target_h_s2 reproduces the legacy LeakyReLU trunk independently
in CUDA. Deleting it requires a target_encoder_forward_only
extraction on the target path (Task 4-style refactor on the
target side). Sub-task in itself.
(f) Three relu_mask removal sites have three different save-buffer
lifetimes (online trunk per-step, IQN-aux per-step in different
trainer, target-sync per-target-update). Three plumbing tasks,
not one.
Decomposition:
- 2c.1: grn_kernel.cu with full LN Jacobian backward + GLU + ELU +
residual-add. Module compiles standalone (additive, dead code).
- 2c.2: gpu_grn.rs Rust wrapper with 5 save-for-backward state
buffers per block (not 3 as the prior plan suggested).
- 2c.3+4: ATOMIC commit — compute_param_sizes 86→95 + fingerprint
rewrite + xavier init for 13 new tensors + all 98 padded_byte_
offset migrations + 3 relu_mask sites with different save-buffer
plumbing each + iqn target trunk delete + target_encoder_forward_
only + mag_concat RMS-match adaptive ISV.
- 2c.5: docs flip.
No code changes. Plan-doc only.
After a first dispatch attempt of monolithic Task 2 (GRN ADOPT) was
correctly refused with a thorough scope assessment, this revision
records the decomposition and the structural facts that drove it:
1. layout_fingerprint_seed() only fingerprints ISV slots, not
param-tensor layout. The "fingerprint will auto-update" assumption
in the original Task 2 spec was wrong for param-tensor reshuffles.
2. compute_param_sizes has 86 tensors (docstring saying 42 is stale).
Inserting GRN's 9 sub-tensors shifts 82 downstream tensors and
98 padded_byte_offset call sites.
3. At least 12 kernels consume h_s2 expecting ReLU non-negativity.
GRN's LayerNorm output is zero-mean (signed). Per-consumer
verification needed before swap.
4. crates/ml-supervised::tft::gated_residual is incompatible as a
port: different formula (sigmoid gate, not GLU split) and
incompatible tensor abstraction. New CUDA kernels from scratch.
Decomposition:
- Task 2a (research, no code): audit h_s2 consumers for ReLU vs
LayerNorm semantics. Output: per-consumer table.
- Task 2b (small, checkpoint break): extend fingerprint seed to
include param-tensor names + sizes. Pearl-aligned: complete
fingerprint coverage rather than partial.
- Task 2c (large, checkpoint break): GRN kernels + 98 call-site
migration + h_s2 consumer shims (per 2a). Blocked on 2a+2b.
Recommended order updated to thread 2a → 2b → 2c. Tasks 1, 6, 3
remain independent of 2c and can land in parallel where useful.
Pearl rules section added documenting the safety constraints
applied throughout the plan.
No code changes. Plan-doc only.
Comprehensive revision to match landed Plan 1+2+3 codebase state. Pattern
mirrors the Plan 3 second revision: per-task "Reality reconciliation"
blocks at the top of each task body identifying what's stale vs. landed,
with concrete file paths in the actual cuda_pipeline tree.
Added:
- Task dependency graph with recommended execution order:
5 (light ISV) → 4 (refactor) → 2 (GRN ADOPT) → 1 (Full VSN) →
6 (aux heads) → 3 (multi-Q IQN) → 7 (audit) → 8 (Argo)
- Per-task implementation surface with concrete trunk hooks:
- Task 1: pre-h_s1 VSN gate in batched_forward.rs::forward_online_raw
- Task 2: GRN audit confirms ADOPT branch (GRN absent from DQN trunk;
only in ml-supervised TFT). Replaces h_s1/h_s2 Linear blocks.
- Task 3: re-scoped — IQN already runs num_quantiles=32 with random τ.
Task is CONSTRAINING to fixed τ ∈ {.05, .25, .5, .75, .95}.
- Task 4: pure Rust API split (no kernel changes, no checkpoint break)
around existing forward_online_raw structure
- Task 5: split into Mode A (light, pre-Task-1, 3 ISV slots) and
Mode B (full, post-Task-1, 7 ISV slots). Mode A recommended first.
- Task 6: aux head loss scaled by ISV[LEARNING_HEALTH] per pearl
- Checkpoint-break consolidation note: Tasks 2/1/6/3 each break checkpoint
compatibility; land in sequence with no Argo run between (one fingerprint
shift per commit; final Argo at Task 8 amortizes retraining cost)
- Task 8 absorbs Plan 3's deferred Argo Tier 1 gate (combined validation)
No code changes.
Plan 4 was drafted 2026-04-24 against an earlier Plan 3 design. After
Plan 3 landed (commits 44539d8f4..3cb083f18), the pre-plan gate
referenced slot names that don't exist in the landed implementation:
- PLAN_PARAMS_0_EMA_INDEX → became READINESS_EMA_INDEX (Task 4)
- STATE_KL_THRESHOLD_EMA_INDEX → eliminated; Task 7 uses kernel-internal
trailing-EMA-of-self pattern (no separate threshold slot)
- TEMPORAL_REWARD_{PERSIST,REGIME_SHIFT,CONSISTENCY}_EMA_INDEX → consolidated
into rc[5] → ISV[68] REWARD_BONUS_EMA_INDEX via Plan 3 Task 6a/6b/6c
Updated:
- Pre-plan slot-existence check matches landed slot names
- Validation-doc gate softened: Plan 3 Argo Tier 1 PASS becomes OPTIONAL;
local 5-epoch multi-fold smoke is the de-facto gate (user choice
to proceed without burning Argo cycles)
- Status table at top shows landed Plan 3 baseline (ISV_TOTAL_DIM=87,
PS_STRIDE=43, fingerprint at [85,86]) and per-task difficulty estimate
- Reality-reconciliation note documents the rename trail
No task body changes; subsequent commits will revise individual task
specs as they become next-up for execution.
Tasks 1, 2, 3, 5 landed on main between a59e7599c and a0abc3da3. The
remaining Task 4/6/7/8/9 bodies in the plan had accumulated drift:
- Task 4: allocated slot 49 (already PLAN_THRESHOLD_INDEX); EMA'd
plan_params[0] (kernel compares against readiness).
- Task 6: allocated 59/60/61 (now collides with Plan 2 Q-quantile
[50..58) and Task 1 reward EMAs [63..69)); required 6 new PS memo
fields and included tuned 0.5e-4f penalty.
- Task 7: allocated 58/63/64 (63/64 collide with REWARD_TRAIL_EMA /
REWARD_MICRO_EMA from Task 1); amplification formula had tuned
2.0× trigger, 0.02 decay, 1.0/2.0 endpoints.
- Task 8: referenced trainer helpers that don't exist
(sample_state_feature_pair, step_scripted, replay_insert_with_
priority_scale); tuned priority_scale=0.5.
- Task 9: used pre-pivot CPU-compute AdaptiveMonitor pattern with
tuned 0.9/0.1 EMA rates.
This revision:
- Adds per-task "Reality reconciliation" sub-header flagging the
stale premise being fixed.
- Marks landed tasks with ✅ LANDED <SHA> and records actual outcomes
(vs. the planned outcomes the original text described).
- Rewrites Task 4/6/7/8/9 bodies to use tail-append slot allocation
(indices recomputed from ISV_TOTAL_DIM at impl time), GPU kernel
producer + read-only AdaptiveMonitor consumer pattern, and
ISV-derived adaptive coefficients in place of tuned constants.
- Splits Task 6 into 6a/6b/6c with independent PS-slot additions
(MIN_PNL / REGIME_SHIFT_BAR / PRE_ENTRY_CONVICTION_EMAs).
- Enumerates concrete trainer-helper prereqs for Task 8.
- Updates Task 10 metric bands to match actual landed ISV slot names.
- Updates exit criteria summary to check off what landed.
No code changes.
Aligns Plan 3 with post-Plan-2 reality:
1. AdaptiveController → AdaptiveMonitor throughout (spec §4.C.6
2026-04-24 revision). Tasks 4 (plan-threshold) and 9 (cql_alpha
seed-coupled) become GPU kernel + read-only CPU monitor pairs
instead of CPU-compute controllers — uniform with Plan 1's
tau/epsilon/gamma/kelly_cap and Plan 2's per-branch γ pattern.
2. Stale ISV slot indices [49..65) replaced with tail-append language.
Current post-Plan-2 ISV_TOTAL_DIM = 63; new slots start at 63 and
grow. Fingerprint at 61-62 auto-re-tails.
3. Architecture paragraph updated to document replay-seed orchestration
(Task 8 dqn_replay_seed.rs) as constructor-time cold-path pre-training
— not a GPU-drives violation.
4. Prerequisites section clarified: Plan 2 state — ISV_TOTAL_DIM 63,
TLOB at SL_TLOB_START, layout fingerprint auto-recomputes.
No scope changes to individual Tasks 1-10. Just terminology + slot
indexing aligned with current main state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User pearl (2026-04-24): the DQN already has every primitive TLOB needs
(gpu_attention, batched_forward/backward, GpuLinear, cuBLASLt handles,
cuda_autograd). Port TLOB's Q/K/V + attention as a composition of
existing primitives. Random-init, trainable end-to-end from the DQN's
reward signal. Uniform with Mamba2, IQL, atoms/γ/τ/ε — all of which
already follow this pattern.
Eliminates:
- ONNX Runtime dependency (was already dead — stripped from ml-supervised)
- Separate supervised pretraining pipeline
- "Freeze vs fine-tune" false dichotomy
- Pretrained-checkpoint-file-not-found failure mode
Prerequisites for the cuBLAS-native design (all satisfied):
- gpu_attention.rs exists
- GpuLinear trainable layer exists
- cuda_autograd over cuBLAS exists
- MBP-10 data already in the DQN data pipeline
What was "BLOCKED on prerequisites" in the Task 6C audit referred to
the OLD pretrained-ONNX design. The cuBLAS-native design has all
prerequisites satisfied — Task 6C can proceed under the new scope.
Pearl captured in memory: pearl_tlob_no_pretraining.md. Generalises to
any future attention/state-space/Neural ODE module: port to cuBLAS,
random init, let the DQN teach it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revises Plan 2 to match the post-Plan-1 reality:
1. AdaptiveController → AdaptiveMonitor throughout (read-only observers
per spec §4.C.6 2026-04-24 revision; GPU kernels compute, CPU reads).
Applies to Task 3 (per-branch gamma) and Task 5 (liquid_mod audit).
2. Task 2 (D.1 Mamba2 backward) scope narrowed from "implement" to
"validate existing". Plan 1 A.5 audit confirmed
mamba2_scan_projected_bwd kernel + mamba2_backward host call are
already fully wired. Task 2 now adds a finite-difference grad-check
smoke + non-zero grad-propagation smoke; escalates if either fails.
3. Task 3 (D.2 per-branch gamma) rewritten for GPU-drives compliance:
- Replace scalar GAMMA_EFF_INDEX=43 with 4 per-branch slots at
43-46 (DIR/MAG/ORD/URG).
- New per_branch_gamma_update_kernel.cu reads v-range + health,
writes 4 slots. Deletes the Plan 1 gamma_update_kernel.cu.
- 4 read-only monitors (or one consolidated PerBranchGammaMonitor).
- ISV slots 44-48 shift downstream; fingerprint re-tails at 50-51;
ISV_TOTAL_DIM grows 49 → 52.
- c51_loss_kernel and iql_value_kernel read per-branch γ from ISV.
- No CPU-side γ computation anywhere.
4. Header architecture block updated: new ISV slot count target,
GPU-drives principle explicit, current Plan-1 layout table inline
for reference.
5. Pre-plan verification updated: expects AdaptiveMonitor trait (not
AdaptiveController); checks for 6 Plan-1 monitor files.
6. Removed "schema version bump 1 → 2" language — layout fingerprint
auto-recomputes from seed bytes; no integer version space exists.
Task 4 (D.5 soft fold transitions), Task 5 content (D.7 liquid audit),
Task 6 (D.3+D.6+D.8 coordinated state-layout migration), Task 7
(validation run) structure preserved. Internal slot numbers in those
tasks will reconcile to the new layout when they land (no pre-emptive
edits since those indices depend on whether Task 1 quantile slots or
Task 3 per-branch gamma slots land first — re-compute at exec time).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User raised that the CPU-controller pattern (where CPU computes adaptive
values from ISV and writes back to ISV) violates the architectural
principle: GPU kernels should compute adaptive decisions; CPU-side code
is pure observation.
Replaces the AdaptiveController trait with read-only AdaptiveMonitor:
- No update() — CPU doesn't compute adaptive values.
- No write_output() — CPU doesn't write adaptive values to ISV.
- read() returns the GPU-computed value from ISV.
- diagnose() emits HEALTH_DIAG snapshot.
- observe() + fire_rate() track how often the GPU-computed value changes.
Reclassifies the 9 adaptive mechanisms:
- 6 reactive get GPU kernel + CPU monitor: atoms, gamma, kelly_cap, tau
(Polyak EMA), epsilon, grad_balancer (last is already GPU-driven).
- 3 static get ISV constructor-write, no monitor: cql_alpha,
conviction_floor, plan_threshold.
New ISV slots for GPU-written adaptive outputs (EPSILON_EFF, TAU_EFF,
GAMMA_EFF, KELLY_CAP_EFF) and CPU-born inputs (EPOCH_IDX, TOTAL_EPOCHS),
plus slots for the 3 static configs.
Rationale:
- Unified adaptive machinery, no CPU-side special cases.
- Per-sample granularity available (Expected SARSA τ in c51_loss_kernel
is already the exemplar — reads ISV q_gap + health per sample).
- Zero CPU→GPU config transfer in any path.
- ISV is single source of truth for every adaptive value.
Plan 1 Tasks 8-17 restructured:
- Task 8 creates AdaptiveMonitor trait (not AdaptiveController).
- Tasks 9, 10, 11, 13, 14, 17: GPU kernel + monitor pairs.
- Tasks 12, 15, 16: static ISV writes only.
Follow-up commits will revert Batch A (d76849f31) and Batch B (4189da563)
which implemented the old CPU-compute pattern, then re-implement under
this design.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Populated docs/dqn-wire-up-audit.md with every pub module and CUDA
kernel in the DQN path. Each entry classified Wired / Partial / Orphan /
Ghost / OUT-of-DQN-scope with the action plan linking to the plan+task
that resolves any non-Wired status.
No Orphan left unclassified. Orphans fall into three buckets:
1. Scheduled for wiring by a later Plan (gpu_statistics → Plan 2 D.2;
tlob_loader → Plan 2 D.8).
2. OUT-of-DQN-scope because supervised consumers exist (PPO kernels,
xLSTM, KAN trainable adapter, flash_attention, benchmarks).
3. Genuinely unused — escalated to user review in the task output,
not deleted autonomously (streaming_dbn_loader, unified_data_loader,
training/orchestrator, training_pipeline, inference_validator,
model_loader_integration, paper_trading/mod.rs,
portfolio_transformer, regime_detection/mod.rs).
Summary: 109 total modules/kernels, 74 wired, 7 partial, 11 orphan,
0 ghost, 17 OUT-of-DQN-scope.
Plan 1 Task 6. Spec §4.A.5, Invariant 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User raised a backward-compat concern: integer "schema version"
semantically implies a family of coexisting versions with upgrade
paths between them, inviting the forbidden pattern
(feedback_no_legacy_aliases.md, feedback_no_partial_refactor.md).
Replace with a compile-time structural fingerprint:
- ISV[0..2) stores a u64 FNV-1a hash of the slot layout (split across
two f32 lanes preserving raw bits).
- The hash is computed by a const fn over the slot list; any slot
change automatically updates the fingerprint. No human decides
"what version is this now".
- Checkpoint load is fail-fast only. Error message does NOT mention
migration as an option.
- Pre-commit hook rejects any `fn migrate_isv|upgrade_isv` to make
the no-migration rule structurally enforced (landed as part of
Plan 1 Task 5 hook extension).
- StateResetRegistry entry renamed ISV_SCHEMA_VERSION →
ISV_LAYOUT_FINGERPRINT (Task 2's landed code touched in Task 5's
commit per no-partial-refactor).
Updated:
- spec §4.A.2 (fingerprint design + rationale)
- spec §5 landing-order note (fingerprint auto-updates on layout change)
- Plan 1 architecture line
- Plan 1 Task 2 StateResetRegistry test + entry naming
- Plan 1 Task 5 — full rewrite of implementation steps
- Plan 1 exit criteria #6
- Plan 2 pre-plan verification (grep fingerprint constants, not version==0)
- Plan 2 Task 6D.1 (update fingerprint seed, not bump version)
- Plan 2 Task 6D exit criteria #7
- Plan 3 pre-plan verification
- docs/isv-slots.md ISV[0..2) row
No code changed. Plan 1 Task 5 is not yet implemented — this commit
realigns the spec + plans so the implementer subagent works from the
corrected design.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
