Plan 3 Task 4.
ISV tail-append:
- [75] READINESS_EMA_INDEX — batch-mean readiness EMA (GPU-written)
- [49] PLAN_THRESHOLD_INDEX — producer upgraded from static constructor
write to GPU kernel (same consumer path unchanged)
- Fingerprint shifted [73,74] → [76,77]; ISV_TOTAL_DIM 75 → 78
Producer (plan_threshold_update_kernel.cu):
- Single-block reduction of readiness_per_sample [N*L]
- Adaptive α = α_base × (1 + 0.5 × |clamp(sharpe, -2, 2)|); α_base=0.05
- Derived: threshold = max(0.1, 0.5 × readiness_ema) → ISV[49]
Consumer sites unchanged (experience_kernels.cu 4 sites + backtest_plan_kernel.cu).
The upgrade is producer-only; consumers keep reading ISV[49] as before but now
receive an adaptive value tracking the policy's actual readiness distribution
rather than a hardcoded 0.5 midpoint.
PlanThresholdMonitor (read-only observer) surfaces plan_threshold.eff +
plan_threshold.readiness_ema for HEALTH_DIAG / controller_activity smoke.
StateResetRegistry: PLAN_THRESHOLD flipped SchemaContract→FoldReset;
READINESS_EMA registered as FoldReset. Both fold-reset arms restore the
cold-start values (0.5 / 1.0) before the first kernel fire on the new fold.
No breaking changes to consumer API.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan 3 Task 6b.
Portfolio-state tail-append:
- PS_REGIME_SHIFT_BAR = 40 (hold_time of first detected regime shift, 0 if none)
- PS_STRIDE 40 → 41
- All 6 hardcoded-stride sites migrated in lockstep
Detector (experience_kernels.cu):
- Adaptive threshold = clamp(0.25 × |clamp(sharpe, -2, 2)|, 0.05, 0.5)
- Fires first bar where |regime_now - PS_PLAN_ENTRY_REGIME| > threshold
- First-shift-only (short-circuits on non-zero PS_REGIME_SHIFT_BAR)
- Uses new ISV_SHARPE_EMA_IDX = 22 macro in state_layout.cuh
Consumer (segment_complete block):
- bars_late_frac = clamp(bars_late / hold_time, 0, 1)
- penalty = shaping × conviction × bars_late_frac × |reward|
- All multiplicands except |reward| in [0,1]; max penalty = |reward|
- reward -= penalty; rc[5] -= penalty (cancels with B.2/C.4/D.4a at
other (i,t) slots; ISV[68] REWARD_BONUS_EMA shows net)
- Consumer resets PS_REGIME_SHIFT_BAR after use
**Iteration history.** First pass multiplied by ISV[Q_DIR_ABS_REF] (~5–50,
an absolute Q-magnitude) AND |reward| — produced penalties 5–50× the
reward, destabilising training (smoke: Return swings ±300–900%, Sharpe
oscillating wildly). Root cause: Q_DIR_ABS_REF is an absolute
magnitude, not a [0,1] coefficient; B.1 uses it as a DENOMINATOR to
normalize q_range, not as a multiplier on an already-unbounded signal.
Fix: drop q_scale, keep |reward| as the only unbounded factor. Smoke
now passes cleanly with fold-2 best Sharpe 117.92 (up from T6a's 100.10).
No new ISV slot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan 3 Task 6a.
Portfolio-state tail-append (shared-contract migration, all in same commit):
- PS_INTRA_TRADE_MIN_PNL = 39 (symmetric to PS_INTRA_TRADE_MAX_PNL = 21)
- PS_STRIDE 39 -> 40
- 6 PORTFOLIO_STRIDE hardcoded copies bumped in lockstep
Producer (experience_kernels.cu):
- MIN_PNL tracked per bar (fminf against pnl_pct) in the same block
as MAX_PNL update
- Reset to 0 at all 5 MAX_PNL reset sites (entry, reverse, 2x fold boundary)
Consumer (experience_kernels.cu segment_complete):
- Fires only on reward > 0 AND drawdown_depth > 1e-6
- persist_bonus = shaping x conviction x |min_pnl| x tanh(reward/|min_pnl|)
- reward += persist_bonus; rc[5] += persist_bonus (accumulates with
B.2 entry bonus + C.4 timing bonus — different (i,t) slots per trade)
Self-scaling via tanh: no tuned coefficients. Saturates when recovery
is large relative to drawdown; near-zero when recovery is trivial.
Attribution lands in ISV[68] REWARD_BONUS_EMA via the Task 1 kernel.
No new ISV slot.
Smoke: multi_fold_convergence PASS (fold-2 best Sharpe 100.10, threshold >=80).
HEALTH_DIAG reward_split bonus=17.21 (post-Task-5 rises with new D.4a credit
firing on profitable drawdown recoveries).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan 3 Task 5.
Portfolio-state tail-append (shared-contract migration, all in same commit):
- PS_PEAK_PNL_BAR = 38 (hold_time snapshotted when MAX_PNL updates)
- PS_STRIDE 38 -> 39 in state_layout.cuh and ml-core/state_layout.rs
- PORTFOLIO_STRIDE 38 -> 39 in trade_stats_kernel.cu (hardcoded copy)
- PORTFOLIO_STRIDE 38 -> 39 in gpu_experience_collector.rs allocator
- ps_stride 38 -> 39 in gpu_dqn_trainer.rs launch_kelly_cap_update
Producer (experience_kernels.cu):
- Peak bar snapshotted alongside every MAX_PNL update (uses local
hold_time, not ps[PS_HOLD_TIME], because the portfolio-state commit
block runs later in the kernel).
- Peak bar reset to 0 at every MAX_PNL reset site: plan-entry (1856),
entering_trade (2014), reversing_trade (2019), fold hard-reset (2736),
trade-complete soft-reset (2751).
Consumer (experience_kernels.cu segment_complete block):
- bars_early = max(0, segment_hold_time - PS_PEAK_PNL_BAR)
- timing_bonus = shaping_scale x (bars_early / segment_hold_time)
x |final_pnl| x conviction_core
- reward += timing_bonus; rc[5] += timing_bonus
(accumulates with Task 3 B.2 entry bonus — different (i,t) slots).
No new ISV slot — rc[5] bonus semantics unchanged; B.2 and C.4 share it
via += accumulate semantics (defensively idempotent, but the two sites
fire at distinct (i,t) by construction: entry vs exit).
Self-scaling: shaping_scale x conviction_core x |pnl| keeps the bonus
proportional to trade magnitude, no tuned coefficients.
Smoke multi_fold_convergence (RTX 3050 Ti): all 3 folds complete,
fold-2 best Sharpe 84.44 at epoch 1 (expected ~85 range).
cargo check --workspace clean at 11 warnings baseline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Current Flat opp-cost was conviction-driven but |Q|-scale was implicit.
When training drifts Q magnitudes into ±50, opp-cost becomes invisible
relative to action Q-values → Flat wins argmax. Fix: multiply by
isv_signals_ptr[21] (Q_DIR_ABS_REF_INDEX, EMA of max(|Q_mean|) across
direction bins, populated by update_eval_v_range / q_stats_kernel).
Self-scaling: opp-cost tracks |Q| proportionally throughout training.
No tuned multiplier; relies on existing ISV slot. Floor 1e-3 for cold-
start protection before EMA is warm. rc[4] is now written in the Flat
branch so reward_component_ema kernel picks it up into ISV[67].
No parallel paths — the old formula IS modified in place.
Plan 3 Task 2. Spec §4.B.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 7th plan_isv dimension: PLAN_ISV_REMAINING_FRACTION = max(0, min(1,
(plan_target_bars - hold_time) / plan_target_bars)) when plan active, else
0. Exposes temporal pressure to the policy.
State vector grows 104 → 112 (105 + 7 padding for 8-alignment).
SL_PORTFOLIO_PLAN_DIM 6 → 7. SL_PADDING_DIM 0 → 7.
Both training (experience_env_step) and val (backtest_plan_state_isv) write
the new slot identically — preserves train/val state-distribution parity.
All hardcoded stride-6 references in backtest_plan_kernel.cu replaced with
SL_PORTFOLIO_PLAN_DIM. plan_isv_buf allocation updated to n_windows * 7.
Stale offset comments ([86..92)) corrected to [98..105) across all files.
No behavioural change to existing dimensions. New signal is additive.
Plan 2 Task 6A. Spec §4.D.6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The liquid_tau_rk4_step kernel modelled f(x) = (1/tau)*(1-x), which has
fixed point x=1.0 for any tau. liquid_mod_buf was initialised to [1.0;4]
and the dynamics could never move it away from that fixed point since every
kernel call reduces to f(1.0)=0 (no perturbation path exists). The
velocity_mod multiplier in c51_grad_kernel was therefore always 1.0 — a
mathematical identity with zero measurable effect on spread_scale.
Additionally, no ISV slot existed for liquid_mod (violates §4.C.6
GPU-drives spec), and the associated LiquidTrainableAdapter supervised
path was never wired into the DQN backward pass.
Decision C (delete): remove liquid_tau_rk4_step kernel (experience_kernels.cu),
liquid_mod param + velocity_mod line (c51_grad_kernel.cu), liquid_mod_buf
allocation, liquid_tau_rk4_kernel field, update_liquid_tau() method and its
call site in update_eval_v_range() (gpu_dqn_trainer.rs). per_branch_q_gap_ema_buf
is retained — it is used independently for trajectory backtracking state
snapshot/restore. Supervised LiquidTrainableAdapter unchanged.
cargo check -p ml: 8 warnings (baseline), 0 errors.
Audit docs updated: dqn-wire-up-audit.md + ml-supervised-to-dqn-concept-audit.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SoftReset registry category (isv_grad_balance_targets, isv_grad_scale_limit)
implemented via bootstrap-write + existing kernel's EMA. CPU writes bootstrap
values (1.0 for targets, 2.0 for limit) at fold boundary; grad_balance_isv_update
kernel's adaptive-rate EMA (alpha in [0.01, 0.30]) converges from bootstrap
toward observed values over subsequent epochs (implicit decay_bars via EMA time
constant).
Option A chosen (minimal): the existing grad_balance_isv_update kernel already
uses adaptive-rate EMA — not a hard-write — so bootstrap-at-fold-boundary is
sufficient. The EMA time constant ~1/alpha provides the decay, satisfying the
decay_bars=500 intent without a new ISV slot (Option B rejected as over-
engineered: new ISV slot + kernel change for no material behavioral improvement).
Changes:
- training_loop.rs::reset_named_state: add dispatch arms for both SoftReset
entries, writing bootstrap constants (CPU-born input, not adaptive output;
complies with spec §4.C.6 GPU-drives-CPU-reads)
- trainer/mod.rs: fold-boundary reset loop now calls both fold_reset_entries()
and soft_reset_entries(); stale "handled separately (Plan 2)" comment removed
- smoke_tests/soft_reset.rs: 4 CPU-only tests verify registry classification,
entry count, mutual exclusivity from fold_reset, and bootstrap constant invariants
- docs/dqn-wire-up-audit.md: D.5 SoftReset dispatch row added
No CPU-side adaptive computation. No new ISV slot. No behavioral change to
training beyond writing known-good bootstrap values at fold boundaries.
Plan 2 Task 4. Spec §4.D.5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plan 1 A.5 audit found mamba2_scan_projected_bwd kernel + host call at
gpu_dqn_trainer.rs::mamba2_backward are ALREADY fully wired in the
adam_grad CUDA-graph child. Plan 2 Task 2 narrows from "implement" to
"validate".
Two smoke tests confirm correctness:
- mamba2_backward_gradients_propagate: grad Frobenius norm 0.25 after 3
epochs (>> 1e-8 threshold), ruling out silent no-op like compute_iqr
had pre-Task-A.6.
- mamba2_backward_grad_check: kernel-level reference check (B=2 K=2
SH2=4 STATE_D=4); max rel_err d_gate=2.6e-7, d_x_out=2.6e-7,
d_context=6.7e-8 — all well within 15% threshold (near machine
epsilon, confirming bit-identical host/GPU results).
No production code change — test-only accessors exposed via #[cfg(test)]
impl blocks on GpuDqnTrainer and DQNTrainer. Audit doc updated.
Plan 2 Task 2. Spec §4.D.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Local smoke-test run after Plan 1 C.6 completion surfaced three issues:
1. StateResetRegistry missing dispatch arms for the 8 ISV slots
pre-allocated in ac9bcab94 (isv_epoch_idx, isv_epsilon_eff, isv_tau_eff,
isv_gamma_eff, isv_kelly_cap_eff, + 3 SchemaContract which already no-op).
Fold boundary would error "unknown name 'isv_epoch_idx'". Added dispatch
arms in training_loop.rs::reset_named_state — reset each to 0.0; GPU
kernels repopulate on next epoch.
2. controller_activity smoke test's single 50% threshold was designed for
reactive CPU-compute controllers. Under GPU-drives-CPU-reads, tau is a
Polyak-EMA cosine schedule that fires every epoch by design (95%),
gamma is health-coupled monotonic (may fire every epoch as health
drifts). Split threshold per-controller: reactive (anti_lr, grad_clip,
cql_alpha, cost_anneal) = 0.50; schedule-based (tau, gamma) = 1.00.
3. examples/train_baseline_rl was broken since the f64→f32 ABI refactor
(d64adc14f) — hp_f64 returning f64 assigned to f32 fields. Added
hp_f32 helper that narrows JSON-born f64→f32 at ingest boundary. Use
hp_f32 for f32 fields, hp_f64 for f64 fields (learning_rate,
entropy_coefficient, weight_decay). No more "as f32" casts at call
sites. Also fixed replay_buffer_vram_fraction + bars_per_day f64→f32.
Local smoke tests now pass:
- controller_activity: ok (1 passed, 29.5s)
- multi_fold_convergence: ok (1 passed, 3 folds x 20 epochs, 534.9s)
- 24 new monitor + registry unit tests: all passing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test observed `mon.observe(0.0)` first but the monitor's initial
last_value is also 0.0, so the first observation didn't fire. Test
expected 2/3 but got 1/3.
Changed observe sequence to (1.0, 1.0, 1.5) mirroring grad_balancer's
correct pattern: first differs from initial 0.0 → fires, second matches
→ no fire, third differs → fires = 2/3.
No production code change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
atoms_update_kernel.cu: 4-block kernel (grid=(4,1,1), block=(256,1,1)) replaces
the CPU loop that launched adaptive_atom_positions 4× per branch. Each block reads
ISV v-range slots [23..31) and spacing_raw params[52..56); computes softmax +
cumsum; writes atom_positions_buf[branch * num_atoms]. Same formula as the
existing adaptive_atom_positions kernel in experience_kernels.cu.
Deleted recompute_atom_positions and warm_start_atom_positions from GpuDqnTrainer
and their fused_training.rs delegates. step_atom_positions now calls
launch_atoms_update. training_loop.rs: recompute_atom_positions → launch_atoms_update;
warm_start_atom_positions + compute_reward_quantiles block removed.
AtomsMonitor reads ISV[V_CENTER_DIR_INDEX=23] as representative scalar;
diagnose() exposes all 8 v-range slots [23..31). state_reset_registry.rs
"follow-up task" descriptions updated for all 4 adaptive slots.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kelly_cap_update_kernel.cu: single-thread cold-path kernel aggregates Kelly
win/loss stats across all n_envs from portfolio_states[n_envs, PS_STRIDE].
Computes mean half-Kelly × health-coupled safety multiplier (0.5+0.5*health)
and writes ISV[KELLY_CAP_EFF_INDEX=44]. Matching Bayesian priors from
trade_physics.cuh (prior_wins=2, prior_losses=2) prevent cold-start collapse.
GpuExperienceCollector::portfolio_states_dev_ptr() added — exposes device
pointer for the portfolio_states buffer without a GPU→CPU roundtrip.
GpuDqnTrainer::launch_kelly_cap_update takes ps_dev_ptr + n_envs.
FusedTrainingCtx delegates to trainer. training_loop.rs launches at epoch
boundary after gamma_update (requires both fused_ctx and gpu_experience_collector).
KellyCapMonitor reads ISV[44] for HEALTH_DIAG via AdaptiveMonitor trait.
state_reset_registry.rs description updated; audit docs updated (Invariant 7).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tau_update kernel computes Polyak-EMA tau from ISV[EPOCH_IDX=39,
TOTAL_EPOCHS=40, LEARNING_HEALTH=12] with cosine schedule + health-coupled
floor (rises to 0.01 at full collapse). Single-thread cold-path kernel
writes ISV[TAU_EFF_INDEX=42].
TauMonitor is a read-only observer exposing tau_eff, epoch_idx, total_ep,
health, progress, fire_rate in DiagSnapshot.
Wires EPOCH_IDX_INDEX CPU-born input write at epoch start via existing
write_isv_signal_at accessor (pinned-memory zero-copy). Also launches
tau_update kernel at epoch boundary before any tau consumer.
Adds read_isv_signal_at() to GpuDqnTrainer (symmetric counterpart to
write_isv_signal_at).
Deleted: compute_cosine_annealed_tau free function (fused_training.rs),
apply_health_coupled_tau_floor method (GpuDqnTrainer), last_tau_eff
cached field + last_tau_eff() delegate. 4 consumer sites in
fused_training.rs now read ISV[TAU_EFF_INDEX] directly.
Tests: 3 monitor unit tests pass. cargo check -p ml at 8-warning baseline.
Plan 1 Task 13. Spec §4.C.6 (2026-04-24 revision).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the static-configuration branch of Plan 1 C.6: cql_alpha,
conviction_floor (no-op if IQL_BRANCH_SCALE_FLOOR already serves),
plan_threshold written to dedicated ISV slots at constructor. Consumer
kernels (CQL, backtest_plan, experience) read from ISV instead of
config fields / hardcoded literals.
Also pre-allocates ISV slots for the 6 upcoming GPU-kernel tasks
(atoms, gamma, kelly_cap, tau, epsilon outputs + CPU-born epoch inputs).
Those slots start at 0; GPU kernels in follow-up commits fill them.
New ISV slots:
- EPOCH_IDX_INDEX=39, TOTAL_EPOCHS_INDEX=40 (CPU-born inputs)
- EPSILON_EFF_INDEX=41, TAU_EFF_INDEX=42, GAMMA_EFF_INDEX=43,
KELLY_CAP_EFF_INDEX=44 (GPU-written in follow-up tasks)
- CQL_ALPHA_INDEX=45, PLAN_THRESHOLD_INDEX=46 (static config; this commit)
- Task 15 confirmed no-op: IQL_BRANCH_SCALE_FLOOR_INDEX=36 already
serves conviction-floor role (constructor + ISV read in iql kernel).
Layout fingerprint auto-updated via seed-byte edits; fingerprint
re-tail at [47..49). ISV_TOTAL_DIM 39 -> 49.
GpuDqnTrainConfig gains total_epochs field; fused_training.rs passes
hyperparams.epochs at construction; default 0 for smoke tests.
write_isv_signal_at bound extended from ISV_DIM(23) to ISV_TOTAL_DIM(49)
so tail slots are writable by CPU.
cql_alpha consumer: compute_cql_logit_gradients reads base from
ISV[CQL_ALPHA_INDEX] instead of config.cql_alpha; adaptive formula
(base x health x (1-regime_stability)) unchanged.
plan_threshold consumers: experience_kernels.cu (experience_state_gather,
experience_action_select, experience_env_step) and backtest_plan_kernel.cu
(backtest_plan_state_isv) read plan_thr from ISV[ISV_PLAN_THRESHOLD_IDX=46]
via isv_signals pointer already present in both kernels; null-guard defaults
to 0.5f for smoke-scale runs without ISV warmup.
ISV_PLAN_THRESHOLD_IDX=46 defined in state_layout.cuh (included by both
plan kernel files); value must match PLAN_THRESHOLD_INDEX in gpu_dqn_trainer.rs.
StateResetRegistry entries added for all new slots: SchemaContract for
TOTAL_EPOCHS/CQL_ALPHA/PLAN_THRESHOLD; FoldReset for EPOCH_IDX and
GPU-written per-fold slots.
No behavioural change: all threshold/base values remain at their prior
defaults; consumers now read adaptive values from ISV instead of baked-in
config or hardcoded literals.
Plan 1 Tasks 12, 15, 16 + pre-allocation for 9, 10, 11, 13, 14.
Spec §4.C.6 (2026-04-24 GPU-drives-CPU-reads revision).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reverts commits d76849f31 (Batch A: atoms, gamma, kelly_cap, cql_alpha)
and 4189da563 (Batch B: tau, epsilon, conviction_floor, plan_threshold).
The reverted commits implemented 8 controllers under the old
AdaptiveController trait which had CPU-side update() that computed
adaptive values and write_output() that pushed them to ISV. This
violates the architectural principle codified in the §4.C.6 revision
(commit cf36091ee): GPU kernels compute all adaptive decisions; CPU is
pure observation.
The 8 migrations will be re-implemented under the new AdaptiveMonitor
pattern:
- 6 reactive mechanisms (atoms, gamma, kelly_cap, tau, epsilon,
grad_balancer) get new GPU kernels + read-only CPU monitors.
- 3 static mechanisms (cql_alpha, conviction_floor, plan_threshold)
get ISV constructor-writes (no kernel, no monitor).
After this revert:
- AdaptiveController trait is back on main (from Task 8's 419c24b4f).
It will be replaced with AdaptiveMonitor in the next commit per the
revised Plan 1 Task 8.
- StateResetRegistry (from Task 2's b688827d6) stays intact.
- Tasks 1-7 completed work unchanged.
Tests: cargo check -p ml at 8-warning baseline after revert.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unified protocol for every adaptive controller in the DQN trainer.
FireRateStats for controller_activity smoke. DiagSnapshot for
standardised HEALTH_DIAG emission. IsvBus abstraction over pinned
device-mapped memory (&mut [f32] wrapper).
No concrete controller migration yet — Tasks 9-17 migrate existing
controllers (atoms → gamma → Kelly → cql_alpha → tau → epsilon →
conviction_floor → plan_threshold → balancer) one per sub-commit.
Old scaffolding for each controller is removed in the SAME commit
that migrates its last consumer to the new protocol, per
feedback_no_partial_refactor.md.
Tests: 2 unit tests on FireRateStats and DiagSnapshot.
Plan 1 Task 8. Spec §4.C.6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Enumerated every DtoH/HtoD/memcpy call in the DQN hot path
(run_full_step → child graphs). Classified 55 call sites:
31 OK-pinned, 20 COLD-PATH, 4 MIGRATED.
Fix 1 (gpu_dqn_trainer.rs): removed dead cuMemcpyDtoHAsync_v2 in
run_causal_intervention_unconditional — result (causal_mean_scratch)
was never consumed; now stays on device.
Fix 2 (fused_training.rs + training_loop.rs): compute_iqr() was
called inside submit_aux_ops (captured aux_child graph). Its sync
cuMemcpyDtoH_v2 cannot be graph-captured — silently ran only during
capture, then HtoD replayed stale IQR data on every step. Removed
from submit_aux_ops; added refresh_iqn_iqr() called once per epoch
in process_epoch_boundary.
Fix 3 (gpu_iqn_head.rs): tau_buf (CudaSlice<f32>) + tau_host (f32) +
cuMemcpyHtoDAsync_v2 each step replaced by tau_pinned (*mut f32) +
tau_dev_ptr (u64) via cuMemAllocHost_v2(DEVICEMAP) +
cuMemHostGetDevicePointer_v2. CPU writes *tau_pinned = tau; EMA
kernel reads via tau_dev_ptr — zero PCIe overhead. Drop updated.
Audit table populated in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Populated docs/dqn-wire-up-audit.md with every pub module and CUDA
kernel in the DQN path. Each entry classified Wired / Partial / Orphan /
Ghost / OUT-of-DQN-scope with the action plan linking to the plan+task
that resolves any non-Wired status.
No Orphan left unclassified. Orphans fall into three buckets:
1. Scheduled for wiring by a later Plan (gpu_statistics → Plan 2 D.2;
tlob_loader → Plan 2 D.8).
2. OUT-of-DQN-scope because supervised consumers exist (PPO kernels,
xLSTM, KAN trainable adapter, flash_attention, benchmarks).
3. Genuinely unused — escalated to user review in the task output,
not deleted autonomously (streaming_dbn_loader, unified_data_loader,
training/orchestrator, training_pipeline, inference_validator,
model_loader_integration, paper_trading/mod.rs,
portfolio_transformer, regime_detection/mod.rs).
Summary: 109 total modules/kernels, 74 wired, 7 partial, 11 orphan,
0 ghost, 17 OUT-of-DQN-scope.
Plan 1 Task 6. Spec §4.A.5, Invariant 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements spec §4.A.2 structural layout fingerprint with tail placement
rather than head placement (spec alternative: §4.A.2 Step 5.3 alt).
Head placement (ISV[0..2)) was rejected because isv_signals[0] and [1]
are actively written by the isv_signal_update kernel (Q-drift EMA and
gradient-norm EMA). Shifting those would require updating every literal
reference in experience_kernels.cu — a larger change than warranted for
pure contract enforcement. Tail placement leaves all existing indices
intact, touches zero kernel .cu files, and fulfils the same design contract.
Key changes:
- ISV_LAYOUT_FINGERPRINT_LO_INDEX = 37, HI_INDEX = 38 (u64 across 2×f32).
- LAYOUT_FINGERPRINT_CURRENT: u64 = FNV-1a of slot-list seed bytes.
Value: 0x85d4d76b578a7c17. Any slot change updates seed bytes,
which updates the hash automatically.
- Constructor writes fingerprint after zero-init; calls
check_layout_fingerprint() to self-verify before returning.
- check_layout_fingerprint(): reads pinned slots [37..39), recomposes u64,
fails-fast on mismatch with "retrain required" message.
- Error message does NOT mention migration as an option.
- Pre-commit hook rejects `fn migrate_isv|upgrade_isv` names — makes the
no-migration rule structurally enforced (check_no_isv_migrations).
- ISV_TOTAL_DIM: 37 → 39.
- Zero existing index shifts (no kernel literal sites affected).
- StateResetRegistry entry renamed ISV_SCHEMA_VERSION → ISV_LAYOUT_FINGERPRINT.
- ResetCategory::SchemaContract docstring updated to remove "migration" framing.
- docs/isv-slots.md: updated table + design note for tail placement.
Tests: state_reset_registry 3 unit tests pass with renamed entry.
cargo check -p ml clean (pre-existing warnings only).
Plan 1 Task 5. Spec §4.A.2 (tail-placement alternative).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Define MAG_QUARTER=0, MAG_HALF=1, MAG_FULL=2, NUM_MAGNITUDES=3 in
state_layout.cuh. Migrate the one unambiguous semantic comparison in
trade_physics.cuh (compute_target_position_4branch magnitude scaling).
Arithmetic operations on mag_idx (mag_idx±1, 2-mag_idx, mag_idx<2) are
runtime values, not semantic comparisons, and are not migrated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Define DIR_SHORT=0, DIR_HOLD=1, DIR_LONG=2, DIR_FLAT=3, NUM_DIRECTIONS=4 in
state_layout.cuh. Migrate all literal dir_idx/raw_dir comparisons to named
constants across experience_kernels.cu (15 sites), trade_physics.cuh (3 sites),
and backtest_metrics_kernel.cu (2 sites). Raw integer comparisons (== 0/1/2/3)
eliminated from all direction-branch consumers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add BRANCH_DIR/BRANCH_MAG/BRANCH_ORD/BRANCH_URG/NUM_BRANCHES to
state_layout.cuh. Migrate the 4 clear literal branch-index accesses
in branch_grad_balance_kernel.cu (branch_norms_dev[0..3] in
grad_balance_isv_update).
Other branch-keyed arrays (branch_starts, branch_lens, branch_norms,
branch_scales) use the runtime `branch = blockIdx.y` variable or loop
counter — no raw literal index, so no migration needed. Rust call sites
use struct fields (branch_0_size etc.) or loop vars — not literals.
No behavioural change; pure refactor.
Plan 1 Task 4D. Spec §3 Invariant 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add PLAN_PARAM_* constants to state_layout.cuh and replace raw
plan_params/pp slot accesses in experience_kernels.cu and
backtest_plan_kernel.cu.
Sites migrated: pp[0..5] in both files (12 sites), plan_params_ptr
indexed accesses at slots 0, 1, 4 in experience_kernels.cu (4 sites),
plan_params[w*6+4] in backtest_plan_kernel.cu (1 site). Loop variable
pp[p] in noisy-plan path left as-is (p is a runtime loop counter, not
a literal slot index).
No behavioural change; pure refactor.
Plan 1 Task 4C. Spec §3 Invariant 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add PLAN_ISV_* constants to state_layout.cuh and replace raw
plan_isv[N] and pisv[N] accesses in experience_kernels.cu and
backtest_plan_kernel.cu. backtest_plan_kernel.cu gains an
#include "state_layout.cuh" so the constants are visible.
6 code sites migrated (plan_isv[0..5] in experience_kernels.cu),
plus 6 pisv[N] local-array accesses in backtest_plan_kernel.cu
(same semantic slots).
No behavioural change; pure refactor.
Plan 1 Task 4B. Spec §3 Invariant 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace raw ps[N] access across all CUDA kernels with PS_* named
constants defined in state_layout.cuh. 32 constants total covering
the full 38-slot portfolio state: position/cash/value, DSR stats,
Kelly stats, plan fields, and OFI scratch range.
Notable deviations from pre-written plan mapping: the plan's
PS_VALUE/PS_POSITION/PS_CASH values (0,1,2) had the wrong semantics.
Code wins (feedback_trust_code_not_docs): ps[0]=position, ps[1]=cash,
ps[2]=portfolio_value. All 148 raw ps[digit] accesses in
experience_kernels.cu and trade_stats_kernel.cu migrated. In-comment
range references (e.g. "ps[30..37]") left as documentation.
trade_stats_kernel.cu: migrated base+14 offset to PS_KELLY_WIN_COUNT.
docs/dqn-named-dims.md: PS_* table corrected to match actual code layout.
No behavioural change; pure refactor.
Plan 1 Task 4A. Spec §3 Invariant 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the ad-hoc scattered fold-boundary reset calls with a single
registry-driven iteration. Adding new fold-reset state now requires
adding a registry entry AND a dispatch arm in reset_named_state in the
same commit (Invariant 2 Wire-It-Up).
Step 3.4 correction: plan_state entry removed from the registry —
plan_state_buf exists only in GpuBacktestEvaluator (val path), not in
the training-path fused ctx. No training-side fold reset is applicable.
New behaviour: isv_learning_health, isv_sharpe_ema, isv_q_means are
now properly reset to baseline at each fold boundary (previously unset,
which allowed signals from fold N to bias fold N+1 initialisation).
Tests: 3 registry unit tests pass; cargo check -p ml clean (8 pre-existing
warnings only, no new).
Authority: spec §4.A.1. Plan 1 Task 3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Typed classification of every piece of training state by reset
lifecycle: FoldReset, WindowReset, SoftReset(decay_bars),
TrainingPersist, SchemaContract.
Replaces the previous vibes-driven fold-boundary reset logic with an
explicit registry. Adding new state requires adding an entry in the
same commit (Invariant 2 Wire-It-Up).
Consumer wiring (reset_fold_state, reset_window_state helpers that
iterate the registry) follows in Task 3 of Plan 1.
Tests: 3 unit tests covering category lookup, fold-reset filtering,
unknown-name handling.
Authority: docs/superpowers/specs/2026-04-24-dqn-v2-unified-design.md §4.A.1
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eliminates the f64→f32 cudarc ABI trap (feedback_cudarc_f64_f32_abi.md,
task #82) at the type level: hyperparameters consumed by CUDA kernels
now live as f32 in Rust, cast once at the TOML/PSO ingest boundary
instead of at every kernel call site.
Structs changed:
- DQNHyperparameters (crates/ml/src/trainers/dqn/config.rs) —
~85 scalar fields migrated from f64 → f32. Covers all
kernel-facing scalars: reward weights (w_pnl/w_dd/w_idle,
dd_threshold, cea_weight, micro_reward_*, price_confirm_weight,
book_aggression_weight, hold_quality_weight), exploration
(epsilon_* and the 4 branch mults, noisy_sigma_*, count_bonus,
noise_sigma, q_gap_threshold), distributional RL (v_min, v_max,
reward_scale, iqn_lambda, qr_kappa, spectral_*,
gradient_collapse_multiplier), fill simulation (5 fill_*
fields), risk/Kelly (kelly_fractional, kelly_max_fraction,
max_leverage, max_position_absolute, minimum_profit_factor),
ensemble/curiosity (curiosity_weight,
curiosity_q_penalty_lambda, ensemble_*, beta_*, variance_cap),
anti-LR (anti_lr_*, adversarial_dd_threshold,
beta_penalty_strength), walk-forward (wf_*),
experience (avg_spread, transaction_cost_multiplier,
holding_cost_rate, churn_penalty_scale, contract_multiplier,
margin_pct, tick_size, bars_per_day, cash_reserve_percent),
misc kernel scalars (gamma, tau, huber_delta, q_clip_*,
shrink_perturb_*, regime_replay_decay, per_alpha,
per_beta_start, dt_target_return, etc.). Also
`noisy_epsilon_floor: Option<f32>` and
`count_bonus_coefficient: Option<f32>`.
- `computed_v_min` / `computed_v_max` now return f32.
- `compute_max_position` returns f32 (f64 internally for the
notional division).
Fields preserved as f64 (precision-sensitive, NOT kernel-facing
scalars — per task spec and feedback_cudarc_f64_f32_abi.md):
- `learning_rate` — tested at 1e-10 tolerance; f32 rounds to
2e-12 for a 1e-5 LR.
- `entropy_coefficient` — tested at 1e-9 tolerance.
- `weight_decay` — tiny 1e-5..1e-3 range.
- `adam_epsilon` — 1e-8 default; f32 preserves denorms here but
paired with weight_decay/learning_rate for symmetry.
- `gradient_clip_norm: Option<f64>` — grad norms are f64
accumulators by project convention.
- `min_loss_improvement_pct`, `q_value_floor` — early-stopping
long-horizon stats.
- `cql_alpha` — flows into DQNConfig (ml-dqn) still f64.
- `min_learning_rate`, `lr_min` — paired with learning_rate.
- Family intensity scalars (6 `*_intensity` fields) — f64 PSO
search space; the intensity applies via `as f32` at each call
site in `apply_family_scaling`.
No checkpoint format change: DQNHyperparameters has
`#[derive(Debug, Clone)]` only (not Serialize/Deserialize), so the
TOML-ingest path is the only serde boundary and already casts
explicitly via `hp.field = v as f32;` in
`DqnTrainingProfile::apply_to`. PSO hyperopt bounds stay f64 in
`SearchSpaceSection` and cast at the adapter boundary.
Call-site impact:
- ~30 `as f32` casts removed from hot paths (fused_training
FusedConfig builder, training_loop kernel launches,
constructor DQNConfig builder, action.rs GPU action selector,
trainer/mod.rs WF config). Kernels now receive the hp field
directly via `&hp.x`.
- ~75 `as f32` casts added at the `apply_to` / hyperopt-adapter
ingest boundary — the single conversion point.
- Cross-crate contracts (DQNConfig in ml-dqn, PortfolioTracker
in ml-core, GAECalculator, DropoutScheduler, NoisySigmaScheduler,
KellyOptimizerConfig, RewardConfig) retain their f64 signatures;
ml calls cast at the boundary with `f64::from(hp.x)` so the
contract is explicit and greppable.
Two test-side adjustments:
- `test_kelly_fields_are_public` now asserts `f32` for
kelly_fractional / kelly_max_fraction (these migrated).
- `test_early_stopping_termination` uses `f64::from(...)` to
preserve the f64 threshold computation against the now-f32
`gradient_collapse_multiplier`.
Verified:
- `SQLX_OFFLINE=true CARGO_INCREMENTAL=0 RUSTC_WRAPPER=sccache
cargo check --workspace` — clean, zero new warnings.
- `cargo check --workspace --tests` — clean.
- `cargo test -p ml --lib training_profile::tests` — all 18
migrated tests pass. The one pre-existing failure
(`test_production_profile_applies_all_sections` n_steps
mismatch, 1 vs 5) reproduces on stashed baseline, so it is
unrelated to this change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final commit toward task #94 val plan_isv parity. Wires the plan
machinery inside the chunked val backtest loop so state positions
[86..92) carry real plan signals matching training instead of zero-
fill. This is the structural distribution-shift fix behind the
observed val trade count of 21 / 214,654 bars (0.0098%).
Phase 6 inside `submit_dqn_step_loop_cublas`, per chunk, after env
step advances portfolio:
1. Forward on the LAST-STEP N rows of chunked_states — a targeted
`compute_q_values_to(last_step_states, N, scratch_q)` rewrites
`save_h_s2` to exactly [N, SH2] for the final step. Needed
because the batched `compute_q_values_to` for N*chunk_len rows
leaves save_h_s2 at an arbitrary last-sub-batch slice.
2. `compute_plan_params(plan_params_buf, N)` runs the trade-plan
MLP on that clean save_h_s2, producing [N, 6] plan_params.
3. `backtest_plan_state_isv` — Flat↔Positioned activation /
deactivation on plan_state[N, 7] and writes plan_isv_buf[N, 6]
consumed by the next chunk's first `launch_gather` for state
positions [86..92).
ch_q_values is reused as the Q-scratch for step (1) — its original
chunk Q-values were already consumed by Phase 4 action-select.
Epoch-boundary diagnostic `launch_plan_diag_and_log` after the step
loop: launches `backtest_plan_diag_reduce` (1-block, 256 threads),
DtoH-copies the 8-float summary, syncs, and emits a single
`tracing::info!(target: "val_plan_diag", ...)` line with the six
labelled plan_isv means plus active_frac / n_active.
Scalars passed with exact i32 types (current_step, max_len,
feat_dim, n_windows) per feedback_cudarc_f64_f32_abi. Kernels
pulled via `plan_state_isv_kernel.as_ref().ok_or_else(...)` — they
are loaded in `ensure_action_select_ready` on first eval.
Not captured inside a CUDA Graph: the backtest step loop uses direct
kernel launches (evaluator comment: graphs SIGSEGV on 500K+ launches).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second commit toward task #94 val plan_isv parity. Adds structural
wiring in GpuBacktestEvaluator:
- plan_params_buf [N, 6] : live plan MLP output per step
- plan_state_buf [N, 7] : persistent stored plan across bars
- plan_diag_buf [8] : epoch-end diagnostic reduction output
- plan_state_isv_kernel : lazy-loaded CudaFunction from
backtest_plan_kernel.cubin
- plan_diag_reduce_kernel : diagnostic reduction kernel
Buffers allocated zero-initialised in the constructor; kernels loaded
alongside the action_select and scatter_intent kernels in
`ensure_action_select_ready` (same module-load pattern).
Remaining work (follow-up commit):
- Wire QValueProvider::compute_plan_params into the chunked loop to
populate plan_params_buf from the trainer's save_h_s2 after each
forward pass.
- Launch backtest_plan_state_isv after the env_batch_kernel each
chunk to update plan_state_buf and plan_isv_buf for the next
chunk's state gather.
- Launch backtest_plan_diag_reduce + DtoH of plan_diag_buf at epoch
boundary for HEALTH_DIAG emission.
The chunked batch layout (N*chunk_len samples per forward pass)
requires careful slicing to extract the last-step's plan_params for
the plan_state update — handled in the follow-up commit to avoid
rushing a subtle integration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Val-Flat-collapse fix#5 infrastructure (task #94, 2026-04-24). The
val backtest's `plan_isv_buf` was zero-filled, causing state positions
[86..92) to be OOD relative to training and biasing val argmax toward
Flat/Hold on 99.99% of bars. Research-agent audit confirmed this is a
documented architectural gap, not a bug per se: val's portfolio state
has no plan slots (8 vs training's 30+), so there was no mechanism to
carry plan signals across bars in val.
This commit lays the foundation for closing that gap:
1. `backtest_plan_kernel.cu` (new) — two kernels:
- `backtest_plan_state_isv`: per-window Flat→Positioned plan
activation + plan deactivation on return to Flat + plan_isv[0..5]
computation matching training's formula at
experience_kernels.cu:596-619. Reads live plan_params from the
plan MLP, writes plan_state (persistent across bars) and
plan_isv_out. Extracts raw_close from features buffer for
unrealized P&L ratios.
- `backtest_plan_diag_reduce`: single-block reduction emitting 8
diagnostic floats for plan activity logging (active fraction,
per-slot mean magnitudes, raw active count).
2. `QValueProvider::compute_plan_params` trait method — allows the val
evaluator to run the plan MLP on the trainer's most recent trunk
hidden state (save_h_s2), filling plan_params[N, 6] for use in the
state-update kernel.
3. `FusedTrainingQValueProvider::compute_plan_params` impl — runs
`launch_trade_plan_forward` then DtoD-copies `plan_params_buf` to
the caller's output in the same chunk-loop pattern as
`compute_q_values_to`.
4. build.rs — register `backtest_plan_kernel.cu` for compilation.
Next commit will wire these into `GpuBacktestEvaluator`:
- Add `plan_params_buf[N, 6]`, `plan_state_buf[N, 7]`,
`plan_diag_buf[8]` device allocations.
- Load the two kernels into CudaFunction slots.
- Call sequence per step: `compute_q_values_to` →
`compute_plan_params` → action_select → env_step →
`backtest_plan_state_isv` (reads updated portfolio, writes
plan_state + plan_isv_buf for next step).
- Epoch boundary: `backtest_plan_diag_reduce` + DtoH + HEALTH_DIAG
emission.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Val-Flat-collapse fix#3 revision (task #94, 2026-04-24). Prior
commit 543e3c11b used `-0.5 × holding_cost_rate × vol_proxy` — the
0.5 is a hardcoded tuned constant violating
`feedback_isv_for_adaptive_bounds.md` and
`feedback_adaptive_not_tuned.md`.
Replace with ISV-driven per-sample conviction, already computed by
the action-select kernel and threaded into env_step via
`conviction_ptr → conviction_core`:
reward_flat = -shaping_scale * holding_cost_rate
* conviction_core * vol_proxy_flat
`conviction_core ∈ [0, 1]` = direction-branch Q-range normalised by
`ISV[21]` (q_dir_abs_ref EMA). Self-adapting properties:
- Cold start / ISV[21] uninitialised → fallback 1.0 → full penalty,
encourages early exploration out of the flat equilibrium.
- Low conviction (uncertain direction) → penalty scales toward 0
→ doing nothing is acceptable when there's no signal (matches
real-world: flat cost is only real when there's opportunity).
- High conviction (strong directional edge) → penalty scales up
→ Flat becomes expensive ONLY where the model itself says
there's an edge. Forces the policy to take action exactly
where it has belief, not blindly.
Continuity: conviction is continuous ∈ [0, 1], no step function. The
temporal Mamba2 layers in the trunk feed the Q-values that drive
conviction, so conviction inherits temporal history — the model can
learn "market has been signalling for N bars, time to try" without
an explicit time-since-last-trade feature.
Training-time exploration (Boltzmann sampling, epsilon floor 2%,
NoisyNets σ, count bonus) is unchanged. The Flat-cost shifts the
learned Q-values so that deterministic val argmax picks trade-
actions where the training-time exploration already found edge.
Hold semantics retained:
- Hold while position≠0 (in-trade stance) → positioned-bar
holding-cost branch (unchanged)
- Hold while position=0 AND Flat (no-op outcomes) → this branch
→ conviction-scaled opportunity cost.
The distinction is structural by portfolio state, not by dir label.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Val-Flat-collapse fix#4 (task #94, 2026-04-24). Research-agent audit
of train-bv2n5 identified a positive-feedback loop driving atom
utilisation to 1%:
util < 0.4 → γ -= 0.01 (toward 0.90 floor)
γ × delta_z shrinks → TD targets concentrate on fewer bins
→ more collapse → util lower → γ lower ...
The original controller had the right structure but the wrong
direction on the low-util branch. Flipping the sign (`-=` → `+=`)
breaks the loop: when atoms collapse, raising γ widens the effective
TD-target support `γ × atom_support`, pushing targets across more
bins of the C51 grid and giving the network more distributional
signal to learn from.
Evidence from train-bv2n5 (pre-fix): atom_util stuck at 0.01 for 16
epochs; γ slowly drifted toward 0.90. No mechanism recovered.
Change localised to a single `else if util < 0.4` branch at
`training_loop.rs:660-675`. Same step size (0.005) as the healthy
branch avoids overshoot on the recovery path. Clamps retained
[0.90, 0.95] at this layer; the fused trainer re-clamps to
[0.90, 0.995] after regime adjustment downstream.
Other pending atom-util fixes from the audit (deferred pending
validation):
- Atom-entropy regularisation term in C51 loss (medium risk)
- Absolute v_half floor in update_eval_v_range (low risk)
- TD-target dithering before Bellman projection (low risk)
This single-line fix addresses the dominant mechanism — research
agent labelled it "lowest risk, highest leverage". If util remains
low after this, the others stack additively.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
