Commit Graph

573 Commits

Author SHA1 Message Date
jgrusewski
63183bb6a4 feat(dqn-v2): Plan 4 Task 2c.3c.2 — backward infra additions (launch_dw_only_no_bias + saxpy_inplace)
Plan 4 Task 2c.3c.2. Additive only — no production callers (2c.3c.4
wires them).

Two infrastructure additions for the GRN trunk backward chain:

1. launch_dw_only_no_bias on CublasBackwardSet: variant of launch_dw_only
   that skips the bias-grad kernel call. Linear_residual in h_s1 GRN
   block has no bias, so calling launch_dw_only with db=0u64 would
   segfault the bias-grad kernel.

2. saxpy_inplace on CublasGemmSet: y += alpha * x for element-wise
   gradient accumulation. h_s2 GRN's identity residual needs
   d_h_s1 += d_pre_ln_h_s2 after Linear_a_h_s2's backward overwrites
   d_h_s1 with d_x = d_linear_a @ W_a. Implementation reuses the
   existing dqn_saxpy_f32_kernel (already used by the experience
   collector's IQR/ensemble-variance Q-bonus paths) — no new kernel,
   no cuBLAS legacy-handle stream-binding work, kernel handle loaded
   once at CublasGemmSet::new from DQN_UTILITY_CUBIN.

Both methods sit dead-code until 2c.3c.4's wire-up commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:45:49 +02:00
jgrusewski
8fc41e3398 feat(dqn-v2): Plan 4 Task 2c.3c.1 — GrnBlock backward phase split (additive)
Plan 4 Task 2c.3c.1. Additive only — no production callers (2c.3c.4
wires them). Mirrors 2c.3b's forward phase split for the same
composition reason: cuBLAS Linear_b backward must run BETWEEN GLU
backward and ELU backward.

New methods on GrnBlock:
- backward_raw_phase1: LN_bwd + LN_dgamma_dbeta_p1 + LN_dgamma_dbeta_p2
  + GLU_bwd. Produces d_linear_b_out for caller's Linear_b backward,
  plus d_pre_LN (= d_residual = d_glu_out via implicit residual split).
- backward_raw_phase2: ELU_bwd. Takes d_elu_out from caller's Linear_b
  backward, produces d_linear_a_out for caller's Linear_a backward.

The existing monolithic backward() is left as-is for surface-area
minimization; will be deleted with 2c.5 cleanup if no production
callers emerge.

No CUDA kernel changes. Pure Rust dispatcher split.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:37:51 +02:00
jgrusewski
ad1e5c5e2d feat(dqn-v2): Plan 4 Task 2c.3b — encoder forward swap to GRN
Plan 4 Task 2c.3b. Forward-path runtime restored. Backward path stays
gated by 2c.3a panic markers until 2c.3c swaps it to GRN backward.

Changes:
- CublasGemmSet now owns four GrnBlock instances (online + target,
  h_s1 + h_s2) and 14 GRN scratch buffers. Constructor allocates them
  alongside the existing branch streams + workspaces.
- New `forward_raw_phase1` + `forward_raw_phase2` methods on GrnBlock
  split the existing forward kernel sequence at the cuBLAS Linear_b
  boundary so the encoder can dispatch
  Linear_a → ELU → Linear_b → Linear_residual → GLU+LN in order.
  No new GRN kernels.
- Swap encoder_forward_only to GRN forward composition: cuBLAS Linear_a
  + bias → forward_raw_phase1 (in-place ELU + DtoD save) → cuBLAS
  Linear_b + bias → cuBLAS Linear_residual (h_s1 only; h_s2 routes
  h_s1_ptr as identity residual) → forward_raw_phase2 (DtoD save +
  GLU + residual+LN).
- New target_encoder_forward_only mirrors the online split using
  dedicated grn_h_s1_target / grn_h_s2_target instances; called with
  save_for_backward=false because the GRN backward only runs against
  the online network (target is inference-only).
- Remove panic gates from encoder_forward_only, forward_target_raw,
  forward_online_f32. forward_target_raw and forward_online_f32 now
  delegate trunk encoding to the new (target_)encoder_forward_only.
- All forward methods on CublasGemmSet migrated from `&self` to
  `&mut self`; trainer call sites refactored to direct
  `self.cublas_forward.method()` so disjoint-borrow rules let
  `self.launch_*` helpers execute alongside.
- Delete orphaned iqn_trunk_forward_kernel + IqnHead::trunk_forward_kernel
  field + IqnKernels.trunk_fwd field + load("iqn_trunk_forward_kernel")
  call. Replace the cuBLAS fallback in
  gpu_iqn_head.rs::execute_training_pipeline with a hard error: IQN
  now requires set_cached_target_h_s2 to be called with the buffer
  written by BatchedForward::target_encoder_forward_only, so online
  and target share ONE trunk implementation. Dead cuBLAS scaffolding
  (trunk_l1_gemm/trunk_l2_gemm/trunk_h1_scratch/launch_trunk_bias_relu)
  marked #[allow(dead_code)] to keep the diff compact.
- Audit doc updated with Task 2c.3b row.

Backward gates intentionally kept on backward_full,
apply_iqn_trunk_gradient, apply_ensemble_diversity_backward — they
panic loudly until 2c.3c swaps the relu_mask trunk backward to the
GRN backward chain.

Smoke (`cargo test … multi_fold_convergence --ignored`): forward path
runs cleanly through online + target encoder forwards (logs 4×
`GrnBlock initialised` confirming both online + DDQN sets allocate
online + target instances). Smoke then panics inside backward_full,
which is the retained 2c.3a gate. Smoke completion is therefore
deferred to 2c.3c per the spec's "KEEP backward gates" non-negotiable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:15:47 +02:00
jgrusewski
f7697851f7 feat(dqn-v2): Plan 4 Task 2c.3a — GRN trunk param-tensor reshuffle (runtime-broken intentionally)
Plan 4 Task 2c.3a. Build-clean prep commit. Lays down the GRN param-tensor
layout (86 -> 95 tensors) and migrates all 93 padded_byte_offset call
sites + ancillary index references in lockstep. Does NOT swap the trunk
forward — that's Task 2c.3b. Explicit panics at every trunk-forward and
trunk-backward caller ensure any accidental runtime execution fails
loudly rather than producing silent garbage.

Param-tensor reshuffle:
- Delete: W_S1, b_S1, W_S2, b_S2 (old trunk's 4 Linear tensors)
- Insert: 7 h_s1 GRN tensors (W_a, b_a, W_b, b_b, W_residual, gamma, beta)
- Insert: 6 h_s2 GRN tensors (no W_residual — SH1 == SH2 makes residual
  identity)
- All tensor indices >= 4 in OLD layout shift +9 in NEW layout
- NUM_WEIGHT_TENSORS 86->95; FIRST_ISV_TENSOR 68->77

layout_fingerprint_seed() updated. New LAYOUT_FINGERPRINT_CURRENT:
0xcf3a24b0a1f70057 (was 0xa504d3c2f275b8af).

Xavier init paths added for the 13 new tensors:
- W matrices (Linear_a, Linear_b, Linear_residual): Xavier
- LayerNorm gamma_h_s1, gamma_h_s2: 1.0
- LayerNorm beta_h_s1, beta_h_s2: 0.0
- All biases (b_a_h_s1, b_b_h_s1, b_a_h_s2, b_b_h_s2): 0.0

Encoder gates: BatchedForward::encoder_forward_only,
BatchedForward::forward_target_raw, BatchedForward::forward_online_f32,
BatchedBackward::backward_full, GpuDqnTrainer::apply_iqn_trunk_gradient,
GpuDqnTrainer::apply_ensemble_diversity_backward all panic at function
entry. Original bodies preserved (with #[allow(unreachable_code,
unused_variables)]) for Task 2c.3b/c to swap GRN forward + backward
in place.

Spectral-norm descriptor (13 matrices): slots [0]/[1] re-mapped to GRN's
w_a_h_s1/w_a_h_s2 — shapes match legacy W_s1/W_s2 exactly, so the
existing spectral-norm constraint transfers cleanly to the GRN's first
Linear. Linear_b / Linear_residual not yet covered (Task 2c.3c).

Smoke intentionally NOT run. Build-clean is the validation. Task 2c.3b
will swap the encoder forward (gpu_grn::GrnBlock::forward); Task 2c.3c
will swap the trunk backward (gpu_grn::GrnBlock::backward) and run smoke.

Validation:
- cargo check --workspace: 11 baseline warnings (unchanged)
- cargo build -p ml --lib: all 58 cubins compile clean
- 93 padded_byte_offset call sites all migrated; spectral_norm
  descriptor + branch_w_base + decoder_forward_only + value_head
  indices all updated in lockstep per feedback_no_partial_refactor.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 12:48:39 +02:00
jgrusewski
4402dbaf3d audit(dqn-v2): h_s2 audit anchor corrections (post-2c.3+4 reconnaissance)
The 2c.3+4 dispatch agent surfaced two stale anchors in the
2026-04-25 h_s2 consumer audit:

1. Row 11 (IQN target trunk): the kernel referenced as
   `iqn_compute_target_h_s2` in `iqn_dual_head_kernel.cu:1031` is
   actually `iqn_trunk_forward_kernel`, and it is ALREADY ORPHANED
   (loaded into IqnHead::trunk_forward_kernel but never invoked).
   The real IQN target trunk runs through cuBLAS iqn_lt_matmul calls
   in gpu_iqn_head.rs::execute_training_pipeline:820-846 (with a
   cached fast-path at 803-847). Mitigation reduced to: delete dead
   kernel + extract target_encoder_forward_only + replace cuBLAS
   fallback path with that extraction.

2. Row 22 (relu_mask sites): line numbers drifted. Audit said
   5581/5798; current code is 5655/5697/5872. Crucially, 5697 is
   the h_s1 mask, not h_s2/IQN-aux — the audit's "3 sites" claim
   needs verification per site before editing. Updated to the
   current line numbers with a "verify before editing" note.

These are documentation-only corrections. The mitigation strategy
(GRN backward chain replaces relu_mask; target_encoder_forward_only
unifies trunk implementation) is unchanged in spirit.
2026-04-25 12:29:47 +02:00
jgrusewski
b7f941f7fa feat(dqn-v2): Plan 4 Task 2c.2 — gpu_grn.rs Rust wrapper (additive, no callers)
Plan 4 Task 2c.2. Additive Rust wrapper around the kernels landed in
2c.1 (`grn_kernel.cu`, commit 68197c2c2). NO production callers in
this commit; dead code until Task 2c.3+4 wires it into the trunk
encoder.

GrnBlock struct holds:
- 5 save-for-backward state buffers per block (elu_post, linear_b_out,
  sigmoid_b, ln_mean, ln_rstd, ln_normed) plus 2 partial-reduction
  scratch buffers (dgamma_partials, dbeta_partials)
- 8 CudaFunction handles for the kernels in grn_kernel.cu
- has_residual_projection flag (true for h_s1 SD->SH1, false for
  h_s2 SH1->SH2)

forward() sequences elu_inplace -> DtoD save post-ELU -> DtoD save
linear_b_out -> glu_forward -> residual+layernorm. cuBLAS GEMMs
(Linear_a, Linear_b, optional Linear_residual) called by caller
outside the wrapper.

backward() sequences LN backward dx (full Jacobian) -> LN backward
dgamma/dbeta two-phase (no atomicAdd) -> GLU backward -> ELU backward.
Residual split is pure pointer aliasing; cuBLAS Linear backward GEMMs
called by caller between steps 4 and 6.

ELU backward saves the POST-activation per the kernel's docstring
(grn_elu_backward recovers f'(x_pre) from x_post via the identity
exp(x_pre) = x_post + 1 for x_post < 0). The wrapper DtoD-copies the
post-ELU buffer into state.elu_post immediately after the in-place
ELU launch.

LN dgamma/dbeta phase-1 partial-reduction allocation is sized for the
maximum batch (num_blocks_b = ceil(B/256)); runtime backward verifies
the runtime batch fits.

Pattern matches gpu_tlob.rs and gpu_attention.rs for idiom
consistency. Module is dead code; production wiring + numerical
validation happens in 2c.3+4.

GRN_CUBIN ref added in gpu_dqn_trainer.rs alongside other Plan 4
kernel cubins. mod.rs declares pub mod gpu_grn. Wire-up audit doc
updated with Task 2c.2 entry + 1 new Orphan-by-design row (will
reclassify Wired at 2c.3+4 alongside the 2c.1 kernel entry).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 12:15:36 +02:00
jgrusewski
68197c2c25 feat(dqn-v2): Plan 4 Task 2c.1 — GRN kernel module (forward + backward, additive)
Plan 4 Task 2c.1. Additive module — kernels compile and load via the
existing kernel-loading infrastructure but have NO production callers
in this commit. Task 2c.3+4 wires them into the trunk encoder.

Eight kernels in grn_kernel.cu (Linear_a/Linear_b/Linear_residual are
cuBLAS GEMMs, not new kernels):
- grn_elu_inplace: element-wise ELU (canonical α=1.0)
- grn_glu_forward: GLU split + sigmoid (saves sigmoid for backward)
- grn_residual_layernorm_forward: residual add + LN, saves mean/rstd/normed
- grn_layernorm_backward_dx: FULL JACOBIAN (not the simplified
  attn_layer_norm_bwd_dx which would silently propagate
  approximation into trunk gradients)
- grn_layernorm_backward_dgamma_dbeta_p1: per-block partial reduction
  (no atomicAdd per feedback_no_atomicadd.md)
- grn_layernorm_backward_dgamma_dbeta_p2: final reduce across blocks
- grn_glu_backward
- grn_elu_backward

LN backward formula (full Jacobian per pearl_cold_path):
  d_out_g = d_out * gamma
  s1 = sum_d(d_out_g)
  s2 = sum_d(d_out_g * normed)
  d_x = (1/H) * rstd * (H * d_out_g - s1 - normed * s2)

Layout convention: row-major [B, H] (sample-major, matches spec §4.E.2
and dt_layernorm_kernel) — intentionally different from
attention_kernel.cu's [D, B] col-major; the trunk encoder downstream
of GRN uses row-major buffers, so per-row LN avoids a transpose.

build.rs registers grn_kernel.cu (kernel count: 57 → 58). Verified
nvcc compiles cleanly via `cargo build -p ml --lib`. Module is dead
code until 2c.3+4. Wire-up audit updated with the additive entry per
Invariant 7.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 12:02:36 +02:00
jgrusewski
e9100073b9 plan(dqn-v2): Plan 4 Task 2c GRN — further decomposed into 2c.1/2c.2/2c.3+4/2c.5
A first dispatch of monolithic Task 2c was refused with a second scope
assessment that surfaced three architectural facts beyond what 2a/2b
had captured:

(d) attn_layer_norm_bwd_dx is a SIMPLIFIED element-wise approximation
    (d_x = d_out * gamma / std), not the full LN Jacobian. Reusing
    it in the GRN backward would propagate the approximation into
    trunk gradients silently. Task 2c.1 must write a new full
    Jacobian: (1/D) * rstd * (D * d_out_g - sum_d(d_out_g) -
    normed * sum_d(d_out_g * normed)).

(e) IQN target-trunk migration is not a clean swap. iqn_compute_
    target_h_s2 reproduces the legacy LeakyReLU trunk independently
    in CUDA. Deleting it requires a target_encoder_forward_only
    extraction on the target path (Task 4-style refactor on the
    target side). Sub-task in itself.

(f) Three relu_mask removal sites have three different save-buffer
    lifetimes (online trunk per-step, IQN-aux per-step in different
    trainer, target-sync per-target-update). Three plumbing tasks,
    not one.

Decomposition:
- 2c.1: grn_kernel.cu with full LN Jacobian backward + GLU + ELU +
  residual-add. Module compiles standalone (additive, dead code).
- 2c.2: gpu_grn.rs Rust wrapper with 5 save-for-backward state
  buffers per block (not 3 as the prior plan suggested).
- 2c.3+4: ATOMIC commit — compute_param_sizes 86→95 + fingerprint
  rewrite + xavier init for 13 new tensors + all 98 padded_byte_
  offset migrations + 3 relu_mask sites with different save-buffer
  plumbing each + iqn target trunk delete + target_encoder_forward_
  only + mag_concat RMS-match adaptive ISV.
- 2c.5: docs flip.

No code changes. Plan-doc only.
2026-04-25 11:53:10 +02:00
jgrusewski
c55c24699a feat(dqn-v2): Plan 4 Task 2b — extend layout fingerprint to cover param-tensor layout
Prerequisite for Plan 4 Task 2c (GRN ADOPT). The current
layout_fingerprint_seed() only covers ISV slot names + indices;
param-tensor layout shifts (which GRN insertion will cause) pass
silently through checkpoint load.

Extends the seed string to include all 86 param-tensor positions
by canonical name. Any structural reshuffle (insert/delete/rename
a tensor) now triggers a different fingerprint and fail-fast at
checkpoint load.

Pragmatic scope (Option A): tensor names + positions, not sizes.
Sizes depend on runtime config (shared_h1, shared_h2, etc.) and
can't be embedded in a const fn. Size mismatches between checkpoint
and current binary are caught by safetensors deserialization
separately. Task 2c's GRN insertion is structural (new tensor
names at new positions) — Option A suffices.

This commit IS a checkpoint break: every existing checkpoint's
fingerprint matches the old seed and fails load. Behavior change
zero; cold-start smoke passes at baseline Sharpe range
(fold-2 best Sharpe = 96.65).

New fingerprint value: 0xa504d3c2f275b8af.

Pearl-aligned: complete fingerprint coverage for what it claims
to protect. Partial coverage was worse than no coverage because
it implied safety where there wasn't.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:44:07 +02:00
jgrusewski
f00c056b49 audit(dqn-v2): h_s2 consumer ReLU-vs-LayerNorm semantics audit (Plan 4 Task 2a)
Prerequisite for Plan 4 Task 2c (GRN ADOPT). Inventories every
trunk-h_s2 consumer and classifies whether each tolerates GRN's
zero-mean signed output or requires the legacy ReLU non-negativity.

Output drives Task 2c mitigation strategy: which consumers need
relu(h_s2) shims at consumption time, and which can take signed
input directly.

No code changes — research only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:17:28 +02:00
jgrusewski
c0d3d7b2ff plan(dqn-v2): Plan 4 Task 2 GRN — decomposed into 2a/2b/2c with reality checks
After a first dispatch attempt of monolithic Task 2 (GRN ADOPT) was
correctly refused with a thorough scope assessment, this revision
records the decomposition and the structural facts that drove it:

1. layout_fingerprint_seed() only fingerprints ISV slots, not
   param-tensor layout. The "fingerprint will auto-update" assumption
   in the original Task 2 spec was wrong for param-tensor reshuffles.
2. compute_param_sizes has 86 tensors (docstring saying 42 is stale).
   Inserting GRN's 9 sub-tensors shifts 82 downstream tensors and
   98 padded_byte_offset call sites.
3. At least 12 kernels consume h_s2 expecting ReLU non-negativity.
   GRN's LayerNorm output is zero-mean (signed). Per-consumer
   verification needed before swap.
4. crates/ml-supervised::tft::gated_residual is incompatible as a
   port: different formula (sigmoid gate, not GLU split) and
   incompatible tensor abstraction. New CUDA kernels from scratch.

Decomposition:
- Task 2a (research, no code): audit h_s2 consumers for ReLU vs
  LayerNorm semantics. Output: per-consumer table.
- Task 2b (small, checkpoint break): extend fingerprint seed to
  include param-tensor names + sizes. Pearl-aligned: complete
  fingerprint coverage rather than partial.
- Task 2c (large, checkpoint break): GRN kernels + 98 call-site
  migration + h_s2 consumer shims (per 2a). Blocked on 2a+2b.

Recommended order updated to thread 2a → 2b → 2c. Tasks 1, 6, 3
remain independent of 2c and can land in parallel where useful.
Pearl rules section added documenting the safety constraints
applied throughout the plan.

No code changes. Plan-doc only.
2026-04-25 11:11:13 +02:00
jgrusewski
c247d9cab0 refactor(dqn-v2): delete legacy CPU-DtoH per_branch_vsn_mean / per_branch_target_drift
Per pearl_cold_path_no_exception_to_gpu_drives.md and explicit user
direction ("remove the legacy paths!"), the two Plan-1-era host-side
DtoH+CPU-loop helpers are GONE — not just routed around. Replaced
with GPU kernel reductions writing to ISV slots.

Deleted (160 lines):
- GpuDqnTrainer::per_branch_vsn_mean()    — DtoH params slice + host abs+mean loop
- GpuDqnTrainer::per_branch_target_drift() — DtoH (target+online) slices + host RMS loop
- FusedTrainingCtx::per_branch_vsn_mean() and per_branch_target_drift() wrappers

Added (GPU-only producers, ~150 lines):
- target_drift_kernel.cu: 2-block reduction RMS(target − online) for mag/dir
  branches. 256-thread smem tree-reduce per block; thread 0 EMA-updates
  ISV slot via pinned device-mapped (no DtoH).
- ISV slots [92] TARGET_DRIFT_MAG_EMA_INDEX, [93] TARGET_DRIFT_DIR_EMA_INDEX
- Fingerprint shifted [90,91] → [94,95]; ISV_TOTAL_DIM 92 → 96
- Trainer accessors: branch_param_slice_indices(), target_params_buf_device_ptr()
- Collector launcher launch_target_drift_ema_inplace()
- Constructor cold-start writes for the 2 new slots

HEALTH_DIAG site refactor:
- vsn_mag/vsn_dir read from ISV[VSN_MAG_EMA_INDEX=87], ISV[88] (Task 5
  GPU-driven kernel produced these, replacing the legacy VSN scalars)
- drift_mag/drift_dir read from ISV[TARGET_DRIFT_MAG_EMA_INDEX=92], ISV[93]
- All 4 reads via FusedTrainingCtx::read_isv_signal_at — pinned device-mapped
  so host reads are coherent with GPU kernel writes without explicit DtoH

isv-slots.md: rows for [87..89] updated to reflect GPU-only producers
(replaces stale text claiming host-DtoH); 2 new rows for [92, 93];
fingerprint shifted to [94, 95]; ISV_TOTAL_DIM bumped 92 → 96.

Smoke fold-2 best Sharpe 100.45 at ep5; per-fold best_val_metric
3.75/10.09/20.21 (avg 11.35) — within Plan 3 T5 baseline (95-117 range).

Pearl validation: this commit demonstrates the cold-path-no-exception
rule applied retroactively. The legacy methods existed for an entire
plan generation; "cold path is fine" was the rationale. New rule:
if a value is computed (reduction/EMA/RMS), the compute is in a kernel,
period — regardless of frequency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:02:23 +02:00
jgrusewski
fbc299fa27 feat(dqn-v2): E.4 encoder/decoder Rust API split — additive, no behavior change
Plan 4 Task 4. Pure Rust API refactor — no kernel changes, no parameter
changes, no checkpoint break, no new ISV slot.

New types:
- StateEncoder: wraps the trunk encoder portion (h_s1 + h_s2 GEMMs)
- ValueDecoder: per-branch decoder (4 instances: Dir, Mag, Ord, Urg)
- EncoderOutput: { h_s2_dev_ptr, h_s2_dim, batch_size }
- BranchDecoderOutput: Q-per-action + V_short + V_long (Plan 2 D.3
  horizon-decomposed value already landed) + num_atoms + batch_size

BatchedForward gains 2 helper methods:
- encoder_forward_only(...) — extracted trunk encoder dispatch
- decoder_forward_only(branch_idx, ...) — single-stream per-branch
  dispatch mirroring the loop body in forward_online_raw

forward_online_raw stays intact and callable; existing call sites
(training_loop, eval, experience collector) unchanged. The trunk
portion of forward_online_raw now routes through encoder_forward_only
internally — lossless extraction (same launch sequence, same
workspace usage). The new wrappers are ADDITIVE attachment points for
Plan 4 Task 1 (Full VSN, attaches pre-encoder), Task 3 (multi-Q IQN,
attaches at decoder), and Task 6 (auxiliary heads, attaches at decoder
peer). Migration of existing callers is deferred — the goal of this
task is establishing a clean type boundary, not migrating call sites.

Smoke (RTX 3050 Ti, 5 epochs, 3 folds, T5 baseline): fold 2 best
val Sharpe = 109.06 at epoch 3 (within RNG noise of T5 landing's
fold-2 best of 95.09; the refactor is behaviorally a no-op since
encoder_forward_only is invoked with byte-identical args).

cargo check --workspace: 11 warnings (matches baseline).

No checkpoint break. No new ISV slot. No new CUDA kernels. No call-site
migration in this commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 10:40:05 +02:00
jgrusewski
cfc4ccb72f feat(dqn-v2): E.5 Mode A attention-focus ISV — fully GPU-driven
Plan 4 Task 5 Mode A. Light, ISV-only, no model parameters added,
no checkpoint break.

ISV tail-append:
- [87] VSN_MAG_EMA_INDEX
- [88] VSN_DIR_EMA_INDEX
- [89] MAMBA2_RETENTION_EMA_INDEX
- Fingerprint shifted [85,86] → [90,91]; ISV_TOTAL_DIM 87 → 92

Producer (attention_focus_ema_kernel.cu):
- Single kernel, 3-block multi-reduction, fully GPU-driven
- Block 0 reduces magnitude-branch VSN-weight slice of params buffer
- Block 1 reduces direction-branch VSN-weight slice
- Block 2 reduces mamba2_h_enriched buffer
- Each block: 256-thread smem tree-reduce → mean |x| → EMA-update
  ISV slot via pinned device-mapped (no explicit DtoH)
- Adaptive α matches Plan 3 Task 3 convention

GPU-only correction (per user direction "no dtoh, pinned memory"):
First-pass agent implementation used host-DtoH + CPU loop for the
Mamba2 retention scalar (mirroring the pre-existing
per_branch_vsn_mean() pattern). User flagged this as a violation
of spec §4.C.6 GPU-drives-CPU-reads even on cold path. Rewrote
the kernel to do all 3 reductions on-device via smem block-reduce,
reading params/mamba2 buffers via raw_ptr() and writing to pinned
ISV slots directly. Removed mamba2_retention_mean() from
GpuDqnTrainer + FusedTrainingCtx (was the host-DtoH culprit).

Read-only AttentionMonitor mirrors PlanThresholdMonitor pattern.
StateResetRegistry: all 3 slots FoldReset.

Smoke fold-2 best Sharpe 95.09, avg best_val_metric 10.98 — within
Plan 3 baseline range. ISV slots populate non-zero from epoch 2.

Pearl: cold-path is not an exception to GPU-drives-CPU-reads. If
the producer is computing a value (not just reading a CPU-side
constant), the compute belongs on GPU regardless of frequency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 10:17:29 +02:00
jgrusewski
fab7758916 plan(dqn-v2): Plan 4 full-scope reality reconciliation
Comprehensive revision to match landed Plan 1+2+3 codebase state. Pattern
mirrors the Plan 3 second revision: per-task "Reality reconciliation"
blocks at the top of each task body identifying what's stale vs. landed,
with concrete file paths in the actual cuda_pipeline tree.

Added:
- Task dependency graph with recommended execution order:
  5 (light ISV) → 4 (refactor) → 2 (GRN ADOPT) → 1 (Full VSN) →
  6 (aux heads) → 3 (multi-Q IQN) → 7 (audit) → 8 (Argo)
- Per-task implementation surface with concrete trunk hooks:
  - Task 1: pre-h_s1 VSN gate in batched_forward.rs::forward_online_raw
  - Task 2: GRN audit confirms ADOPT branch (GRN absent from DQN trunk;
    only in ml-supervised TFT). Replaces h_s1/h_s2 Linear blocks.
  - Task 3: re-scoped — IQN already runs num_quantiles=32 with random τ.
    Task is CONSTRAINING to fixed τ ∈ {.05, .25, .5, .75, .95}.
  - Task 4: pure Rust API split (no kernel changes, no checkpoint break)
    around existing forward_online_raw structure
  - Task 5: split into Mode A (light, pre-Task-1, 3 ISV slots) and
    Mode B (full, post-Task-1, 7 ISV slots). Mode A recommended first.
  - Task 6: aux head loss scaled by ISV[LEARNING_HEALTH] per pearl
- Checkpoint-break consolidation note: Tasks 2/1/6/3 each break checkpoint
  compatibility; land in sequence with no Argo run between (one fingerprint
  shift per commit; final Argo at Task 8 amortizes retraining cost)
- Task 8 absorbs Plan 3's deferred Argo Tier 1 gate (combined validation)

No code changes.
2026-04-25 09:30:09 +02:00
jgrusewski
3bfbcd7137 plan(dqn-v2): Plan 4 reality reconciliation
Plan 4 was drafted 2026-04-24 against an earlier Plan 3 design. After
Plan 3 landed (commits 44539d8f4..3cb083f18), the pre-plan gate
referenced slot names that don't exist in the landed implementation:

- PLAN_PARAMS_0_EMA_INDEX → became READINESS_EMA_INDEX (Task 4)
- STATE_KL_THRESHOLD_EMA_INDEX → eliminated; Task 7 uses kernel-internal
  trailing-EMA-of-self pattern (no separate threshold slot)
- TEMPORAL_REWARD_{PERSIST,REGIME_SHIFT,CONSISTENCY}_EMA_INDEX → consolidated
  into rc[5] → ISV[68] REWARD_BONUS_EMA_INDEX via Plan 3 Task 6a/6b/6c

Updated:
- Pre-plan slot-existence check matches landed slot names
- Validation-doc gate softened: Plan 3 Argo Tier 1 PASS becomes OPTIONAL;
  local 5-epoch multi-fold smoke is the de-facto gate (user choice
  to proceed without burning Argo cycles)
- Status table at top shows landed Plan 3 baseline (ISV_TOTAL_DIM=87,
  PS_STRIDE=43, fingerprint at [85,86]) and per-task difficulty estimate
- Reality-reconciliation note documents the rename trail

No task body changes; subsequent commits will revise individual task
specs as they become next-up for execution.
2026-04-25 09:21:36 +02:00
jgrusewski
e1626876af plan(dqn-v2): mark Tasks 8+9 LANDED; 11/12 done 2026-04-25 09:15:43 +02:00
jgrusewski
3cb083f182 feat(dqn-v2): B.3 + C.5 GPU-only replay seed warm-start + CQL α ramp
Plan 3 Tasks 8 + 9. Single commit because Task 9 directly consumes Task 8's
seed-fraction signal; no useful intermediate state.

ISV tail-append:
- [82] SEED_STEPS_TARGET_INDEX — config replay_seed_steps (CPU constructor write)
- [83] SEED_STEPS_DONE_INDEX — GPU-incremented per collect_experiences_gpu
- [84] SEED_FRAC_EMA_INDEX — adaptive EMA of (1 - done/target)
- Fingerprint shifted [80,81] → [85,86]; ISV_TOTAL_DIM 82 → 87

GPU-only design (per user direction "fully gpu driven no cpu involvement"):
- 4 scripted policies as ONE CUDA kernel (scripted_policy_kernel.cu)
- Per-sample policy mix (40% uniform LCG / 20% momentum / 20% mean-rev /
  20% vwap-deviation) deterministic by `i % 5`
- Action source switched at launch boundary (CPU per-epoch read of ISV slot
  decides which kernel to dispatch; the action computation itself is 100% GPU)
- No CPU physics mirror — existing GPU `experience_env_step` runs unchanged

seed_step_counter_update_kernel.cu:
- Single-thread cold-path; increments DONE, computes FRAC = max(0, 1-done/target)
- Adaptive α matches Task 3/4 convention (α_base × (1 + 0.5×|clamp(sharpe,±2)|))

cql_alpha_seed_update_kernel.cu (Task 9):
- target = config.cql_alpha × max(0, 1 - seed_frac)
- During seed phase (frac=1) → target=0 → CQL α decays to 0 (no pessimism on
  exploration data); as frac → 0 → CQL α ramps to config value
- Updates ISV[CQL_ALPHA_INDEX=48]; CQL gradient kernel reads slot 48 via
  pinned device-mapped ISV (Plan 1 Task 12 consumer pattern unchanged)

Registry: SEED_STEPS_DONE + SEED_FRAC_EMA both FoldReset; CQL_ALPHA flipped
SchemaContract → FoldReset; SEED_STEPS_TARGET stays SchemaContract.

Read-only monitors (mirror PlanThresholdMonitor / StateKlMonitor pattern):
- monitors/seed_monitor.rs — surfaces ISV[82..85) for HEALTH_DIAG +
  controller_activity smoke fire-rate
- monitors/cql_alpha_monitor.rs — surfaces ISV[48] + ISV[84] dependency

Smoke (RTX 3050 Ti, 3 folds × 5 epochs, dqn-smoketest profile with
replay_seed_steps=1000 override so seed phase completes mid-fold):
- All 3 folds saved best-checkpoint
- Fold 2 best Sharpe = 92.4938 at epoch 1 (target range 80-120) ✓
- Per-fold val_metric: f0=3.80 / f1=9.73 / f2=20.24 (loss-based)
- HEALTH_DIAG[3..4] cql_alpha=0.0500 with health=0.49 → base ≈ 0.10 from ISV[48],
  consistent with kernel ramping toward final×(1-frac); regime gate
  (1-regime)×health applies on top
- 11 cargo check warnings (matches pre-task baseline; no new warnings)
- 6/6 monitor unit tests pass (read/diagnose/observe×fire_rate)

Smoke override rationale: smoke runs ~200 samples per collect (4 episodes ×
50 timesteps) × 5 epochs × 3 folds ≈ 3000 total. Default 100k target would
keep entire smoke in seed phase. Override to 1000 lets the seed→network
transition complete mid-fold so the CQL α ramp is observable.

Per pearl_one_unbounded_signal_per_reward.md: cql_alpha is bounded (clamped
to config_final × (1 - seed_frac) ∈ [0, config_final]), composes safely with
downstream CQL loss.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:13:39 +02:00
jgrusewski
0c48dc7192 plan(dqn-v2): mark Task 7 LANDED 2026-04-25 02:26:39 +02:00
jgrusewski
673eb66124 feat(dqn-v2): C.3 state-distribution KL divergence + adaptive amplification
Plan 3 Task 7.

ISV tail-append:
- [78] STATE_KL_TRAIN_VAL_EMA_INDEX — moment-match KL on OFI dims, EMA
- [79] STATE_KL_AMPLIFICATION_INDEX — Flat-trap escape multiplier ∈ [1,2]
- Fingerprint shifted [76,77] → [80,81]; ISV_TOTAL_DIM 78 → 82

Producer (state_kl_divergence_kernel.cu):
- Per-dim Gaussian moment-match KL summed over OFI block (32 dims)
- Adaptive α matches Task 3 convention: α_base × (1 + 0.5 × |sharpe|), α_base=0.05
- Amp formula: ratio = new_ema / prev_ema; target = 1 + clamp(ratio−1, 0, 1)
- Kernel-internal trailing-EMA-of-self pattern — no separate threshold ISV slot
- Single block, one thread per OFI dim, smem reduction, no atomicAdd, no DtoH

Val-side launch: GpuBacktestEvaluator exposes val_state_sample() returning
(ptr, n) into the chunked_states_buf the val pass populates. Train side uses
the experience-collector's existing states buffer. Both buffers in same
CUDA context; launch enqueues on collector's stream after compute_validation_loss.

Consumer (experience_kernels.cu):
- B.1 opp_cost ×= kl_amp (penalty intensifies on train/val drift)
- B.2 bonus ×= kl_amp (novelty bonus intensifies on drift)
- fmaxf(1.0, ISV[STATE_KL_AMP]) guard at consumers handles cold-start 0 → no-op

Per pearl_one_unbounded_signal_per_reward.md: amp ∈ [1,2] is bounded; stacks
safely with B.1's q_abs_ref unbounded multiplicand. No formula blow-out risk.

Constructor cold-start: KL=0.0, amp=1.0 (consumer no-op until kernel fires).
Registry: both slots FoldReset with cold-start values restored on fold boundary.
Read-only StateKLMonitor surfaces both slots in diagnose snapshot.

Smoke results: All 3 folds pass cleanly with diverse epoch convergence
patterns (best at ep5/ep2/ep3 — addresses prior concern about always-ep1
in fold 2). Per-fold best Sharpe: 105.13, 83.28, 103.49. Average
best_val_metric 10.64.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 02:26:09 +02:00
jgrusewski
aa8e6f8ec1 plan(dqn-v2): mark Task 4 LANDED 2026-04-25 01:36:24 +02:00
jgrusewski
8f59c1e3b8 feat(dqn-v2): B.4 adaptive plan threshold — upgrade PLAN_THRESHOLD_INDEX static→GPU-driven
Plan 3 Task 4.

ISV tail-append:
- [75] READINESS_EMA_INDEX — batch-mean readiness EMA (GPU-written)
- [49] PLAN_THRESHOLD_INDEX — producer upgraded from static constructor
  write to GPU kernel (same consumer path unchanged)
- Fingerprint shifted [73,74] → [76,77]; ISV_TOTAL_DIM 75 → 78

Producer (plan_threshold_update_kernel.cu):
- Single-block reduction of readiness_per_sample [N*L]
- Adaptive α = α_base × (1 + 0.5 × |clamp(sharpe, -2, 2)|); α_base=0.05
- Derived: threshold = max(0.1, 0.5 × readiness_ema) → ISV[49]

Consumer sites unchanged (experience_kernels.cu 4 sites + backtest_plan_kernel.cu).
The upgrade is producer-only; consumers keep reading ISV[49] as before but now
receive an adaptive value tracking the policy's actual readiness distribution
rather than a hardcoded 0.5 midpoint.

PlanThresholdMonitor (read-only observer) surfaces plan_threshold.eff +
plan_threshold.readiness_ema for HEALTH_DIAG / controller_activity smoke.

StateResetRegistry: PLAN_THRESHOLD flipped SchemaContract→FoldReset;
READINESS_EMA registered as FoldReset. Both fold-reset arms restore the
cold-start values (0.5 / 1.0) before the first kernel fire on the new fold.

No breaking changes to consumer API.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 01:35:07 +02:00
jgrusewski
3af2c79def plan(dqn-v2): mark Task 6c LANDED 2026-04-25 01:11:04 +02:00
jgrusewski
0bbe97ed85 feat(dqn-v2): D.4c conviction consistency bonus — reward stable pre-entry deliberation
Plan 3 Task 6c.

Portfolio-state tail-append:
- PS_PRE_ENTRY_CONVICTION_EMA = 41 (EMA mean of conviction_core during Flat)
- PS_PRE_ENTRY_CONVICTION_VAR_EMA = 42 (EMA of squared deviations)
- PS_STRIDE 41 -> 43
- All 6 hardcoded-stride sites migrated in lockstep

Producer (experience_kernels.cu Flat branch):
- Per-bar EMA update alpha=0.05 (matches Task 1 reward-ema convention)
- Welford-style: delta = c - mean; var_ema = (1-alpha)*(var + alpha*delta^2)

Consumer (experience_kernels.cu entering_trade block):
- ratio = stddev/mean; stability = clamp(0, 1, 1 - ratio/0.2)
- Fires only when ratio < 0.2 (stable pre-entry conviction)
- bonus = shaping x vol_proxy x stability x conviction_core
- All multiplicands in [0,1] except vol_proxy (<=0.01); max bonus ~ 0.01
- Mirrors B.2 novelty-bonus structure — one bounded shape replaced (novelty -> stability)
- rc[5] += bonus; both EMA slots reset at entry, reversal, fold/episode boundary

Per pearl_one_unbounded_signal_per_reward.md: exactly ONE unbounded
multiplicand (vol_proxy); all others bounded. No `q_scale x |reward|`
style blowout possible.

No new ISV slot.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 01:10:03 +02:00
jgrusewski
efb4e529a4 plan(dqn-v2): mark Task 6a, 6b LANDED with iteration history 2026-04-25 00:51:10 +02:00
jgrusewski
5ebebc564a feat(dqn-v2): D.4b regime-shift penalty — bounded-by-|reward|
Plan 3 Task 6b.

Portfolio-state tail-append:
- PS_REGIME_SHIFT_BAR = 40 (hold_time of first detected regime shift, 0 if none)
- PS_STRIDE 40 → 41
- All 6 hardcoded-stride sites migrated in lockstep

Detector (experience_kernels.cu):
- Adaptive threshold = clamp(0.25 × |clamp(sharpe, -2, 2)|, 0.05, 0.5)
- Fires first bar where |regime_now - PS_PLAN_ENTRY_REGIME| > threshold
- First-shift-only (short-circuits on non-zero PS_REGIME_SHIFT_BAR)
- Uses new ISV_SHARPE_EMA_IDX = 22 macro in state_layout.cuh

Consumer (segment_complete block):
- bars_late_frac = clamp(bars_late / hold_time, 0, 1)
- penalty = shaping × conviction × bars_late_frac × |reward|
- All multiplicands except |reward| in [0,1]; max penalty = |reward|
- reward -= penalty; rc[5] -= penalty (cancels with B.2/C.4/D.4a at
  other (i,t) slots; ISV[68] REWARD_BONUS_EMA shows net)
- Consumer resets PS_REGIME_SHIFT_BAR after use

**Iteration history.** First pass multiplied by ISV[Q_DIR_ABS_REF] (~5–50,
an absolute Q-magnitude) AND |reward| — produced penalties 5–50× the
reward, destabilising training (smoke: Return swings ±300–900%, Sharpe
oscillating wildly). Root cause: Q_DIR_ABS_REF is an absolute
magnitude, not a [0,1] coefficient; B.1 uses it as a DENOMINATOR to
normalize q_range, not as a multiplier on an already-unbounded signal.
Fix: drop q_scale, keep |reward| as the only unbounded factor. Smoke
now passes cleanly with fold-2 best Sharpe 117.92 (up from T6a's 100.10).

No new ISV slot.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 00:49:47 +02:00
jgrusewski
7773417761 feat(dqn-v2): D.4a persistence credit — reward trades that held through drawdown
Plan 3 Task 6a.

Portfolio-state tail-append (shared-contract migration, all in same commit):
- PS_INTRA_TRADE_MIN_PNL = 39 (symmetric to PS_INTRA_TRADE_MAX_PNL = 21)
- PS_STRIDE 39 -> 40
- 6 PORTFOLIO_STRIDE hardcoded copies bumped in lockstep

Producer (experience_kernels.cu):
- MIN_PNL tracked per bar (fminf against pnl_pct) in the same block
  as MAX_PNL update
- Reset to 0 at all 5 MAX_PNL reset sites (entry, reverse, 2x fold boundary)

Consumer (experience_kernels.cu segment_complete):
- Fires only on reward > 0 AND drawdown_depth > 1e-6
- persist_bonus = shaping x conviction x |min_pnl| x tanh(reward/|min_pnl|)
- reward += persist_bonus; rc[5] += persist_bonus (accumulates with
  B.2 entry bonus + C.4 timing bonus — different (i,t) slots per trade)

Self-scaling via tanh: no tuned coefficients. Saturates when recovery
is large relative to drawdown; near-zero when recovery is trivial.
Attribution lands in ISV[68] REWARD_BONUS_EMA via the Task 1 kernel.

No new ISV slot.

Smoke: multi_fold_convergence PASS (fold-2 best Sharpe 100.10, threshold >=80).
HEALTH_DIAG reward_split bonus=17.21 (post-Task-5 rises with new D.4a credit
firing on profitable drawdown recoveries).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 00:07:48 +02:00
jgrusewski
87b52ea946 plan(dqn-v2): Plan 3 second revision — reconcile with landed reality
Tasks 1, 2, 3, 5 landed on main between a59e7599c and a0abc3da3. The
remaining Task 4/6/7/8/9 bodies in the plan had accumulated drift:

- Task 4: allocated slot 49 (already PLAN_THRESHOLD_INDEX); EMA'd
  plan_params[0] (kernel compares against readiness).
- Task 6: allocated 59/60/61 (now collides with Plan 2 Q-quantile
  [50..58) and Task 1 reward EMAs [63..69)); required 6 new PS memo
  fields and included tuned 0.5e-4f penalty.
- Task 7: allocated 58/63/64 (63/64 collide with REWARD_TRAIL_EMA /
  REWARD_MICRO_EMA from Task 1); amplification formula had tuned
  2.0× trigger, 0.02 decay, 1.0/2.0 endpoints.
- Task 8: referenced trainer helpers that don't exist
  (sample_state_feature_pair, step_scripted, replay_insert_with_
  priority_scale); tuned priority_scale=0.5.
- Task 9: used pre-pivot CPU-compute AdaptiveMonitor pattern with
  tuned 0.9/0.1 EMA rates.

This revision:
- Adds per-task "Reality reconciliation" sub-header flagging the
  stale premise being fixed.
- Marks landed tasks with  LANDED <SHA> and records actual outcomes
  (vs. the planned outcomes the original text described).
- Rewrites Task 4/6/7/8/9 bodies to use tail-append slot allocation
  (indices recomputed from ISV_TOTAL_DIM at impl time), GPU kernel
  producer + read-only AdaptiveMonitor consumer pattern, and
  ISV-derived adaptive coefficients in place of tuned constants.
- Splits Task 6 into 6a/6b/6c with independent PS-slot additions
  (MIN_PNL / REGIME_SHIFT_BAR / PRE_ENTRY_CONVICTION_EMAs).
- Enumerates concrete trainer-helper prereqs for Task 8.
- Updates Task 10 metric bands to match actual landed ISV slot names.
- Updates exit criteria summary to check off what landed.

No code changes.
2026-04-24 23:48:07 +02:00
jgrusewski
a0abc3da35 feat(dqn-v2): C.4 temporal timing bonus on trade exit — peak-bar proximity
Plan 3 Task 5.

Portfolio-state tail-append (shared-contract migration, all in same commit):
- PS_PEAK_PNL_BAR = 38 (hold_time snapshotted when MAX_PNL updates)
- PS_STRIDE 38 -> 39 in state_layout.cuh and ml-core/state_layout.rs
- PORTFOLIO_STRIDE 38 -> 39 in trade_stats_kernel.cu (hardcoded copy)
- PORTFOLIO_STRIDE 38 -> 39 in gpu_experience_collector.rs allocator
- ps_stride 38 -> 39 in gpu_dqn_trainer.rs launch_kelly_cap_update

Producer (experience_kernels.cu):
- Peak bar snapshotted alongside every MAX_PNL update (uses local
  hold_time, not ps[PS_HOLD_TIME], because the portfolio-state commit
  block runs later in the kernel).
- Peak bar reset to 0 at every MAX_PNL reset site: plan-entry (1856),
  entering_trade (2014), reversing_trade (2019), fold hard-reset (2736),
  trade-complete soft-reset (2751).

Consumer (experience_kernels.cu segment_complete block):
- bars_early = max(0, segment_hold_time - PS_PEAK_PNL_BAR)
- timing_bonus = shaping_scale x (bars_early / segment_hold_time)
                 x |final_pnl| x conviction_core
- reward += timing_bonus; rc[5] += timing_bonus
  (accumulates with Task 3 B.2 entry bonus — different (i,t) slots).

No new ISV slot — rc[5] bonus semantics unchanged; B.2 and C.4 share it
via += accumulate semantics (defensively idempotent, but the two sites
fire at distinct (i,t) by construction: entry vs exit).

Self-scaling: shaping_scale x conviction_core x |pnl| keeps the bonus
proportional to trade magnitude, no tuned coefficients.

Smoke multi_fold_convergence (RTX 3050 Ti): all 3 folds complete,
fold-2 best Sharpe 84.44 at epoch 1 (expected ~85 range).
cargo check --workspace clean at 11 warnings baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 23:34:27 +02:00
jgrusewski
b5d19c1004 feat(dqn-v2): B.2 ISV-driven trade-attempt bonus — novelty at Flat→Positioned
Plan 3 Task 3.

ISV tail-append:
- [71] TRADE_ATTEMPT_RATE_EMA — Flat→Positioned transition rate EMA
       (GPU-written by trade_attempt_rate_ema_update, adaptive α)
- [72] TRADE_TARGET_RATE — reference rate, CPU-frozen at epoch 5
- Fingerprint shifted [69,70] → [73,74]; ISV_TOTAL_DIM 71 → 75

Producer kernel (trade_rate_ema_kernel.cu):
- Single-block reduction of flat_to_pos_per_sample [N*L] (no atomicAdd)
- Adaptive EMA: α = α_base × (1 + 0.5 × |clamp(sharpe, -2, 2)|)
- α_base = 0.05 (matches reward_component_ema convention)
- Launched from training_loop alongside reward_component_ema

Consumer (experience_env_step):
- Flat→Positioned site: novelty = max(0, 1 - attempt/target)
- bonus = conviction_core × vol_proxy × novelty
- reward += shaping_scale × bonus; rc[5] captures bonus for ISV[68]
- Explicit freeze gate: target_raw > 1e-6f, so bonus is structurally
  inert pre-freeze (prevents spurious novelty=1.0 on epoch-1 when
  attempt_rate is still 0)

Epoch-5 freeze (training_loop):
- measured = ISV[TRADE_ATTEMPT_RATE_EMA]; floor at 0.001
- Prevents novelty from sticking at 1.0 post-freeze

StateResetRegistry: both slots registered as FoldReset with per-slot
reset dispatch arms in training_loop's fold-boundary path.

Smoke: multi_fold_convergence passes (fold 2 best Sharpe 87.55 —
 slightly above Task 2 baseline 85.6, within noise; bonus inert
 in 5-epoch smoke so training matches pre-B.2 baseline as designed).
cargo check clean at 11 warnings baseline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 23:11:20 +02:00
jgrusewski
12bba98ecd feat(dqn-v2): B.1 Flat opp-cost scales with ISV[Q_DIR_ABS_REF] — self-scaling no-tuning
Current Flat opp-cost was conviction-driven but |Q|-scale was implicit.
When training drifts Q magnitudes into ±50, opp-cost becomes invisible
relative to action Q-values → Flat wins argmax. Fix: multiply by
isv_signals_ptr[21] (Q_DIR_ABS_REF_INDEX, EMA of max(|Q_mean|) across
direction bins, populated by update_eval_v_range / q_stats_kernel).

Self-scaling: opp-cost tracks |Q| proportionally throughout training.
No tuned multiplier; relies on existing ISV slot. Floor 1e-3 for cold-
start protection before EMA is warm. rc[4] is now written in the Flat
branch so reward_component_ema kernel picks it up into ISV[67].

No parallel paths — the old formula IS modified in place.

Plan 3 Task 2. Spec §4.B.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 22:18:14 +02:00
jgrusewski
44539d8f4b feat(dqn-v2): C.2 reward-component attribution — 6 ISV EMAs + HEALTH_DIAG split
New GPU kernel reward_component_ema reads per-sample per-component
rewards [popart, cf, trail, micro, opp_cost, bonus] and updates 6 ISV
EMA slots via adaptive-rate EMA (α=0.05). CPU-side RewardComponentMonitor
is a read-only observer per spec §4.C.6; HEALTH_DIAG emits reward_split
line each epoch.

experience_kernels.cu extended with reward_components_per_sample [N*L,6]
output param; rc[0]=popart, rc[1]=cf, rc[3]=micro populated; rc[2,4,5]
are structural placeholders for Plan 3 B.1/B.2/C.4/D.4.

ISV slots 63-68 added (REWARD_{POPART,CF,TRAIL,MICRO,OPP_COST,BONUS}_EMA);
fingerprint shifted 61-62 → 69-70; ISV_TOTAL_DIM 63 → 71.
Layout fingerprint seed updated; auto-recomputes on next run.

StateResetRegistry: isv_reward_component_emas → FoldReset.
training_loop.rs: reset_named_state arm zeroes slots 63..69 at fold boundary.
docs/isv-slots.md + docs/dqn-wire-up-audit.md updated (+2 Wired rows, 107 total).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 22:08:40 +02:00
jgrusewski
a59e7599c8 plan(dqn-v2): Plan 3 revision — GPU-drives-CPU-reads, ISV tail-append, AdaptiveMonitor
Aligns Plan 3 with post-Plan-2 reality:

1. AdaptiveController → AdaptiveMonitor throughout (spec §4.C.6
   2026-04-24 revision). Tasks 4 (plan-threshold) and 9 (cql_alpha
   seed-coupled) become GPU kernel + read-only CPU monitor pairs
   instead of CPU-compute controllers — uniform with Plan 1's
   tau/epsilon/gamma/kelly_cap and Plan 2's per-branch γ pattern.

2. Stale ISV slot indices [49..65) replaced with tail-append language.
   Current post-Plan-2 ISV_TOTAL_DIM = 63; new slots start at 63 and
   grow. Fingerprint at 61-62 auto-re-tails.

3. Architecture paragraph updated to document replay-seed orchestration
   (Task 8 dqn_replay_seed.rs) as constructor-time cold-path pre-training
   — not a GPU-drives violation.

4. Prerequisites section clarified: Plan 2 state — ISV_TOTAL_DIM 63,
   TLOB at SL_TLOB_START, layout fingerprint auto-recomputes.

No scope changes to individual Tasks 1-10. Just terminology + slot
indexing aligned with current main state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 21:46:38 +02:00
jgrusewski
3c18ebd637 feat(dqn-v2): D.8 Plan 2 Task 6C — TLOB cuBLAS port, train+val parity
Implement TLOB attention (OFI[32]→TLOB[16]) using existing DQN cuBLAS
infrastructure.  Random Xavier init, trained end-to-end via DQN reward.
Both training (experience_env_step) and val (backtest_plan_kernel) write
TLOB features to SL_TLOB_START=74, preserving state-dist parity.

State layout changes:
  - SL_TLOB_DIM=16 inserted at SL_OFI_START+SL_OFI_DIM (=74)
  - SL_MTF_START 74→90, SL_PORTFOLIO_START 90→106, SL_PLAN_ISV_START 98→114
  - SL_PADDING_START 105→121, SL_STATE_DIM 112→128
  - state_layout.rs mirrored: TLOB_START=74, STATE_DIM=128

New files:
  - cuda_pipeline/gpu_tlob.rs: GpuTlob with 9 cublasLt GEMM descriptors,
    forward/backward/adam_step, copy_params_from for val weight sync
  - cuda_pipeline/tlob_kernel.cu: tlob_sdp_forward, tlob_sdp_backward,
    tlob_write_states, tlob_read_grad_from_states kernels

Wiring:
  - fused_training.rs: TLOB forward before trunk; TLOB backward+Adam Phase 6
  - training_loop.rs: per-epoch ISV[60]=TLOB_REGIME_FOCUS_EMA update
  - gpu_backtest_evaluator.rs: val TLOB forward after each launch_gather,
    set_tlob_from_training syncs weights from training TLOB each epoch
  - metrics.rs: creates eval GpuTlob on evaluator stream, syncs params
  - gpu_dqn_trainer.rs: ISV_TOTAL_DIM 60→63, fingerprint indices 58/59→61/62

ISV slot audit: isv-slots.md updated (slot 60=TLOB_REGIME_FOCUS_EMA,
fingerprint shifted to 61/62, ISV_TOTAL_DIM=63).
Wire-up audit: dqn-wire-up-audit.md updated (gpu_tlob.rs + tlob_kernel.cu).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 21:38:42 +02:00
jgrusewski
f3a8a5ff62 spec(dqn-v2): D.8 TLOB pivot — cuBLAS port using existing DQN infra, no ONNX, no pretraining
User pearl (2026-04-24): the DQN already has every primitive TLOB needs
(gpu_attention, batched_forward/backward, GpuLinear, cuBLASLt handles,
cuda_autograd). Port TLOB's Q/K/V + attention as a composition of
existing primitives. Random-init, trainable end-to-end from the DQN's
reward signal. Uniform with Mamba2, IQL, atoms/γ/τ/ε — all of which
already follow this pattern.

Eliminates:
- ONNX Runtime dependency (was already dead — stripped from ml-supervised)
- Separate supervised pretraining pipeline
- "Freeze vs fine-tune" false dichotomy
- Pretrained-checkpoint-file-not-found failure mode

Prerequisites for the cuBLAS-native design (all satisfied):
- gpu_attention.rs exists
- GpuLinear trainable layer exists
- cuda_autograd over cuBLAS exists
- MBP-10 data already in the DQN data pipeline

What was "BLOCKED on prerequisites" in the Task 6C audit referred to
the OLD pretrained-ONNX design. The cuBLAS-native design has all
prerequisites satisfied — Task 6C can proceed under the new scope.

Pearl captured in memory: pearl_tlob_no_pretraining.md. Generalises to
any future attention/state-space/Neural ODE module: port to cuBLAS,
random init, let the DQN teach it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 21:03:57 +02:00
jgrusewski
98bb1c4dc1 feat(dqn-v2): D.3 horizon-decomposed V — widen IQL value head to 2 outputs
IQL value head FC expanded from [D_in × 1] to [D_in × 2]. v_out_buf shape
[B] → [B*2]. Consumer code sums the two outputs (V = V_short + V_long)
wherever a scalar V was previously read. Both outputs trained via the
same expectile loss (horizon-specific regression is a follow-up
enhancement — current form provides the architectural capacity for
horizon decomposition without per-horizon targeting).

Changes:
- total_params: w3 H*1+b3[1] → w3 H*2+b3[2]
- gemm_fwd_v: M=1→2; gemm_bwd_dw3: M=1→2; gemm_bwd_dh2: K=1→2
- v_out_buf, dv_buf, loss_buf: [B] → [B*2]
- iql_expectile_loss kernel: new num_heads param; q_taken[b]=q_taken[idx/num_heads]
- iql_loss_reduce kernel: new num_heads param; normalises by B (not B*num_heads)
- bias_add for b3: out_dim=1→2, N=B→B*2
- db3 bias_grad_reduce: gridDim.y=1→2 for per-head gradient accumulation
- V_W3_SIZE/V_B3_SIZE macros: H→H*2, 1→2 (used by iql_forward_kernel)
- iql_forward_kernel: updated for 2-output col-major [2,B] write
- 4 consumer kernels: v_out[b] → v_out[b*2+0] + v_out[b*2+1]
- Xavier init: w3 fan_out 1→2, w3_end H→H*2

Checkpoint compat: IQL parameter count changes. Layout fingerprint
recomputes; old checkpoints fail-fast at load per spec §4.A.2.
Retrain required.

Smoke test: training runs 28s, best Sharpe=80.59 (baseline ~80), 0 errors.
Unit tests: 889 pass / 12 fail (12 pre-existing, 0 regressions introduced).

Plan 2 Task 6B. Spec §4.D.3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 20:56:24 +02:00
jgrusewski
3e5e3ff206 feat(dqn-v2): D.6 plan_isv[6] = remaining_fraction
Adds 7th plan_isv dimension: PLAN_ISV_REMAINING_FRACTION = max(0, min(1,
(plan_target_bars - hold_time) / plan_target_bars)) when plan active, else
0. Exposes temporal pressure to the policy.

State vector grows 104 → 112 (105 + 7 padding for 8-alignment).
SL_PORTFOLIO_PLAN_DIM 6 → 7. SL_PADDING_DIM 0 → 7.

Both training (experience_env_step) and val (backtest_plan_state_isv) write
the new slot identically — preserves train/val state-distribution parity.

All hardcoded stride-6 references in backtest_plan_kernel.cu replaced with
SL_PORTFOLIO_PLAN_DIM. plan_isv_buf allocation updated to n_windows * 7.
Stale offset comments ([86..92)) corrected to [98..105) across all files.

No behavioural change to existing dimensions. New signal is additive.

Plan 2 Task 6A. Spec §4.D.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 20:36:46 +02:00
jgrusewski
ab9c7e43a8 delete(dqn): D.7 liquid_mod audit — ODE identity, remove from c51_grad + experience kernels
The liquid_tau_rk4_step kernel modelled f(x) = (1/tau)*(1-x), which has
fixed point x=1.0 for any tau. liquid_mod_buf was initialised to [1.0;4]
and the dynamics could never move it away from that fixed point since every
kernel call reduces to f(1.0)=0 (no perturbation path exists). The
velocity_mod multiplier in c51_grad_kernel was therefore always 1.0 — a
mathematical identity with zero measurable effect on spread_scale.

Additionally, no ISV slot existed for liquid_mod (violates §4.C.6
GPU-drives spec), and the associated LiquidTrainableAdapter supervised
path was never wired into the DQN backward pass.

Decision C (delete): remove liquid_tau_rk4_step kernel (experience_kernels.cu),
liquid_mod param + velocity_mod line (c51_grad_kernel.cu), liquid_mod_buf
allocation, liquid_tau_rk4_kernel field, update_liquid_tau() method and its
call site in update_eval_v_range() (gpu_dqn_trainer.rs). per_branch_q_gap_ema_buf
is retained — it is used independently for trajectory backtracking state
snapshot/restore. Supervised LiquidTrainableAdapter unchanged.

cargo check -p ml: 8 warnings (baseline), 0 errors.
Audit docs updated: dqn-wire-up-audit.md + ml-supervised-to-dqn-concept-audit.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 20:18:59 +02:00
jgrusewski
f7a91d906d feat(dqn-v2): D.5 soft fold-boundary transitions via bootstrap-write + EMA
SoftReset registry category (isv_grad_balance_targets, isv_grad_scale_limit)
implemented via bootstrap-write + existing kernel's EMA. CPU writes bootstrap
values (1.0 for targets, 2.0 for limit) at fold boundary; grad_balance_isv_update
kernel's adaptive-rate EMA (alpha in [0.01, 0.30]) converges from bootstrap
toward observed values over subsequent epochs (implicit decay_bars via EMA time
constant).

Option A chosen (minimal): the existing grad_balance_isv_update kernel already
uses adaptive-rate EMA — not a hard-write — so bootstrap-at-fold-boundary is
sufficient. The EMA time constant ~1/alpha provides the decay, satisfying the
decay_bars=500 intent without a new ISV slot (Option B rejected as over-
engineered: new ISV slot + kernel change for no material behavioral improvement).

Changes:
- training_loop.rs::reset_named_state: add dispatch arms for both SoftReset
  entries, writing bootstrap constants (CPU-born input, not adaptive output;
  complies with spec §4.C.6 GPU-drives-CPU-reads)
- trainer/mod.rs: fold-boundary reset loop now calls both fold_reset_entries()
  and soft_reset_entries(); stale "handled separately (Plan 2)" comment removed
- smoke_tests/soft_reset.rs: 4 CPU-only tests verify registry classification,
  entry count, mutual exclusivity from fold_reset, and bootstrap constant invariants
- docs/dqn-wire-up-audit.md: D.5 SoftReset dispatch row added

No CPU-side adaptive computation. No new ISV slot. No behavioral change to
training beyond writing known-good bootstrap values at fold boundaries.

Plan 2 Task 4. Spec §4.D.5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 20:08:36 +02:00
jgrusewski
d071501e44 feat(dqn-v2): D.2 per-branch gamma — GPU kernel + monitor (spec §4.D.2 + §4.C.6)
Replaces scalar GAMMA_EFF_INDEX=43 with 4 per-branch slots at 43-46
(DIR, MAG, ORD, URG). New per_branch_gamma_update_kernel.cu reads
per-branch v-range + health and writes 4 slots. Deletes
gamma_update_kernel.cu per feedback_no_partial_refactor.md.

Per-branch base/max: direction [0.92, 0.99] (trend horizon),
magnitude [0.88, 0.95], order [0.85, 0.93], urgency [0.80, 0.90]
(execution horizon). Kernel diverges each branch's γ from uniform
bootstrap based on its branch's Q-stats.

ISV slots downstream of 43 shift +3: KELLY (44→47), CQL_ALPHA (45→48),
PLAN_THRESHOLD (46→49), Q_P05_* (47→50..54), Q_P95_* (51→54..58),
fingerprint (55-56 → 58-59). ISV_TOTAL_DIM grows 57 → 60.
Layout fingerprint auto-recomputes from updated seed bytes.

c51_loss_kernel reads per-branch γ from ISV[43+d] inside the branch loop;
iql_compute_per_sample_support reads ISV[43+d] per-branch for half_w.
kelly_cap_update_kernel write index updated 44→47.
q_quantile_kernel slot bases updated 47/51 → 50/54.
state_layout.cuh ISV_PLAN_THRESHOLD_IDX updated 46→49.

CPU-side PerBranchGammaMonitor (gamma_monitor.rs) is read-only observer
of all 4 slots; read() returns mean, diagnose() exposes individual γ values
+ spread + health. No CPU-side γ computation (spec §4.C.6 GPU-drives).

Plan 2 Task 3. Spec §4.D.2 + §4.C.6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 20:01:38 +02:00
jgrusewski
345867c599 test(dqn-v2): D.1 Mamba2 backward — grad-check validation (already wired)
Plan 1 A.5 audit found mamba2_scan_projected_bwd kernel + host call at
gpu_dqn_trainer.rs::mamba2_backward are ALREADY fully wired in the
adam_grad CUDA-graph child. Plan 2 Task 2 narrows from "implement" to
"validate".

Two smoke tests confirm correctness:
- mamba2_backward_gradients_propagate: grad Frobenius norm 0.25 after 3
  epochs (>> 1e-8 threshold), ruling out silent no-op like compute_iqr
  had pre-Task-A.6.
- mamba2_backward_grad_check: kernel-level reference check (B=2 K=2
  SH2=4 STATE_D=4); max rel_err d_gate=2.6e-7, d_x_out=2.6e-7,
  d_context=6.7e-8 — all well within 15% threshold (near machine
  epsilon, confirming bit-identical host/GPU results).

No production code change — test-only accessors exposed via #[cfg(test)]
impl blocks on GpuDqnTrainer and DQNTrainer. Audit doc updated.

Plan 2 Task 2. Spec §4.D.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 19:40:52 +02:00
jgrusewski
2cf9927647 feat(dqn-v2): C.1 quantile-based atom support (Plan 2 Task 1)
Replaces half = max(10*q_gap, 3*std_ema, floor) tuned constants with
quantile-based: half = max(|q_p95 - v_center|, |v_center - q_p5|)
clamped to [min_half_floor, abs_half]. Covers observed Q distribution
directly.

New GPU kernel q_quantile_reduce reads per-sample Q-values from q_out_buf,
sorts per branch (bitonic for power-of-2 N, quickselect otherwise), writes
ISV[Q_P05_*=47..51) and ISV[Q_P95_*=51..55). Cold-path per-epoch (4 blocks,
1 thread each). No atomicAdd.

ISV slots 47-54 added at tail (8 total). Fingerprint shifted to 55-56.
ISV_TOTAL_DIM grows 49 -> 57. Fingerprint seed updated; recomputes hash
at compile time (new value: 0xbf6c400c026d77e3).

Launch order: reduce_current_q_stats (populates q_out_buf) ->
q_quantile_reduce (writes ISV P5/P95) -> reduce_current_q_stats_per_branch
-> update_eval_v_range (reads ISV P5/P95 for half-width).

Bootstrap: q_p05 = v_min, q_p95 = v_max (matches current atom range at
cold start). FoldReset: isv_q_quantiles dispatch arm resets to bootstrap
values at fold boundary.

StateResetRegistry: isv_q_quantiles as FoldReset; dispatch arm added in
training_loop.rs::reset_named_state.

Docs: isv-slots.md rows [47..57) updated, fingerprint tail reference
corrected to [55..57). dqn-wire-up-audit.md: q_quantile_kernel.cu added.

Plan 2 Task 1. Spec §4.C.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 19:29:26 +02:00
jgrusewski
79b7d3fc8b plan(dqn-v2): Plan 2 revision — GPU-drives architecture + Mamba2 validation scope
Revises Plan 2 to match the post-Plan-1 reality:

1. AdaptiveController → AdaptiveMonitor throughout (read-only observers
   per spec §4.C.6 2026-04-24 revision; GPU kernels compute, CPU reads).
   Applies to Task 3 (per-branch gamma) and Task 5 (liquid_mod audit).

2. Task 2 (D.1 Mamba2 backward) scope narrowed from "implement" to
   "validate existing". Plan 1 A.5 audit confirmed
   mamba2_scan_projected_bwd kernel + mamba2_backward host call are
   already fully wired. Task 2 now adds a finite-difference grad-check
   smoke + non-zero grad-propagation smoke; escalates if either fails.

3. Task 3 (D.2 per-branch gamma) rewritten for GPU-drives compliance:
   - Replace scalar GAMMA_EFF_INDEX=43 with 4 per-branch slots at
     43-46 (DIR/MAG/ORD/URG).
   - New per_branch_gamma_update_kernel.cu reads v-range + health,
     writes 4 slots. Deletes the Plan 1 gamma_update_kernel.cu.
   - 4 read-only monitors (or one consolidated PerBranchGammaMonitor).
   - ISV slots 44-48 shift downstream; fingerprint re-tails at 50-51;
     ISV_TOTAL_DIM grows 49 → 52.
   - c51_loss_kernel and iql_value_kernel read per-branch γ from ISV.
   - No CPU-side γ computation anywhere.

4. Header architecture block updated: new ISV slot count target,
   GPU-drives principle explicit, current Plan-1 layout table inline
   for reference.

5. Pre-plan verification updated: expects AdaptiveMonitor trait (not
   AdaptiveController); checks for 6 Plan-1 monitor files.

6. Removed "schema version bump 1 → 2" language — layout fingerprint
   auto-recomputes from seed bytes; no integer version space exists.

Task 4 (D.5 soft fold transitions), Task 5 content (D.7 liquid audit),
Task 6 (D.3+D.6+D.8 coordinated state-layout migration), Task 7
(validation run) structure preserved. Internal slot numbers in those
tasks will reconcile to the new layout when they land (no pre-emptive
edits since those indices depend on whether Task 1 quantile slots or
Task 3 per-branch gamma slots land first — re-compute at exec time).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 19:04:56 +02:00
jgrusewski
536eea20bf fix(dqn-v2): local-smoke fallout — StateResetRegistry dispatch arms, controller_activity thresholds, example hp_f32 helper
Local smoke-test run after Plan 1 C.6 completion surfaced three issues:

1. StateResetRegistry missing dispatch arms for the 8 ISV slots
   pre-allocated in ac9bcab94 (isv_epoch_idx, isv_epsilon_eff, isv_tau_eff,
   isv_gamma_eff, isv_kelly_cap_eff, + 3 SchemaContract which already no-op).
   Fold boundary would error "unknown name 'isv_epoch_idx'". Added dispatch
   arms in training_loop.rs::reset_named_state — reset each to 0.0; GPU
   kernels repopulate on next epoch.

2. controller_activity smoke test's single 50% threshold was designed for
   reactive CPU-compute controllers. Under GPU-drives-CPU-reads, tau is a
   Polyak-EMA cosine schedule that fires every epoch by design (95%),
   gamma is health-coupled monotonic (may fire every epoch as health
   drifts). Split threshold per-controller: reactive (anti_lr, grad_clip,
   cql_alpha, cost_anneal) = 0.50; schedule-based (tau, gamma) = 1.00.

3. examples/train_baseline_rl was broken since the f64→f32 ABI refactor
   (d64adc14f) — hp_f64 returning f64 assigned to f32 fields. Added
   hp_f32 helper that narrows JSON-born f64→f32 at ingest boundary. Use
   hp_f32 for f32 fields, hp_f64 for f64 fields (learning_rate,
   entropy_coefficient, weight_decay). No more "as f32" casts at call
   sites. Also fixed replay_buffer_vram_fraction + bars_per_day f64→f32.

Local smoke tests now pass:
- controller_activity: ok (1 passed, 29.5s)
- multi_fold_convergence: ok (1 passed, 3 folds x 20 epochs, 534.9s)
- 24 new monitor + registry unit tests: all passing

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:58:13 +02:00
jgrusewski
0aec55dd70 fix(dqn-v2): atoms_monitor fire_rate test initial-value bug
The test observed `mon.observe(0.0)` first but the monitor's initial
last_value is also 0.0, so the first observation didn't fire. Test
expected 2/3 but got 1/3.

Changed observe sequence to (1.0, 1.0, 1.5) mirroring grad_balancer's
correct pattern: first differs from initial 0.0 → fires, second matches
→ no fire, third differs → fires = 2/3.

No production code change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:32:11 +02:00
jgrusewski
d7f1b9e9ae plan(dqn-v2): Plan 1 substrate + refactor complete (Tasks 9-17)
Appends Plan 1 completion note to the substrate plan doc, recording
all 10 commit SHAs from Tasks 8-17 and the exit criteria status.
Tasks 8-17 fully implemented: AdaptiveMonitor trait, 6 CPU-side read-only
monitors (atoms, gamma, kelly_cap, tau, epsilon, grad_balancer), 5 cold-path
GPU kernels (atoms_update, gamma_update, kelly_cap_update, tau_update,
epsilon_update), 3 static ISV constructor writes (cql_alpha, conviction_floor,
plan_threshold). All adaptive computation is now GPU-driven; CPU never
computes adaptive values (§4.C.6). Tasks 18.1-18.7 (validation run) pending.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:29:34 +02:00
jgrusewski
048c78d5f7 feat(dqn): Plan 1 Task 9 — atoms_update GPU kernel + AtomsMonitor + delete CPU paths
atoms_update_kernel.cu: 4-block kernel (grid=(4,1,1), block=(256,1,1)) replaces
the CPU loop that launched adaptive_atom_positions 4× per branch. Each block reads
ISV v-range slots [23..31) and spacing_raw params[52..56); computes softmax +
cumsum; writes atom_positions_buf[branch * num_atoms]. Same formula as the
existing adaptive_atom_positions kernel in experience_kernels.cu.

Deleted recompute_atom_positions and warm_start_atom_positions from GpuDqnTrainer
and their fused_training.rs delegates. step_atom_positions now calls
launch_atoms_update. training_loop.rs: recompute_atom_positions → launch_atoms_update;
warm_start_atom_positions + compute_reward_quantiles block removed.

AtomsMonitor reads ISV[V_CENTER_DIR_INDEX=23] as representative scalar;
diagnose() exposes all 8 v-range slots [23..31). state_reset_registry.rs
"follow-up task" descriptions updated for all 4 adaptive slots.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:28:19 +02:00
jgrusewski
1e90f74c8f feat(dqn): Plan 1 Task 11 — kelly_cap_update GPU kernel + KellyCapMonitor
kelly_cap_update_kernel.cu: single-thread cold-path kernel aggregates Kelly
win/loss stats across all n_envs from portfolio_states[n_envs, PS_STRIDE].
Computes mean half-Kelly × health-coupled safety multiplier (0.5+0.5*health)
and writes ISV[KELLY_CAP_EFF_INDEX=44]. Matching Bayesian priors from
trade_physics.cuh (prior_wins=2, prior_losses=2) prevent cold-start collapse.

GpuExperienceCollector::portfolio_states_dev_ptr() added — exposes device
pointer for the portfolio_states buffer without a GPU→CPU roundtrip.
GpuDqnTrainer::launch_kelly_cap_update takes ps_dev_ptr + n_envs.
FusedTrainingCtx delegates to trainer. training_loop.rs launches at epoch
boundary after gamma_update (requires both fused_ctx and gpu_experience_collector).

KellyCapMonitor reads ISV[44] for HEALTH_DIAG via AdaptiveMonitor trait.
state_reset_registry.rs description updated; audit docs updated (Invariant 7).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:19:45 +02:00
jgrusewski
c1b6848c21 feat(dqn-v2): C.6 Task 10 — gamma GPU kernel + CPU monitor (discount factor)
gamma_update kernel computes health-coupled gamma from ISV[LEARNING_HEALTH=12]:
gamma_eff = gamma_min + (gamma_base - gamma_min) * health. Single-thread
cold-path kernel writes ISV[GAMMA_EFF_INDEX=43].

GammaMonitor is a read-only observer exposing gamma_eff, health, fire_rate.

Consumer migration: fill_gamma_buf and IQL gamma computation now read
ISV[GAMMA_EFF_INDEX] via read_isv_signal_at (pinned, zero-copy). Training
loop passes hyperparams.gamma as gamma_base to the kernel.

Deleted: apply_adaptive_gamma method (GpuDqnTrainer + FusedTrainingCtx
delegates), set_adaptive_gamma method, adaptive_gamma field (GpuDqnTrainer
+ DQNTrainer), last_gamma_eff cached field + last_gamma_eff() delegate.
StateResetRegistry entry for adaptive_gamma removed (field gone).

Smoke test generalization.rs updated to check config gamma_base instead
of deleted adaptive_gamma field.

Tests: 3 monitor unit tests pass. cargo check -p ml at 8-warning baseline.

Plan 1 Task 10. Spec §4.C.6 (2026-04-24 revision).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 18:05:44 +02:00
jgrusewski
d346e03d48 feat(dqn-v2): C.6 Task 14 — epsilon GPU kernel + CPU monitor (exploration)
epsilon_update kernel computes effective epsilon from ISV[EPOCH_IDX=39,
TOTAL_EPOCHS=40, LEARNING_HEALTH=12] with cosine schedule (eps_start ->
eps_end) plus health-coupled boost (up to +0.1 at collapse). Single-thread
cold-path kernel writes ISV[EPSILON_EFF_INDEX=41].

EpsilonMonitor is a read-only observer exposing eps_eff, epoch_idx,
total_ep, health, progress, fire_rate in DiagSnapshot.

Wires epsilon kernel launch at epoch boundary alongside tau kernel. Consumer
migration: log_training_config ISV-adaptive epsilon block now reads
ISV[EPSILON_EFF_INDEX] instead of computing `base_floor * (0.5 + volatility)`.

Tests: 3 monitor unit tests pass. cargo check -p ml at 8-warning baseline.

Plan 1 Task 14. Spec §4.C.6 (2026-04-24 revision).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 17:53:30 +02:00