Second half of Phase B4b. Wires the trade-outcome label column through
the replay buffer (struct field, allocation, scatter on insert, gather
on sample, direct-to-trainer pointer + setter) and connects the
trainer's aux_to_label_buf to receive per-batch sampled labels.
Completes the end-to-end producer → ring → trainer i32 path the K=3
sparse CE consumer reads.
Replay buffer changes (crates/ml-dqn/src/gpu_replay_buffer.rs):
- New struct fields: aux_outcome_labels (capacity-sized ring),
sample_aux_outcome_labels (mbs-sized fallback gather), trainer_aux_
outcome_labels_ptr (direct-path destination)
- New aux_outcome_labels_ptr field on GpuBatchPtrs
- insert_batch signature: new aux_outcome arg between aux_conf_in and
bs. Scatters via existing K-generic scatter_insert_i32 (same kernel
the K=2 aux_sign_labels uses).
- sample_proportional direct gather when trainer_aux_outcome_labels_ptr
!= 0; fallback gather otherwise. Mirrors aux_sign_labels direct/
fallback semantic exactly.
- New setter set_trainer_aux_outcome_labels_ptr mirrors
set_trainer_aux_conf_ptr.
Collector emission:
- New aux_outcome_labels field on GpuExperienceBatch
- Populated at end of collect_experiences_gpu via dtod_clone_i32 from
Phase B4b-1's per-(env, t) producer scratch.
Trainer wireup:
- aux_to_label_buf_ptr() accessor on GpuDqnTrainer (mirrors
aux_nb_label_buf_ptr)
- trainer_aux_to_label_buf_ptr() delegating accessor on
FusedTrainingCtx
- New set_trainer_aux_outcome_labels_ptr call in training_loop at the
same site where set_trainer_buffers + set_trainer_aux_conf_ptr fire.
Two call sites updated (lines ~835 + ~2857).
- insert_batch call in training_loop passes &gpu_batch.aux_outcome
_labels as new arg.
Test fixtures updated: 4 smoke test files + 3 unit-test fixtures in
gpu_replay_buffer.rs alloc zero-init aux_outcome i32 arg.
End-to-end chain complete:
trade_outcome_label_kernel (A2)
→ collector per-step launch (B3)
→ collector emission (B4b-1)
→ replay-buffer insert + scatter (B4b-2)
→ PER sample + direct gather (B4b-2)
→ trainer aux_heads_forward.loss_reduce (B4) reads sparse {-1,0,1,2}
→ trainer aux_heads_backward (B4) computes per-sample partials
→ Adam SAXPY (B1+B4) updates W1, b1, W2, b2 at [163..167)
The "degraded predict-Profit-everywhere" cold-start from Phase B4 is
resolved. K=3 head trains on real sparse trade-outcome labels.
Verification:
- cargo check -p ml clean (21 warnings, none new).
- cargo test -p ml --lib → 1016/0 on clean runs; pre-existing
NoisyLinear flake still surfaces ~30-50% of runs (unrelated to
vNext work — see ebc1b1502 / 20e1aea27 commit notes).
Remaining vNext work:
- B5: input concat 256 → 262 with plan_params
- Phase C: 3-slot state assembly (state[121..124])
- Phase D: 12-weight W atom-shift (4 actions × 3 outcomes)
- Phase E: dW backward + Adam for W[4, 3]
- Phase F: validation smoke at β=0.5 structural prior
Audit: docs/dqn-wire-up-audit.md Phase B4b-2 section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Threads `self.aux_conf_at_state_buf` into the `c51_loss_batched` launch
in `GpuDqnTrainer::launch_c51_loss`. Position matches the kernel's
appended trailing arg from the previous commit.
Tests added in `crates/ml-dqn/src/gpu_replay_buffer.rs::tests`:
- `aux_gate_high_confidence_passes_full_target` (CPU pure-math):
gate(aux_conf=0.5, threshold=0.10, temp=0.05) > 0.99 proves
high-confidence reward pass-through.
- `aux_gate_low_confidence_attenuates_reward` (CPU pure-math):
gate(aux_conf=0.02, threshold=0.10, temp=0.05) < 0.20 proves
the uncertain-state neutralizer semantic.
- `aux_gate_temp_floor_keeps_gate_finite` (CPU pure-math):
sweeps {temp, aux_conf, threshold} and asserts finite gate ∈ [0,1]
across the ISV-controllable parameter range — proves the
fmaxf(temp, 1e-3) floor keeps the kernel numerically safe.
- `aux_conf_direct_to_trainer_gather_populates_destination` (GPU
behavioral): wires a fresh CudaSlice<f32> as the trainer
destination, inserts 8 transitions with strictly-positive distinct
aux_conf values, samples 1, asserts the trainer destination
buffer post-sample holds a value from the inserted set (NOT the
alloc_zeros sentinel) — proves the direct-gather wiring actually
populates the trainer buffer with non-trivial data.
All 3 CPU math tests + 1 GPU integration test pass on RTX 3050.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Class B audit, per_update_pa was applying alpha twice:
- priorities[idx] = |td|^α + ε ← α applied once
- priorities_pa[idx] = priorities^α ← α applied AGAIN, gives |td|^(α²)
Sum-tree sampler reads priorities_pa (verified at gpu_replay_buffer.rs:778
for per_prefix_scan + line 799 for per_sample), so effective sampling
exponent was α² = 0.36 instead of α = 0.6 (default). High-TD events were
sampled only ~2.5× more than low-TD events instead of ~10× — PER's
prioritization power was attenuated by ~4×, washing out the signal from
sparse trade-close events (3% of buffer).
Fix (Option B — preserves variable semantics): the contract documented at
gpu_replay_buffer.rs:1285 is `priorities_pa[i] = priorities[i]^alpha`.
Store raw |td|+ε in `priorities` (one source of truth) and compute α
once for `priorities_pa`. Same pattern applied to pow_alpha_diverse_f32
in replay_buffer_kernels.cu (the health<0.8 fallback path) which had the
identical double-α bug.
per_insert_pa NOT changed — it was already correct (priorities=effective,
priorities_pa=effective^α, no double application).
priority_update_f32 NOT changed — single-buffer kernel, verified UNUSED
(no Rust caller); harmless idle code, deletion deferred.
Behavioral test: gpu_replay_buffer::tests::
test_per_priority_single_exponent_no_double_alpha (GPU-gated #[ignore])
inserts 16 slots, fires update_priorities_gpu with TD errors spanning
[0.001, 100.0], asserts priorities_pa ratio matches (100/0.001)^0.6 ≈
1995× within 5%. Pre-fix would yield (100/0.001)^(0.6²) ≈ 100× — the
20× miss makes regression instant-detect.
Cumulative WR-plateau fix series (commit 11):
- Class C bug 1 + P0-B (8f218cab2)
- P0-C MIN_HOLD_TARGET (316db416b)
- P0-A REWARD_POS/NEG_CAP (394de7d43)
- P1 var_floor (c4b6d6ef2)
- P0-A-downstream (657972a4b)
- P1-Producer adaptive Kelly priors
- Class A audit batches 4-A / 4-B / 4-B fixup (9fb980da2)
- Class B #1 fold-boundary PER buffer clear (658fec493)
- Class B #2 (this commit)
Verification:
- cargo check -p ml-dqn -p ml --tests --all-targets — clean
- cargo test -p ml-dqn --release --lib gpu_replay_buffer::tests::
-- --ignored — 3/3 pass
- cargo test -p ml --test sp14_oracle_tests --release -- --ignored
— 24/24 pass
- cargo test -p ml --test sp15_phase1_oracle_tests --release
-- --ignored — 36/36 pass (incl. per_sampler_* oracles)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Class B audit, the PER ring buffer was never cleared from the
single source of truth at `DQN::reset_for_fold`. fold N+1 was
training on a 50/50 mix of stale fold N-1 experiences + fresh
fold N — averaging behavior locked learning to a "policy-agnostic"
middle ground producing 46-48% WR across SP3-SP15.
Fix: extend `DQN::reset_for_fold` to call `clear_replay_buffer()`
internally. Single source of truth — every fold reset clears the
buffer atomically. Removed the now-redundant explicit
`self.clear_replay_buffer().await?` call at the top of
`DQNTrainer::reset_for_fold` per `feedback_no_legacy_aliases`.
Tradeoff: loses cross-fold experience transfer (fold N+1 starts
from scratch on fold N data only). User-approved Option (a) per
Class B audit 2026-05-08.
Signature changes (atomic per `feedback_no_partial_refactor`):
- `DQN::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNAgentType::reset_for_fold`: `()` → `Result<(), MLError>`
- `DQNTrainer::reset_for_fold` (async): unchanged signature; the
agent.reset_for_fold call now propagates the Result.
`GpuReplayBuffer::clear` extended: it already zeroed
`priorities_pa` + `prefix_sum` but left `priorities` unzeroed.
Sampler reads `priorities_pa` (not `priorities`) so this was not
a sampling-contamination vector, but is leftover state that
`priorities_slice()` consumers would inherit. Fixed for
completeness alongside the fold-boundary-clear lift.
Behavioral test added: `test_clear_purges_priorities_and_blocks_sampling`
inserts 64 transitions, samples once to populate prefix_sum +
sample buffers, steps the β counter, then clears and asserts:
- len() == 0, is_empty() == true, can_sample(1) == false
- write_cursor() == 0, current_step == 0
- priorities[..n] all zero (DtoH readback)
- priorities_pa[..n] all zero (DtoH readback)
No mocks; exercises actual GPU memset_zeros + DtoH path.
GPU-gated `#[ignore]` — runs at deploy time.
Cumulative WR-plateau fix series (commit 10):
- Class C bug 1 + P0-B (8f218cab2)
- P0-C, P0-A, P1, P0-A-downstream, P1-Producer, 4-A, 4-B, 4-B fixup
- Class B #1 (this commit)
Verification:
- cargo check -p ml-dqn -p ml --tests --all-targets: clean
- cargo test -p ml-dqn --release --lib: 266 pass / 16 pre-existing
GPU-config failures unchanged (verified via stash-baseline)
- cargo test -p ml --test sp14_oracle_tests --test
sp15_phase1_oracle_tests --release: 5 pass / 36 ignored (GPU)
Audit doc entry appended at docs/dqn-wire-up-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1.3.b-followup (5b394f103) landed the main per-env redesign but
deferred dd_trajectory + per_insert_pa migration as Followup-B because
the contract shape is per-(env, t) buffer [N*L], not per-env tile.
This commit:
- Reshapes dd_trajectory_decreasing_kernel to per-env grid
[n_envs, 1, 1] x [1, 1, 1]; writes dd_trajectory_per_env[env]
- Migrates per_insert_pa to per-transition lookup via
env_id = (j % (n_envs * lookback)) / lookback; reads
dd_trajectory_per_env[env_id]
- Allocates two collector-owned per-env tiles
(sp15_dd_trajectory_per_env + sp15_dd_trajectory_prev_dd_per_env);
deletes the trainer-owned [1] sp15_dd_trajectory_prev_dd scratch +
its setter wiring (replaced by unconditional collector ownership,
mirrors sp15_dd_state_per_env pattern)
- Extends dd_state_reduce_kernel to mean-aggregate per-env
trajectory tile -> ISV[DD_TRAJECTORY_DECREASING_INDEX=439] for
HEALTH_DIAG diagnostic preservation
- Wires per-env tile dev_ptr + (n_envs, lookback) dims into
GpuReplayBuffer via new setters (set_sp15_dd_trajectory_per_env_ptr,
set_sp15_per_env_dims) called from training_loop
- Migrates 4 dd_trajectory + 1 PER oracle tests to per-env contract
- Adds NEW behavioral test
per_sampler_weights_per_transition_not_uniform_across_batch:
n_envs=2 batch with env-0 trajectory=1, env-1 trajectory=0; asserts
priorities[env-0 slots]=3.0 and priorities[env-1 slots]=1.0 in the
SAME insert batch (Wave 4.3 would produce uniform 3.0 OR uniform
1.0 across all 8 priorities — the test directly fails the old
implementation)
- Layout fingerprint marker DD_TRAJECTORY_PER_ENV=sp15_phase_1_3_b_followup_B
(greenfield checkpoints OK per spec Q1)
Fixes the Wave 4.3 PER limitation: ISV[DD_TRAJECTORY_DECREASING_INDEX=
439] was read ONCE per insert call and applied uniformly to ALL
n_envs * lookback * 2 transitions in the batch, so the recovery
oversample was statistically biased (whichever env wrote ISV[439]
most recently determined the boost for ALL inserted transitions).
Per-transition lookup fixes this — each transition's weight reflects
its own env's recovery context.
Atomic per feedback_no_partial_refactor: kernel reshape + 2 collector-
owned per-env tiles + trainer struct cleanup + reduction kernel
extension + per_insert_pa kernel signature change + 3 new GPU PER
replay buffer fields/setters/accessors + per_insert_pa launch site
update + collector launch sequence update + state-reset registry
rename (1->2 entries) + 2 dispatch arms + training_loop wiring
delete/replace + 4 dd_trajectory test migrations + 1 PER test
migration + 1 NEW behavioral test + 2 dd_state_reduce callsite
null updates + audit doc all in this commit.
Closes the per-env DD redesign chain end-to-end. SP15 reward shaping
+ recovery-curriculum PER oversample now correctly apply per-env
context everywhere they're consumed; no more "whichever env wrote
last wins" paths.
Verified: all dd_state, dd_trajectory, per_sampler, final_reward,
plasticity oracle tests green; ml lib HOLDS the Phase 1.3.b-followup
baseline (947 pass / 12 fail) on RTX 3050 Ti — no new regressions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3.5.5 (69b8fdb61) landed dd_trajectory_kernel writing
ISV[DD_TRAJECTORY_DECREASING_INDEX=439] per-step but DEFERRED the PER
sampler integration AND the production launch. Wave 4.3 wires both
atomically per feedback_wire_everything_up.
Investigation finding — Case B (existing TD-error-driven priority
sampler): the GPU PER architecture in
crates/ml-dqn/src/gpu_replay_buffer.rs already weights replay by
per-transition priority (raw priority + priorities_pa = priority^alpha
feeding the prefix-sum sampler). Recovery boost slots in cleanly as a
multiplier on priorities[idx] at insert time — no architectural
refactor, no new sampling-path machinery.
Recovery-oversample formula:
effective = base_priority × (1.0 + ISV[440] × ISV[439])
priorities[idx] = effective
priorities_pa[idx] = effective^alpha
When a transition is in a recovery (DD shrinking from non-trivial DD,
ISV[439]=1.0), it lands in the buffer at 1 + ω(2.0) × δ(1.0) = 3.0×
the baseline max_priority, then sampled 3.0^0.6 ≈ 1.93× more often
than baseline (alpha=0.6 PER convention) until per_update_pa
overwrites priority based on TD-error and normal PER takes over.
Recovery transitions get amplified gradient signal — the agent learns
recovery dynamics over typical "average-DD" Bellman noise.
Atomic landings (per feedback_no_partial_refactor):
* crates/ml-dqn/src/per_kernels.cu — per_insert_pa kernel signature
extended (+priorities, +isv pointers); kernel writes both columns
of the (priority, priority^alpha) row pair; nullable ISV ptr falls
back to boost_factor=1.0
* crates/ml-dqn/src/gpu_replay_buffer.rs — new isv_signals_dev_ptr
field + setter + accessor + priorities_pa_slice accessor; redundant
pre-3.5.5.b scatter_insert_f32 broadcast of max_priority REMOVED
(now subsumed by per_insert_pa's priorities[idx]=effective store)
* crates/ml/src/cuda_pipeline/gpu_experience_collector.rs — new
sp15_dd_trajectory_prev_dd_dev_ptr field + setter; per-step
launch_sp15_dd_trajectory_decreasing invocation gated on both
ISV ptr + prev_dd ptr being non-zero, placed immediately after
launch_sp15_dd_state in the env-step loop
* crates/ml/src/trainers/dqn/trainer/training_loop.rs — two new
wiring blocks for the collector's prev_dd ptr and the replay
buffer's ISV ptr, mirroring the existing
set_isv_signals_ptr / set_sp15_alpha_warm_count_ptr plumbing
* crates/ml/tests/sp15_phase1_oracle_tests.rs — new oracle test
per_sampler_weights_recovery_transitions_higher verifying both
columns of the priority-buffer write equal exactly the
boosted/baseline values (3.0 vs 1.0; 3.0^0.6 vs 1.0^0.6) and the
boosted/baseline ratio = 3.0 within 1e-5 — the recovery oversample
factor by construction
Phase 3.5.5 deferred consumer eliminated per
feedback_wire_everything_up. This closes the SP15 Phase 3.5
recovery-dynamics chain (3.5.2 + 3.5.3 + 3.5.4 + 3.5.4.b + 3.5.5 +
3.5.5.b all wired). DD_TRAJECTORY_FLOOR (slot 441) and ISV-driven
RECOVERY_OVERSAMPLE_WEIGHT (slot 440) producers remain documented
follow-ups per feedback_isv_for_adaptive_bounds — kernel reads from
slots rather than literals so consumer migration is a no-op when the
producers land.
Verified on RTX 3050 Ti (CUDA 12.9, sm_86):
* cargo check -p ml-dqn / -p ml --features cuda clean
* 3 of 3 dd_trajectory oracle tests still green
* 1 of 1 new per_sampler oracle test green (3.0 vs 1.0 priority
bias, ratio = 3.000... within 1e-5)
* 8 of 8 (1 ignored) gpu_residency replay-buffer + adamw smoke
tests green
* 4 of 4 PER smoke tests green
* Full ml lib suite IMPROVES Wave 4.2 baseline: 947 pass / 12 fail
(was 946 pass / 13 fail; the previously-failing
test_dqn_checkpoint_round_trip now passes — likely the redundant
pre-3.5.5.b scatter_insert_f32 was racing with the immediate
per_insert_pa overwrite under particular timing conditions; the
merged single-kernel write closes that race)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final piece of the SP13 Layer B chain. B1.1a flipped the aux head from K=1
MSE regression to K=2 softmax CE classification but aux_nb_label_buf was
zero-init — model was training on "all bars are class 0 (down)". B1.1b
lands the producer kernel that fills i32 -1/0/1 labels from the 30-bar
price trajectory, the replay direct-path 8th gather that carries those
labels into the trainer, and the experience collector hoist that ensures
bar_indices_pinned is always populated (producer + hindsight relabel both
consume it). Aux head finally trains on real classification signal.
Recovery commit: this completes a B1.1b agent dispatch that crashed
mid-edit. The implementer had landed ~95% of the cascade (kernel file,
build.rs, replay buffer signature + direct path, fused_training getter,
trainer accessor, both training_loop.rs callers, kernel field + cubin
loader on the experience collector) before being killed. The missing
pieces (experience collector launch + bar_indices_pinned hoist + 6
producer tests + audit doc) were completed manually post-crash and
verified end-to-end.
Three contracts (atomic single commit per feedback_no_partial_refactor):
1. NEW aux_sign_label_kernel.cu producer — pure per-thread O(1) map
reading targets[bar*6+2] (raw_close column) at bar and bar+lookahead,
writing -1 (skip if bar+lookahead >= total_bars), 0 (down/flat under
strict greater-than tie-break), or 1 (up). Replaces B0 alloc_zeros.
2. Replay direct-path 8th gather — set_trainer_buffers gains 8th arg
trainer_aux_sign_labels_ptr; direct branch in sample_proportional
adds gather_i32_scalar into the trainer ptr; fallback gather wrapped
in if !direct_to_trainer (avoids wasted DtoD). Direct-mode
GpuBatchPtrs return points aux_sign_labels_ptr at trainer ptr.
3. Experience collector bar_indices_pinned cpu-fill hoist — moved out
of if hindsight_fraction > 0.0 so producer + hindsight share it.
Files (9 total):
- crates/ml/src/cuda_pipeline/aux_sign_label_kernel.cu (NEW)
- crates/ml/build.rs (cubin registration)
- crates/ml/src/cuda_pipeline/gpu_experience_collector.rs (kernel
field + cubin loader + struct init + hoist + producer launch)
- crates/ml-dqn/src/gpu_replay_buffer.rs (8th arg + direct gather +
fallback skip + GpuBatchPtrs return)
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs (aux_nb_label_buf_ptr
accessor mirrors 6 existing trainer-buf accessors)
- crates/ml/src/trainers/dqn/fused_training.rs
(trainer_aux_sign_labels_buf_ptr getter)
- crates/ml/src/trainers/dqn/trainer/training_loop.rs (both
set_trainer_buffers callers updated)
- crates/ml/tests/sp13_layer_b_oracle_tests.rs (6 NEW producer tests)
- docs/dqn-wire-up-audit.md (B1.1b section)
Hard rules upheld:
- feedback_no_partial_refactor: every consumer of the 3 contracts
migrates atomically
- feedback_no_atomicadd: producer is pure map; no reductions
- feedback_cpu_is_read_only: producer GPU-only; only host work is
pre-existing bar_indices_pinned cpu-fill (hoisted unchanged)
- feedback_no_stubs: kernel output flows through real chain — ring
buffer → direct gather → aux_nb_label_buf → CE consumer
- feedback_no_legacy_aliases: 8-arg setter gets
#[allow(clippy::too_many_arguments)] not an alias shim
- feedback_no_htod_htoh_only_mapped_pinned: targets_buf and
bar_indices_pinned both pre-existing mapped-pinned
Build + test:
- cargo check --workspace clean (only pre-existing warnings)
- cargo check --workspace --tests clean
- 17 tests in sp13_layer_b_oracle_tests.rs:
- 2 CPU-only (fingerprint bump + HEALTH_DIAG snap stable) pass
- 15 GPU on RTX 3050 Ti pass (9 B1.1a + 6 new B1.1b producer):
aux_sign_label_monotone_up_all_ones
aux_sign_label_monotone_down_all_zeros
aux_sign_label_flat_all_zeros_strict_gt
aux_sign_label_last_30_bars_skip
aux_sign_label_boundary_first_valid_last_skip
aux_sign_label_multi_episode_per_episode_skip
Producer tests cover every edge case in the kernel:
- Monotone trajectories (label=1 / label=0 across all valid bars)
- Flat tie-break (strict greater-than means flat → 0)
- Skip sentinel for last lookahead bars
- First/last bar boundary (bar=0 valid, bar=L-1 skip)
- Multi-episode global skip semantics
Next: Smoke A — L40S 5-epoch validation of full SP13 stack
(P0a + P0b + B0 + B0.1 + B1.0 + B1.1a + B1.1b). Expected: aux head
trains on real K=2 softmax CE labels; aux_dir_acc_short_ema rises above
0.5 within first epoch (vs B1.1a degraded baseline at 0.5);
HEALTH_DIAG aux_b1_diag emits per-epoch with n_down/n_up/n_skip/mask_frac.
If aux_dir_acc_short_ema > 0.55 by epoch 5, B1.1b is validated and the
chain merges to main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure plumbing: threads a new i32 column (aux_sign_labels) through every
layer of the replay path so B1 can wire the aux head's CrossEntropy
classification target without touching any aggregator or batch-shape
contract on its own. No consumer reads the column yet — all labels are
zero-initialized; smoke between B0 and B1 should be bit-identical to
parent 0ad5b6fa4 modulo allocator entropy.
Why split B0 from B1: original Layer B Commit 1 bundled replay plumbing
with the aux head contract change (1->2 dim, MSE->CE, slot 117 retire,
fingerprint bump). Four implementer subagents in a row hit
NEEDS_CONTEXT on brief-accuracy issues — too large for a single
dispatch. Split keeps B0 mechanical and isolates B1 for fresh
implementer with audit-derived brief.
Wiring (data flow):
1. scatter_insert_i32 kernel mirrors scatter_insert_u32 but preserves
-1 sentinels (u32 would alias to 4294967295). gather_i32_scalar at
#18c is symmetric.
2. ReplayKernels.scatter_insert_i32 field + ld() loader.
3. GpuReplayBuffer.aux_sign_labels: CudaSlice<i32> ring [capacity] +
sample_aux_sign_labels: CudaSlice<i32> per-batch gather dst.
4. insert_batch signature gains aux_sign: &CudaSlice<i32>. Body
scatter-launches alongside actions/rewards/dones using same
(ci, cpi, bsi) tuple.
5. sample_proportional Step 3c launches gather_i32_scalar from ring
into sample buffer. Both GpuBatchPtrs return paths set
aux_sign_labels_ptr to sample buffer raw_ptr.
6. GpuBatchPtrs.aux_sign_labels_ptr: u64 publicly exposed.
7. GpuExperienceBatch.aux_sign_labels field — collect_experiences_gpu
alloc_zeros total-sized i32 slice (B1 replaces with producer kernel).
8. Single production caller training_loop.rs:2032 threads through.
Audit-verified call site cardinality (per implementer 4 finding):
- 1 production insert_batch caller (training_loop.rs:2032)
- 1 in-file unit test caller (signature updated)
- 2 GpuBatchPtrs construction sites (both inside sample_proportional)
- 1 GpuExperienceBatch construction site (collect_experiences_gpu)
Implementer 4 caught the over-counted 4 sites claim from the original
brief — the 2 extras were doc-comment references to
insert_batch_with_episode_ids (no separate function exists).
Hard rules upheld:
- feedback_no_partial_refactor: every insert_batch consumer migrates
atomically (1 prod + 1 test, both updated)
- feedback_no_stubs: column carries real data through real kernels;
zero values are valid i32 that B1 overwrites with -1/0/1
- feedback_isv_for_adaptive_bounds: no new ISV slots; slot 117
AUX_LABEL_SCALE_EMA retirement deferred to B1 atomic
- feedback_no_atomicadd: no new producers/reductions
- feedback_cpu_is_read_only: alloc_zeros + scatter + gather all GPU
Build: cargo check --workspace clean in 21s.
Audit: docs/dqn-wire-up-audit.md SP13 Layer B B0 section added with
wiring diagram, call site cardinality table, and B1 next-steps for
fresh-implementer dispatch.
Files: 92 LOC net across 4 source files + audit doc:
- crates/ml-dqn/src/replay_buffer_kernels.cu (+21)
- crates/ml-dqn/src/gpu_replay_buffer.rs (+54)
- crates/ml/src/cuda_pipeline/gpu_experience_collector.rs (+17)
- crates/ml/src/trainers/dqn/trainer/training_loop.rs (+1)
- docs/dqn-wire-up-audit.md (B0 section)
Next: B1 — aux head 1->2 dim, MSE->CE loss, aux_dir_acc reads softmax,
aux_pred_to_isv_tanh logit-diff rewrite, slot 117 retirement,
dqn_param_layout fingerprint bump, kernel populates aux_sign_labels
with real -1/0/1 labels, aux_b1_diag HEALTH_DIAG, 17+ unit tests.
B1 dispatched to fresh implementer with this audit doc as truth-source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
W1 SEMANTIC: Pearl 2 iqn_branch[b] is now consumed at Pearl 5 per-pass
IQN budget. Previous code used iqn_trunk/4 in both parallel + sequential
IQN paths, silently averaging Pearl 2's per-branch IQN differentiation.
Now: iqn_budget_per_branch = iqn_branch[branch_idx] / 4.0 inside the
per-branch loop — sum across 4 branches = iqn_trunk magnitude preserved,
per-branch differentiation preserved. iqn_trunk renamed _iqn_trunk in
the destructure (still in-scope comments only).
W2 DELETE: ATOM_NUM_ATOMS_BASE + NOISY_SIGMA_BASE imports removed from
fused_training.rs:44 — consumers are atoms_update_kernel.cu (Layer B)
and experience_kernels.cu; imports were speculative additions.
W3 DELETE: v_blocks at gpu_iql_trainer.rs:756 — both kernel launches
used v_blocks2; v_blocks was stale dead code.
W4 DELETE: OrderRouter import from ml-dqn/src/dqn.rs — import only,
never used in the file body.
W5 DELETE: get_snapshots_for_timestamp import from data_loading.rs —
imported but never called; only OFICalculator + get_trades_for_bar used.
W6 DELETE: update_target_networks method (42 lines) from ml-dqn/dqn.rs —
orphan with zero call sites; real path uses fused CUDA EMA kernel.
Cascaded: convergence_half_life import deleted (now unused).
W7-W13 FILE-LEVEL #![allow(unsafe_code)]: cublas_algo_deterministic.rs
and sp4_wiener_ema.rs — workspace unsafe_code = "warn" fires for every
unsafe impl/fn; both files require unsafe for CUDA/cuBLAS interop;
pattern matches mapped_pinned.rs, gpu_iqn_head.rs, gpu_weights.rs, etc.
W14-W15 VISIBILITY SCOPE: ControllerPrevValues + ControllerFireCounts
in trainer/mod.rs changed pub → pub(crate); consumed only within ml.
Per feedback_no_hiding: zero #[allow(...)] item-level suppressions added;
2 file-level #![allow(unsafe_code)] follow established crate convention.
cargo check (ml + ml-dqn) + cargo build --release + cargo test --lib
all CLEAN (0 warnings; 13/13 lib tests pass).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Smoke smoke-test-vh9bj revealed: after F+H Q-drift kill fix, fold 0
trains successfully (5 epochs, Best Sharpe 36.03), but fold 1 fails
at epoch 1 with "Gradient collapse detected for 5 consecutive epochs"
despite epoch 1 grad_norm=296,417 (healthy). Root cause: the per-step
gradient_collapse_counter on DQN was already near patience (5) from
fold 0's late-epoch near-zero-grad steps, and DQNTrainer::reset_for_fold
didn't clear it. Plus, training_steps accumulating across folds made
the `past_warmup` gate always-true in fold 1+, removing the warmup
grace period for data distribution shifts.
Add DQN::reset_for_fold zeroing both. Wire from DQNTrainer::reset_for_fold
alongside the A.1 prev_epoch_q_mean reset. Pure additive — same
fold-boundary state-reset gap pattern as A.1, A.2, F.
Predicted impact: fold 1+ now starts with counter=0 and warmup window
restored; gradient collapse check has its full per-fold grace period.
Atomic cleanup per feedback_no_partial_refactor.md:
- Delete crates/ml-dqn/src/regime_conditional.rs and all
RegimeConditional* exports from lib.rs. regime_classifier.rs
(RegimeType, RegimeClassConfig) is kept — used by validation layer
and ml-regime-detection crate.
- Rewrite DQNAgentType to wrap DQN directly (no RegimeConditionalDQN
field, no 3-head delegation gymnastics).
- DQNAgentType::get_count_bonuses_branched no longer returns hardcoded
None — it delegates to DQN::get_count_bonuses_branched() and UCB
count bonuses reach the GPU action selector for the first time
(ghost feature from Phase 0 deferral). action.rs caller simplified
to direct destructure of fixed arrays, no Option matching.
- Drop 4 regime-threshold fields from DQNConfig (regime_adx_idx,
regime_cusum_idx, regime_adx_threshold, regime_cusum_threshold) +
matching Default and aggressive() builder entries.
- serialize_model rewritten to serialize single DQN branching network
directly (no trending__/ranging__/volatile__ prefix namespace).
- curriculum.rs import fixed: crate::dqn::regime_conditional::RegimeType
→ crate::dqn::RegimeType (re-exported from regime_classifier).
- Constructor drops RegimeConditionalDQN::new_on_device, calls
DQN::new_on_device directly.
- Stale doc comments in dqn.rs, moe.rs, regime_classifier.rs updated.
- Wire-up audit updated.
Phase 3 MoE (commit a52d99613) provides the regime-conditioned behavior
the legacy 3-head architecture pretended to do — gate sees ADX (40)
and CUSUM (41) as part of the full 128-dim state vector and learns its
own decomposition, strictly subsuming the threshold classifier.
Smoke: 3/3 folds passed (405s), gate util=0.119,0.119,0.169,... preserved,
all fold checkpoints written. Workspace: 0 errors across all crates,
tests, and examples.
Spec: docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md §7.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Atomic removal of 5 boolean feature flags from DQNConfig per
`feedback_no_feature_flags.md`, all of which gated dead or redundant code.
(use_iqn already removed in da632446c.)
- `use_dueling`, `use_distributional`, `use_noisy_nets`, `use_branching`:
pure-cosmetic always-on flags. Only metadata strings remained.
4 `if true /* always on */` blocks in dqn.rs collapsed to live arms;
dead else arms (legacy non-branching code paths, 5-flat ExposureLevel)
deleted.
- `use_count_bonus`: redundant with `count_bonus_coefficient` (the live
numeric kill-switch). Recording made unconditional; coefficient is the
single dial.
- `use_cvar_action_selection`: dead post-use_iqn cleanup (only consumers
were inside the deleted IQN arms). cvar_alpha retained for cuda_pipeline
CVaR loss usage.
count_bonus.rs API tightened: `bonuses_branched_f32(&self,
exp_out: &mut [f32], ord_out: &mut [f32], urg_out: &mut [f32])` write-into
API alongside the existing Vec<f64> form (kept for tests). Production
DQN::get_count_bonuses_branched() returns fixed `([f32; 4], [f32; 3],
[f32; 3])` arrays — allocation-free, f32 throughout, fed directly to GPU
action selector via stack-allocated buffers.
Test 0.F outputs bit-identical vs pre-cleanup (sigma_C51, argmax,
Thompson counts) — confirms legacy use_* arms were unreachable as
expected.
`docs/dqn-wire-up-audit.md` updated per Invariant 7.
Precondition for the MoE regime redesign per
`docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`use_iqn` is exactly the `use_/enable_` boolean banned by
`feedback_no_feature_flags.md`. It gated dead code: production training
runs IQN unconditionally through `cuda_pipeline/gpu_iqn_head.rs` +
`iqn_dual_head_kernel`, properly wired into the branching architecture
with the `FIXED_TAUS` 5-quantile schedule. Nothing in the cuda_pipeline
production path ever read `use_iqn` or `iqn_network`.
The legacy `iqn_network: Option<QuantileNetwork>` field on `DQN` was a
parallel CPU-side network from a pre-branching era, structurally
unreachable in production: every consumer was gated behind the
`if true /* use_branching: always on */` arm at `q_values_for_batch`,
so the IQN else-if at L1822 was dead code. Training optimised
`iqn_network` parameters in isolation; inference never read them. That's
the train/inference mismatch the L283 comment ("IQN trains base
q_network but inference uses dist_dueling network (zero gradients)")
was working around by **disabling** the feature instead of fixing the
inference path. Per `feedback_no_quickfixes.md` + `feedback_no_hiding.md`
the fix is to remove the dead path entirely.
Strip:
- `DQNConfig::use_iqn` field + parses + checkpoint hash + metadata.
`dqn.use_iqn` / `dqn.iqn_embedding_dim` / `dqn.iqn_num_quantiles` /
`dqn.iqn_kappa` from older checkpoints are silently dropped on load
(same pattern used for `use_dueling`). `iqn_lambda` stays — the
cuda_pipeline dual head consumes it as the IQN aux loss weight.
- 3 vestigial config fields (`iqn_num_quantiles`, `iqn_kappa`,
`iqn_embedding_dim`) — never read in production; kernel-side macros
(`IQN_NUM_QUANTILES = 5`, embed_dim 64, kappa 1.0) are the actual
config.
- 4 default builders (`Default`, `aggressive`, `conservative`,
`emergency_safe_defaults`) drop the 4 IQN-related fields each.
- `DQN::iqn_network` field + initialisation block in `new_with_stream`.
- 6 conditional gates in `select_action`, `select_action_with_confidence`,
`select_action_inference`, `q_values_for_batch` — all collapse to the
live (branching or standard-Q) arm.
- `DQN::get_state_embedding` (only consumed by deleted IQN paths).
- The entire `crates/ml-dqn/src/quantile_regression.rs` module (392 LOC)
+ its 2 lib.rs exports. Nothing outside `ml-dqn` ever imported it
(the `quantile_huber_loss` reference in `gpu_iqn_head.rs` is a CUDA
kernel name string, unrelated to this Rust module).
Downstream call sites:
- `crates/ml/src/trainers/dqn/{config,fused_training,trainer/constructor}.rs`:
drop `iqn_num_quantiles` / `iqn_embedding_dim` / `iqn_kappa` references
off `DQNConfig`; substitute kernel-fixed literals (64, 1.0) where
`GpuDqnTrainConfig` / `GpuIqnConfig` still expect them.
- `crates/ml/examples/evaluate_baseline.rs`: drop two `iqn_num_quantiles`
hyperparam reads (their `..DQNConfig::default()` fallbacks now stand
alone).
- `crates/ml/tests/dqn_action_collapse_fix_test.rs`: drop the
`assert!(!config.use_iqn, "...gradient dead zone")` and the explicit
`config.use_iqn = false` setter; the dead-zone pathology is now
structurally impossible.
- `crates/ml/tests/dqn_inference_test.rs`: drop `config.use_iqn = false`.
- `services/trading_service/src/services/dqn_model.rs`: drop the
`iqn={}` debug-log field.
- `crates/ml/src/trainers/dqn/distributional_q_tests.rs`: ship Test 0.F
(Plan A Task 8 #186) — converged-checkpoint extraction harness +
`MappedF32Buffer` (mapped pinned f32 mirror) + `compute_sigma_c51_test`
kernel handle. The structural assertions panic on the local 5-epoch
smoke checkpoint as designed (under-converged: sigma_C51 spread ~1.4%
across directions, P(active)=0.4330 >= 0.20). Docstring rewritten to
drop `use_iqn=false` framing and the legacy "Tier-B-prime" caveat;
Tier-A version is GPU-integration-only per
`feedback_no_cpu_forwards.md` (CPU is read-only).
`docs/dqn-wire-up-audit.md` updated per Invariant 7.
Build: `cargo check --workspace --tests` clean (0 errors).
Test: `cargo test -p ml --lib distributional_q_tests::test_0f
-- --ignored --nocapture` produces bit-identical sigma_C51 /
argmax / Thompson values vs pre-removal — confirms branching
forward path is the same code post-cleanup as pre-cleanup
(legacy `use_iqn` arms were unreachable, as expected).
Net: 12 files, +458 / -785 lines (327 LOC deleted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`DQN::load_from_safetensors` was a documented no-op that only checked
file existence and returned Ok(()). Real weight restoration happened
only via the fused trainer's params_buf path; any caller loading a
checkpoint via the DQN struct itself (e.g., DqnInferenceAdapter::
from_checkpoint used by the ensemble) got a fresh untrained DQN with
no error raised — a silent production bug.
Adds BranchingDuelingQNetwork::load_from_named_slices (symmetric
counterpart of existing named_weight_slices) using the same D2D
memcpy primitive as copy_weights_from. Validates names AND shapes;
returns Err on mismatch. Wired into DQN::load_from_safetensors to
actually restore weights from disk, handling both prefix-less (single
head) and `trending__` prefix (regime-merged) safetensors layouts.
Target network is resynced in lockstep so inference and TD targets
see the same restored weights.
Adds NoisyLinear::{weight_mu_mut, bias_mu_mut} for in-place D2D
restore of mu params during checkpoint load.
Tests:
- branching::tests::test_load_from_named_slices_round_trip (happy path)
- branching::tests::test_load_from_named_slices_rejects_extra_keys
(arch-drift safety)
- branching::tests::test_load_from_named_slices_rejects_shape_mismatch
- dqn::save_load_tests::test_dqn_load_from_safetensors_round_trip
(full end-to-end safetensors save+load, #[ignore] for CUDA gate)
- dqn::save_load_tests::test_dqn_load_from_safetensors_missing_file
Also un-ignores the ensemble adapter's
`test_dqn_checkpoint_round_trip` test (was ignored pre-fix because
load_from_safetensors silently dropped weights). Disables NoisyNet
on both sides for deterministic argmax comparison. Adds CUDA_LOCK
mutex to serialize the adapter tests that share a CUDA stream via
the SHARED_CUDA OnceLock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two ghost-feature fixes from the pre-L40S cleanup audit (tasks #66
and #68 tracked internally).
### #66 — delete apply_accumulated_gradients (dead code from removed path)
The agent-audit confirmed this function is residue from a prior
Candle-gradient-tracking training path that was REMOVED (see the
now-deleted test crates/ml/tests/test_var_source_gradients.rs which
was `#[ignore]`'d with the comment "Candle gradient tracking removed
-- DQN/PPO use custom CUDA backward passes"). Evidence:
- `DQN::optimizer: Option<GpuAdamW>` is always `None` — never
initialised anywhere in the codebase.
- `DQN` uses `OwnedGpuLinear` + `NoisyLinear` with standalone
`CudaSlice<f32>` BY DESIGN (see comment at branching.rs:988-989
"this layout exists to avoid a GpuVarStore intermediate"). There
is no GpuVarStore to feed GpuAdamW.
- Zero external callers for `DqnTrainer::apply_accumulated_gradients`,
`DQN::apply_accumulated_gradients`, `RegimeConditionalDQN::
apply_accumulated_gradients`, or the various `optimizer_vars()`
wrappers.
- The only test exercising this path was `#[ignore]`'d with the
"Candle gradient tracking removed" rationale.
- Production training goes through the fused CUDA trainer
(`trainers/dqn/fused_training.rs`) which applies gradients into a
flat `params_buf` — a separate, live path.
Deleted: 6 functions across 3 files + the stale test. Preserving a
ghost that has zero callers, zero initialisation path, and a design
direction explicitly chosen AWAY from its premise is not "keep and
wire" — it's accumulating fiction. Per feedback_no_functionality_
removal.md the rule preserves FUNCTIONAL features; this wasn't one.
### #68 — TFT honest error message
Audit found TFT is architecturally incompatible with GpuAdamW in its
current form (not a wiring gap, an architectural absence):
- `TemporalFusionTransformer` has no GpuVarStore, no `parameters()`,
no `named_parameters()` accessor
- Internal layers use `StreamLinear` which has NO `backward()`
- `TFTModel` trait only exposes `forward()`, `get_config()`,
`clear_cache()` — no gradient accessor
- `TemporalFusionTransformer::train()` runs forward + accumulates
loss but never calls backward or optimizer step
- `TrainableTFT` adapter's `backward()` explicitly returns "not
supported — use TFTTrainer::train() instead"
The previous error string "TFT optimizer not yet migrated to
GpuAdamW" implied 99% done and a simple constructor wiring would
finish it. Actual gap: GpuVarStore threading through 5+ layers +
StreamLinear→GpuLinear migration + backward ops for softmax
attention / layer norm 3D / quantile monotonicity chain / stack +
trait extension for var_store accessor + train-loop gradient
assembly. ~3-7 day dedicated architectural project.
New error message states this explicitly so no one wastes time
chasing an "almost migrated" fiction. Full-lift tracked separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- ml-dqn/dqn.rs: `apply_accumulated_gradients` is a scaffolding method
whose real optimizer step lives in the fused CUDA trainer. The
`grads` map was already being dropped silently; reword the comment
to describe that split explicitly (incidental: see trainer path for
the live gradient application).
- ml-features/mbp10_loader.rs: strip the "TODO optimize with binary
search" parenthetical from the docstring. Linear search over the
sorted snapshot slice is the intended behaviour for current call
sites.
- ml-hyperopt/optimizer.rs: `optimize_two_phase` short-circuits after
Phase A because `DQNTrainer` is not `Clone`. Describe that limit
and point callers at `optimize_parallel` (which requires `M: Clone`)
rather than a hypothetical Phase B.
- ml-checkpoint/signer.rs: `fetch_key_from_vault` is currently an
env-var resolver. Reword to say so plainly — no Vault client is
wired into this crate, production uses K8s secrets injected as env.
- backtesting/dbn_replay.rs: `DbnReplayEngine::from_bytes` remains an
Err stub because `DbnParser` is gated behind the `databento`
feature which this crate does not enable. Replace the pseudocode
block with a declarative comment.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HEALTH_DIAG `sigma_mag` / `sigma_dir` were reporting raw weight_sigma
mean (constant 0.0320 across schedules) because reset_noise_with_sigma
only scaled the noise epsilon samples, not the underlying σ tensor.
Fix: added current_sigma_scale: f32 field on NoisyLinear (default 1.0),
updated in reset_noise_with_sigma, multiplied through in sigma_mean()
accessor. Also propagated through copy_params_from so target-net sync
does not reset the reported effective σ.
The HEALTH_DIAG field now reflects the scheduled effective σ, enabling
H7 detection signal to actually observe schedule attenuation.
Closes Track 4 E2 TUNE finding from Phase 1 triage.
Wires Track 1 noisy_mag/noisy_dir + Track 4 sigma_mean fields with the
mean |sigma| across each action branch's NoisyLinear fc + out layers.
Branch 0 = direction, Branch 1 = magnitude. H7 detection signal — if
magnitude branch has 2x larger σ than direction, NoisyNets noise is
dominating the magnitude head's effective signal.
Cross-crate API chain (no shortcuts):
* ml-dqn::noisy_layers::NoisyLinear::sigma_mean() -> mean |weight_σ| + |bias_σ|
* ml-dqn::branching::BranchingDuelingQNetwork::branch_noisy_sigma_mean(idx)
averages fc + out σ for the named branch
* ml-dqn::dqn::DQN::branch_noisy_sigma_mean(idx) — None-tolerant proxy
* ml::trainers::dqn::config::DQNAgentType::branch_noisy_sigma_mean(idx)
delegates through primary_head
* HEALTH_DIAG reads via self.agent.read().await at epoch boundary
Pinned-readback pattern not used here — NoisyLinear weights live in
candle-managed CudaSlices, not flat trainer params buffer. Per-call
dtoh of ~256 + 768 floats × 2 layers × 4 branches = ~8KB total per
epoch. Negligible.
Track 0.10 (exploration entropy + sigma_mean) is now COMPLETE — the
sigma_mean field that was previously stubbed is now real.
Task 0.6 remains partial: VSN mask (vsn_mag, vsn_dir) and target drift
(drift_mag, drift_dir) still stubbed — they require separate accessors
on different layer types (VSN module + target_params_buf reductions).
Two bugs caught by the L40S smoke (train-qhgj6) that couldn't surface on
local RTX-3050 single-fold runs:
1. PER dtoh inside CUDA Graph capture (Fold 1 crash)
Failure: CUDA_ERROR_STREAM_CAPTURE_INVALIDATED at per_prefix_scan on
Fold 1 re-capture. Chain: fused_training parent graph captures →
memcpy_dtoh + cuStreamSynchronize in gpu_replay_buffer::update_priorities_gpu
(health<0.8 diversity path) poisons the stream → subsequent per_sample
kernel on the same stream sees an invalidated capture context.
The prior comment claimed "runs once per epoch, DtoH cost acceptable"
— wrong, it runs every priority update when health<0.8 (common during
Fold handoff when health_cache is re-seeded low). Any dtoh inside
capture invalidates regardless of latency.
Proper fix (no shortcut):
* New kernel actions_sum_scale_reduce_u32 — single-block deterministic
tree reduction over sample_actions (u32) → writes (sum*1000)/n as i32
to a device-accessible slot. No atomics (consistent with the 1/N
determinism policy from commit c82386500).
* mean_action_scaled storage is pinned + device-mapped (cuMemAllocHost
+ cuMemHostGetDevicePointer — same pattern as rng_step_dev_ptr and
size_dev_ptr elsewhere in the file). Zero-copy between host and
device, graph-safe, no explicit free needed (process-exit cleanup,
matches existing pattern).
* pow_alpha_diverse_f32 now takes const int* mean_action_scaled_ptr
and does a plain global load — NOT __ldg. The read-only cache used
by __ldg is not guaranteed coherent with device-mapped host memory;
multi-trial smoke regression caught it (median q_gap collapsed
from 2.0 → 0.15 with __ldg, recovered to 2.8 with plain load).
Verified: multi-trial smoke 5/5 pass, median_q_gap=2.80 (beats 2.00
baseline), Best Sharpe peaks 19-30 per trial. No stream capture
invalidation.
2. evaluate step CLI drift in Argo template
evaluate_baseline's Args struct uses --models-dir and --output (single
file path). Template was passing --checkpoint-dir and --output-dir,
causing clap to reject the invocation. Fixed argument names + added
mkdir for the eval subdir + updated the comment to pin the source of
truth for future drift catches.
Both fixes are graph-capture-clean and match the "wire properly or delete"
discipline. No masking, no feature flags, no dead params.
gpu_replay_buffer.rs had three sites that cast `&CudaSlice<i32>` to
`&CudaSlice<u32>` via raw-pointer transmutes to satisfy kernel
signatures (gather_u32, scatter_insert_u32). This changes the element
type of a `&mut` reference through `as *mut`, which is UB under Rust's
strict aliasing model even though i32 and u32 share a memory layout.
Episode-id values are always non-negative (counters of the form
`(write_cursor + j) % capacity`), so u32 is the correct storage type.
Changed:
- 3 field types: episode_ids, sample_episode_ids, insert_ep_buf
(CudaSlice<i32> → CudaSlice<u32>)
- Allocators: a32i → a32u at 4 sites
- Vec<i32> → Vec<u32> at ep_ids_host, with `as u32` cast
- Deleted 3 raw-ptr transmute blocks (lines 491-492, 689-690)
- Public sample_episode_ids_ref() return type i32 → u32
- Deleted now-unused a32i helper (fn never called after the change)
Per BORROW learned pattern option (3): "change the declared type".
Both dtod_copy wrappers (noisy_layers.rs:372, gpu_dqn_trainer.rs:13220)
already call memcpy_dtod_async internally — the name didn't advertise
the async semantics, which caused the PINMEM scan to false-positive on
them.
Per user question "why use wrappers?" — they provide per-call-site
error context (label / op / idx) with ~5 LOC of setup each. The value
is real but small; renaming to dtod_copy_async makes the async
semantics visible at every call site and lets the scan regex
(`\bdtod_copy\b`) stop matching them.
27 sites across 4 files: noisy_layers.rs (7), gpu_dqn_trainer.rs
(def + many callers), gpu_iqn_head.rs (3), fused_training.rs (imports).
MultiAssetPortfolioTracker.opportunity_scores field and its two public
methods (set_opportunity_scores, select_active_symbol) had zero callers
workspace-wide per rg. Also eliminates the FALLBACK-001 symbols[0]
default — that else branch was inside the now-removed dead method.
Deleted: field + 2 init sites + 2 pub methods + architecture doc line.
Per user directive "no deferred tasks, solve properly" — was previously
marked as DEFERRED (FALLBACK-001 false-positive flagging dead code).
DQNConfig.enable_q_value_clipping was true at all 4 construction sites;
the else branch returned the pre-clamp tensor unchanged (dead identity
path). Q-value clipping is a BUG #37 fix — clipping happens after
forward pass so gradients still flow, and prevents Q-value explosions
from poisoning action selection. Per feedback_no_feature_flags.md, the
flag is gone — clipping is now unconditional. Kept q_value_clip_min /
q_value_clip_max since they're still passed to clamp().
QNetworkConfig had three write-only fields with zero workspace readers:
use_spectral_norm, spectral_norm_iterations, use_residual. Defaults set
at the single Default impl, never checked in forward/backward. Deleted
all three + their Default init. use_gpu kept — line 138 gates CUDA
construction and errors without it (genuine sanity check).
RainbowNetworkConfig.use_spectral_norm + spectral_norm_iterations were
declared but never read anywhere in the workspace. Defaulted false, set
false at the sole construction site. Pure write-only pair. Deleted both
fields + 2 construction sites. Serde default (no deny_unknown_fields)
keeps old JSON configs deserializable.
DQNConfig.use_soft_updates was true at all 4 construction sites; the
else branch (legacy hard update) was unreachable. Deleted field + 4
setters + collapsed update_target_networks() to the unconditional
cosine-annealed EMA path. Hard-copy mode removed per
feedback_no_feature_flags.md.
DQNConfig.use_regime_conditioning was declared, defaulted true at 3
construction sites, but never read anywhere in the workspace. Classic
write-only flag (like use_different_seeds in DEAD-001). Deleted field +
docstring + 3 assignment sites. Regime conditioning is already
unconditional per the trainer — the flag was vestigial.
LoggingConfig.enable_network_diagnostics was declared and defaulted but
never read anywhere (only test round-trip assertions existed). Deleted
field + Default init + two test assertions that validated the round-trip.
Zero production callers per rg.
The field was gated by validate() to always be true (absolute-dollar mode
caused Q-value explosion ±50,000). All call sites already set true. Deleted:
- RewardConfig.use_percentage_pnl field + Default init
- Absolute-dollar branch in calculate_pnl_reward (dead)
- Validation guard against false
- RewardConfigBuilder field + builder method + build() init
- Constructor call site
Percentage-based P&L is now unconditional per feedback_no_feature_flags.md.
EnsembleConfig.use_different_seeds was set but never read anywhere in
the workspace. rg confirmed zero readers. Field removed from struct,
Default impl, and new() constructor (3 sites).
When learning_health < 0.8, the PER priority update switches from the
standard per_update_pa kernel to pow_alpha_diverse_f32, which multiplies
each priority by (1 + 2*(1-health)*|action - mean_action|). This rescues
the replay buffer's diversity signal during Q-collapse by surfacing
experiences whose action deviates from the batch mean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostics served their purpose — confirmed OFI flows through full
pipeline on H100 (state_gather → PER → trainer). Remove:
- OFI host verify readback (upload_ofi_features)
- STATE_GATHER_DIAG pinned memory readback (timestep loop)
- Dead bf16 scatter_insert kernel (states are f32, was never called)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
scatter_insert_f32 is a 1D scalar kernel (5 args: dst, src, cursor, cap,
batch_size). insert_batch called it with 6 args for state matrices, passing
state_dim as batch_size. CUDA silently dropped the 6th arg (actual batch_size).
Result: only 96 floats (1 state row) inserted per experience batch into PER.
The model was training on ~99.99% uninitialized GPU memory. This bug affected
ALL state features, not just OFI — market features and portfolio were also
garbage in PER-sampled training batches.
Fix: added scatter_insert_f32_rows kernel (2D-aware, 6 args: dst, src,
cursor, cap, state_dim, batch_size) matching the existing scatter_insert
(bf16) pattern. States and next_states now use the row-aware kernel.
Verified locally: OFI_DIAG shows non-zero values through the full chain
(state_gather → env_step → PER insert → PER sample → trainer states_buf).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed the legacy non-branching Sequential q_network and target_network
from DQNAgent. These were never used (branching+dueling always active)
but allocated VRAM and ran noise resets every step. -190 lines.
Config tuning for dense micro-reward system:
- n_steps: 5→1 (TD(0), micro-rewards cancel over n>1)
- tau: 0.007→0.01 (faster target tracking for TD(0))
- c51_alpha_max: 0.5→1.0 (full C51, PopArt handles normalization)
- curiosity_weight: 0.1→0.0 (dense micro-reward replaces curiosity)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Action space 81→108: direction(4) × magnitude(3) × order(3) × urgency(3).
Short(0), Hold(1), Long(2), Flat(3). Hold keeps current position with
zero transaction cost, giving the model a "do nothing" option.
Changes: trade_physics.cuh (Hold returns current_position), env_step
(Hold skips trade execution), action.rs (ExposureLevel::Hold variant),
config (branch_0_size 3→4), all match arms updated across 11 files.
Counterfactual mirror: Short↔Long, Hold↔Hold, Flat↔Flat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Changed per_prefix_scan, per_sample, and is_weights_f32 kernels to accept
size as a const int* pointer (pinned device-mapped) instead of a baked int.
Graph replay now uses the CURRENT buffer size, not the capture-time value.
Host updates the pinned size on every insert_batch and clear.
per_sample also reads rng_step from pinned pointer (GPU-side increment).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: rng_step was passed as 0u64 (null) to increment_step_counters
kernel, causing atomicAdd on address 0 → CUDA_ERROR_ILLEGAL_ADDRESS →
cascading CUBLAS_STATUS_EXECUTION_FAILED on all subsequent operations.
Fixes:
- Replace all atomicAdd with plain writes (single-thread kernel)
- Add null guards for optional pointers (iqn_t, attn_t, rng_step)
- Allocate pinned device-mapped rng_step in GpuReplayBuffer
- Wire rng_step_dev_ptr from replay buffer to FusedTrainingCtx
- Remove stale atomicAdd comment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PER gather now writes directly to GpuDqnTrainer's padded buffers via
gather_f32_rows_padded, gather_f32_scalar, and gather_i32_scalar kernels
compiled into the replay buffer cubin. set_trainer_buffers() wires stable
device pointers at init; sample_proportional uses them when available,
falling back to intermediate buffers otherwise. memset_zeros calls in
the sampling hot path converted to raw cuMemsetD8Async.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
