da5e564ccfad43071f579a824ef122dc7017d837
557 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
da5e564ccf |
feat(sp22-vnext): Phase B4b-2 — replay-buffer scatter + trainer setter
Second half of Phase B4b. Wires the trade-outcome label column through
the replay buffer (struct field, allocation, scatter on insert, gather
on sample, direct-to-trainer pointer + setter) and connects the
trainer's aux_to_label_buf to receive per-batch sampled labels.
Completes the end-to-end producer → ring → trainer i32 path the K=3
sparse CE consumer reads.
Replay buffer changes (crates/ml-dqn/src/gpu_replay_buffer.rs):
- New struct fields: aux_outcome_labels (capacity-sized ring),
sample_aux_outcome_labels (mbs-sized fallback gather), trainer_aux_
outcome_labels_ptr (direct-path destination)
- New aux_outcome_labels_ptr field on GpuBatchPtrs
- insert_batch signature: new aux_outcome arg between aux_conf_in and
bs. Scatters via existing K-generic scatter_insert_i32 (same kernel
the K=2 aux_sign_labels uses).
- sample_proportional direct gather when trainer_aux_outcome_labels_ptr
!= 0; fallback gather otherwise. Mirrors aux_sign_labels direct/
fallback semantic exactly.
- New setter set_trainer_aux_outcome_labels_ptr mirrors
set_trainer_aux_conf_ptr.
Collector emission:
- New aux_outcome_labels field on GpuExperienceBatch
- Populated at end of collect_experiences_gpu via dtod_clone_i32 from
Phase B4b-1's per-(env, t) producer scratch.
Trainer wireup:
- aux_to_label_buf_ptr() accessor on GpuDqnTrainer (mirrors
aux_nb_label_buf_ptr)
- trainer_aux_to_label_buf_ptr() delegating accessor on
FusedTrainingCtx
- New set_trainer_aux_outcome_labels_ptr call in training_loop at the
same site where set_trainer_buffers + set_trainer_aux_conf_ptr fire.
Two call sites updated (lines ~835 + ~2857).
- insert_batch call in training_loop passes &gpu_batch.aux_outcome
_labels as new arg.
Test fixtures updated: 4 smoke test files + 3 unit-test fixtures in
gpu_replay_buffer.rs alloc zero-init aux_outcome i32 arg.
End-to-end chain complete:
trade_outcome_label_kernel (A2)
→ collector per-step launch (B3)
→ collector emission (B4b-1)
→ replay-buffer insert + scatter (B4b-2)
→ PER sample + direct gather (B4b-2)
→ trainer aux_heads_forward.loss_reduce (B4) reads sparse {-1,0,1,2}
→ trainer aux_heads_backward (B4) computes per-sample partials
→ Adam SAXPY (B1+B4) updates W1, b1, W2, b2 at [163..167)
The "degraded predict-Profit-everywhere" cold-start from Phase B4 is
resolved. K=3 head trains on real sparse trade-outcome labels.
Verification:
- cargo check -p ml clean (21 warnings, none new).
- cargo test -p ml --lib → 1016/0 on clean runs; pre-existing
NoisyLinear flake still surfaces ~30-50% of runs (unrelated to
vNext work — see
|
||
|
jgrusewski
|
491bf7d3e6 |
feat(sp22-vnext): Phase B4b-1 — per-(env, t) label kernel output + collector buffer
First half of Phase B4b (replay-buffer label scatter chain). Amends the Phase A2 trade_outcome_label_kernel to emit a per-(env, t) output column alongside the existing per-env tile, and adds the collector-side buffer + per-step launch arg. Kernel amendment (trade_outcome_label_kernel.cu): - Added NULL-tolerant `out_labels_per_sample` arg after existing `out_labels`. When non-NULL, writes `out_labels_per_sample[env*L + t] = label` at the same offset as the `trade_close_per_sample[env*L + t]` read. NULL = no-op (preserves Phase A2/A3 contract for callers passing old signature). - Pattern mirrors the K=2 head's `aux_sign_labels` per-(i, t) ring column that threads through the replay buffer. Collector field + alloc + launch: - New struct field `exp_aux_to_label_per_sample: CudaSlice<i32>` sized `[alloc_episodes × alloc_timesteps]`. Sentinel -1 (mask) populated by alloc_zeros + per-step kernel writes — survives until a trade-close event overwrites the env's slot at that t. - Updated Phase B3 launcher in collect_experiences_gpu to pass `self.exp_aux_to_label_per_sample.raw_ptr()` as the new arg. Why split B4b into B4b-1 + B4b-2: the full replay-buffer wireup mirrors the K=2 head's aux_sign_labels pattern across ~8 distinct code sites (replay-buffer struct field, sample destination buffer, direct-to-trainer pointer, setter method, scatter on insert, gather direct, gather fallback, GpuBatchPtrs field). Splitting lets us validate the per-(i, t) producer in isolation before touching the consumer pipeline. B4b-1 (this commit) = producer chain complete. Per-(env, t) column populated correctly every rollout step. Consumer wiring (replay- buffer scatter + trainer setter) is B4b-2's scope. Verification: - cargo check -p ml clean (21 warnings, none new). - cargo test -p ml --lib → 1016/0 on clean runs; pre-existing test_dqn_checkpoint_round_trip NoisyLinear flake still surfaces ~50-70% of full-suite runs (flake predates Phase B4b, unrelated to trade-outcome head — disable_noise() zeros ε but leaves some other randomness source intact). Audit: docs/dqn-wire-up-audit.md Phase B4b-1 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
20e1aea27c |
feat(sp22-vnext): Phase B4 — trainer-side replay-batch chain wireup
Wires the trade-outcome head's forward + loss reduce + backward +
per-sample partial reduce + SAXPY into the trainer's aux_heads_forward
and aux_heads_backward methods. Adam SAXPY for the 4 new weight tensors
at [163..167) extends uniformly via the existing aux_param_specs array
iteration.
Changes to aux_heads_forward:
- Steps 7 + 8 appended after K=2 next-bar head's loss reduce
- K=3 forward reads weights at [163..167) (Phase B1) → writes to
aux_to_* save-for-backward tiles (Phase B2)
- K=3 loss_reduce writes aux_to_loss_scalar_buf + aux_to_valid_count_buf
Changes to aux_heads_backward:
- K=3 backward appended after K=5 regime backward, emits per-sample
partials (dW1, db1, dW2, db2) + per-sample dh_s2_aux
- aux_param_specs array extended from 8 → 12 entries. Per-tensor
reduce + SAXPY loop iterates uniformly, scaling each by aux_weight
- K=3 dh_s2_aux SAXPY appended after K=5's, all three heads'
gradients flow into aux trunk's dh_s2_aux_accum (encoder stop-grad
enforced structurally by aux_trunk_backward's missing dx_in output)
Label semantic (cold-start): aux_to_label_buf is alloc_zeros (all 0
= Profit) until Phase B4b lands replay-buffer label scatter. Model
trains on "predict Profit everywhere" — degraded but well-defined
(no NaN). Mirrors K=2 head's known-degraded state between B1.1a
(forward landed) and B1.1b (label producer wired).
Adam SAXPY: existing global SAXPY iterates 0..NUM_WEIGHT_TENSORS
(now 167) — 4 new weight slots get gradient SAXPYs followed by
Adam m/v updates uniformly. Architectural payoff of Phase B1's
NUM_WEIGHT_TENSORS bump.
Test flake mitigation: added bind_to_thread() to
ensemble::adapters::dqn::tests::shared_device() mirroring the
cuda_stream() test-helper pattern from the fix sweep at
|
||
|
jgrusewski
|
b28b349ac3 |
feat(sp22-vnext): Phase B3 — collector-side rollout buffers + forward chain wireup
Adds collector-side trade-outcome head: 5 struct fields + allocations
+ per-step forward + per-step label producer launches in the rollout
loop. Mirrors the K=2 next-bar head's collector wireup at K=3.
Collector struct additions:
- exp_aux_to_fwd: AuxTradeOutcomeForwardOps (3 kernel handles)
- exp_aux_to_hidden_buf [alloc_episodes × H=128] saved post-ELU
- exp_aux_to_logits_buf [alloc_episodes × K=3] saved logits
- exp_aux_to_softmax_buf [alloc_episodes × K=3] softmax tile
- exp_aux_to_label_buf [alloc_episodes] i32 sparse {-1, 0, 1, 2}
Per-step launches in collect_experiences_gpu rollout loop:
1. aux_trade_outcome_forward — launched immediately after the K=2
sibling's forward_next_bar, parallel on the same stream. Reads
exp_h_s2_aux + weights at flat-buffer indices [163..167) (Phase
B1 additions). Writes hidden/logits/softmax tiles. No consumer
yet — Phase C wires state assembly; Phase B4 wires trainer
scatter.
2. trade_outcome_label_kernel — launched immediately after
experience_env_step on the same stream, reading the save-for-
backward buffers (pnl_vs_target_at_close_per_env, pnl_vs_stop_at_
close_per_env) that env_step just wrote at segment_complete.
Stream-implicit producer→consumer ordering. Emits per-env
{-1, 0, 1, 2} labels — sparse, ~95-99% bars produce -1 (mask).
Dead-code discipline per feedback_wire_everything_up: every kernel arg
+ producer site is real wiring (not NULL placeholder) — only the
absence of consumers reading the produced tiles is "dead". The smoke
run produces softmax tiles + labels every step bit-identical to
pre-vNext baseline (no consumer = no effect on training behavior).
Phase B4 next: trainer-side replay-batch chain (forward + loss_reduce
+ backward + Adam SAXPY for the 4 new weight tensors).
Audit: docs/dqn-wire-up-audit.md Phase B3 section.
Verification:
- cargo check -p ml clean (21 warnings, none new on aux_to_*).
- cargo test -p ml --lib → 1016 passing / 0 failing (unchanged from
post-fix-sweep baseline at
|
||
|
jgrusewski
|
ebc1b15023 |
fix(tests): repair 14 pre-existing test failures across ml crate
Lib test suite was at 14 failures from accumulated layout/contract drift.
Fixed each by tracing root cause; lib suite now 1016 passing / 0 failing.
Failures fixed (test → root cause → fix):
1. sp14_isv_slots::sp20_isv_slots_reserved_510_to_520 — ISV_TOTAL_DIM
pin drifted from SP20-era 520 to current 538 via SP21/SP22 H6 bus
growth. Slot positions 510-519 still pinned. Fix: relax total-dim
check to >= 520; keep slot-position asserts tight.
2-3. gradient_budget::test_spectral_norm_all_heads_no_panic +
test_spectral_norm_constrains_operator_norm — test config used
cfg.state_dim=16 but trainer uses STATE_DIM=128 when bottleneck
off; also DuelingWeightBacking slices [2]/[3] used pre-GRN w_s2
shape, but post-GRN it's w_b_h_s1 [2*SH1, SH1] = 2048. Fix: add
s1_input_dim_for_test helper; correct slice sizes to GRN layout.
4-9. dqn::trainer::tests::test_feature_vector_to_state +
test_single_sample_batch + test_batched_action_selection +
test_batched_vs_sequential_action_selection_consistency +
test_batch_size_mismatch_larger/smaller_than_configured —
feature_vector_to_state wrapper falsely advertised "no OFI" by
signature but body required strict OFI. Contract violation —
broke 6 unit tests + hyperopt's public convert_to_state APIs.
Fix: wrapper now genuinely produces 45-dim no-OFI state; OFI-
strict callers use _with_ofi with Some(idx).
10. state_kl_monitor::observe_tracks_fire_rate_on_change — last_amp
initialized to 1.0 coincidentally equaled first observation value,
silently suppressing first fire. Test expected first observation
always fires (cold-start semantic). Fix: Option<f32> sentinel —
None means "no prior baseline" → first observe ALWAYS fires.
11. cuda_pipeline::tests::test_eval_action_select_thompson_picks_
proportionally — test launched experience_action_select kernel
with 1 missing arg (v_logits_dir from SP17 Commit C). Kernel
read garbage pointer → SIGSEGV. Also threshold 0.70 calibrated
for pre-SP17 raw-A distribution; post-SP17 sampler uses mean-zero
softmax(V + A_centered), Long still dominates ~4× but P(Long)
drops to ~0.66. Fix: add zero-filled v_logits_buf at correct
arg position; recalibrate threshold 0.70 → 0.60.
12. cuda_pipeline::tests::test_ppo_gpu_data_upload — flaked in parallel
test run with CUDA_ERROR_INVALID_CONTEXT from MappedF32Buffer's
cudaHostAlloc. cuda_stream() test helper used OnceLock to share
a CudaStream but didn't bind_to_thread on subsequent calls. CUDA
contexts are per-thread state. Fix: helper now calls bind_to_
thread() on every invocation (idempotent — mirrors trainer/
constructor.rs:111's pattern).
13. ppo::tests::test_reward_computation — test created hold action
with ExposureLevel::Flat but is_hold() matches only Hold (Flat
= close position = transaction with cost). So hold and buy got
same transaction-cost reduction; reward_hold > reward_buy assert
failed. Fix: use ExposureLevel::Hold for no-cost hold semantic.
14. training_profile::tests::test_production_profile_applies_all_
sections — pinned n_steps==5 and tau==0.005 (pre-TD(0)); production
flipped to n_steps=1, tau=0.01 per dqn-production.toml ("dense
micro-rewards cancel over n>1 bars; faster target tracking for
TD(0)"). Fix: update pins to current production values.
All 14 fixes target root causes — no #[ignore] masking. Verified:
- Lib-only run: cargo test -p ml --lib → 1016/0 (passed/failed)
- Stash-test pre-changes: 1001/15 (confirms 14 fixed atomically)
Remaining flake under `cargo test -p ml --tests`:
- ensemble::adapters::dqn::tests::test_dqn_checkpoint_round_trip —
CUDA-context-race-adjacent flake under parallel `--tests` mode;
passes in isolation and in `--lib` mode (single binary). Predates
this commit; deferred as separate triage.
Audit: docs/dqn-wire-up-audit.md "Fix sweep" section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
b98dc2730d |
feat(sp22-vnext): Phase B2 — trade-outcome trainer saved-tensor + partial buffers
Adds 11 buffer fields + 2 orchestrator ops handles (fwd + bwd) to the
trainer struct, mirroring the existing aux_nb_* / aux_partial_nb_*
pattern at K=3 instead of K=2.
Trainer struct additions:
- aux_to_fwd: AuxTradeOutcomeForwardOps (Phase B0 scaffold)
- aux_to_bwd: AuxTradeOutcomeBackwardOps
- aux_to_hidden_buf [B, H=128] saved post-ELU
- aux_to_logits_buf [B, K=3] saved logits
- aux_to_softmax_buf [B, K=3] saved softmax (3 future consumers)
- aux_to_label_buf [B] i32 sparse {-1, 0, 1, 2}
- aux_to_loss_scalar_buf [1] mean CE
- aux_to_valid_count_buf [1] B_valid for backward
- aux_dh_s2_to_buf [B, SH2] SAXPYs into dh_s2_aux_accum
- aux_partial_to_w1 [B, H, SH2] per-sample dW1
- aux_partial_to_b1 [B, H] per-sample db1
- aux_partial_to_w2 [B, K=3, H] per-sample dW2
- aux_partial_to_b2 [B, K=3] per-sample db2
Memory: aux_partial_to_w1 = 256 MB at B=2048 — identical to K=2 head's
partial size (same SH2, same H). Total new aux-to footprint ≈ 260 MB.
The existing aux_param_grad_final_buf scratch is sized to the largest
tensor across all aux heads; trade-outcome head's largest is W1 [H, SH2]
= 32,768 floats — identical to K=2/K=5 W1s. No resize needed.
Cold-start label semantics: alloc_zeros yields label 0 (Profit) for
every sample. Until the producer wires in (B3), the trainer's CE loss
treats every sample as "should have predicted Profit" — degraded but
well-defined (no NaN). Mirrors the K=2 head's known-degraded state
between B1.1a and B1.1b.
No FoldReset registration: these buffers are overwritten every batch
— no stale-state-leak risk across folds (matches the existing aux_nb_*
pattern).
Phase B3 next: collector-side rollout buffers + forward chain wireup
into collect_experiences_gpu (per-env softmax → per-(i, t) fan-out
scatter for trainer's aux_to_softmax_buf population).
Audit: docs/dqn-wire-up-audit.md Phase B2 section.
Cargo check clean (21 warnings, none new on aux_to_* fields).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
205e46c171 |
feat(sp22-vnext): Phase B1 — trade-outcome head weight tensors + Xavier init
Adds the 4 weight tensors (W1, b1, W2, b2) for the SP22 H6 vNext
trade-outcome aux head into the trainer's flat params_buf at indices
[163..167). Adam machinery (m/v moment buffers, SAXPY iteration over
0..NUM_WEIGHT_TENSORS) picks up the new tensors uniformly — no
per-tensor wiring needed.
Changes:
- NUM_WEIGHT_TENSORS bumped 163 → 167. Most of the 54 references are
&[u64; NUM_WEIGHT_TENSORS] array-size generics that resize
uniformly with the constant.
- compute_param_sizes() adds 4 new size entries:
[163] aux_to_w1 [H=128, SH2=256] = 32,768 floats
[164] aux_to_b1 [H=128] = 128 floats
[165] aux_to_w2 [K=3, H=128] = 384 floats
[166] aux_to_b2 [K=3] = 3 floats
Total: 33,283 floats = ~133 KB params, ~266 KB Adam state.
- compute_param_sizes() debug_assert updated 163 → 167.
- Xavier fan_dims added: (H, SH2) for W1, (0, 0) for biases (zero-init),
(K=3, H) for W2. Cold-start: logits ≈ 0 → softmax ≈ uniform 1/3 → no
Profit/Stop/Timeout preference per pearl_first_observation_bootstrap.
SH2 stays at 256 in this commit (mirrors K=2 head exactly). The spec's
Phase B input concat (256 → 262 with plan_params 6-dim) will re-shape
slot [163] to 128 × 262 = 33,536 floats later — small touch-up vs the
full B1 commit.
Verification:
- cargo check -p ml clean (21 warnings, none new).
- 14 pre-existing test failures stashed-verified unrelated (OFI features
missing, ISV slot count drift — independent of NUM_WEIGHT_TENSORS).
Phase B2 next: saved-tensor + per-sample partial buffers (hidden_post,
logits, softmax, valid_count, dW*_partial, dh_s2_aux_out).
Audit: docs/dqn-wire-up-audit.md Phase B1 section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
108a426c38 |
feat(sp22-vnext): Phase B0 — AuxTradeOutcome ops struct scaffolding
First Rust-side commit of Phase B for the SP22 H6 vNext trade-outcome aux head. Pure orchestrator scaffolding mirroring AuxHeadsForwardOps / AuxHeadsBackwardOps. Zero production callers — additive per the gpu_grn / aux_trunk commit-by-commit ordering convention (scaffold → trainer fields → collector wireup). Adds: - gpu_dqn_trainer.rs: 4 cubin embeds (TRADE_OUTCOME_LABEL_CUBIN, AUX_TRADE_OUTCOME_FORWARD_CUBIN, AUX_TRADE_OUTCOME_LOSS_REDUCE_CUBIN, AUX_TRADE_OUTCOME_BACKWARD_CUBIN). All #[allow(dead_code)] until B1+. - gpu_aux_heads.rs: AUX_OUTCOME_K = 3 constant (parallel to AUX_NEXT_BAR_K). - gpu_aux_heads.rs: AuxTradeOutcomeForwardOps struct holding 3 kernel handles (forward + loss_reduce + label producer). Launch methods: forward(), loss_reduce(), compute_label(). Per-env label kernel uses grid ceil(n_envs/256) — pure per-env map, no reduction. - gpu_aux_heads.rs: AuxTradeOutcomeBackwardOps struct holding 1 kernel handle (backward). Launch method: backward(). Reuses existing K-generic aux_param_grad_reduce from AuxHeadsBackwardOps — no separate reducer needed. Contract shapes mirror AuxHeadsForwardOps/AuxHeadsBackwardOps so subsequent wireup commits plug in with minimal contract drift. Phase B proper (input concat 256→262 with plan_params) becomes a small change touching only the buffer fill + W1 shape once B1-B4 land the wireup. Phase B1 next: trainer struct fields for W1/b1/W2/b2 + Adam state + Xavier init + reset registry entries. Audit: docs/dqn-wire-up-audit.md Phase B0 section. Cargo check clean (21 warnings, none new). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
9f4a25e623 |
feat(sp22-vnext): Phase A5 — aux_trade_outcome backward kernel (Phase A complete)
K=3 backward kernel that closes the forward → loss → backward chain for the trade-outcome aux head. Mirrors `aux_next_bar_backward` (K=2 sibling) line-for-line because gradient flow is K-independent: `d_logits = (softmax − one_hot)/B_valid` propagated through `Linear → ELU → Linear` chain via standard softmax-CE derivative. Per-sample partials (caller reduces via existing K-generic `aux_param_ grad_reduce` kernel): dW1_partial [B, H=128, SH2=256], db1_partial [B, H] dW2_partial [B, K=3, H], db2_partial [B, K=3] dh_s2_aux_out [B, SH2] Mask handling: labels[b] == -1 zeros the K-vector → all downstream partials zero (chain rule's multiplicative zero). All-skip batch produces valid_count=0 → d_logits=0 for every row → zero gradients across the board, no NaN. Sparse-label gradient amplification: B_valid is typically ~1-5% of nominal batch (trade-close events are rare), so inv_B = 1/B_valid is much larger than the K=2 sibling's inv_B = 1/(~B). Per-trade-close gradients have proportionally higher magnitude — correct credit assignment (rare signal speaks louder) but Phase E's Adam may need class-weighted CE or per-group LR tuning. ELU backward via post-activation identity: f'(x) = (h_post > 0) ? 1 : 1 + h_post — recovers derivative without re-evaluating x_pre. SP14 Phase C.5b separation preserved: reads h_s2_aux (aux trunk output), writes dh_s2_aux_out SAXPYing into dh_s2_aux_accum. Q's encoder structurally protected (aux_trunk_backward has no dx_in output). Phase A5 (this commit) is dead code — no Rust launcher. Phase B will land the full launcher chain (gpu_aux_heads.rs parallel ops struct, collector struct fields for W1/W2/b1/b2/Adam-state/saved-tensors/dW- partials, wireup in collect_experiences_gpu). Cubin: aux_trade_outcome_backward_kernel.cubin (24.8 KB). ═══ Phase A complete ═══ A1: ISV slots (none needed — reuses padding 121-123) A2: trade_outcome_label_kernel.cu (label producer) |
||
|
jgrusewski
|
3ddcfb8868 |
feat(sp22-vnext): Phase A4 — aux_trade_outcome loss reduce kernel
K=3 sparse cross-entropy reduce over the trade-outcome softmax tile produced by `aux_trade_outcome_forward` (Phase A3). Mirrors the K=2 sibling `aux_next_bar_loss_reduce` structurally: single-block shmem-tree reduce, two parallel partial strips (loss_numer + valid_count) reduced lockstep, fmaxf(p_tgt, 1e-30) numerical floor, fmaxf(valid, 1.0) all- skip-batch guard, valid_count_out[1] save-for-backward. Kept as SEPARATE kernel from the K=2 sibling: - Diagnostic isolation (distinct HEALTH_DIAG slot, distinct cubin in profiles for clean per-loss-source attribution) - Sparse-label semantic clarity (~95-99% mask=-1 vs ~50-100% valid for the K=2 next-bar head) - Future per-class weighting headroom (Profit/Stop/Timeout 3:1-10:1 imbalance will likely need class-weighted CE — surgical mod here without touching the K=2 head's contract) Phase A4 (this commit) is dead code — no Rust launcher yet. Phase A5 lands backward; Phase B wires the full forward→loss→backward chain. Discipline: feedback_no_atomicadd (single-block tree-reduce), feedback_ cpu_is_read_only (pure GPU), pearl_first_observation_bootstrap (sentinel 0 valid_count produces zero gradients gracefully on cold start). Audit: docs/dqn-wire-up-audit.md Phase A4 section. Cubin: aux_trade_outcome_loss_reduce_kernel.cubin (9.9 KB) compiles clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
07728f9efc |
feat(sp22-vnext): Phase A3 — aux_trade_outcome forward kernel + save-for-backward wireup
Phase A3 of the SP22 H6 vNext trade-outcome aux head (per
docs/plans/2026-05-14-sp22-h6-vNext-trade-outcome-aux.md). Two atomic pieces:
1. NEW kernel `aux_trade_outcome_forward_kernel.cu` — K=3 softmax aux head
forward (Linear → ELU → Linear → stable softmax). Mirrors
`aux_next_bar_forward` (K=2) but emits {Profit, Stop, Timeout} probs.
Saved tensors {hidden_out, logits_out, softmax_out} ready for A4 (loss
reduce) and A5 (backward). Dead code at this commit — no Rust launcher
yet. Registered in build.rs::kernels_with_common, cubin verified.
2. Save-for-backward buffers `pnl_vs_target_at_close_per_env` +
`pnl_vs_stop_at_close_per_env` ([alloc_episodes] f32 device-resident).
Producer: `experience_env_step::segment_complete` writes the trade's
realized P&L ratios vs profit_target / stop_loss at close (inline-
computed from `pre_trade_position × (raw_close − entry_price) /
(ps[PS_PLAN_PROFIT_TARGET] × prev_equity)`, symmetric-clamped to
[-2, +2] per pearl_symmetric_clamp_audit — same formula as the
sibling experience_state_gather's plan_isv[PLAN_ISV_PNL_VS_TARGET/_STOP]
slots). Consumer (eventual A4/A5 wireup): trade_outcome_label_kernel
classifies each close into {Profit, Stop, Timeout} via the ≥1.0
threshold-hit predicate.
Wireup discipline per feedback_registry_entries_need_dispatch_arms:
- New struct fields on GpuExperienceCollector
- stream.alloc_zeros at construct site
- Kernel-launch .arg() threading at experience_env_step launch
- StateResetRegistry entries (FoldReset sentinel 0.0)
- training_loop::reset_named_state dispatch arms
- All 10 registry pin tests pass including
every_fold_and_soft_reset_entry_has_dispatch_arm
Audit doc updated: docs/dqn-wire-up-audit.md Phase A3 section.
Next phases (per spec): A4 = aux_trade_outcome_loss_reduce (sparse CE,
mask=-1), A5 = aux_trade_outcome_backward, Phase B = 262-dim input
(h_s2_aux || plan_params), Phase C = 3-slot state assembly, Phase D =
12-weight W atom-shift, Phase E = dW + Adam, Phase F = validation smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
26ce7ba690 |
feat(sp22-vNext): Phase A2 — trade-outcome label producer kernel
First foundation kernel for the H6 vNext trade-outcome aux head. Per-env classification at trade-close events into K=3 outcomes: - 0 = Profit (pnl_vs_target >= 1.0) - 1 = Stop (pnl_vs_stop >= 1.0) - 2 = Timeout (neither threshold hit) Sparse labels — most bars get -1 mask. Priority: Profit > Stop > Timeout. Pure per-env map; no atomicAdd, no reduction. Launch: grid(ceil(N/256)), block(256). Registered in build.rs. Cubin compiles clean. Currently dead code — launcher wireup comes in Phase A3+ commits. See docs/plans/2026-05-14-sp22-h6-vNext-trade-outcome-aux.md for the full vNext architecture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
d0c037a3d2 |
feat(sp22): H6 Phase 3 FALSIFIED + vNext spec (trade-outcome aux head)
Decisive smoke train-xrkb7 @
|
||
|
jgrusewski
|
ebc7144434 |
feat(sp22): H6 Phase 3 RE-ACTIVATED with corrected aux head
Smoke train-8zwtf @
|
||
|
jgrusewski
|
465abc7e9b |
fix(sp14): dial back aux head/trunk enlargement after L40S OOM
Initial 8x head + 2x trunk-H2 (commit
|
||
|
jgrusewski
|
787ee7b86c |
feat(sp14/sp22): enlarge aux head + trunk capacity (8x head, 2x trunk-H2)
After Path C confirmed aux head's 28% accuracy at H=60 is the H6 Phase 3 bottleneck (not the mechanism itself), enlarge aux capacity: - AUX_HIDDEN_DIM: 32 -> 256 (8x, matches input dim, removes bottleneck) - AUX_TRUNK_H2: 128 -> 256 (2x, uniform trunk width) Architecture changes: - Aux head: 256 -> Linear -> 256 -> ELU -> Linear -> 2 (was 256->32->2) - Aux trunk: 256 -> 256 -> 256 -> 256 (was 256 -> 256 -> 128 -> 256) - +160K params total (mostly aux_nb_w1 + aux_rg_w1: [256, 256] each) Side effects: - Checkpoint fingerprint change (intentional) - Thread utilisation improves: AUX_BLOCK=256 threads x H=256 = 1:1 (vs 8:1 at H=32 — most threads idle previously) Phase 3 mechanism stays DORMANT (W=0, beta=0) for this validation smoke. Verdict criteria: aux_dir_acc improves from 0.28 toward 0.50+ with the larger capacity. If yes, re-activate Phase 3 priors. If no, the bottleneck is signal/horizon, not capacity. Cargo build clean (full nvcc rebuild). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2c0911981a |
feat(sp22): H6 Phase 3 DORMANT - mechanism falsified, infrastructure preserved
Path C investigation revealed the aux head at epoch 1 is severely
anti-predictive at H=60 bars:
- Aux predicts UP 83% of the time
- Labels are 17% UP, 83% DOWN
- Accuracy = 28% (vs 50% random)
This means H6 Phase 3 hypothesis cannot help WR — atom-shift on an
anti-predictive signal produces no discriminative bias. Confirmed by
full-mechanism smoke at
|
||
|
jgrusewski
|
b4e26a3b45 |
feat(sp22): H6 Phase 3 α/β — RESTORE structural priors
Verification smoke train-5t6vb @ |
||
|
jgrusewski
|
79945987a5 |
fix(sp22): W-read guards across ALL atom-shift kernels
Verify smoke train-t5885 partial result: dir CLEAN, mag NaN. The isfinite(W) guard added in compute_expected_q didn't extend to the other atom-shift consumers. Adam-corrupted W propagates through them via 0*NaN=NaN. Adds isfinite(w_aux[a]) guards to all atom-shift kernels: - mag_concat_qdir - quantile_q_select - c51_loss_kernel (eq_per_action shift + effective_reward W[a0]/W[best_next_a]) Also strengthens c51_aux_dw_kernel: guard sp/isw/dz/gamma/done (not just dz<1e-7 which doesn't catch NaN). Any NaN/Inf input -> skip sample (no dW contribution). Defense-in-depth complete: NaN cannot propagate through any atom-shift path regardless of source. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
57cf5c1e08 |
fix(sp22): defensive NaN guards on Step 8/11 dW + Adam path
Bisect cut #1 (train-6zbcn) DEFINITIVELY identified Step 8/11 as the NaN source. With launches disabled, q_var/q_by_action/v_a_means all CLEAN. Forward atom-shift kernels confirmed innocent. Defensive fix (belt-and-suspenders): 1. c51_aux_dw_kernel: clamp NaN/inf total to 0 at dW write site. 2. adam_w_aux_kernel: early-return if any input non-finite. 3. compute_expected_q: guard w_aux[a] read with isfinite (in addition to state_121). Together these prevent any NaN propagation through atom-shift regardless of where the upstream NaN originated. Step 8/11 launches RESTORED in submit_adam_ops. The underlying NaN source is still unidentified but defensively neutralised. Smoke verification next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a1945de031 |
bisect(sp22): disable Step 8/11 launches (c51_aux_dw + adam_w_aux)
User-requested static-analysis-first bisect (Option 2): comment out both launch_c51_aux_dw and launch_adam_w_aux in submit_adam_ops to isolate forward vs backward as NaN source. With these disabled: - Forward atom-shift kernels still run (no-op with W=0) - W stays at [0,0,0,0] (no Adam update) - dW kernel doesn't run Hypothesis test: - Clean smoke -> backward is the bug - NaN persists -> forward is the bug Audit doc updated. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a683c4bc52 |
fix(sp22): defensive NaN guard on state_121 reads in atom-shift kernels
Root cause: IEEE-754 rule 0 * NaN = NaN defeats W=0 safety. Two smokes proved the bug propagates regardless of W magnitude — pattern identical at W=[-0.5,0,+0.5,0] (train-th8pj) and W=[0,0,0,0] (train-gs4gx). The state_121 = batch_states[i * state_dim + 121] read in 4 atom-shift kernels can contain NaN (upstream source: aux head softmax producing NaN at some rollout step, stored in replay buffer, sampled into trainer's batch). 0 * NaN = NaN propagates through all atom-shift arithmetic. Defensive fix: guard each state_121 read with isfinite check, fallback to 0 if NaN/inf. Applies to: - compute_expected_q (experience_kernels.cu) - mag_concat_qdir (experience_kernels.cu) - quantile_q_select (experience_kernels.cu) - c51_loss_batched (c51_loss_kernel.cu) — state_121 AND next_state_121 - c51_aux_dw_kernel (s121 AND ns121) This unblocks Phase 3 mechanism validation. The actual state_121 NaN source (aux head or state assembly) is to be investigated separately. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
617eb61aab |
diagnostic(sp22): zero W prior + beta prior to isolate NaN source
First smoke at
|
||
|
jgrusewski
|
ff98edc774 |
fix(sp22): H6 Phase 3 beta - activate via 0.5 structural prior (B6-minimal)
Bug: state_reset_registry has FoldReset entries for sp22_reward_aux_align_ema
and sp22_aux_align_scale, but reset_named_state dispatch had no matching arms
-> unknown name error at first fold boundary with these registry entries.
Latent in
|
||
|
jgrusewski
|
9c0aaacdfb |
docs(sp22): smoke scope — α ACTIVE β DEAD (B6 not landed)
Auditing the β reward shaping path revealed a producer gap: - experience_kernels.cu:3614-3628 correctly computes r_aux_align from scale_β = isv_signals[537] - But slot 537 has NO PRODUCER — SP11 controller still emits 6 components (N_COMPONENTS=6.0f), Phase B6 extension to 7 components hasn't landed - → scale_β stays at alloc_zeros 0 → r_aux_align = 0 → β no-op Current smoke validates α atom-shift mechanism ONLY. β remains dormant. Updated verdict outcome table: WR-shift implies α works alone; WR-stable with dist-shift implies α active but β needed (land B6 next, ~1-2hr). Phase B6 scope documented (single-kernel edit to SP11 controller + launcher update for N_COMPONENTS). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a57ec3da8b |
docs(sp22): smoke verdict template for H6 Phase 3 α
Defines verdict criteria for the train-qsltr smoke at commit
|
||
|
jgrusewski
|
3a004256a8 |
docs(sp22): H6 Phase 3 α Phase D scoping refinement
Architectural finding: eval evaluator's Q-value computation routes through
QValueProvider::compute_q_and_b_logits_to (fused_training.rs:4892) which
delegates to trainer.replay_forward_for_q_values — already atom-shift-wired
since commit
|
||
|
jgrusewski
|
5106e3b117 |
feat(sp22): H6 Phase 3 α Phase C1 — collector W ptr setter
Wires the trainer's `w_aux_to_q_dir [4]` device pointer into the GpuExperienceCollector so rollout-time action selection sees the active atom-shift for the direction branch. Changes: - gpu_experience_collector.rs: new field aux_w_to_q_dir_dev_ptr (u64, default 0) + setter set_aux_w_to_q_dir_ptr(). Both rollout launchers (compute_expected_q + quantile_q_select) now pass W ptr + exp_states_f32 + STATE_DIM_PADDED instead of NULL placeholders. NULL-safe via the kernels' existing aux_shift_active gating. - gpu_dqn_trainer.rs: w_aux_to_q_dir field promoted to pub(crate) for cross-module access via raw_ptr(). - trainers/dqn/trainer/training_loop.rs: new wire-up block after SP15 warm-count setter, mirroring the established setter pattern. NULL-safe on test scaffolds where fused_ctx or collector is absent. End-state — rollout activation: - compute_expected_q + quantile_q_select now return shifted E[Q] / quantile-blends per direction action during rollout. - With trainer's W trained each step (Step 8+11 Adam), the rollout policy's direction-action distribution actively reflects the learned aux→policy coupling. state_121's per-env value drives a per-(env, action) bias of magnitude W[a] (≤0.5 initial prior, learned thereafter). Verification: cargo check -p ml --lib clean (0 errors, 21 pre-existing warnings). Trainer + collector now smoke-ready end-to-end for the trainer/rollout side. Eval-side activation pending Phase D. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
5d163e0e1d |
feat(sp22): H6 Phase 3 α — adaptive W via dW backward + Adam (B9 Steps 8+11)
Step 8 — c51_aux_dw_kernel (new):
- Per-action block tree-reduce: grid=(4,1,1), block=(256,1,1). One block
per W index a, tree-reduces dW[a] across batch via warp shuffle + shmem.
Zero atomicAdd per pearl_no_atomicadd.
- Per-sample contributions:
a == a_d: dW[a] += inv_batch × isw × (SP_b/dz) × state_121
a == a*: dW[a] += inv_batch × isw × (-γ(1-done)) × (SP_b/dz) × next_state_121
c51_loss_kernel forward — new scratch outputs:
- aux_target_a_dir_buf[B] (i32): saves best_next_a for d==0 after Step c
sampling.
- aux_proj_logdiff_dir_buf[B] (f32): saves SP_b = Σ_n p_target_n ×
(current_lp[upper_n] - current_lp[lower_n]) after Step d's projection
via re-derivation of lower_n/upper_n (matching Huber compression +
clamp arithmetic of block_bellman_project_f).
Step 11 — adam_w_aux_kernel (new):
- Standard Adam with bias correction, grid=(1,1,1), block=(4,1,1).
- Graph-capture-safe: lr via self.lr_dev_ptr pointer arg; step via
self.ptrs.t_buf pointer arg (matches main Adam pattern). beta/eps
as value args from sp5_isv_slots constants.
- Bias-correction denominator floored at 1e-30 to avoid /0.
Trainer wiring (submit_adam_ops):
- launch_c51_aux_dw + launch_adam_w_aux added right after
launch_adam_update. Both inside the captured adam_child graph.
- New trainer fields: aux_target_a_dir_buf, aux_proj_logdiff_dir_buf,
c51_aux_dw_kernel, adam_w_aux_kernel. Cubin statics SP22_C51_AUX_DW_CUBIN
+ SP22_ADAM_W_AUX_CUBIN added.
NULL-safety:
- aux_shift_active=false in c51_loss_kernel forward → both scratch
buffers stay at alloc_zeros 0 → dW reads 0 → Adam W is a no-op.
- aux_target_a_dir_out / aux_proj_logdiff_dir_out are NULL-tolerant.
Deferred (deliberate scope):
- dL/dstate_121 backward (c51 → aux head): refinement, not correctness;
aux head trains via own supervised CE loss.
- Phase C1 collector W ptr setter.
- Phase D (eval-side aux infrastructure).
Verification:
- cargo build -p ml --lib: 0 errors, 21 pre-existing warnings.
- nvcc full recompile clean (1m05s for sm_89 target).
- All forward atom-shift consumers + adaptive W backward + Adam now wired.
End-state: adaptive W trains from structural prior [-0.5, 0, +0.5, 0]
via projection log-diff gradient. Aux head trains independently via
supervised CE. Together they form learned cross-coupling from aux
direction predictions to dir-branch Q distribution shifts. Smoke can
now measure adaptive W's effect on WR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
a98f299823 |
feat(sp22): H6 Phase 3 α — atom-shift forward-side complete (B9 Steps 7+9+10)
Steps 7+9+10 land all FORWARD-path consumers of atom positions:
Step 7 — c51_loss_kernel atom-shift threading:
- 5 new args (w_aux, batch_states, next_batch_states, aux_dir_prob_index,
state_dim). NULL-safe — any NULL collapses to 0 shift, bit-identical
to pre-Phase-3-α.
- Per-sample state_121 + next_state_121 hoisted once at top of kernel.
- eq_per_action[a] += W[a]*next_state_121 in dir branch (d==0) before
Expected SARSA target-action sampling. Biases a* toward aux-aligned.
- Bellman projection: effective_reward = reward + γ*Δ_target*(1-done)
- Δ_online substituted for reward in block_bellman_project_f call.
Math derivation: see prior commit
|
||
|
jgrusewski
|
195c051e7a |
feat(sp22): H6 Phase 3 α — atom-shift wired through compute_expected_q (B9 Step 5+6)
W structural prior init kernel (Step 5): - aux_w_prior_init_kernel.cu: 4-thread one-time init writing W[a] = [-0.5, 0.0, +0.5, 0.0] (Short / Hold / Long / Flat). Hardcoded values in kernel — no HtoD per feedback_no_htod_htoh_only_mapped_pinned.md. - build.rs registration + gpu_dqn_trainer.rs cubin static + handle field + new() launch after alloc_zeros for w_aux_to_q_dir. compute_expected_q atom-shift (Step 6): - experience_kernels.cu signature grows 4 args (w_aux, batch_states, aux_dir_prob_index, state_dim). NULL-safe — collapses to 0 shift when either pointer is NULL, bit-identical to pre-Phase-3-α. - Per-action inner loop computes aux_atom_shift = w_aux[a] * state_121 once per (b, a) for d==0 only. Inner z-loop applies z_val += aux_atom_shift before all S/TZ/TZ²/TLM accumulators consume it. Launcher updates (3 trainer sites + 1 collector site): - populate_q_out: passes W + current states (online path). - replay_forward_for_q_values: passes W + current states (online replay). - compute_denoise_target_q: passes W + next_states (target on s'; W shared across online/target). - gpu_experience_collector.rs: passes NULL W + NULL states until Phase C1 wires the trainer's W ptr through a setter. Architectural notes: - Action selection (every compute_expected_q call) now uses shifted atom positions for direction branch (d==0). Other branches stay bit-identical (shift = 0). Step 7 (c51_loss_kernel) + Step 8 (c51_grad_kernel) will close the loop on training loss + W gradient in the same atomic commit to avoid the gradient mismatch trap per feedback_no_partial_refactor.md. - Adam wireup for w_aux_to_q_dir lands at Step 11; until then W stays at structural prior values (no Adam step modifies it). Verification: - cargo check -p ml --lib: 0 errors, 21 pre-existing warnings (Phase 3b baseline parity). - Audit doc updated with B9 Step 5+6 checkpoint entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
08fd5803c4 |
refactor(sp22): H6 Phase 3 α cleanup + runbook revision for atom-shift design
Same-session cleanup of Phase 3b scalar-bias residue (commit cb80b74ce's
additions that became architecturally redundant under the 2026-05-13
atom-shift revision). The scalar-bias α design is mathematically
ineffective in C51 distributional Q-learning (softmax-shift-invariance
→ W gradient = 0 → never trains). The revised design threads
`aux_atom_shift[b, a] = W[a] * state_121[b]` through compute_expected_q
+ c51_loss_kernel + c51_grad_kernel + mag_concat_qdir; dW + dstate
gradients integrate into c51_grad_kernel's projection backward.
Files
─────
- crates/ml/build.rs:
Removed `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu`
from kernels_with_common. Replaced with comment block documenting
C51 softmax-invariance reason + spec/audit pointers.
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs:
Removed `pub(crate) static SP22_AUX_TO_Q_DIR_BIAS_CUBIN` and
`SP22_AUX_TO_Q_DIR_BIAS_BWD_CUBIN` static declarations.
Removed 3 struct fields (aux_to_q_dir_bias_kernel,
aux_to_q_dir_bias_backward_dw_kernel,
aux_to_q_dir_bias_backward_dstate_kernel) and their new()
loading blocks + struct construction entries.
KEPT: w_aux_to_q_dir + adam_m_w_aux + adam_v_w_aux + dw_aux_buf
(still needed for atom-shift Adam-trained W).
Updated W doc-comment to describe atom-shift threading (was
scalar-bias) and structural-prior initialization `[-0.5, 0.0,
+0.5, 0.0]`.
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu +
crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu:
Source files stay on disk as committed dead code (commit
|
||
|
jgrusewski
|
648078ce20 |
docs(sp22): H6 Phase 3 α spec revision — atom-position shift (not scalar Q bias)
Architectural finding during Phase 3c implementation attempt:
Original spec: α adds `W[a] * state_121[b]` to scalar Q values. Adam-
trained W via dloss/dQ gradient.
Actual codebase: C51 distributional Q-learning. Q is computed from a
probability distribution over fixed atom positions; C51 loss is KL
divergence over distributions, not MSE over scalars. Adding a constant
to logits is softmax-shift-invariant; adding to scalar expected_Q
happens post-loss-path. Either way, the KL loss is invariant under
scalar Q bias → W gradient = 0 → W never trains → α is mathematically
ineffective as originally specified.
Principled fix
──────────────
Per-(b, a) atom-position shift:
z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b] for all n
Shifts atom positions state-dependently. C51 Bellman projection
re-projects target onto shifted positions → loss depends on
`W[a] * state_121[b]` smoothly → W gradient flows from projection
arithmetic.
Implementation cost grows from ~25-35 hr to ~40-60 hr because the
atom-shift threads through every kernel that uses atom positions
for the direction branch:
- compute_expected_q (action selection)
- c51_loss_kernel (Bellman projection)
- c51_grad_kernel (backward dW + dstate from projection arithmetic)
- mag_concat_qdir (magnitude-branch conditioning Q_dir compute)
- quantile_q_select / iqn_dual_head_kernel (investigate)
The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become architecturally redundant under the revised
design. The dW + dstate gradient computations integrate into
c51_grad_kernel's existing projection backward. Phase A kernels
stay as compiled cubins (committed in
|
||
|
jgrusewski
|
cb80b74ce9 |
feat(sp22): H6 Phase 3b — α infrastructure loaded (NOT yet wired)
Phase 3b builds on Phase 3a ( |
||
|
jgrusewski
|
2e4c7ebf60 |
feat(sp22): H6 Phase 3a — 7-component contract migration + β producer (WIP)
Phase 3a builds on the Phase A foundation ( |
||
|
jgrusewski
|
464bc5f7a4 |
feat(sp22): H6 Phase 3 — Phase A foundation (WIP checkpoint, additive)
Phase A of the SP22 H6 Phase 3 implementation runbook
(docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md). Purely
additive: new ISV slots + new kernel files + build.rs registration
+ state_reset_registry entries. Nothing in the running training or
eval pipeline consumes this infrastructure yet — the existing
6-component contract is untouched. Phase A is committable as a clean
WIP foundation; Phases B-F (7-component contract migration, α
plumbing, A2 eval-side wiring, verification, smoke) resume in a
future session.
Files
─────
- crates/ml/src/cuda_pipeline/sp22_isv_slots.rs (NEW)
REWARD_AUX_ALIGN_EMA_INDEX = 536 — EMA of 7th reward component
SP22_AUX_ALIGN_SCALE_INDEX = 537 — SP11-controller scale_β
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
ISV_TOTAL_DIM bumped 536 → 538 (SP22 H6 Phase 3 +2 slots)
Layout fingerprint extended with SLOT_536 + SLOT_537 entries
- crates/ml/src/cuda_pipeline/mod.rs
pub mod sp22_isv_slots
- crates/ml/src/trainers/dqn/state_reset_registry.rs
sp22_reward_aux_align_ema (FoldReset, sentinel 0)
sp22_aux_align_scale (FoldReset, sentinel 0)
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu (NEW)
α forward: Q_dir[b, a] += W_aux[a] * state_121[b]
Reads state_121 from encoder INPUT (batch_states), bypassing
the cold encoder weight for slot 121. No atomicAdd, no host
branches, capture-safe.
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu (NEW)
Two kernels in one .cu:
- aux_to_q_dir_bias_backward_dw: per-action block tree-reduce
computing dW[a] = Σ_b state_121[b] * dq_dir[b, a]
- aux_to_q_dir_bias_backward_dstate: per-sample sum
dstate_121[b] += Σ_a W[a] * dq_dir[b, a] (option-(i) gradient
routing: both α and encoder paths get signal)
- crates/ml/build.rs
Both new cubins registered in kernels_with_common (compiled
successfully by nvcc; cubins exist at OUT_DIR/aux_to_q_dir_bias_*.cubin)
- docs/dqn-wire-up-audit.md
Phase A entry documenting the checkpoint + remaining-work
breakdown for next session.
Verification
────────────
- cargo check -p ml --features cuda: 0 errors, 21 pre-existing
warnings (Phase 2 baseline parity)
- nvcc compiles both new cubins (3.6 KB fwd, 10.6 KB bwd)
- No runtime impact: no consumer wires the new infrastructure yet
Resumes in
──────────
Future session executes Phases B-F per the runbook:
- B: 11 tasks — 7-component contract migration cascade
- C: 1 task — α collector rollout-side wiring
- D: 7 tasks — A2 eval-side aux trunk + α + state-gather
- E: 7 tasks — verification gates
- F: 5 tasks — audit doc append + atomic commit (incl. this Phase A
+ Phase B-F changes) + push + smoke + verdict
Estimated remaining: ~28-43 hr engineering + ~37 min smoke wall-clock.
Phase B partial work (300-line diff for experience_kernels.cu + reward_component_ema_kernel.cu — 7-stride migration + β producer + preamble update) saved at /tmp/sp22-h6-phase3-b-partial.patch (local-only).
Refs
────
- docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md (spec)
- docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md (runbook)
- pearl_no_partial_refactor (Phase A is additive; safe to commit alone)
- pearl_no_atomicadd (backward block tree-reduce, no atomics)
- pearl_no_host_branches_in_captured_graph (kernel capture-safety)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
f7fc83879e |
docs(sp22): H6 Phase 2 smoke verdict — FALSIFIED + Phase 3 framing
Workflow `train-bw28b` on sp20-aux-h-fixed @
|
||
|
jgrusewski
|
f9192f70a5 |
feat(sp22): H6 Phase 2 — recenter state[121] to [-1, +1] (atomic)
Phase 1 post-mortem traced an actual `pearl_first_observation_bootstrap` violation in my own H6 implementation: state slot 121 wrote `aux_softmax[env, 1] = p_up ∈ [0, 1]` with sentinel 0.5, but every OTHER state slot uses 0 as the "no signal" baseline (zero-padding, feature_mask, ofi-missing, mtf-missing). The encoder had to learn TWO things about slot 121 (directional mapping + non-zero bias offset) instead of one. Phase 2 fixes the encoding to match the project convention BEFORE declaring H6 fully falsified. Mechanism change ──────────────── - `aux_softmax_to_per_env_kernel.cu` writes `2*p_up - 1 ∈ [-1, +1]` instead of `p_up`. Still structurally bounded (softmax components in [0, 1] sum to 1). - Cold-start + FoldReset sentinel: 0.5 → 0.0 via the same pure-GPU `fill_f32` path. No HtoD per `feedback_no_htod_htoh_only_mapped_pinned`. - NULL-fallback in 3 state-gather kernels (training + backtest-per-step + backtest-chunk): 0.5f → 0.0f. - Constant + device-function comment updates to document the recentered encoding. Atomic per `feedback_no_partial_refactor`: the encoding contract spans 5 source files; partial migration produces inconsistent slot semantics between training and eval. Verification gates (all clean) ────────────────────────────── - cargo check -p ml --features cuda: 0 errors, 21 pre-existing warnings (parity with Phase 1 baseline) - gpu_backtest_validation: 4/4 expected-passing tests still pass; 2 pre-existing PnL-assertion failures bit-identical to Phase 1 (confirms recentering does not perturb scripted-policy paths) - compute-sanitizer --tool=memcheck: ERROR SUMMARY: 0 errors Smoke dispatch deferred pending an orthogonal investigation into the 2 pre-existing gpu_backtest_validation failures (stale action constants in the tests; addressed in a follow-up commit, NOT a Phase 2 regression). Verdict criteria (per spec, evaluated after smoke) ────────────────────────────────────────────────── - WR > 50.5% within 3 epochs → recentering binding, H6 + Phase 2 sufficient → justify A2. - a_var for mag/ord/urg > 1e-3 → sub-branches gradient-coupled under recentered signal. - WR pinned at 50.1–50.2% → Phase 2 falsified, pivot to amplitude scaling or deeper hypothesis. Refs ──── - docs/plans/2026-05-12-sp22-h6-phase2-recenter.md (spec) - docs/plans/2026-05-12-sp22-h6-phase2-recenter-runbook.md (this plan) - pearl_first_observation_bootstrap (sentinel = 0) - feedback_no_partial_refactor (5-file atomic) - feedback_no_htod_htoh_only_mapped_pinned (fill_f32, not HtoD) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
ce1a81552b |
docs(sp22): H6 smoke verdict — falsified at cycle 1 (50.21% WR)
Workflow train-cr9hl on sp20-aux-h-fixed @
|
||
|
jgrusewski
|
7fc9799343 |
feat(sp22): H6 Phase 1 — aux→policy state bridge (atomic)
Wires the aux head's per-env directional probability into policy STATE
slot 121 (AUX_DIR_PROB_INDEX = PADDING_START + 0), preserving the trunk-
separation invariant from `pearl_separate_aux_trunk_when_shared_starves`.
H1 (label horizon) confirmed aux learns 78% dir-acc within-fold at H=200
but the policy was walled off; this commit conducts that signal through
the state input with a one-step lag.
Mechanism (rollout-time, collector-only)
────────────────────────────────────────
- State[env, t] reads `prev_aux_dir_prob[env]` (= p_up from step t-1).
- After aux forward at step t, the new copy kernel writes
`aux_softmax[env, 1]` → `prev_aux_dir_prob[env]` for step t+1.
- Cold-start + FoldReset seed the buffer to 0.5 (neutral; p_up = 50%)
via the pure-GPU `fill_f32` kernel — no HtoD per
`feedback_no_htod_htoh_only_mapped_pinned.md`.
- Launch sits in the same `isv_signals && trainer_params != 0` gate as
the aux forward, so when aux is skipped the cache keeps its previous
(sentinel or last-good) value instead of copying stale `alloc_zeros`.
Three state-gather kernels updated atomically (per
`feedback_no_partial_refactor.md`):
- `experience_state_gather` — training, reads `aux_dir_prob_per_env[i]`
- `backtest_state_gather` — eval (single-step), NULL → 0.5 (A3 fallback)
- `backtest_state_gather_chunk` — eval (chunked), NULL → 0.5 (A3 fallback)
`assemble_state` gained a 7th param `float aux_dir_prob` written to
`out[SL_PADDING_START + 0]`; the remaining 6 padding slots stay zero
for 8-alignment.
Phase 1 scope = training-side + eval A3 NULL fallback. A2 (aux trunk
forward in eval) is deferred per the runbook — gates on whether the
smoke moves WR off the 50.1–50.2% plateau.
New files
─────────
- crates/ml/src/cuda_pipeline/aux_softmax_to_per_env_kernel.cu
Modified
────────
- crates/ml-core/src/state_layout.rs (+AUX_DIR_PROB_INDEX)
- crates/ml/src/cuda_pipeline/state_layout.cuh (+assemble_state param)
- crates/ml/src/cuda_pipeline/experience_kernels.cu
(3 state-gather kernels + NULL-defensive sentinel)
- crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
(per-env buffer + 2 kernel handles + cold-start fill + copy launch
+ FoldReset re-fill + state-gather arg)
- crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs
(NULL aux_dir_prob_per_env at all 3 launchers for A3)
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
(SP22_AUX_SOFTMAX_TO_PER_ENV_CUBIN static)
- crates/ml/src/cuda_pipeline/gpu_action_selector.rs
(EPSILON_GREEDY_CUBIN → pub(crate) so collector reuses fill_f32)
- crates/ml/build.rs (register new kernel)
- docs/dqn-wire-up-audit.md
(## 2026-05-12 — SP22 H6 implementation: Phase 1 entry)
Verification (all three gates clean, no smoke yet)
──────────────────────────────────────────────────
- cargo check -p ml --features cuda: 0 errors
- gpu_backtest_validation: 4/4 expected-passing tests still pass
(the 2 PnL-assertion failures are pre-existing per the runbook)
- compute-sanitizer --tool=memcheck: 0 CUDA errors
Refs
────
- docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md
- docs/plans/2026-05-12-sp22-h6-next-session-prompt.md
- pearl_separate_aux_trunk_when_shared_starves
- pearl_first_observation_bootstrap (sentinel = 0.5 cold-start)
- feedback_no_htod_htoh_only_mapped_pinned (fill_f32 not HtoD)
- feedback_no_partial_refactor (3 state-gather kernels atomic)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
f50a974fb6 |
docs(sp22): H1 falsified + H6 aux→policy state bridge design
H1 result (commit |
||
|
jgrusewski
|
e8814079d9 |
Revert "experiment(sp22): H1 — pin aux pred horizon at 200 bars (atomic)"
This reverts commit
|
||
|
jgrusewski
|
9adbca8262 |
experiment(sp22): H1 — pin aux pred horizon at 200 bars (atomic)
Hypothesis test for SP22 H1 (label horizon mismatch). Finding from v9/v10 HEALTH_DIAG: aux_dir_acc=28-47% (BELOW RANDOM) across all observed cycles. Root cause: adaptive aux_horizon_update collapses H back to ~1.7 bars (observed avg winning hold time), making the aux label HFT microstructure noise. Experiment: 1. Bump SENTINEL_AUX_PRED_HORIZON_BARS 60.0 → 200.0 2. Disable launch_aux_horizon_chain call so H stays at sentinel Predicted: if aux_dir_acc rises >50% → H1 confirmed; if stays ≤50% → escalate to H2/H4 per SP22 plan. Cost: 1 smoke ~30min, kill early on cycle 1-2 trend. Files changed: - crates/ml/src/cuda_pipeline/sp14_isv_slots.rs - crates/ml/src/trainers/dqn/trainer/training_loop.rs - docs/dqn-wire-up-audit.md (H1 experiment entry) Reverts if H1 falsified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
07332bf057 |
docs(sp21): close-out + v10 findings + SP22 WR plateau plan
SP21 T2.2 status: cascade wiring COMPLETE.
- All E1-E8 enrichment producers wired to consumers
- Local compute-sanitizer clean on eval-baseline closure path
- v9 cycle 1 confirms cascade live: win_conc=1.72, curric_conc=0.31,
hindsight_mag=1.47e-5
- Eval pipeline now hard-fails on GPU error, no CPU fallback,
factored-action branch sizes + default ISV all wired
v10 hypothesis test (10 epochs, killed at E8 with clear answer):
- val_Sharpe peaks at E3 (174), monotonically degrades to E8 (66, -62%)
- WR pinned at 50.1-50.2% across ALL 8 epochs (no improvement)
- PF pinned at 1.00-1.01
- Cascade controllers stable but cannot move policy off the plateau
- Verdict: 3-epoch baseline structure IS optimal; longer training
monotonically degrades. WR plateau is upstream of the cascade.
Implication: SP21 T2.2 cascade is necessary but insufficient for
project_goal_wr_55_pf_2 (WR≥55%, PF≥2.0). Need new SP arc to address
the WR=50% plateau at its source.
New plan: docs/plans/2026-05-12-sp22-wr-plateau-investigation.md
Hypotheses (priority order):
H1: Label horizon mismatch (cheapest, most likely)
H2: Action-space pathology (Hold hiding directional signal)
H3: Reward shape (no directional gradient)
H4: Feature representation gap (MTF features zero-padded)
H5: Bar resolution itself is too noisy (longshot)
Each hypothesis tested as atomic smoke with one variable changed
vs v9 baseline (peak val=174, WR=50.1%).
Phase 1 milestone: WR ≥ 51% on val for ≥ 1 fold.
Phase 2: WR ≥ 53% across 3 folds.
Phase 3: WR ≥ 55% AND PF ≥ 2.0 (SP20+ goal achieved).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
d482efc416 |
fix(sp21): T2.2 Phase 8.4-fix — winner-concentration z-score separation (atomic)
v8 smoke (train-96wfk) confirmed win_conc=0.0000 across all 9 cycles
even when PF crossed 1.0. The Phase 8.2/8.4 threshold tweaks couldn't
fix it because the underlying formula `top_mean / all_mean` is
sign-unstable around `all_mean=0`. PER alpha boost path was dark for
any cold-start policy below PF=1 — exactly the regime where the
boost matters most.
Old: if all_mean ≤ (0.01 * pnl_std).max(0.0) { return 0.0; }
top_mean / all_mean
→ guard trips at PF≤1 → 0.0 (every v8 cycle)
New: ((top_mean - all_mean) / pnl_std).max(0.0)
→ z-score separation: how many pnl_stds above the overall mean
does the top decile sit? By construction top_mean ≥ all_mean,
so separation is non-negative even when all_mean is negative.
→ bounded [0, ∞), scale-invariant, profitability-agnostic.
Expected v9 magnitudes (v8 cycle-1-like inputs):
top_mean ≈ 5e-6, all_mean ≈ 1e-7, pnl_std ≈ 1.08e-5
separation = (5e-6 - 1e-7) / 1.08e-5 ≈ 0.45
Healthy PF>1 cycles likely 0.2..1.5 range.
Pearls honoured:
- pearl_controller_anchors_isv_driven: pnl_std is the signal-driven
scale anchor (replaces hardcoded all_mean denominator)
- feedback_isv_for_adaptive_bounds: z-score formulation removes the
hardcoded multiplier dependency that motivated 8.2 and 8.4's
threshold adjustments
- feedback_no_quickfixes: structural reformulation, not a threshold
tweak (8.2 and 8.4 already showed threshold tweaks couldn't fix
the sign-instability)
Verification:
cargo check -p ml --features cuda # clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9f2d0fffb5 |
fix(sp21): T2.2 Phase 8.7 — default ISV buffer in evaluator (atomic)
v8 smoke (train-96wfk, commit
|
||
|
jgrusewski
|
cb7ace0063 |
feat(sp21): T2.2 Phase 8.6 — curriculum weights via z-score exp(-sharpe) (atomic)
v8 smoke (train-96wfk, commit
|
||
|
jgrusewski
|
5694eb4df2 |
fix(sp21): T2.2 Phase 8.5 — wire factored-action branch sizes into closure-based eval (atomic)
v7 smoke (train-fv4s8, commit
|
||
|
jgrusewski
|
5186701982 |
feat(sp21): T2.2 Phase 8.4 — v6 loose ends (win_conc / curric_conc / hindsight_mag display) (atomic)
Three targeted fixes for v6 smoke (train-x4m96) findings that Phase 8.2
left on the table. v6 cycle-by-cycle showed `win_conc=0.0000` and
`curric_conc=0.0000` pinned across all 9 cycles, and `hindsight_mag`
printed `0.0000` due to format-width rounding.
Fix 1: compute_winner_concentration guard threshold 0.1 → 0.01 × pnl_std
v6 had pnl_std ≈ 1e-5 and val-trade all_mean ≈ 1e-7..1e-6. At 0.1×
the threshold was 1e-6, still above typical all_mean → short-circuit
fired every cycle. At 0.01× (threshold 1e-7) healthy small-but-positive
policies emit a non-zero signal; degenerate-strategy guard preserved.
Fix 2: compute_curriculum_concentration formula
Was: 1 - entropy(weights) / log(n) (entropy-deficit, normalized)
Now: CV(weights) / sqrt(n − 1) (normalized coefficient of variation)
Entropy-deficit is ~weight_std² near uniform. v6's 8 contiguous segments
had per-segment Sharpes in a tight band, so normalized weights stayed
within ~0.5% of 1/n and deficit collapsed to <1e-4 (only cycle 9
reached 0.0001). CV scales linearly with weight_std/weight_mean,
dramatically more sensitive in the near-uniform regime. Cold-start
(uniform) still returns 0; one-hot returns 1.
Fix 3: hindsight_mag display format {:.4} → {:.2e}
pnl_std ≈ 1e-5 on volume bars → hindsight magnitudes are similar scale,
printed as 0.0000 under {:.4} even when the count is non-zero. Matches
pnl_std={:.2e} format already in the log.
Pearls honoured:
- feedback_isv_for_adaptive_bounds: thresholds derived from pnl_std
- pearl_first_observation_bootstrap: uniform/empty inputs → 0.0 sentinel
- pearl_controller_anchors_isv_driven: CV anchor (one-hot = sqrt(n-1))
is structural, not magic
- feedback_no_quickfixes: entropy → CV is a structural reformulation
Verification:
cargo check -p ml --features cuda # clean
Expected v8 smoke (when dispatched):
- win_conc rises to ~1.5..3.0 (top_decile_mean / all_mean ratio)
- curric_conc enters 0.05..0.20 range as segments develop differential
difficulty; grows toward ~0.4 with overfit divergence
- hindsight_mag prints as 5e-6 or similar (real value, not zero)
If these fire, Phase 7 (alpha boost via E6/E7/E8) is finally exercised
end-to-end on volume bars.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
23b89a90e9 |
fix(sp21): T2.2 Phase 8.3+9 — eval pipeline GPU-only, hard-fail, delete CPU path (atomic)
Combined Phase 8.3 (visibility + hard-fail) and Phase 9 (CPU path
removal) per pearl_no_deferrals_for_complementary_fixes. Surfaced by
v6 smoke (train-x4m96) where:
[DQN GPU] Fold 0 GPU eval failed: GpuBacktestEvaluator::evaluate failed
for fold 0. Falling back to CPU path.
[DQN] Fold 0 evaluation failed: Failed to create DQN state tensor for
bar 0: Dimension mismatch: expected 54, got 45
Both fold 0 and fold 1 hit this; workflow exited 0, masking eval
failure for every smoke run since STATE_DIM grew beyond legacy 54.
Root cause (silent GPU failure):
evaluate_dqn_fold_gpu calls evaluator.evaluate(closure, portfolio_dim: 3)
but GpuBacktestEvaluator initialises portfolio_dim = PORTFOLIO_BASE_DIM (8)
+ MTF_DIM (16) = 24 per canonical state layout. gather_states asserts
match → returns MLError::ConfigError. Caller wraps with .with_context()
+ warn!("... {}", e) — `{}` strips anyhow chain, hiding root cause.
The CPU fallback runs with a separate stale 45-dim state builder
(42 from extract_ml_features + 3 portfolio) → fails at GpuTensor::
from_host shape validation against the model's 54-feature default.
Fixes (all atomic):
1. portfolio_dim: 3 → 24 at all 4 call sites (DQN x2, PPO, supervised).
The GPU evaluator's gather_kernel handles full 128-dim state
assembly (Market 42 + OFI 32 + TLOB 16 + MTF 16 + Portfolio 8 +
PlanISV 7 + Padding 7); caller just declares correct portfolio_dim.
2. Surface anyhow chain: {} → {:#} in error messages.
3. Hard-fail on GPU eval failure: anyhow::bail! (no CPU fallback).
Per user directive: "hard fail on gpu panic, cpu path strictly
forbidden should be removed entirely!"
4. DELETE CPU DQN eval path entirely:
- fn evaluate_dqn_fold
- fn build_chunk_states (stale 45-dim state builder)
- fn simulate_chunk_trades
- fn compute_metrics + struct ComputedMetrics
- struct PortfolioState
5. DELETE coupled surrogate-noise machinery:
- struct SurrogateSampler + impl
- fn load_surrogate_marginals
- fn compute_pooled_sharpe
- Surrogate init blocks in main
- ACTION_MARGINALS / POOLED_SHARPE emission blocks
6. DELETE coupled CLI flags:
- --gpu-eval / --no-gpu-eval (GPU mandatory)
- --surrogate-mode, --surrogate-seed, --surrogate-marginals
- --emit-action-marginals, --emit-pooled-sharpe
7. Collapse `if args.gpu_eval { ... }` blocks to direct calls; cleaner
control flow, no gpu_handled tracking.
Pearls honoured:
- feedback_no_cpu_test_fallbacks: GPU oracle only
- feedback_no_partial_refactor: stale CPU layout from pre-STATE_DIM=128 era
- feedback_no_hiding: error chain now visible via {:#}
- feedback_no_legacy_aliases: no deprecated --no-gpu-eval wrapper
- pearl_no_deferrals_for_complementary_fixes: 8.3+9 combined
Files changed:
- crates/ml/examples/evaluate_baseline.rs:
−892 net lines (1066 del, 174 ins; 2817 → 1925)
- docs/dqn-wire-up-audit.md: 2026-05-12 audit entry
Verification:
- cargo check -p ml --example evaluate_baseline --features cuda # clean
- cargo check --workspace --features cuda # clean
- cargo test -p ml --lib --features cuda financials # 7/7
Note: OFI/TLOB/MTF feature-set fidelity is a separate concern. The GPU
gather_kernel handles state assembly; caller currently provides zeroed
OFI (LobBar.ofi = 0.0) and no MTF data. Eval will run, but on degraded
features. Faithful feature wiring deferred to a later Phase once
eval-runs-at-all is validated by v7 smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|