1305d6531b18307e9d1228c9a0f9505cefbd94de
1286 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
3286dc7dee |
feat(sp22-vnext): Phase C-1 — K=3 softmax → per-env 3-slot cache (producer)
First half of Phase C. Lands the producer side of the K=3 trade- outcome aux head's state bridge: new kernel populates a per-env 3-slot cache from the K=3 softmax tile every rollout step. The consumer side (state gather reading from this cache → state slots [121..124)) lands in Phase C-2. Mirrors the K=2 head's existing aux_softmax_to_per_env_kernel exactly at K=3: - K=2: prev_aux_dir_prob[env] = 2*softmax[env, 1] - 1 (recentered) - K=3: prev_aux_outcome_probs[env, k] = softmax[env, k] for k in [0, 3) Changes: - state_layout.rs: 3 new constants AUX_OUTCOME_PROFIT_INDEX = 121, AUX_OUTCOME_STOP_INDEX = 122, AUX_OUTCOME_TIMEOUT_INDEX = 123. PROFIT_INDEX aliases AUX_DIR_PROB_INDEX (same value, different semantic). Phase C-2 flips slot 121's meaning from K=2's recentered p_up to K=3's p_Profit. - aux_outcome_softmax_to_per_env_kernel.cu: new kernel + cubin. - gpu_dqn_trainer.rs: new SP22_AUX_OUTCOME_SOFTMAX_TO_PER_ENV_CUBIN embed. - gpu_experience_collector.rs: 2 new struct fields (cache buffer + kernel handle); cubin load + alloc in constructor; struct-init; per-step launch in rollout loop after K=3 forward. - build.rs: kernel registered. Encoding shift K=2 → K=3: K=2 used recentered [-1, +1] to match "no signal = 0" baseline of every other slot. K=3 keeps raw softmax probabilities [0, 1]. Cold-start sentinel 0.0 for all 3 slots = "no prediction yet" (mask). The 3-slot natural distribution is more informative than a scalar. Dead-code status: producer populates cache every step but experience_state_gather doesn't read from it yet — state slot 121 still receives K=2's prev_aux_dir_prob write. Phase C-2 swaps the state gather's source from K=2 cache to K=3 cache (3-slot write). Why split C into C-1 + C-2: experience_state_gather is a hot-path kernel with many consumers. Updating it touches training collector, eval-side backtest evaluator, Rust launcher arg list. C-2 lands that as an atomic state-semantic flip; C-1 lands the GPU-side scaffolding independently so the producer chain can be validated first. Verification: - cargo check -p ml clean. - cargo test -p ml --lib → 1016/0 green. Audit: docs/dqn-wire-up-audit.md Phase C-1 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
68f0481a9e |
feat(sp22-vnext): Phase B5a — input concat kernel scaffolding
Lands the plan-conditioning concat kernel for the K=3 trade-outcome forward as reusable scaffolding. Phase B5b (integration) is deferred with rationale: Phase C (state slots) is more critical for testing the K=3 head's effect on policy behavior, and can land independently of the plan-conditioning refinement. NEW kernel aux_to_input_concat_kernel.cu: - Writes [B, SH2+P] from h_s2_aux [B, 256] || plan_params [B, 6] - Pure GPU map; one thread per output element, no atomicAdd - NULL-tolerant on plan_params (zeros trailing P cols when source unavailable, e.g., collector cold-start where the trade plan head doesn't run) - Registered in build.rs; cubin compiles (5.7 KB). Dead code at this commit — no Rust launcher yet. Why B5 is split + B5b deferred: Full Phase B5 (integration) requires three coordinated changes: 1. Forward path: bump aux_to_fwd.forward() to SH2=262 + 262-dim input 2. Backward stride mismatch: backward emits dh_s2_aux_to_buf [B, 262], but dh_s2_aux_accum (input to aux trunk backward) is [B, 256]. A direct SAXPY mismatches row strides (262 vs 256) and corrupts the trunk's upstream gradient. Needs a strided-SAXPY kernel. 3. Collector-path plan_params unavailability: trade plan head only runs trainer-side. Workarounds: zero-fill, add trade plan to collector, or skip K=3 forward in collector. All have trade-offs. Phase B5b would need (1) strided-SAXPY kernel and (2) collector plan_params decision. Real work but NOT on the critical path for testing the K=3 head's effect on WR. Why Phase C should land first: The K=3 head currently trains on real labels (post-B4b) but doesn't influence policy behavior. Phase C wires the head's softmax into state slots [121..124) = (p_Profit, p_Stop, p_Timeout), replacing the K=2 single-slot 121 = 2*p_up - 1. WITH Phase C the policy reads aux's outcome predictions as state features → behavior changes → testable. Without Phase C, validation runs would show "K=3 head trains and converges" but predictions don't reach the policy → WR signal isn't a function of K=3 at all. We'd be testing nothing. Recommendation: skip the full B5 for now, do Phase C next, then Phase D (atom-shift). Phase B5b (plan-conditioning) is a refinement we add IF Phase C/D's no-plan-params version shows promise but plateaus below the WR ≥ 0.55 target. Audit: docs/dqn-wire-up-audit.md Phase B5a section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
da5e564ccf |
feat(sp22-vnext): Phase B4b-2 — replay-buffer scatter + trainer setter
Second half of Phase B4b. Wires the trade-outcome label column through
the replay buffer (struct field, allocation, scatter on insert, gather
on sample, direct-to-trainer pointer + setter) and connects the
trainer's aux_to_label_buf to receive per-batch sampled labels.
Completes the end-to-end producer → ring → trainer i32 path the K=3
sparse CE consumer reads.
Replay buffer changes (crates/ml-dqn/src/gpu_replay_buffer.rs):
- New struct fields: aux_outcome_labels (capacity-sized ring),
sample_aux_outcome_labels (mbs-sized fallback gather), trainer_aux_
outcome_labels_ptr (direct-path destination)
- New aux_outcome_labels_ptr field on GpuBatchPtrs
- insert_batch signature: new aux_outcome arg between aux_conf_in and
bs. Scatters via existing K-generic scatter_insert_i32 (same kernel
the K=2 aux_sign_labels uses).
- sample_proportional direct gather when trainer_aux_outcome_labels_ptr
!= 0; fallback gather otherwise. Mirrors aux_sign_labels direct/
fallback semantic exactly.
- New setter set_trainer_aux_outcome_labels_ptr mirrors
set_trainer_aux_conf_ptr.
Collector emission:
- New aux_outcome_labels field on GpuExperienceBatch
- Populated at end of collect_experiences_gpu via dtod_clone_i32 from
Phase B4b-1's per-(env, t) producer scratch.
Trainer wireup:
- aux_to_label_buf_ptr() accessor on GpuDqnTrainer (mirrors
aux_nb_label_buf_ptr)
- trainer_aux_to_label_buf_ptr() delegating accessor on
FusedTrainingCtx
- New set_trainer_aux_outcome_labels_ptr call in training_loop at the
same site where set_trainer_buffers + set_trainer_aux_conf_ptr fire.
Two call sites updated (lines ~835 + ~2857).
- insert_batch call in training_loop passes &gpu_batch.aux_outcome
_labels as new arg.
Test fixtures updated: 4 smoke test files + 3 unit-test fixtures in
gpu_replay_buffer.rs alloc zero-init aux_outcome i32 arg.
End-to-end chain complete:
trade_outcome_label_kernel (A2)
→ collector per-step launch (B3)
→ collector emission (B4b-1)
→ replay-buffer insert + scatter (B4b-2)
→ PER sample + direct gather (B4b-2)
→ trainer aux_heads_forward.loss_reduce (B4) reads sparse {-1,0,1,2}
→ trainer aux_heads_backward (B4) computes per-sample partials
→ Adam SAXPY (B1+B4) updates W1, b1, W2, b2 at [163..167)
The "degraded predict-Profit-everywhere" cold-start from Phase B4 is
resolved. K=3 head trains on real sparse trade-outcome labels.
Verification:
- cargo check -p ml clean (21 warnings, none new).
- cargo test -p ml --lib → 1016/0 on clean runs; pre-existing
NoisyLinear flake still surfaces ~30-50% of runs (unrelated to
vNext work — see
|
||
|
jgrusewski
|
491bf7d3e6 |
feat(sp22-vnext): Phase B4b-1 — per-(env, t) label kernel output + collector buffer
First half of Phase B4b (replay-buffer label scatter chain). Amends the Phase A2 trade_outcome_label_kernel to emit a per-(env, t) output column alongside the existing per-env tile, and adds the collector-side buffer + per-step launch arg. Kernel amendment (trade_outcome_label_kernel.cu): - Added NULL-tolerant `out_labels_per_sample` arg after existing `out_labels`. When non-NULL, writes `out_labels_per_sample[env*L + t] = label` at the same offset as the `trade_close_per_sample[env*L + t]` read. NULL = no-op (preserves Phase A2/A3 contract for callers passing old signature). - Pattern mirrors the K=2 head's `aux_sign_labels` per-(i, t) ring column that threads through the replay buffer. Collector field + alloc + launch: - New struct field `exp_aux_to_label_per_sample: CudaSlice<i32>` sized `[alloc_episodes × alloc_timesteps]`. Sentinel -1 (mask) populated by alloc_zeros + per-step kernel writes — survives until a trade-close event overwrites the env's slot at that t. - Updated Phase B3 launcher in collect_experiences_gpu to pass `self.exp_aux_to_label_per_sample.raw_ptr()` as the new arg. Why split B4b into B4b-1 + B4b-2: the full replay-buffer wireup mirrors the K=2 head's aux_sign_labels pattern across ~8 distinct code sites (replay-buffer struct field, sample destination buffer, direct-to-trainer pointer, setter method, scatter on insert, gather direct, gather fallback, GpuBatchPtrs field). Splitting lets us validate the per-(i, t) producer in isolation before touching the consumer pipeline. B4b-1 (this commit) = producer chain complete. Per-(env, t) column populated correctly every rollout step. Consumer wiring (replay- buffer scatter + trainer setter) is B4b-2's scope. Verification: - cargo check -p ml clean (21 warnings, none new). - cargo test -p ml --lib → 1016/0 on clean runs; pre-existing test_dqn_checkpoint_round_trip NoisyLinear flake still surfaces ~50-70% of full-suite runs (flake predates Phase B4b, unrelated to trade-outcome head — disable_noise() zeros ε but leaves some other randomness source intact). Audit: docs/dqn-wire-up-audit.md Phase B4b-1 section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
20e1aea27c |
feat(sp22-vnext): Phase B4 — trainer-side replay-batch chain wireup
Wires the trade-outcome head's forward + loss reduce + backward +
per-sample partial reduce + SAXPY into the trainer's aux_heads_forward
and aux_heads_backward methods. Adam SAXPY for the 4 new weight tensors
at [163..167) extends uniformly via the existing aux_param_specs array
iteration.
Changes to aux_heads_forward:
- Steps 7 + 8 appended after K=2 next-bar head's loss reduce
- K=3 forward reads weights at [163..167) (Phase B1) → writes to
aux_to_* save-for-backward tiles (Phase B2)
- K=3 loss_reduce writes aux_to_loss_scalar_buf + aux_to_valid_count_buf
Changes to aux_heads_backward:
- K=3 backward appended after K=5 regime backward, emits per-sample
partials (dW1, db1, dW2, db2) + per-sample dh_s2_aux
- aux_param_specs array extended from 8 → 12 entries. Per-tensor
reduce + SAXPY loop iterates uniformly, scaling each by aux_weight
- K=3 dh_s2_aux SAXPY appended after K=5's, all three heads'
gradients flow into aux trunk's dh_s2_aux_accum (encoder stop-grad
enforced structurally by aux_trunk_backward's missing dx_in output)
Label semantic (cold-start): aux_to_label_buf is alloc_zeros (all 0
= Profit) until Phase B4b lands replay-buffer label scatter. Model
trains on "predict Profit everywhere" — degraded but well-defined
(no NaN). Mirrors K=2 head's known-degraded state between B1.1a
(forward landed) and B1.1b (label producer wired).
Adam SAXPY: existing global SAXPY iterates 0..NUM_WEIGHT_TENSORS
(now 167) — 4 new weight slots get gradient SAXPYs followed by
Adam m/v updates uniformly. Architectural payoff of Phase B1's
NUM_WEIGHT_TENSORS bump.
Test flake mitigation: added bind_to_thread() to
ensemble::adapters::dqn::tests::shared_device() mirroring the
cuda_stream() test-helper pattern from the fix sweep at
|
||
|
jgrusewski
|
b28b349ac3 |
feat(sp22-vnext): Phase B3 — collector-side rollout buffers + forward chain wireup
Adds collector-side trade-outcome head: 5 struct fields + allocations
+ per-step forward + per-step label producer launches in the rollout
loop. Mirrors the K=2 next-bar head's collector wireup at K=3.
Collector struct additions:
- exp_aux_to_fwd: AuxTradeOutcomeForwardOps (3 kernel handles)
- exp_aux_to_hidden_buf [alloc_episodes × H=128] saved post-ELU
- exp_aux_to_logits_buf [alloc_episodes × K=3] saved logits
- exp_aux_to_softmax_buf [alloc_episodes × K=3] softmax tile
- exp_aux_to_label_buf [alloc_episodes] i32 sparse {-1, 0, 1, 2}
Per-step launches in collect_experiences_gpu rollout loop:
1. aux_trade_outcome_forward — launched immediately after the K=2
sibling's forward_next_bar, parallel on the same stream. Reads
exp_h_s2_aux + weights at flat-buffer indices [163..167) (Phase
B1 additions). Writes hidden/logits/softmax tiles. No consumer
yet — Phase C wires state assembly; Phase B4 wires trainer
scatter.
2. trade_outcome_label_kernel — launched immediately after
experience_env_step on the same stream, reading the save-for-
backward buffers (pnl_vs_target_at_close_per_env, pnl_vs_stop_at_
close_per_env) that env_step just wrote at segment_complete.
Stream-implicit producer→consumer ordering. Emits per-env
{-1, 0, 1, 2} labels — sparse, ~95-99% bars produce -1 (mask).
Dead-code discipline per feedback_wire_everything_up: every kernel arg
+ producer site is real wiring (not NULL placeholder) — only the
absence of consumers reading the produced tiles is "dead". The smoke
run produces softmax tiles + labels every step bit-identical to
pre-vNext baseline (no consumer = no effect on training behavior).
Phase B4 next: trainer-side replay-batch chain (forward + loss_reduce
+ backward + Adam SAXPY for the 4 new weight tensors).
Audit: docs/dqn-wire-up-audit.md Phase B3 section.
Verification:
- cargo check -p ml clean (21 warnings, none new on aux_to_*).
- cargo test -p ml --lib → 1016 passing / 0 failing (unchanged from
post-fix-sweep baseline at
|
||
|
jgrusewski
|
ebc1b15023 |
fix(tests): repair 14 pre-existing test failures across ml crate
Lib test suite was at 14 failures from accumulated layout/contract drift.
Fixed each by tracing root cause; lib suite now 1016 passing / 0 failing.
Failures fixed (test → root cause → fix):
1. sp14_isv_slots::sp20_isv_slots_reserved_510_to_520 — ISV_TOTAL_DIM
pin drifted from SP20-era 520 to current 538 via SP21/SP22 H6 bus
growth. Slot positions 510-519 still pinned. Fix: relax total-dim
check to >= 520; keep slot-position asserts tight.
2-3. gradient_budget::test_spectral_norm_all_heads_no_panic +
test_spectral_norm_constrains_operator_norm — test config used
cfg.state_dim=16 but trainer uses STATE_DIM=128 when bottleneck
off; also DuelingWeightBacking slices [2]/[3] used pre-GRN w_s2
shape, but post-GRN it's w_b_h_s1 [2*SH1, SH1] = 2048. Fix: add
s1_input_dim_for_test helper; correct slice sizes to GRN layout.
4-9. dqn::trainer::tests::test_feature_vector_to_state +
test_single_sample_batch + test_batched_action_selection +
test_batched_vs_sequential_action_selection_consistency +
test_batch_size_mismatch_larger/smaller_than_configured —
feature_vector_to_state wrapper falsely advertised "no OFI" by
signature but body required strict OFI. Contract violation —
broke 6 unit tests + hyperopt's public convert_to_state APIs.
Fix: wrapper now genuinely produces 45-dim no-OFI state; OFI-
strict callers use _with_ofi with Some(idx).
10. state_kl_monitor::observe_tracks_fire_rate_on_change — last_amp
initialized to 1.0 coincidentally equaled first observation value,
silently suppressing first fire. Test expected first observation
always fires (cold-start semantic). Fix: Option<f32> sentinel —
None means "no prior baseline" → first observe ALWAYS fires.
11. cuda_pipeline::tests::test_eval_action_select_thompson_picks_
proportionally — test launched experience_action_select kernel
with 1 missing arg (v_logits_dir from SP17 Commit C). Kernel
read garbage pointer → SIGSEGV. Also threshold 0.70 calibrated
for pre-SP17 raw-A distribution; post-SP17 sampler uses mean-zero
softmax(V + A_centered), Long still dominates ~4× but P(Long)
drops to ~0.66. Fix: add zero-filled v_logits_buf at correct
arg position; recalibrate threshold 0.70 → 0.60.
12. cuda_pipeline::tests::test_ppo_gpu_data_upload — flaked in parallel
test run with CUDA_ERROR_INVALID_CONTEXT from MappedF32Buffer's
cudaHostAlloc. cuda_stream() test helper used OnceLock to share
a CudaStream but didn't bind_to_thread on subsequent calls. CUDA
contexts are per-thread state. Fix: helper now calls bind_to_
thread() on every invocation (idempotent — mirrors trainer/
constructor.rs:111's pattern).
13. ppo::tests::test_reward_computation — test created hold action
with ExposureLevel::Flat but is_hold() matches only Hold (Flat
= close position = transaction with cost). So hold and buy got
same transaction-cost reduction; reward_hold > reward_buy assert
failed. Fix: use ExposureLevel::Hold for no-cost hold semantic.
14. training_profile::tests::test_production_profile_applies_all_
sections — pinned n_steps==5 and tau==0.005 (pre-TD(0)); production
flipped to n_steps=1, tau=0.01 per dqn-production.toml ("dense
micro-rewards cancel over n>1 bars; faster target tracking for
TD(0)"). Fix: update pins to current production values.
All 14 fixes target root causes — no #[ignore] masking. Verified:
- Lib-only run: cargo test -p ml --lib → 1016/0 (passed/failed)
- Stash-test pre-changes: 1001/15 (confirms 14 fixed atomically)
Remaining flake under `cargo test -p ml --tests`:
- ensemble::adapters::dqn::tests::test_dqn_checkpoint_round_trip —
CUDA-context-race-adjacent flake under parallel `--tests` mode;
passes in isolation and in `--lib` mode (single binary). Predates
this commit; deferred as separate triage.
Audit: docs/dqn-wire-up-audit.md "Fix sweep" section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
b98dc2730d |
feat(sp22-vnext): Phase B2 — trade-outcome trainer saved-tensor + partial buffers
Adds 11 buffer fields + 2 orchestrator ops handles (fwd + bwd) to the
trainer struct, mirroring the existing aux_nb_* / aux_partial_nb_*
pattern at K=3 instead of K=2.
Trainer struct additions:
- aux_to_fwd: AuxTradeOutcomeForwardOps (Phase B0 scaffold)
- aux_to_bwd: AuxTradeOutcomeBackwardOps
- aux_to_hidden_buf [B, H=128] saved post-ELU
- aux_to_logits_buf [B, K=3] saved logits
- aux_to_softmax_buf [B, K=3] saved softmax (3 future consumers)
- aux_to_label_buf [B] i32 sparse {-1, 0, 1, 2}
- aux_to_loss_scalar_buf [1] mean CE
- aux_to_valid_count_buf [1] B_valid for backward
- aux_dh_s2_to_buf [B, SH2] SAXPYs into dh_s2_aux_accum
- aux_partial_to_w1 [B, H, SH2] per-sample dW1
- aux_partial_to_b1 [B, H] per-sample db1
- aux_partial_to_w2 [B, K=3, H] per-sample dW2
- aux_partial_to_b2 [B, K=3] per-sample db2
Memory: aux_partial_to_w1 = 256 MB at B=2048 — identical to K=2 head's
partial size (same SH2, same H). Total new aux-to footprint ≈ 260 MB.
The existing aux_param_grad_final_buf scratch is sized to the largest
tensor across all aux heads; trade-outcome head's largest is W1 [H, SH2]
= 32,768 floats — identical to K=2/K=5 W1s. No resize needed.
Cold-start label semantics: alloc_zeros yields label 0 (Profit) for
every sample. Until the producer wires in (B3), the trainer's CE loss
treats every sample as "should have predicted Profit" — degraded but
well-defined (no NaN). Mirrors the K=2 head's known-degraded state
between B1.1a and B1.1b.
No FoldReset registration: these buffers are overwritten every batch
— no stale-state-leak risk across folds (matches the existing aux_nb_*
pattern).
Phase B3 next: collector-side rollout buffers + forward chain wireup
into collect_experiences_gpu (per-env softmax → per-(i, t) fan-out
scatter for trainer's aux_to_softmax_buf population).
Audit: docs/dqn-wire-up-audit.md Phase B2 section.
Cargo check clean (21 warnings, none new on aux_to_* fields).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
205e46c171 |
feat(sp22-vnext): Phase B1 — trade-outcome head weight tensors + Xavier init
Adds the 4 weight tensors (W1, b1, W2, b2) for the SP22 H6 vNext
trade-outcome aux head into the trainer's flat params_buf at indices
[163..167). Adam machinery (m/v moment buffers, SAXPY iteration over
0..NUM_WEIGHT_TENSORS) picks up the new tensors uniformly — no
per-tensor wiring needed.
Changes:
- NUM_WEIGHT_TENSORS bumped 163 → 167. Most of the 54 references are
&[u64; NUM_WEIGHT_TENSORS] array-size generics that resize
uniformly with the constant.
- compute_param_sizes() adds 4 new size entries:
[163] aux_to_w1 [H=128, SH2=256] = 32,768 floats
[164] aux_to_b1 [H=128] = 128 floats
[165] aux_to_w2 [K=3, H=128] = 384 floats
[166] aux_to_b2 [K=3] = 3 floats
Total: 33,283 floats = ~133 KB params, ~266 KB Adam state.
- compute_param_sizes() debug_assert updated 163 → 167.
- Xavier fan_dims added: (H, SH2) for W1, (0, 0) for biases (zero-init),
(K=3, H) for W2. Cold-start: logits ≈ 0 → softmax ≈ uniform 1/3 → no
Profit/Stop/Timeout preference per pearl_first_observation_bootstrap.
SH2 stays at 256 in this commit (mirrors K=2 head exactly). The spec's
Phase B input concat (256 → 262 with plan_params 6-dim) will re-shape
slot [163] to 128 × 262 = 33,536 floats later — small touch-up vs the
full B1 commit.
Verification:
- cargo check -p ml clean (21 warnings, none new).
- 14 pre-existing test failures stashed-verified unrelated (OFI features
missing, ISV slot count drift — independent of NUM_WEIGHT_TENSORS).
Phase B2 next: saved-tensor + per-sample partial buffers (hidden_post,
logits, softmax, valid_count, dW*_partial, dh_s2_aux_out).
Audit: docs/dqn-wire-up-audit.md Phase B1 section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
108a426c38 |
feat(sp22-vnext): Phase B0 — AuxTradeOutcome ops struct scaffolding
First Rust-side commit of Phase B for the SP22 H6 vNext trade-outcome aux head. Pure orchestrator scaffolding mirroring AuxHeadsForwardOps / AuxHeadsBackwardOps. Zero production callers — additive per the gpu_grn / aux_trunk commit-by-commit ordering convention (scaffold → trainer fields → collector wireup). Adds: - gpu_dqn_trainer.rs: 4 cubin embeds (TRADE_OUTCOME_LABEL_CUBIN, AUX_TRADE_OUTCOME_FORWARD_CUBIN, AUX_TRADE_OUTCOME_LOSS_REDUCE_CUBIN, AUX_TRADE_OUTCOME_BACKWARD_CUBIN). All #[allow(dead_code)] until B1+. - gpu_aux_heads.rs: AUX_OUTCOME_K = 3 constant (parallel to AUX_NEXT_BAR_K). - gpu_aux_heads.rs: AuxTradeOutcomeForwardOps struct holding 3 kernel handles (forward + loss_reduce + label producer). Launch methods: forward(), loss_reduce(), compute_label(). Per-env label kernel uses grid ceil(n_envs/256) — pure per-env map, no reduction. - gpu_aux_heads.rs: AuxTradeOutcomeBackwardOps struct holding 1 kernel handle (backward). Launch method: backward(). Reuses existing K-generic aux_param_grad_reduce from AuxHeadsBackwardOps — no separate reducer needed. Contract shapes mirror AuxHeadsForwardOps/AuxHeadsBackwardOps so subsequent wireup commits plug in with minimal contract drift. Phase B proper (input concat 256→262 with plan_params) becomes a small change touching only the buffer fill + W1 shape once B1-B4 land the wireup. Phase B1 next: trainer struct fields for W1/b1/W2/b2 + Adam state + Xavier init + reset registry entries. Audit: docs/dqn-wire-up-audit.md Phase B0 section. Cargo check clean (21 warnings, none new). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
9f4a25e623 |
feat(sp22-vnext): Phase A5 — aux_trade_outcome backward kernel (Phase A complete)
K=3 backward kernel that closes the forward → loss → backward chain for the trade-outcome aux head. Mirrors `aux_next_bar_backward` (K=2 sibling) line-for-line because gradient flow is K-independent: `d_logits = (softmax − one_hot)/B_valid` propagated through `Linear → ELU → Linear` chain via standard softmax-CE derivative. Per-sample partials (caller reduces via existing K-generic `aux_param_ grad_reduce` kernel): dW1_partial [B, H=128, SH2=256], db1_partial [B, H] dW2_partial [B, K=3, H], db2_partial [B, K=3] dh_s2_aux_out [B, SH2] Mask handling: labels[b] == -1 zeros the K-vector → all downstream partials zero (chain rule's multiplicative zero). All-skip batch produces valid_count=0 → d_logits=0 for every row → zero gradients across the board, no NaN. Sparse-label gradient amplification: B_valid is typically ~1-5% of nominal batch (trade-close events are rare), so inv_B = 1/B_valid is much larger than the K=2 sibling's inv_B = 1/(~B). Per-trade-close gradients have proportionally higher magnitude — correct credit assignment (rare signal speaks louder) but Phase E's Adam may need class-weighted CE or per-group LR tuning. ELU backward via post-activation identity: f'(x) = (h_post > 0) ? 1 : 1 + h_post — recovers derivative without re-evaluating x_pre. SP14 Phase C.5b separation preserved: reads h_s2_aux (aux trunk output), writes dh_s2_aux_out SAXPYing into dh_s2_aux_accum. Q's encoder structurally protected (aux_trunk_backward has no dx_in output). Phase A5 (this commit) is dead code — no Rust launcher. Phase B will land the full launcher chain (gpu_aux_heads.rs parallel ops struct, collector struct fields for W1/W2/b1/b2/Adam-state/saved-tensors/dW- partials, wireup in collect_experiences_gpu). Cubin: aux_trade_outcome_backward_kernel.cubin (24.8 KB). ═══ Phase A complete ═══ A1: ISV slots (none needed — reuses padding 121-123) A2: trade_outcome_label_kernel.cu (label producer) |
||
|
jgrusewski
|
3ddcfb8868 |
feat(sp22-vnext): Phase A4 — aux_trade_outcome loss reduce kernel
K=3 sparse cross-entropy reduce over the trade-outcome softmax tile produced by `aux_trade_outcome_forward` (Phase A3). Mirrors the K=2 sibling `aux_next_bar_loss_reduce` structurally: single-block shmem-tree reduce, two parallel partial strips (loss_numer + valid_count) reduced lockstep, fmaxf(p_tgt, 1e-30) numerical floor, fmaxf(valid, 1.0) all- skip-batch guard, valid_count_out[1] save-for-backward. Kept as SEPARATE kernel from the K=2 sibling: - Diagnostic isolation (distinct HEALTH_DIAG slot, distinct cubin in profiles for clean per-loss-source attribution) - Sparse-label semantic clarity (~95-99% mask=-1 vs ~50-100% valid for the K=2 next-bar head) - Future per-class weighting headroom (Profit/Stop/Timeout 3:1-10:1 imbalance will likely need class-weighted CE — surgical mod here without touching the K=2 head's contract) Phase A4 (this commit) is dead code — no Rust launcher yet. Phase A5 lands backward; Phase B wires the full forward→loss→backward chain. Discipline: feedback_no_atomicadd (single-block tree-reduce), feedback_ cpu_is_read_only (pure GPU), pearl_first_observation_bootstrap (sentinel 0 valid_count produces zero gradients gracefully on cold start). Audit: docs/dqn-wire-up-audit.md Phase A4 section. Cubin: aux_trade_outcome_loss_reduce_kernel.cubin (9.9 KB) compiles clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
07728f9efc |
feat(sp22-vnext): Phase A3 — aux_trade_outcome forward kernel + save-for-backward wireup
Phase A3 of the SP22 H6 vNext trade-outcome aux head (per
docs/plans/2026-05-14-sp22-h6-vNext-trade-outcome-aux.md). Two atomic pieces:
1. NEW kernel `aux_trade_outcome_forward_kernel.cu` — K=3 softmax aux head
forward (Linear → ELU → Linear → stable softmax). Mirrors
`aux_next_bar_forward` (K=2) but emits {Profit, Stop, Timeout} probs.
Saved tensors {hidden_out, logits_out, softmax_out} ready for A4 (loss
reduce) and A5 (backward). Dead code at this commit — no Rust launcher
yet. Registered in build.rs::kernels_with_common, cubin verified.
2. Save-for-backward buffers `pnl_vs_target_at_close_per_env` +
`pnl_vs_stop_at_close_per_env` ([alloc_episodes] f32 device-resident).
Producer: `experience_env_step::segment_complete` writes the trade's
realized P&L ratios vs profit_target / stop_loss at close (inline-
computed from `pre_trade_position × (raw_close − entry_price) /
(ps[PS_PLAN_PROFIT_TARGET] × prev_equity)`, symmetric-clamped to
[-2, +2] per pearl_symmetric_clamp_audit — same formula as the
sibling experience_state_gather's plan_isv[PLAN_ISV_PNL_VS_TARGET/_STOP]
slots). Consumer (eventual A4/A5 wireup): trade_outcome_label_kernel
classifies each close into {Profit, Stop, Timeout} via the ≥1.0
threshold-hit predicate.
Wireup discipline per feedback_registry_entries_need_dispatch_arms:
- New struct fields on GpuExperienceCollector
- stream.alloc_zeros at construct site
- Kernel-launch .arg() threading at experience_env_step launch
- StateResetRegistry entries (FoldReset sentinel 0.0)
- training_loop::reset_named_state dispatch arms
- All 10 registry pin tests pass including
every_fold_and_soft_reset_entry_has_dispatch_arm
Audit doc updated: docs/dqn-wire-up-audit.md Phase A3 section.
Next phases (per spec): A4 = aux_trade_outcome_loss_reduce (sparse CE,
mask=-1), A5 = aux_trade_outcome_backward, Phase B = 262-dim input
(h_s2_aux || plan_params), Phase C = 3-slot state assembly, Phase D =
12-weight W atom-shift, Phase E = dW + Adam, Phase F = validation smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
26ce7ba690 |
feat(sp22-vNext): Phase A2 — trade-outcome label producer kernel
First foundation kernel for the H6 vNext trade-outcome aux head. Per-env classification at trade-close events into K=3 outcomes: - 0 = Profit (pnl_vs_target >= 1.0) - 1 = Stop (pnl_vs_stop >= 1.0) - 2 = Timeout (neither threshold hit) Sparse labels — most bars get -1 mask. Priority: Profit > Stop > Timeout. Pure per-env map; no atomicAdd, no reduction. Launch: grid(ceil(N/256)), block(256). Registered in build.rs. Cubin compiles clean. Currently dead code — launcher wireup comes in Phase A3+ commits. See docs/plans/2026-05-14-sp22-h6-vNext-trade-outcome-aux.md for the full vNext architecture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
d0c037a3d2 |
feat(sp22): H6 Phase 3 FALSIFIED + vNext spec (trade-outcome aux head)
Decisive smoke train-xrkb7 @
|
||
|
jgrusewski
|
ebc7144434 |
feat(sp22): H6 Phase 3 RE-ACTIVATED with corrected aux head
Smoke train-8zwtf @
|
||
|
jgrusewski
|
465abc7e9b |
fix(sp14): dial back aux head/trunk enlargement after L40S OOM
Initial 8x head + 2x trunk-H2 (commit
|
||
|
jgrusewski
|
787ee7b86c |
feat(sp14/sp22): enlarge aux head + trunk capacity (8x head, 2x trunk-H2)
After Path C confirmed aux head's 28% accuracy at H=60 is the H6 Phase 3 bottleneck (not the mechanism itself), enlarge aux capacity: - AUX_HIDDEN_DIM: 32 -> 256 (8x, matches input dim, removes bottleneck) - AUX_TRUNK_H2: 128 -> 256 (2x, uniform trunk width) Architecture changes: - Aux head: 256 -> Linear -> 256 -> ELU -> Linear -> 2 (was 256->32->2) - Aux trunk: 256 -> 256 -> 256 -> 256 (was 256 -> 256 -> 128 -> 256) - +160K params total (mostly aux_nb_w1 + aux_rg_w1: [256, 256] each) Side effects: - Checkpoint fingerprint change (intentional) - Thread utilisation improves: AUX_BLOCK=256 threads x H=256 = 1:1 (vs 8:1 at H=32 — most threads idle previously) Phase 3 mechanism stays DORMANT (W=0, beta=0) for this validation smoke. Verdict criteria: aux_dir_acc improves from 0.28 toward 0.50+ with the larger capacity. If yes, re-activate Phase 3 priors. If no, the bottleneck is signal/horizon, not capacity. Cargo build clean (full nvcc rebuild). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
2c0911981a |
feat(sp22): H6 Phase 3 DORMANT - mechanism falsified, infrastructure preserved
Path C investigation revealed the aux head at epoch 1 is severely
anti-predictive at H=60 bars:
- Aux predicts UP 83% of the time
- Labels are 17% UP, 83% DOWN
- Accuracy = 28% (vs 50% random)
This means H6 Phase 3 hypothesis cannot help WR — atom-shift on an
anti-predictive signal produces no discriminative bias. Confirmed by
full-mechanism smoke at
|
||
|
jgrusewski
|
b4e26a3b45 |
feat(sp22): H6 Phase 3 α/β — RESTORE structural priors
Verification smoke train-5t6vb @ |
||
|
jgrusewski
|
79945987a5 |
fix(sp22): W-read guards across ALL atom-shift kernels
Verify smoke train-t5885 partial result: dir CLEAN, mag NaN. The isfinite(W) guard added in compute_expected_q didn't extend to the other atom-shift consumers. Adam-corrupted W propagates through them via 0*NaN=NaN. Adds isfinite(w_aux[a]) guards to all atom-shift kernels: - mag_concat_qdir - quantile_q_select - c51_loss_kernel (eq_per_action shift + effective_reward W[a0]/W[best_next_a]) Also strengthens c51_aux_dw_kernel: guard sp/isw/dz/gamma/done (not just dz<1e-7 which doesn't catch NaN). Any NaN/Inf input -> skip sample (no dW contribution). Defense-in-depth complete: NaN cannot propagate through any atom-shift path regardless of source. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
57cf5c1e08 |
fix(sp22): defensive NaN guards on Step 8/11 dW + Adam path
Bisect cut #1 (train-6zbcn) DEFINITIVELY identified Step 8/11 as the NaN source. With launches disabled, q_var/q_by_action/v_a_means all CLEAN. Forward atom-shift kernels confirmed innocent. Defensive fix (belt-and-suspenders): 1. c51_aux_dw_kernel: clamp NaN/inf total to 0 at dW write site. 2. adam_w_aux_kernel: early-return if any input non-finite. 3. compute_expected_q: guard w_aux[a] read with isfinite (in addition to state_121). Together these prevent any NaN propagation through atom-shift regardless of where the upstream NaN originated. Step 8/11 launches RESTORED in submit_adam_ops. The underlying NaN source is still unidentified but defensively neutralised. Smoke verification next. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a1945de031 |
bisect(sp22): disable Step 8/11 launches (c51_aux_dw + adam_w_aux)
User-requested static-analysis-first bisect (Option 2): comment out both launch_c51_aux_dw and launch_adam_w_aux in submit_adam_ops to isolate forward vs backward as NaN source. With these disabled: - Forward atom-shift kernels still run (no-op with W=0) - W stays at [0,0,0,0] (no Adam update) - dW kernel doesn't run Hypothesis test: - Clean smoke -> backward is the bug - NaN persists -> forward is the bug Audit doc updated. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a683c4bc52 |
fix(sp22): defensive NaN guard on state_121 reads in atom-shift kernels
Root cause: IEEE-754 rule 0 * NaN = NaN defeats W=0 safety. Two smokes proved the bug propagates regardless of W magnitude — pattern identical at W=[-0.5,0,+0.5,0] (train-th8pj) and W=[0,0,0,0] (train-gs4gx). The state_121 = batch_states[i * state_dim + 121] read in 4 atom-shift kernels can contain NaN (upstream source: aux head softmax producing NaN at some rollout step, stored in replay buffer, sampled into trainer's batch). 0 * NaN = NaN propagates through all atom-shift arithmetic. Defensive fix: guard each state_121 read with isfinite check, fallback to 0 if NaN/inf. Applies to: - compute_expected_q (experience_kernels.cu) - mag_concat_qdir (experience_kernels.cu) - quantile_q_select (experience_kernels.cu) - c51_loss_batched (c51_loss_kernel.cu) — state_121 AND next_state_121 - c51_aux_dw_kernel (s121 AND ns121) This unblocks Phase 3 mechanism validation. The actual state_121 NaN source (aux head or state assembly) is to be investigated separately. Cargo check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
617eb61aab |
diagnostic(sp22): zero W prior + beta prior to isolate NaN source
First smoke at
|
||
|
jgrusewski
|
ff98edc774 |
fix(sp22): H6 Phase 3 beta - activate via 0.5 structural prior (B6-minimal)
Bug: state_reset_registry has FoldReset entries for sp22_reward_aux_align_ema
and sp22_aux_align_scale, but reset_named_state dispatch had no matching arms
-> unknown name error at first fold boundary with these registry entries.
Latent in
|
||
|
jgrusewski
|
9c0aaacdfb |
docs(sp22): smoke scope — α ACTIVE β DEAD (B6 not landed)
Auditing the β reward shaping path revealed a producer gap: - experience_kernels.cu:3614-3628 correctly computes r_aux_align from scale_β = isv_signals[537] - But slot 537 has NO PRODUCER — SP11 controller still emits 6 components (N_COMPONENTS=6.0f), Phase B6 extension to 7 components hasn't landed - → scale_β stays at alloc_zeros 0 → r_aux_align = 0 → β no-op Current smoke validates α atom-shift mechanism ONLY. β remains dormant. Updated verdict outcome table: WR-shift implies α works alone; WR-stable with dist-shift implies α active but β needed (land B6 next, ~1-2hr). Phase B6 scope documented (single-kernel edit to SP11 controller + launcher update for N_COMPONENTS). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a57ec3da8b |
docs(sp22): smoke verdict template for H6 Phase 3 α
Defines verdict criteria for the train-qsltr smoke at commit
|
||
|
jgrusewski
|
3a004256a8 |
docs(sp22): H6 Phase 3 α Phase D scoping refinement
Architectural finding: eval evaluator's Q-value computation routes through
QValueProvider::compute_q_and_b_logits_to (fused_training.rs:4892) which
delegates to trainer.replay_forward_for_q_values — already atom-shift-wired
since commit
|
||
|
jgrusewski
|
5106e3b117 |
feat(sp22): H6 Phase 3 α Phase C1 — collector W ptr setter
Wires the trainer's `w_aux_to_q_dir [4]` device pointer into the GpuExperienceCollector so rollout-time action selection sees the active atom-shift for the direction branch. Changes: - gpu_experience_collector.rs: new field aux_w_to_q_dir_dev_ptr (u64, default 0) + setter set_aux_w_to_q_dir_ptr(). Both rollout launchers (compute_expected_q + quantile_q_select) now pass W ptr + exp_states_f32 + STATE_DIM_PADDED instead of NULL placeholders. NULL-safe via the kernels' existing aux_shift_active gating. - gpu_dqn_trainer.rs: w_aux_to_q_dir field promoted to pub(crate) for cross-module access via raw_ptr(). - trainers/dqn/trainer/training_loop.rs: new wire-up block after SP15 warm-count setter, mirroring the established setter pattern. NULL-safe on test scaffolds where fused_ctx or collector is absent. End-state — rollout activation: - compute_expected_q + quantile_q_select now return shifted E[Q] / quantile-blends per direction action during rollout. - With trainer's W trained each step (Step 8+11 Adam), the rollout policy's direction-action distribution actively reflects the learned aux→policy coupling. state_121's per-env value drives a per-(env, action) bias of magnitude W[a] (≤0.5 initial prior, learned thereafter). Verification: cargo check -p ml --lib clean (0 errors, 21 pre-existing warnings). Trainer + collector now smoke-ready end-to-end for the trainer/rollout side. Eval-side activation pending Phase D. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
5d163e0e1d |
feat(sp22): H6 Phase 3 α — adaptive W via dW backward + Adam (B9 Steps 8+11)
Step 8 — c51_aux_dw_kernel (new):
- Per-action block tree-reduce: grid=(4,1,1), block=(256,1,1). One block
per W index a, tree-reduces dW[a] across batch via warp shuffle + shmem.
Zero atomicAdd per pearl_no_atomicadd.
- Per-sample contributions:
a == a_d: dW[a] += inv_batch × isw × (SP_b/dz) × state_121
a == a*: dW[a] += inv_batch × isw × (-γ(1-done)) × (SP_b/dz) × next_state_121
c51_loss_kernel forward — new scratch outputs:
- aux_target_a_dir_buf[B] (i32): saves best_next_a for d==0 after Step c
sampling.
- aux_proj_logdiff_dir_buf[B] (f32): saves SP_b = Σ_n p_target_n ×
(current_lp[upper_n] - current_lp[lower_n]) after Step d's projection
via re-derivation of lower_n/upper_n (matching Huber compression +
clamp arithmetic of block_bellman_project_f).
Step 11 — adam_w_aux_kernel (new):
- Standard Adam with bias correction, grid=(1,1,1), block=(4,1,1).
- Graph-capture-safe: lr via self.lr_dev_ptr pointer arg; step via
self.ptrs.t_buf pointer arg (matches main Adam pattern). beta/eps
as value args from sp5_isv_slots constants.
- Bias-correction denominator floored at 1e-30 to avoid /0.
Trainer wiring (submit_adam_ops):
- launch_c51_aux_dw + launch_adam_w_aux added right after
launch_adam_update. Both inside the captured adam_child graph.
- New trainer fields: aux_target_a_dir_buf, aux_proj_logdiff_dir_buf,
c51_aux_dw_kernel, adam_w_aux_kernel. Cubin statics SP22_C51_AUX_DW_CUBIN
+ SP22_ADAM_W_AUX_CUBIN added.
NULL-safety:
- aux_shift_active=false in c51_loss_kernel forward → both scratch
buffers stay at alloc_zeros 0 → dW reads 0 → Adam W is a no-op.
- aux_target_a_dir_out / aux_proj_logdiff_dir_out are NULL-tolerant.
Deferred (deliberate scope):
- dL/dstate_121 backward (c51 → aux head): refinement, not correctness;
aux head trains via own supervised CE loss.
- Phase C1 collector W ptr setter.
- Phase D (eval-side aux infrastructure).
Verification:
- cargo build -p ml --lib: 0 errors, 21 pre-existing warnings.
- nvcc full recompile clean (1m05s for sm_89 target).
- All forward atom-shift consumers + adaptive W backward + Adam now wired.
End-state: adaptive W trains from structural prior [-0.5, 0, +0.5, 0]
via projection log-diff gradient. Aux head trains independently via
supervised CE. Together they form learned cross-coupling from aux
direction predictions to dir-branch Q distribution shifts. Smoke can
now measure adaptive W's effect on WR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
a98f299823 |
feat(sp22): H6 Phase 3 α — atom-shift forward-side complete (B9 Steps 7+9+10)
Steps 7+9+10 land all FORWARD-path consumers of atom positions:
Step 7 — c51_loss_kernel atom-shift threading:
- 5 new args (w_aux, batch_states, next_batch_states, aux_dir_prob_index,
state_dim). NULL-safe — any NULL collapses to 0 shift, bit-identical
to pre-Phase-3-α.
- Per-sample state_121 + next_state_121 hoisted once at top of kernel.
- eq_per_action[a] += W[a]*next_state_121 in dir branch (d==0) before
Expected SARSA target-action sampling. Biases a* toward aux-aligned.
- Bellman projection: effective_reward = reward + γ*Δ_target*(1-done)
- Δ_online substituted for reward in block_bellman_project_f call.
Math derivation: see prior commit
|
||
|
jgrusewski
|
7eae832f24 |
docs(sp22): H6 Phase 3 runbook — atom-shift math derivation for Steps 7+8
Derived the projection-arithmetic transformation that collapses per-(b, a) atom-position shift into a SINGLE per-sample reward adjustment: effective_reward = reward + gamma_eff * Delta_target * (1-done) - Delta_online where Delta_online = W[a_d] * state_121[b] (taken-action shift on online support) Delta_target = W[a*] * next_state_121[b] (sampled-target shift on target dist) The Bellman projection (block_bellman_project_f body) needs NO arithmetic change — only the reward arg passed in. This dramatically reduces Step 7's kernel surgery surface area. For Step 8 backward, derived the gradient paths: dL/dDelta_online = (1/dz) * sum_n p_target_n * (log_p_online[upper_n] - log_p_online[lower_n]) dL/dDelta_target = -gamma*(1-done) * dL/dDelta_online dL/dW[a_d] += dL/dDelta_online * state_121 dL/dW[a*] += dL/dDelta_target * next_state_121 dL/dstate_121[b] += dL/dDelta_online * W[a_d] dL/dnext_state_121[b] += dL/dDelta_target * W[a*] Critical constraint documented: Steps 7+8+11 MUST land atomically. Splitting Step 7 forward from Step 8 backward creates a silent gradient path through the aux head (missing dDelta_online/dnetwork_weights). Splitting Step 8 from Step 11 Adam accumulates dW without updating W — no learning. Per feedback_no_partial_refactor.md. The math now makes Step 7+8 a transcription job rather than exploration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
195c051e7a |
feat(sp22): H6 Phase 3 α — atom-shift wired through compute_expected_q (B9 Step 5+6)
W structural prior init kernel (Step 5): - aux_w_prior_init_kernel.cu: 4-thread one-time init writing W[a] = [-0.5, 0.0, +0.5, 0.0] (Short / Hold / Long / Flat). Hardcoded values in kernel — no HtoD per feedback_no_htod_htoh_only_mapped_pinned.md. - build.rs registration + gpu_dqn_trainer.rs cubin static + handle field + new() launch after alloc_zeros for w_aux_to_q_dir. compute_expected_q atom-shift (Step 6): - experience_kernels.cu signature grows 4 args (w_aux, batch_states, aux_dir_prob_index, state_dim). NULL-safe — collapses to 0 shift when either pointer is NULL, bit-identical to pre-Phase-3-α. - Per-action inner loop computes aux_atom_shift = w_aux[a] * state_121 once per (b, a) for d==0 only. Inner z-loop applies z_val += aux_atom_shift before all S/TZ/TZ²/TLM accumulators consume it. Launcher updates (3 trainer sites + 1 collector site): - populate_q_out: passes W + current states (online path). - replay_forward_for_q_values: passes W + current states (online replay). - compute_denoise_target_q: passes W + next_states (target on s'; W shared across online/target). - gpu_experience_collector.rs: passes NULL W + NULL states until Phase C1 wires the trainer's W ptr through a setter. Architectural notes: - Action selection (every compute_expected_q call) now uses shifted atom positions for direction branch (d==0). Other branches stay bit-identical (shift = 0). Step 7 (c51_loss_kernel) + Step 8 (c51_grad_kernel) will close the loop on training loss + W gradient in the same atomic commit to avoid the gradient mismatch trap per feedback_no_partial_refactor.md. - Adam wireup for w_aux_to_q_dir lands at Step 11; until then W stays at structural prior values (no Adam step modifies it). Verification: - cargo check -p ml --lib: 0 errors, 21 pre-existing warnings (Phase 3b baseline parity). - Audit doc updated with B9 Step 5+6 checkpoint entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
08fd5803c4 |
refactor(sp22): H6 Phase 3 α cleanup + runbook revision for atom-shift design
Same-session cleanup of Phase 3b scalar-bias residue (commit cb80b74ce's
additions that became architecturally redundant under the 2026-05-13
atom-shift revision). The scalar-bias α design is mathematically
ineffective in C51 distributional Q-learning (softmax-shift-invariance
→ W gradient = 0 → never trains). The revised design threads
`aux_atom_shift[b, a] = W[a] * state_121[b]` through compute_expected_q
+ c51_loss_kernel + c51_grad_kernel + mag_concat_qdir; dW + dstate
gradients integrate into c51_grad_kernel's projection backward.
Files
─────
- crates/ml/build.rs:
Removed `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu`
from kernels_with_common. Replaced with comment block documenting
C51 softmax-invariance reason + spec/audit pointers.
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs:
Removed `pub(crate) static SP22_AUX_TO_Q_DIR_BIAS_CUBIN` and
`SP22_AUX_TO_Q_DIR_BIAS_BWD_CUBIN` static declarations.
Removed 3 struct fields (aux_to_q_dir_bias_kernel,
aux_to_q_dir_bias_backward_dw_kernel,
aux_to_q_dir_bias_backward_dstate_kernel) and their new()
loading blocks + struct construction entries.
KEPT: w_aux_to_q_dir + adam_m_w_aux + adam_v_w_aux + dw_aux_buf
(still needed for atom-shift Adam-trained W).
Updated W doc-comment to describe atom-shift threading (was
scalar-bias) and structural-prior initialization `[-0.5, 0.0,
+0.5, 0.0]`.
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu +
crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu:
Source files stay on disk as committed dead code (commit
|
||
|
jgrusewski
|
648078ce20 |
docs(sp22): H6 Phase 3 α spec revision — atom-position shift (not scalar Q bias)
Architectural finding during Phase 3c implementation attempt:
Original spec: α adds `W[a] * state_121[b]` to scalar Q values. Adam-
trained W via dloss/dQ gradient.
Actual codebase: C51 distributional Q-learning. Q is computed from a
probability distribution over fixed atom positions; C51 loss is KL
divergence over distributions, not MSE over scalars. Adding a constant
to logits is softmax-shift-invariant; adding to scalar expected_Q
happens post-loss-path. Either way, the KL loss is invariant under
scalar Q bias → W gradient = 0 → W never trains → α is mathematically
ineffective as originally specified.
Principled fix
──────────────
Per-(b, a) atom-position shift:
z_n_effective[b, a, n] = z_n_base + W[a] * state_121[b] for all n
Shifts atom positions state-dependently. C51 Bellman projection
re-projects target onto shifted positions → loss depends on
`W[a] * state_121[b]` smoothly → W gradient flows from projection
arithmetic.
Implementation cost grows from ~25-35 hr to ~40-60 hr because the
atom-shift threads through every kernel that uses atom positions
for the direction branch:
- compute_expected_q (action selection)
- c51_loss_kernel (Bellman projection)
- c51_grad_kernel (backward dW + dstate from projection arithmetic)
- mag_concat_qdir (magnitude-branch conditioning Q_dir compute)
- quantile_q_select / iqn_dual_head_kernel (investigate)
The Phase A `aux_to_q_dir_bias_kernel.cu` + `aux_to_q_dir_bias_backward_kernel.cu` become architecturally redundant under the revised
design. The dW + dstate gradient computations integrate into
c51_grad_kernel's existing projection backward. Phase A kernels
stay as compiled cubins (committed in
|
||
|
jgrusewski
|
cb80b74ce9 |
feat(sp22): H6 Phase 3b — α infrastructure loaded (NOT yet wired)
Phase 3b builds on Phase 3a ( |
||
|
jgrusewski
|
2e4c7ebf60 |
feat(sp22): H6 Phase 3a — 7-component contract migration + β producer (WIP)
Phase 3a builds on the Phase A foundation ( |
||
|
jgrusewski
|
464bc5f7a4 |
feat(sp22): H6 Phase 3 — Phase A foundation (WIP checkpoint, additive)
Phase A of the SP22 H6 Phase 3 implementation runbook
(docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md). Purely
additive: new ISV slots + new kernel files + build.rs registration
+ state_reset_registry entries. Nothing in the running training or
eval pipeline consumes this infrastructure yet — the existing
6-component contract is untouched. Phase A is committable as a clean
WIP foundation; Phases B-F (7-component contract migration, α
plumbing, A2 eval-side wiring, verification, smoke) resume in a
future session.
Files
─────
- crates/ml/src/cuda_pipeline/sp22_isv_slots.rs (NEW)
REWARD_AUX_ALIGN_EMA_INDEX = 536 — EMA of 7th reward component
SP22_AUX_ALIGN_SCALE_INDEX = 537 — SP11-controller scale_β
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
ISV_TOTAL_DIM bumped 536 → 538 (SP22 H6 Phase 3 +2 slots)
Layout fingerprint extended with SLOT_536 + SLOT_537 entries
- crates/ml/src/cuda_pipeline/mod.rs
pub mod sp22_isv_slots
- crates/ml/src/trainers/dqn/state_reset_registry.rs
sp22_reward_aux_align_ema (FoldReset, sentinel 0)
sp22_aux_align_scale (FoldReset, sentinel 0)
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_kernel.cu (NEW)
α forward: Q_dir[b, a] += W_aux[a] * state_121[b]
Reads state_121 from encoder INPUT (batch_states), bypassing
the cold encoder weight for slot 121. No atomicAdd, no host
branches, capture-safe.
- crates/ml/src/cuda_pipeline/aux_to_q_dir_bias_backward_kernel.cu (NEW)
Two kernels in one .cu:
- aux_to_q_dir_bias_backward_dw: per-action block tree-reduce
computing dW[a] = Σ_b state_121[b] * dq_dir[b, a]
- aux_to_q_dir_bias_backward_dstate: per-sample sum
dstate_121[b] += Σ_a W[a] * dq_dir[b, a] (option-(i) gradient
routing: both α and encoder paths get signal)
- crates/ml/build.rs
Both new cubins registered in kernels_with_common (compiled
successfully by nvcc; cubins exist at OUT_DIR/aux_to_q_dir_bias_*.cubin)
- docs/dqn-wire-up-audit.md
Phase A entry documenting the checkpoint + remaining-work
breakdown for next session.
Verification
────────────
- cargo check -p ml --features cuda: 0 errors, 21 pre-existing
warnings (Phase 2 baseline parity)
- nvcc compiles both new cubins (3.6 KB fwd, 10.6 KB bwd)
- No runtime impact: no consumer wires the new infrastructure yet
Resumes in
──────────
Future session executes Phases B-F per the runbook:
- B: 11 tasks — 7-component contract migration cascade
- C: 1 task — α collector rollout-side wiring
- D: 7 tasks — A2 eval-side aux trunk + α + state-gather
- E: 7 tasks — verification gates
- F: 5 tasks — audit doc append + atomic commit (incl. this Phase A
+ Phase B-F changes) + push + smoke + verdict
Estimated remaining: ~28-43 hr engineering + ~37 min smoke wall-clock.
Phase B partial work (300-line diff for experience_kernels.cu + reward_component_ema_kernel.cu — 7-stride migration + β producer + preamble update) saved at /tmp/sp22-h6-phase3-b-partial.patch (local-only).
Refs
────
- docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md (spec)
- docs/plans/2026-05-13-sp22-h6-phase3-alpha-beta-runbook.md (runbook)
- pearl_no_partial_refactor (Phase A is additive; safe to commit alone)
- pearl_no_atomicadd (backward block tree-reduce, no atomics)
- pearl_no_host_branches_in_captured_graph (kernel capture-safety)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
9056d461d8 |
docs(sp22): H6 Phase 3 implementation runbook — 36 tasks across 6 phases
Bite-sized task plan derived from the Phase 3 spec (docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md). Tasks grouped into six phases; all EDIT tasks land in a single atomic commit at Phase F per `feedback_no_partial_refactor`. There are NO intermediate commits across Phase A–E; each task's cargo-check step is the in-flight gate. Phase A (5 tasks): foundation — new ISV slots (REWARD_AUX_ALIGN_EMA_INDEX + SP22_AUX_ALIGN_SCALE_INDEX), state_reset_registry entries, new alpha forward + backward kernels, build.rs registration. Phase B (11 tasks): 7-component contract migration — experience_kernels.cu (producer + beta), backtest_env_kernel.cu (eval producer + beta), reward_component_ema_kernel.cu (0..7 iteration), reward_decomp_diag_kernel.cu (column 6), reward_component_mag_ratio_compute_kernel.cu (axis 6), reward_subsystem_controller_kernel.cu (7-weight emit + w_aux_align anchor), health_diag_kernel.cu (snap slot), gpu_experience_collector.rs (7-slot buffer + HEALTH_DIAG print), gpu_dqn_trainer.rs (7-slot buffer + HEALTH_DIAG + alpha param + Adam + captured graph wiring), reward_component_monitor.rs (reader), training_loop.rs (sp11_reward + reward_split print lines). Phase C (1 task): alpha plumbing collector-side / rollout-time captured graph (Phase C is otherwise folded into Phase B Task B9). Phase D (7 tasks): A2 eval-side — aux trunk weight loading + per-window prev_aux_dir_prob buffer + per-step aux trunk forward + softmax-to-per-env launch + alpha forward + non-NULL state-gather wiring across 3 launchers + W ptr sharing trainer ↔ evaluator. Phase E (7 tasks): verification gates — cargo check, gpu_backtest_validation, compute-sanitizer, captured-graph integrity (deferred to smoke), alpha weight movement (deferred to smoke), 7-component contract sanity (deferred to smoke), A2 eval-side aux activity (deferred to validation eval). Phase F (5 tasks): audit doc append (Invariant 7), single atomic commit covering all changes, push, smoke dispatch, verdict monitoring + audit doc verdict. Total: 36 tasks. Each task has exact file paths, exact commands with expected output, and code-block-level specificity for kernel signatures, struct field additions, and HEALTH_DIAG format-string extensions. Self-review section maps every spec requirement to a specific task. Refs ──── - docs/plans/2026-05-12-sp22-h6-phase3-alpha-beta.md (spec) - pearl_no_partial_refactor (single atomic commit Phase F) - feedback_push_before_deploy (Phase F3) - feedback_no_redundant_monitor (Phase F5 single-channel waiter) - feedback_kill_runs_on_anomaly_quickly (verdict criteria) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
88318ddf8e |
docs(sp22): H6 Phase 3 spec — include A2 eval-side wiring (per "do eval too")
Folded A2 (eval-side α + β + aux trunk forward) into the Phase 3 spec
per user directive. Brings eval pipeline to production parity with
training-time rollout.
A2 components (6 sub-items)
───────────────────────────
A2.1 — Aux trunk weight loading at eval init (shared-source-of-truth
with trainer, mirroring existing GEMM weight pattern).
A2.2 — Per-window `prev_aux_dir_prob` buffer at eval ([n_windows] f32,
fill_f32(0.0) init + per-evaluate() reset).
A2.3 — Aux trunk forward at eval per-step (reuse `AuxHeadsForward`
orchestrator if API permits; fall back to thin eval-only
variant if too training-coupled).
A2.4 — α at eval (post-Q_dir bias launch, reuses training kernel).
A2.5 — β at eval (mirror of training producer in
`backtest_env_kernel.cu::segment_complete`; same NULL-fallback
semantics; reads scale_beta from training-emitted ISV slot).
A2.6 — Wire `prev_aux_dir_prob.raw_ptr()` (non-NULL) into all 3
backtest state-gather launchers.
7-component contract migration EXTENDED
───────────────────────────────────────
Added `backtest_env_kernel.cu` to the atomic 7-component migration
list (eval-side reward stride 6 → 7 alongside training-side). All
reward-component consumers across training AND eval paths migrate
in one commit per `feedback_no_partial_refactor`.
Scope estimate updated
──────────────────────
~34–47 hr engineering (5–6 working days), up from ~21–31 hr in the
within-phase-follow-ups version. Bulk of addition is A2.1–A2.3
greenfield work — the eval pipeline had no aux infrastructure
before.
Verification gates expanded (gate 7 added)
──────────────────────────────────────────
7. **A2 eval-side aux activity check**: post-cycle-1 validation eval
shows non-zero `r_aux_align` in eval-side WindowMetrics reward
decomposition. If eval r_aux_align ≈ 0 while training-side > 0 →
A2 wiring failure. gpu_backtest_validation tests should still
pass; potential tolerance adjustment for extended metrics
(CVaR/Omega) if β shifts numerics meaningfully; the four
directional tests remain bit-identical because constant_action_model
bypasses Q-network and β is no-op at test-time (sentinel-zero
scale_β since tests don't run SP11 controller).
Out of scope (sole remaining)
─────────────────────────────
Only the aux-trunk-gradient-flow-back-through-state[121] item, which
is a property statement (preserved by H6 design's stop-grad), not a
deferral.
Refs
────
- pearl_separate_aux_trunk_when_shared_starves (A2.1 aux trunk source-
of-truth pattern)
- pearl_no_partial_refactor (7-component migration includes eval-side
backtest_env_kernel atomically)
- pearl_no_deferrals_for_complementary_fixes (combined plan now
spans training + eval)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
3fc67ab9ba |
docs(sp22): H6 Phase 3 spec — fold within-phase follow-ups into scope
Per the directive "no follow ups include in spec", expanded the
Phase 3 design to include three previously-out-of-scope items:
1. β as a separate 7th reward component (not added to r_trail)
────────────────────────────────────────────────────────────
Contract change from `reward_components_per_sample[6]` →
`reward_components_per_sample[7]`. Migrated atomically per
`feedback_no_partial_refactor` across every consumer:
- experience_kernels.cu (producer site at segment_complete)
- reward_component_ema_kernel.cu (iterate 0..7)
- reward_decomp_diag_kernel.cu (emit column 6)
- reward_component_mag_ratio_compute_kernel.cu (axis 6)
- reward_subsystem_controller_kernel.cu (7-weight output)
- gpu_experience_collector.rs + gpu_dqn_trainer.rs (buffer alloc
+ HEALTH_DIAG print format)
- reward_component_monitor.rs (7-component EMA reader)
- health_diag_kernel.cu (snap layout)
- training_loop.rs (sp11_reward + reward_split lines)
New ISV slot REWARD_AUX_ALIGN_EMA_INDEX for the component EMA.
2. SP11 controller emits scale_β adaptively
─────────────────────────────────────────
New ISV slot `SP22_AUX_ALIGN_SCALE_INDEX`. Producer:
reward_subsystem_controller_kernel extended to 7 weight outputs.
Anchor: REWARD_AUX_ALIGN_EMA_INDEX. Target: ~10% of total reward
magnitude. Bounds [0.05, 2.0]. Cold-start sentinel 0 per
`pearl_first_observation_bootstrap` — β no-op until first emit.
state_reset_registry registers both slots as FoldReset category.
3. Encoder-state[121] gradient routing — DECISION committed
─────────────────────────────────────────────────────────
Option (i) — both paths open. `dbatch_states[121] +=` flows
gradient into both α's W and the encoder's column for slot 121.
Long-term robust: encoder may eventually learn additively (belt
+ suspenders). Empirical observable: compare W magnitudes vs
encoder column-norm for state[121] in post-smoke telemetry.
Explicitly OUT of scope (with reasoning, not deferral)
──────────────────────────────────────────────────────
A2 (eval-side α + β + aux trunk forward at eval): the same
architectural deferral logic from H6 Phase 1. Adding aux
infrastructure to GpuBacktestEvaluator should follow positive
training-side evidence rather than precede it. A3 NULL-fallback at
eval makes both α and β no-ops at eval-time, isolating the verdict
to training-side. Becomes a separate spec if Phase 3 confirms.
Scope estimate updated
──────────────────────
~21–31 hr engineering (roughly 3-4 working days) — up from the
earlier "~10–16 hr" YAGNI estimate. The bulk of the addition is the
7-component contract migration touching ~8 kernel/source files
atomically and the SP11 controller extension.
Verification gates expanded
───────────────────────────
6. **7-component contract sanity** (smoke cycle 1): HEALTH_DIAG
sp11_reward shows 7 weights including w_aux; reward_split shows 7
components including aux_align; both non-zero within the first
epoch after the SP11 controller's first emit. If still zero,
either the controller emit is broken OR the β producer isn't
firing.
Refs
────
- pearl_no_deferrals_for_complementary_fixes (combined plan)
- pearl_no_partial_refactor (7-component atomic contract)
- pearl_controller_anchors_isv_driven (SP11 scale_β anchor)
- pearl_first_observation_bootstrap (cold-start sentinel 0)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
754aea6995 |
docs(sp22): H6 Phase 3 spec — combined α (bypass-head) + β (event-driven aux bonus)
Phase 2 verdict gave mechanistic evidence that the encoder's weights
for state[121] are essentially unused (action distribution
bit-identical across Phase 1 and Phase 2 encodings; drift < 0.5% on
every action bin). The aux signal reaches state[121] (`pred_tanh =
0.66`) but the policy ignores it because slot 121's encoder weights
need many training steps to grow from zero.
Phase 3 routes aux around the cold-encoder bottleneck via two
non-overlapping mechanisms (combined per
`pearl_no_deferrals_for_complementary_fixes`):
α (post-encoder bypass-head)
────────────────────────────
- New learnable param `W_aux_to_Q_dir: [4]` f32 (one weight per
direction action). Adam states `m`, `v` ([4] each). 12 floats total.
- Forward (rollout + training graphs, post-Q_dir):
Q_dir[b, a] += W_aux_to_Q_dir[a] * state_121[b]
- Backward (training only):
dW[a] = Σ_b state_121[b] * dQ_dir[b, a] (per-action block reduce)
dstate_121[b] += Σ_a W[a] * dQ_dir[b, a] (per-sample sum)
- New cubins:
aux_to_q_dir_bias_kernel.cu (forward)
aux_to_q_dir_bias_backward_kernel.cu (backward)
- Zero-init; Phase-3 step-0 output identical to Phase-2 baseline.
Adam-driven growth tracks aux-Q correlation.
- Reads state_121 from the encoder INPUT (batch_states[121]), bypassing
the encoder entirely.
β (event-driven aux-aligned trade-close bonus)
──────────────────────────────────────────────
At trade-close (existing segment_complete branch in
experience_env_step):
position_sign = sign(prev_position)
aux_at_close = state_121[env]
alignment = max(0, aux_at_close * position_sign) ∈ [0, +1]
profit_term = max(0, realized_pnl) ∈ [0, +∞)
r_aux_align = scale_beta * alignment * profit_term
Added to existing r_trail (no new reward component for first test —
YAGNI). scale_beta = 0.5 fixed. Non-negative-only (no anti-alignment
penalty; let r_trail's loss carry the negative signal). Event-driven
per `pearl_event_driven_reward_density_alignment`. Bounded by
`pearl_one_unbounded_signal_per_reward` (realized_pnl is the single
unbounded multiplicand, which IS the natural per-trade scale).
Why combined
────────────
α: direct forward-pass channel (aux shifts Q in real-time).
β: indirect training-signal channel (aux-aligned profitable trades
get bigger gradient → encoder learns to use state[121] faster, AND
the policy learns to take aux-aligned trades).
Decision points captured (with rationale)
─────────────────────────────────────────
| Question | Decision | Rationale |
| α scope | 4-weight vector | aux is directionally signed |
| β formula | non-negative only | avoid confusing policy when aux wrong |
| β scale | fixed 0.5 | minimum scope for verdict experiment |
| state[121] grad routing | both paths open | simplest; α dominates initially |
| β placement | added to r_trail | YAGNI; 7th component deferred |
| W sharing | trainer holds, collector reads | mirrors GEMM pattern |
Verification (all required clean before commit)
───────────────────────────────────────────────
- cargo check -p ml --features cuda (0 errors, 21 pre-existing
warnings expected — parity with Phase 2 baseline)
- gpu_backtest_validation (6/6 pass; post-Phase-2-test-fix baseline)
- compute-sanitizer --tool=memcheck (0 errors)
- Captured-graph integrity check (CAPTURE_PHASE_FORWARD_DONE +
parent compose visible in smoke logs)
- α weight movement check (W_aux_to_Q_dir non-zero after epoch 1)
Verdict criteria
────────────────
- WR > 50.5% within 3 epochs → confirmed → justify A2.
- W_aux_to_Q_dir non-zero magnitudes → α gradient flowed.
- Action distribution diverges from Phase 1/2 baselines → policy is
exercising different actions.
- a_var [m, o, u] > 1e-3 → sub-branches gradient-coupled.
- WR pinned + W non-zero + action distribution moved → bypass routing
is working but aux signal isn't actionable on current fixtures →
pivot to deeper hypothesis (per-branch IQN tau audit, reward
density rewrite, longer horizon).
- WR pinned + W near-zero → gradient flow broken; kernel debug.
Estimated effort: ~10–16 hr engineering + ~37 min smoke wall-clock.
Refs
────
- docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md (Phase 1)
- docs/plans/2026-05-12-sp22-h6-phase2-recenter.md (Phase 2)
- pearl_no_deferrals_for_complementary_fixes (combined plan)
- pearl_event_driven_reward_density_alignment (β rationale)
- pearl_one_unbounded_signal_per_reward (β bound)
- pearl_first_observation_bootstrap (Phase 2 sentinel = 0)
- pearl_no_host_branches_in_captured_graph (kernel discipline)
- pearl_separate_aux_trunk_when_shared_starves (aux trunk untouched)
- pearl_no_atomicadd (backward reduce discipline)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
f7fc83879e |
docs(sp22): H6 Phase 2 smoke verdict — FALSIFIED + Phase 3 framing
Workflow `train-bw28b` on sp20-aux-h-fixed @
|
||
|
jgrusewski
|
f9192f70a5 |
feat(sp22): H6 Phase 2 — recenter state[121] to [-1, +1] (atomic)
Phase 1 post-mortem traced an actual `pearl_first_observation_bootstrap` violation in my own H6 implementation: state slot 121 wrote `aux_softmax[env, 1] = p_up ∈ [0, 1]` with sentinel 0.5, but every OTHER state slot uses 0 as the "no signal" baseline (zero-padding, feature_mask, ofi-missing, mtf-missing). The encoder had to learn TWO things about slot 121 (directional mapping + non-zero bias offset) instead of one. Phase 2 fixes the encoding to match the project convention BEFORE declaring H6 fully falsified. Mechanism change ──────────────── - `aux_softmax_to_per_env_kernel.cu` writes `2*p_up - 1 ∈ [-1, +1]` instead of `p_up`. Still structurally bounded (softmax components in [0, 1] sum to 1). - Cold-start + FoldReset sentinel: 0.5 → 0.0 via the same pure-GPU `fill_f32` path. No HtoD per `feedback_no_htod_htoh_only_mapped_pinned`. - NULL-fallback in 3 state-gather kernels (training + backtest-per-step + backtest-chunk): 0.5f → 0.0f. - Constant + device-function comment updates to document the recentered encoding. Atomic per `feedback_no_partial_refactor`: the encoding contract spans 5 source files; partial migration produces inconsistent slot semantics between training and eval. Verification gates (all clean) ────────────────────────────── - cargo check -p ml --features cuda: 0 errors, 21 pre-existing warnings (parity with Phase 1 baseline) - gpu_backtest_validation: 4/4 expected-passing tests still pass; 2 pre-existing PnL-assertion failures bit-identical to Phase 1 (confirms recentering does not perturb scripted-policy paths) - compute-sanitizer --tool=memcheck: ERROR SUMMARY: 0 errors Smoke dispatch deferred pending an orthogonal investigation into the 2 pre-existing gpu_backtest_validation failures (stale action constants in the tests; addressed in a follow-up commit, NOT a Phase 2 regression). Verdict criteria (per spec, evaluated after smoke) ────────────────────────────────────────────────── - WR > 50.5% within 3 epochs → recentering binding, H6 + Phase 2 sufficient → justify A2. - a_var for mag/ord/urg > 1e-3 → sub-branches gradient-coupled under recentered signal. - WR pinned at 50.1–50.2% → Phase 2 falsified, pivot to amplitude scaling or deeper hypothesis. Refs ──── - docs/plans/2026-05-12-sp22-h6-phase2-recenter.md (spec) - docs/plans/2026-05-12-sp22-h6-phase2-recenter-runbook.md (this plan) - pearl_first_observation_bootstrap (sentinel = 0) - feedback_no_partial_refactor (5-file atomic) - feedback_no_htod_htoh_only_mapped_pinned (fill_f32, not HtoD) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
cffc95152c |
docs(sp22): H6 Phase 2 implementation runbook
Bite-sized 12-task runbook for the recentering fix specified in `docs/plans/2026-05-12-sp22-h6-phase2-recenter.md`. Tasks 1-5 are the 5 atomic edits (kernel write recentered to `2*p - 1`, sentinels 0.5 → 0.0 across host + 3 NULL-fallback kernels, plus comment updates in state_layout.rs / state_layout.cuh). Tasks 6-8 are the three verification gates (cargo check / gpu_backtest_validation / compute-sanitizer) identical to H6 Phase 1 — all required clean before commit. Task 9 appends the H6 Phase 2 entry to the audit doc (Invariant 7 satisfaction). Task 10 is the single atomic commit per `feedback_no_partial_refactor` covering all 5 source files + audit doc. Task 11 dispatches the 3-epoch baseline smoke on ci-training-l40s. Task 12 monitors for the cycle-1 verdict using the same single-channel grep-anchored bg waiter pattern as Phase 1 (per `feedback_no_redundant_monitor`). Every task has exact file paths, complete code blocks for edits (showing old + new strings for Edit-tool consumption), exact commands with expected output, and explicit kill criteria for verification gate failures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
7883c5ca1b |
docs(sp22): H6 Phase 2 spec — recenter state[121] to [-1, +1]
Post-mortem of the H6 Phase 1 smoke (WR = 50.21% verdict in `docs/dqn-wire-up-audit.md`) traced an actual pearl violation in the H6 implementation itself: state slot 121 uses the [0, 1] range (`aux_softmax[env, 1] = p_up`) with sentinel 0.5, while every OTHER state slot uses 0 as the "no signal" baseline (zero-padding, feature_mask, ofi-missing, mtf-missing). Per `pearl_first_observation_bootstrap`: "sentinel = 0; first observation replaces directly." The H6 design violated this for slot 121 alone. The encoder must learn TWO things about slot 121: (1) the directional mapping AND (2) the appropriate bias offset for the non-zero baseline — every other dim is single-step (mapping only). Phase 2 is the simplest possible amplification fix: rewrite the bridge to use the same convention as every other state slot. If the encoder can't gradient-couple even after this fix, H6 is truly falsified and we pivot to amplitude scaling or deeper hypothesis. Change scope (atomic per `feedback_no_partial_refactor`): - aux_softmax_to_per_env_kernel.cu: write `2*p_up - 1` instead of `p_up` - gpu_experience_collector.rs: cold-start + FoldReset sentinel 0.5 → 0.0 - experience_kernels.cu: NULL-fallback in 3 state-gather kernels 0.5f → 0.0f - state_layout.rs / state_layout.cuh: comment updates to reflect [-1, +1] range and 0 sentinel Estimated effort: ~45 min walltime (edits + verification gates) + ~25–40 min smoke wall-clock. Verdict criteria (cycle 1 WR + a_var [d/m/o/u] in HEALTH_DIAG[0]): - WR > 50.5% → recentering binding, H6 + Phase 2 sufficient → justify A2 - a_var for mag/ord/urg > 1e-3 → sub-branches learning under recentered signal - WR pinned at 50.1–50.2% → Phase 2 falsified, pivot to amplitude scaling or deeper hypothesis Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
ce1a81552b |
docs(sp22): H6 smoke verdict — falsified at cycle 1 (50.21% WR)
Workflow train-cr9hl on sp20-aux-h-fixed @
|
||
|
jgrusewski
|
7fc9799343 |
feat(sp22): H6 Phase 1 — aux→policy state bridge (atomic)
Wires the aux head's per-env directional probability into policy STATE
slot 121 (AUX_DIR_PROB_INDEX = PADDING_START + 0), preserving the trunk-
separation invariant from `pearl_separate_aux_trunk_when_shared_starves`.
H1 (label horizon) confirmed aux learns 78% dir-acc within-fold at H=200
but the policy was walled off; this commit conducts that signal through
the state input with a one-step lag.
Mechanism (rollout-time, collector-only)
────────────────────────────────────────
- State[env, t] reads `prev_aux_dir_prob[env]` (= p_up from step t-1).
- After aux forward at step t, the new copy kernel writes
`aux_softmax[env, 1]` → `prev_aux_dir_prob[env]` for step t+1.
- Cold-start + FoldReset seed the buffer to 0.5 (neutral; p_up = 50%)
via the pure-GPU `fill_f32` kernel — no HtoD per
`feedback_no_htod_htoh_only_mapped_pinned.md`.
- Launch sits in the same `isv_signals && trainer_params != 0` gate as
the aux forward, so when aux is skipped the cache keeps its previous
(sentinel or last-good) value instead of copying stale `alloc_zeros`.
Three state-gather kernels updated atomically (per
`feedback_no_partial_refactor.md`):
- `experience_state_gather` — training, reads `aux_dir_prob_per_env[i]`
- `backtest_state_gather` — eval (single-step), NULL → 0.5 (A3 fallback)
- `backtest_state_gather_chunk` — eval (chunked), NULL → 0.5 (A3 fallback)
`assemble_state` gained a 7th param `float aux_dir_prob` written to
`out[SL_PADDING_START + 0]`; the remaining 6 padding slots stay zero
for 8-alignment.
Phase 1 scope = training-side + eval A3 NULL fallback. A2 (aux trunk
forward in eval) is deferred per the runbook — gates on whether the
smoke moves WR off the 50.1–50.2% plateau.
New files
─────────
- crates/ml/src/cuda_pipeline/aux_softmax_to_per_env_kernel.cu
Modified
────────
- crates/ml-core/src/state_layout.rs (+AUX_DIR_PROB_INDEX)
- crates/ml/src/cuda_pipeline/state_layout.cuh (+assemble_state param)
- crates/ml/src/cuda_pipeline/experience_kernels.cu
(3 state-gather kernels + NULL-defensive sentinel)
- crates/ml/src/cuda_pipeline/gpu_experience_collector.rs
(per-env buffer + 2 kernel handles + cold-start fill + copy launch
+ FoldReset re-fill + state-gather arg)
- crates/ml/src/cuda_pipeline/gpu_backtest_evaluator.rs
(NULL aux_dir_prob_per_env at all 3 launchers for A3)
- crates/ml/src/cuda_pipeline/gpu_dqn_trainer.rs
(SP22_AUX_SOFTMAX_TO_PER_ENV_CUBIN static)
- crates/ml/src/cuda_pipeline/gpu_action_selector.rs
(EPSILON_GREEDY_CUBIN → pub(crate) so collector reuses fill_f32)
- crates/ml/build.rs (register new kernel)
- docs/dqn-wire-up-audit.md
(## 2026-05-12 — SP22 H6 implementation: Phase 1 entry)
Verification (all three gates clean, no smoke yet)
──────────────────────────────────────────────────
- cargo check -p ml --features cuda: 0 errors
- gpu_backtest_validation: 4/4 expected-passing tests still pass
(the 2 PnL-assertion failures are pre-existing per the runbook)
- compute-sanitizer --tool=memcheck: 0 CUDA errors
Refs
────
- docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md
- docs/plans/2026-05-12-sp22-h6-next-session-prompt.md
- pearl_separate_aux_trunk_when_shared_starves
- pearl_first_observation_bootstrap (sentinel = 0.5 cold-start)
- feedback_no_htod_htoh_only_mapped_pinned (fill_f32 not HtoD)
- feedback_no_partial_refactor (3 state-gather kernels atomic)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
ff9ec76f18 |
docs(sp22): H6 implementation runbook + next-session prompt
Two new plan docs to enable clean session handoff: 1. docs/plans/2026-05-12-sp22-h6-aux-policy-state-bridge.md Detailed 10-step implementation runbook for H6 (aux→policy state bridge). Each step has file paths, kernel signatures, expected diff, and risk callouts. Estimated ~5hr for Phase 1 (training-side + A3 eval fallback). A2 (eval-side aux integration) is +1 day follow-up if H6 confirmed. 2. docs/plans/2026-05-12-sp22-h6-next-session-prompt.md Concise context-loading prompt to paste into next session. Includes state of the world, runbook pointer, verification gates, and start-here pointer. Why staged: H6 implementation crosses kernel-level state-layout contract (every consumer of STATE_DIM must migrate atomically per feedback_no_partial_refactor). Doing it RIGHT requires careful incremental verification with compute-sanitizer between steps — not safely batched into a tail-end session. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |