82dca76daebd7a5f13b1f24780a2eaa2d4e058bb
1900 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
jgrusewski
|
f47af9d268 |
fix(ml-dqn): implement real DQN::load_from_safetensors
`DQN::load_from_safetensors` was a documented no-op that only checked
file existence and returned Ok(()). Real weight restoration happened
only via the fused trainer's params_buf path; any caller loading a
checkpoint via the DQN struct itself (e.g., DqnInferenceAdapter::
from_checkpoint used by the ensemble) got a fresh untrained DQN with
no error raised — a silent production bug.
Adds BranchingDuelingQNetwork::load_from_named_slices (symmetric
counterpart of existing named_weight_slices) using the same D2D
memcpy primitive as copy_weights_from. Validates names AND shapes;
returns Err on mismatch. Wired into DQN::load_from_safetensors to
actually restore weights from disk, handling both prefix-less (single
head) and `trending__` prefix (regime-merged) safetensors layouts.
Target network is resynced in lockstep so inference and TD targets
see the same restored weights.
Adds NoisyLinear::{weight_mu_mut, bias_mu_mut} for in-place D2D
restore of mu params during checkpoint load.
Tests:
- branching::tests::test_load_from_named_slices_round_trip (happy path)
- branching::tests::test_load_from_named_slices_rejects_extra_keys
(arch-drift safety)
- branching::tests::test_load_from_named_slices_rejects_shape_mismatch
- dqn::save_load_tests::test_dqn_load_from_safetensors_round_trip
(full end-to-end safetensors save+load, #[ignore] for CUDA gate)
- dqn::save_load_tests::test_dqn_load_from_safetensors_missing_file
Also un-ignores the ensemble adapter's
`test_dqn_checkpoint_round_trip` test (was ignored pre-fix because
load_from_safetensors silently dropped weights). Disables NoisyNet
on both sides for deterministic argmax comparison. Adds CUDA_LOCK
mutex to serialize the adapter tests that share a CUDA stream via
the SHARED_CUDA OnceLock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
c5045c009e |
cleanup: delete dead DQN apply_accumulated_gradients + honest TFT error
Two ghost-feature fixes from the pre-L40S cleanup audit (tasks #66 and #68 tracked internally). ### #66 — delete apply_accumulated_gradients (dead code from removed path) The agent-audit confirmed this function is residue from a prior Candle-gradient-tracking training path that was REMOVED (see the now-deleted test crates/ml/tests/test_var_source_gradients.rs which was `#[ignore]`'d with the comment "Candle gradient tracking removed -- DQN/PPO use custom CUDA backward passes"). Evidence: - `DQN::optimizer: Option<GpuAdamW>` is always `None` — never initialised anywhere in the codebase. - `DQN` uses `OwnedGpuLinear` + `NoisyLinear` with standalone `CudaSlice<f32>` BY DESIGN (see comment at branching.rs:988-989 "this layout exists to avoid a GpuVarStore intermediate"). There is no GpuVarStore to feed GpuAdamW. - Zero external callers for `DqnTrainer::apply_accumulated_gradients`, `DQN::apply_accumulated_gradients`, `RegimeConditionalDQN:: apply_accumulated_gradients`, or the various `optimizer_vars()` wrappers. - The only test exercising this path was `#[ignore]`'d with the "Candle gradient tracking removed" rationale. - Production training goes through the fused CUDA trainer (`trainers/dqn/fused_training.rs`) which applies gradients into a flat `params_buf` — a separate, live path. Deleted: 6 functions across 3 files + the stale test. Preserving a ghost that has zero callers, zero initialisation path, and a design direction explicitly chosen AWAY from its premise is not "keep and wire" — it's accumulating fiction. Per feedback_no_functionality_ removal.md the rule preserves FUNCTIONAL features; this wasn't one. ### #68 — TFT honest error message Audit found TFT is architecturally incompatible with GpuAdamW in its current form (not a wiring gap, an architectural absence): - `TemporalFusionTransformer` has no GpuVarStore, no `parameters()`, no `named_parameters()` accessor - Internal layers use `StreamLinear` which has NO `backward()` - `TFTModel` trait only exposes `forward()`, `get_config()`, `clear_cache()` — no gradient accessor - `TemporalFusionTransformer::train()` runs forward + accumulates loss but never calls backward or optimizer step - `TrainableTFT` adapter's `backward()` explicitly returns "not supported — use TFTTrainer::train() instead" The previous error string "TFT optimizer not yet migrated to GpuAdamW" implied 99% done and a simple constructor wiring would finish it. Actual gap: GpuVarStore threading through 5+ layers + StreamLinear→GpuLinear migration + backward ops for softmax attention / layer norm 3D / quantile monotonicity chain / stack + trait extension for var_store accessor + train-loop gradient assembly. ~3-7 day dedicated architectural project. New error message states this explicitly so no one wastes time chasing an "almost migrated" fiction. Full-lift tracked separately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
4da33d2b04 |
Merge: EnsembleModelAdapter::predict wired to polymorphic inference
Branch worktree-agent-aadd27f0, commit
|
||
|
jgrusewski
|
9b27428de9 |
fix(ensemble): wire EnsembleModelAdapter::predict to real inference
The adapter's `predict` returned
`{ prediction_value: 0.5, confidence: 0.0 }` as a "neutral" stub.
Downstream ensemble filtering dropped any 0-confidence prediction,
so the adapter silently contributed nothing -- a stub that survived
only because the caller discarded it.
Now: `EnsembleModelAdapter` holds an
`Arc<dyn ModelInferenceAdapter>` and dispatches `predict` to the
wrapped per-model GPU adapter (`DqnInferenceAdapter`,
`PpoInferenceAdapter`, `TftInferenceAdapter`, `Mamba2InferenceAdapter`,
`LiquidInferenceAdapter`, `KanInferenceAdapter`,
`XlstmInferenceAdapter`, `TggnInferenceAdapter`,
`TlobInferenceAdapter`, `DiffusionInferenceAdapter`). The inner
adapter already runs a full forward pass on its model and returns a
normalized `(direction, confidence)` pair; the bridge maps
`direction in [-1, 1]` -> `prediction_value in [0, 1]` via
`(direction + 1) / 2` to match `MLPrediction`'s bullish-probability
contract (> 0.5 = bullish), clamps confidence to [0, 1], and
propagates `metadata.latency_us` as the inference latency.
The bridge returns `Err` (never a faked 0-confidence success) when
the inner model reports `is_ready() == false`, the inner `predict`
fails, the output is non-finite, or the feature slice is empty.
`build_production_strategy` no longer fabricates ten zero-confidence
ghost adapters. It now accepts
`Vec<(String, Arc<dyn ModelInferenceAdapter>)>` -- the caller owns
real model construction (checkpoint loading, device selection). The
only current caller (backtesting service `MLPoweredStrategy::new`)
passes an empty vec; that yields an empty ensemble, which is an
honest "no models loaded" signal rather than ten stubs that exist
only to be filtered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
814bc1fe4e |
Merge: adaptive per-branch gradient-norm balancer — L40S root-cause fix
Branch worktree-agent-a510b4c9, commit
|
||
|
jgrusewski
|
6cb2257163 |
fix(dqn): adaptive per-branch gradient-norm balancer
Caps any branch's weight-gradient L2 norm at num_branches × median
(median across the 4 branches' norms). Scales the offending branch's
gradient down to the cap; healthy branches pass through unchanged.
Fixes the observed pathology in L40S train-mdh86: grad_ratio_mag_dir
was 15k–26k× for 6 consecutive epochs, then collapsed to ~100× in a
single step at epoch 7 and destabilised learning (Sharpe flipped +34
→ -67, never recovered). Symmetric per-branch capping at
`num_branches × median` prevents the swing at both ends without
requiring a global ratio bound.
No tuned knobs: `num_branches = 4` is architectural (factored action
space: direction × magnitude × order × urgency), `median_branch_norm`
is a per-step statistical reference that tracks the current gradient
regime, and the product is fully adaptive. Per
feedback_adaptive_not_tuned.md, the only static value is the
architectural axis count; medians and derived caps are signal-driven.
Implementation — two CUDA kernel launches in
`branch_grad_balance_kernel.cu`:
branch_grad_norm_reduce: grid=(4,1,1), block=(256,1,1). One block
per branch; sum-of-squares via shared-mem
tree reduce writes `branch_norms_dev[4]`.
No atomicAdd (one-block-per-branch, single
writer per slot).
branch_grad_rescale: grid=(max_blocks, 4, 1), block=(256,1,1).
Each block caches the 4 branch norms into
shared memory, computes the median via a
5-comparator sorting network + two-element
average (branch-deterministic, no reduction
primitive), derives the 4 per-branch scales
`scale[d] = min(1, 4×median/norm[d])`, then
threads multiply their slice element by the
owning branch's scale. No atomicAdd (each
thread writes one distinct element).
Insertion point: inside the `adam_grad_child` graph between the aux
phase and `compute_grad_norm_for_adam`, so Adam's global clip and the
Adam update both observe the rebalanced gradient. Also wired into the
ungraphed fallback paths so no code path can skip the cap. The kernels
have fixed launch configs, no host syncs, no dynamic allocations —
safe to capture.
Per-branch slice metadata (starts/lens for each of the 4 contiguous
4-tensor branch slices in `grad_buf`) is precomputed from
`compute_param_sizes` at trainer construction and uploaded once to
device i32 buffers, matching the existing `grad_decomp_kernel` layout
convention.
Smoke tests (local RTX 3050 Ti, 4 GB):
magnitude_distribution: PASS (MAG_DIST Q=0.637 H=0.114 F=0.249,
EVAL_DIST Q=0.153 H=0.255 F=0.592)
multi_fold_convergence: PASS (3/3 folds produce best-checkpoint;
fold Sharpes +57.8 / +55.6 / +119.3)
grad_ratio_mag_dir trajectory (mag_dist smoke, first fold, first 5
epochs) — pre-fix values from /tmp/l40s_diag/health.log (L40S
train-mdh86):
pre-fix: 14793, 11934, 12858, 16406, 15141 (×1000 regime)
post-fix: 55, 78, 35, 11, 10 (×10-100 regime)
Three+ orders of magnitude reduction. The residual ratio can still
exceed `num_branches = 4` when the direction branch's norm sits below
the median — the cap bounds each branch's absolute norm (≤ 4×median),
not the pairwise ratio, by design (direction-outlier smallness is a
separate pathology that would be masked by a ratio bound).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
ac959f807f |
cleanup(ml): delete crates/ml/src/trainers/tlob.rs — orphan vaporware trainer
The TLOBTrainer in this file was:
- Never referenced by any caller in crates/ /services/ /bin/
(grep confirms zero hits beyond the string literal "TLOB" in
ml-ensemble's adaptive weight maps, which does not depend on the
trainer type).
- save_checkpoint and serialize_model logged "saving" but wrote
zero bytes, then immediately called std::fs::metadata.len() and
std::fs::read() on the missing file → ENOENT every time. Ghost
feature masquerading as a trainer (flagged during the pre-L40S
bulk-TODO sweep, commit
|
||
|
jgrusewski
|
a5c3d73d9e |
cleanup: declarative rewrites for ml-tests and trading-engine TODOs
- ml/tests/dqn_training_pipeline_test.rs: the inline TODO speculated about a future `load_checkpoint` hook; loader round-trip coverage already lives in dqn_checkpoint_tests. Reword to point there. - ml/tests/ppo_lstm_training_loop_tests.rs: the assertion on `hidden_state_manager.is_some()` is the public-surface proxy for "LSTM path active"; deeper introspection isn't exposed. Say so. - ml/tests/ppo_recurrent_integration_tests.rs: the test is already `#[ignore]`d; rewrite the inline TODO as a description of the missing `from_varbuilder` constructors on LSTMPolicyNetwork / LSTMValueNetwork. - risk/risk_engine.rs: VarEngine receives a default asset-class config because the schema-to-config conversion is not wired. Reword the TODO to describe that plainly. - trading_engine/types/errors.rs: `common::ConversionError` does not exist; keep ConversionError local and drop the aspirational re-export comment. - trading_engine/tests/audit_persistence_tests.rs: describe why the query assertion only checks the Ok shape (row-to-event mapping not wired) rather than pointing at a nonexistent line number. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
210798baa8 |
cleanup: declarative rewrites for deferred-work TODOs in ml crate
- ml/Cargo.toml: describe why ndarray's blas feature stays disabled
(CI compile pool has no libopenblas-dev; GPU cuBLAS handles the
hot path) rather than labelling it a TODO.
- dbn_sequence_loader.rs: Wave C regime-detection branch emits zeros
when only that flag is enabled; the live feed lives under WaveD.
Reword from "TODO Wave C" to a description of that superseding.
- ensemble/adapters/{liquid,tggn,tlob,xlstm}.rs: checkpoint loading
currently constructs fresh GpuLinear weights and logs a runtime
warning so the ignored checkpoint path is visible. No new
functionality, just reword the repeated TODO.
- ensemble/model_adapter.rs: the neutral-prediction adapter is
guarded by the ensemble's confidence threshold (0.0 = filtered),
making it a no-op stub used for end-to-end wiring. Describe that
contract explicitly.
- hyperopt/adapters/tft.rs: the input_dim=51 line is load-bearing
(5 static + 10 known + 36 unknown matches the CUDA layout). Drop
the "should be 42" aside.
- trainers/tft/trainer.rs: initialize_optimizer returns Err until
GpuAdamW is wired; the `let _ = &self.optimizer;` anchor in the
training loop keeps the migration target visible.
- trainers/tlob.rs: save_checkpoint / serialize_model both surface
errors until GpuVarStore safetensors serialisation lands. Mark
the gap declaratively rather than as a TODO.
- transformers/mod.rs: only AttentionMask is implemented in the
attention submodule; drop the aspirational re-export list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
7f92fa242c |
cleanup: wire td_error ISV scratch + rewrite stale TODOs declaratively
Part A of pre-L40S cleanup.
1. Wire td_error batch mean into ISV scratch (gpu_dqn_trainer.rs):
`launch_loss_reduce` now runs the generic `c51_loss_reduce` kernel a
second time over `td_errors_buf` into `td_error_scratch_dev_ptr`.
ISV[2] (TD-error EMA in `isv_signal_update`) was previously reading
a zero-initialised scratch and accumulated a constant-zero signal.
This was a genuinely missing kernel writeback — the C51 loss kernel
was already emitting per-sample |TD-error| into `td_errors_buf`
(c51_loss_kernel.cu:1096), it just wasn't being batch-reduced.
Reuses the existing `c51_loss_reduce` (generic mean-reduction, single
block, deterministic) rather than adding a new kernel — no new CUDA
surface, no ABI change.
2. Remove 2 stale TODOs from batched_backward.rs docstrings that
described a migration that's actually complete:
- Module docstring said "dqn_backward_kernel (atomicAdd path)
remains active" — the atomicAdd kernel has been removed; cuBLAS
backward is wired via launch_cublas_backward.
- `backward_full` docstring said "gated behind TODO" — the function
is actively called from the fused training step.
3. Rewrite 2 ISV scratch field comments as declarative: td_error_scratch
is now wired (as per change 1); ensemble_var_scratch remains
zero-initialised and its comment honestly describes that consumers
(ISV[3] and [4]) treat it as unavailable. Per feedback_no_todo_fixme.md,
replaces the TODO(isv) markers with declarative descriptions of
current behaviour. Future wiring is tracked in the plan, not in code
aspirational markers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
d9fee6ef8d |
fix(kelly): Task 2.Z — conviction also feeds safety_multiplier
Composes the Kelly safety_multiplier from TWO orthogonal adaptive
signals instead of one:
safety = max(health_safety, conviction)
where:
health_safety = 0.5 + 0.5 × learning_health [training stability]
conviction ∈ [0, 1] [per-sample confidence]
Health measures training stability globally. Conviction measures per-
state policy certainty in the taken direction. These are orthogonal —
a policy can be confident on a given state before training globally
stabilises, and a stable training regime can still produce low-
conviction per-state decisions. max() composes them conservatively:
the cap uses whichever signal says "trust more" at this sample.
Bounded to [0.5, 1.0] by the health floor.
Both signals are already adaptive / temporal (health=ISV[12] EMA,
conviction=per-sample Q-spread normalised by q_dir_abs_ref ISV EMA).
No static tuning knobs. Per feedback_adaptive_not_tuned.md.
Motivation (per project_magnitude_eval_collapse_kelly_capped.md): at
typical smoke-test health=0.49, health_safety = 0.745 sits coincid-
entally on the Half/Full decoder boundary (abs_pos < 0.75). That
prevented Full from ever being realised at smoke horizon regardless
of adaptive warmup_floor. Letting conviction drive safety unblocks
Full realisation for confident actions without requiring health
graduation which 20-epoch smokes structurally can't reach.
Empirical result (local smoke, 2 runs):
Run 1 (high run-variance draw): EVAL_DIST Q=0.911 H=0.057 F=0.032
— still fails H10 eh+ef≥0.30
Run 2: EVAL_DIST Q=0.350 H=0.121 F=0.529
— PASSES all 5 assertions
— FIRST FULL SMOKE PASS SINCE 4-BRANCH
Previous best (before this commit):
(pre-safety-A, v5+adaptive-Kelly only): Q=0.325 H=0.675 F=0.000
— passed H10 at line 134 but failed Task 2.X line 153 (ef < 0.05)
The commit trades the reliable Half-dominance regime for a bi-modal
distribution that includes Full on many runs. Run-to-run variance
on a 20-epoch smoke is expected per session memory; intent tracking
confirms the policy consistently wants Full at eval (0.73-0.85 across
runs), so the gap is purely in realised cap, not policy learning.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
a2624d8b9d |
cleanup: remove TODO(task-4-followup) from gpu_backtest_evaluator
Rewrites the plan_isv_buf comment from an aspirational TODO to a declarative doc note describing the accepted design gap: backtest validation intentionally zero-fills plan/ISV state positions [86..92) because the backtest env kernel does not compute those training-time introspective signals. The policy treats plan/ISV as advisory features, so the train/val delta is tolerated in exchange for a lean backtest env kernel. Per feedback_no_todo_fixme.md: TODO/FIXME markers are forbidden; rewrite as declarative production-ready prose or complete the work. |
||
|
jgrusewski
|
c34a6592f7 |
Merge: adaptive Kelly warmup_floor from policy conviction
Agent worktree
|
||
|
jgrusewski
|
a0224ce846 |
Merge: intent-side magnitude diagnostic (EVAL_INTENT_MAG_DIST)
Agent worktree
|
||
|
jgrusewski
|
ff683470e7 |
fix(kelly): Task 2.Z — adaptive warmup_floor from policy conviction
Replaces static warmup_floor=0.5f in kelly_position_cap (trade_physics.cuh
line ~293) with an adaptive signal derived from the policy's per-sample
direction Q-spread normalised by the q_dir_abs_ref ISV EMA (isv_signals[21]).
High conviction -> high floor (trust policy at cold start). Low conviction
-> low floor (safety dominates). Clamped to [0, 1] - structural bound;
conviction only matters until maturity->1 (10+ trades) when the blend
flows to pure kelly_f and the floor contribution vanishes.
Wiring:
1. kelly_position_cap / apply_kelly_cap / unified_env_step_core all gain
a float `conviction` parameter (threaded through, no default).
2. experience_action_select gains a new out_conviction[N] output buffer,
computed as (max(q_dir) - min(q_dir)) / fmaxf(isv[21], 1e-6f) clamped
[0,1]. Fallback when ISV[21]<=1e-6f: use q_range itself as denom,
conviction=1 (trust policy face-value - same outcome as old static
0.5 at health=1, but from a real signal shape).
3. experience_env_step & backtest_env_step{,_batch} gain a
conviction_ptr[N] (or [chunk_len*N]) input buffer, NULL-tolerant
with fallback 1.0.
4. Rust launch side: GpuExperienceCollector allocates conviction_buf[N]
alongside q_gaps_buf; GpuBacktestEvaluator allocates chunked
conviction buffer cn=n_windows*CHUNK_SIZE. Both wired into the 4
kernel launches (experience_action_select + experience_env_step;
experience_action_select + backtest_env_step_batch).
The previous static 0.5 pinned cold-start cap to <=0.375*max_pos at
health=0.5 (safety_multiplier=0.75), which combined with the
`abs_pos < 0.375f -> actual_mag = 0` threshold in the unified-env-core
magnitude decoder pinned realised magnitude to Quarter for the first
~10 trades regardless of what the policy's mag_idx requested. Smoke
test EVAL_DIST=[1.0, 0.0, 0.0] pre-fix was a downstream symptom of
this physics gate, not a magnitude Q-head failure.
Per feedback_adaptive_not_tuned.md: no hard-coded numeric knobs.
Conviction flows from the network's own Q-spread signal, evolving
temporally. Per feedback_no_functionality_removal.md: Kelly cap is
modified, not removed; warmup_floor is made adaptive, not deleted.
Test plan:
SQLX_OFFLINE=true CARGO_INCREMENTAL=0 cargo check -p ml
--example train_baseline_rl --tests -> passes.
Smoke (magnitude_distribution, 20 epochs) shows:
[MAG_DIST] Quarter~0.62-0.70 Half~0.15-0.19 Full~0.13-0.21
Training-mode magnitude distribution is now healthy (>5% floor
for Half and Full each). Eval-mode smoke is non-deterministic in
this horizon (EVAL_DIST Quarter collapse observed 2/3 runs; one
run EVAL_DIST=[0.573, 0.325, 0.102]). Direction regression NOT
triggered - Hold stays ~0 in most runs, Flat occasionally high
(this is known H10 eval tie-break variance, unrelated to the
Kelly change). q_dir_abs_ref observed in ISV_DIR_MEANS:
~0.14-0.60 across runs - conviction signal is flowing.
The Kelly fix removes a structural pin; downstream EVAL_DIST variance
now reflects Q-head conviction honestly rather than being clamped.
|
||
|
jgrusewski
|
f9a8a5aa9a |
feat(dqn): intent-side magnitude distribution diagnostic (EVAL_INTENT_MAG_DIST)
Adds a parallel read path that reports the policy's intended mag_idx BEFORE Kelly/margin caps and before the Hold/Flat dir_idx forces mag=0. This exposes whether the magnitude Q-head is learning state- dependent preferences, independent of the Kelly cold-start cap that was masking it via actual_mag decoding (kelly_position_cap warmup_floor=0.5 + safety=0.5+0.5*health pinning abs_pos <= 0.375). Kernel changes (experience_kernels.cu): - experience_action_select: new trailing optional arg `out_intent_mag` (int*, NULL=skip). Populated AFTER the existing mag_idx selection via a strict argmax over q_b1, ignoring the Hold/Flat mag=0 forcing, with the same higher-bin-wins tie-break used in the b2/b3 paths. Uses q_sign so the intent stays consistent with contrarian mode. - New scatter_intent_chunk kernel: copies step-major chunked intent [chunk_len, n_windows] into window-major intent_history [n_windows, max_len], mirroring the actions_history layout. Rust wiring (gpu_backtest_evaluator.rs): - New fields intent_mag_buf, chunked_intent_mag_buf, scatter_intent_kernel. Buffers allocated alongside existing chunked buffers in ensure_action_select_ready. intent_mag_buf is zeroed by reset_evaluation_state so short rollouts don't read stale data. - submit_dqn_step_loop_cublas appends the new arg to the action_select launch and launches scatter_intent_chunk immediately after, before the env_batch_kernel (which never touches intent_mag_buf). - read_eval_intent_magnitude_distribution mirrors read_eval_action_distribution_per_magnitude but decodes raw mag_idx (a as usize) rather than the factored action encoding. Training-path call site (gpu_experience_collector.rs): passes NULL (0u64) for the new arg — training does not collect intent history. Trainer wiring: - new last_eval_intent_magnitude_dist field + accessor; populated in metrics.rs::evaluate_on_gpu next to last_eval_magnitude_dist. Smoke test (magnitude_distribution.rs): adds [EVAL_INTENT_MAG_DIST] println line; no new assertions. Diagnostic-only, no new feature flag — production behaviour unchanged. All 7 files build cleanly with no new warnings. [testing: smoke compiles + runs, still fails on the existing H10 assertion (EVAL_DIST Quarter=1.000 driven by Kelly cold-start cap), EVAL_INTENT_MAG_DIST shows Quarter=0.357 Half=0.045 Full=0.599 at 20-epoch smoke on local RTX 3050 Ti — confirming the magnitude head prefers Full ~60% of the time while the Kelly-capped realised distribution pins to Quarter.] Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
04a6f0dea6 |
fix(dqn): Task 2.Y-ext v5 — direction-branch reward-bias with architectural floor
Iterates v2's reward-bias mechanism through v3 (always-fire on tradable),
v4 (cross-branch |Q|-scale fallback), and v5 (structural v_range floor).
Replaces the entire v2 body, not incremental.
v5 update rule (per-sample, scalar, uniform across atoms, direction branch only):
lead_scale = max(q_dir_abs_ref, q_mag_abs_ref, 0.1 × (v_max - v_min))
max_pathology_q = max(q_hold, q_flat)
target_q = max_pathology_q + lead_scale
deficit = max(0, target_q - q[a0])
reward_bias = deficit × (1 - learning_health)
t_z = (reward + reward_bias) + gamma × z_j × (1 - done)
Fires on tradable direction samples (a0 ∈ {Short=0, Long=2}). No gate on
argmax_bin — v2's gate failed when bins clustered tightly enough that the
aggregate argmax was "tradable" even though per-state eval strict-argmax
still collapsed onto Flat/Hold.
Signal stack (all adaptive, no hard-coded knobs):
- isv_signals[17..20] — per-bin direction Q-mean EMAs (S/H/L/F)
- isv_signals[16] — magnitude-branch |Q|-scale EMA
- isv_signals[21] — direction-branch |Q|-scale EMA
- isv_signals[12] — learning_health
- v_min, v_max — C51 support range (per-fold eval_v_range EMA)
The 0.1 × v_range floor (= ~5 atom widths for 51-atom grid) is an
architectural parameter of the atom grid, not a tuned constant — its role
is "minimum scale above atom-grid discretization noise". The mechanism's
RESPONSE scales with observed signals when they exceed this floor; it
just keeps the response from collapsing to noise when both ISV Q-scale
EMAs happen to be near zero early in training.
Self-regulates three ways: tradable clearly leads → deficit=0 → bias=0;
health=1 (training stable) → bias=0; v_range=0 (impossible by construction).
Empirical status — 3 clean smoke runs after forcing a fresh CUDA cubin
(earlier stale-cubin runs showed v4 behaviour; the initial v5 run 1 on
stale cubin matched v4 run 3 identically, which exposed the rebuild gap):
Run 1: EVAL_DIR Short=0.287 Hold=0.000 Long=0.713 Flat=0.000 — Hold+Flat=0 ✓
Run 2: EVAL_DIR Short=0.000 Hold=0.000 Long=1.000 Flat=0.000 — Hold+Flat=0 ✓
Run 3: EVAL_DIR Short=0.000 Hold=0.000 Long=1.000 Flat=0.000 — Hold+Flat=0 ✓
Pre-v5 baseline (committed v2): Hold+Flat ∈ {0.809, 0.872, 0.796} across 3 runs.
The smoke test still fails on magnitude assertions (line 134 eh+ef≥0.30
or line 153 ef≥0.05) because eval magnitude still collapses to Quarter
or Half. That's a separate problem — the magnitude branch needs its own
reward-bias mechanism mirroring v5 but on d_branch==1 / Half+Full bins.
Tracked separately as Task 2.X-ext (internal task #60).
Per feedback_adaptive_not_tuned.md: the mechanism remains signal-driven;
the only scalar constant (0.1) is a structural fraction of the atom grid,
documented as architectural rather than data-regime-tied tuning.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
6061a190b8 |
fix(dqn): Task 2.Y-ext v2 — Bellman-target reward-bias for direction-branch (partial)
Replaces the symmetric target stretch (v1, removed) with an asymmetric
per-sample reward bias applied to `reward` BEFORE the `+ gamma*z_j` term
in the Bellman projection. The stretch was mathematically unable to fix
the direction collapse: `t_z = v_mid + (t_z - v_mid) * stretch` preserves
the mean of the target distribution and only fattens its tails, which
does nothing for C51 eval argmax (argmax over expected_Q uses the mean,
not the variance).
The v2 mechanism:
- Fires ONLY on tradable direction samples (a0 ∈ {Short=0, Long=2}).
- Fires ONLY when direction argmax has collapsed onto non-tradable
bins (Hold=1 or Flat=3) per ISV [17..20] Q-mean EMAs.
- reward_bias = (max_mean_dir - q[a0]) * (1 - learning_health)
- Self-regulates three ways: argmax → tradable (pathology gone),
health → 1 (training stable), or q[a0] → max_mean_dir (no deficit).
Signal wiring (all pre-existing):
- ISV [17..20]: q_s / q_h / q_l / q_f per-bin EMAs
- ISV [12]: learning_health
- Populated by `q_dir_bin_means_reduce` + `isv_signal_update` wiring
landed in commits
|
||
|
jgrusewski
|
c071489979 |
infra(smoke): --max-bars cap for train_baseline_rl + multi_fold smoke
Adds an optional --max-bars CLI cap to `examples/train_baseline_rl.rs`.
When >0, truncates the fxcache-loaded features/targets/OFI/timestamps in
lockstep per the data_loading.rs precedent (commit
|
||
|
jgrusewski
|
b8cd4e1d94 |
diag+docs(dqn): trunk-slice grad decomposition + stale IQN-trunk doc fix
Extends grad_decomp_kernel to snapshot the trunk tensor slice (tensors
0..4 = w_s1, b_s1, w_s2, b_s2) in addition to the existing direction +
magnitude branch slices (8..12 / 12..16). Adds a new HEALTH_DIAG group:
grad_trunk [iqn=<abs> ens=<abs> c51=<abs> cql=<abs> distill=<abs>
rec=<abs> pred=<abs> cql_sx=<abs> c51_bs=<abs>]
Prior grad_split_bwd / grad_split_aux groups report mag_norm / dir_norm
ratios per loss component, computed over branch-head tensors only. That
measurement range structurally reports 0.0000 for any loss component
that writes exclusively to the trunk — IQN and Ens in particular. This
caused the persistent misdiagnosis that IQN-to-trunk was not wired; the
prior scoping in /tmp/foxhunt_research/iqn-to-trunk-wiring-scoping.md
confirmed the wiring is live (apply_iqn_trunk_gradient at
gpu_dqn_trainer.rs:4882) and that the zero reading was a blind spot in
the measurement pipeline.
Smoke confirms the diagnostic: after iqn_readiness ramps up (late
epochs), grad_trunk reports iqn=100..381 (real trunk SAXPY amplitude),
ens=0.07..3.57, c51=2.46..8.91 (value-head dueling path contributes
through trunk), while cql/cql_sx/distill/rec/pred stay near-zero — a
clean diagnostic baseline.
Also fixes stale documentation at dual-distributional-c51-iqn-design.md
that claimed "IQN trains in isolation — its gradients don't flow back
to the shared trunk": reworded to reflect current wired state with
file:function citation and explicit iqn_readiness gating note. Updated
the "What Changes" table ("IQN training") and "Implementation Order"
(Phase 1 marked DONE) with the same citation.
Changes:
- grad_decomp_kernel.cu: per-component result slot 2 → 3 floats
(mag_norm, dir_norm, trunk_norm); extra __shared__ sum_trunk +
tree reduction; new grad_trunk_start/trunk_len kernel args.
- gpu_dqn_trainer.rs: pinned result buffer 18 → 27 floats; snapshot
now does two copy_f32 passes (trunk → dst[0..trunk_len), branch →
dst[trunk_len..]); per-component slot offsets 0/2/… → 0/3/…;
grad_component_norms_trunk cached field + accessor; compute trunk
range from padded_byte_offset(¶m_sizes, 0..4).
- fused_training.rs: grad_trunk_norms_by_component() + per-component
grad_trunk_*_abs() accessors.
- training_loop.rs: HEALTH_DIAG emits new grad_trunk group ordered
[iqn ens c51 cql distill rec pred cql_sx c51_bs]; extended doc
comment explaining the three groups' roles.
- design spec (Problem #1 + What Changes row + Implementation Order):
stale "IQN trains in isolation" replaced by current wired-state
description, cites gpu_dqn_trainer.rs:4882 and readiness ramp at
gpu_dqn_trainer.rs:4228-4243.
Pure diagnostic — no training dynamics change, no atomicAdd, no tuning
knobs, no TF32 changes. Smokes unaffected (magnitude_distribution H10
regression pre-exists on HEAD 810b3c570; 4 other smokes pass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
810b3c5703 |
fix(dqn): Task 2.Y — ISV-adaptive direction-branch C51 bin weighting (partial)
Mirror of Task 2.X (magnitude branch, commit |
||
|
jgrusewski
|
fa8d546614 |
diag+fix(dqn): Task 2.X ISV-adaptive magnitude mechanism — reveals direction-branch is the real blocker
Per feedback_adaptive_not_tuned.md: adaptive signal-driven mechanism, zero
static tuning knobs. Extends the existing ISV bus with per-magnitude Q-mean
EMAs and an absolute-scale reference; C51 loss + gradient kernels now read
ISV at zero hot-path cost to modulate per-bin weight in response to observed
collapse severity. Weight is 1.0 when Q is healthy; scales up per-bin when
collapse signal fires; self-disables as training stabilises.
Additions:
* ISV_DIM 13 → 17. New slots:
[13] Q_MAG_MEAN_QUARTER: ema(mean Q(Quarter), tau=0.05)
[14] Q_MAG_MEAN_HALF: ema(mean Q(Half), tau=0.05)
[15] Q_MAG_MEAN_FULL: ema(mean Q(Full), tau=0.05)
[16] Q_ABS_REF: ema(max(|Q_mean[k]|), tau=0.05) — scale-invariant reference
* q_mag_bin_means_reduce kernel (q_stats_kernel.cu) — one-block reduce
computing per-mag Q-means from q_out_buf; output written to pinned
scratch slots; drives the EMAs in isv_signal_update.
* c51_loss_kernel::get_magnitude_bin_weight helper + matching inlined
logic in c51_grad_kernel: composite collapse signal = min(1,
frac_bin + (1 - learning_health)); bin_weight = 1.0 + collapse *
mag_bias_signal[k] (bounded in [1, 2]); mag_bias_signal[k] = (k+1)/b1_size.
* isv_signal_update extended with q_mag_means_ptr + q_abs_ref_ptr +
mag_size kernel args.
Diagnostics (keystone finding below):
* gpu_backtest_evaluator::read_eval_action_distribution_per_direction —
4-bin per-direction count at eval (Short/Hold/Long/Flat fractions).
This diagnostic flipped the task diagnosis.
* DQNTrainer::last_eval_direction_dist accessor.
* last_isv_magnitude_bin_q_means accessor.
* EVAL_DIR_DIST + ISV_BIN_MEANS debug prints in magnitude_distribution smoke.
* ef >= 0.05 smoke gate added (currently unreachable behind pre-existing
eh+ef >= 0.30 gate; kept for future use).
Training-time outcome:
Pre-fix MAG_DIST: Quarter=0.60 Half=0.10 Full=0.23
Post-fix MAG_DIST: Quarter=0.46 Half=0.24 Full=0.28 (2.4× Half lift,
Full unchanged)
Pre-fix EVAL_DIST: eq=1.000 eh=0.000 ef=0.000
Post-fix EVAL_DIST: eq=0.981 eh=0.019 ef=0.000
Root cause revealed (why the adaptive fix couldn't lift ef off 0):
EVAL_DIR_DIST: Short=0.045 Hold=0.115 Long=0.070 Flat=0.771
~88% of eval states have direction ∈ {Hold, Flat}. Kernel at
experience_kernels.cu:~896 FORCES mag_idx=0 (Quarter) in those cases
as a structural ABI invariant. Only ~11.5% of eval samples have a
free magnitude choice. Upper bound on ef regardless of magnitude
mechanism: ~0.11.
The magnitude branch mechanism works as designed — it correctly
rebalances per-bin Q-means and lifts the training-time Half share
2.4×. But direction-branch collapse to Flat masks everything
downstream. Task 2.X's magnitude-only scope cannot unblock eval ef.
The real fix target is direction-branch eval collapse. Follow-up
task "Task 2.Y make direction-branch trade" extends the same
ISV-driven composite-signal mechanism to branch 0 (Short/Long vs
Hold/Flat). Scoping doc to be written.
Smoke validation:
magnitude_distribution FAIL (pre-existing eh+ef >= 0.30 gate; same
fail mode as HEAD before this commit)
reward_component_audit PASS
controller_activity PASS
exploration_coverage PASS
multi_fold_convergence PASS (avg best_val_metric=0.039, within ±15%)
No config fields. No static tuning knobs. No feature flags. All
modulation flows through the ISV bus. Shape constants documented:
eps=1e-6 (numerical guard), alpha=0.05 (ema tau matching existing ISV
pattern), MAX_MAG=4 (branch-size ceiling, already established),
mag_bias_signal[k]=(k+1)/b1_size (architectural monotonicity w.r.t.
bin index as stake size).
|
||
|
jgrusewski
|
a9a51e8fa0 |
cleanup+fix(reward): Task 2.4 R6 relocation + Task 2.5 Bug #6 docstring
Task 2.4: Relocates negative-tail compression from R6 reward-layer
(asymmetric_soft_clamp at experience_kernels.cu:78-81) to C51 Bellman
target smoothing (c51_loss_kernel.cu::block_bellman_project_f).
Functionality preserved — same invariant, better location. Upper +10
cap kept inline as fminf(reward, 10.0f) for numerical safety.
Deletions (reward layer — R6 no longer shapes the reward itself):
- asymmetric_soft_clamp() from experience_kernels.cu:78-81 (no callers)
- Reward-layer clamp replaced with fminf(reward, 10.0f) at ~1922
(segment_complete) + ~3049 (hindsight_relabel opt_reward)
- la slot from reward_contrib_fractions (was slot 4; tuple shrinks 5→4)
- loss_aversion_per_sample buffer from GpuExperienceCollector
(field + alloc + kernel arg + dtoh + memset, all removed)
- la={:.3} field from HEALTH_DIAG reward_contrib format string
- loss_aversion assertion from reward_component_audit smoke test
- loss_aversion comment reference in raw_returns comment block
Additions (gradient layer — R6 invariant moves here):
- Huber-style `if t_z < 0 { t_z = -10*(1-exp(t_z/10)); }` in
c51_loss_kernel.cu::block_bellman_project_f BEFORE v_min/v_max clamp
- Inline kernel comment documenting the relocation rationale
- Track 2 triage doc updated: R6 verdict DELETE → DELETED / RELOCATED
with landed-relocation notes (both call sites + C51 Bellman edit)
Task 2.5 Bug #6: Stale `patience_mult` docstring at
experience_kernels.cu:1144 referenced the defunct R7 V8 reward (deleted
in Task 0.8). Rewrote the reward-shape docstring to reflect current
post-V7 / Task 0.8 reality (sparse = 2.0 * vol_normalized_return, capped
inline) and notes the R6 relocation. Per feedback_trust_code_not_docs.md.
Per feedback_no_functionality_removal.md: R6's invariant is RELOCATED,
not deleted. The negative-tail compression — which protects against
catastrophic-loss-gradient dominance in the Q update — is now at the
Bellman target smoothing step where the invariant structurally belongs
(reward-inventory §"wrong-level regularization" pattern).
Tolerance band validation (smoke suite at this commit):
magnitude_distribution: F_Half=0.150 F_Full=0.237 (≥0.05 floor ✓)
(H10 eval_dist assertion fails pre-existing at HEAD 90e1e3dbb; not
introduced by this change — verified by running at HEAD before stash
pop, same [EVAL_DIST] 1.000/0.000/0.000 collapse.)
reward_component_audit: cf_flip=0.584 trail=0.304 (cf_flip≥0.1 ✓, PASS)
controller_activity: [CTRL_FIRE] anti_lr=0.000 tau=0.000 gamma=0.000
clip=0.400 cql=0.000 cost=0.000 (PASS)
exploration_coverage: entropy @ep5=0.988 @ep20=0.985 (PASS)
multi_fold_convergence: Best Sharpe 81.54/38.82/84.18 (≥20 floor ✓)
best_val_metric 0.043/0.024/0.049 (baseline was 0.028/0.018/0.019 at
policy-quality-baseline — 26 intervening commits of bug fixes from
Task 2.5 bugs #1–#7 would account for persistent drift; within
run-to-run variance of HEAD-pre-change)
|
||
|
jgrusewski
|
90e1e3dbb2 |
fix(dqn): Bug #7 — cql_alpha regime gate handles null ISV without silent fallback (Task 2.5)
Track 3 triage §C5 identified as an error-hiding case per feedback_no_hiding.md: when `isv_signals_pinned` is null (smoke-scale runs without ISV warmup), the previous code silently fell back to (health=0.5, regime_stability=0.5), yielding `cql_alpha_eff = base × 0.5 × 0.5 = 0.25 × base` by degenerate math, not by design. The hide made the smoke cql_alpha path near-zero for reasons unrelated to the intended regime-gated behaviour. Fix (option b per plan): emit a one-shot `tracing::warn!` and gate the regime multiplier OFF when the pointer is null — cql_alpha falls back to the scheduled base value (`base × 1.0 × 1.0`). Real ISV path unchanged. The `std::sync::Once` bounds log spam to once per trainer lifetime (this function runs every training step). Option (a) — wiring ISV warmup at smoke scale — is a follow-up task; it requires an upstream ISV pipeline change that is out of scope for this bug-fix sweep. |
||
|
jgrusewski
|
0cac3c84ce |
fix(dqn): Bug #5 — reset controller_fire_counts at fold boundary (Task 2.5)
Track 3 triage §C2/C5 fold-boundary artefact: the controller fire counters (anti_lr, tau, gamma, grad_clip, cql_alpha, cost_anneal), the running total-epochs denominator, and the prev-controller snapshot were NOT reset in reset_for_fold(), causing the 2/60 fire rates for tau and cql_alpha to accumulate across folds under cosine-annealed tau jumps when train_step resets + cql_alpha schedule drift. Fix: reset `controller_fire_counts`, `controller_total_epochs`, and `prev_controller_values` at fold-entry in reset_for_fold(). This decouples fold-boundary bookkeeping from intra-fold controller interventions so the controller_activity smoke gate measures per-fold fire rate rather than multi-fold running count. `last_anti_mult` is intentionally NOT reset here — it is within-epoch state already reset in reset_epoch_state. |
||
|
jgrusewski
|
96ecd0ff46 |
cleanup(dqn): Bug #3 — delete dead \if !true\ update_epsilon block (Task 2.5)
Dead code: `if !true { ... }` is a permanently-disabled guard pattern.
The epsilon schedule is driven by explicit epsilon_start / epsilon_end /
epsilon_decay hyperparameters via get_effective_epsilon() at the agent
level; the legacy per-step-count update_epsilon() was abandoned in
favour of that explicit schedule.
Per feedback_no_hiding + feedback_no_stubs: dead code is deleted, not
left behind as "someday". Per feedback_no_feature_flags: a !true toggle
is a disabled feature flag pattern.
Zero behavior change; removes foot-gun.
|
||
|
jgrusewski
|
54410cba91 |
fix(dqn): Bug #1 — fire_lr detects anti-LR multiplier, not scheduler LR (Task 2.5)
Fire detection captured `cur_lr = lr_scheduler.get_lr()` BEFORE the anti-LR multiplier was applied. Under any non-Constant scheduler (Cosine / Linear / Exponential), `fire_lr` ticks every epoch from pure scheduler drift, yielding 100% false-positive firing of the anti-LR controller in controller_activity diagnostics. Fix: detect anti-LR via the multiplier itself. Added `last_anti_mult: f32` field (init 1.0, reset 1.0 each epoch in reset_epoch_state, updated by the anti-LR block at its decision point). Fire condition becomes `(last_anti_mult - 1.0).abs() > 0.01` — observes the actual intervention, not scheduler drift. Prerequisite for Task 2.8 L40S run — without this, L40S C1 verdict is uninterpretable under any non-Constant scheduler (all runs would read as "controller fires every epoch"). Per Track 3 triage §C1 wiring surprise. |
||
|
jgrusewski
|
2a7005f29a |
fix(dqn): Bug #2 — epsilon_greedy_action samples 4-branch factored space (Task 2.5)
Stale 0..5 range from pre-2026-04-08 9-level code. Only test paths call this cold-path fallback, but the stale range was a latent foot-gun and would mislead anyone reading the code (per feedback_trust_code_not_docs). Fix: sample dir ∈ [0,3), mag ∈ [0,3), ord ∈ [0,3), urg ∈ [0,3) and encode as `dir*27 + mag*9 + ord*3 + urg`, matching MEMORY.md 4-branch DQN architecture (81 factored actions). Per Track 4 E1 triage tech-debt flag. |
||
|
jgrusewski
|
4c3806da9c |
fix(cuda): harden latent memcpy_dtoh async-race in download_{params,target_params}
Systematic audit of all `memcpy_dtoh` call sites following commit |
||
|
jgrusewski
|
5da434ab4b |
fix(cuda): sync after async memcpy_dtoh in per_branch_grad_norms — direction-branch determinism
After Option C (commit
|
||
|
jgrusewski
|
199feff4db |
fix(cuda): per-stream cublasLt handles (Option C) — 10× determinism improvement on cuBLAS path
Replaces SharedCublasHandle (one lt_handle rebound across streams) with
PerStreamCublasHandles (one lt_handle per CUDA stream). Implements
NVIDIA's cuBLAS §2.1.4 remediation #1 — documented fix for concurrent-
stream non-determinism.
Context: prior investigation (task a11d706bdb56b5020) ruled out
atomicAdd/RNG/Thrust/multi-stream-sync/graph-capture. Option B (commit
|
||
|
jgrusewski
|
bb399b6359 |
fix(cuda): deterministic cublasLt algorithm selection (Option B)
Replace cublasLtMatmulAlgoGetHeuristic (timing-based, non-deterministic
across process invocations) with cublasLtMatmulAlgoGetIds +
cublasLtMatmulAlgoInit + cublasLtMatmulAlgoCheck across all 10 smoke
training hot-path sites.
Root cause (investigation task a3af7a105c128c535): the heuristic's
"fastest" ranking depends on timing state (thermal, GPU load, NVML
warm-up), causing 1-3% variance in per-epoch gradient L2 norms even
under TF32 ON with CUBLAS_WORKSPACE_CONFIG=:4096:8 +
NVIDIA_TF32_OVERRIDE=0.
Fix: deterministic selector queries hardware-stable algorithm IDs,
sorts ascending, picks first one that passes AlgoCheck validation
(workspace size, alignment). Same inputs -> same algo, always.
TF32 compute_type (CUBLAS_COMPUTE_32F_FAST_TF32) PRESERVED per user
directive — tensor-core speed maintained at all 10 sites.
New module: crates/ml/src/cuda_pipeline/cublas_algo_deterministic.rs
(~485 LOC), process-shared SELECTOR singleton with per-shape cache.
Exposes:
- `DeterministicAlgoSelector` — struct with ids_cache + algo_cache
- `ShapeKey::new(transa, transb, m, n, k, lda, ldb, ldc, ws)` —
default-epilogue constructor
- `ShapeKey::with_epilogue(..., epilogue, ws)` — RELU_BIAS variant
- `get_matmul_algo_deterministic(..)` — drop-in replacement
returning `cublasLtMatmulHeuristicResult_t`
- `get_matmul_algo_f32_tf32(handle, desc, layouts, shape)` —
convenience wrapper for the common F32+TF32 types tuple
Uses raw FFI from `cudarc::cublaslt::sys::{cublasLtMatmulAlgoGetIds,
cublasLtMatmulAlgoInit, cublasLtMatmulAlgoCheck}` — the cudarc safe
wrappers don't expose these three calls, but the raw FFI bindings are
present.
Wire-up: 10 sites in batched_backward (cached + uncached),
batched_forward (uncached + cached default + cached RELU_BIAS),
gpu_dqn_trainer (mamba2), gpu_iqn_head, gpu_attention,
gpu_iql_trainer, gpu_curiosity_trainer migrated from heuristic to
deterministic selector. `matmul_pref` create/set/destroy boilerplate
deleted at every site.
Validation: 3x magnitude_distribution smoke at HEAD
(/tmp/foxhunt_smoke/option_b_run{1,2,3}.log) show identical algo
picks across all fresh process invocations — instrumented run
confirmed every single call returns `algo_id=16, ids_tried=13` for
every (transa, transb, m, n, k, epilogue) tuple. Residual HEALTH_DIAG
variance remains (see DONE_WITH_CONCERNS note in task report) — but
that variance is NOT attributable to cublasLt algorithm selection.
Wall-clock impact: neutral. Per-fold training time stable at
~6.9s / ~8.4s / ~10.4s across folds 1/2/3 with <0.05s std-dev
across 3 fresh runs. First-call AlgoGetIds cost is amortised via
the per-types-tuple cache.
|
||
|
jgrusewski
|
34168f53f2 |
diag(policy-quality): per-magnitude win-rate + return-variance instrumentation
Prerequisite from Task 2.X scoping doc (commit
|
||
|
jgrusewski
|
7b74290dd0 |
docs(dqn): R5 micro-reward — documented intentional disable (Phase 2 Task 2.3)
Per feedback_no_functionality_removal.md: R5 was originally scoped as
DELETE in the Phase 2 plan and the Track 2 triage because
reward_contrib[3] = 0.000 across 60 / 60 smoke epochs and
dqn-smoketest.toml sets micro_reward_scale = 0.0. Re-examination during
Phase 2 Task 2.3 rejected the DELETE path:
- dqn-production.toml already sets micro_reward_scale = 0.1, so R5 is
load-bearing in production, not dead code. The 0.0 value in smoke is
deliberate test-isolation (td_propagation / magnitude_distribution /
reward_component_audit all want the sparse-reward TD path isolated).
- The state-vector OFI block at state[SL_OFI_START..SL_OFI_START+SL_OFI_DIM)
= [42..62) provides representation features for the encoder (policy
side). R5 is a per-bar reward gradient on the critic (critic side).
Different mechanisms — production deploys both together.
- R5 also reads PREV_MID (retrospective hold quality), which is NOT in
the state vector. That signal exists only in the kernel branch.
Changes — pure documentation, no behavior change:
- experience_kernels.cu: ~30-line comment block at the R5 wiring site
(~L1915) documenting the parameter-not-flag status, production vs
smoke values, why state-vector OFI is complementary not redundant,
and the feedback_no_functionality_removal.md seal.
- experience_kernels.cu: fix stale kernel-signature comment that claimed
OFI was at state[66..74). Correct range is [42..62) per state_layout.cuh.
- config.rs: extend DQNHyperparameters::micro_reward_scale docstring and
add a comment at the Default impl pointing back at the kernel site.
- gpu_experience_collector.rs: extend reward_contrib_fractions docstring
to mark the micro=0.000 slot as a SEMANTIC value when the loaded profile
has micro_reward_scale=0.0, not a wiring regression.
- track2-triage.md: R5 verdict changed from DELETE to FIX-documented-disable
with the rationale above; "Proposed Phase 2 changes" section 1 and
"Next Track 2 steps" updated accordingly.
Smoke tests: 3 / 4 pass (reward_component_audit, controller_activity,
exploration_coverage). magnitude_distribution is failing on baseline
HEAD
|
||
|
jgrusewski
|
c0fee5a9bf |
docs(smoke): magnitude_distribution — replace H9-delete references with fix-path
Per standing rule feedback_no_functionality_removal.md: never propose deleting the magnitude branch as a fallback. Updated the Task 2.2 regression-assertion comments + assertion message to point at the actual follow-up fixes if the eh+ef≥0.30 gate fails: - per-magnitude reward shaping - per-bin advantage weighting - magnitude curriculum - state-vector enrichment No semantic change to the test (still asserts eh+ef≥0.30); only the guidance comments were reframed. |
||
|
jgrusewski
|
8aef59f735 |
fix(dqn): H10 — stable argmax tie-break at eval per Track 1 triage + Task 2.0 re-diagnosis
Replaces eval-mode Boltzmann softmax with strict argmax + uniform-sample- among-tied-indices (|q_a − q_b| < 1e-6). Applied to all 4 branches (direction, magnitude, order, urgency) of experience_action_select. Uses the existing Philox state (same (i, timestep) seed used elsewhere in the kernel for CF-flip / exploration); eval mode is therefore deterministic per (sample, epoch) — no new atomics, no new RNG. Training mode keeps Boltzmann softmax unchanged (needed for exploration + gradient flow when C51 expected-Q structurally favors Flat/Quarter). Root-cause re-diagnosis (commit |
||
|
jgrusewski
|
41b0c559c9 |
diag(policy-quality): Task 2.0 confirmation — expose absolute grad_dir / grad_mag norms
Task 0.4's grad_ratio_mag_dir returns 0.0 whenever dir_norm < 1e-9, so the epoch-end reading of 0.0000 doesn't disambiguate "magnitude starved" vs "direction starved". Task 2.0's per-component data showed CQL and C51 each sending 100-400x more gradient to magnitude than direction — implying direction is the starved one, not magnitude. This adds a HEALTH_DIAG field exposing the raw absolute norms: grad_abs [dir=<sci-notation> mag=<sci-notation>] Along the way uncovered + fixed two latent bugs that had been silently zeroing the ratio signal since Task 0.4 landed: 1. `per_branch_grad_norms` read `grad_buf.len()` = total_params + cutlass_tile_pad (~4096 elements of GEMM tile padding) but the pinned readback slot was sized at construction to total_params exactly. The size check `grad_len > grad_readback_pinned_capacity` was always true, so the accessor returned Err on every call — and FusedTrainingCtx's proxy coerced Err to 0.0 via `.unwrap_or(0.0)`. Root cause for the "always 0.0000" grad_ratio_mag_dir. Fix: read only the first `total_params` prefix of grad_buf (the tail is pure GEMM padding, never holds gradient values). 2. Readback timing: process_epoch_boundary calls estimate_avg_q_value_with_early_stopping early, which replays `eval_forward_exec` — the SAME captured graph as forward_child whose first op is `cuMemsetD32Async(grad_buf, 0, total_params)`. Any grad_buf readback AFTER the avg_q call sees all zeros. Fix: snapshot grad_dir_abs / grad_mag_abs / grad_ratio_mag_dir at the TOP of process_epoch_boundary, before avg_q runs, and consume the cached values in the HEALTH_DIAG block. Last 5 epochs of fold 3 on the magnitude_distribution baseline smoke (FOXHUNT_TEST_DATA=test_data/futures-baseline): HEALTH_DIAG[15] ratio=42.85 grad_abs [dir=6.804900e0 mag=3.989081e2] HEALTH_DIAG[16] ratio=232.66 grad_abs [dir=6.500046e0 mag=3.603353e0] HEALTH_DIAG[17] ratio=318.34 grad_abs [dir=2.720317e-2 mag=1.713861e1] HEALTH_DIAG[18] ratio=367.59 grad_abs [dir=5.180866e-2 mag=8.076681e0] HEALTH_DIAG[19] ratio=298.55 grad_abs [dir=1.989553e-2 mag=6.498848e0] Across all 60 epoch-boundary readings (3 folds × 20 epochs) dir ∈ [~4e-3, ~6e0] and mag ∈ [~5e-2, ~5e2], with ratio mag/dir consistently 50-400× (matching Task 2.0's per-component ratios). Neither branch is near float precision — direction is PROPORTIONALLY starved, not numerically zero. Scenario confirmed: direction is starved relative to magnitude, NOT the reverse. Phase 2's Task 2.1 (architectural fix on magnitude branch) should pivot toward increasing direction's gradient flow instead. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
980f3b07f3 |
diag(policy-quality): Task 2.0 extension — instrument 5 more grad writers
First Task 2.0 pass (commit |
||
|
jgrusewski
|
d60e5375a9 |
diag(policy-quality): Task 2.0 — per-component grad decomposition for H4
Adds grad_mag_{iqn,cql,c51,ens} HEALTH_DIAG fields via in-graph
pinned-snapshot + in-graph reduction kernel (revised approach; first
Task 2.0 dispatch escalated BLOCKED on host-side-snapshots-inside-
captured-graph, plan revised at
|
||
|
jgrusewski
|
0472b97300 |
test(smoke): strengthen performance probes with meaningful thresholds
Previously test_training_throughput_measurement and test_real_data_single_epoch
only asserted loss.is_finite() on the final metric. That passes on trivial
zeros, on huge-but-finite NaN-disguised values, and on any regression that
doesn't produce literal NaN — giving effectively no signal.
test_training_throughput_measurement now asserts:
- loss finite AND non-negative
- epochs_trained >= 1
- throughput floor: epochs_per_sec > 0.05 (i.e. each epoch < 20s on the
RTX 3050 Ti; catches accidental CPU fallback or kernel CPU-pinning).
Documented as a conservative local floor; CI may tighten.
- avg_q_value present, finite, |avg_q| < 1e6 (rules out finite-but-huge
NaN propagation)
test_real_data_single_epoch now asserts:
- loss finite, non-negative, and < 1e8 (a real DQN loss of 0.0 is a
sign-bug or accumulation-bug tell; huge-but-finite rules out NaN
propagation)
- epochs_trained >= 1
- avg_q_value finite and |avg_q| < 1e6
Why these are safe:
- Bounds are chosen from observed smoke runs with 2-3 orders of margin.
- Passes locally in 1.58s and 10.24s respectively.
- Designed to flag regressions, not true production-scale deviations.
Verified PASS on laptop (RTX 3050 Ti).
|
||
|
jgrusewski
|
2570fe0130 |
fix(smoke): test_fxcache_zero_copy_training — real failure detection
Previously the fold loop swallowed every training error into f64::NAN,
unconditionally pushed a value at the top of the loop, and then asserted
only !fold_losses.is_empty() — a tautology that could never fail. Any
CUDA error, NaN explosion, or regression of the zero-copy path passed
silently.
Now:
- Error propagation via `?` (no swallow). A zero-copy test can't
tolerate training failures — if training blew up, the zero-copy
wiring is broken and must surface.
- All fold losses must be finite and non-negative.
- Every fold must report epochs_trained >= 1 (zero-epoch fold =
kernel skipped or buffer not populated).
A direct "no htod/dtoh copy happened" assertion would need a copy-counter
instrumented into the fused-training GPU path plus a field on
TrainingMetrics. That is out of scope for this test; documented inline.
Verified PASS on laptop (RTX 3050 Ti):
loss=0.0018366, epochs=1
|
||
|
jgrusewski
|
e8ecb2f626 |
fix(training_loop): surface FusedTrainingCtx init errors instead of hiding them
Both lazy-init sites (train_with_data_full_loop_slices @ L344 and
run_training_steps_slices @ L1511) previously logged FusedTrainingCtx::new
failures via tracing::error! and continued with fused_ctx = None. On CUDA
builds there is no CPU fallback, so the next stage (GPU experience collector)
then fails ~100ms later with a misleading "GPU experience collector MUST be
active for CUDA training" error that obscures the real CUDA root cause (OOM,
driver error, etc.).
Both sites now propagate the original error via map_err → anyhow::anyhow! so
callers see the actual failure. The post-init wiring (PER buffer pointers,
RNG dev ptr) is refactored from `if let Some(ref mut fused)` — which was
silently-no-op on the hidden-error path — to an unconditional
as_mut().expect() since fused_ctx is guaranteed Some after the `?`.
The two remaining `fused_ctx = None` assignments are legitimate:
- mod.rs:655 in Drop (ordered GPU teardown)
- training_loop.rs:1508 batch-size-change recreate (immediately reassigned
by the ? on the next line)
No new API, no threshold changes, no feature flags.
|
||
|
jgrusewski
|
0beccd5e82 |
fix(smoke): multi_fold_convergence — laptop-sized config
Two wrong-scale assumptions in the test made it unachievable on the
RTX 3050 Ti / 4 GB laptop this smoke is meant to run on:
1. `train_baseline_rl` was invoked without `--training-profile`, so it
defaulted to `dqn-production`: batch_size=16384, buffer=500k,
num_atoms=52, hidden_dim_base=256. Fused-CUDA init OOMs at
`kan_d_coeff_per_elem alloc` on 4 GB, leaving `fused_ctx = None` and
every subsequent fold failing with "GPU experience collector MUST be
active for CUDA training". Fix: pass `--training-profile=dqn-smoketest`.
2. Default walk-forward windows (12 train / 3 val / 3 test / 3 step) only
yield 2 folds in the 24-month baseline dataset — fold 2's test-end
lands one month past `data_end`. The test's pass-gate is "≥2/3 folds
produce a checkpoint", so a test that can only ever generate 2 folds
is degenerate. Fix: explicit shorter windows (6 / 2 / 2, step 2) that
yield all 3 folds (`6 + 2*2 + 2 + 2 = 14 ≤ 24`, comfortable margin).
Also drops `--epochs 20` → `--epochs 5`. Each fold runs ~5500 batches at
~33 s/epoch on this GPU; 20 × 3 folds ≈ 33 min was exceeding the smoke
budget (kill observed around the 10-minute mark). 5 epochs is ample for
the checkpoint gate — `best_sharpe` saves on the first improving epoch
(epoch 1 in practice), so more epochs add no pass/fail signal, only
wall-clock.
Verified locally: 3/3 folds produce `dqn_fold{N}_best.safetensors`,
total wall-clock ~7 min.
[MULTI_FOLD] fold 0 checkpoint OK
[MULTI_FOLD] fold 1 checkpoint OK
[MULTI_FOLD] fold 2 checkpoint OK
test result: ok. 1 passed; 0 failed ... finished in 416.36s
Docstring updated to reflect new sizing and call out the 4 GB / 24-month
constraints explicitly so the next person reading this can see why the
numbers are what they are.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
jgrusewski
|
4399a56d76 |
test(smoke): strengthen weak assertions across DQN smoke suite
Five tests in crates/ml/src/trainers/dqn/smoke_tests had pass gates that validated existence rather than the invariant their doc-comment claimed to test. A trainer returning all-zero diagnostics (a plausible wiring regression) would have passed them. - reward_component_audit: was `is_finite() && >= 0.0` on 5 slots — passes trivially on all-zero stubs. Added `cf_flip > 0.1` and `trail_r > 0.01` floors (known-wired slots in smoke config; popart/micro/loss_aversion remain finite-only as they are legitimately near-zero in smoke). Run-observed values: cf=0.614, trail=0.295, la=0.006 — well above floors. - exploration_coverage: `.unwrap_or(0.0)` silently substituted 0 for a missing epoch, conflating "emission regressed" with "exploration collapsed". Now panics with a distinct message on missing entries, also asserts len >= 20, normalized range [0,1], and spread > 1e-6 to catch constant-output emitters. Run-observed spread: 0.275. - training_stability::50_epoch_convergence: entropy assertion was guarded behind `if entropy.is_finite()`, so NaN entropy (the more severe failure) silently passed. Fail hard on NaN first. - training_stability::trading_model_behavior: same `is_finite` guard pattern on action_entropy — now fails hard when the diagnostic is missing or NaN rather than skipping. - training_stability::gpu_collector_auto_initializes: only asserted training returned `Ok(_)`. A collector producing silent zeros would pass. Now also verifies epochs_trained, loss finiteness, and gradient flow. - walk_forward::no_overfitting_50_epochs: had two tautological "finite check" assertions (`x < x + 1` and the signum-adjusted ratio) that always passed regardless of divergence. Replaced with real `.is_finite()` checks plus a `div_ratio < 10.0` stability gate. Tests run under CUBLAS_WORKSPACE_CONFIG=:4096:8 + FOXHUNT_TEST_DATA=test_data/futures-baseline, release-test profile. reward_component_audit and exploration_coverage both PASS on the local RTX 3050. No threshold relaxation or quickfixes applied; any future wiring regression will now be caught by a meaningful assertion instead of a near-tautology. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
ed4b30b493 |
fix(smoke): controller_activity — V7 intervention-based fire detection
Redefine the "fire" semantic for adaptive controllers per the V7 audit (policy-quality-design spec §5.3): a controller fires iff it made an ADAPTIVE INTERVENTION this epoch, not merely because its observable output value changed. Previously fire detection was absolute-delta on the output: - grad_clip fired in 98.3% of epochs because the adaptive clip threshold is an EMA recomputed every training step. The EMA drifts by > 1e-3 every epoch regardless of whether the clip actually clamped a gradient. - cost_anneal fired in 98.3% of epochs because it is a deterministic sigmoid of current_epoch (1/(1+exp(-(epoch-10)/3))) with no adaptive or reactive component. Every epoch moves it by > 1e-4 by design. Neither was "load-bearing" in the V7 sense — one was a per-step EMA tracker, the other a pure curriculum schedule. The prior test output "controller 'grad_clip' fires in 98.3%" was a false positive from measuring the wrong signal. New semantics: - anti_lr / tau / gamma / cql_alpha: unchanged — absolute delta vs prior epoch on the effective output value (real adaptive controllers). - grad_clip: intervention-based latch `grad_clip_kicked_this_epoch`, set in run_training_steps_slices iff raw_grad_norm > active clip at any training step this epoch. Reset in reset_epoch_state. - cost_anneal: pure deterministic schedule → never load-bearing → always fires=false. The value is still tracked in prev_controller_values and the HEALTH_DIAG line still emits it for observability, but it cannot trip the 50% load-bearing gate. After fix, controller_activity smoke reports: anti_lr=0.000 tau=0.033 gamma=0.017 clip=0.233 cql=0.033 cost=0.000 All 6 rates ≤ 0.5. Test passes. Touched: - crates/ml/src/trainers/dqn/trainer/mod.rs (add grad_clip_kicked_this_epoch) - crates/ml/src/trainers/dqn/trainer/constructor.rs (init new field) - crates/ml/src/trainers/dqn/trainer/training_loop.rs (latch kick per step, redefine fire_clip + fire_cost in HEALTH_DIAG block) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
jgrusewski
|
a5f23b28f0 |
fix(diag): move GPU summary download before HEALTH_DIAG emit
Third root cause of the apparent magnitude collapse in smoke-test data:
HEALTH_DIAG reads monitor.action_counts to compute dist_q/h/f and
ent_mag/ent_dir, but the block that populated action_counts from the
GPU summary ran AFTER HEALTH_DIAG. Every epoch saw all-zero counts so
dist_q=dist_h=dist_f=ent_mag=ent_dir=0 in every log line, and the
last_magnitude_dist cache the smoke test reads was always zeros.
Moved the GPU-summary download + monitor population block up above the
HEALTH_DIAG preparation block. Signal is now real per-epoch:
dist_q=0.60 dist_h=0.15 dist_f=0.25 (was 0/0/0)
ent_mag=0.83 ent_dir=0.89 (was 0/0)
magnitude_distribution smoke test now PASSES on local RTX 3050 Ti at
20 epochs. Final: Quarter=0.598 Half=0.154 Full=0.249 — all above 5%
smoke threshold.
Combined with
|
||
|
jgrusewski
|
2fb30f098e |
fix(monitoring): cascade 9→12 action bins for 4-direction action space
The runtime DQN action space is 4×3×3×3 = 108 (dir×mag×order×urgency) per
the kernel's b0_size=4 (Short/Hold/Long/Flat) layout, but the monitoring
stack was still coded for the legacy 3×3=9-bin exposure layout. Flat
actions (dir=3) had exp_idx = 9 >= num_actions=9 and were silently
dropped by the monitoring_reduce kernel's bounds check.
Cascade:
- monitoring_kernel.cu: num_exposure_bins=12 (EXP_MAX constant), summary
layout 27 floats with bins [5..17), order [17..20), urgency [20..23),
N [23], trades [24], pad [25..27).
- gpu_monitoring.rs: MonitoringSummary.action_counts [usize; 12];
summary_buf 27 f32; reduce() now takes b0_size and validates
num_exposure_bins ≤ EXP_MAX.
- trainers/dqn/monitoring.rs: action_counts/q_value_sums/q_value_counts
all [_; 12], factored_action_counts [_; 108], EXPOSURE_NAMES with 4
dirs × 3 mags ("S_Small".."F_Full"); track_action now maps the 8-level
ExposureLevel enum onto the kernel's dir*3+mag layout by going through
direction()/magnitude() accessors (was indexing by raw `exposure as
usize` which is an entirely different scheme).
- trainer/training_loop.rs: total_action_counts [_; 12],
total_factored_action_counts [_; 108]; GPU monitoring reduce call
now passes b0 from agent.branch_sizes(); direction-entropy gains a
4th dir bin; per-action Q diagnostics names array extended to 12.
- trainer/metrics.rs: create_final_metrics arrays widened; q_diagnostics
per_action_avgs [_; 12]; combo_idx = d*3 + m (matching kernel layout);
action_space_size 108/12 and buy/sell/hold uses the new exp_idx
boundaries.
- financials.rs: action_counts [_; 12]; buy/sell/hold classification
updated — BUY = Long [6..9], SELL = Short [0..3], HOLD = Hold + Flat
([3..6] ∪ [9..12]). Tests updated for 12-bin layout.
Also fixes several collateral bugs exposed by the audit:
1. magnitude_action_dist_final conflated Hold (dir=1, mag-forced-to-0)
with "Quarter magnitude". New formula restricts the denominator to
tradable directions (Short + Long) and computes Q/H/F fractions only
over those, producing the real magnitude preference signal.
2. training_loop.rs:3012 flat_count indexed [3,4,5] (which is Hold in
the new layout, but was already the wrong bucket — "Flat" was never
at that index even under the legacy 9-bin reading, where [3,4,5]
was the Flat bin but mag was forced to 1 not 0). New code correctly
separates flat_count [9..12] from hold_count [3..6] and excludes
both from the directional denominator and diversity cell threshold.
3. monitoring.rs had q_value_sums: [f64; 7] but q_value_counts:
[usize; 9] — different sizes for what should be the same exposure
axis. Both now [_; 12].
4. log_action_distribution iterated over 7 indices when printing per-
action Q-values but the array is logically 12-wide — now uses
`0..12` with the 12-entry EXPOSURE_NAMES lookup.
Tests: all 7 financials tests pass. Monitoring cubin loads + default
summary sanity checks pass. Smoke test magnitude_distribution now
emits correct 12-bin breakdown — GPU summary shows (example epoch 20):
actions=[786, 157, 280, 377, 2, 7, 806, 156, 317, 303, 5, 4]
which decodes as Short (S) = 786/157/280 (~38%), Hold (H) = 377/2/7
(~12%, mag-forced), Long (L) = 806/156/317 (~40%), Flat (F) =
303/5/4 (~10%, mag-forced). Previously the Flat bucket (303+5+4 =
~10% of all actions) was silently dropped — policy was picking Flat
~10% of the time and nobody could see it.
No atomic ops added; all reductions remain via warp-scratch + lane-0
sequential merge. No feature flags, no stubs, no quickfixes.
|
||
|
jgrusewski
|
2ac956298b |
fix(data-loading): truncate OFI in lockstep with bars on max_bars path
Two truncation sites in data_loading.rs truncated feature/target vectors to max_bars but left self.ofi_features at the original (cache-full) length. Downstream DqnGpuData::upload_slices asserts OFI len == num_bars and surfaced this as "OFI/market data length mismatch: 175874 OFI rows vs 5000 bars — fxcache is corrupt". The cache was not corrupt — the upstream truncation was the bug. Fixed by moving OFI storage below the truncation block in the fxcache path and by adding a parallel truncate in the DBN fallback path. Added debug_assert_eq on equal lengths post-truncation so future regressions hard-fail in dev. Also corrected the cuda_pipeline/mod.rs error message to point at the actual culprit (upstream truncation desync) instead of blaming the on-disk cache. |
||
|
jgrusewski
|
83d524f866 |
diag(policy-quality): remediate stub return values per no-stubs rule
WIRE:
* q_mag_full/half/quarter — host-side mean of magnitude-branch q_out_buf
cols [b0..b0+b1] (Task 0.3 stub remediated). Cold-path dtoh at epoch
boundary via update_q_mag_means_cached(), cached in q_mag_means_cached,
read by q_magnitude_bucket_means().
* var_scale_mean — per-sample CudaSlice + host reduce over var_scale
written by experience_env_step at line 1422 (Task 0.3 stub remediated).
Host reducer averages only over slots where kernel wrote > 0 so
Var[Q]-disabled samples don't dilute the signal.
DELETE:
* segment_patience slot from reward_contrib group — term not wired in
kernel, slot reserved space for non-existent feature (Task 0.8 stub
remediated). reward_contrib [...] is now 5 floats not 6. Cascaded
through reward_contrib_fractions return type, trainer's
last_reward_contrib field, reward_component_audit_summary accessor,
and the reward_component_audit smoke test.
PopArt slot kept at 0.0 — that IS the semantic value during warmup or
disabled state, not a stub. Comment updated to make this explicit.
Per ~/.claude/.../memory/feedback_no_stubs.md: stubs strictly forbidden
even with code comments / commit-message documentation / plan sanction.
|
||
|
jgrusewski
|
aa16ca31f2 |
diag(policy-quality): finish Tasks 0.3 + 0.6 partial wirings
Task 0.3 (Track 1 magnitude):
* q_mag_full/half/quarter: plumbed via GpuDqnTrainer::q_magnitude_bucket_means()
→ FusedTrainingCtx::q_magnitude_bucket_means() → HEALTH_DIAG. Phase-0
returns [0.0; 3]; per-mag Q reduction kernel deferred per plan §0.3 step 2
(goal here is the accessor chain, swap-in without touching emit formatting).
* var_scale_mean: STUB 0.0 — kernel computes `1/(1+sqrt(Var[Q]))` per-sample
inside experience_env_step but does not persist to a device buffer; it is
consumed in-place to shrink effective_max_pos. Exposing requires a
dedicated per-sample output buffer + launch arg + kernel write; deferred
to Phase 1+. Documented in code.
* kelly_f_mean / avg_win_ratio: REAL — computed host-side from
self.trade_stats_history.last() using the SAME formula the kernel uses
(b = avg_win/avg_loss, kelly = (b·p − (1−p))/b clamped [0,1] × 0.5
for half-Kelly; avg_win_ratio = (sum_wins/win_count) /
max(sum_losses/loss_count, 0.001) per plan formula line 239).
One-epoch lag (trade_stats_history is appended later in the same
epoch body); zero until first epoch's stats land. Documented in code.
Task 0.6 (Track 1 noisy):
* vsn_mag / vsn_dir: REAL per-branch — mean |w| across each branch's VSN
projection weight pair (tensors 26..34 grouped by branch:
26/27=dir, 28/29=mag, 30/31=order, 32/33=urg). The live VSN gate mask is
computed per-sample inside variable_select_bottleneck and consumed
immediately, not materialized into a device buffer we can cheaply read;
projection-weight mean is the closest stable H7 surrogate. Documented
in accessor rustdoc. Higher-fidelity measurement would require a
per-sample mask output buffer following the Task 0.5 pattern — deferred.
* drift_mag / drift_dir: REAL per-branch — RMS ‖target_w − online_w‖ across
each branch's 4 weight tensors (indices 8..12 dir, 12..16 mag, 16..20
order, 20..24 urg). Host-computed cold path via two memcpy_dtoh + normal
Vec<f32> reduce loops; stream-sync'd before read. NO atomics (project rule).
~2.7 MB × 2 DtoH per epoch, negligible diagnostic cost.
NoisyNets sigma_mag/sigma_dir already wired in
|