CUDA kernel launches via cudarc::launch_builder and MappedF32::new
(cuMemHostAlloc DEVICEMAP FFI) inherently require unsafe blocks. The
workspace-wide '-W unsafe-code' lint produced ~80 warnings across these
files, all structurally identical. Match the established pattern from
ml-backtesting/src/harness.rs and ml-backtesting/src/sim/mod.rs: single
file-level allow with a comment explaining the rationale.
Files: alpha_dqn_h600_smoke, alpha_baseline, cublaslt_debug,
gpu_walk_forward, cuda_pipeline/mod, hyperopt adapters (mamba2, ppo),
DQN smoke_tests (helpers, td_propagation), trainers/ppo, and two
ml/tests files. Documented in dqn-wire-up-audit.md.
Three things landing atomically because they're load-bearing for each other:
1. **Trend-scanning leakage fix** — trend_scanning.rs was emitting OLS slope+t-stat
over a *forward* window [t, t+L]. With the Phase 1a label = sign(price[t+60]
− price[t]), the forward feature window overlaps the label window, contaminating
it. Purged walk-forward only sterilizes forward-looking *labels* that cross
the train/val split, not forward-looking *features* that peek inside the same
horizon the label measures. The leak inflated MLP accuracy from 0.49
(legacy 74-dim baseline) to 0.75 — vanished to 0.50 after switching to a
trailing window. Bounded the perfect-fit t-stat sentinel from ±1e6 → ±20
(p<1e-30 is already meaningless); eliminated the 16k corruption-cap drops.
2. **Variable-dim alpha column** — fxcache schema now carries the alpha-feature
width via metadata (`alpha_feature_dim`), not a compile-time constant. Same
on-disk format hosts the 134-dim bar-level stack OR the 81-dim snapshot stack.
Reader + auto-detect honor the metadata-declared dim; downstream MLP auto-sizes
`in_dim`. Single schema, no forks.
3. **Snapshot pipeline (Phase 1c falsification)** — `snapshot_pipeline.rs`: 81-dim
per-MBP10-snapshot extractor reusing 10 snapshot-native alpha blocks + 6 new
snapshot-specific features (time-since-trade, time-since-snap, event-rate,
spread-bps, L1-imbalance, microprice-mid drift). `precompute_features` gets
`--row-unit snapshot` flag; emits one fxcache row per LOB update (1.97M rows
from MBP-10 data vs 206K for bar mode).
**Smoke verdict on real data** (ES.FUT, 1.97M snapshots, 384K val):
- Bar-level honest alpha: accuracy=0.5005, AUC=0.5043 (no signal)
- **Snapshot-level alpha**: accuracy=0.5241, AUC=0.6849 (real signal, 384K val)
- GBM corroboration: accuracy=0.5401 (non-linear partitioning sees more)
- Horizon decay: alpha peaks at K=20-50 snapshots (~5-25ms), gone by K=500
- Regime-conditional: spread-Q4 quintile hits 0.752 accuracy on 76k samples
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per Class A audit-fix Batch 4-A (deferred from P1-wiring/P1-producer due to
audit-doc errors). Fixes 2 of 4 deferred items; Batch 4-B handles plan_threshold
floor + MIN_HOLD_TEMPERATURE in a separate commit.
Item 1: DD saturation floor (the upper end of the DD ramp at trade_physics.cuh:154
in apply_margin_cap, NOT line 548 as the audit doc claimed — that line is a
magnitude action constant; the actual saturation floor lives in apply_margin_cap)
- NEW slot DD_SATURATION_FLOOR_ADAPTIVE_INDEX=458
- Producer dd_saturation_floor_update_kernel.cu — p75(per-env DD_MAX) × 1.5
via Welford `mean + Z_75 × sigma` estimator with `max(p75, mean)` robustness
guard, mirrors P0-A REWARD_POS_CAP producer pattern (Pearl-A bootstrap +
Welford α=0.01)
- Cold-start fallback: 0.25f (DD_SATURATION_FLOOR_DEFAULT in state_layout.cuh)
- Bounds: [0.10, 0.50] (Category-1 dimensional safety)
- Distinct from SP15_DD_THRESHOLD_INDEX=421 (the SP15 quadratic DD-penalty
*trigger* threshold, a *lower* bound; this slot is the *upper* end of the
linear position-size scaling ramp dd_scale = max(0.05, 1.0 − dd_frac/floor))
- Threaded `isv_signals_ptr` into `apply_margin_cap` with NULL-tolerant
cold-start fallback to DD_SATURATION_FLOOR_DEFAULT
- 4 oracle tests (Pearl-A bootstrap, no-DD guard, bounds clamp, Welford EMA)
Item 2: Legacy compute_drawdown_penalty path → Case A (DELETED)
- Decision rationale: SP15's quadratic asymmetric DD penalty
(compute_sp15_final_reward_kernel.cu:154 via sp15_dd_penalty helper) runs
unconditionally as a post-modifier on the SP11-composed reward with
ISV-driven λ_dd (slot 420) and DD threshold (slot 421). Layering the legacy
linear-ramp penalty inside the SP11 composer on top of the SP15 quadratic
creates double-counting of DD shaping — exactly the code-smell the Class A
audit was designed to eliminate. Per `feedback_no_legacy_aliases.md` and
`feedback_no_partial_refactor.md`.
- Atomic deletion across:
- `compute_drawdown_penalty` device function (trade_physics.cuh)
- Single call site at experience_kernels.cu:3822
- `dd_threshold` and `w_dd` kernel arguments
- `w_dd` Rust config field (gpu_experience_collector.rs +
trainers/dqn/config.rs DQNHyperparameters)
- `w_dd` profile section + dispatch (training_profile.rs RewardSection,
OptimizableParameterRanges, FixedRewardParameters, ParamLookup
dispatch, profile→hyperparam mapping, test assertion)
- `w_dd *= rki` risk-intensity multiplier (config.rs)
- `w_dd` TOML keys (dqn-hyperopt.toml × 2, dqn-localdev.toml,
dqn-production.toml, dqn-smoketest.toml)
- Stale doc comments on hyperopt/adapters/dqn.rs + config.rs
risk_intensity field
- `config.dd_threshold` SURVIVES (still consumed by `launch_sp15_dd_state`
as the dd_budget for DD_PCT scaling). Documented in field comment.
ISV_TOTAL_DIM: 458 → 459 (Item 1 adds 1 slot; Item 2 is pure deletion)
Cumulative WR-plateau fix series (this is commit 7):
- Class C bug 1 + P0-B (8f218cab2)
- P0-C (316db416b)
- P0-A (394de7d43)
- P1 wiring (c4b6d6ef2) — 1 of 4 wireable
- P0-A downstream (657972a4b)
- P1 producer (87d597d5d)
- audit-fix 4-A (this commit)
Verification: 16 sp14_oracle_tests pass (incl. 4 new), 36 sp15_phase1_oracle_tests
pass, 12 sp14_isv_slots layout tests pass, 4 state_reset_registry tests pass
(every-FoldReset-arm-has-dispatch contract holds), workspace cargo check clean.
Per feedback_isv_for_adaptive_bounds + feedback_no_partial_refactor +
feedback_no_legacy_aliases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflicts auto-resolved with -X theirs strategy. sp5 has more
evolved MappedF32Buffer API + more recent work; main's via-pinned
cleanup landed in parallel as a lateral migration. Taking sp5's
side preserves the cleaner consolidated upload pattern.
The *_via_pinned helpers (clone_to_device_f32_via_pinned,
upload_f32_via_pinned, clone_to_device_i32_via_pinned,
upload_i32_via_pinned, upload_u32_via_pinned) were temporary scaffolding
during the SP4 mapped-pinned migration. The canonical replacement is
MappedF32Buffer / MappedI32Buffer / MappedU32Buffer in the same module —
direct allocation + write_from_slice + device_ptr with no additional
indirection.
Each caller now inlines the staging + DtoD pattern directly:
- Allocate MappedXxxBuffer (cuMemHostAlloc DEVICEMAP)
- Write data via write_from_slice (no memcpy, mapped pinned coherence)
- alloc_zeros::<T> CudaSlice for device-resident destination
- memcpy_dtod_async from staging.dev_ptr to dst
- stream.synchronize() before staging drops
This commit migrates all callers atomically (per
feedback_no_partial_refactor) and removes the now-unused helpers in the
same commit. grep -rn 'via_pinned' returns 0 matches after this lands.
The two local file-private helpers in gpu_experience_collector.rs
(upload_host_to_cuda_f32_via_pinned / upload_host_to_cuda_i32_via_pinned)
were already correctly using MappedF32Buffer directly; they were renamed
to drop the _via_pinned suffix.
Touched: gpu_attention.rs, gpu_backtest_evaluator.rs, gpu_dqn_trainer.rs,
gpu_experience_collector.rs, gpu_her.rs, gpu_iql_trainer.rs,
gpu_iqn_head.rs, gpu_ppo_collector.rs, gpu_tlob.rs, gpu_walk_forward.rs,
gpu_weights.rs, mapped_pinned.rs, mod.rs, hyperopt/adapters/mamba2.rs,
hyperopt/adapters/ppo.rs, trainers/dqn/trainer/training_loop.rs,
trainers/ppo.rs, docs/dqn-wire-up-audit.md (18 files, +930/-363 LOC).
Cargo check workspace clean. Cargo test ml --lib: 925 passed, 17 failed
(all 17 are pre-existing GPU/data infrastructure failures unrelated to
this change).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Fix 29 audit rows #14 and #15 (Bug-1 contract drift in val/HPO
close-price extraction). Both sites read `target[0]` thinking it is
raw_close, but post Bug-1 (commit `5a5dd0fed`) `target[0]` is
preproc_close (z-normed log-return). The `fv[3]` fallback path is
unreachable on every production code path because
`set_val_data_from_slices` (in `dqn/trainer/mod.rs:1704`) always yields
`Vec<f64>` of length 6 from `[f64; 6]` slices post `TARGET_DIM=6` bump
(commit `063fd2716`), so `target.len() >= 2` is an always-true guard.
Per `feedback_no_hiding` the dead fallback is removed in the same edit
rather than left as a silent wrong-units path.
Sites fixed:
- crates/ml/src/trainers/dqn/trainer/metrics.rs:576
(val window_prices for GpuBacktestEvaluator)
- crates/ml/src/hyperopt/adapters/dqn.rs:2492
(HPO adapter val_close_prices for window-aggregated backtest)
Both now read `target[TARGET_RAW_CLOSE]` (col 2). Both import the named
constant from `crate::fxcache` so a future column rename moves the call
site with the writer (Fix 27 prevention pattern).
Verification:
- SQLX_OFFLINE=true cargo check -p ml --offline (8.03s) clean.
- No GPU code changed; cubin not affected.
Affects val Sharpe annualization + window equity curves on training
metrics; affects HPO val score on every trial. Pre-fix would yield
"prices" of magnitude ~stddev(log_return) (~7e-5 for ES 1-min) feeding
into PnL math that expects dollar prices, producing degenerate
backtest output. Post-fix prices are real raw_close values.
References Fix 27 Bug B (host-side Welford) — same kind of bug surfaced
at val/HPO consumer rather than training Welford. References
feedback_no_hiding (delete unreachable fallbacks rather than leaving
silent wrong-units fall-through), feedback_no_partial_refactor (both
consumers of the same (fv, target) tuple convention migrate together),
feedback_trust_code_not_docs (the `target.len() >= 2` guard read as
defensive but was masking a contract drift).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- trainers/ppo.rs: 2 sites (features, targets in set_raw_market_data)
- hyperopt/adapters/ppo.rs: 3 sites (features, targets in upload path;
action_indices in run_gpu_backtest forward closure)
- hyperopt/adapters/mamba2.rs: 1 site (gpu_to_stream)
All clone_htod_f32 and bare stream.clone_htod calls rewritten to
mapped_pinned::clone_to_device_{f32,i32}_via_pinned.
Audit row appended (Fix 7) in docs/dqn-gpu-hot-path-audit.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Smoke and localdev profiles previously used data_source = "ohlcv", divergent
from production (mbp10). Mismatch silently produced cache-key collisions in
the SHA256 hash path: smoke fxcache could not be loaded by production-shape
training without an explicit override. Local-dev fxcache regen also required
remembering to pass --data-source ohlcv to match.
Globalize mbp10 as the default everywhere it isn't deliberately overridden:
- config/training/dqn-smoketest.toml: data_source = "mbp10"
- config/training/dqn-localdev.toml: data_source = "mbp10"
- training_profile.rs: doc Default → "mbp10"
- trainers/dqn/config.rs: Default impl → "mbp10"
- hyperopt/adapters/dqn.rs: default → "mbp10"
- examples/precompute_features.rs: doc updated
- fxcache.rs / feature_cache.rs: discover_and_load + cache-key tests
use "mbp10" arguments
- docs/dqn-wire-up-audit.md: new entry per Invariant 7
Documentation strings retained "ohlcv" only where they document the two
available choices (config.rs:946, training_profile.rs:83).
Local fxcache regenerated to v6 mbp10:
test_data/feature-cache/13c0b086a975cc7e2384377a2cd0e97738c9410292fcfecb5807c29bf885cb48.fxcache
(175874 bars, 55 MB, OFI_DIM=32). Stale v5 ohlcv fxcache untracked
from git index (already gitignored post-79578bbaf).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eliminates the f64→f32 cudarc ABI trap (feedback_cudarc_f64_f32_abi.md,
task #82) at the type level: hyperparameters consumed by CUDA kernels
now live as f32 in Rust, cast once at the TOML/PSO ingest boundary
instead of at every kernel call site.
Structs changed:
- DQNHyperparameters (crates/ml/src/trainers/dqn/config.rs) —
~85 scalar fields migrated from f64 → f32. Covers all
kernel-facing scalars: reward weights (w_pnl/w_dd/w_idle,
dd_threshold, cea_weight, micro_reward_*, price_confirm_weight,
book_aggression_weight, hold_quality_weight), exploration
(epsilon_* and the 4 branch mults, noisy_sigma_*, count_bonus,
noise_sigma, q_gap_threshold), distributional RL (v_min, v_max,
reward_scale, iqn_lambda, qr_kappa, spectral_*,
gradient_collapse_multiplier), fill simulation (5 fill_*
fields), risk/Kelly (kelly_fractional, kelly_max_fraction,
max_leverage, max_position_absolute, minimum_profit_factor),
ensemble/curiosity (curiosity_weight,
curiosity_q_penalty_lambda, ensemble_*, beta_*, variance_cap),
anti-LR (anti_lr_*, adversarial_dd_threshold,
beta_penalty_strength), walk-forward (wf_*),
experience (avg_spread, transaction_cost_multiplier,
holding_cost_rate, churn_penalty_scale, contract_multiplier,
margin_pct, tick_size, bars_per_day, cash_reserve_percent),
misc kernel scalars (gamma, tau, huber_delta, q_clip_*,
shrink_perturb_*, regime_replay_decay, per_alpha,
per_beta_start, dt_target_return, etc.). Also
`noisy_epsilon_floor: Option<f32>` and
`count_bonus_coefficient: Option<f32>`.
- `computed_v_min` / `computed_v_max` now return f32.
- `compute_max_position` returns f32 (f64 internally for the
notional division).
Fields preserved as f64 (precision-sensitive, NOT kernel-facing
scalars — per task spec and feedback_cudarc_f64_f32_abi.md):
- `learning_rate` — tested at 1e-10 tolerance; f32 rounds to
2e-12 for a 1e-5 LR.
- `entropy_coefficient` — tested at 1e-9 tolerance.
- `weight_decay` — tiny 1e-5..1e-3 range.
- `adam_epsilon` — 1e-8 default; f32 preserves denorms here but
paired with weight_decay/learning_rate for symmetry.
- `gradient_clip_norm: Option<f64>` — grad norms are f64
accumulators by project convention.
- `min_loss_improvement_pct`, `q_value_floor` — early-stopping
long-horizon stats.
- `cql_alpha` — flows into DQNConfig (ml-dqn) still f64.
- `min_learning_rate`, `lr_min` — paired with learning_rate.
- Family intensity scalars (6 `*_intensity` fields) — f64 PSO
search space; the intensity applies via `as f32` at each call
site in `apply_family_scaling`.
No checkpoint format change: DQNHyperparameters has
`#[derive(Debug, Clone)]` only (not Serialize/Deserialize), so the
TOML-ingest path is the only serde boundary and already casts
explicitly via `hp.field = v as f32;` in
`DqnTrainingProfile::apply_to`. PSO hyperopt bounds stay f64 in
`SearchSpaceSection` and cast at the adapter boundary.
Call-site impact:
- ~30 `as f32` casts removed from hot paths (fused_training
FusedConfig builder, training_loop kernel launches,
constructor DQNConfig builder, action.rs GPU action selector,
trainer/mod.rs WF config). Kernels now receive the hp field
directly via `&hp.x`.
- ~75 `as f32` casts added at the `apply_to` / hyperopt-adapter
ingest boundary — the single conversion point.
- Cross-crate contracts (DQNConfig in ml-dqn, PortfolioTracker
in ml-core, GAECalculator, DropoutScheduler, NoisySigmaScheduler,
KellyOptimizerConfig, RewardConfig) retain their f64 signatures;
ml calls cast at the boundary with `f64::from(hp.x)` so the
contract is explicit and greppable.
Two test-side adjustments:
- `test_kelly_fields_are_public` now asserts `f32` for
kelly_fractional / kelly_max_fraction (these migrated).
- `test_early_stopping_termination` uses `f64::from(...)` to
preserve the f64 threshold computation against the now-f32
`gradient_collapse_multiplier`.
Verified:
- `SQLX_OFFLINE=true CARGO_INCREMENTAL=0 RUSTC_WRAPPER=sccache
cargo check --workspace` — clean, zero new warnings.
- `cargo check --workspace --tests` — clean.
- `cargo test -p ml --lib training_profile::tests` — all 18
migrated tests pass. The one pre-existing failure
(`test_production_profile_applies_all_sections` n_steps
mismatch, 1 vs 5) reproduces on stashed baseline, so it is
unrelated to this change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix kernel-read gap
Adds 12 features to the DQN input pipeline:
- 10 MicrostructureState::snapshot()[0..10] slots that were previously computed
every bar and then discarded before reaching fxcache: ofi_trajectory,
realized_variance, hawkes_intensity, book_pressure (weighted 10-level),
spread_dynamics, aggression_ratio, queue_depletion_asymmetry,
order_count_flux, intra_bar_momentum, regime_score.
- 2 TLOB-novel slots derived directly from Mbp10Snapshot:
order_count_imbalance = (Σbid_ct − Σask_ct) / Σ(bid_ct + ask_ct),
microprice_residual = (weighted_mid − mid) / mid.
Also fixes a production gap: ofi_acceleration (slot 18) and
toxicity_gradient (slot 19) were persisted to fxcache via OFI_DIM=20
but the OFI embed kernel (experience_kernels.cu:6146-6173) only read
[0..18), silently discarding them every bar. Kernel extended to
consume full SL_OFI_DIM=32.
Dimension bumps (all 8-aligned):
OFI_DIM 20 → 32
FXCACHE_VERSION 4 → 5 (invalidates existing caches; regen via
precompute_features)
STATE_DIM 96 → 104
PADDING_DIM 4 → 0 (OFI expansion consumed padding, still 8-aligned)
STATE_DIM_PADDED 128 (unchanged)
OFI_EMBED_IN 18 → 32 (MLP input width; W/grad/Adam/m/v buffers
resized in lockstep via named constants)
fxcache regen results (175874 bars ES.FUT 2024-Q1):
deltas_nonzero: 175781 / 175874 (99.9 percent)
book_aggression: 102137 / 175874 (58.1 percent)
microstructure[20-30): 175874 / 175874 (100 percent)
tlob_novel[30-32): 133615 / 175874 (76.0 percent)
Compile status: SQLX_OFFLINE=true CARGO_INCREMENTAL=0 cargo check
--workspace --tests passes cleanly (0 errors, pre-existing warnings
only).
Test results:
fxcache roundtrip (unit + integration): PASS (4+6 tests)
magnitude_distribution smoke: ran through epoch 1 successfully
(OFI_DIAG fires, state_dim=104 confirmed, feature_dim=74 in
validation kernel); epoch 2 OOM on local RTX 3050 Ti (4 GB) —
expected hardware limit from state_dim growth. Full 20-epoch run
requires L40S/H100 CI verification.
multi_fold_convergence smoke: not verified locally (same VRAM
ceiling applies). L40S/H100 CI verification required.
The new slots follow the existing OFICalculator/MicrostructureState
pattern and consume signals already computed by ml-features — no new
crate, no ONNX, no stubs. All 12 sources were audited against their
implementation before persistence; every slot traces back to real
Mbp10Snapshot or MicrostructureState math.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- ml/Cargo.toml: describe why ndarray's blas feature stays disabled
(CI compile pool has no libopenblas-dev; GPU cuBLAS handles the
hot path) rather than labelling it a TODO.
- dbn_sequence_loader.rs: Wave C regime-detection branch emits zeros
when only that flag is enabled; the live feed lives under WaveD.
Reword from "TODO Wave C" to a description of that superseding.
- ensemble/adapters/{liquid,tggn,tlob,xlstm}.rs: checkpoint loading
currently constructs fresh GpuLinear weights and logs a runtime
warning so the ignored checkpoint path is visible. No new
functionality, just reword the repeated TODO.
- ensemble/model_adapter.rs: the neutral-prediction adapter is
guarded by the ensemble's confidence threshold (0.0 = filtered),
making it a no-op stub used for end-to-end wiring. Describe that
contract explicitly.
- hyperopt/adapters/tft.rs: the input_dim=51 line is load-bearing
(5 static + 10 known + 36 unknown matches the CUDA layout). Drop
the "should be 42" aside.
- trainers/tft/trainer.rs: initialize_optimizer returns Err until
GpuAdamW is wired; the `let _ = &self.optimizer;` anchor in the
training loop keeps the migration target visible.
- trainers/tlob.rs: save_checkpoint / serialize_model both surface
errors until GpuVarStore safetensors serialisation lands. Mark
the gap declaratively rather than as a TODO.
- transformers/mod.rs: only AttentionMask is implemented in the
attention submodule; drop the aspirational re-export list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mass deletion of the "v7 gem" reward terms identified in the Phase 1
inventory as category errors — each one rewarded an outcome-adjacent
behavior instead of encoding the underlying physics, and each
empirically hurt validation metrics more than it helped training
stability.
Deleted from experience_kernels.cu and all plumbing (Rust configs,
launch args, hyperopt logs, TOML entries):
* order_credit_weight - reward redundant with compute_tx_cost
(order_type_idx already differentiates
fills by order type)
* risk_efficiency_weight - reward double-counted drawdown penalty
asymmetrically (only on winners)
* urgency_credit_weight - reward was vol-normalized unrealized P&L,
pure rename of core return
* commitment_lambda - triple-counted churn + tx_cost
* w_dsr - kernel wrote DSR EMA but no longer added
to reward (dead); removed the EMA
bookkeeping too
* dsr_eta - kernel arg for the deleted DSR EMA
* position_entropy_weight - rewarded action-bucket diversity
regardless of outcome; histogram buffer
+ zero-init removed too
* exit_timing_weight - already inactive (used raw_next future
price, comment-deleted earlier)
* ofi_reward_weight - dead plumbing; OFI already passed as
feature through state[OFI_START..]
* opportunity_cost_scale - penalized flat when Q-gap wide;
redundant with Q-values themselves
Kernel arg count: experience_env_step_batch shrank from ~55 to ~45 args.
Rust-side config surface reduced correspondingly.
Results on E1 smoke test (20-epoch):
BEFORE any Phase 2 work:
Val Sharpe -120 to -150, MaxDD 10-15%, Sharpe_raw -0.39
AFTER reward_noise + Kelly (both envs) + urgency + this sweep:
Val Sharpe -17 to -22 (7× better)
Val MaxDD 0.27% (40× better)
Val Sharpe_raw ~-0.09 (4× better)
Training Sharpe_raw ~0 (stabilized from ±20 swings)
Final q_gap 0.1712 (highest yet, collapse mechanism fine)
The extreme train-Sharpe swings (+17 one epoch, -13 next) were not
learning dynamics — they were shaping-term noise. Core reward (P&L +
drawdown + churn + holding + tx_cost + Kelly physics cap) gives training
metrics that actually reflect what the model does.
Inventory doc (docs/superpowers/specs/2026-04-21-phase1-reward-inventory.md)
extended with a "better-form taxonomy" section: every deleted gem has
a correct layer it belongs to (physics, feature, diagnostic, gradient-
level regularization — not reward). Kelly cap and Q-target smoothing
are already relocated; others are scheduled per the taxonomy's P1/P2/P3
priority list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 second relocation: the kelly_sizing_weight reward penalty
(experience_kernels.cu:1727-1746 — penalized deviation from Kelly-optimal
sizing) is deleted. Kelly is now a physics constraint in trade_physics.cuh:
the environment refuses to let the agent over-lever, not the reward
scoring the agent for matching a formula.
New helper in trade_physics.cuh (shared device function, reusable by
the forthcoming unified env kernel):
kelly_position_cap(win_count, loss_count, sum_wins, sum_losses,
max_position, safety_multiplier)
Applied in experience_kernels.cu between margin cap and execute_trade,
with health-coupled safety multiplier:
safety = 0.5 + 0.5 × health
- health=1 (healthy): full Kelly — trust the learned policy
- health=0 (collapsing): half Kelly — constrain when decisions less
reliable
Cold-start warmup (critical — otherwise balanced priors yield kelly_f=0
until real trades accumulate, starving Q-learning):
maturity = min(1.0, total_trades / 10)
effective_kelly = maturity × kelly_f + (1 - maturity) × 0.5
Early on (0 trades): cap dominated by 50% floor.
As real trades accumulate (10+): pure data-driven Kelly.
Validation env (backtest_env_kernel.cu) does NOT yet get the Kelly cap —
that requires extending its portfolio state or adding a separate
kelly_stats buffer, which naturally belongs in the Phase 3 unified env
kernel refactor. The current asymmetry is a KNOWN temporary — training
is constrained, validation is not — and will be resolved when both
kernels share the same env_step() device function.
Also completes removal of kelly_sizing_weight from all plumbing:
- experience_kernels.cu: kernel arg deleted
- gpu_experience_collector.rs: launch arg, config field, default
- training_loop.rs: hyperparam propagation
- config.rs: field, default, intensity clamp (with tombstone)
- hyperopt/adapters/dqn.rs: log reference
- config/training/*.toml (6 entries across 4 files): orphan configs
(none were wired to a profile parser field)
Verification:
- cargo check -p ml --lib clean
- E1 smoke test passes: final epoch q_gap=0.1109, health=0.51 (warmup
floor of 0.5 gives early exploration enough room; floor of 0.25
was too tight and failed at q_gap=0.0496)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 2 kick-off from the env-unification design. First relocation: the
reward_noise_scale field that perturbed training rewards is deleted, and
its regularization effect moves to the correct layer — Q-target label
smoothing in c51_loss_kernel.cu — now health-coupled rather than fixed.
Before:
- reward += pseudo_noise × max(|reward| × 0.05, 0.01) (in env reward path)
- Q-target label smoothing = fixed LABEL_SMOOTHING_EPS = 0.01
After:
- Reward untouched by noise. Core reward = actual outcome + aligned penalties.
- Q-target label smoothing eps_eff = 0.02 × (1 − health) read from ISV[12]
- health=1 (healthy): eps_eff=0, sharp targets preserved
- health=0.5: eps_eff=0.01, matches old fixed behavior at mid-health
- health=0 (collapsing): eps_eff=0.02, maximum regularization prevents
overcommitment to the collapsed distribution
Why health-coupled:
Same insight as the distillation SAXPY fix — every fixed kernel scalar is
a temporal-coupling candidate when we have the ISV pinned buffer available.
Regularization strength should scale INVERSELY with network health: it's
most needed exactly when things are falling apart.
Files touched:
- c51_loss_kernel.cu: LABEL_SMOOTHING_EPS const replaced with
LABEL_SMOOTHING_BASE + in-kernel health read from isv_signals[12]
- experience_kernels.cu: deleted reward noise block + kernel arg
- gpu_experience_collector.rs: dropped launch .arg + config field + default
- training_loop.rs: dropped hyperparam propagation
- config.rs: deleted field + intensity clamp + default (with tombstone)
- hyperopt/adapters/dqn.rs: dropped log reference
- config/training/*.toml (4 files): dropped orphan reward_noise_scale
entries (none were being parsed — the profile parser had no field)
Verification:
- `cargo check -p ml --lib` clean
- E1 smoke test passes: final q_gap=0.1190, health=0.52 (health-coupled
smoothing at ~mid-health matches old fixed behavior, collapse-prevention
mechanism intact)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OFI (Order Flow Imbalance) features are mandatory — the model is worthless
without MBP-10 order book data. Every silent fallback that degraded to zeros
has been converted to a hard error.
Critical fixes:
- build_batch_states used *8 instead of *20 for OFI dimensions — every
walk-forward backtest was reading corrupted OFI features from adjacent memory
- precompute_features early exit skipped cache rebuild when stale v2/v3
cache existed with has_ofi=false — now validates has_ofi before skipping
- 14 silent OFI fallback paths converted to hard errors across data loading,
training loop, experience collector, state construction, metrics, hyperopt
Dead code removed (-751 lines):
- DoubleBufferedLoader (superseded by init_from_fxcache)
- GpuBufferPool (superseded by init_from_fxcache)
- DqnGpuData::upload legacy method (no OFI support)
- CPU training fallback path (CUDA always required)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
target_dim expansion: adds raw_open (OHLCV) and mid_price_open
(MBP-10 midpoint at bar formation) to fxcache targets. FXCACHE_VERSION
2→3 for auto-rebuild. Legacy v2 files handled with close-price fallback.
Spec v5 adds 3 pearls:
- Bar duration encoding in Mamba2 (continuous-time SSM awareness)
- Order book center of mass from all 10 MBP-10 levels (aggression signal)
- Retrospective hold quality bonus (teaches exit timing)
Plus: Hold action (4th direction), DSR Sharpe EMA fix, counterfactual
magnitude/order sign fix, MFT mid-price mark-to-market.
OFI embed MLP now 18→10 (was 16→8). Mamba2 width SH2+10 (was SH2+8).
Attention width D+10 (was D+8).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed: enforce_hold(), min_hold_bars from config/kernels/backtest.
Added: holding_cost_rate (inventory penalty), churn_threshold_bars +
churn_penalty_scale (graduated flip penalty) to reward in
experience_env_step and backtest kernels.
The model learns optimal hold timing from cost signals:
- Per-trade tx cost prevents churning (existing)
- Inventory penalty makes large positions expensive to hold
- Churn penalty graduates cost for rapid flips
- Temporal attention learns when holding cost > expected profit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fxcache: OFI_DIM=20 (pub const), RECORD_F64_COUNT=66, 272 bytes/bar.
constructor: state_dim 74→86 (aligned 88) with OFI.
experience collector: ofi_dim detection 8→20.
All [f64; 8] → [f64; 20]. Existing 8 features preserved at 0-7.
New 12 features zero-padded until precompute is extended.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
evaluate_dqn_graphed now takes Option<&mut dyn QValueProvider>.
Training eval passes Some(fused_ctx), standalone eval uses closure path.
Removed dead evaluate_dqn non-graphed branch from evaluate_baseline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the evaluator's independent CublasForward with QValueProvider
trait that routes Q-value computation through the trainer's graphed
CublasForward. This eliminates the separate cuBLAS handle that caused
evaluation non-determinism.
Architecture:
- QValueProvider trait in q_value_provider.rs (compute_q_values_to)
- FusedTrainingCtx implements it (chunks through trainer batch_size=64)
- GpuDqnTrainer::compute_q_values_graphed captures eval forward in
CUDA Graph (cuBLAS forward + compute_expected_q) on first call
- Pre-captured at deterministic point (right after mega graph)
- evaluate_dqn_graphed takes &mut dyn QValueProvider (mandatory)
Cleanup (-398 lines):
- Deleted evaluator's internal CublasForward + all chunked scratch buffers
- Deleted compute_q_values (non-graphed), flatten_weights_for_cublas,
ensure_cublas_ready, compute_backtest_param_sizes
- Deleted evaluate_dqn (fallback path)
- Removed dead imports and stale comments
Training: fully bit-identical across runs (CUDA Graph replay).
Evaluation: deterministic within process (graph replay), minor
variation across processes (cublasLtMatmul capture non-determinism).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 4-branch DQN (direction x magnitude) had 3 degenerate variants
(Short25, Flat, Long25) that all mapped to 0.0 target exposure when
direction=Flat, causing 82% Flat collapse. Collapse these into a
single Flat variant, giving 7 levels (ShortSmall/Half/Full, Flat,
LongSmall/Half/Full) and 63 total factored actions (7x3x3).
- ExposureLevel enum: 9 variants -> 7 (add direction/magnitude/from_dir_mag)
- FactoredAction: 81 -> 63 total actions, from_index/to_index updated
- DQN epsilon-greedy: use from_dir_mag() instead of dir*3+mag indexing
- DQN config: num_actions default 9 -> 7
- PPO action space: 45 -> 63 actions, action masking updated
- Signal adapter CUDA kernel: 5-bin -> 7-bin exposure aggregation
- All tests updated for new variant names and index ranges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete bf16 elimination across all crates (ml, ml-core, ml-dqn, ml-ppo,
ml-supervised). Zero half::bf16, __nv_bfloat16, or CudaSlice<half::bf16>
references remain (verified by grep).
CUDA: All 60+ .cu kernels and .cuh headers converted to native float.
- Half-precision intrinsics (__hmul, __hadd, __hdiv) → float operators
- atomicAddBF16 → native atomicAdd(float)
- bf16 wrapper functions → f32 identity passthroughs
Rust: All CudaSlice<half::bf16> → CudaSlice<f32> across 90+ files.
- htod_f32_to_bf16/dtoh_bf16_to_f32 → htod_f32/dtoh_f32 (direct, no conversion)
- Deleted bf16 mirror infrastructure (DuelingWeightSetBf16, alloc_bf16_mirror, etc.)
- Renamed params_bf16→params_flat, d_value_logits_bf16→d_value_logits, etc.
- Fixed .to_f32() sed damage on Decimal::to_f32() and rng.f32()
FxCache: Single f32 disk format (was bf16/f64 dual-version).
- Deleted --bf16 CLI flag from precompute_features
- PVC cache files need regeneration via precompute_features
TF32 tensor cores activated via cublasLtMatmul CUBLAS_COMPUTE_32F — no
explicit TF32 types needed. Storage is pure f32 everywhere.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
H100 baseline (20 epochs) showed gradient collapse at epoch 10: C51
cross-entropy converges its distributional fit before the policy converges,
leaving zero gradient signal. The collapse happened 5 epochs after C51
reached alpha=1.0 (pure C51, zero MSE).
Fix: cap the C51 alpha ramp at c51_alpha_max (default 0.5) so MSE always
contributes (1 - alpha_max) of the primary gradient. MSE loss measures
Q-error directly and only goes to zero when Q-values are correct, not
just when the distribution shape is right.
- c51_alpha_max added to DQNHyperparameters (default 0.5)
- Added to PSO search space as 15th dimension (range [0.3, 0.9])
- Added to TOML profiles: smoketest, production, hyperopt
- Training loop caps alpha ramp at alpha_max instead of 1.0
- All 907 ml tests + 300 ml-core tests + 6 smoke tests pass
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes:
1. spread_cost removed contract_multiplier ($50) — was computing in dollars
but PnL is in points, making spread 12.5× actual. The model saw every
trade as guaranteed loss → preferred Flat.
2. Mirror universe now negates ALL 17 directional features (returns, MACD,
Bollinger, SMA ratios, regression slope, CUSUM) + flips RSI. Was only
negating 4, teaching wrong direction on 50% of epochs.
3. tx_cost_multiplier updated to 0.18 bps (IBKR ES RT=$4.50/contract).
min_hold_bars=10 for volume bars (~26s at 23 bars/min).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire 4th magnitude branch (size=3) into BranchingDuelingQNetwork construction,
DQNAgentType::branch_sizes() return type, training loop diversity metrics,
backtest config in metrics.rs and hyperopt adapter, and DT pre-training config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Log v7 reward params after apply_family_scaling() so their scaled values are
visible in trial logs. Update loss_shaping_intensity doc comment to reference
v7 params (CEA, order credit, risk efficiency) instead of v6.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The preload's key-based fxcache lookup failed because the cache key
hash depends on file metadata (size+mtime) which differs between
the precompute pod and hyperopt pod (different PVC mount paths).
Fix: load the first .fxcache file from the cache directory directly.
The precompute step generates exactly one fxcache per dataset — no
ambiguity. Eliminates 285s DBN fallback loading.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace stream.synchronize() with cuStreamWaitEvent for cross-stream
visibility of restored weights. The trainer writes best params on its
stream, records a CudaEvent, and the evaluator waits on that event
on ITS stream via cuStreamWaitEvent — purely GPU-side dependency,
zero CPU blocking.
Previous fix (stream.synchronize) was a CPU stall that hid the issue.
This is the proper CUDA approach: inter-stream event dependencies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
If a trial corrupts GPU state (CUDA_ERROR_ILLEGAL_ADDRESS from extreme
training), the inter-trial sync now drains the error instead of
propagating it — preventing all subsequent trials from failing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The data preload created a dummy DQNTrainer with buffer_size=1, which
fails the GPU PER capacity check (capacity must be >= batch_size).
This caused the preload to fail silently, falling back to per-trial
data loading from disk — 50 × 5s = 250s wasted.
Fix: set buffer_size = max(batch_size, 1024) so the PER allocation
succeeds. The preload trainer doesn't train — it just loads data into
a shared Arc for all trials.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three issues fixed:
1. Best checkpoint was loaded via safetensors → Candle VarMap, but the
evaluator now reads from fused_ctx GPU buffers. The VarMap weights
were never used — evaluator always saw last-epoch weights, not best.
2. Added save_best_params/restore_best_params to FusedTrainingCtx:
- save: async DtoD copy params_bf16 → best_params_snapshot
- restore: async DtoD copy best_params_snapshot → params_bf16 + unflatten
Zero sync, zero Candle, zero safetensors roundtrip.
3. Hyperopt evaluation now calls restore_best_gpu_params() instead of
loading safetensors checkpoint. Pure GPU path end-to-end.
Added params_bf16()/params_bf16_mut() accessors to GpuDqnTrainer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Same bug as the training loop validation: hyperopt's backtest evaluator
read weights from agent.get_q_network_vars() (Candle VarMap) which is
NOT synced during fused CUDA training. All previous hyperopt runs were
evaluating with epoch-0 weights — Sharpe scores were meaningless.
Fix: read directly from fused_ctx.online_dueling_ref() /
online_branching_ref() via new DQNTrainer::fused_online_weights() method.
Falls back to error if fused_ctx not available (required in CUDA builds).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Preloaded fxcache truncated to max_bars from training profile
(1.1M → 5000 bars for smoketest, ~40x faster per trial)
- Consolidated 3 redundant DqnTrainingProfile::load() calls into 2
(VRAM gate reuses base_hp in train_with_params)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Load TOML profile BEFORE VRAM budget gate so hidden_dim/batch_size
reflect the actual profile (smoketest=64 vs production=256)
- Fixes backtest_metrics=None: VRAM gate was pruning all trials using
conservative() defaults (hidden=256, batch=1024) on RTX 3050
- Add test_hyperopt_trial_produces_backtest_metrics regression test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add training_profile field to DQNTrainer (default: "dqn-production")
- Smoketest uses "dqn-smoketest" profile for RTX 3050 compatible sizes
- DQNTrainer fields (data_source, mbp10_data_dir, trades_data_dir) are
source of truth — TOML profile provides training params only
- Fix relative path resolution in preload_data for mbp10/trades dirs
- Add preload_data() call in hyperopt test (fxcache shared via Arc)
- Buffer size floor of 1024 in hyperopt param application
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds `has_ofi: bool` to `FxCacheData` and the fxcache binary header
(stored in `reserved[0]`). Propagates through load_fxcache, write_fxcache,
all callers (precompute_features, train_baseline_rl, hyperopt dqn adapter),
existing roundtrip tests, and adds two new has_ofi-specific roundtrip tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Different instruments (ES.FUT vs NQ.FUT) and data source modes (ohlcv vs mbp10)
now produce distinct .fxcache keys, preventing silent overwrites and wrong-bar-type
loads at cache lookup time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Normalize features once in precompute_features (single source of truth).
NormStats saved alongside .fxcache for inference denormalization.
Removed per-fold NormStats computation from train_baseline_rl, hyperopt
adapter, and smoketests — fxcache is pre-normalized, consumers use as-is.
NOTE: features still show raw price values (max=18000+) because
test_data/futures-baseline contains multiple symbols (ES, NQ, ZN, 6E)
mixed into one dataset. The feature extraction pipeline needs
investigation — log returns between different symbols produce garbage.
This is tracked as a separate task.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Epoch duration self-balances: bigger GPU → bigger auto-scaled batch →
fewer steps per epoch. The manual cap created 7 different values
(0, 8, 64, 100, 200, 300, 2000) across configs/tests/examples, making
behavior inconsistent between environments.
Removed from: DQNHyperparameters, training profiles (smoketest,
localdev, production), CLI args, Argo templates, hyperopt adapter,
all test overrides, supervised example.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
