After training completes, the example now loads the best checkpoint
into a fresh DQN and runs inference on 5 synthetic state vectors,
printing action, max Q-value, and Q-spread for each sample. Errors
are handled gracefully with match on Result so the example never
panics on inference failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Validates the complete DQN checkpoint lifecycle: train 5 epochs with
DQNHyperparameters::conservative(), save via checkpoint callback, load
into a fresh DQN with architecture auto-detected from checkpoint tensor
metadata (noisy vs standard layers, state_dim), and run 100 inference
passes asserting valid action indices, finite Q-values, and non-zero
Q-values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OOM recovery loop halves `current_batch_size` on each retry, but
`train_step_single_batch()` and `train_step_with_accumulation()` both
called `agent.train_step(None)`, which makes the agent sample from its
replay buffer at the original `config.batch_size` -- ignoring the
reduction entirely. This meant OOM retries would always fail with the
same allocation size.
Fix: when `current_batch_size < hyperparams.batch_size`, manually
sample from the replay buffer at the reduced size and pass the explicit
batch via `agent.train_step(Some(experiences))`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace misleading doc comments and TODOs that described "true gradient
accumulation" with accurate documentation. The method runs N sequential
mini-batch training steps (each with its own optimizer.step()), not
gradient accumulation in the strict sense. Also switch debug log
messages from self.hyperparams.batch_size to self.current_batch_size
so they reflect the dynamically-adjusted batch size after any OOM
recovery halving.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wraps the train_step() dispatch in a retry loop (up to 3 attempts) that
detects CUDA OOM errors via string matching on the anyhow error chain and
halves the tracked batch size on each retry. Adds a current_batch_size
field to DQNTrainer for monitoring the effective batch size.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the static MAX_BATCH_SIZE=230 hard-coded cap with dynamic GPU memory
detection via AutoBatchSizer. When nvidia-smi is available, the trainer queries
actual free GPU memory and calculates an optimal batch size. When unavailable
(CI environments), it falls back to the previous static cap of 230.
Behavioral change: instead of returning an error when batch_size exceeds the
limit, the trainer now clamps it down to the safe maximum. This is more
user-friendly and enables OOM recovery in a later task.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The train() callback was changed to (epoch, data, is_best) but
this test still used (epoch, data). Update all 5 call sites.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Assertion 7 runs DqnStrategy through ValidationHarness with real
6E.FUT data to verify the full train->validate pipeline works
end-to-end. Produces 15 folds with finite Sharpe ratio.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Assertions verified on real 6E.FUT data:
1. All 20 epochs complete (no premature early stopping)
2. Loss decreases >5% (gradient flow works) — actual: 22.4%
3. All per-epoch losses are finite (no NaN/Inf)
4. Q-value divergence (model develops action preferences)
5. Checkpoint round-trip (save/load weight integrity)
6. Epsilon decayed below 0.5 (exploration schedule ran)
Also fixes divide-by-zero in triple_barrier.rs:103 when
entry_price_cents is zero (guard in both barrier tracker
and trainer caller).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add public accessor methods for smoke test verification:
- loss_history() — per-epoch training loss
- val_loss_history() — per-epoch validation loss
- get_agent_epsilon() — current epsilon from DQN agent
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single integration test that verifies the complete train → checkpoint →
validate pipeline works on real 6E.FUT data. 7 assertions covering
loss convergence, Q-value divergence, checkpoint round-trip, epsilon
decay, and Sharpe improvement vs untrained baseline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removed across 66 files:
- 49 instances of "// use crate::safe_operations; // DISABLED"
- 11 instances of "// use error_handling::{...}; // crate doesn't exist"
- 2 instances of "// use crate::Optimizer; // not available"
- 5 disabled test placeholder blocks (/* ... */) in ensemble/
- 1 disabled From impl in lib.rs (38 lines)
- 1 disabled test module in model.rs (113 lines)
- 1 disabled code block in integration/distillation.rs (41 lines)
- Various other disabled imports with explanation comments
All of this code references modules/crates that were removed during
prior refactoring waves and is preserved in git history. Removing it
reduces noise and makes the codebase easier to navigate.
1922 lib tests passing, compilation clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PPO's circuit_breaker.rs was a 276-line copy-paste of DQN's version
(only difference: "PPO" vs "DQN" in log messages). Since ppo/ppo.rs
already imports from dqn::circuit_breaker, the PPO copy was dead code.
Replace with a 6-line re-export module. Tests still pass via the
canonical DQN implementation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes dead code across 28 files:
- 31 commented-out "DISABLED" import lines (mostly safe_operations, error_handling)
- Commented-out module declarations in lib.rs (deployment, model_loader_integration, tests)
- Commented-out re-exports in lib.rs (training_pipeline, deployment::ModelVersion)
- Commented-out adaptive strategy modules in regime/mod.rs
All are in git history if ever needed. Net -74 lines removed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces ml/src/types/ohlcv.rs as the single source of truth for
OHLCVBar (DateTime<Utc>, f64). Replaces 13 identical struct definitions
scattered across features/, regime/, real_data_loader, and evaluation/.
The f32 backtesting variant in evaluation/metrics.rs is renamed to
OHLCVBarF32 to distinguish it from the canonical type. The regime_adx.rs
i64-timestamp variant was safely migrated since its timestamp field was
never accessed. The orchestrator's Bar alias is replaced with OHLCVBar.
39 files changed, -151 net lines removed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Created ml/src/types/ohlcv.rs as the single source of truth for OHLCVBar
(DateTime<Utc> timestamp, f64 OHLCV fields). Replaced all 13 duplicate
definitions across features/, regime/, real_data_loader, and evaluation/
with imports from crate::types::OHLCVBar.
Key changes:
- New: ml/src/types/mod.rs + ohlcv.rs with canonical OHLCVBar
(derives: Debug, Clone, Copy, PartialEq, Serialize, Deserialize + Default)
- Renamed: evaluation::metrics::OHLCVBar → OHLCVBarF32 (genuinely
different type: f32 fields, i64 timestamp for compact backtesting)
- Eliminated all import aliases (ExtractionOHLCVBar, RegimeOHLCVBar,
PriceOHLCVBar, VolumeOHLCVBar) in dbn_sequence_loader.rs and pipeline.rs
- Renamed regime::orchestrator::Bar → OHLCVBar (same fields, just aliased)
- Updated 39 files total (13 definitions removed, imports normalized)
1883 lib tests passing, compilation clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs full ValidationHarness on actual Databento 6E.FUT 1-minute OHLCV
bars (~30k bars). Extracts 15-dim features (5 OHLCV + 10 technical
indicators), configures DQN with walk-forward validation, and prints
a detailed report including DSR, PBO, permutation test, and per-regime
breakdown.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Critical fixes found during pre-GPU-test code audit:
- DSR SE formula: (kurt-1)/4 → kurt/4 for excess kurtosis input
- PBO CSCV: replace circular fold ranking with IS/OOS mean comparison
- DqnStrategy evaluate: fix off-by-one (loop over returns, not bars)
- DqnStrategy action mapping: handle small num_actions (3→5) directly
instead of FactoredAction decomposition (which maps all to Short100)
- Walk-forward: document rolling window as deliberate design choice
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move validation.rs to validation/financial.rs, create validation/mod.rs
with re-exports for backward compatibility, and remove orphan
numerical_tests.rs that referenced nonexistent types.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace fixed τ=0.005 Polyak averaging with cosine-annealed EMA schedule
(BYOL/MoCo v3). τ(t) = τ_final - (τ_final - τ_base)·(cos(πt/T)+1)/2.
Early training: τ ≈ 0.005 (fast adaptation while model is learning)
Late training: τ ≈ 0.0005 (stability to prevent bootstrap error drift)
This is critical for offline RL — fixed τ causes target drift that
accumulates over training since we can't collect corrective data.
CQL handles Q-value overestimation; annealed EMA handles target stability.
New config fields: tau_final, tau_anneal_steps. Set tau_anneal_steps=0
to revert to fixed τ behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 3 integration tests verifying the complete IQN+CQL training pipeline:
full training loop with both features, IQN-only mode, and CVaR risk-aware
action selection. Module re-exports for QuantileConfig/QuantileNetwork
were already present from Wave 26.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add copy_weights_from() to QuantileNetwork for target sync. Wire IQN
target updates into both Polyak averaging (soft) and hard copy (legacy)
update paths in train_step(). Also fix agent.rs tests that still
referenced state_dim=52 after consolidation to 51.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire IQN into select_action() greedy branch. When use_iqn is enabled,
computes quantile-based Q-values via uniform quantile sampling. CVaR mode
(use_cvar_action_selection) optimizes for worst-case outcomes instead of
mean, providing risk-averse action selection for conservative trading.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 3-way loss branch in train_step(): IQN > C51 > scalar. The IQN path
implements Dabney et al. 2018b using quantile Huber loss — no scatter_add
needed (bypasses Candle BUG #36). Also adds get_state_embedding() helper
to extract base Q-network hidden layer outputs for IQN, stores VarMap in
QuantileNetwork for optimizer access, and includes IQN vars in optimizer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add iqn_network and iqn_target_network as Option<QuantileNetwork> fields
to the DQN struct. When use_iqn is enabled, DQN::new() creates both networks
with QuantileConfig derived from DQNConfig (embed_dim from last hidden layer).
Each network gets its own VarMap for independent parameter tracking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix NaN panic at dqn.rs:1716 with unwrap_or(Ordering::Equal)
- Fix Adam epsilon 1e-8 → 1.5e-4 per Hessel et al. 2018
- Remove duplicate DistributionalType enum from quantile_regression.rs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Research-validated design for fixing critical DQN issues:
- CQL regularization for offline RL training on historical data
- IQN integration replacing broken C51 (Candle scatter_add bug)
- CVaR risk-aware action selection
- State dimension consolidation (51 is canonical)
- NaN panic fix, Adam epsilon fix per Rainbow paper
Verified against 20+ papers and 2026 SOTA.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract ~1,300 lines from the 4,755-line trainer.rs into four focused
sub-modules to improve maintainability and code navigation:
- monitoring.rs (290 lines): TrainingMonitor for per-epoch reward,
action, and Q-value tracking with health validation
- data_loading.rs (719 lines): Parquet/DBN data loading, MBP-10 OFI
integration, feature caching, and preprocessing pipeline
- risk.rs (145 lines): Volatility-adjusted epsilon, risk-adjusted
rewards, Kelly criterion sizing, and risk tracker updates
- features.rs (211 lines): Full feature extraction (51-dim), feature
statistics calculation, z-score normalization, synthetic features
All public API paths preserved via mod.rs re-exports. Fields accessed
across module boundaries changed to pub(crate) visibility.
Verified: cargo check --workspace (0 errors), cargo test -p ml --lib
(1,817 passed, 0 failed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
