- Replace Silverman's bandwidth (h = 1.06σn^(-1/5)) with Scott's rule
(h = 0.7σn^(-1/(d+4))) for tighter kernels in high-D parameter spaces
- Add best-trial injection: always evaluate EI at best known point plus
5 small perturbations (±5%), preventing optimizer from forgetting peaks
- Scale n_candidates dynamically: max(256, 8*n_dims) instead of fixed 100
- Reduce gamma from 0.25 to 0.15 when trials < 50 for tighter exploitation
- Wire model_name through PSO/TPE paths for per-trial Prometheus metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes from deep investigation audit (LOW/MEDIUM priority):
1. CVaR penalty: hard cliff (0 or 10) → smooth ramp with gradient signal
for PSO. Formula: min(10, max(0, -cvar-0.05)*200).
2. Clip outliers leakage: data_loading.rs now computes clip bounds from
training portion only (first 80%), then applies to full series.
Log returns and windowed normalize are causal (no leakage).
3. Noisy sigma scheduler: hyperopt now matches conservative() defaults
(enabled, initial=0.8, final=0.4) so hyperopt-found params
generalize to train_best without scheduler mismatch.
4. evaluate_supervised.rs: NormStats fallback from test data (leakage)
replaced with bail! matching evaluate_baseline.rs behavior.
5. Doc comments: stale 27D references updated to 31D (4 locations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add device_pool to DQN/PPO hyperopt trainers for round-robin GPU assignment
per trial. Binary detects all CUDA devices, scales VRAM budget by GPU count.
Single-GPU: no behavior change (pool of 1).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NoisyLinear layers have different parameter names (sigma_weight/sigma_bias)
than standard Linear layers. Hardcoding use_noisy_nets=false caused
checkpoint loading to silently fail when the training run used noisy nets,
resulting in empty folds in the eval report.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
num_atoms, dueling_hidden_dim, v_min/v_max, gamma were using defaults
instead of hyperopt values — causing tensor shape mismatch on checkpoint
load (e.g. output layer 45×200=9000 vs 45×51=2295). Also fixed use_iqn
to read from use_qr_dqn key (matching trainer's field mapping).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt found hold_penalty_weight=1.818, curiosity_weight=0.402, etc.
but walk-forward training used tiny defaults (0.01, 0.1), causing the
agent to learn "holding is optimal" → 0 trades across all 50 epochs.
Now passes: hold_penalty_weight, max_position_absolute, huber_delta,
entropy_coefficient, curiosity_weight, weight_decay, kelly_fractional,
kelly_max_fraction from hyperopt JSON to DQNHyperparameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Walk-forward DQN training hardcoded epsilon_start=1.0 with noisy_nets=true,
forcing pure random exploration for all 50 epochs (0 trades, 0 Sharpe).
Now reads epsilon, PER, dueling, distributional, noisy nets, and QR-DQN
params from the hyperopt JSON. When noisy_nets=true, defaults epsilon to
0.05 instead of 1.0.
Evaluator now falls back to highest dqn_fold{N}_epoch{E}.safetensors when
_best.safetensors doesn't exist (early stopping saves epoch checkpoints).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire LR scheduler to actually update the Adam optimizer (was logged but
never applied). Add update_learning_rate to DQN, RegimeConditionalDQN,
and DQNAgentType so decay_factor propagates through all agent variants.
Fix train/eval parity: CLI defaults 51→54 features, 3→45 actions;
DQN eval always uses 3-layer hidden_dims; PPO eval uses 5-layer value
network matching trainer. enhanced_ml.rs hardcoded config updated from
state_dim=16/num_actions=3 to 54/45.
Fix Candle F32/BF16 traps: replace `Tensor * 0.5` (f64 literal) with
broadcast_mul(Tensor::full(0.5_f32)) in quantile_regression.rs (2x),
dqn.rs Huber loss, and IQN gamma multiplication. Prevents panics on
Ampere+ BF16 GPUs.
Add urgency_weight() multiplier to training cost model in reward.rs
and portfolio_tracker.rs — urgency dimension (Patient/Normal/Aggressive)
now affects learned value function, matching evaluate_baseline.rs.
Fix equity tracking: additive (equity += ret) → multiplicative
(equity *= 1.0 + ret) in compute_metrics. Fix total return calc.
Fix PER beta annealing: epochs*70 → epochs*1000 (~1015 actual steps
per epoch for 130k bars / batch 128).
Fix silent target network freeze: mutex lock failure now propagates
error instead of silently skipping weight update.
Make save_checkpoint atomic (write .tmp then rename). Make NormStats
write atomic with error logging instead of silent discard.
Convert EnsembleConfig::new assert! → Result<Self, MLError> with
14 call site updates. Fix hyperopt result serialization to warn
instead of silently dropping to Value::Null.
Fix clippy MSRV mismatch: clippy.toml 1.75 → 1.85 matching Cargo.toml.
2732 tests pass, 0 clippy warnings across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQNAgent::load_from_safetensors: validate architecture hash before
loading weights (was bypassing validation entirely - P0 production gap)
- DQNEnsemble::save_to_directory: embed architecture metadata via
safetensors::serialize_to_file instead of bare VarMap::save (save/load
round-trip was guaranteed to fail)
- architecture_hash: hash hidden_dims.len() before values to prevent
theoretical collision between different-length configs
- train_baseline_rl: NormStats write now atomic (write .tmp then rename)
with cleanup guard on rename failure; serialization error is now loud
(error! + return None) instead of silently discarded
- train_baseline_rl: checkpoint rename failure cleans up orphaned .tmp
- hyperopt_baseline_rl: fix clippy single_match_else (match on equality
check -> if/else)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Wire position-delta cost model: costs only apply on position changes,
proportional to delta (not flat per-bar)
- Differentiate order types: Market=15bps, LimitMaker=5bps, IoC=10bps
- Apply urgency weight: Patient=0.5x, Normal=1.0x, Aggressive=1.5x
- Append 3 portfolio features (position, unrealized PnL, drawdown) to match
training's 54-dim state vector (was 51, causing shape mismatch on load)
- Fix default num_actions 3->45 to match training's factored action space
- Add --bars-per-year CLI flag for per-symbol annualization
(ES/NQ=347760, 6E=345000, ZN=105840)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Atomic checkpoint writes: write to .tmp then fs::rename() (POSIX atomic)
- Remove redundant thread::sleep(50ms) after CUDA tensor readback sync
- NormStats: bail on missing file instead of computing from test data (lookahead bias)
- q_value_std: warn when missing from training metrics instead of silent 0.0 fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change --optimizer default from "pso" to "tpe" since TPE has better
sample efficiency in 25D spaces. Fix clippy let_underscore_must_use
on CUDA sync tensor readback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add optimize_with_tpe() function that uses Tree-Parzen Estimator for
Bayesian hyperparameter optimization. Add --optimizer=tpe CLI flag
to hyperopt_baseline_rl binary (default: pso for backward compat).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split model overhead into two constants: MODEL_OVERHEAD_MB (pure
model weights for batch-size capping) and TRIAL_VRAM_MB (total
per-trial VRAM for concurrent hyperopt planning). DQN trials
empirically consume ~7 GB each on L40S (model + GPU replay buffer +
experience collector + CUDA allocations + fragmentation), not the
200 MB previously estimated. This caused plan_hyperopt to compute
128 concurrent trials instead of the actual 5, inflating PSO
particles from 20→128 and total trials from 20→384 via .max()
instead of .min(), guaranteeing a 4h timeout kill.
Fix auto-scaling to: (1) match particles to GPU concurrency for
maximum hardware utilization on any node, (2) cap particles at
max_trials to never inflate the trial budget, (3) never auto-inflate
the total trial count.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The auto-detect heuristic used cpus/2 ("smart cap"), designed for
CPU-bound workloads. DQN/PPO trials are GPU-bound — each rayon
thread submits CUDA kernels and waits on cudaDeviceSynchronize(),
using minimal CPU. cpus-1 is the correct cap.
On L40S-1-48G (8 vCPU): 3 threads → 7 threads (2.3× more trials).
Also bumps CI CPU limit 7500m→8000m to expose all 8 cores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The batch span processor needs a tokio runtime for gRPC transport and
periodic flush. Async services already have one via #[tokio::main], but
sync training binaries (hyperopt, train, evaluate) don't.
Previous approach (making binaries async with #[tokio::main]) caused
"Cannot start a runtime from within a runtime" panics because the ML
crate's internal code creates its own tokio runtimes for block_on().
New approach: build_otel_tracer() detects runtime context via
Handle::try_current(). If absent, it creates a dedicated 1-worker
multi-thread runtime stored in a process-lifetime OnceLock. The worker
thread actively polls the OTLP batch export task.
Reverts training binaries to sync fn main() so internal runtime creation
(hyperopt adapters, DQN/PPO trainers) continues working as before.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 training binaries (hyperopt_baseline_rl, hyperopt_baseline_supervised,
train_baseline_rl, train_baseline_supervised, evaluate_baseline,
evaluate_supervised) used sync fn main() but the OTLP batch exporter
requires a tokio runtime (tonic/hyper-util gRPC transport). This caused
an immediate panic on CI when OTEL_EXPORTER_OTLP_ENDPOINT was set.
Fix: #[tokio::main(flavor = "current_thread")] on all 6 binaries.
Also fix pre-existing clippy warnings (shadow, let_underscore_must_use,
doc_markdown, cognitive_complexity, integer_division, unsafe_code).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both DQN and PPO eval paths used old 3-action index matching (0=Buy,
1=Sell, 2=Hold). Now uses FactoredAction.target_exposure() for
exposure-weighted returns and order-type-specific transaction costs.
PPO path had .to_int() which doesn't exist on FactoredAction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN PER: defer td_errors to_vec1() after loss.to_scalar() — piggyback on
existing pipeline flush instead of forcing premature GPU→CPU stall
- PPO trajectories: capacity-hint Vec allocations, extend_flat_states methods,
states_flat field on TrajectoryBatch for zero-copy GPU upload
- TGGN validate(): batch N per-sample losses on GPU → single to_scalar() sync
(was N GPU→CPU syncs)
- Liquid backward(): batch grad-norm per-param sqr().sum_all() on GPU → single
to_scalar() sync (was N GPU→CPU syncs per optimizer step)
- Liquid validate(): same N→1 GPU sync reduction as TGGN
- DQN trainer: restore EpochPrefetcher/DoubleBufferedLoader API (wrongly deleted)
- train_baseline_rl: wire DoubleBuffer GPU pre-upload — after CPU prefetch
completes, immediately upload next fold to GPU via DqnGpuData::upload() so
next fold starts with data already resident on GPU
2478 tests pass, 0 clippy warnings, 0 compile errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Emit set_eval_metrics (directional_accuracy, sharpe, profit_factor, return)
per fold for supervised model evaluation. Start metrics server on :9094.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Emit set_eval_metrics (win_rate, sharpe, profit_factor, return) per fold
for DQN/PPO evaluation. Start metrics server on :9094. Feeds training
cockpit Grafana dashboard eval panels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace tracing_subscriber::fmt() with init_observability() which adds
JSON structured logging + optional OTLP export to Tempo. When
OTEL_EXPORTER_OTLP_ENDPOINT env var is unset, falls back to fmt-only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Emit set_epoch, set_epoch_loss, set_validation_loss, set_iteration_seconds
after each DQN/PPO fold completes. Feeds training cockpit Grafana dashboard.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously train_baseline_rl.rs only passed commission (tx_cost_bps) to
DQN/PPO trainers, ignoring bid-ask spread slippage. Now computes per-fold
average spread via spread_cost_bps() (same as evaluate_baseline) and passes
total cost (commission + spread) to both trainers.
Removes #[allow(dead_code)] — function is now used by all 4 example binaries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace raw DQN::new() + manual training loop in the walk-forward
training binary with DQNTrainer, which automatically activates:
- Mixed precision (BF16/F16 auto-detected from GPU)
- Dynamic batch sizing (AutoBatchSizer + HardwareBudget)
- Gradient accumulation
- Full Rainbow DQN (PER, dueling, C51, noisy nets, n-step)
- Regime-conditional Q-networks
- Portfolio tracking, Kelly sizing, entropy regularization
The walk-forward fold structure (data loading, feature extraction,
window generation, normalization) stays in the binary — only per-fold
training delegates to DQNTrainer::train_with_preloaded_data().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Quality audit found 2 dead code paths in the GPU optimization commit:
1. Data caching: preload_data() was defined on all 10 hyperopt adapters
but never called. Now wired in both hyperopt binaries (RL + supervised)
before the trial loop. Each model preloads training data once into
Arc<Vec<...>>, eliminating per-trial disk I/O.
2. PPO mixed precision: config.mixed_precision was stored but never used
in forward passes. Added forward_mixed() to PolicyNetwork and
ValueNetwork (same BF16/FP16 pattern as DQN's NetworkLayers). Stored
on network structs and auto-applied via forward(). Wired in
PPO::with_device() for MLP networks.
Also fixes missing mixed_precision field in 2 test files and
trading_service PPOConfig literal.
5 files changed, +152/-20. 2418 tests pass, workspace compiles clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire BF16/FP16 mixed precision end-to-end for DQN and PPO with auto-detection
from GPU name (Ampere+ → BF16, Volta/Turing → FP16). Add hidden_dim_base to
hyperopt and wire through training/eval binaries. Reduce GPU sync points: make
DQN NaN checks periodic (every 100 steps), replace PPO GAE GPU round-trip with
pure CPU implementation. Cache training data across hyperopt trials for all 10
models via Arc. Batch DQN experience storage (128x fewer lock acquisitions).
Correct VRAM constants and batch bounds for all 9 supervised model adapters.
28 files changed, +1207/-208 lines. 2418 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Prometheus training metrics from example-local baseline_common/
to common::metrics::{server,training_metrics} following the existing
grpc_metrics.rs pattern. Fix 29 let_underscore_must_use clippy errors
in push_metrics.rs, 3 shadow lint errors in training binaries, and
demote gradient clipping log from warn to debug.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add metrics server lifecycle (init, port 9094, active_workers) to both
hyperopt_baseline_rl and hyperopt_baseline_supervised. All 18 metrics
are registered and exposed; inner PSO trial loops can be instrumented
incrementally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create shared baseline_common/metrics.rs module that registers all 18
dashboard-expected metrics (11 gauges, 5 counters, 2 histograms) and
spawns a lightweight HTTP metrics server on port 9094.
Instrument train_baseline_supervised and train_baseline_rl with:
- Epoch progress, training/validation loss gauges
- Checkpoint save timing, size, and failure counters
- NaN/gradient explosion detection counters
- Data loading latency histograms
- Active workers lifecycle (1 on start, 0 on exit)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove artificial swarm_size cap from plan_hyperopt() that limited
concurrent trials to n_particles (default 20). Now concurrency is
purely VRAM-driven with a hardware cap of 128 threads.
optimize_parallel() auto-scales n_particles to match GPU budget:
- L4 24GB: ~65 concurrent DQN trials (was 20)
- H100 80GB: 128 concurrent DQN trials (was 20)
- CPU/small GPU: falls back to configured n_particles
max_trials scales proportionally to ensure 3+ PSO iterations
for convergence with larger swarms.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Auto-detect now considers both CPU count and GPU VRAM when choosing
concurrent trial count. Prevents OOM on smaller GPUs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous "auto-detect" logic forced CPU which was wrong — GPU is
faster for forward/backward even with parallel trials (DQN/PPO use
<350MB of 24GB VRAM across 7 threads).
Changes:
- Remove --device flag, always require CUDA GPU
- Add DQNTrainer::new_with_device() to share CUDA context across trials
- Propagate hyperopt device to internal DQN trainer (was ignoring it)
- PPO/DQN adapters error on missing GPU instead of silent CPU fallback
- Downgrade batch-size clamping from warn to debug
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN/PPO networks are tiny (3 layers × 128 neurons). Running parallel
hyperopt on GPU wastes cores because CUDA context serializes across
threads — 5 trials on L4 only used 2000m of 6000m requested CPU.
Changes:
- Add --device flag to hyperopt_baseline_rl (auto/cpu/cuda)
- Auto mode forces CPU for parallel runs (no CUDA contention)
- CPU mode uses all available cores (no 2-core reserve)
- Add with_device() builder to DQN/PPO hyperopt trainers
- Downgrade "portfolio value <= 0" and GPU utilization warnings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enable concurrent trial evaluation for DQN/PPO hyperparameter
optimization via clone-per-particle pattern — each PSO particle
clones the trainer and trains independently, replacing the previous
Arc<Mutex> serialization bottleneck. On L4 (8 vCPU) this yields
~4-5x throughput improvement.
Changes:
- DQNTrainer/PPOTrainer: Clone with Arc<AtomicUsize> trial counter
- DQNTrainer: replace unsafe mutable aliasing with Arc<Mutex> for
best_trial tracking
- ArgminOptimizer: add optimize_parallel() with ParallelObjectiveFunction
and scoped-thread LHS evaluation
- CLI: --parallel 0 (auto-detect CPUs-2), --initial-capital 35000,
--tx-cost-bps 0.1 (IBKR ES all-in)
- CI: both hyperopt jobs use --parallel 0 + IBKR ES cost defaults
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend hyperopt infrastructure to support all 10 ML models (DQN, PPO +
8 supervised). Previously only TFT and Mamba2 had hyperopt trainers.
- Add HyperparameterOptimizable impl for Liquid, TGGN, TLOB, KAN, xLSTM, Diffusion
- Create shared_data.rs with common data prep utilities (build_flat_pairs,
build_sequence_pairs, write_trial_result_json)
- Extend hyperopt_baseline_supervised binary to dispatch all 8 models
(individual, "both" for tft+mamba2, "all" for all 8)
- Add CI jobs: 7 train-validate + 10 hyperopt jobs for all models
- Fix DiffusionMetrics NaN default, XLSTMMetrics serde, safe indexing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
evaluate_supervised looked for checkpoints at {models_dir}/{model}_fold{N}_best
but training saves to {models_dir}/{model}/{model}_fold{N}_best (model subdirectory).
Also saves NormStats JSON per fold during training so evaluation uses
training-time normalization instead of computing from test data (data leakage).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New evaluate_supervised binary runs walk-forward inference on supervised
model checkpoints (TFT, Mamba2, etc.), converts directional predictions
to trading signals, and computes Sharpe/MaxDD/WinRate/DirAccuracy.
CI changes:
- train-validate-dqn → train-validate-rl (trains+evals DQN+PPO)
- train-validate-tft now runs evaluate_supervised after training
- web/api fallback rules added to train-validate and deploy stages
- evaluate_supervised added to compile-services and Docker images
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
