8-task plan across 3 phases: data pipeline (types, cache, manager,
prepared dataset), asset selection (universe, scorer, selector),
and integration wiring. 7 new files, ~51 tests estimated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers automated data downloading/caching for ML training and
a tiered asset selection funnel (universe → predictability → regime → signal).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three pillars: 7-gate conviction system, autonomous feedback loop
with kill switch, Rust-native model registry. QuestDB analytics layer.
~55 tests, 7 new files planned across ml/ and trading_service/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7-gate conviction system, autonomous feedback loop with kill switch,
and Rust-native model registry — backed by PostgreSQL + QuestDB.
Resolve merge conflicts in hyperopt/adapters/mod.rs and enhanced_ml.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Scale model_memory_mb by activation_multiplier in resolve_batch_size()
so TFT (2.5x) gets proportionally smaller batches than DQN (1.0x)
- Add cached_capabilities() with OnceLock to avoid spawning nvidia-smi
on every call to CampaignConfig::dqn_default/ppo_default/PpoTrainer::new
- Add PartialEq to GpuCapabilities and simplify serde roundtrip test
- Update stale "RTX 3050 Ti" and "Exceeds GPU limit (230)" comments
to reflect dynamic detection
- Document Auto variant silent CPU fallback behavior (no callers affected)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add DeviceConfig and GpuCapabilities re-exports to ml::prelude so
downstream crates can import GPU management types via a single
`use ml::prelude::*` statement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded max_batch_size: 230 in CampaignConfig::dqn_default()
and ppo_default() with dynamic GPU detection via GpuCapabilities::detect()
and resolve_batch_size(). Update test to no longer assert <= 230.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Device::cuda_if_available(0).unwrap_or(Device::Cpu) with
DeviceConfig::Auto.resolve().unwrap_or(Device::Cpu) in DQN, PPO, TFT,
and Mamba2 ensemble inference adapters so device selection goes through
the centralized GPU detection module.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add check_gradients_finite() to gradient_utils.rs that detects NaN/Inf
in GradStore before optimizer step. Uses efficient sum_all approach
where any NaN element produces a NaN sum. Includes 3 unit tests
(finite pass, NaN detected, empty vars pass).
DQN and PPO training loops already wired via gradient_accumulation
module's check_gradients_finite (all 6 paths: standard policy/value,
remainder policy/value, LSTM policy/value).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add check_gradients_finite() utility that scans GradStore for NaN/Inf
values. Wire into DQN (after gradient accumulation, before optimizer step)
and PPO (both MLP and LSTM training paths, after backward pass).
Prevents silent model corruption from exploding gradients or numerical
instability — training halts immediately with a descriptive error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace local DeviceConfig enum definition in liquid/candle_cfc.rs with
a re-export from the central crate::gpu::DeviceConfig, eliminating
duplication while preserving all existing API and tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add train_launcher.sh that detects local GPU VRAM and routes training
to local or Scaleway cloud. Auto-selects batch size per model based on
available VRAM tier. Maps model names to actual ml/examples/train_*.rs
cargo targets. Document Scaleway GPU instance types, setup procedure,
batch size tables, and cost estimates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8 bite-sized tasks with TDD approach, exact file paths, and complete code.
Covers: DeviceConfig relocation, GpuCapabilities detection,
ModelMemoryEstimate per-model profiles, and wiring into PPO trainer
and hyperopt campaign.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Centralizes GPU device selection, capability detection, and per-model
batch size optimization to replace hardcoded RTX 3050 Ti limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 16 unit tests for Enhanced ML service components:
- EnsembleConfig::default() values (min_models, thresholds, voting)
- FeaturePreprocessor::classify_feature_type() for price, volume,
technical, sentiment, and unknown features
- FeaturePreprocessor normalization (z-score, tanh fallback, zero std_dev)
- FeaturePreprocessor default stats validation
- RuntimeModelInfo creation for all 5 model types (DQN/PPO/TFT/Mamba/LNN)
- ModelPerformanceMetrics::default() zero initialization
- FeatureNormStats volatility default bounds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 25 unit tests for the RiskServiceImpl pure functions:
- Parametric VaR fallback formula (notional * 0.02)
- Equal contribution percentage for N symbols (including empty)
- Drawdown computation (empty, positive PnL, negative, mixed)
- Returns from executions (empty, single, sorted, zero-price filtering)
- Volatility (empty, single, constant, known series)
- Sharpe ratio (insufficient data, zero vol, positive returns)
- Sortino ratio (insufficient data, no downside, mixed)
- VaR square-root-of-time scaling (1d→5d→30d)
- Concentration risk level thresholds
- Risk constants validation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add optional correlation matrix parameter to mean-variance optimization.
When provided, builds full covariance matrix (Sigma[i][j] = corr[i][j] *
vol_i * vol_j) instead of diagonal-only. Existing API unchanged — callers
pass None by default. New allocate_with_correlations() public method for
correlated optimization. Five new tests: identity-matches-diagonal,
correlated-differs-from-diagonal, invalid dimensions, non-square matrix,
and non-MeanVariance delegation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add defensive checks to FeaturePreprocessor::normalize():
- Return 0.0 with warning log for NaN/Inf input values
- Clamp z-score output to [-10, 10] to prevent extreme values
- Add 4 unit tests covering NaN, Inf, -Inf, and extreme value clamping
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the TODO for spawning a background Redis health-check task
with an architectural decision comment. Local AtomicBool provides
immediate process-level protection; Redis monitoring is deferred to
multi-service deployment with a clear implementation roadmap.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move position fetching before VaR calculation in both get_va_r and
get_risk_metrics so the portfolio notional is computed from real
position data (sum of |quantity * avg_price|) instead of the fake
confidence_level * 1_000_000.0 placeholder. Falls back to 100_000.0
when the portfolio is empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces hardcoded zeros in AssetAllocation with real position data
queried from agent_orders. Adds fetch_current_positions() helper that
derives net quantity per symbol (buy - sell) and reuses it in both
allocate_portfolio and rebalance_portfolio to eliminate SQL duplication.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- NoiseScheduler: precomputed cosine/linear alpha_bar schedules
- Denoiser: FC network with sinusoidal time embedding + SiLU + residual
- DDIMSampler: deterministic fast sampling (10 steps from 1000 timesteps)
- DiffusionTrainableAdapter: UnifiedTrainable for unified training pipeline
- Hyperopt adapter with ParameterSpace (9 params, batch ≤64 for 4GB GPU)
- ModelType::Diffusion registered in common + coordinator
- 41 tests passing (config=3, noise=7, denoiser=4, sampler=5, trainable=12, hyperopt=7)
- OOM-safe: FC denoiser instead of U-Net, small hidden dims, conservative defaults
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 2-3 of ensemble expansion:
- KAN (Kolmogorov-Arnold Network): B-spline basis, layer, network, trainable adapter
- TLOB UnifiedTrainable adapter with 3D input support (batch, seq, features)
- Hyperopt adapters for both KAN and TLOB (ParameterSpace + metrics)
- ModelType::KAN variant registered in common, coordinator, lib.rs
- 44 new tests, all passing, zero warnings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Closes the integration gap between Liquid CfC and DQN/PPO by adding:
- LiquidTrainer with gRPC progress callbacks, early stopping, and
checkpoint management (ml/src/trainers/liquid.rs)
- LiquidModel wrapper in enhanced_ml.rs for hot-loading from safetensors
- Ensemble weight rebalance: DQN 0.25, PPO 0.25, TFT 0.20, Mamba2 0.15,
Liquid-CfC 0.15
Verified: 81 liquid unit tests + 3 integration tests + 211 trading_service
tests pass, 0 compile errors across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
