Detailed TDD implementation plan for wiring all 10 ML models into
the ensemble coordinator. Covers 5 new inference adapters (KAN, TGGN,
xLSTM, TLOB, Diffusion), main.rs registration with equal weights,
market data wiring, E2E integration test, Docker/CI validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire existing components into end-to-end pipeline: 5 missing inference
adapters, 10-model registration, market data feeding, integration test,
Docker build validation, and CI pipeline triage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove duplicate AssetClass enum from data_pipeline module and replace
with a re-export from asset_selection, which is now the canonical
location. This ensures data_pipeline::AssetClass and
asset_selection::AssetClass are the same type, enabling direct
comparison and preventing subtle type mismatch bugs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
10-task plan: unify AssetClass, add trading_symbol field, futures_baseline()
preset, TOML config loading, DatasetSpec::from_universe() wiring, data
reorganization into cache structure, stale data cleanup, manifest generation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Define futures baseline universe (ES, NQ, ZN, 6E) with micro
contract mapping for limited capital. Reorganize scattered test
data into cache structure. Download spec for 730 days of OHLCV-1m.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use tokio::task::block_in_place for safe sync→async bridging
- Replace count(*) with count(1) in division context (QuestDB parser bug)
- Add coalesce() for stddev NULL handling (single-row edge case)
- Consolidate integration tests into single lifecycle test with DROP+CREATE
- All 4 QuestDB tests pass against real QuestDB 8.2.3
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace stub metrics provider with real QuestDBMetricsProvider that
queries QuestDB via PostgreSQL wire protocol (port 8812) for rolling
model Sharpe, win rates, confidence buckets, and ensemble metrics.
- Add QuestDB service to docker-compose.yml (8.2.3, ports 9009/8812/9003)
- Create questdb_metrics.rs with 5 SQL queries and graceful fallback
- Wire QuestDBMetricsProvider into main.rs (replaces StubMetricsProvider)
- Fix unused Instant import in feedback_loop.rs (#[cfg(test)] scope)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FeedbackLoop runs as a background tokio task, periodically running
weight and gate optimization cycles with kill switch monitoring and
retraining trigger detection. Wired into main.rs with a stub
MetricsProvider until QuestDB is deployed.
7 new tests covering kill switch, freeze/unfreeze, retraining triggers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gate optimizer adjusts conviction gate thresholds based on win-rate per
confidence bucket with cooldown and kill switch safety rails. Model
registry provides lifecycle management (Candidate → Staging → Production
→ Archived) with InMemoryModelRegistry for testing. P&L attribution
decomposes realized trade P&L into per-model contributions using signal
alignment.
24 new tests across 3 modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces the asset_selection module with AssetClass enum (Equity/ETF/Future),
UniverseAsset struct, and AssetUniverse with filtering, lookup, and a us_starter()
preset of 7 highly liquid US assets. Includes scorer/selector placeholder stubs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adjusts model weights based on rolling Sharpe ratios with:
- EMA smoothing (alpha=0.1)
- Bounds: [0.05, 0.40] per model
- Max step: 0.03 per cycle
- 24h cooldown, 7-day grace period for new models
- Kill switch freeze/unfreeze
8 tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Non-critical path: if QuestDB is unavailable, metrics buffer locally
(up to 10,000 entries) and flush when connection is restored.
Feature-gated under `questdb` feature. 6 tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds optional ConvictionGateEvaluator field to EnsembleCoordinator.
When configured, predict() evaluates all 7 gates before returning.
If any gate rejects, returns HOLD with rejection metadata.
Includes quorum ratio, trading session, and regime volatility helpers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8-task plan across 3 phases: data pipeline (types, cache, manager,
prepared dataset), asset selection (universe, scorer, selector),
and integration wiring. 7 new files, ~51 tests estimated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers automated data downloading/caching for ML training and
a tiered asset selection funnel (universe → predictability → regime → signal).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three pillars: 7-gate conviction system, autonomous feedback loop
with kill switch, Rust-native model registry. QuestDB analytics layer.
~55 tests, 7 new files planned across ml/ and trading_service/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7-gate conviction system, autonomous feedback loop with kill switch,
and Rust-native model registry — backed by PostgreSQL + QuestDB.
Resolve merge conflicts in hyperopt/adapters/mod.rs and enhanced_ml.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Scale model_memory_mb by activation_multiplier in resolve_batch_size()
so TFT (2.5x) gets proportionally smaller batches than DQN (1.0x)
- Add cached_capabilities() with OnceLock to avoid spawning nvidia-smi
on every call to CampaignConfig::dqn_default/ppo_default/PpoTrainer::new
- Add PartialEq to GpuCapabilities and simplify serde roundtrip test
- Update stale "RTX 3050 Ti" and "Exceeds GPU limit (230)" comments
to reflect dynamic detection
- Document Auto variant silent CPU fallback behavior (no callers affected)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add DeviceConfig and GpuCapabilities re-exports to ml::prelude so
downstream crates can import GPU management types via a single
`use ml::prelude::*` statement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded max_batch_size: 230 in CampaignConfig::dqn_default()
and ppo_default() with dynamic GPU detection via GpuCapabilities::detect()
and resolve_batch_size(). Update test to no longer assert <= 230.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Device::cuda_if_available(0).unwrap_or(Device::Cpu) with
DeviceConfig::Auto.resolve().unwrap_or(Device::Cpu) in DQN, PPO, TFT,
and Mamba2 ensemble inference adapters so device selection goes through
the centralized GPU detection module.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add check_gradients_finite() to gradient_utils.rs that detects NaN/Inf
in GradStore before optimizer step. Uses efficient sum_all approach
where any NaN element produces a NaN sum. Includes 3 unit tests
(finite pass, NaN detected, empty vars pass).
DQN and PPO training loops already wired via gradient_accumulation
module's check_gradients_finite (all 6 paths: standard policy/value,
remainder policy/value, LSTM policy/value).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add check_gradients_finite() utility that scans GradStore for NaN/Inf
values. Wire into DQN (after gradient accumulation, before optimizer step)
and PPO (both MLP and LSTM training paths, after backward pass).
Prevents silent model corruption from exploding gradients or numerical
instability — training halts immediately with a descriptive error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace local DeviceConfig enum definition in liquid/candle_cfc.rs with
a re-export from the central crate::gpu::DeviceConfig, eliminating
duplication while preserving all existing API and tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add train_launcher.sh that detects local GPU VRAM and routes training
to local or Scaleway cloud. Auto-selects batch size per model based on
available VRAM tier. Maps model names to actual ml/examples/train_*.rs
cargo targets. Document Scaleway GPU instance types, setup procedure,
batch size tables, and cost estimates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8 bite-sized tasks with TDD approach, exact file paths, and complete code.
Covers: DeviceConfig relocation, GpuCapabilities detection,
ModelMemoryEstimate per-model profiles, and wiring into PPO trainer
and hyperopt campaign.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Centralizes GPU device selection, capability detection, and per-model
batch size optimization to replace hardcoded RTX 3050 Ti limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 16 unit tests for Enhanced ML service components:
- EnsembleConfig::default() values (min_models, thresholds, voting)
- FeaturePreprocessor::classify_feature_type() for price, volume,
technical, sentiment, and unknown features
- FeaturePreprocessor normalization (z-score, tanh fallback, zero std_dev)
- FeaturePreprocessor default stats validation
- RuntimeModelInfo creation for all 5 model types (DQN/PPO/TFT/Mamba/LNN)
- ModelPerformanceMetrics::default() zero initialization
- FeatureNormStats volatility default bounds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 25 unit tests for the RiskServiceImpl pure functions:
- Parametric VaR fallback formula (notional * 0.02)
- Equal contribution percentage for N symbols (including empty)
- Drawdown computation (empty, positive PnL, negative, mixed)
- Returns from executions (empty, single, sorted, zero-price filtering)
- Volatility (empty, single, constant, known series)
- Sharpe ratio (insufficient data, zero vol, positive returns)
- Sortino ratio (insufficient data, no downside, mixed)
- VaR square-root-of-time scaling (1d→5d→30d)
- Concentration risk level thresholds
- Risk constants validation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add optional correlation matrix parameter to mean-variance optimization.
When provided, builds full covariance matrix (Sigma[i][j] = corr[i][j] *
vol_i * vol_j) instead of diagonal-only. Existing API unchanged — callers
pass None by default. New allocate_with_correlations() public method for
correlated optimization. Five new tests: identity-matches-diagonal,
correlated-differs-from-diagonal, invalid dimensions, non-square matrix,
and non-MeanVariance delegation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add defensive checks to FeaturePreprocessor::normalize():
- Return 0.0 with warning log for NaN/Inf input values
- Clamp z-score output to [-10, 10] to prevent extreme values
- Add 4 unit tests covering NaN, Inf, -Inf, and extreme value clamping
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
