# Full-Stack ML Integration Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. **Goal:** Take the ML codebase from "4 models that train independently" to "paper-trading pipeline producing simulated P&L from a 4-model ensemble." **Architecture:** 4-phase approach: (1) delete 88 stale test files and commit unstaged adapter code, (2) wire all 4 model adapters into the EnsembleCoordinator, (3) build hyperopt campaign runner with automated validation, (4) add paper-trading pipeline with simulated fills and P&L tracking. **Tech Stack:** Rust, candle (ML framework), safetensors (checkpoints), tokio (async), Databento (market data) --- ## Phase 1: Codebase Cleanup ### Task 1: Delete stale test files using `WorkingDQN` 88 test files in `ml/tests/` reference the removed `WorkingDQN`/`WorkingDQNConfig` types and will not compile. They must be deleted. **Files:** - Delete: All 88 files listed below **Step 1: Delete all stale WorkingDQN test files** Run: ```bash cd /home/jgrusewski/Work/foxhunt grep -rl "WorkingDQN\|WorkingDQNConfig" ml/tests/ | xargs rm -f ``` **Step 2: Verify deletion** Run: `ls ml/tests/*.rs | wc -l` Expected: ~482 files remain (570 - 88 = ~482) **Step 3: Commit** ```bash git add -u ml/tests/ git commit -m "chore(ml): delete 88 stale test files referencing removed WorkingDQN type" ``` --- ### Task 2: Fix test files using wrong crate name `foxhunt_ml::` 4 test files use `foxhunt_ml::` instead of the correct `ml::` crate name. **Files:** - Fix: `ml/tests/dqn_full_gradient_flow_integration_test.rs` - Fix: `ml/tests/dqn_gradient_flow_isolation_test.rs` - Fix: `ml/tests/tft_int8_forward_integration_test.rs` - Fix: `ml/tests/tft_int8_integration_test.rs` **Step 1: Fix crate name in all 4 files** Run: ```bash cd /home/jgrusewski/Work/foxhunt sed -i 's/foxhunt_ml::/ml::/g' \ ml/tests/dqn_full_gradient_flow_integration_test.rs \ ml/tests/dqn_gradient_flow_isolation_test.rs \ ml/tests/tft_int8_forward_integration_test.rs \ ml/tests/tft_int8_integration_test.rs ``` **Step 2: Check if the fixed files compile** Run: `SQLX_OFFLINE=true cargo check -p ml --tests 2>&1 | grep -i "error\[" | head -20` If these files have other compilation errors beyond the crate name, delete them too: ```bash rm -f ml/tests/dqn_full_gradient_flow_integration_test.rs \ ml/tests/dqn_gradient_flow_isolation_test.rs \ ml/tests/tft_int8_forward_integration_test.rs \ ml/tests/tft_int8_integration_test.rs ``` **Step 3: Commit** ```bash git add ml/tests/ git commit -m "fix(ml): fix or remove test files with wrong foxhunt_ml:: crate name" ``` --- ### Task 3: Commit unstaged ensemble adapter files The Mamba2 and TFT inference adapters already exist but are unstaged. The `adapters/mod.rs` already exports them. **Files:** - Stage: `ml/src/ensemble/adapters/mod.rs` (modified — added mamba2/tft modules) - Stage: `ml/src/ensemble/adapters/mamba2.rs` (new — 320 lines, with tests) - Stage: `ml/src/ensemble/adapters/tft.rs` (new — 484 lines, with tests) **Step 1: Run the adapter unit tests** Run: ```bash SQLX_OFFLINE=true cargo test -p ml --lib ensemble::adapters::mamba2 -- --nocapture SQLX_OFFLINE=true cargo test -p ml --lib ensemble::adapters::tft -- --nocapture ``` Expected: All tests pass (3 tests each module) **Step 2: Commit** ```bash git add ml/src/ensemble/adapters/mod.rs ml/src/ensemble/adapters/mamba2.rs ml/src/ensemble/adapters/tft.rs git commit -m "feat(ensemble): add Mamba2 and TFT inference adapters with ring buffer + tests" ``` --- ### Task 4: Commit unstaged Mamba2 loss module The directional MSE loss function and bilinear discretization fix are unstaged. **Files:** - Stage: `ml/src/mamba/loss.rs` (new — 112 lines) - Stage: `ml/src/mamba/mod.rs` (modified — bilinear discretization fix) **Step 1: Verify Mamba2 tests still pass** Run: `SQLX_OFFLINE=true cargo test -p ml --lib mamba -- --nocapture 2>&1 | tail -10` Expected: Tests pass **Step 2: Commit** ```bash git add ml/src/mamba/loss.rs ml/src/mamba/mod.rs git commit -m "feat(mamba): add directional MSE loss and bilinear discretization fix" ``` --- ### Task 5: Verify full compilation **Step 1: Check lib compilation** Run: `SQLX_OFFLINE=true cargo check -p ml --lib 2>&1 | tail -5` Expected: No errors **Step 2: Check remaining integration tests compile** Run: `SQLX_OFFLINE=true cargo check -p ml --tests 2>&1 | grep "error\[" | wc -l` Expected: 0 errors (or identify remaining broken files and delete them) **Step 3: If errors remain, delete offending files and re-commit** ```bash # Identify remaining broken test files SQLX_OFFLINE=true cargo check -p ml --tests 2>&1 | grep "error\[" | grep -oP 'ml/tests/\S+\.rs' | sort -u | xargs rm -f git add -u ml/tests/ git commit -m "chore(ml): remove remaining broken integration test files" ``` **Step 4: Run lib tests** Run: `SQLX_OFFLINE=true cargo test -p ml --lib 2>&1 | tail -20` Expected: All pass, zero failures --- ## Phase 2: Ensemble Coordinator Integration ### Task 6: Create `InferenceEnsemble` — lightweight coordinator wrapping 4 adapters The existing `EnsembleCoordinator` in `coordinator.rs` is async and uses `Features`/`ModelPrediction` types from the old ensemble system. Rather than refactoring it, create a new lightweight synchronous coordinator that uses the `ModelInferenceAdapter` trait. **Files:** - Create: `ml/src/ensemble/inference_ensemble.rs` - Modify: `ml/src/ensemble/mod.rs` (add `pub mod inference_ensemble;`) **Step 1: Write the failing test** In `ml/src/ensemble/inference_ensemble.rs`: ```rust #[cfg(test)] mod tests { use super::*; use crate::ensemble::inference_adapter::{ EnsemblePrediction, FeatureVector, ModelInferenceAdapter, PredictionMeta, }; /// Dummy adapter for testing ensemble aggregation struct DummyAdapter { name: String, direction: f64, confidence: f64, ready: bool, } impl ModelInferenceAdapter for DummyAdapter { fn model_name(&self) -> &str { &self.name } fn is_ready(&self) -> bool { self.ready } fn predict(&self, _features: &FeatureVector) -> crate::MLResult { Ok(EnsemblePrediction { model_name: self.name.clone(), direction: self.direction, confidence: self.confidence, metadata: PredictionMeta { latency_us: 100, quantiles: None, attention_weights: None, q_values: None, }, }) } } #[test] fn test_ensemble_equal_weight_aggregation() { let adapters: Vec> = vec![ Box::new(DummyAdapter { name: "A".into(), direction: 1.0, confidence: 0.8, ready: true }), Box::new(DummyAdapter { name: "B".into(), direction: -1.0, confidence: 0.6, ready: true }), ]; let ensemble = InferenceEnsemble::new(adapters); let fv = FeatureVector { values: vec![0.0; 51], timestamp: 0 }; let result = ensemble.predict(&fv).unwrap(); // Weighted: (1.0*0.8 + (-1.0)*0.6) / (0.8+0.6) = 0.2/1.4 ≈ 0.143 assert!(result.direction > 0.0, "Net direction should be positive"); assert!(result.direction < 0.5, "Net direction should be modest"); } #[test] fn test_ensemble_skips_unready_models() { let adapters: Vec> = vec![ Box::new(DummyAdapter { name: "A".into(), direction: 1.0, confidence: 0.9, ready: true }), Box::new(DummyAdapter { name: "B".into(), direction: -1.0, confidence: 0.9, ready: false }), ]; let ensemble = InferenceEnsemble::new(adapters); let fv = FeatureVector { values: vec![0.0; 51], timestamp: 0 }; let result = ensemble.predict(&fv).unwrap(); // Only model A is ready, so direction should be strongly positive assert!(result.direction > 0.5); } #[test] fn test_ensemble_custom_weights() { let adapters: Vec> = vec![ Box::new(DummyAdapter { name: "A".into(), direction: 1.0, confidence: 1.0, ready: true }), Box::new(DummyAdapter { name: "B".into(), direction: -1.0, confidence: 1.0, ready: true }), ]; let mut ensemble = InferenceEnsemble::new(adapters); // Give model A 3x weight of model B ensemble.set_weight("A", 0.75); ensemble.set_weight("B", 0.25); let fv = FeatureVector { values: vec![0.0; 51], timestamp: 0 }; let result = ensemble.predict(&fv).unwrap(); // (1.0 * 0.75 * 1.0 + (-1.0) * 0.25 * 1.0) = 0.5 (before normalization) assert!(result.direction > 0.3, "A's weight should dominate"); } } ``` **Step 2: Run tests to verify they fail** Run: `SQLX_OFFLINE=true cargo test -p ml --lib ensemble::inference_ensemble -- --nocapture 2>&1 | tail -5` Expected: Compilation error — `InferenceEnsemble` not defined **Step 3: Write minimal implementation** ```rust //! Lightweight synchronous ensemble that aggregates ModelInferenceAdapter predictions. use std::collections::HashMap; use crate::ensemble::inference_adapter::{ EnsemblePrediction, FeatureVector, ModelInferenceAdapter, PredictionMeta, }; use crate::{MLError, MLResult}; /// Synchronous ensemble coordinator wrapping N model adapters. /// /// Collects predictions from all ready adapters and produces a /// confidence-weighted consensus direction. pub struct InferenceEnsemble { adapters: Vec>, weights: HashMap, } impl InferenceEnsemble { pub fn new(adapters: Vec>) -> Self { Self { adapters, weights: HashMap::new(), } } /// Set a custom weight for a model (by name). Default weight is 1.0. pub fn set_weight(&mut self, model_name: &str, weight: f64) { self.weights.insert(model_name.to_string(), weight); } /// Predict from all ready models and aggregate. pub fn predict(&self, features: &FeatureVector) -> MLResult { let start = std::time::Instant::now(); let mut predictions = Vec::new(); for adapter in &self.adapters { if !adapter.is_ready() { continue; } match adapter.predict(features) { Ok(pred) => predictions.push(pred), Err(e) => { tracing::warn!(model = adapter.model_name(), error = %e, "Adapter prediction failed, skipping"); } } } if predictions.is_empty() { return Err(MLError::InferenceError("No ready models in ensemble".into())); } // Confidence-weighted aggregation let mut weighted_direction = 0.0; let mut total_weight = 0.0; for pred in &predictions { let model_weight = self.weights.get(&pred.model_name).copied().unwrap_or(1.0); let w = model_weight * pred.confidence; weighted_direction += pred.direction * w; total_weight += w; } let direction = if total_weight > 0.0 { (weighted_direction / total_weight).clamp(-1.0, 1.0) } else { 0.0 }; let confidence = if !predictions.is_empty() { let avg_conf: f64 = predictions.iter().map(|p| p.confidence).sum::() / predictions.len() as f64; avg_conf.clamp(0.0, 1.0) } else { 0.0 }; let latency_us = start.elapsed().as_micros() as u64; let model_names: Vec = predictions.iter().map(|p| p.model_name.clone()).collect(); Ok(EnsemblePrediction { model_name: format!("ENSEMBLE({})", model_names.join("+")), direction, confidence, metadata: PredictionMeta { latency_us, quantiles: None, attention_weights: None, q_values: None, }, }) } /// Number of adapters currently ready. pub fn ready_count(&self) -> usize { self.adapters.iter().filter(|a| a.is_ready()).count() } } ``` **Step 4: Run tests to verify they pass** Run: `SQLX_OFFLINE=true cargo test -p ml --lib ensemble::inference_ensemble -- --nocapture` Expected: 3 tests pass **Step 5: Commit** ```bash git add ml/src/ensemble/inference_ensemble.rs ml/src/ensemble/mod.rs git commit -m "feat(ensemble): add InferenceEnsemble coordinator with confidence-weighted aggregation" ``` --- ### Task 7: Integration test — 4-model ensemble prediction Create an integration test that constructs all 4 real adapters (DQN, PPO, Mamba2, TFT) with random weights, feeds feature vectors, and verifies the ensemble produces bounded predictions. **Files:** - Create: `ml/tests/ensemble_inference_integration_test.rs` **Step 1: Write the test** ```rust //! Integration test: 4-model InferenceEnsemble //! //! Creates DQN, PPO, Mamba2, and TFT adapters with default/small configs, //! feeds feature vectors through the ensemble, and verifies output bounds. #![allow(unused_crate_dependencies)] use ml::dqn::{DQNConfig, DQN}; use ml::ensemble::adapters::{ DqnInferenceAdapter, Mamba2InferenceAdapter, PpoInferenceAdapter, TftInferenceAdapter, }; use ml::ensemble::inference_adapter::FeatureVector; use ml::ensemble::inference_ensemble::InferenceEnsemble; use ml::mamba::Mamba2Config; use ml::ppo::PPOConfig; use ml::tft::TFTConfig; #[test] fn test_4_model_ensemble_prediction() { // Create DQN adapter let dqn_config = DQNConfig::default(); let dqn = DQN::new(&dqn_config).expect("DQN creation"); let dqn_adapter = DqnInferenceAdapter::new(dqn); // Create PPO adapter let ppo_config = PPOConfig::default(); let ppo = ml::ppo::PPO::new(&ppo_config).expect("PPO creation"); let ppo_adapter = PpoInferenceAdapter::new(ppo); // Create Mamba2 adapter with small config let mamba2_config = Mamba2Config { d_model: 64, d_state: 16, d_head: 16, num_heads: 2, expand: 2, num_layers: 1, max_seq_len: 8, ..Default::default() }; let mamba2_adapter = Mamba2InferenceAdapter::new(mamba2_config, 4) .expect("Mamba2 adapter creation"); // Create TFT adapter with small config let tft_config = TFTConfig { input_dim: 20, hidden_dim: 32, num_heads: 2, num_layers: 1, prediction_horizon: 5, sequence_length: 4, num_quantiles: 9, num_static_features: 6, num_known_features: 6, num_unknown_features: 8, dropout_rate: 0.0, ..Default::default() }; let tft_adapter = TftInferenceAdapter::new(tft_config, 4) .expect("TFT adapter creation"); // Build ensemble let adapters: Vec> = vec![ Box::new(dqn_adapter), Box::new(ppo_adapter), Box::new(mamba2_adapter), Box::new(tft_adapter), ]; let ensemble = InferenceEnsemble::new(adapters); // Feed 5 feature vectors (enough to fill Mamba2 + TFT buffers at seq_len=4) for i in 0..5 { let fv = FeatureVector { values: vec![0.1 * (i as f64 + 1.0); 51], timestamp: 1_700_000_000 + i, }; let result = ensemble.predict(&fv); // DQN and PPO are always ready (single-step models) // Mamba2 and TFT need 4 vectors to fill their buffers assert!(result.is_ok(), "Ensemble prediction should not error: {:?}", result.err()); let pred = result.unwrap(); assert!(pred.direction >= -1.0 && pred.direction <= 1.0, "direction {} out of [-1,1]", pred.direction); assert!(pred.confidence >= 0.0 && pred.confidence <= 1.0, "confidence {} out of [0,1]", pred.confidence); } // After 5 FVs, all 4 models should be ready assert_eq!(ensemble.ready_count(), 4, "All 4 models should be ready after 5 FVs"); } ``` **Step 2: Run test** Run: `SQLX_OFFLINE=true cargo test -p ml --test ensemble_inference_integration_test -- --nocapture` Expected: 1 test passes Note: This test may need adjustments to the import paths depending on how DQN/PPO constructors are exposed. If DQN/PPO adapters need the model passed differently, adjust accordingly. The key pattern is: construct model → wrap in adapter → add to ensemble. **Step 3: Commit** ```bash git add ml/tests/ensemble_inference_integration_test.rs git commit -m "test(ensemble): add 4-model inference ensemble integration test" ``` --- ## Phase 3: Hyperopt Campaign Runner ### Task 8: Create hyperopt campaign CLI runner Build a CLI entrypoint that runs a hyperopt campaign for a given model type. **Files:** - Create: `ml/src/hyperopt/campaign.rs` - Modify: `ml/src/hyperopt/mod.rs` (add `pub mod campaign;`) **Step 1: Write the failing test** In `ml/src/hyperopt/campaign.rs`: ```rust #[cfg(test)] mod tests { use super::*; #[test] fn test_campaign_config_defaults() { let config = CampaignConfig::dqn_default(); assert_eq!(config.model_type, ModelType::DQN); assert_eq!(config.num_trials, 50); assert!(config.max_batch_size > 0); } #[test] fn test_campaign_config_ppo() { let config = CampaignConfig::ppo_default(); assert_eq!(config.model_type, ModelType::PPO); assert_eq!(config.num_trials, 30); } #[test] fn test_results_dir_creation() { let config = CampaignConfig::dqn_default(); let dir = config.results_dir(); assert!(dir.starts_with("ml/hyperopt_results/dqn/")); } } ``` **Step 2: Write implementation** ```rust //! Hyperopt campaign runner — configures and executes multi-trial optimization. use std::path::PathBuf; /// Model type for campaign targeting. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum ModelType { DQN, PPO, } /// Campaign configuration. #[derive(Debug, Clone)] pub struct CampaignConfig { pub model_type: ModelType, pub num_trials: usize, pub data_dir: PathBuf, pub max_batch_size: usize, pub early_stopping_eta: usize, pub max_epochs_per_trial: usize, pub results_base_dir: PathBuf, } impl CampaignConfig { pub fn dqn_default() -> Self { Self { model_type: ModelType::DQN, num_trials: 50, data_dir: PathBuf::from("test_data/real/databento/ml_training"), max_batch_size: 230, // RTX 3050 Ti 4GB VRAM limit early_stopping_eta: 3, max_epochs_per_trial: 81, results_base_dir: PathBuf::from("ml/hyperopt_results"), } } pub fn ppo_default() -> Self { Self { model_type: ModelType::PPO, num_trials: 30, data_dir: PathBuf::from("test_data/real/databento/ml_training"), max_batch_size: 230, early_stopping_eta: 3, max_epochs_per_trial: 81, results_base_dir: PathBuf::from("ml/hyperopt_results"), } } /// Generate a timestamped results directory path. pub fn results_dir(&self) -> String { let model_name = match self.model_type { ModelType::DQN => "dqn", ModelType::PPO => "ppo", }; let timestamp = chrono::Utc::now().format("%Y%m%d_%H%M%S"); format!("{}/{}/{}", self.results_base_dir.display(), model_name, timestamp) } } ``` **Step 3: Run tests** Run: `SQLX_OFFLINE=true cargo test -p ml --lib hyperopt::campaign -- --nocapture` Expected: 3 tests pass **Step 4: Commit** ```bash git add ml/src/hyperopt/campaign.rs ml/src/hyperopt/mod.rs git commit -m "feat(hyperopt): add campaign configuration with DQN/PPO defaults" ``` --- ### Task 9: Campaign execution and results persistence Implement the actual trial loop that drives the hyperopt campaign. **Files:** - Modify: `ml/src/hyperopt/campaign.rs` (add `run_campaign` async fn) **Step 1: Write the test (smoke test with 2 trials)** ```rust #[tokio::test] async fn test_campaign_smoke_2_trials() { let data_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")) .parent() .unwrap() .join("test_data/real/databento/ml_training_small"); if !data_dir.exists() { eprintln!("Skipping: test data not found at {}", data_dir.display()); return; } let mut config = CampaignConfig::dqn_default(); config.num_trials = 2; config.max_epochs_per_trial = 5; config.data_dir = data_dir; let results = run_campaign(config).await.expect("Campaign should complete"); assert_eq!(results.trials_completed, 2); assert!(results.best_loss.is_finite()); } ``` **Step 2: Implement `run_campaign`** This function should: 1. Create the results directory 2. Loop for `num_trials` iterations 3. Sample random DQNParams using `ParameterSpace` trait 4. Train for `max_epochs_per_trial` epochs 5. Use `EarlyStoppingObserver` for pruning 6. Track best params and loss 7. Save best checkpoint and params JSON to results dir **Step 3: Run test** Run: `SQLX_OFFLINE=true cargo test -p ml campaign_smoke -- --nocapture` Expected: Passes (may take 30-60s for 2 trials x 5 epochs) **Step 4: Commit** ```bash git add ml/src/hyperopt/campaign.rs git commit -m "feat(hyperopt): implement campaign runner with trial loop and results persistence" ``` --- ### Task 10: Post-campaign automated validation After hyperopt finds best params, automatically run them through the ValidationHarness. **Files:** - Modify: `ml/src/hyperopt/campaign.rs` (add `validate_best` fn) **Step 1: Write the test** ```rust #[test] fn test_campaign_results_serialization() { let results = CampaignResults { trials_completed: 10, best_loss: 0.42, best_params_json: "{\"lr\": 0.001}".to_string(), best_checkpoint_path: Some("path/to/best.safetensors".into()), total_time_seconds: 120.0, }; let json = serde_json::to_string(&results).expect("serialize"); let back: CampaignResults = serde_json::from_str(&json).expect("deserialize"); assert_eq!(back.trials_completed, 10); } ``` **Step 2: Implement `CampaignResults` struct with serde** **Step 3: Commit** ```bash git add ml/src/hyperopt/campaign.rs git commit -m "feat(hyperopt): add CampaignResults serialization and validation hook" ``` --- ## Phase 4: Paper Trading Pipeline ### Task 11: Define `TradeSignal` type in ml crate The ML-side signal that flows from ensemble to trading. **Files:** - Create: `ml/src/ensemble/signal.rs` - Modify: `ml/src/ensemble/mod.rs` **Step 1: Write test** ```rust #[test] fn test_trade_signal_from_ensemble_prediction() { let pred = EnsemblePrediction { model_name: "ENSEMBLE(DQN+PPO)".into(), direction: 0.6, confidence: 0.85, metadata: PredictionMeta::default(), }; let signal = TradeSignal::from_prediction(&pred, "6E.FUT"); assert_eq!(signal.action, TradeAction::Buy); assert_eq!(signal.symbol, "6E.FUT"); assert!(signal.confidence > 0.8); } ``` **Step 2: Implement** ```rust /// Trade action derived from ensemble prediction. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum TradeAction { Buy, Sell, Hold, } /// Signal emitted by the ML ensemble for the trading engine. #[derive(Debug, Clone)] pub struct TradeSignal { pub action: TradeAction, pub symbol: String, pub confidence: f64, pub direction: f64, pub timestamp_us: i64, } impl TradeSignal { pub fn from_prediction(pred: &EnsemblePrediction, symbol: &str) -> Self { let action = if pred.direction > 0.1 { TradeAction::Buy } else if pred.direction < -0.1 { TradeAction::Sell } else { TradeAction::Hold }; Self { action, symbol: symbol.to_string(), confidence: pred.confidence, direction: pred.direction, timestamp_us: chrono::Utc::now().timestamp_micros(), } } } ``` **Step 3: Run tests, commit** ```bash git add ml/src/ensemble/signal.rs ml/src/ensemble/mod.rs git commit -m "feat(ensemble): add TradeSignal type with Buy/Sell/Hold action mapping" ``` --- ### Task 12: PaperBroker — simulated fill engine Lives in the ml crate for now (no cross-crate dependency needed for paper trading). **Files:** - Create: `ml/src/paper_trading/mod.rs` - Create: `ml/src/paper_trading/broker.rs` - Modify: `ml/src/lib.rs` (add `pub mod paper_trading;`) **Step 1: Write test** ```rust #[test] fn test_paper_broker_buy_and_sell() { let mut broker = PaperBroker::new(100_000.0, 0.0001); // $100k, 1bps slippage let fill = broker.execute_signal(&TradeSignal { action: TradeAction::Buy, symbol: "6E.FUT".into(), confidence: 0.9, direction: 0.5, timestamp_us: 0, }, 1.0850); // current price assert!(fill.is_some()); let fill = fill.unwrap(); assert!(fill.fill_price > 1.0850); // slippage applied assert_eq!(broker.position_size(), 1); // 1 contract long } ``` **Step 2: Implement PaperBroker** - Track cash, positions, commission - `execute_signal` takes `TradeSignal` + current market price → `Option` - Apply slippage: `fill_price = price * (1 + slippage * direction_sign)` - Commission per trade (configurable, default $2.50 per contract) - Track P&L: `unrealized_pnl(current_price)` and `realized_pnl()` **Step 3: Run tests, commit** ```bash git add ml/src/paper_trading/ ml/src/lib.rs git commit -m "feat(paper_trading): add PaperBroker with simulated fills and position tracking" ``` --- ### Task 13: PnL Tracker — cumulative returns and Sharpe **Files:** - Create: `ml/src/paper_trading/pnl_tracker.rs` - Modify: `ml/src/paper_trading/mod.rs` **Step 1: Write test** ```rust #[test] fn test_pnl_tracker_sharpe_calculation() { let mut tracker = PnLTracker::new(252); // 252-bar rolling window // Simulate 10 bars of small positive returns for i in 0..10 { tracker.record_return(0.001 + 0.0001 * (i as f64)); } let sharpe = tracker.rolling_sharpe(); assert!(sharpe.is_finite()); assert!(sharpe > 0.0, "Positive returns should yield positive Sharpe"); let drawdown = tracker.max_drawdown(); assert!(drawdown >= 0.0); assert!(drawdown <= 1.0); } ``` **Step 2: Implement** - Ring buffer of returns (configurable window, default 252) - `rolling_sharpe()`: `mean / std * sqrt(252)` - `max_drawdown()`: track peak equity, compute max drop - `cumulative_return()`: product of `(1 + r)` - 1 **Step 3: Run tests, commit** ```bash git add ml/src/paper_trading/pnl_tracker.rs ml/src/paper_trading/mod.rs git commit -m "feat(paper_trading): add PnLTracker with rolling Sharpe and max drawdown" ``` --- ### Task 14: Paper trading pipeline integration test End-to-end test: replay historical OHLCV data → feature extraction → ensemble → signals → paper broker → P&L. **Files:** - Create: `ml/tests/paper_trading_integration_test.rs` **Step 1: Write test** ```rust //! Paper trading pipeline integration test. //! //! Replays synthetic OHLCV data through the full pipeline: //! Feature extraction → Ensemble → TradeSignal → PaperBroker → PnL #![allow(unused_crate_dependencies)] use ml::ensemble::adapters::{DqnInferenceAdapter, PpoInferenceAdapter}; use ml::ensemble::inference_adapter::FeatureVector; use ml::ensemble::inference_ensemble::InferenceEnsemble; use ml::ensemble::signal::TradeSignal; use ml::paper_trading::broker::PaperBroker; use ml::paper_trading::pnl_tracker::PnLTracker; #[test] fn test_paper_trading_pipeline_synthetic_data() { // Build ensemble with 2 models (DQN + PPO for speed) // ... construct adapters ... let ensemble = InferenceEnsemble::new(adapters); let mut broker = PaperBroker::new(100_000.0, 0.0001); let mut tracker = PnLTracker::new(252); // Simulate 100 bars of synthetic price data let mut price = 1.0850; for i in 0..100 { // Synthetic feature vector let fv = FeatureVector { values: vec![price; 51], timestamp: 1_700_000_000 + i, }; if let Ok(pred) = ensemble.predict(&fv) { let signal = TradeSignal::from_prediction(&pred, "6E.FUT"); broker.execute_signal(&signal, price); } // Random walk price price += 0.0001 * ((i % 7) as f64 - 3.0); // Record bar return let pnl = broker.unrealized_pnl(price); tracker.record_equity(100_000.0 + pnl); } // Assertions let sharpe = tracker.rolling_sharpe(); assert!(sharpe.is_finite(), "Sharpe should be finite"); let drawdown = tracker.max_drawdown(); assert!(drawdown.is_finite(), "Drawdown should be finite"); assert!(drawdown <= 1.0, "Drawdown should be <= 100%"); println!("Pipeline test complete:"); println!(" Cumulative return: {:.4}%", tracker.cumulative_return() * 100.0); println!(" Rolling Sharpe: {:.4}", sharpe); println!(" Max drawdown: {:.4}%", drawdown * 100.0); println!(" Final equity: ${:.2}", 100_000.0 + broker.unrealized_pnl(price)); } ``` **Step 2: Run test** Run: `SQLX_OFFLINE=true cargo test -p ml --test paper_trading_integration_test -- --nocapture` Expected: Passes, prints P&L summary **Step 3: Commit** ```bash git add ml/tests/paper_trading_integration_test.rs git commit -m "test(paper_trading): add end-to-end pipeline integration test" ``` --- ## Phase 5: Final Verification ### Task 15: Full compilation and test suite verification **Step 1: Verify lib compilation** Run: `SQLX_OFFLINE=true cargo check -p ml --lib` Expected: No errors **Step 2: Verify integration test compilation** Run: `SQLX_OFFLINE=true cargo check -p ml --tests` Expected: No errors **Step 3: Run all lib tests** Run: `SQLX_OFFLINE=true cargo test -p ml --lib 2>&1 | tail -30` Expected: All pass **Step 4: Run key integration tests** Run: ```bash SQLX_OFFLINE=true cargo test -p ml --test ensemble_inference_integration_test -- --nocapture SQLX_OFFLINE=true cargo test -p ml --test paper_trading_integration_test -- --nocapture SQLX_OFFLINE=true cargo test -p ml --test production_training_smoke_test -- --nocapture ``` Expected: All pass **Step 5: Clippy check** Run: `SQLX_OFFLINE=true cargo clippy -p ml -- -D warnings 2>&1 | tail -20` Expected: No errors (warnings may exist but no denials) --- ## Constraints Checklist - [ ] GPU: RTX 3050 Ti, 4GB VRAM — batch sizes clamped at 230 - [ ] `SQLX_OFFLINE=true` for all cargo commands - [ ] Clippy denials: `unwrap_used`, `expect_used`, `panic`, `indexing_slicing` — safe patterns only - [ ] No live broker integration (paper-trading only) - [ ] Single asset (6E.FUT only) ## Summary | Phase | Tasks | Key Deliverable | |-------|-------|----------------| | 1: Cleanup | 1-5 | Zero compilation errors across all targets | | 2: Ensemble | 6-7 | `InferenceEnsemble::predict()` with 4 models | | 3: Hyperopt | 8-10 | Campaign runner producing best params + checkpoint | | 4: Paper Trading | 11-14 | Full pipeline: data → ensemble → signals → P&L | | 5: Verification | 15 | All tests green, clippy clean |