docs: real-data ensemble backtest and hyperopt deploy design

Two-phase design:
A) Real 6E.FUT data → 4-model ensemble → PaperBroker → P&L metrics
B) Best hyperopt params → train → deploy to ensemble → ValidationHarness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs/plans/2026-02-21-real-data-backtest-and-deploy-design.md

@@ -0,0 +1,101 @@
+# Real-Data Ensemble Backtest & Hyperopt Deploy Design
+
+**Date**: 2026-02-21
+**Goal**: Prove the 4-model ensemble works end-to-end on real 6E.FUT data, then close the hyperopt-to-production gap.
+
+## Problem
+
+All pipeline components exist (4 models, ensemble, paper broker, PnL tracker, validation harness, hyperopt) but they've never been connected end-to-end on real market data. Ensemble tests use mocks for Mamba2/TFT. Paper trading tests use synthetic data. Hyperopt saves best params but doesn't deploy them.
+
+## Phase A: Real-Data Ensemble Backtest
+
+### Data Flow
+
+```
+Real 6E.FUT bars (DBN files from test_data/real/databento/ml_training_small)
+        │
+        ▼
+Feature extraction (51-dim FeatureVector per bar)
+        │
+        ▼
+InferenceEnsemble (DQN + PPO + Mamba2 + TFT, random weights)
+        │ warm-up: first 50 bars fill sequence buffers, no trading
+        ▼
+TradeSignal (Buy/Sell/Hold, threshold ±0.1)
+        │
+        ▼
+PaperBroker ($100k, 1bps slippage, $2.50 commission)
+        │
+        ▼
+PnLTracker → Sharpe, max drawdown, cumulative return
+```
+
+### Key Decisions
+
+- **Dataset**: `test_data/real/databento/ml_training_small` (~361 files of 6E.FUT)
+- **Model weights**: Random (proving pipeline correctness, not alpha)
+- **Warm-up**: 50 bars to fill Mamba2/TFT sequence buffers before trading starts
+- **Feature extraction**: Reuse existing real_data_loader to build FeatureVectors from OHLCV bars
+- **Success criteria**: Pipeline completes without panics, all metrics are finite and bounded. Positive Sharpe NOT required (random weights).
+
+### What This Proves
+
+- All 4 real model adapters work together in ensemble
+- Feature extraction → prediction → signal → fill → P&L path is connected
+- Sequence models (Mamba2, TFT) handle real data dimensions correctly
+- PaperBroker produces realistic fill simulation
+- PnLTracker computes valid Sharpe/drawdown from real price movements
+
+## Phase B: Hyperopt → Deploy → Validate
+
+### Data Flow
+
+```
+Best params JSON (from ml/hyperopt_results/)
+        │
+        ▼
+DQNParams deserialization → DQNHyperparameters
+        │
+        ▼
+DQNTrainer::train(real_data) → checkpoint.safetensors
+        │
+        ▼
+DqnInferenceAdapter::from_checkpoint(path)  ← NEW CAPABILITY
+        │
+        ▼
+InferenceEnsemble (trained DQN + random PPO/Mamba2/TFT)
+        │
+        ▼
+ValidationHarness (walk-forward, 5 folds)
+        │
+        ▼
+ValidationVerdict: Pass / Marginal / Fail
+    DSR p-value, PBO ratio, permutation p-value
+```
+
+### Key Decisions
+
+- **Start with DQN only** — it has 16+ hyperopt runs with best Sharpe up to 3.16
+- **New capability needed**: `DqnInferenceAdapter::from_checkpoint(path)` to load trained weights
+- **Validation**: Walk-forward with 5 folds, DSR p<0.05, PBO<0.25
+- **PPO/Mamba2/TFT deployment**: Deferred until DQN proves the pattern works
+
+### What This Proves
+
+- Hyperopt results can be deployed to the inference pipeline
+- Trained models produce statistically validated alpha (or not — that's valuable info too)
+- The full loop: optimize → train → deploy → validate is closed
+
+## Constraints
+
+- GPU: RTX 3050 Ti, 4GB VRAM — batch sizes capped at 230
+- `SQLX_OFFLINE=true` for all cargo commands
+- Clippy denials: `unwrap_used`, `expect_used`, `panic`, `indexing_slicing`
+- Graceful skip if test data not found (CI-safe)
+
+## Non-Goals
+
+- Live trading or real broker integration
+- Multi-asset support (6E.FUT only)
+- Optimizing for positive Sharpe in Phase A (random weights)
+- Deploying all 4 models from hyperopt (DQN first)