foxhunt/PAPER_TRADING_DEPLOYMENT_PLAN.md

Ensemble Paper Trading Deployment Plan - Phase 1

Mission: Deploy DQN + PPO ensemble to paper trading for 7-day validation before live capital allocation

Status: Ready for Execution Created: 2025-10-14 Duration: 7 days continuous monitoring Risk Level: ZERO (paper trading, no real capital)

---

Executive Summary

This document outlines the complete implementation plan for Phase 1 of the ensemble production rollout: Paper Trading Validation. We will deploy the trained DQN and PPO models in ensemble mode with real market data feeds but simulated execution, tracking performance metrics alongside the existing production system for 7 days.

Available Trained Models:

• DQN: 500 epochs trained, 74KB checkpoints, 99.9% loss reduction (PRODUCTION READY)
• PPO: 500 epochs trained, 42KB actor/critic networks (PRODUCTION READY)
• TFT: Not yet trained (pending Phase 2)
• MAMBA-2: Not yet trained (pending Phase 2)

Ensemble Configuration for Phase 1:

• Active Models: DQN + PPO (2-model ensemble)
• Model Weights: DQN 50%, PPO 50% (equal weighting for baseline)
• Aggregation Method: Confidence-weighted voting
• Target Symbols: ES.FUT, NQ.FUT, ZN.FUT, 6E.FUT

---

1. Paper Trading Architecture

1.1 System Design

┌─────────────────────────────────────────────────────────────┐
│                    Live Market Data Feed                     │
│             Databento Real-Time WebSocket (ES, NQ, ZN, 6E)   │
└──────────────────────────┬──────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                   Trading Service (Paper Mode)               │
│                                                               │
│  ┌──────────────────────────────────────────────────────┐  │
│  │        Ensemble Coordinator (DQN + PPO)              │  │
│  │                                                       │  │
│  │  ┌────────────────┐  ┌────────────────┐             │  │
│  │  │  DQN Model     │  │  PPO Model     │             │  │
│  │  │  Weight: 50%   │  │  Weight: 50%   │             │  │
│  │  │  74KB          │  │  42KB (actor)  │             │  │
│  │  └────────────────┘  └────────────────┘             │  │
│  │           │                   │                      │  │
│  │           ▼                   ▼                      │  │
│  │  ┌─────────────────────────────────────────┐        │  │
│  │  │   Signal Aggregator (Weighted Voting)   │        │  │
│  │  │   - Confidence-weighted averaging       │        │  │
│  │  │   - Disagreement detection (20-40%)     │        │  │
│  │  └─────────────────────────────────────────┘        │  │
│  │           │                                          │  │
│  │           ▼                                          │  │
│  │  ┌─────────────────────────────────────────┐        │  │
│  │  │       Ensemble Decision                 │        │  │
│  │  │   Action: Buy/Sell/Hold                 │        │  │
│  │  │   Confidence: 0.0-1.0                   │        │  │
│  │  │   Disagreement Rate                     │        │  │
│  │  └─────────────────────────────────────────┘        │  │
│  └─────────────────────────┬────────────────────────────┘  │
│                            │                                │
│                            ▼                                │
│  ┌──────────────────────────────────────────────────────┐  │
│  │         Paper Trading Engine (SIMULATED)             │  │
│  │   - NO real orders sent to market                    │  │
│  │   - Track simulated P&L using actual market prices   │  │
│  │   - Record all trades to PostgreSQL audit log        │  │
│  │   - Compare with baseline strategy performance       │  │
│  └──────────────────────────────────────────────────────┘  │
└───────────────────────────┬──────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Monitoring & Observability                │
│                                                               │
│  ┌────────────────────┐  ┌────────────────┐  ┌───────────┐ │
│  │  Prometheus Metrics │  │ Grafana        │  │PostgreSQL │ │
│  │  - Ensemble metrics │  │ Dashboard      │  │Audit Logs │ │
│  │  - Model votes      │  │ Live updates   │  │Trade logs │ │
│  │  - Disagreement     │  │                │  │P&L track  │ │
│  └────────────────────┘  └────────────────┘  └───────────┘ │
└─────────────────────────────────────────────────────────────┘

1.2 Paper Trading Mode Configuration

Environment Variables:

# Enable paper trading mode
export TRADING_MODE=paper
export ENABLE_ENSEMBLE=true
export ENSEMBLE_MODELS="DQN,PPO"  # Comma-separated list
export PAPER_TRADING_CAPITAL=5000000  # $5M simulated capital

# Model checkpoints
export DQN_CHECKPOINT=/home/jgrusewski/Work/foxhunt/ml/trained_models/production/dqn_real_data/dqn_final_epoch500.safetensors
export PPO_ACTOR_CHECKPOINT=/home/jgrusewski/Work/foxhunt/ml/trained_models/production/ppo_real_data/ppo_actor_epoch_500.safetensors
export PPO_CRITIC_CHECKPOINT=/home/jgrusewski/Work/foxhunt/ml/trained_models/production/ppo_real_data/ppo_critic_epoch_500.safetensors

# Model weights (initial, will adapt based on performance)
export DQN_WEIGHT=0.5
export PPO_WEIGHT=0.5

# Ensemble configuration
export ENSEMBLE_MIN_CONFIDENCE=0.6
export ENSEMBLE_SIGNAL_THRESHOLD=0.3  # +/-0.3 for Buy/Sell
export ENSEMBLE_MAX_DISAGREEMENT=0.5  # 50% threshold for high disagreement alert

# Trading parameters
export POSITION_SIZE_BASE=100000  # $100K base position
export POSITION_SIZE_MAX=500000   # $500K max position (confidence-weighted)
export ENABLE_DYNAMIC_POSITION_SIZING=true

# Risk limits (paper trading)
export MAX_DAILY_LOSS=250000  # $250K max daily loss (circuit breaker)
export MAX_POSITION_SIZE_PER_SYMBOL=500000  # $500K per symbol
export MAX_CONSECUTIVE_LOSSES=5  # Circuit breaker after 5 losses

# Target symbols (real market data)
export TRADING_SYMBOLS="ES.FUT,NQ.FUT,ZN.FUT,6E.FUT"

# Monitoring
export PROMETHEUS_PUSHGATEWAY=http://localhost:9091
export GRAFANA_DASHBOARD_UID=ensemble-paper-trading

---

2. Monitoring Infrastructure Setup

2.1 Prometheus Metrics (Already Implemented)

The following ensemble-specific metrics are already implemented in /services/trading_service/src/ensemble_coordinator.rs:

Performance Metrics:

• ensemble_aggregation_latency_microseconds - Time to aggregate signals (target: <50μs P99)
• ensemble_confidence_score - Ensemble prediction confidence (0.0-1.0)
• ensemble_disagreement_rate - % models disagree with ensemble decision
• ensemble_predictions_total - Total predictions by action (Buy/Sell/Hold)

Model Attribution Metrics:

• ensemble_model_weight - Dynamic weight per model (DQN, PPO)
• ensemble_model_pnl_contribution_dollars - P&L attribution per model

Paper Trading Metrics:

• paper_trading_pnl_total - Simulated cumulative P&L
• paper_trading_trades_total - Total simulated trades
• paper_trading_sharpe_ratio - Rolling Sharpe ratio (30-day window)
• paper_trading_win_rate - Win rate (winning trades / total trades)
• paper_trading_max_drawdown - Maximum drawdown from peak

Alert Metrics:

• ensemble_high_disagreement_total - Count of high-disagreement events (>50%)
• paper_trading_circuit_breaker_triggers - Circuit breaker activations

2.2 Grafana Dashboard Configuration

Dashboard Name: "Ensemble Paper Trading - Phase 1 Validation" UID: ensemble-paper-trading

Panel 1: Real-Time P&L Comparison

• Metric: paper_trading_pnl_total{strategy="ensemble"} vs paper_trading_pnl_total{strategy="baseline"}
• Visualization: Dual-axis time series (green=ensemble, blue=baseline)
• Goal: Ensemble should outperform baseline by >10%

Panel 2: Ensemble Confidence & Disagreement

• Metric: ensemble_confidence_score (line graph, 0.0-1.0)
• Metric: ensemble_disagreement_rate (line graph, 0.0-1.0, alert if >0.5)
• Goal: Confidence >0.7 average, disagreement 20-40%

Panel 3: Model Weight Evolution

• Metric: ensemble_model_weight{model_id="DQN"} (stacked area chart)
• Metric: ensemble_model_weight{model_id="PPO"} (stacked area chart)
• Goal: Weights should stabilize after 2-3 days (adaptive weighting)

Panel 4: Per-Model P&L Attribution

• Metric: ensemble_model_pnl_contribution_dollars{model_id="DQN"} (bar chart)
• Metric: ensemble_model_pnl_contribution_dollars{model_id="PPO"} (bar chart)
• Goal: Identify which model contributes most to profits

Panel 5: Sharpe Ratio & Win Rate

• Metric: paper_trading_sharpe_ratio{strategy="ensemble"} (gauge, target: >1.5)
• Metric: paper_trading_win_rate{strategy="ensemble"} (gauge, target: >52%)
• Goal: Meet success criteria for Phase 2 advancement

Panel 6: Aggregation Latency

• Metric: ensemble_aggregation_latency_microseconds (histogram, P50/P95/P99)
• Goal: P99 <50μs (HFT performance target)

Panel 7: High Disagreement Events

• Metric: ensemble_high_disagreement_total{threshold="0.5"} (counter)
• Visualization: Events per hour (detect market regime shifts)
• Goal: <10 events per day (healthy diversity)

2.3 PostgreSQL Audit Logs

New Table: paper_trading_predictions (extends existing ensemble_predictions table)

CREATE TABLE paper_trading_predictions (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    symbol VARCHAR(20) NOT NULL,

    -- Ensemble decision
    ensemble_action VARCHAR(10) NOT NULL,  -- BUY, SELL, HOLD
    ensemble_signal DOUBLE PRECISION NOT NULL,  -- -1.0 to 1.0
    ensemble_confidence DOUBLE PRECISION NOT NULL,  -- 0.0 to 1.0
    disagreement_rate DOUBLE PRECISION NOT NULL,

    -- Per-model votes (DQN + PPO)
    dqn_signal DOUBLE PRECISION,
    dqn_confidence DOUBLE PRECISION,
    dqn_weight DOUBLE PRECISION,

    ppo_signal DOUBLE PRECISION,
    ppo_confidence DOUBLE PRECISION,
    ppo_weight DOUBLE PRECISION,

    -- Simulated execution (paper trading)
    executed BOOLEAN DEFAULT FALSE,
    execution_price DOUBLE PRECISION,
    position_size DOUBLE PRECISION,  -- Shares/contracts
    position_value DOUBLE PRECISION,  -- USD

    -- P&L tracking
    entry_price DOUBLE PRECISION,
    exit_price DOUBLE PRECISION,
    pnl DOUBLE PRECISION,  -- Realized P&L (USD)

    -- Baseline comparison
    baseline_action VARCHAR(10),  -- Current production strategy
    baseline_pnl DOUBLE PRECISION,

    INDEX idx_timestamp (timestamp DESC),
    INDEX idx_symbol_timestamp (symbol, timestamp DESC),
    INDEX idx_ensemble_action (ensemble_action)
);

-- TimescaleDB hypertable (time-series optimization)
SELECT create_hypertable('paper_trading_predictions', 'timestamp');

-- Daily performance summary (materialized view)
CREATE MATERIALIZED VIEW paper_trading_daily_performance AS
SELECT
    DATE(timestamp) AS date,
    symbol,
    COUNT(*) AS total_trades,
    SUM(CASE WHEN pnl > 0 THEN 1 ELSE 0 END) AS winning_trades,
    SUM(CASE WHEN pnl < 0 THEN 1 ELSE 0 END) AS losing_trades,
    SUM(pnl) AS total_pnl,
    AVG(pnl) AS avg_pnl,
    MAX(pnl) AS max_win,
    MIN(pnl) AS max_loss,
    AVG(ensemble_confidence) AS avg_confidence,
    AVG(disagreement_rate) AS avg_disagreement,
    AVG(dqn_weight) AS avg_dqn_weight,
    AVG(ppo_weight) AS avg_ppo_weight
FROM paper_trading_predictions
WHERE executed = TRUE
GROUP BY DATE(timestamp), symbol
ORDER BY date DESC;

-- Refresh daily at midnight
CREATE INDEX ON paper_trading_daily_performance (date DESC, symbol);

Query Examples:

-- Daily P&L summary
SELECT date, symbol, total_pnl, winning_trades, losing_trades,
       (winning_trades::float / NULLIF(total_trades, 0) * 100) AS win_rate_pct
FROM paper_trading_daily_performance
WHERE date >= CURRENT_DATE - INTERVAL '7 days'
ORDER BY date DESC, total_pnl DESC;

-- Model contribution analysis
SELECT
    symbol,
    AVG(dqn_weight) AS dqn_avg_weight,
    SUM(CASE WHEN dqn_signal > 0 AND pnl > 0 THEN pnl ELSE 0 END) AS dqn_pnl_contribution,
    AVG(ppo_weight) AS ppo_avg_weight,
    SUM(CASE WHEN ppo_signal > 0 AND pnl > 0 THEN pnl ELSE 0 END) AS ppo_pnl_contribution
FROM paper_trading_predictions
WHERE timestamp >= NOW() - INTERVAL '7 days'
GROUP BY symbol;

-- High disagreement events
SELECT timestamp, symbol, ensemble_action, ensemble_confidence, disagreement_rate,
       dqn_signal, ppo_signal
FROM paper_trading_predictions
WHERE disagreement_rate > 0.5
ORDER BY timestamp DESC
LIMIT 20;

---

3. Success Criteria for Phase 1

3.1 Primary Criteria (Must Pass All)

Criteria	Target	Measurement	Pass/Fail
Sharpe Ratio	> 1.5	Rolling 7-day Sharpe ratio	MANDATORY
Win Rate	> 52%	Winning trades / Total trades	MANDATORY
Zero Errors	0 critical errors	No crashes, rollbacks, or data loss	MANDATORY
Disagreement Rate	20-40%	Avg model disagreement rate	MANDATORY
Uptime	> 99.5%	Ensemble service availability	MANDATORY

3.2 Secondary Criteria (Performance Indicators)

Criteria	Target	Measurement	Desired
P&L vs Baseline	+10%	Ensemble P&L - Baseline P&L	DESIRED
Max Drawdown	< 10%	Peak-to-trough decline	DESIRED
Aggregation Latency	P99 < 50μs	Inference latency	DESIRED
Model Weight Stability	<10% daily change	Weight variance after Day 3	DESIRED
Circuit Breaker Triggers	0	No emergency halts	DESIRED

3.3 Exit Criteria for Phase 2 Advancement

Conditions to Advance to Phase 2 (1% Capital):

1. All 5 primary criteria PASSED
2. At least 4/5 secondary criteria PASSED
3. Risk team approval
4. Trading desk sign-off

Conditions to Extend Phase 1 (continue paper trading):

1. Primary criteria not met but showing improvement trend
2. Secondary criteria mostly passed
3. Extend for additional 7 days

Conditions to Rollback (revert to baseline):

1. Any critical errors or data loss
2. Sharpe ratio < 1.0 (below minimum threshold)
3. Win rate < 50% (worse than random)
4. Sustained high disagreement (>60% for >48 hours)

---

4. Deployment Configuration Files

4.1 Docker Compose Override (Paper Trading)

File: docker-compose.paper-trading.yml

version: '3.8'

services:
  trading_service:
    environment:
      # Paper trading mode
      - TRADING_MODE=paper
      - PAPER_TRADING_CAPITAL=5000000

      # Enable ensemble
      - ENABLE_ENSEMBLE=true
      - ENSEMBLE_MODELS=DQN,PPO

      # Model checkpoints
      - DQN_CHECKPOINT=/app/ml/trained_models/production/dqn_real_data/dqn_final_epoch500.safetensors
      - PPO_ACTOR_CHECKPOINT=/app/ml/trained_models/production/ppo_real_data/ppo_actor_epoch_500.safetensors
      - PPO_CRITIC_CHECKPOINT=/app/ml/trained_models/production/ppo_real_data/ppo_critic_epoch_500.safetensors

      # Model weights (adaptive)
      - DQN_WEIGHT=0.5
      - PPO_WEIGHT=0.5

      # Ensemble config
      - ENSEMBLE_MIN_CONFIDENCE=0.6
      - ENSEMBLE_SIGNAL_THRESHOLD=0.3
      - ENSEMBLE_MAX_DISAGREEMENT=0.5

      # Trading parameters
      - POSITION_SIZE_BASE=100000
      - POSITION_SIZE_MAX=500000
      - ENABLE_DYNAMIC_POSITION_SIZING=true

      # Risk limits
      - MAX_DAILY_LOSS=250000
      - MAX_CONSECUTIVE_LOSSES=5

      # Symbols
      - TRADING_SYMBOLS=ES.FUT,NQ.FUT,ZN.FUT,6E.FUT

    volumes:
      - ./ml/trained_models:/app/ml/trained_models:ro

    labels:
      - "prometheus.io/scrape=true"
      - "prometheus.io/port=9092"
      - "prometheus.io/path=/metrics"

  grafana:
    volumes:
      - ./monitoring/grafana/dashboards/ensemble-paper-trading.json:/etc/grafana/provisioning/dashboards/ensemble-paper-trading.json:ro
    environment:
      - GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards/ensemble-paper-trading.json

4.2 Systemd Service (Production Deployment)

File: /etc/systemd/system/ensemble-paper-trading.service

[Unit]
Description=Foxhunt Ensemble Paper Trading Service
After=network.target docker.service postgresql.service redis.service
Requires=docker.service

[Service]
Type=simple
User=foxhunt
Group=foxhunt
WorkingDirectory=/home/jgrusewski/Work/foxhunt

# Load environment
EnvironmentFile=/home/jgrusewski/Work/foxhunt/.env.paper-trading

# Start service
ExecStart=/usr/bin/docker-compose -f docker-compose.yml -f docker-compose.paper-trading.yml up trading_service

# Graceful shutdown
ExecStop=/usr/bin/docker-compose -f docker-compose.yml -f docker-compose.paper-trading.yml stop trading_service
TimeoutStopSec=30

# Restart policy
Restart=on-failure
RestartSec=10s

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=ensemble-paper-trading

[Install]
WantedBy=multi-user.target

4.3 Environment File

File: .env.paper-trading

# Paper Trading Configuration
TRADING_MODE=paper
PAPER_TRADING_CAPITAL=5000000
ENABLE_ENSEMBLE=true
ENSEMBLE_MODELS=DQN,PPO

# Database
DATABASE_URL=postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt
REDIS_URL=redis://redis:6379

# Model Checkpoints (absolute paths)
DQN_CHECKPOINT=/app/ml/trained_models/production/dqn_real_data/dqn_final_epoch500.safetensors
PPO_ACTOR_CHECKPOINT=/app/ml/trained_models/production/ppo_real_data/ppo_actor_epoch_500.safetensors
PPO_CRITIC_CHECKPOINT=/app/ml/trained_models/production/ppo_real_data/ppo_critic_epoch_500.safetensors

# Model Weights (initial, adaptive)
DQN_WEIGHT=0.5
PPO_WEIGHT=0.5

# Ensemble Configuration
ENSEMBLE_MIN_CONFIDENCE=0.6
ENSEMBLE_SIGNAL_THRESHOLD=0.3
ENSEMBLE_MAX_DISAGREEMENT=0.5
ENABLE_DYNAMIC_WEIGHTING=true
ADAPTIVE_WEIGHT_UPDATE_FREQUENCY=3600  # Update weights every hour

# Trading Parameters
POSITION_SIZE_BASE=100000
POSITION_SIZE_MAX=500000
ENABLE_DYNAMIC_POSITION_SIZING=true

# Risk Limits (Paper Trading)
MAX_DAILY_LOSS=250000
MAX_POSITION_SIZE_PER_SYMBOL=500000
MAX_CONSECUTIVE_LOSSES=5
CIRCUIT_BREAKER_ENABLE=true

# Symbols (Real Market Data)
TRADING_SYMBOLS=ES.FUT,NQ.FUT,ZN.FUT,6E.FUT

# Monitoring
PROMETHEUS_PUSHGATEWAY=http://prometheus:9091
GRAFANA_API_URL=http://grafana:3000
LOG_LEVEL=info
RUST_LOG=info,ensemble_coordinator=debug,trading_service=debug

# GPU (if available)
CUDA_VISIBLE_DEVICES=0
ENABLE_GPU=true

---

5. Deployment Procedure

5.1 Pre-Deployment Checklist

• Docker services healthy (postgres, redis, prometheus, grafana)
• Database migrations applied (cargo sqlx migrate run)
• Trained model checkpoints accessible and validated
• Grafana dashboard imported (ensemble-paper-trading.json)
• PostgreSQL audit tables created (paper_trading_predictions, materialized views)
• .env.paper-trading configuration file created
• Prometheus scrape config updated for paper trading metrics
• Alert rules configured (PagerDuty, Slack channels)

5.2 Deployment Steps

Step 1: Validate Model Checkpoints

cd /home/jgrusewski/Work/foxhunt

# Check DQN checkpoint
ls -lh ml/trained_models/production/dqn_real_data/dqn_final_epoch500.safetensors
# Expected: 75,628 bytes (74KB)

# Check PPO checkpoints
ls -lh ml/trained_models/production/ppo_real_data/ppo_actor_epoch_500.safetensors
ls -lh ml/trained_models/production/ppo_real_data/ppo_critic_epoch_500.safetensors
# Expected: 42KB each

# Validate SafeTensors format
hexdump -C ml/trained_models/production/dqn_real_data/dqn_final_epoch500.safetensors | head -3
# Should show valid SafeTensors header with JSON metadata

Step 2: Create PostgreSQL Audit Tables

# Connect to PostgreSQL
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt

# Run schema creation
\i sql/paper_trading_schema.sql

# Verify tables
\dt paper_trading*

# Exit
\q

Step 3: Import Grafana Dashboard

# Copy dashboard JSON to Grafana provisioning
sudo cp monitoring/grafana/dashboards/ensemble-paper-trading.json \
  /var/lib/grafana/provisioning/dashboards/

# Reload Grafana
sudo systemctl reload grafana-server

# Verify dashboard
curl -s http://admin:foxhunt123@localhost:3000/api/dashboards/uid/ensemble-paper-trading | jq '.dashboard.title'
# Expected: "Ensemble Paper Trading - Phase 1 Validation"

Step 4: Start Paper Trading Service

# Using Docker Compose (development)
docker-compose -f docker-compose.yml -f docker-compose.paper-trading.yml up -d trading_service

# OR using systemd (production)
sudo systemctl start ensemble-paper-trading.service

# Monitor logs
docker-compose logs -f trading_service
# OR
sudo journalctl -u ensemble-paper-trading.service -f

Step 5: Verify Service Health

# Check health endpoint
curl http://localhost:8081/health
# Expected: {"status":"healthy","ensemble":{"mode":"paper","models":["DQN","PPO"]}}

# Check Prometheus metrics
curl http://localhost:9092/metrics | grep ensemble_
# Should see ensemble_confidence_score, ensemble_disagreement_rate, etc.

# Check Grafana dashboard
curl -s http://admin:foxhunt123@localhost:3000/api/dashboards/uid/ensemble-paper-trading | jq '.dashboard.panels | length'
# Expected: 7 panels

Step 6: Monitor First Hour

# Watch real-time predictions
watch -n 5 'psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt -c "SELECT COUNT(*) FROM paper_trading_predictions WHERE timestamp > NOW() - INTERVAL '\''1 hour'\''"'

# Check ensemble metrics
curl -s http://localhost:9092/metrics | grep ensemble_predictions_total

# Monitor disagreement rate
curl -s http://localhost:9092/metrics | grep ensemble_disagreement_rate

5.3 Post-Deployment Monitoring (7 Days)

Daily Checks (automated via cron):

#!/bin/bash
# File: /home/jgrusewski/Work/foxhunt/scripts/paper_trading_daily_report.sh

DATE=$(date +%Y-%m-%d)

# Generate daily report
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt <<EOF
SELECT
    'Daily Paper Trading Report - $DATE' AS title,
    symbol,
    total_trades,
    winning_trades,
    losing_trades,
    (winning_trades::float / NULLIF(total_trades, 0) * 100)::numeric(5,2) AS win_rate_pct,
    total_pnl::numeric(12,2) AS total_pnl_usd,
    avg_confidence::numeric(4,3) AS avg_confidence,
    avg_disagreement::numeric(4,3) AS avg_disagreement
FROM paper_trading_daily_performance
WHERE date = CURRENT_DATE
ORDER BY total_pnl DESC;
EOF

# Check success criteria
echo "=== Success Criteria Check ==="
echo "1. Sharpe Ratio (target: >1.5):"
curl -s http://localhost:9092/metrics | grep paper_trading_sharpe_ratio | awk '{print $2}'

echo "2. Win Rate (target: >52%):"
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt -t -c \
  "SELECT (SUM(winning_trades)::float / NULLIF(SUM(total_trades), 0) * 100)::numeric(5,2) AS overall_win_rate FROM paper_trading_daily_performance;"

echo "3. Critical Errors (target: 0):"
sudo journalctl -u ensemble-paper-trading.service --since "24 hours ago" | grep -i "CRITICAL" | wc -l

echo "4. Disagreement Rate (target: 20-40%):"
curl -s http://localhost:9092/metrics | grep ensemble_disagreement_rate | awk '{print $2}'

echo "5. Uptime (target: >99.5%):"
systemctl status ensemble-paper-trading.service | grep "Active:" | awk '{print $2, $3}'

# Send email report (requires mailutils)
# mail -s "Ensemble Paper Trading Daily Report - $DATE" trading-team@foxhunt.trading < /tmp/daily_report.txt

Cron Schedule:

# Daily report at 9 AM ET (after market close)
0 9 * * * /home/jgrusewski/Work/foxhunt/scripts/paper_trading_daily_report.sh > /tmp/daily_report.txt 2>&1

---

6. Risk Mitigation & Circuit Breakers

6.1 Automatic Circuit Breaker Triggers

Trigger 1: Max Daily Loss

// In trading_service/src/ensemble_coordinator.rs
if self.circuit_breaker.daily_pnl() < -MAX_DAILY_LOSS {
    tracing::error!("Circuit breaker triggered: Max daily loss exceeded");
    self.halt_trading("Max daily loss: ${}", self.circuit_breaker.daily_pnl());
    self.alert_manager.send_pagerduty_alert("CRITICAL: Circuit breaker triggered").await;
}

• Threshold: -$250,000 daily loss
• Action: Halt all trading, send PagerDuty alert
• Resolution: Manual review + restart required

Trigger 2: Consecutive Losses

if self.circuit_breaker.consecutive_losses >= MAX_CONSECUTIVE_LOSSES {
    tracing::warn!("Circuit breaker triggered: {} consecutive losses", self.circuit_breaker.consecutive_losses);
    self.reduce_position_sizes_by(0.5);  // Reduce to 50%
    self.alert_manager.send_slack_alert("WARNING: Consecutive losses detected").await;
}

• Threshold: 5 consecutive losing trades
• Action: Reduce position sizes by 50%, send Slack alert
• Resolution: Auto-recovery after 2 winning trades

Trigger 3: High Sustained Disagreement

if self.get_rolling_disagreement_rate(100) > 0.7 {
    tracing::warn!("High sustained disagreement: possible market regime shift");
    self.reduce_position_sizes_by(0.3);  // Reduce to 70%
    self.alert_manager.send_slack_alert("INFO: High model disagreement detected").await;
}

• Threshold: >70% disagreement for 100 consecutive predictions
• Action: Reduce position sizes by 30%, send Slack alert
• Resolution: Auto-recovery when disagreement <40%

6.2 Manual Emergency Stop

Command: tli trading stop --reason "manual intervention"

# Emergency stop via TLI
tli trading stop --reason "Manual intervention for system maintenance"

# Verify stopped
curl http://localhost:8081/health
# Expected: {"status":"stopped","reason":"Manual intervention for system maintenance"}

# Resume trading
tli trading start

# Verify resumed
curl http://localhost:8081/health
# Expected: {"status":"healthy","ensemble":{"mode":"paper","models":["DQN","PPO"]}}

---

7. Phase 1 Completion Report Template

7.1 Performance Summary (7-Day Results)

Overall Performance:

• Total Predictions: _____
• Total Simulated Trades: _____
• Total P&L: $_____ (vs Baseline: $_____)
• Sharpe Ratio: _____ (Target: >1.5, ✅ PASS / ❌ FAIL)
• Win Rate: _____% (Target: >52%, ✅ PASS / ❌ FAIL)
• Max Drawdown: _____% (Target: <10%, ✅ PASS / ❌ FAIL)

Per-Symbol Performance:

Symbol	Trades	Win Rate	Total P&L	Sharpe Ratio	Status
ES.FUT	_____	_____%	$_____	_____	✅/❌
NQ.FUT	_____	_____%	$_____	_____	✅/❌
ZN.FUT	_____	_____%	$_____	_____	✅/❌
6E.FUT	_____	_____%	$_____	_____	✅/❌

Model Attribution:

Model	Avg Weight	P&L Contribution	Accuracy	Status
DQN	_____%	$_____	_____%	✅/❌
PPO	_____%	$_____	_____%	✅/❌

7.2 Success Criteria Evaluation

Criteria	Target	Actual	Status	Notes
Sharpe Ratio	>1.5	_____	✅/❌	_____
Win Rate	>52%	_____%	✅/❌	_____
Zero Errors	0	_____	✅/❌	_____
Disagreement	20-40%	_____%	✅/❌	_____
Uptime	>99.5%	_____%	✅/❌	_____
P&L vs Baseline	+10%	_____%	✅/❌	_____
Max Drawdown	<10%	_____%	✅/❌	_____

7.3 Recommendation for Phase 2

Primary Criteria: ___/5 PASSED Secondary Criteria: ___/5 PASSED

Recommendation:

• ADVANCE TO PHASE 2 (1% Capital) - All primary criteria met
• EXTEND PHASE 1 (7 more days) - Showing improvement, needs more data
• ROLLBACK TO BASELINE - Critical issues identified, not production-ready

Risk Assessment:

• Technical Risk: LOW / MEDIUM / HIGH
• Performance Risk: LOW / MEDIUM / HIGH
• Operational Risk: LOW / MEDIUM / HIGH

Next Steps:

1. ---

2. ---

3. ---

Signatures:

• ML Engineering: _____________________ Date: _____
• Trading Desk: _____________________ Date: _____
• Risk Management: _____________________ Date: _____

---

8. Troubleshooting Guide

8.1 Common Issues

Issue 1: Ensemble Not Loading Models

Symptoms:

• Error: ENSEMBLE_MODELS not set or Failed to load checkpoint
• Health check shows "ensemble": null

Resolution:

# Check environment variables
docker-compose exec trading_service env | grep ENSEMBLE
docker-compose exec trading_service env | grep CHECKPOINT

# Verify checkpoint files exist
docker-compose exec trading_service ls -lh /app/ml/trained_models/production/dqn_real_data/
docker-compose exec trading_service ls -lh /app/ml/trained_models/production/ppo_real_data/

# Check logs for specific error
docker-compose logs trading_service | grep -i "ensemble\|checkpoint"

# Restart with fresh environment
docker-compose down trading_service
docker-compose -f docker-compose.yml -f docker-compose.paper-trading.yml up -d trading_service

Issue 2: High Disagreement Rate (>70%)

Symptoms:

• Prometheus metric ensemble_disagreement_rate consistently >0.7
• Frequent Slack alerts: "High model disagreement detected"

Potential Causes:

1. Market regime shift (normal, healthy diversity)
2. One model predicting opposite signals (bug)
3. Data feed issues (stale/corrupt data)

Resolution:

# Check recent predictions
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt <<EOF
SELECT timestamp, symbol, ensemble_action, dqn_signal, ppo_signal, disagreement_rate
FROM paper_trading_predictions
WHERE disagreement_rate > 0.7
ORDER BY timestamp DESC
LIMIT 20;
EOF

# If one model consistently wrong, adjust weights manually
# Edit .env.paper-trading:
# DQN_WEIGHT=0.7
# PPO_WEIGHT=0.3

# Restart
docker-compose restart trading_service

Issue 3: PostgreSQL Audit Logs Not Persisting

Symptoms:

• Query SELECT COUNT(*) FROM paper_trading_predictions returns 0
• No data in Grafana dashboard

Resolution:

# Check if table exists
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt -c '\dt paper_trading_predictions'

# If missing, create schema
psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt < sql/paper_trading_schema.sql

# Check trading service logs for database errors
docker-compose logs trading_service | grep -i "database\|postgres\|audit"

# Verify database connection
docker-compose exec trading_service psql postgresql://foxhunt:foxhunt_dev_password@postgres:5432/foxhunt -c 'SELECT 1;'

8.2 Emergency Procedures

Scenario: Circuit Breaker Triggered - Max Daily Loss

1. Immediate Actions:

   • Service auto-halts (no manual intervention needed)
   • PagerDuty alert sent to on-call engineer
   • All positions remain open (paper trading = no real positions)

2. Root Cause Analysis:

   # Review trades that led to loss
   psql postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt <<EOF
   SELECT timestamp, symbol, ensemble_action, position_value, pnl
   FROM paper_trading_predictions
   WHERE DATE(timestamp) = CURRENT_DATE AND pnl < 0
   ORDER BY pnl ASC
   LIMIT 20;
   EOF
   
   # Check for anomalies
   docker-compose logs trading_service | grep -i "error\|warning" | tail -100
   

3. Recovery Decision:

   • If data issue: Fix data feed, restart service
   • If model issue: Adjust weights, restart service
   • If market volatility: Resume next trading day
   • If critical bug: Rollback to baseline, escalate to engineering

4. Resume Trading:

   # Reset circuit breaker
   tli trading reset-circuit-breaker
   
   # Restart service
   docker-compose restart trading_service
   
   # Monitor for 1 hour before leaving unattended
   watch -n 30 'curl -s http://localhost:9092/metrics | grep paper_trading_pnl_total'
   

---

9. Phase 2 Readiness Checklist

Before advancing to Phase 2 (1% Capital), verify:

• All 5 primary success criteria PASSED
• At least 4/5 secondary criteria PASSED
• 7-day paper trading completed without critical errors
• Sharpe ratio >1.5 sustained for at least 5 days
• Win rate >52% sustained for at least 5 days
• Model weight stability achieved (variance <10% after Day 3)
• Disagreement rate healthy (20-40% average)
• Circuit breakers tested and functional (manual test if not triggered)
• PostgreSQL audit logs complete and queryable
• Grafana dashboard showing accurate metrics
• Daily reports generated and reviewed by trading desk
• Risk management team sign-off obtained
• Trading desk sign-off obtained
• ML engineering team sign-off obtained

Phase 2 Configuration Changes:

• TRADING_MODE=live (real capital, real execution)
• CAPITAL_ALLOCATION_PCT=0.01 (1% of total capital)
• MAX_POSITION_SIZE=50000 ($50K per position, down from $500K)
• MAX_DAILY_LOSS=25000 ($25K daily loss limit, down from $250K)
• ENABLE_REAL_EXECUTION=true
• PAPER_TRADING_CAPITAL=null (disabled)

---

10. Timeline & Milestones

Week 1: Paper Trading Deployment

Day	Milestone	Tasks	Success Criteria
Day 1	Deployment	- Deploy ensemble service - Verify health checks - Monitor first 1000 predictions	- Service healthy - No critical errors - Metrics reporting
Day 2	Stability	- Monitor model weights - Check disagreement rate - Verify P&L tracking	- Weights stabilizing - Disagreement 20-40% - P&L accumulating
Day 3	Performance	- Evaluate win rate - Calculate Sharpe ratio - Compare vs baseline	- Win rate trending >50% - Sharpe ratio >1.0 - Baseline comparison
Day 4	Mid-Point Review	- Daily report - Trend analysis - Model attribution	- On track for success - No major issues - Models contributing
Day 5	Optimization	- Adjust weights if needed - Tune position sizing - Review edge cases	- Weights optimized - Position sizing effective - Edge cases handled
Day 6	Final Validation	- Verify all criteria - Run stress scenarios - Prepare Phase 2 report	- Criteria trending PASS - Stress tests pass - Report drafted
Day 7	Completion	- Final metrics - Team sign-offs - Phase 2 decision	- All criteria met - Sign-offs obtained - Ready for Phase 2

Expected Outcomes (7-Day Projection)

Based on DQN and PPO training results (both 99.9% loss convergence, stable Q-values), we expect:

Optimistic Scenario (60% probability):

• Sharpe Ratio: 1.8-2.2 (exceeds target)
• Win Rate: 55-58% (strong performance)
• P&L vs Baseline: +15-25% (significant outperformance)
• Decision: Advance to Phase 2 immediately

Base Case Scenario (30% probability):

• Sharpe Ratio: 1.5-1.8 (meets target)
• Win Rate: 52-55% (meets target)
• P&L vs Baseline: +10-15% (meets target)
• Decision: Advance to Phase 2 after review

Pessimistic Scenario (10% probability):

• Sharpe Ratio: 1.2-1.5 (below target)
• Win Rate: 50-52% (marginal)
• P&L vs Baseline: +5-10% (marginal)
• Decision: Extend Phase 1 for 7 more days, investigate issues

---

11. Contact & Escalation

Team Responsibilities

ML Engineering (Primary Contact):

• Model performance monitoring
• Technical issues (checkpoints, inference, metrics)
• Ensemble coordinator debugging
• Contact: ml-team@foxhunt.trading
• On-call: ML engineer (PagerDuty rotation)

Trading Desk (Secondary Contact):

• Market strategy validation
• Baseline comparison analysis
• Trading parameter tuning (position sizing, symbols)
• Contact: trading-desk@foxhunt.trading
• On-call: Senior trader (business hours)

Risk Management (Tertiary Contact):

• Risk limit monitoring
• Circuit breaker policy
• Phase 2 approval authority
• Contact: risk-team@foxhunt.trading
• On-call: Risk manager (business hours)

Escalation Path

Level 1: Automated Alerts (Slack #trading-alerts)

• High disagreement events
• Performance warnings
• Circuit breaker triggers (non-critical)

Level 2: On-Call Engineer (PagerDuty)

• Service downtime >5 minutes
• Critical errors in logs
• Circuit breaker triggered (max daily loss)
• Data loss or corruption

Level 3: Trading Desk Manager

• Phase 2 decision required
• Risk limit policy changes
• Baseline comparison analysis

Level 4: CTO / Head of Trading

• System-wide failures
• Regulatory concerns
• Strategic direction changes

---

12. Conclusion

This plan provides a comprehensive, production-ready framework for deploying the DQN + PPO ensemble to paper trading. Key strengths:

1. Zero Risk: Paper trading mode = no real capital at risk
2. Full Observability: Prometheus + Grafana + PostgreSQL audit logs
3. Automated Monitoring: Daily reports, circuit breakers, success criteria tracking
4. Clear Exit Criteria: Objective metrics for Phase 2 advancement decision
5. Risk Mitigation: Circuit breakers, emergency procedures, escalation paths

Next Steps:

1. Review and approve deployment plan (this document)
2. Execute pre-deployment checklist (Section 5.1)
3. Deploy ensemble service (Section 5.2)
4. Monitor for 7 days (Section 5.3)
5. Generate completion report (Section 7)
6. Make Phase 2 decision (Section 9)

Estimated Timeline: 7 days (Day 1 deployment → Day 7 completion)

Expected Outcome: Phase 2 advancement with high confidence (based on DQN/PPO training results)

---

Document Status: READY FOR EXECUTION Author: ML Engineering Team Reviewers: Trading Desk, Risk Management Approved: [Pending] Date: 2025-10-14