foxhunt/TEMPORAL_TRAIN_VAL_SPLIT_ANALYSIS.md

Temporal Train/Val Split Issue - Comprehensive Analysis

Date: 2025-11-03 Status: ANALYSIS ONLY - No code changes made Priority: MEDIUM (affects validation metrics, but NOT hyperopt objective) Scope: DQN, TFT, MAMBA-2, PPO trainers

---

Executive Summary

The codebase implements a sequential 80/20 temporal split across multiple trainers (DQN, TFT, MAMBA-2, PPO) that creates temporal leakage in validation data:

• Training data: First 80% of time series (e.g., Jan-Sep)
• Validation data: Last 20% of time series (e.g., Oct-Dec)
• Problem: Model trains on past data, validates on future → inflates validation performance

However, this is NOT critical for hyperopt because DQN's objective function uses avg_episode_reward (NOT validation loss). The validation loss is only used for:

1. Early stopping (plateau detection)
2. Best model checkpointing (saves model with best val loss)
3. Monitoring/logging (informational only)

Severity Assessment: MEDIUM - not critical for hyperopt, but affects early stopping accuracy and best model selection

---

Current Implementation Analysis

1. DQN Trainer (dqn.rs)

Data Split Locations:

• Line 1165-1167: Parquet loading path
• Line 1276-1278: DBN file loading path

Code:

// Split training data 80/20 for train/validation
let split_idx = (training_data.len() * 80) / 100;
let train_data = training_data[..split_idx].to_vec();
let val_data = training_data[split_idx..].to_vec();

Validation Usage (lines 854-878):

// Compute validation loss
let val_loss = self.compute_validation_loss().await?;
info!("Epoch {}/{}: val_loss={:.6}", epoch + 1, self.hyperparams.epochs, val_loss);

// Track metrics for early stopping
self.val_loss_history.push(val_loss);

// Save best model checkpoint if validation loss improved
if train_step_count > 0 && val_loss < self.best_val_loss {
    self.best_val_loss = val_loss;
    self.best_epoch = epoch + 1;
    // ... save checkpoint
}

compute_validation_loss() Implementation (lines 556-582):

async fn compute_validation_loss(&self) -> Result<f64> {
    if self.val_data.is_empty() {
        return Ok(0.0);
    }

    let mut total_loss = 0.0;
    let sample_size = self.val_data.len().min(1000); // Sample up to 1000 for speed

    for (feature_vec, target) in self.val_data.iter().take(sample_size) {
        let state = self.feature_vector_to_state(feature_vec)?;

        // Calculate reward
        let current_close = if target.len() >= 2 { target[0] } else { feature_vec[3] };
        let next_close = if target.len() >= 2 { target[1] } else { current_close };
        let reward = self.calculate_reward(current_close, next_close);

        // Get Q-values for the state
        let q_values = self.get_q_values(&state).await?;
        let max_q = q_values.iter().copied().fold(f64::NEG_INFINITY, f64::max);

        // Loss = (predicted_q - reward)^2
        let loss = (max_q - reward as f64).powi(2);
        total_loss += loss;
    }

    Ok(total_loss / sample_size as f64)
}

Early Stopping (lines 617-649):

fn check_early_stopping(&self, avg_q_value: f64, epoch: usize) -> Option<String> {
    // ... validation checks ...

    // Criterion 2: Validation loss plateau check
    if self.val_loss_history.len() >= self.hyperparams.plateau_window {
        let window = self.hyperparams.plateau_window;
        let recent_losses: Vec<f64> = self.val_loss_history
            .iter()
            .rev()
            .take(window)
            .copied()
            .collect();

        if let (Some(&first), Some(&last)) = (recent_losses.first(), recent_losses.last()) {
            let improvement = last - first;

            if improvement < 0.001 {
                return Some(format!(
                    "Validation loss plateau detected (improvement: {:.6})",
                    improvement
                ));
            }
        }
    }
}

2. DQN Hyperopt Adapter (adapters/dqn.rs)

Objective Function (lines 873-883):

fn extract_objective(metrics: &Self::Metrics) -> f64 {
    // CRITICAL: Maximize episode rewards (negative because optimizer MINIMIZES)
    //
    // We optimize for avg_episode_reward, NOT validation loss, because:
    // 1. Loss minimization rewards tiny batches (batch_size=32-43) that prevent learning
    // 2. Low batch sizes → noisy gradients → Q-values stay near zero → low loss
    // 3. Episode rewards measure actual trading performance (PnL)
    //
    // The optimizer minimizes this objective, so we negate rewards to maximize them.
    -metrics.avg_episode_reward
}

Key Finding: Hyperopt uses avg_episode_reward, NOT validation loss. This means temporal leakage in validation data does NOT affect hyperopt results.

Metrics Struct (lines 150-163):

pub struct DQNMetrics {
    pub train_loss: f64,
    pub val_loss: f64,           // ← Computed but NOT used in objective
    pub avg_q_value: f64,
    pub final_epsilon: f64,
    pub epochs_completed: usize,
    pub avg_episode_reward: f64, // ← THIS is the objective (not val_loss)
}

3. TFT Trainer (tft_parquet.rs)

Data Split (lines 57-66):

// Split into train/val (80/20)
let split_idx = (training_data.len() as f64 * 0.8) as usize;
let train_data = training_data[..split_idx].to_vec();
let val_data = training_data[split_idx..].to_vec();

Usage: TFT creates separate data loaders for train and val (lines 107-112):

let train_loader = TFTDataLoader::new(train_data.clone(), current_batch_size, true);
let val_loader = TFTDataLoader::new(
    val_data.clone(),
    self.get_training_config().validation_batch_size,
    false,  // Not training
);

Impact: TFT actively uses validation data during training loop for loss monitoring and early stopping. Temporal leakage affects TFT training more directly.

4. MAMBA-2 Trainer (mamba2.rs)

No explicit train/val split in trainer code. Validation data is passed from external loaders (lines 359-373):

pub async fn train_dbn(
    &mut self,
    train_data: &[(Tensor, Tensor)],
    val_data: &[(Tensor, Tensor)],  // ← Validation data source unclear
    // ...
) -> Result<TrainingMetrics>

Note: Need to check where val_data is created for MAMBA-2 in hyperopt adapter.

5. PPO Trainer (ppo.rs)

No explicit 80/20 split found in trainer code. PPO uses trajectory-based training (not direct time-series split).

---

Impact Assessment

1. Hyperopt Impact: LOW

Why? Hyperopt objective is avg_episode_reward, NOT validation loss:

• DQN hyperopt (adapters/dqn.rs:873-883): Objective = -metrics.avg_episode_reward
• Validation loss is computed but NOT used in optimization
• Episode reward is calculated from trading actions (PnL), not validation data

Evidence:

Hyperopt Trial #1 (best):
- Objective: 2.4023 (episode reward)
- Validation loss: 0.XXX (not reported in objective)

2. Early Stopping Impact: MEDIUM

Affected: DQN early stopping (check_early_stopping, lines 617-649)

• Uses val_loss_history for plateau detection
• Temporal leakage inflates validation loss (future data looks easier to predict)
• Plateau detection threshold (0.001 improvement) may trigger too early/late

Consequence:

• May stop training prematurely if val loss plateaus on future data
• Or may miss convergence if past data was harder than future data

3. Best Model Selection: MEDIUM

Affected: Model checkpoint management (lines 862-878)

if train_step_count > 0 && val_loss < self.best_val_loss {
    self.best_val_loss = val_loss;
    self.best_epoch = epoch + 1;
    // Save checkpoint
}

• Selects model based on lowest validation loss
• Temporal leakage means best model on future data may not be best on unseen data
• Model trained on Jan-Sep, validated on Oct-Dec, tested on Jan-Sep bias

4. TFT Impact: MEDIUM-HIGH

• TFT actively uses validation data during training
• Affects loss computation, early stopping, and model selection
• More severe than DQN because TFT uses val loss directly in training loop

5. Production Deployment Impact: UNKNOWN

• Backtesting uses walk-forward validation (barrier_backtest.rs)
• If models are trained with temporal leakage, backtesting results may be optimistic
• Need to verify backtesting code doesn't reuse temporal leakage

---

Detailed Analysis: Why Hyperopt is NOT Critical

Objective Function Decoupling

Hyperopt -> Optimization Loop
  |
  +---> DQN Trial Training
         |
         +---> compute_validation_loss()  [← computes but NOT used]
         +---> avg_episode_reward  [← THIS is the objective]

Optimizer minimizes: -avg_episode_reward (to maximize rewards)
Not affected by: validation loss (used only for early stopping)

Episode Reward vs. Validation Loss

Episode Reward (optimization objective):

// Calculated during training loop (lines 728-749)
let reward = match action {
    TradingAction::Buy => {
        (price_change / 10.0).clamp(-1.0, 1.0) as f32
    },
    TradingAction::Sell => {
        (-price_change / 10.0).clamp(-1.0, 1.0) as f32
    },
    TradingAction::Hold => {
        -0.0001_f32
    },
};

• Independent of validation data split
• Depends on: action selection + price changes
• Temporal leakage has NO effect on episode rewards

Validation Loss (used only for early stopping):

// Calculated on held-out data
let loss = (max_q - reward as f64).powi(2);

• Depends on validation data quality
• Affected by temporal leakage
• Used only for plateau detection, not optimization

---

Fix Options Assessment

Option A: Stratified Sampling (Every Nth Sample)

Implementation:

// Sample every 5th bar for validation (stratified)
let val_data: Vec<_> = training_data
    .iter()
    .enumerate()
    .filter(|(i, _)| i % 5 == 0)
    .map(|(_, d)| d.clone())
    .collect();

let train_data: Vec<_> = training_data
    .iter()
    .enumerate()
    .filter(|(i, _)| i % 5 != 0)
    .map(|(_, d)| d.clone())
    .collect();

Pros:

• ✅ Preserves temporal order (no leakage)
• ✅ Maintains temporal features
• ✅ Easy to implement
• ✅ Predictable validation set size

Cons:

• ❌ Reduces training data by 20% (4 out of 5 samples)
• ❌ Validation set is smaller (harder to evaluate)
• ❌ Alternating pattern may not be optimal

Recommendation: ✅ GOOD FIT - Best for this codebase

---

Option B: Walk-Forward Validation (Multiple Windows)

Implementation:

// Create k windows of (train, test) pairs
// Window 1: [0-20%] train, [20-40%] test
// Window 2: [0-40%] train, [40-60%] test
// Window 3: [0-60%] train, [60-80%] test
// etc.

let n_windows = 5;
let window_size = data.len() / n_windows;

for i in 1..n_windows {
    let train_end = i * window_size;
    let test_end = (i + 1) * window_size;

    let train = &data[..train_end];
    let test = &data[train_end..test_end];
}

Pros:

• ✅ No temporal leakage
• ✅ More robust (multiple validation windows)
• ✅ Better for time-series evaluation
• ✅ Prevents overfitting to specific test period

Cons:

• ❌ Requires multiple training runs (5-10x slower)
• ❌ Complex implementation
• ❌ Hyperopt trials would take 5-10x longer
• ❌ Not suitable for rapid hyperopt (already 15-minute runs)

Recommendation: ❌ TOO EXPENSIVE - Hyperopt is already fast (14 min), this would make it 70-140 min

---

Option C: Random Shuffling

Implementation:

// Shuffle all data, then split 80/20
let mut shuffled = training_data.clone();
shuffled.shuffle(&mut rng);

let split_idx = (shuffled.len() * 80) / 100;
let train_data = shuffled[..split_idx].to_vec();
let val_data = shuffled[split_idx..].to_vec();

Pros:

• ✅ Completely eliminates temporal leakage
• ✅ Simple to implement
• ✅ Fair distribution of past/future data

Cons:

• ❌ BREAKS temporal features (rolling windows, momentum, seasonality)
• ❌ Adjacent bars are critical for feature extraction
• ❌ Model learns temporal relationships that don't transfer to shuffled data
• ❌ Fundamental incompatibility with time-series features

Recommendation: ❌ NOT VIABLE - Destroys temporal dependencies

---

Recommended Fix: Option A (Stratified Sampling)

Implementation Plan

For DQN trainer (dqn.rs, lines 1165-1167):

// Instead of sequential split:
let split_idx = (training_data.len() * 80) / 100;
let train_data = training_data[..split_idx].to_vec();
let val_data = training_data[split_idx..].to_vec();

// Use stratified sampling:
let mut train_data = Vec::new();
let mut val_data = Vec::new();

for (i, sample) in training_data.into_iter().enumerate() {
    if i % 5 == 0 {
        val_data.push(sample);
    } else {
        train_data.push(sample);
    }
}

For TFT trainer (tft_parquet.rs, lines 57-66):

// Current:
let split_idx = (training_data.len() as f64 * 0.8) as usize;
let train_data = training_data[..split_idx].to_vec();
let val_data = training_data[split_idx..].to_vec();

// Proposed:
let mut train_data = Vec::new();
let mut val_data = Vec::new();

for (i, sample) in training_data.into_iter().enumerate() {
    if i % 5 == 0 {
        val_data.push(sample);
    } else {
        train_data.push(sample);
    }
}

For MAMBA-2/PPO:

• Need to investigate where validation splits occur in hyperopt adapters

Expected Outcomes

Before:

Training: [1-80% of time] Jan-Sep
Validation: [80-100% of time] Oct-Dec
Result: Model trains on past, tests on future (inflated val loss)

After (Stratified):

Training: Every 1, 2, 3, 4 sample (~80%)
Validation: Every 5th sample (~20%)
Result: Mixed temporal distribution (no leakage)

Risk Assessment

Low Risk because:

1. Hyperopt objective (avg_episode_reward) unaffected
2. Only validation loss changes (already not used in optimization)
3. Early stopping may be more conservative (better for safety)
4. No architectural changes needed

Potential Issues:

1. Validation set size reduced (1000 → 800 samples typical)
2. Early stopping plateau detection may behave differently
3. Need to retrain and verify no regression

---

Code Locations Summary

File	Lines	Component	Impact
ml/src/trainers/dqn.rs	1165-1167, 1276-1278	Data split	HIGH (2 locations)
ml/src/trainers/dqn.rs	556-582	compute_validation_loss	MEDIUM (monitoring only)
ml/src/trainers/dqn.rs	617-649	check_early_stopping	MEDIUM (plateau detection)
ml/src/trainers/dqn.rs	862-878	best model checkpoint	MEDIUM (model selection)
ml/src/hyperopt/adapters/dqn.rs	873-883	extract_objective	LOW (NOT used)
ml/src/trainers/tft_parquet.rs	57-66	Data split	MEDIUM (active usage)
ml/src/trainers/ppo.rs	?	Data split	UNKNOWN (no split found)
ml/src/trainers/mamba2.rs	?	Data split	UNKNOWN (external source)

---

Production Impact Assessment

Current System Status

Positive:

• ✅ DQN hyperopt uses avg_episode_reward (immune to temporal leakage)
• ✅ Best hyperparameters identified correctly (Policy LR 1e-6, Value LR 0.001)
• ✅ Training converges properly (episode rewards improve)

Negative:

• ⚠️ Validation loss may not reflect true out-of-sample performance
• ⚠️ Early stopping may trigger at wrong time
• ⚠️ Best model selection based on future data (not past)

Does This Explain Known Issues?

Pod 0hczpx9nj1ub88 (PPO stagnation):

• Loss stagnated at 1.158-1.159 for 200+ epochs
• Temporal leakage: NOT the cause (single learning rate issue)
• Root cause: Single --learning-rate 0.001 for both networks (should be 1e-6 for policy, 0.001 for value)

DQN Epoch 50 Early Stop:

• Temporal leakage: Could contribute to premature stopping
• min_epochs_before_stopping=50 + validation plateau detection
• Still not a bug (intentional early stopping), but temporal split may trigger it earlier than deserved

---

Severity Classification

Impact Matrix

System	Severity	Reason	Fix Urgency
DQN Hyperopt	🟢 LOW	Objective = episode reward (not val loss)	Not urgent
DQN Training	🟡 MEDIUM	Early stopping + best model selection	Moderate
TFT Training	🟡 MEDIUM	Active val loss usage in loop	Moderate
PPO Training	🔴 UNKNOWN	No split found in code	Investigate
MAMBA-2	🔴 UNKNOWN	Validation source unclear	Investigate
Production Backtesting	🔴 UNKNOWN	May inherit trained model bias	Investigate

---

Questions Answered

Q1: Is validation loss used in hyperopt objective?

A1: NO

fn extract_objective(metrics: &Self::Metrics) -> f64 {
    -metrics.avg_episode_reward  // ← Only this is used
}

Validation loss is computed but not passed to optimizer. Hyperopt is NOT affected by temporal leakage.

Q2: Does this affect DQN hyperopt results?

A2: NO, hyperopt is unaffected

• Objective: -avg_episode_reward
• Episode reward: Calculated from trading actions, independent of val data split
• Validation loss: Logged but not used in optimization

Q3: What does validation loss impact?

A3: Early stopping and model checkpointing

1. Plateau detection (line 642): if improvement < 0.001 stops training
2. Best model (line 863): Saves model if val_loss < self.best_val_loss
3. Logging (line 855): Information only, doesn't affect training

Q4: Is this a bug or feature?

A4: Bug with design intent

• Intent: Separate training and validation sets
• Implementation: Sequential 80/20 split (naive approach)
• Bug: Creates temporal leakage (training on past, validating on future)
• Consequence: Inflated validation performance, unrealistic early stopping

Q5: Should we fix it immediately?

A5: NO - Analysis only, fix decision deferred

• Hyperopt is not affected (uses episode reward)
• Validation metrics are informational
• Production impact unknown (backtesting code not reviewed)
• Fix requires careful testing to ensure no regression

---

Recommendations

1. Immediate (No Action)

• ✅ Continue current hyperopt runs
• ✅ DQN hyperopt results are valid
• ✅ PPO dual learning rates production-ready

2. Short Term (1-2 Days)

• 🔍 Investigate PPO and MAMBA-2 validation data sources
• 🔍 Review backtesting code (barrier_backtest.rs) for similar issues
• 📊 Run A/B test: stratified split vs. current split (DQN training)

3. Medium Term (Optional, 4-8 Hours)

• If A/B test shows benefit: Implement Option A (stratified sampling)
• Update DQN, TFT trainers with new split logic
• Retrain models and verify no regression in episode rewards
• Update CLAUDE.md with new findings

4. Analysis Only (Until Decision Made)

• ✅ Do NOT change code
• ✅ Do NOT retrain models
• ✅ Do NOT update hyperparameters
• ⏳ Await decision on Option A implementation

5. Production Deployment

• Use current models (hyperopt is valid)
• Monitor validation loss trends during training
• If early stopping occurs too early, investigate with walk-forward validation

---

Summary for Code Review

Aspect	Finding	Status
Temporal Leakage Exists?	YES (sequential 80/20 split)	✅ Confirmed
Affects Hyperopt?	NO (objective = episode reward)	✅ Confirmed
Affects Early Stopping?	YES (plateau detection)	⚠️ Medium impact
Affects Model Selection?	YES (best val loss)	⚠️ Medium impact
Affects TFT?	YES (more active val usage)	⚠️ Medium impact
Recommended Fix?	Option A (stratified sampling)	📋 Pending decision
Implementation Cost?	1-2 hours per trainer	💰 Low
Rollout Risk?	LOW (hyperopt unaffected)	🟢 Safe

---

References

Code Locations:

• DQN trainer: /home/jgrusewski/Work/foxhunt/ml/src/trainers/dqn.rs
• DQN hyperopt: /home/jgrusewski/Work/foxhunt/ml/src/hyperopt/adapters/dqn.rs
• TFT trainer: /home/jgrusewski/Work/foxhunt/ml/src/trainers/tft_parquet.rs
• MAMBA-2 trainer: /home/jgrusewski/Work/foxhunt/ml/src/trainers/mamba2.rs
• PPO trainer: /home/jgrusewski/Work/foxhunt/ml/src/trainers/ppo.rs
• Backtesting: /home/jgrusewski/Work/foxhunt/ml/src/backtesting/barrier_backtest.rs

Documentation:

• CLAUDE.md: Checkpoint/Resume Investigation section (lines ~390-450)
• ML_TRAINING_PARQUET_GUIDE.md: Data loading documentation

---

Appendix: Technical Deep Dive

Why Hyperopt is Safe (Detailed Explanation)

Hyperopt Trial Flow:

1. DQNTrainer::train_with_data_full_loop()
   ├─ Create training_data from Parquet/DBN
   ├─ Split: train_data (80%), val_data (20%)
   ├─ Loop epochs:
   │  ├─ Phase 1: Collect experiences using training_data
   │  ├─ Phase 2: Train on replay buffer (experiences from Phase 1)
   │  ├─ Phase 3: compute_validation_loss() on val_data
   │  ├─ Phase 4: check_early_stopping() using val_loss_history
   │  └─ Record: episode rewards (from Phase 1), val loss (from Phase 3)
   └─ Return: TrainingMetrics with avg_episode_reward

2. DQNTrainer::extract_objective(metrics)
   └─ return -metrics.avg_episode_reward  // <- Hyperopt uses ONLY this

Why Episode Reward is Unaffected:

• Episode reward = cumulative reward from trading actions
• Action selection: Based on Q-values (deterministic argmax at test time)
• Reward calculation: Based on price changes (independent of train/val split)
• Temporal leakage: Affects VALIDATION loss, NOT episode reward

Example:

Epoch 1:
  Training loss: 2.5 (trains on Jan-Sep data)
  Episode reward: 150 (PnL from actions on training_data)
  Validation loss: 1.2 (validates on Oct-Dec data) ← LEAKAGE HERE

Hyperopt sees:
  avg_episode_reward = 150
  avg_val_loss = 1.2 (not used!)
  → Objective = -150 (to maximize rewards)

Temporal leakage inflates validation loss (1.2 should be higher if past data), but hyperopt optimizer never sees this value.

---

Appendix: Walk-Forward Validation (Detailed)

Why NOT recommended for DQN hyperopt:

Current hyperopt setup:

• Duration: 14.3 minutes
• Trials: 63
• Cost: $0.06
• GPU time per trial: ~13 seconds

Walk-forward with 5 windows:

• Duration: 14.3 × 5 = 71.5 minutes (5 trials × 5 windows)
• Cost: $0.06 × 5 = $0.30
• Per-window validation: More robust but slower

For production model:

• Could use walk-forward (robust evaluation)
• For hyperopt: Too slow (diminishing returns)
• Recommendation: Option A (stratified) is better for hyperopt, walk-forward for final validation

---

End of Analysis Report