foxhunt/docs/agent7_missing_getters.md

Agent 7: Missing DQNTrainer Getters Analysis

Required Getters for Overfitting Detection

To build ValidationMetrics history, we need access to per-epoch data from DQNTrainer.

---

Current Status

✅ Existing Getters (verified in trainer.rs)

1. pub fn get_best_val_loss(&self) -> f64 (line 3390)

   • Returns: Single scalar (best validation loss)
   • Issue: We need history not just best value

2. pub fn get_best_epoch(&self) -> usize (line 3397)

   • Returns: Epoch number with best val loss
   • Issue: Not a history vector

3. pub fn get_val_data(&self) -> &[(FeatureVector51, Vec<f64>)] (line 3406)

   • Returns: Validation dataset (raw data, not loss history)
   • Issue: Not loss history

4. pub fn get_agent(&self) -> &Arc<RwLock<DQNAgentType>> (line 3448)

   • Returns: DQN agent reference
   • Usage: Can query current state but not history

---

❌ Missing Getters (need to add)

1. Training Loss History

/// Get training loss per epoch
///
/// Returns vector where `history[i]` is the average training loss
/// for epoch `i` (0-indexed).
///
/// Used by hyperopt adapter to build ValidationMetrics for overfitting detection.
pub fn get_loss_history(&self) -> &[f64] {
    &self.loss_history
}

Backing field: loss_history: Vec<f64> (line 337, already exists) Verification: Exists in struct, just needs public getter

---

2. Validation Loss History

/// Get validation loss per epoch
///
/// Returns vector where `history[i]` is the validation loss
/// for epoch `i` (0-indexed).
///
/// Used by hyperopt adapter to detect train/val divergence (overfitting).
pub fn get_val_loss_history(&self) -> &[f64] {
    &self.val_loss_history
}

Backing field: val_loss_history: Vec<f64> (line 344, already exists) Verification: Exists in struct, just needs public getter

---

3. Q-Value History

/// Get mean Q-value per epoch
///
/// Returns vector where `history[i]` is the average Q-value
/// across all actions for epoch `i` (0-indexed).
///
/// Used by hyperopt adapter to monitor Q-value stability.
pub fn get_q_value_history(&self) -> &[f64] {
    &self.q_value_history
}

Backing field: q_value_history: Vec<f64> (line 338, already exists) Verification: Exists in struct, just needs public getter

---

4. Action Distribution

/// Get final action distribution (BUY%, SELL%, HOLD%)
///
/// Returns the percentage of each action taken during training
/// as of the most recent epoch.
///
/// Used by hyperopt adapter to detect action collapse.
pub fn get_action_distribution(&self) -> (f64, f64, f64) {
    // Get from portfolio tracker or compute from recent episode data
    self.portfolio_tracker.get_action_distribution()
        .unwrap_or((0.33, 0.33, 0.34))  // Default uniform
}

Backing field: Needs computation from portfolio_tracker or new tracking Verification: Need to check if PortfolioTracker has this method

---

5. Final Gradient Norm

/// Get most recent gradient norm
///
/// Returns the L2 norm of gradients from the most recent training step.
/// Used to detect gradient explosion.
///
/// Returns `None` if no gradient has been computed yet.
pub fn get_final_gradient_norm(&self) -> Option<f64> {
    self.last_gradient_norm
}

Backing field: Need to add last_gradient_norm: Option<f64> to struct Verification: Currently not tracked, needs new field

---

6. Q-Value Standard Deviation

/// Get Q-value standard deviation
///
/// Returns the standard deviation of Q-values across all actions
/// from the most recent epoch. Used to detect Q-value instability.
///
/// Returns `None` if not computed yet.
pub fn get_q_value_std(&self) -> Option<f64> {
    self.last_q_value_std
}

Backing field: Need to add last_q_value_std: Option<f64> to struct Verification: Currently not tracked, needs new field

---

Implementation Plan

Phase 1: Add Simple History Getters (5 minutes)

Add to ml/src/trainers/dqn/trainer.rs (around line 3400):

/// Get training loss history per epoch
pub fn get_loss_history(&self) -> &[f64] {
    &self.loss_history
}

/// Get validation loss history per epoch
pub fn get_val_loss_history(&self) -> &[f64] {
    &self.val_loss_history
}

/// Get Q-value history per epoch
pub fn get_q_value_history(&self) -> &[f64] {
    &self.q_value_history
}

Risk: None (just exposing existing private fields)

---

Phase 2: Add Derived Getters (10 minutes)

/// Get final action distribution from portfolio tracker
pub fn get_action_distribution(&self) -> (f64, f64, f64) {
    // Query portfolio tracker for action counts
    let (buy_count, sell_count, hold_count) = self.portfolio_tracker
        .get_action_counts();

    let total = (buy_count + sell_count + hold_count) as f64;
    if total == 0.0 {
        return (0.33, 0.33, 0.34);  // Default uniform
    }

    (
        buy_count as f64 / total,
        sell_count as f64 / total,
        hold_count as f64 / total,
    )
}

Risk: Low (depends on PortfolioTracker API)

---

Phase 3: Add Tracking Fields (15 minutes)

Add to struct (around line 340):

pub struct DQNTrainer {
    // ... existing fields ...

    /// Last computed gradient norm (for gradient explosion detection)
    last_gradient_norm: Option<f64>,

    /// Last computed Q-value std (for Q-value instability detection)
    last_q_value_std: Option<f64>,
}

Update in train loop (around line 1950):

// After computing gradients
let grad_norm = compute_gradient_norm(&gradients);
self.last_gradient_norm = Some(grad_norm);

// After computing Q-values
let q_std = compute_q_value_std(&q_values);
self.last_q_value_std = Some(q_std);

Add getters:

pub fn get_final_gradient_norm(&self) -> Option<f64> {
    self.last_gradient_norm
}

pub fn get_q_value_std(&self) -> Option<f64> {
    self.last_q_value_std
}

Risk: Medium (requires finding where gradients are computed)

---

Verification Checklist

Before Implementation

• Verify loss_history, val_loss_history, q_value_history exist in struct
• Check if PortfolioTracker has get_action_counts() or similar
• Identify where gradients are computed in training loop
• Identify where Q-value statistics are computed

After Implementation

• Run cargo build --package ml (should compile)
• Run cargo test --package ml (all tests should pass)
• Verify getters return expected data types
• Test with hyperopt integration

---

Alternative Approach (If Getters Too Complex)

Use Default Values in Overfitting Detection

Instead of requiring all fields, use defaults where data is unavailable:

// In hyperopt adapter
let action_dist = trainer.get_action_distribution()
    .unwrap_or([0.33, 0.33, 0.34]);  // Default uniform

let gradient_norm = trainer.get_final_gradient_norm()
    .unwrap_or(1.0) as f32;  // Default safe value

let q_value_std = trainer.get_q_value_std()
    .unwrap_or(10.0) as f32;  // Default safe value

Pros: Works immediately without DQNTrainer changes Cons: Less accurate overfitting detection (missing some signals)

---

Recommended Path Forward

Minimal Viable Product (MVP)

Only add Phase 1 getters (3 simple history getters):

• get_loss_history()
• get_val_loss_history()
• get_q_value_history()

Use defaults for:

• Action distribution: [0.33, 0.33, 0.34] (uniform)
• Gradient norm: Final epoch gradient (from logs)
• Q-value std: Estimated from Q-value mean

Rationale:

• Train/val divergence is the primary overfitting signal
• Requires only train_loss and val_loss histories
• Other fields are supplementary (nice to have)
• Can be enhanced in Phase 2

Full Implementation

Add all getters (Phase 1-3):

• Most accurate overfitting detection
• Enables all ValidationMetrics features
• More work upfront but cleaner long-term

---

Summary

✅ Easy Wins (5 minutes)

pub fn get_loss_history(&self) -> &[f64] { &self.loss_history }
pub fn get_val_loss_history(&self) -> &[f64] { &self.val_loss_history }
pub fn get_q_value_history(&self) -> &[f64] { &self.q_value_history }

⚠️ Moderate Effort (10 minutes)

pub fn get_action_distribution(&self) -> (f64, f64, f64) { /* compute */ }

🔧 Requires New Tracking (15 minutes)

pub fn get_final_gradient_norm(&self) -> Option<f64> { /* new field */ }
pub fn get_q_value_std(&self) -> Option<f64> { /* new field */ }

💡 Recommended: Start with Easy Wins

Implement Phase 1 only, use defaults for other fields. Can enhance later.

---

Next Step: Add 3 simple getters to trainer.rs and test overfitting detection.