foxhunt/WAVE3_A5_HANDOFF.txt

================================================================================
WAVE 3 - AGENT A5 HANDOFF: IMPLEMENT MULTI-OBJECTIVE FUNCTIONS
================================================================================

CURRENT STATUS:
  ✅ Agent A4 Complete: 15 unit tests created (579 lines)
  ⏳ Agent A5 Task: Implement 6 component functions

YOUR TASK:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Implement the 6 component functions for the multi-objective optimization
function. All tests are already written and will guide your implementation.

STEP 1: Extend DQNMetrics (1-2 hours)
───────────────────────────────────────────────────────────────────────────

File: ml/src/hyperopt/adapters/dqn.rs

Add these fields to DQNMetrics struct:
  pub action_distribution: [f64; 3],  // [BUY%, SELL%, HOLD%]
  pub avg_gradient_norm: f64,         // Average gradient norm

Update train_with_params() to populate these fields:
  - Track actions via TrainingMonitor in ml/src/trainers/dqn.rs
  - Track gradient norms during training
  - Calculate percentages and averages

STEP 2: Implement Component Functions (2-3 hours)
───────────────────────────────────────────────────────────────────────────

File: ml/src/hyperopt/adapters/dqn.rs

Function 1: calculate_reward_component(metrics: &DQNMetrics) -> f64
  Formula: -metrics.avg_episode_reward
  Test: test_reward_component_normalization()

Function 2: calculate_diversity_penalty(metrics: &DQNMetrics) -> f64
  Formula:
    entropy = -Σ(p_i * ln(p_i)) for i in [BUY, SELL, HOLD]
    max_entropy = ln(3) ≈ 1.099
    diversity_score = entropy / max_entropy
    penalty = 1.0 - diversity_score
  
  Special case: ln(0) = 0 in entropy calculation
  Tests: 
    - test_diversity_penalty_balanced_actions()
    - test_diversity_penalty_extreme_hold()
    - test_edge_case_zero_entropy()

Function 3: calculate_stability_penalty(metrics: &DQNMetrics) -> f64
  Formula:
    q_penalty = if Q < 0.5 { |0.5 - Q| }
                else if Q > 10.0 { (Q - 10.0) * 0.1 }
                else { 0.0 }
    
    grad_penalty = if grad_norm > 10.0 { (grad_norm - 10.0) * 0.5 }
                   else { 0.0 }
    
    stability_penalty = q_penalty + grad_penalty
  
  Tests:
    - test_stability_penalty_good_qvalues()
    - test_stability_penalty_gradient_explosion()
    - test_edge_case_q_value_boundaries()
    - test_edge_case_gradient_norm_threshold()

Function 4: calculate_completion_penalty(metrics: &DQNMetrics, target: usize) -> f64
  Formula: (target_epochs - epochs_completed) / target_epochs
  Test: test_completion_penalty_early_stop()

Function 5: violates_hard_constraints(metrics: &DQNMetrics) -> bool
  Constraints:
    1. avg_q_value < -10.0 → true (Q-value floor)
    2. train_loss > 1000.0 → true (loss ceiling)
    3. epochs_completed < 20 → true (minimum epochs)
    Otherwise → false
  
  Tests:
    - test_hard_constraint_q_value_floor()
    - test_hard_constraint_training_loss_ceiling()
    - test_hard_constraint_minimum_epochs()

Function 6: calculate_multiobjective(metrics: &DQNMetrics, target: usize) -> f64
  Formula:
    if violates_hard_constraints(metrics) {
        return 1e6;  // Penalty
    }
    
    let reward = calculate_reward_component(metrics);
    let diversity = calculate_diversity_penalty(metrics);
    let stability = calculate_stability_penalty(metrics);
    let completion = calculate_completion_penalty(metrics, target);
    
    0.8 * reward + 0.1 * diversity + 0.05 * stability + 0.05 * completion
  
  Tests:
    - test_multiobjective_optimal_trial()
    - test_multiobjective_broken_trial()
    - test_weight_sensitivity()
    - All 3 hard constraint tests

STEP 3: Replace extract_objective() (1 hour)
───────────────────────────────────────────────────────────────────────────

File: ml/src/hyperopt/adapters/dqn.rs

Replace this code (lines 874-884):
  fn extract_objective(metrics: &Self::Metrics) -> f64 {
      -metrics.avg_episode_reward
  }

With:
  fn extract_objective(metrics: &Self::Metrics) -> f64 {
      calculate_multiobjective(metrics, 50)  // 50 = target epochs
  }

STEP 4: Run Tests (5 minutes)
───────────────────────────────────────────────────────────────────────────

Verify all 15 tests pass:
  cargo test -p ml --test dqn_hyperopt_multiobjective_test

Expected result:
  test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

REFERENCE FILES:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

1. Test File (defines all expected behavior):
   ml/tests/dqn_hyperopt_multiobjective_test.rs

2. Design Document (explains why):
   DQN_HYPEROPT_OBJECTIVE_ANALYSIS.md

3. Test Report (describes tests):
   WAVE3_A4_MULTIOBJECTIVE_TESTS_REPORT.md

4. This Handoff:
   WAVE3_A5_HANDOFF.txt

TIME BUDGET:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  Step 1 (Extend DQNMetrics): 1-2 hours
  Step 2 (Implement functions): 2-3 hours
  Step 3 (Replace extract_objective): 1 hour
  Step 4 (Run tests): 5 minutes
  
  Total: 4-6 hours

SUCCESS CRITERIA:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✅ All 15 tests pass
  ✅ DQNMetrics extended with 2 new fields
  ✅ All 6 component functions implemented
  ✅ extract_objective() replaced with calculate_multiobjective()
  ✅ No compiler warnings (except test dead_code warning)

EXPECTED IMPACT:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

After your implementation, hyperopt will:
  ✅ Reject trials with Q-value collapse (< -10.0)
  ✅ Reject trials with numerical explosion (loss > 1000.0)
  ✅ Reject trials with early stopping (< 20 epochs)
  ✅ Penalize trials with degenerate actions (>95% single action)
  ✅ Reward trials with balanced actions (20-40% per action)
  ✅ Reward trials with healthy Q-values (-10.0 to 10.0)

This will prevent Trial #31-style failures where reward is high but training
is catastrophically unstable.

================================================================================
END OF WAVE 3 AGENT A5 HANDOFF
================================================================================