diff --git a/docs/plans/2026-03-02-additional-training-metrics.md b/docs/plans/2026-03-02-additional-training-metrics.md new file mode 100644 index 000000000..8c8249f6a --- /dev/null +++ b/docs/plans/2026-03-02-additional-training-metrics.md @@ -0,0 +1,875 @@ +# Additional Training Metrics Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Add 17 new Prometheus metrics (12 always-on gauges + 5 opt-in verbose) to provide full RL diagnostics, gradient health, and hyperopt intra-trial visibility across the monitoring pipeline. + +**Architecture:** Two-tier metrics system. Tier 1 (12 gauges) — zero-cost atomic writes in every trainer's epoch loop. Tier 2 (5 metrics) — gated behind `FOXHUNT_VERBOSE_METRICS=1`. All metrics flow through the existing Prometheus → monitoring_service → gRPC → fxt CLI pipeline. No new queries needed (wildcard `foxhunt_training_*|foxhunt_hyperopt_*` already captures new metrics). + +**Tech Stack:** Rust, prometheus crate, tonic/prost (gRPC), protobuf, Candle ML framework + +**Build command:** `SQLX_OFFLINE=true cargo check --workspace` +**Test command:** `SQLX_OFFLINE=true cargo test -p --lib` +**Clippy rules:** `#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::indexing_slicing)]` — use `.get()`, `?`, `.ok_or()` instead. + +--- + +### Task 1: Register Tier 1 metrics + verbose gate in training_metrics.rs + +**Files:** +- Modify: `crates/common/src/metrics/training_metrics.rs` + +This task adds 12 new gauge registrations, 5 verbose metric registrations with env-var gating, and typed helper functions for all 17 new metrics. + +**Context:** The file currently has 24 metrics registered in `init()` (lines 32-168) and typed helpers (lines 174-297). Tests are at lines 299-340. The module uses helper functions from `super::` (`register_gauge_vec`, `set_gauge_vec`, `register_histogram_vec`, `observe_histogram_vec`). Labels use `&["model", "fold"]` for training metrics and `&["model"]` for hyperopt metrics. + +**Step 1: Add the verbose gate** + +At the top of the file, after the `use super::{...}` block (line 23), add: + +```rust +use std::sync::OnceLock; + +static VERBOSE: OnceLock = OnceLock::new(); + +/// Returns true if `FOXHUNT_VERBOSE_METRICS` env var is set. +/// Checked once at first call and cached. +pub fn verbose_enabled() -> bool { + *VERBOSE.get_or_init(|| std::env::var("FOXHUNT_VERBOSE_METRICS").is_ok()) +} +``` + +**Step 2: Register Tier 1 gauges in `init()`** + +After the existing `register_gauge("foxhunt_training_active_workers", ...)` block (line 130) and before the `// Hyperopt gauges` comment (line 132), add these 9 training gauges: + +```rust + // RL diagnostic gauges (model + fold) + _ = register_gauge_vec("foxhunt_training_q_value_mean", "Mean Q-value per epoch (DQN)", mf); + _ = register_gauge_vec("foxhunt_training_q_value_max", "Max Q-value per epoch (DQN)", mf); + _ = register_gauge_vec("foxhunt_training_policy_entropy", "Policy entropy (PPO)", mf); + _ = register_gauge_vec("foxhunt_training_kl_divergence", "KL divergence old vs new policy (PPO)", mf); + _ = register_gauge_vec("foxhunt_training_advantage_mean", "Mean advantage estimate (PPO)", mf); + _ = register_gauge_vec("foxhunt_training_replay_buffer_size", "Replay buffer occupancy (DQN)", mf); + + // Gradient & training health gauges (model + fold) + _ = register_gauge_vec("foxhunt_training_gradient_norm", "L2 gradient norm after backward", mf); + _ = register_gauge_vec("foxhunt_training_learning_rate", "Effective learning rate", mf); + _ = register_gauge_vec("foxhunt_training_epoch_duration_seconds", "Wall-clock time per epoch", mf); +``` + +After the existing `register_gauge_vec("foxhunt_hyperopt_mode", ...)` block (line 152) and before the `// Hyperopt counters` comment (line 154), add these 3 hyperopt gauges: + +```rust + // Hyperopt intra-trial gauges (model only) + _ = register_gauge_vec("foxhunt_hyperopt_trial_epoch", "Current epoch within active trial", m); + _ = register_gauge_vec("foxhunt_hyperopt_trial_best_loss", "Best loss in current trial", m); + _ = register_gauge_vec("foxhunt_hyperopt_elapsed_seconds", "Total wall-clock time since hyperopt start", m); +``` + +**Step 3: Register Tier 2 verbose metrics in `init()`** + +At the end of `init()`, before the closing `}`, add: + +```rust + // Tier 2: Verbose metrics (only registered when FOXHUNT_VERBOSE_METRICS is set) + if verbose_enabled() { + let mfl: &[&str] = &["model", "fold", "layer"]; + _ = register_histogram_vec( + "foxhunt_training_batch_loss", + "Per-batch loss distribution", + mf, + vec![0.001, 0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0], + ); + _ = register_histogram_vec( + "foxhunt_training_gradient_norm_per_layer", + "Per-layer gradient L2 norm", + mfl, + vec![0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0], + ); + _ = register_gauge_vec("foxhunt_training_replay_priority_mean", "Mean PER priority", mf); + _ = register_gauge_vec("foxhunt_training_advantage_std", "Advantage standard deviation (PPO)", mf); + _ = register_gauge_vec("foxhunt_training_value_explained_variance", "Value function explained variance (PPO)", mf); + } +``` + +**Step 4: Add typed helpers for Tier 1** + +After the existing `set_hyperopt_mode` helper (line 297), add: + +```rust +// --------------------------------------------------------------------------- +// Tier 1: RL diagnostics +// --------------------------------------------------------------------------- + +pub fn set_q_value_stats(model: &str, fold: &str, mean: f64, max: f64) { + set_gauge_vec("foxhunt_training_q_value_mean", &[model, fold], mean); + set_gauge_vec("foxhunt_training_q_value_max", &[model, fold], max); +} + +pub fn set_policy_entropy(model: &str, fold: &str, entropy: f64) { + set_gauge_vec("foxhunt_training_policy_entropy", &[model, fold], entropy); +} + +pub fn set_kl_divergence(model: &str, fold: &str, kl: f64) { + set_gauge_vec("foxhunt_training_kl_divergence", &[model, fold], kl); +} + +pub fn set_advantage_mean(model: &str, fold: &str, mean: f64) { + set_gauge_vec("foxhunt_training_advantage_mean", &[model, fold], mean); +} + +pub fn set_replay_buffer_size(model: &str, fold: &str, size: f64) { + set_gauge_vec("foxhunt_training_replay_buffer_size", &[model, fold], size); +} + +// --------------------------------------------------------------------------- +// Tier 1: Gradient & training health +// --------------------------------------------------------------------------- + +pub fn set_gradient_norm(model: &str, fold: &str, norm: f64) { + set_gauge_vec("foxhunt_training_gradient_norm", &[model, fold], norm); +} + +pub fn set_learning_rate(model: &str, fold: &str, lr: f64) { + set_gauge_vec("foxhunt_training_learning_rate", &[model, fold], lr); +} + +pub fn set_epoch_duration(model: &str, fold: &str, secs: f64) { + set_gauge_vec("foxhunt_training_epoch_duration_seconds", &[model, fold], secs); +} + +// --------------------------------------------------------------------------- +// Tier 1: Hyperopt intra-trial +// --------------------------------------------------------------------------- + +pub fn set_hyperopt_trial_epoch(model: &str, epoch: f64) { + set_gauge_vec("foxhunt_hyperopt_trial_epoch", &[model], epoch); +} + +pub fn set_hyperopt_trial_best_loss(model: &str, loss: f64) { + set_gauge_vec("foxhunt_hyperopt_trial_best_loss", &[model], loss); +} + +pub fn set_hyperopt_elapsed(model: &str, secs: f64) { + set_gauge_vec("foxhunt_hyperopt_elapsed_seconds", &[model], secs); +} + +// --------------------------------------------------------------------------- +// Tier 2: Verbose metrics (no-op when FOXHUNT_VERBOSE_METRICS not set) +// --------------------------------------------------------------------------- + +pub fn observe_batch_loss(model: &str, fold: &str, loss: f64) { + if verbose_enabled() { + observe_histogram_vec("foxhunt_training_batch_loss", &[model, fold], loss); + } +} + +pub fn observe_gradient_norm_per_layer(model: &str, fold: &str, layer: &str, norm: f64) { + if verbose_enabled() { + observe_histogram_vec("foxhunt_training_gradient_norm_per_layer", &[model, fold, layer], norm); + } +} + +pub fn set_replay_priority_mean(model: &str, fold: &str, mean: f64) { + if verbose_enabled() { + set_gauge_vec("foxhunt_training_replay_priority_mean", &[model, fold], mean); + } +} + +pub fn set_advantage_std(model: &str, fold: &str, std: f64) { + if verbose_enabled() { + set_gauge_vec("foxhunt_training_advantage_std", &[model, fold], std); + } +} + +pub fn set_value_explained_variance(model: &str, fold: &str, ev: f64) { + if verbose_enabled() { + set_gauge_vec("foxhunt_training_value_explained_variance", &[model, fold], ev); + } +} +``` + +**Step 5: Add tests** + +Add these tests inside the existing `#[cfg(test)] mod tests` block, after the last test: + +```rust + #[test] + fn test_verbose_disabled_by_default() { + // FOXHUNT_VERBOSE_METRICS not set in test env + // verbose helpers should not panic even when metrics aren't registered + init(); + // These are no-ops when verbose is disabled + observe_batch_loss("dqn", "0", 0.05); + observe_gradient_norm_per_layer("dqn", "0", "fc1", 0.01); + set_replay_priority_mean("dqn", "0", 1.5); + set_advantage_std("ppo", "0", 0.8); + set_value_explained_variance("ppo", "0", 0.7); + } + + #[test] + fn test_tier1_helpers_no_panic() { + init(); + set_q_value_stats("dqn", "0", 12.3, 45.6); + set_policy_entropy("ppo", "0", 1.23); + set_kl_divergence("ppo", "0", 0.008); + set_advantage_mean("ppo", "0", 0.001); + set_replay_buffer_size("dqn", "0", 50000.0); + set_gradient_norm("dqn", "0", 1.23); + set_learning_rate("dqn", "0", 0.001); + set_epoch_duration("dqn", "0", 12.5); + set_hyperopt_trial_epoch("dqn", 5.0); + set_hyperopt_trial_best_loss("dqn", 0.042); + set_hyperopt_elapsed("dqn", 123.4); + } +``` + +**Step 6: Run tests** + +Run: `SQLX_OFFLINE=true cargo test -p common --lib -- training_metrics` +Expected: All tests pass, including the 2 new ones. + +**Step 7: Clippy check** + +Run: `SQLX_OFFLINE=true cargo clippy -p common -- -D warnings` +Expected: 0 errors, 0 warnings. + +**Step 8: Commit** + +```bash +git add crates/common/src/metrics/training_metrics.rs +git commit -m "feat(metrics): register 17 new Prometheus metrics with verbose gating" +``` + +--- + +### Task 2: Extend monitoring.proto with new TrainingSession fields + +**Files:** +- Modify: `services/monitoring_service/proto/monitoring.proto` + +**Context:** `TrainingSession` currently has fields 1-23. The design doc assigns field numbers 24-35 for new fields. Note: the existing hyperopt fields already use numbers 20-23, so the new fields start at 24. + +**Step 1: Add new fields to TrainingSession** + +After the `uint32 hyperopt_trials_failed = 23;` line (line 65), add: + +```protobuf + + // RL diagnostics + float q_value_mean = 24; + float q_value_max = 25; + float policy_entropy = 26; + float kl_divergence = 27; + float advantage_mean = 28; + uint32 replay_buffer_size = 29; + + // Gradient & training health + float gradient_norm = 30; + float learning_rate = 31; + float epoch_duration_seconds = 32; + + // Hyperopt intra-trial + uint32 hyperopt_trial_epoch = 33; + float hyperopt_trial_best_loss = 34; + float hyperopt_elapsed_seconds = 35; +``` + +**Step 2: Verify proto compiles** + +Run: `SQLX_OFFLINE=true cargo check -p monitoring_service` +Expected: Compiles (prost generates the new fields as `f32`/`u32` with default 0). + +Run: `SQLX_OFFLINE=true cargo check -p fxt` +Expected: Compiles (fxt also compiles this proto). + +**Step 3: Commit** + +```bash +git add services/monitoring_service/proto/monitoring.proto +git commit -m "proto(monitoring): add 12 new TrainingSession fields for diagnostics" +``` + +--- + +### Task 3: Wire new metrics through monitoring_service group_into_sessions + +**Files:** +- Modify: `services/monitoring_service/src/service.rs` + +**Context:** The `group_into_sessions` function (line 98) has a `match s.name.as_str()` block (lines 117-158) that maps Prometheus metric names to TrainingSession proto fields. We add 12 new match arms. + +**Step 1: Add match arms** + +Inside the `match s.name.as_str()` block, after the `"foxhunt_hyperopt_mode"` arm (line 153-157) and before the `_ => {}` wildcard (line 158), add: + +```rust + // RL diagnostics + "foxhunt_training_q_value_mean" => session.q_value_mean = s.value as f32, + "foxhunt_training_q_value_max" => session.q_value_max = s.value as f32, + "foxhunt_training_policy_entropy" => session.policy_entropy = s.value as f32, + "foxhunt_training_kl_divergence" => session.kl_divergence = s.value as f32, + "foxhunt_training_advantage_mean" => session.advantage_mean = s.value as f32, + "foxhunt_training_replay_buffer_size" => { + session.replay_buffer_size = s.value as u32; + } + // Gradient & training health + "foxhunt_training_gradient_norm" => session.gradient_norm = s.value as f32, + "foxhunt_training_learning_rate" => session.learning_rate = s.value as f32, + "foxhunt_training_epoch_duration_seconds" => { + session.epoch_duration_seconds = s.value as f32; + } + // Hyperopt intra-trial + "foxhunt_hyperopt_trial_epoch" => session.hyperopt_trial_epoch = s.value as u32, + "foxhunt_hyperopt_trial_best_loss" => { + session.hyperopt_trial_best_loss = s.value as f32; + } + "foxhunt_hyperopt_elapsed_seconds" => { + session.hyperopt_elapsed_seconds = s.value as f32; + } +``` + +**Step 2: Add tests for new metric mapping** + +Add a new test in the `#[cfg(test)] mod tests` block: + +```rust + #[test] + fn test_group_new_diagnostic_metrics() { + let samples = vec![ + MetricSample { + name: "foxhunt_training_current_epoch".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 25.0, + }, + MetricSample { + name: "foxhunt_training_q_value_mean".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 12.3, + }, + MetricSample { + name: "foxhunt_training_q_value_max".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 45.6, + }, + MetricSample { + name: "foxhunt_training_gradient_norm".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 1.23, + }, + MetricSample { + name: "foxhunt_training_learning_rate".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 0.001, + }, + MetricSample { + name: "foxhunt_training_epoch_duration_seconds".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 12.5, + }, + MetricSample { + name: "foxhunt_training_replay_buffer_size".to_owned(), + model: "dqn".to_owned(), + fold: "0".to_owned(), + value: 50000.0, + }, + ]; + let sessions = group_into_sessions(&samples, ""); + assert_eq!(sessions.len(), 1); + let s = &sessions[0]; + assert_eq!(s.current_epoch, 25.0); + assert!((s.q_value_mean - 12.3).abs() < 0.01); + assert!((s.q_value_max - 45.6).abs() < 0.01); + assert!((s.gradient_norm - 1.23).abs() < 0.01); + assert!((s.learning_rate - 0.001).abs() < 0.0001); + assert!((s.epoch_duration_seconds - 12.5).abs() < 0.01); + assert_eq!(s.replay_buffer_size, 50000); + } + + #[test] + fn test_group_ppo_diagnostics() { + let samples = vec![ + MetricSample { + name: "foxhunt_training_policy_entropy".to_owned(), + model: "ppo".to_owned(), + fold: "0".to_owned(), + value: 1.23, + }, + MetricSample { + name: "foxhunt_training_kl_divergence".to_owned(), + model: "ppo".to_owned(), + fold: "0".to_owned(), + value: 0.008, + }, + MetricSample { + name: "foxhunt_training_advantage_mean".to_owned(), + model: "ppo".to_owned(), + fold: "0".to_owned(), + value: 0.001, + }, + ]; + let sessions = group_into_sessions(&samples, ""); + assert_eq!(sessions.len(), 1); + let s = &sessions[0]; + assert!((s.policy_entropy - 1.23).abs() < 0.01); + assert!((s.kl_divergence - 0.008).abs() < 0.001); + assert!((s.advantage_mean - 0.001).abs() < 0.001); + } + + #[test] + fn test_group_hyperopt_intra_trial() { + let samples = vec![ + MetricSample { + name: "foxhunt_hyperopt_mode".to_owned(), + model: "dqn".to_owned(), + fold: "".to_owned(), + value: 1.0, + }, + MetricSample { + name: "foxhunt_hyperopt_trial_epoch".to_owned(), + model: "dqn".to_owned(), + fold: "".to_owned(), + value: 12.0, + }, + MetricSample { + name: "foxhunt_hyperopt_trial_best_loss".to_owned(), + model: "dqn".to_owned(), + fold: "".to_owned(), + value: 0.042, + }, + MetricSample { + name: "foxhunt_hyperopt_elapsed_seconds".to_owned(), + model: "dqn".to_owned(), + fold: "".to_owned(), + value: 123.4, + }, + ]; + let sessions = group_into_sessions(&samples, ""); + assert_eq!(sessions.len(), 1); + let s = &sessions[0]; + assert!(s.is_hyperopt); + assert_eq!(s.hyperopt_trial_epoch, 12); + assert!((s.hyperopt_trial_best_loss - 0.042).abs() < 0.001); + assert!((s.hyperopt_elapsed_seconds - 123.4).abs() < 0.1); + } +``` + +**Step 3: Run tests** + +Run: `SQLX_OFFLINE=true cargo test -p monitoring_service --lib` +Expected: All 7 tests pass (4 existing + 3 new). + +**Step 4: Commit** + +```bash +git add services/monitoring_service/src/service.rs +git commit -m "feat(monitoring): wire 12 new metric→proto field mappings in group_into_sessions" +``` + +--- + +### Task 4: Enhance fxt monitor CLI display + +**Files:** +- Modify: `bin/fxt/src/commands/train/monitor.rs` + +**Context:** The CLI has `print_session_table` (line 127) for the main table, `print_hyperopt_summary` (line 154) for hyperopt, and `print_health_summary` (line 172) for health counters. We add gradient_norm to the main table, an RL diagnostics section, and inline hyperopt trial progress. + +**Step 1: Add gradient norm column to main table** + +Replace the `print_session_table` function (lines 127-152) with: + +```rust +fn print_session_table(sessions: &[TrainingSession]) { + println!( + "{:<10} {:<6} {:<7} {:<10} {:<10} {:<9} {:<10} {:<10}", + "Model".bright_cyan(), + "Fold".bright_cyan(), + "Epoch".bright_cyan(), + "Loss".bright_cyan(), + "Val Loss".bright_cyan(), + "Batch/s".bright_cyan(), + "Grad Norm".bright_cyan(), + "Eval Acc".bright_cyan(), + ); + println!("{}", "-".repeat(82).bright_black()); + + for s in sessions { + let acc = if s.eval_accuracy > 0.0 { + format!("{:.1}%", s.eval_accuracy * 100.0) + } else { + "-".to_owned() + }; + let grad = if s.gradient_norm > 0.0 { + format!("{:.4}", s.gradient_norm) + } else { + "-".to_owned() + }; + println!( + "{:<10} {:<6} {:<7.0} {:<10.4} {:<10.4} {:<9.1} {:<10} {:<10}", + s.model, s.fold, s.current_epoch, s.epoch_loss, s.validation_loss, + s.batches_per_second, grad, acc, + ); + } +} +``` + +**Step 2: Add RL diagnostics section** + +After `print_session_table`, add a new function: + +```rust +fn print_rl_diagnostics(sessions: &[TrainingSession]) { + let rl_sessions: Vec<_> = sessions + .iter() + .filter(|s| s.q_value_mean != 0.0 || s.policy_entropy != 0.0) + .collect(); + if rl_sessions.is_empty() { + return; + } + println!(); + println!("{}", "RL Diagnostics:".bright_cyan()); + for s in &rl_sessions { + if s.q_value_mean != 0.0 || s.q_value_max != 0.0 { + // DQN-style + println!( + " {}: Q-mean={:.2} Q-max={:.2} | buffer={}", + s.model.bright_white(), + s.q_value_mean, + s.q_value_max, + s.replay_buffer_size, + ); + } + if s.policy_entropy != 0.0 || s.kl_divergence != 0.0 { + // PPO-style + println!( + " {}: entropy={:.3} KL={:.4} adv-mean={:.4}", + s.model.bright_white(), + s.policy_entropy, + s.kl_divergence, + s.advantage_mean, + ); + } + } +} +``` + +**Step 3: Enhance hyperopt summary with intra-trial progress** + +Replace the `print_hyperopt_summary` function (lines 154-169) with: + +```rust +fn print_hyperopt_summary(sessions: &[TrainingSession]) { + let hyperopt: Vec<_> = sessions.iter().filter(|s| s.is_hyperopt).collect(); + if hyperopt.is_empty() { + return; + } + println!(); + for s in &hyperopt { + let trial_detail = if s.hyperopt_trial_epoch > 0 { + format!( + " (epoch {}, loss {:.4})", + s.hyperopt_trial_epoch, s.hyperopt_trial_best_loss + ) + } else { + String::new() + }; + let elapsed = if s.hyperopt_elapsed_seconds > 0.0 { + format!(" | {:.0}s elapsed", s.hyperopt_elapsed_seconds) + } else { + String::new() + }; + println!( + "Hyperopt ({}): trial {}/{}{} | best Sharpe {:.2} | {} failures{}", + s.model.bright_magenta(), + s.hyperopt_trial_current, + s.hyperopt_trial_total, + trial_detail, + s.hyperopt_best_objective, + s.hyperopt_trials_failed, + elapsed, + ); + } +} +``` + +**Step 4: Wire RL diagnostics into render_snapshot** + +In `render_snapshot` (line 81), add a call to `print_rl_diagnostics` between `print_session_table` and `print_hyperopt_summary`: + +Change lines 100-102 from: +```rust + print_session_table(&resp.sessions); + print_hyperopt_summary(&resp.sessions); + print_health_summary(&resp.sessions); +``` +to: +```rust + print_session_table(&resp.sessions); + print_rl_diagnostics(&resp.sessions); + print_hyperopt_summary(&resp.sessions); + print_health_summary(&resp.sessions); +``` + +**Step 5: Verify compilation** + +Run: `SQLX_OFFLINE=true cargo check -p fxt` +Expected: Compiles cleanly. + +**Step 6: Commit** + +```bash +git add bin/fxt/src/commands/train/monitor.rs +git commit -m "feat(fxt): add grad norm, RL diagnostics, and hyperopt trial detail to monitor" +``` + +--- + +### Task 5: Record new metrics in DQN trainer + +**Files:** +- Modify: `crates/ml/src/trainers/dqn/trainer.rs` + +**Context:** The DQN trainer already has `use common::metrics::training_metrics;` and records 4 metrics at line 2653. The epoch loop computes `avg_q_value` and `avg_grad_norm` at line 2473. Q-value stats (min, max, mean) are available at line 2580 via `monitor.get_q_value_stats()`. Buffer size is available via `agent.get_replay_buffer_size()` (line 993). + +**Step 1: Add Tier 1 metrics after existing per-epoch recording** + +After the existing `training_metrics::set_batches_per_second(...)` block (~line 2661), add: + +```rust + // Tier 1: RL diagnostics + let (q_min, q_max, q_mean) = monitor.get_q_value_stats(); + training_metrics::set_q_value_stats("dqn", "current", q_mean as f64, q_max as f64); + training_metrics::set_gradient_norm("dqn", "current", avg_grad_norm); + training_metrics::set_epoch_duration("dqn", "current", epoch_duration.as_secs_f64()); + { + let agent = self.agent.read().await; + if let Ok(buf_size) = agent.get_replay_buffer_size() { + training_metrics::set_replay_buffer_size("dqn", "current", buf_size as f64); + } + } +``` + +Note: `q_min`, `q_max`, `q_mean` are already used at line 2580. You need to move/duplicate the call here. The `monitor.get_q_value_stats()` call at line 2580 already exists — use the variables from there or call it again. The simplest approach is to add the metrics recording right after the existing `let (q_min, q_max, q_mean) = monitor.get_q_value_stats();` at line 2580. + +Actually, the cleanest approach: add metrics recording right after line 2580 where `(q_min, q_max, q_mean)` is already bound: + +```rust + // After: let (q_min, q_max, q_mean) = monitor.get_q_value_stats(); + training_metrics::set_q_value_stats("dqn", "current", q_mean as f64, q_max as f64); + training_metrics::set_gradient_norm("dqn", "current", avg_grad_norm); + training_metrics::set_epoch_duration("dqn", "current", epoch_duration.as_secs_f64()); + { + let agent = self.agent.read().await; + if let Ok(buf_size) = agent.get_replay_buffer_size() { + training_metrics::set_replay_buffer_size("dqn", "current", buf_size as f64); + } + } +``` + +Note: `_ = q_min;` may be needed to suppress unused warning if `q_min` was only used in the info log. Check — if `q_min` is already used in the `info!()` macro at line 2582, no suppression needed. + +**Step 2: Verify compilation** + +Run: `SQLX_OFFLINE=true cargo check -p ml` +Expected: Compiles. Check for unused variable warnings. + +**Step 3: Commit** + +```bash +git add crates/ml/src/trainers/dqn/trainer.rs +git commit -m "feat(dqn): record Q-value, gradient norm, epoch duration, buffer size metrics" +``` + +--- + +### Task 6: Record new metrics in PPO trainer + +**Files:** +- Modify: `crates/ml/src/trainers/ppo.rs` + +**Context:** PPO trainer already records 3 metrics at line 668. The epoch loop computes `kl_divergence`, `explained_variance`, `mean_reward`, `std_reward`, `entropy` at line 649 via `self.compute_metrics(...)`. The `PpoEpochMetrics` struct (line 196) has all these fields. + +**Step 1: Add metrics after existing recording** + +After the existing `training_metrics::set_validation_loss(...)` at line 670, add: + +```rust + // Tier 1: PPO diagnostics + training_metrics::set_policy_entropy("ppo", "current", entropy as f64); + training_metrics::set_kl_divergence("ppo", "current", kl_divergence as f64); + training_metrics::set_advantage_mean("ppo", "current", mean_reward as f64); + + // Tier 2: Verbose PPO metrics (no-op when disabled) + training_metrics::set_advantage_std("ppo", "current", std_reward as f64); + training_metrics::set_value_explained_variance("ppo", "current", explained_variance as f64); +``` + +Note: The variables `entropy`, `kl_divergence`, `mean_reward`, `std_reward`, `explained_variance` are computed at line 649 and available in scope. + +**Step 2: Add gradient norm and epoch duration** + +The PPO trainer doesn't currently track gradient norms per epoch. We need to find where the optimizer step happens. Search for the backward/optimizer step in the PPO update logic. If a gradient norm is available after the optimizer step, record it. If not, skip this for PPO (the gradient_norm metric will only be populated for DQN and supervised models). + +Check if the PPO update at `self.ppo_update_step(...)` or similar returns a gradient norm. If not, we only add epoch_duration: + +After the metrics block, add: + +```rust + // Epoch duration (epoch_start must be captured at top of loop) + // Note: PPO epochs are measured by the progress_callback timing +``` + +Actually — looking at the PPO trainer structure, the epoch loop at line 492 iterates over `0..self.hyperparams.epochs`. We need to add timing. At the top of the epoch loop body (after `for epoch in 0..self.hyperparams.epochs {`), add: + +```rust + let epoch_start = std::time::Instant::now(); +``` + +Then after the metrics block: + +```rust + training_metrics::set_epoch_duration("ppo", "current", epoch_start.elapsed().as_secs_f64()); +``` + +**Step 3: Verify compilation** + +Run: `SQLX_OFFLINE=true cargo check -p ml` +Expected: Compiles. + +**Step 4: Commit** + +```bash +git add crates/ml/src/trainers/ppo.rs +git commit -m "feat(ppo): record entropy, KL, advantage, explained variance, epoch duration metrics" +``` + +--- + +### Task 7: Record new metrics in supervised training binary + +**Files:** +- Modify: `crates/ml/examples/train_baseline_supervised.rs` + +**Context:** The supervised binary already records epoch/loss/val_loss/iteration_seconds at lines 564-567. The `adapter.get_learning_rate()` value is printed at line 576 but not recorded as a metric. Epoch timing is already measured at line 554 (`epoch_start`). + +**Step 1: Add learning rate and epoch duration metrics** + +After the existing metrics block (line 567), add: + +```rust + metrics::set_learning_rate(model_name, &fold_str, adapter.get_learning_rate()); + metrics::set_epoch_duration(model_name, &fold_str, elapsed.as_secs_f64()); +``` + +Note: `metrics` is the alias for `training_metrics` (imported as `use common::metrics::training_metrics as metrics;` at line 46). `elapsed` is computed at line 561. `adapter.get_learning_rate()` returns `f64`. + +**Step 2: Verify compilation** + +Run: `SQLX_OFFLINE=true cargo check --example train_baseline_supervised` +Expected: Compiles. (Note: this example may need the full `cargo check -p ml` due to workspace deps.) + +Actually, the correct command is: +Run: `SQLX_OFFLINE=true cargo check -p ml --example train_baseline_supervised` +Expected: Compiles. + +**Step 3: Commit** + +```bash +git add crates/ml/examples/train_baseline_supervised.rs +git commit -m "feat(supervised): record learning rate and epoch duration metrics" +``` + +--- + +### Task 8: Record hyperopt intra-trial metrics in both hyperopt binaries + +**Files:** +- Modify: `crates/ml/examples/hyperopt_baseline_rl.rs` +- Modify: `crates/ml/examples/hyperopt_baseline_supervised.rs` + +**Context:** Both binaries already have `use common::metrics::training_metrics;` and record hyperopt trial/best_objective. They each have a `let start = Instant::now();` at the beginning of each model's optimization function. The `elapsed` is computed after optimization completes. We need to add `elapsed_seconds` recording after optimization completes, before returning results. + +**Step 1: Add elapsed_seconds to hyperopt_baseline_rl.rs** + +In `run_dqn_hyperopt()`, after `training_metrics::set_hyperopt_best_objective("dqn", result.best_objective);` (~line 185), add: + +```rust + training_metrics::set_hyperopt_elapsed("dqn", elapsed); +``` + +In `run_ppo_hyperopt()`, after `training_metrics::set_hyperopt_best_objective("ppo", result.best_objective);` (~line 254), add: + +```rust + training_metrics::set_hyperopt_elapsed("ppo", elapsed); +``` + +**Step 2: Add elapsed_seconds to hyperopt_baseline_supervised.rs** + +The supervised hyperopt binary has 8 model functions (TFT, Mamba2, Liquid, TGGN, TLOB, KAN, xLSTM, Diffusion). In each function, after the `training_metrics::set_hyperopt_best_objective(MODEL, result.best_objective);` line, add: + +```rust + training_metrics::set_hyperopt_elapsed(MODEL, elapsed); +``` + +Where MODEL is the lowercase model name string ("tft", "mamba2", "liquid", "tggn", "tlob", "kan", "xlstm", "diffusion"). + +Specific locations: +- `run_tft_hyperopt()`: after line 150 +- `run_mamba2_hyperopt()`: after line 208 +- `run_liquid_hyperopt()`: after line 267 +- `run_tggn_hyperopt()`: after line 326 +- `run_tlob_hyperopt()`: after line 385 +- `run_kan_hyperopt()`: after line 444 +- `run_xlstm_hyperopt()`: after line 503 +- `run_diffusion_hyperopt()`: after line 562 + +**Step 3: Verify compilation** + +Run: `SQLX_OFFLINE=true cargo check -p ml --example hyperopt_baseline_rl` +Run: `SQLX_OFFLINE=true cargo check -p ml --example hyperopt_baseline_supervised` +Expected: Both compile. + +**Step 4: Commit** + +```bash +git add crates/ml/examples/hyperopt_baseline_rl.rs crates/ml/examples/hyperopt_baseline_supervised.rs +git commit -m "feat(hyperopt): record elapsed_seconds metric in all hyperopt binaries" +``` + +--- + +### Task 9: Full workspace check + clippy + +**Files:** None (verification only) + +**Step 1: Full workspace check** + +Run: `SQLX_OFFLINE=true cargo check --workspace` +Expected: 0 errors. + +**Step 2: Clippy** + +Run: `SQLX_OFFLINE=true cargo clippy --workspace -- -D warnings` +Expected: 0 errors, 0 warnings. + +**Step 3: Run all affected tests** + +Run: `SQLX_OFFLINE=true cargo test -p common --lib -- training_metrics` +Run: `SQLX_OFFLINE=true cargo test -p monitoring_service --lib` +Expected: All tests pass. + +**Step 4: Commit (if any fixes needed)** + +```bash +git commit -am "fix: resolve clippy/test issues from metrics additions" +``` + +Only create this commit if there were issues to fix. If everything passes, skip this step.