# Additional Training Metrics Design

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add 17 new Prometheus metrics (12 always-on gauges + 5 opt-in verbose) to provide full training diagnostics and production monitoring visibility.

**Architecture:** Two-tier metrics system. Tier 1 metrics are zero-cost atomic gauge writes recorded in every trainer's epoch loop. Tier 2 metrics (histograms and per-layer stats) are gated behind `FOXHUNT_VERBOSE_METRICS=1` and disabled by default. All metrics flow through the existing Prometheus → monitoring_service → gRPC → fxt CLI pipeline.

---

## Current State

- 24 Prometheus metrics defined in `crates/common/src/metrics/training_metrics.rs`
- ~20 actively recorded across 5 training binaries
- Monitoring service queries all `foxhunt_training_*` and `foxhunt_hyperopt_*` via wildcard
- Proto has TrainingSession (19 fields) and GpuSnapshot (5 fields)

## Tier 1: Always-On Gauges (12 new)

### RL Diagnostics (6)

| Metric Name | Labels | Source | Purpose |
|---|---|---|---|
| `foxhunt_training_q_value_mean` | model, fold | DQN epoch loop | Detect value overestimation |
| `foxhunt_training_q_value_max` | model, fold | DQN epoch loop | Detect divergence (exploding Q-values) |
| `foxhunt_training_policy_entropy` | model, fold | PPO epoch loop | Detect premature convergence (entropy → 0) |
| `foxhunt_training_kl_divergence` | model, fold | PPO epoch loop | Detect unstable policy updates |
| `foxhunt_training_advantage_mean` | model, fold | PPO epoch loop | Detect advantage bias (should be ~0) |
| `foxhunt_training_replay_buffer_size` | model, fold | DQN epoch loop | Monitor replay buffer occupancy |

### Gradient & Training Health (3)

| Metric Name | Labels | Source | Purpose |
|---|---|---|---|
| `foxhunt_training_gradient_norm` | model, fold | All trainers post-backward | Trend before gradient explosion |
| `foxhunt_training_learning_rate` | model, fold | All trainers per epoch | Correlate LR schedule with loss plateaus |
| `foxhunt_training_epoch_duration_seconds` | model, fold | All trainers | Detect performance regressions |

### Hyperopt Intra-Trial Progress (3)

| Metric Name | Labels | Source | Purpose |
|---|---|---|---|
| `foxhunt_hyperopt_trial_epoch` | model | Hyperopt adapters | Current epoch within active trial |
| `foxhunt_hyperopt_trial_best_loss` | model | Hyperopt adapters | Best loss in current trial |
| `foxhunt_hyperopt_elapsed_seconds` | model | Hyperopt binaries | Wall-clock time since start |

## Tier 2: Opt-In Verbose Metrics (5 new, `FOXHUNT_VERBOSE_METRICS=1`)

| Metric Name | Type | Labels | Purpose |
|---|---|---|---|
| `foxhunt_training_batch_loss` | Histogram | model, fold | Per-batch loss distribution |
| `foxhunt_training_gradient_norm_per_layer` | Histogram | model, fold, layer | Per-layer gradient health |
| `foxhunt_training_replay_priority_mean` | Gauge | model, fold | PER staleness detection |
| `foxhunt_training_advantage_std` | Gauge | model, fold | Advantage normalization quality |
| `foxhunt_training_value_explained_variance` | Gauge | model, fold | Value function prediction quality |

### Gating Mechanism

```rust
// In training_metrics.rs
static VERBOSE: OnceLock<bool> = OnceLock::new();

pub fn verbose_enabled() -> bool {
    *VERBOSE.get_or_init(|| std::env::var("FOXHUNT_VERBOSE_METRICS").is_ok())
}
```

Recording calls for Tier 2 metrics check `verbose_enabled()` and return early if disabled.

## Proto Changes (monitoring.proto)

Add to `TrainingSession`:

```protobuf
// RL diagnostics
float q_value_mean = 20;
float q_value_max = 21;
float policy_entropy = 22;
float kl_divergence = 23;
float advantage_mean = 24;
uint32 replay_buffer_size = 25;

// Gradient & training health
float gradient_norm = 26;
float learning_rate = 27;
float epoch_duration_seconds = 28;

// Hyperopt intra-trial
uint32 hyperopt_trial_epoch = 29;
float hyperopt_trial_best_loss = 30;
float hyperopt_elapsed_seconds = 31;
```

## Service Changes

### monitoring_service/src/service.rs

Extend `group_into_sessions` match arms for all 12 new metric names → proto fields. No new Prometheus queries needed (wildcard already covers `foxhunt_training_*` and `foxhunt_hyperopt_*`).

### monitoring_service/src/prometheus_client.rs

No changes needed.

## CLI Changes (fxt train monitor)

### Main table: add gradient_norm column

```
Model   Fold  Epoch  Loss     Val Loss  Batch/s  Grad Norm  Eval Acc
----------------------------------------------------------------------
dqn     0     25     0.0234   0.0312    148.2    1.23       -
ppo     0     12     0.0156   0.0089    92.1     0.87       -
```

### RL diagnostics section (only when non-zero)

```
RL Diagnostics:
  dqn: Q-mean=12.3 Q-max=45.6 | buffer=50000/100000
  ppo: entropy=1.23 KL=0.008 adv-mean=0.001
```

### Hyperopt intra-trial (inline enhancement)

```
Hyperopt (dqn): trial 3/20 (epoch 12, loss 0.042) | best Sharpe 1.23 | 0 failures
```

## Existing Gap Fills

Wire up eval metrics (accuracy, precision, recall, F1) in DQN and PPO trainers. These proto fields already exist but are never populated for RL models.

## Testing

- Extend `training_metrics.rs` unit tests for new registrations + verbose gating
- Extend `service.rs` `group_into_sessions` tests with new metric → proto field mappings
- Full pipeline tested on next CI build with new training binaries

## Files Modified

| File | Change |
|---|---|
| `crates/common/src/metrics/training_metrics.rs` | +12 gauges, +5 verbose metrics, verbose gate |
| `services/monitoring_service/proto/monitoring.proto` | +12 proto fields |
| `services/monitoring_service/src/service.rs` | +12 match arms in group_into_sessions |
| `bin/fxt/src/commands/train/monitor.rs` | Enhanced display (grad norm col, RL section, hyperopt detail) |
| `crates/ml/src/trainers/dqn/trainer.rs` | Record Q-values, gradient norm, LR, epoch duration, eval metrics, buffer size |
| `crates/ml/src/trainers/ppo.rs` | Record entropy, KL, advantage, gradient norm, LR, epoch duration, eval metrics |
| `crates/ml/examples/train_baseline_supervised.rs` | Record gradient norm, LR, epoch duration |
| `crates/ml/examples/hyperopt_baseline_rl.rs` | Record trial_epoch, trial_best_loss, elapsed_seconds |
| `crates/ml/examples/hyperopt_baseline_supervised.rs` | Record trial_epoch, trial_best_loss, elapsed_seconds |