plan5(task5-A): wire tier 2/3 val_* metrics into HEALTH_DIAG
Plan 5 Task 4 left every tier-2/tier-3 check failing with "metric missing
from aggregate" because the existing 'Validation backtest:' free-form log
line was not parseable by the aggregate-multi-seed-metrics.py block-keyed
parser. Phase A closes that gap end-to-end (CPU-only, no kernel touch):
* metrics.rs::compute_validation_loss — emit a new
HEALTH_DIAG[<epoch>]: val [sharpe=… sortino=… win_rate=…
max_drawdown=… trade_count=… calmar=…
omega_ratio=… total_pnl=… var_95=… cvar_95=…
trades_per_bar=… active_frac=… dir_entropy=…
sharpe_annualised=… profit_factor=…
window_bars=…]
block immediately after the existing 'Validation backtest:' line. All
16 keys derive from the existing GpuBacktestEvaluator WindowMetrics
reduction (no new GPU work):
- sharpe / sortino / win_rate / max_drawdown / total_trades /
calmar / omega_ratio / total_pnl / var_95 / cvar_95 / buy_count /
sell_count / hold_count come straight from m.*
- window_bars = buy + sell + hold (kernel tallies one direction
per bar)
- trades_per_bar = total_trades / window_bars
- active_frac = (buy + sell) / window_bars (kernel folds Hold AND
Flat into hold_count, so 'active' = bars where the policy chose
Short or Long — meets the Tier-2 'not always Hold' intent)
- dir_entropy = -Σ p ln p over the 3-bucket {short, hold-or-flat,
long} distribution. Documented limitation: max log(3) ≈ 1.099
vs spec's 4-bucket 0.8·log(4) ≈ 1.109 ceiling — tier2 dir_entropy
threshold is unreachable from this 3-bucket distribution; resolution
tracked in audit row.
- sharpe_annualised = m.sharpe alias (kernel already multiplies by
sqrt(bars_per_day · 252) at backtest_metrics_kernel:266)
- profit_factor = m.omega_ratio alias (kernel's omega computes
gain_sum/loss_sum at threshold 0, equivalent to per-step PF;
trade-level PF deferred — needs boundary-aware kernel work)
* mod.rs — adds last_val_metrics: Option<[f32; 14]> on DQNTrainer to
snapshot the WindowMetrics-derived values for downstream consumers
(smoke tests, future telemetry).
* constructor.rs — initialises the new field to None.
* aggregate-multi-seed-metrics.py — switches the block→key joiner from
'__' to '_' so 'val [sharpe=…]' surfaces as the bare 'val_sharpe'
aggregate key the tier check scripts and synthetic test fixtures
already expect. The pre-existing '__' joiner was an oversight in
Plan 5 Task 1B that was never validated against actual aggregator
output (the aggregator emitted 90 'block__key' metrics that nothing
consumed; the synthetic good_tier1.json / bad_tier1.json fixtures
were always shaped as 'val_sharpe', confirming the single-underscore
convention was intended). Renaming the 90 existing keys is safe — no
consumers had locked in on the '__' form.
* docs/dqn-wire-up-audit.md — updates Plan 5 Task 4 row to reference
the now-landed wiring and adds a new row documenting the val [...]
HEALTH_DIAG block pipeline + aggregator joiner change + the deferred
4-bucket dir_dist + trade-level PF caveats.
Validation:
cargo check --workspace clean at 11 warnings.
multi_fold_convergence smoke (629s, 3 folds × 5 epochs on RTX 3050 Ti)
PASSES with 3/3 fold checkpoints. Per-fold best Sharpe: -9.78 / 42.46 /
88.18 (within smoke noise band — no perturbation from the additive
CPU-only HEALTH_DIAG line).
scripts/aggregate-multi-seed-metrics.py against /tmp/p5t5a-smoke.log
produces 3 streams (one per fold), 16 val_* keys all present:
val_sharpe, val_sharpe_annualised, val_sortino, val_win_rate,
val_max_drawdown, val_trade_count, val_calmar, val_omega_ratio,
val_total_pnl, val_var_95, val_cvar_95, val_trades_per_bar,
val_active_frac, val_dir_entropy, val_profit_factor, val_window_bars.
check_tier2.py / check_tier3.py rejection messages are now substantive
(threshold-based) rather than "missing key":
Tier 2: trades_per_bar PASS @ 0.0127; active_frac FAIL @ 0.058 (model
mostly Hold on 5-epoch smoke); dir_entropy FAIL @ 0.18 (within
documented 3-bucket vs 4-bucket caveat).
Tier 3: sharpe_annualised FAIL @ -0.25 (5-epoch smoke not converged);
win_rate skipped (192 trades ≤ 500 noise gate); profit_factor
FAIL @ 0.18 (untrained policy).
Real validation pass requires the L40S 60-epoch run (Phase C).
Deferred (out of T5 Phase A scope):
- val_dir_dist_{short,hold,long,flat} per-direction breakdown — kernel
intentionally collapses Hold+Flat for trade-cycle counting; Tier-2's
log(4) threshold needs either a kernel-level split or a 3-bucket
threshold tweak in check_tier2.py.
- Trade-level profit_factor (sum-winner-PnL / sum-loser-PnL) vs the
per-step omega-equivalent emitted here.
- avg_q_value bare-key aggregation — the metric is logged via separate
Prometheus + tracing paths but not inside any HEALTH_DIAG block; out
of T5 Phase A scope and pre-existing.
This commit is contained in:
jgrusewski
@@ -589,6 +589,9 @@ impl DQNTrainer {
|
||||
grad_clip_kicked_this_epoch: false,
|
||||
last_eval_magnitude_dist: [0.0_f32; 3],
|
||||
last_eval_intent_magnitude_dist: [0.0_f32; 3],
|
||||
// Plan 5 Task 5 Phase A — populated after each `compute_validation_loss`
|
||||
// call; consumed by HEALTH_DIAG `val [...]` block emit.
|
||||
last_val_metrics: None,
|
||||
prev_reward_contrib_popart_var: 0.0,
|
||||
last_reward_contrib: [0.0_f32; 4],
|
||||
|
||||
|
||||
@@ -657,6 +657,105 @@ impl DQNTrainer {
|
||||
"Validation backtest: Sharpe={:.2} Sortino={:.2} WinRate={:.1}% MaxDD={:.3}% Trades={:.0} Calmar={:.2}",
|
||||
m.sharpe, m.sortino, m.win_rate * 100.0, m.max_drawdown * 100.0, m.total_trades, m.calmar,
|
||||
);
|
||||
|
||||
// Plan 5 Task 5 Phase A — flat-emit per-epoch validation metrics
|
||||
// in HEALTH_DIAG `val [...]` block format so the
|
||||
// `aggregate-multi-seed-metrics.py` parser picks them up as
|
||||
// top-level `val_*` aggregate keys (joined with single underscore
|
||||
// by the aggregator). Tier-2/tier-3 check scripts read these keys
|
||||
// directly. Derivations (no new GPU work):
|
||||
// * `trade_count` = m.total_trades (kernel's boundary-
|
||||
// stitched count of position-sign cycles)
|
||||
// * `window_bars` = buy + sell + hold (every bar
|
||||
// contributes exactly one direction tally per kernel)
|
||||
// * `trades_per_bar` = trade_count / window_bars
|
||||
// * `active_frac` = (buy + sell) / window_bars (kernel
|
||||
// counts Hold AND Flat into hold_count, so "active" = bars
|
||||
// where the policy chose Short or Long — directly meets the
|
||||
// Tier-2 spec "model is not always Hold" intent)
|
||||
// * `dir_entropy` = Shannon entropy in nats over the
|
||||
// 3-bucket {short, hold-or-flat, long} distribution from the
|
||||
// kernel; max log(3) ≈ 1.0986. Note: the spec compares
|
||||
// against 0.8·log(4) ≈ 1.109 which is unreachable here
|
||||
// because the kernel collapses Hold+Flat into one bucket.
|
||||
// Tier-2's check_tier2.py needs to use the 3-bucket ceiling
|
||||
// (or fall back to per-direction `val_dir_dist_*`); see audit
|
||||
// doc row for follow-up.
|
||||
// * `profit_factor` = m.omega_ratio (per-step gain_sum /
|
||||
// loss_sum from the kernel's bitonic-sorted return array;
|
||||
// mathematically equivalent to profit-factor when threshold
|
||||
// = 0, which is the standard definition. NOT trade-level PF
|
||||
// — that would require boundary-aware per-trade P&L which
|
||||
// the kernel doesn't yet emit; flagged as future enhancement
|
||||
// in audit row).
|
||||
// * `sharpe_annualised` = m.sharpe (kernel already multiplies
|
||||
// by `sqrt(bars_per_day * 252)` — see backtest_metrics_kernel
|
||||
// line 266). Aliased so tier3's primary key is present.
|
||||
let buy = m.buy_count;
|
||||
let sell = m.sell_count;
|
||||
let hold = m.hold_count;
|
||||
let window_bars = buy + sell + hold;
|
||||
let trades_per_bar = if window_bars > 0.0 { m.total_trades / window_bars } else { 0.0 };
|
||||
let active_frac = if window_bars > 0.0 { (buy + sell) / window_bars } else { 0.0 };
|
||||
|
||||
// Shannon entropy in nats over 3-bucket direction distribution
|
||||
// (short / hold-or-flat / long). Uses single-pass log of the
|
||||
// probability; non-positive buckets contribute nothing.
|
||||
let dir_entropy = if window_bars > 0.0 {
|
||||
let p_short = (sell / window_bars).max(0.0);
|
||||
let p_hold = (hold / window_bars).max(0.0);
|
||||
let p_long = (buy / window_bars).max(0.0);
|
||||
let mut h = 0.0_f32;
|
||||
for &p in &[p_short, p_hold, p_long] {
|
||||
if p > 0.0 { h -= p * p.ln(); }
|
||||
}
|
||||
h
|
||||
} else { 0.0 };
|
||||
|
||||
// Snapshot for downstream consumers (smoke tests, future telemetry).
|
||||
self.last_val_metrics = Some([
|
||||
m.sharpe,
|
||||
m.sortino,
|
||||
m.win_rate,
|
||||
m.max_drawdown,
|
||||
m.total_trades,
|
||||
m.calmar,
|
||||
m.omega_ratio,
|
||||
m.total_pnl,
|
||||
m.var_95,
|
||||
m.cvar_95,
|
||||
m.buy_count,
|
||||
m.sell_count,
|
||||
m.hold_count,
|
||||
window_bars,
|
||||
]);
|
||||
|
||||
// HEALTH_DIAG `val [...]` block. Block name `val` + single-`_`
|
||||
// aggregator join → top-level keys `val_sharpe`, `val_sortino`,
|
||||
// ..., `val_profit_factor`. Additive line — does not perturb the
|
||||
// existing giant HEALTH_DIAG line.
|
||||
let epoch = self.current_epoch;
|
||||
tracing::info!(
|
||||
"HEALTH_DIAG[{}]: val [sharpe={:.4} sortino={:.4} win_rate={:.4} max_drawdown={:.4} trade_count={:.0} calmar={:.4} omega_ratio={:.4} total_pnl={:.4} var_95={:.4e} cvar_95={:.4e} trades_per_bar={:.6} active_frac={:.4} dir_entropy={:.4} sharpe_annualised={:.4} profit_factor={:.4} window_bars={:.0}]",
|
||||
epoch,
|
||||
m.sharpe,
|
||||
m.sortino,
|
||||
m.win_rate,
|
||||
m.max_drawdown,
|
||||
m.total_trades,
|
||||
m.calmar,
|
||||
m.omega_ratio,
|
||||
m.total_pnl,
|
||||
m.var_95,
|
||||
m.cvar_95,
|
||||
trades_per_bar,
|
||||
active_frac,
|
||||
dir_entropy,
|
||||
m.sharpe, // sharpe_annualised alias (kernel already annualises)
|
||||
m.omega_ratio, // profit_factor alias (per-step PF == omega at threshold 0)
|
||||
window_bars,
|
||||
);
|
||||
|
||||
m.sharpe as f64
|
||||
} else {
|
||||
0.0
|
||||
|
||||
@@ -308,6 +308,26 @@ pub struct DQNTrainer {
|
||||
/// safety=0.5+0.5×health pins abs_pos ≤ 0.375×max_pos).
|
||||
pub(crate) last_eval_intent_magnitude_dist: [f32; 3],
|
||||
|
||||
/// Plan 5 Task 5 Phase A — flat aggregate of validation-backtest metrics
|
||||
/// from the most recent `compute_validation_loss` invocation. Sourced from
|
||||
/// the GPU `WindowMetrics` reduction (single-window val path); kept on the
|
||||
/// trainer so the next HEALTH_DIAG emit can publish them as a `val [...]`
|
||||
/// block consumable by `scripts/aggregate-multi-seed-metrics.py` + the
|
||||
/// Plan 5 Task 4 tier-2/tier-3 check scripts.
|
||||
///
|
||||
/// Layout (matches the HEALTH_DIAG `val [...]` key order — see
|
||||
/// `metrics.rs::compute_validation_loss`):
|
||||
/// `[sharpe_annualised, sortino, win_rate, max_drawdown, trade_count,
|
||||
/// calmar, omega_ratio, total_pnl, var_95, cvar_95,
|
||||
/// buy_count, sell_count, hold_count, window_bars]`.
|
||||
///
|
||||
/// `None` until the first validation backtest finishes (epoch 0 with
|
||||
/// async stream, or epoch 0 sync). `[bars_per_day]` is implied as a
|
||||
/// hyperparameter; downstream emitters derive `trades_per_bar`,
|
||||
/// `active_frac`, `dir_entropy`, and `profit_factor` from these 14 floats
|
||||
/// without re-running the kernel.
|
||||
pub(crate) last_val_metrics: Option<[f32; 14]>,
|
||||
|
||||
/// Task 0.9 — prior-epoch values of adaptive-controller outputs, used to
|
||||
/// detect "fire" = value changed from the prior epoch. Controllers audited:
|
||||
/// anti-LR, adaptive tau, adaptive gamma, adaptive grad-clip, CQL alpha
|
||||
|
||||
@@ -48,7 +48,8 @@
|
||||
| `config/metric-bands.toml` | `MetricBandsRegistry::load_from_toml` (auto-loaded by `DQNTrainer::new_internal`) | Wired | Plan 5 Task 2 — per-metric warn/error bands (mean ± 5σ / mean ± 10σ); populated by `scripts/populate-metric-bands-from-runs.py` from Plan 5 Task 1B aggregate JSONs | — |
|
||||
| `scripts/populate-metric-bands-from-runs.py` | reads aggregate JSONs from `scripts/aggregate-multi-seed-metrics.py` (Plan 5 Task 1B.2); writes `config/metric-bands.toml` `[bands]` section | Wired | Plan 5 Task 2 helper — N>=3 sample path; N=1 multiplicative fallback | — |
|
||||
| `scripts/argo-train.sh --profile` + `infra/k8s/argo/train-multi-seed-template.yaml` (`profile` parameter, conditional `nsys profile` wrapper, `mc cp` upload to `foxhunt-training-artifacts/profiles/<sha>/`) + `infra/docker/Dockerfile.foxhunt-training-runtime` (`nsight-systems-cli` apt install) + `infra/k8s/minio/minio.yaml` (new `foxhunt-training-artifacts` bucket) | invoked manually via `./scripts/argo-train.sh dqn --profile [...]`; consumed by `scripts/compare-nsys-profiles.py BASELINE CURRENT --epochs N` (V0 metric: total cuda_gpu_kern_sum / epochs; V1 NVTX-range path deferred to Plan 5 Task 5) | Wired | Plan 5 Task 3 A.4.1 — nsys profile harness with regression-comparison script. `--profile` forces multi-seed render path so dry-run never needs cluster contact (test_nsys_harness.sh); MinIO creds optional on `train-single` (warn-skips upload if absent). **Baseline capture deferred to Plan 5 Task 5** (T5 runs the harness on L40S as part of the multi-seed acceptance pass — avoids burning local GPU time in T3) | — |
|
||||
| `scripts/validation/check_tier1.py` + `check_tier2.py` + `check_tier3.py` + `check_all_tiers.py` (+ `tests/test_tier_checks.sh` + `tests/fixtures/{good,bad}_tier1.json`) | reads aggregate JSONs from `scripts/aggregate-multi-seed-metrics.py` (Plan 5 Task 1B.2); invoked manually as `python3 scripts/validation/check_all_tiers.py <metrics.json> [--warmup-end N]` and from Plan 5 Task 5's acceptance pass | Wired | Plan 5 Task 4 — per-tier exit checks for the spec §2 tiered acceptance criteria. Tier 1 (convergence: std/mean ≤ 0.15 on `val_sharpe`/`avg_q_value`/`train_loss` over stable epochs + no avg_q_value > 500 fold-1 explosion + Q-saturation/hot-path-DtoH placeholders); Tier 2 (behavioural: `val_trades_per_bar ≥ 0.005`, `val_active_frac > 0.2`, dir entropy > 0.8·log4); Tier 3 (profitability: `val_sharpe_annualised > 1.0` with per-bar fallback, `val_win_rate ≥ 0.52` gated on >500 trades, `val_profit_factor` mean ≥ 1.1 AND cross-seed std < 0.3). Stdlib only (no numpy/scipy). Defensive missing-metric handling — each missing aggregate key fails the relevant check with an explanatory message rather than silently passing. **Deferred metrics not yet emitted by HEALTH_DIAG** (current aggregator key set is `<block>__<key>` per `aggregate-multi-seed-metrics.py:115`): `val_trades_per_bar`, `val_active_frac`, `val_dir_entropy` / `val_dir_dist_*`, `val_sharpe_annualised`, `val_win_rate`, `val_profit_factor`, `val_trade_count`. Wiring those into `training_loop.rs::log_epoch_metrics_and_financials` (and either flat-emitting or tracking through the band-check harvest path) is the prerequisite for Tiers 2/3 to ever PASS on real data — tracked under Plan 5 Task 5's pre-flight. | — |
|
||||
| `scripts/validation/check_tier1.py` + `check_tier2.py` + `check_tier3.py` + `check_all_tiers.py` (+ `tests/test_tier_checks.sh` + `tests/fixtures/{good,bad}_tier1.json`) | reads aggregate JSONs from `scripts/aggregate-multi-seed-metrics.py` (Plan 5 Task 1B.2); invoked manually as `python3 scripts/validation/check_all_tiers.py <metrics.json> [--warmup-end N]` and from Plan 5 Task 5's acceptance pass | Wired | Plan 5 Task 4 — per-tier exit checks for the spec §2 tiered acceptance criteria. Tier 1 (convergence: std/mean ≤ 0.15 on `val_sharpe`/`avg_q_value`/`train_loss` over stable epochs + no avg_q_value > 500 fold-1 explosion + Q-saturation/hot-path-DtoH placeholders); Tier 2 (behavioural: `val_trades_per_bar ≥ 0.005`, `val_active_frac > 0.2`, dir entropy > 0.8·log4); Tier 3 (profitability: `val_sharpe_annualised > 1.0` with per-bar fallback, `val_win_rate ≥ 0.52` gated on >500 trades, `val_profit_factor` mean ≥ 1.1 AND cross-seed std < 0.3). Stdlib only (no numpy/scipy). Defensive missing-metric handling — each missing aggregate key fails the relevant check with an explanatory message rather than silently passing. Tier-2/tier-3 wiring landed in Plan 5 Task 5 Phase A (`val [...]` HEALTH_DIAG block + aggregator single-`_` joiner) — see next row. | — |
|
||||
| `trainer/metrics.rs::compute_validation_loss` (HEALTH_DIAG `val [...]` block) + `scripts/aggregate-multi-seed-metrics.py::parse_blocks` (single-`_` joiner) + `trainer/mod.rs` (`last_val_metrics: Option<[f32; 14]>`) | Plan 5 Task 4 tier-2/tier-3 check scripts read these as top-level `val_*` aggregate keys. Block emit pipeline: `evaluate_dqn_graphed` → `WindowMetrics{sharpe, sortino, win_rate, max_drawdown, total_trades, calmar, omega_ratio, total_pnl, var_95, cvar_95, buy/sell/hold_count}` (CUDA `compute_backtest_metrics` 14-float reduction, no new GPU work) → CPU derivations (`window_bars = buy+sell+hold`; `trades_per_bar = total_trades / window_bars`; `active_frac = (buy+sell) / window_bars`; `dir_entropy = -Σ p ln p` over the 3-bucket {short, hold-or-flat, long} distribution; `sharpe_annualised = m.sharpe` alias since the kernel already multiplies by `sqrt(bars_per_day · 252)`; `profit_factor = m.omega_ratio` alias since the kernel's omega is `gain_sum / loss_sum` with threshold 0, equivalent to per-step PF) → `tracing::info!("HEALTH_DIAG[{}]: val [sharpe=… sortino=… win_rate=… max_drawdown=… trade_count=… calmar=… omega_ratio=… total_pnl=… var_95=… cvar_95=… trades_per_bar=… active_frac=… dir_entropy=… sharpe_annualised=… profit_factor=… window_bars=…]", current_epoch, …)`. The aggregator joins `<block>_<key>` (single underscore — was `__` before T5 Phase A; tier scripts and synthetic test fixtures already used the bare `val_*` convention) so each emitted key surfaces as a top-level `val_*` aggregate. **Deferred for follow-up** (cannot be emitted from existing kernel data): (a) per-direction distribution `val_dir_dist_{short,hold,long,flat}` — kernel collapses Hold+Flat into hold_count (intentional for the position-sign trade-cycle definition), so `dir_entropy` here ranges over 3 buckets with max `log 3 ≈ 1.099` rather than the spec's 4-bucket `log 4 ≈ 1.386` ceiling; tier2 `check_dir_entropy` compares against `0.8 · log 4 ≈ 1.109` which is unreachable from the 3-bucket distribution. Resolution options: extend `WindowMetrics` with separate Hold/Flat counts (kernel touch) **or** lower tier2's threshold to the 3-bucket equivalent `0.8 · log 3 ≈ 0.879`. (b) trade-level `profit_factor` (sum-of-winning-trade-P&L / sum-of-losing-trade-P&L) — currently aliased to per-step `omega_ratio`; matches the standard PF definition under threshold 0 but a trade-level variant would require boundary-aware per-trade P&L accumulation in the kernel. Cold-path (per validation epoch); zero hot-path overhead; one new tracing line per epoch. | — |
|
||||
| `trainers/dqn/adaptive_monitor.rs` | Read-only observer trait + harness (FireRateStats, DiagSnapshot, IsvBus<'a>); consumers added in Plan 1 Tasks 9-17 (atoms/gamma/kelly_cap/tau/epsilon/grad_balancer monitors) | Wired (consumers added in same plan) | C.6 GPU-drives-CPU-reads | — |
|
||||
| `trainers/dqn/monitors/grad_balancer_monitor.rs` | Read-only observer for grad_balance_isv_update kernel output (ISV slots 31..35); consumers: HEALTH_DIAG + controller_activity smoke | Wired | Plan 1 Task 17 | — |
|
||||
| `trainers/dqn/monitors/tau_monitor.rs` | Read-only observer for tau_update kernel output (ISV slot 42); consumers: HEALTH_DIAG + controller_activity smoke | Wired | Plan 1 Task 13 | — |
|
||||
|
||||
@@ -114,7 +114,10 @@ def extract_health_diag(line: str) -> tuple[int, str] | None:
|
||||
|
||||
def parse_blocks(body: str) -> dict[str, float]:
|
||||
"""Parse all `<block>[<kv> <kv> ...]` segments from a HEALTH_DIAG
|
||||
body. Returns flat dict with keys `<block>__<subkey>`."""
|
||||
body. Returns flat dict with keys `<block>_<subkey>` (single underscore
|
||||
joiner — this is what the Plan 5 Task 4 tier check scripts expect, and
|
||||
what the synthetic test fixtures encode: e.g. `val [sharpe=…]` becomes
|
||||
aggregate key `val_sharpe`)."""
|
||||
out: dict[str, float] = {}
|
||||
for block_match in BLOCK_RE.finditer(body):
|
||||
block_name = block_match.group(1)
|
||||
@@ -123,7 +126,7 @@ def parse_blocks(body: str) -> dict[str, float]:
|
||||
key = kv_match.group(1)
|
||||
val = parse_value(kv_match.group(2))
|
||||
if val is not None:
|
||||
out[f"{block_name}__{key}"] = val
|
||||
out[f"{block_name}_{key}"] = val
|
||||
return out
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user