feat(dqn): Branching DQN + KernelWeightPack + metrics propagation

Branching DQN (Tavakoli et al., 2018):
- 3 independent advantage heads (exposure=5, order=3, urgency=3) in CUDA kernel
- `use_branching` as hyperopt-tunable parameter (index 11 in 29D space)
- Branch weight pointers packed into KernelWeightPack struct

KernelWeightPack refactor:
- 87→38 CUDA kernel params by packing 48 weight pointers into 384-byte #[repr(C)] struct
- UNPACK_WEIGHT_PTRS macro in CUDA header for clean kernel-side access
- Single .arg(&weight_pack) replaces 48 individual .arg() calls

Hyperopt metrics in JSON output:
- Added `metrics: Option<serde_json::Value>` to TrialResult<P>
- `extract_metrics()` trait method with default None (DQN overrides)
- JSON output now includes per-trial backtest metrics (Sharpe, Sortino,
  Calmar, Omega, drawdown, win_rate, trades) + top-level best_metrics

Prometheus backtest gauges (Rust-side, no CUDA):
- 8 new gauges: foxhunt_hyperopt_best_{sharpe,sortino,calmar,omega,
  max_drawdown_pct,win_rate,total_trades,total_return_pct}

Dynamic episode scaling:
- Removed hardcoded 256 episode cap on small GPUs
- VRAM budget calculation in optimal_n_episodes() handles scaling naturally
- Removed stale .min(256) in PPO trainer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-03-11 00:46:53 +01:00
parent 9278561ec5
commit a5ea8713b6
29 changed files with 1510 additions and 307 deletions

View File

@@ -485,15 +485,32 @@ impl GpuHardwareInfo {
};
let optimal = sm_target.min(vram_cap).clamp(MIN_EPISODES, MAX_EPISODES);
// On small GPUs (≤8 GB), the experience collector output tensors compete
// with model backward graph, candle block allocator overhead, and GPU PER
// buffers. Hard-cap to 256 episodes to keep output under ~25 MB.
let small_gpu_cap = if self.free_memory_mb <= 8192.0 { 256 } else { MAX_EPISODES };
let capped = optimal.min(small_gpu_cap);
// Round up to nearest multiple of 256 for clean block scheduling
let aligned = ((capped + 255) / 256) * 256;
// Round up to nearest multiple of 256 for clean block scheduling.
// The VRAM budget cap above already scales dynamically with free memory,
// so no additional hard-cap is needed for small GPUs.
let aligned = ((optimal + 255) / 256) * 256;
aligned.min(MAX_EPISODES)
}
/// Calculate VRAM-optimal replay buffer capacity.
///
/// Allocates `vram_fraction` of free GPU memory to the replay buffer.
/// Each transition stores: 2 × state_dim × f32 (state + next_state)
/// + 4 scalars (action, reward, done, priority) = (2 * state_dim + 4) * 4 bytes.
///
/// Enforces guardrails:
/// - MIN_REPLAY_CAPACITY = 100_000 (below this, sample efficiency degrades)
/// - MAX_REPLAY_CAPACITY = 10_000_000 (above this, priority staleness dominates)
pub fn optimal_replay_capacity(&self, state_dim: usize, vram_fraction: f64) -> usize {
const MIN_REPLAY_CAPACITY: usize = 100_000;
const MAX_REPLAY_CAPACITY: usize = 10_000_000;
let bytes_per_transition = (2 * state_dim + 4) * 4; // f32 each
let budget_bytes = (self.free_memory_mb * vram_fraction * 1024.0 * 1024.0) as usize;
let raw_capacity = budget_bytes / bytes_per_transition.max(1);
raw_capacity.clamp(MIN_REPLAY_CAPACITY, MAX_REPLAY_CAPACITY)
}
}
/// Detect available GPU memory using nvidia-smi
@@ -1047,4 +1064,29 @@ mod tests {
assert!(n >= 128, "Should hit minimum 128 even with low VRAM, got {}", n);
assert!(n <= 1024, "Should be VRAM-capped with only 500MB free, got {}", n);
}
#[test]
fn test_optimal_replay_capacity_h100() {
let hw = GpuHardwareInfo {
device_name: "NVIDIA H100".to_owned(),
total_memory_mb: 81920.0,
free_memory_mb: 72_000.0,
sm_count: 132,
};
let cap = hw.optimal_replay_capacity(56, 0.70);
assert!(cap >= 500_000, "H100 should get at least 500K capacity, got {}", cap);
assert!(cap <= 10_000_000, "Should be capped at max, got {}", cap);
}
#[test]
fn test_optimal_replay_capacity_small_gpu() {
let hw = GpuHardwareInfo {
device_name: "RTX 3050 Ti".to_owned(),
total_memory_mb: 4096.0,
free_memory_mb: 2_000.0,
sm_count: 20,
};
let cap = hw.optimal_replay_capacity(48, 0.70);
assert!(cap >= 100_000, "Small GPU should get at least min capacity");
}
}