foxhunt

gitadmin/foxhunt

Fork 0

Commit Graph

Author	SHA1	Message	Date
jgrusewski	6ba52425ea	feat(infra): Argo workflow templates, drop cuDNN, GPU hotpath fixes - Add compile-and-deploy, train-dqn/ppo/supervised WorkflowTemplates - Add Argo Events (EventSource, Sensor, Service) for webhook triggers - Add NetworkPolicy for compile-and-deploy pods (MinIO/DNS/API egress) - Add convenience scripts: argo-compile-deploy.sh, argo-train.sh - Drop cuDNN feature flags from all 9 ML crates (zero conv ops in codebase) - Switch training runtime base to nvidia/cuda:12.9.1-runtime (saves ~800MB) - Delete unused selective_scan.cu (16KB, zero Rust callers) - Fix GPU hotpath violations in ml-core (NVTX, gradient utils, capabilities) - Fix clippy warnings in ml-dqn (VarMap backticks, const fn) - Add DQN GPU smoketest, backtest evaluator signal adapter fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 01:44:03 +01:00
jgrusewski	edd7ceb0f2	perf(cuda): H100 optimization Wave 0+1 — NVTX profiling, L2 cache pinning, dynamic shmem, async double buffer Wave 0 (NVTX Instrumentation): - Add ml-core::nvtx module with NvtxRange RAII guard (runtime dlopen, zero overhead when absent) - Instrument 10 CUDA pipeline hot paths: experience collector, backtest evaluator, PPO collector, statistics, training guard, monitoring, replay buffer Wave 1 (Low-Effort H100 Optimizations): - L2 cache persistence: pin DQN weights (~23MB BF16) in H100's 50MB L2 via cudaCtxSetLimit(CU_LIMIT_PERSISTING_L2_CACHE_SIZE) — 3.5x effective bandwidth - Dynamic shared memory: GPU-aware tile sizing (228KB H100, 164KB A100, 100KB RTX) via CU_FUNC_ATTRIBUTE_MAX_DYNAMIC_SHARED_SIZE_BYTES opt-in - Async double buffer: sync_staging() with CUDA stream synchronization before swap Validation: 0 clippy errors, 1629 tests passed (308+410+911), 0 gpu-hotpath violations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-11 22:05:01 +01:00
jgrusewski	33cc076eb3	refactor(ml): move compute primitives to ml-core (5b) Move optimizers, gradient_accumulation, gradient_utils, cuda_compat, tensor_ops, and gpu (device config, capabilities, memory profiling) to ml-core. These are shared compute primitives used by all models. Also commit module files for core types (common, config, error, model, traits, types) that were moved from ml to ml-core in task 5a but left staged without being committed. Notable changes: - resolve_batch_size() stays in ml (new batch_size_resolver module) because it depends on memory_optimization::auto_batch_size which has not yet moved to ml-core - FactoredAction legacy bridge converted from inherent impl to extension trait (FactoredActionLegacy) since FactoredAction is now defined in ml-core, not ml - candle-optimisers added to ml-core dependencies (needed by Adam) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-08 15:13:41 +01:00

Author

SHA1

Message

Date

jgrusewski

6ba52425ea

feat(infra): Argo workflow templates, drop cuDNN, GPU hotpath fixes

- Add compile-and-deploy, train-dqn/ppo/supervised WorkflowTemplates
- Add Argo Events (EventSource, Sensor, Service) for webhook triggers
- Add NetworkPolicy for compile-and-deploy pods (MinIO/DNS/API egress)
- Add convenience scripts: argo-compile-deploy.sh, argo-train.sh
- Drop cuDNN feature flags from all 9 ML crates (zero conv ops in codebase)
- Switch training runtime base to nvidia/cuda:12.9.1-runtime (saves ~800MB)
- Delete unused selective_scan.cu (16KB, zero Rust callers)
- Fix GPU hotpath violations in ml-core (NVTX, gradient utils, capabilities)
- Fix clippy warnings in ml-dqn (VarMap backticks, const fn)
- Add DQN GPU smoketest, backtest evaluator signal adapter fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-12 01:44:03 +01:00

jgrusewski

edd7ceb0f2

perf(cuda): H100 optimization Wave 0+1 — NVTX profiling, L2 cache pinning, dynamic shmem, async double buffer

Wave 0 (NVTX Instrumentation):
- Add ml-core::nvtx module with NvtxRange RAII guard (runtime dlopen, zero overhead when absent)
- Instrument 10 CUDA pipeline hot paths: experience collector, backtest evaluator,
  PPO collector, statistics, training guard, monitoring, replay buffer

Wave 1 (Low-Effort H100 Optimizations):
- L2 cache persistence: pin DQN weights (~23MB BF16) in H100's 50MB L2 via
  cudaCtxSetLimit(CU_LIMIT_PERSISTING_L2_CACHE_SIZE) — 3.5x effective bandwidth
- Dynamic shared memory: GPU-aware tile sizing (228KB H100, 164KB A100, 100KB RTX)
  via CU_FUNC_ATTRIBUTE_MAX_DYNAMIC_SHARED_SIZE_BYTES opt-in
- Async double buffer: sync_staging() with CUDA stream synchronization before swap

Validation: 0 clippy errors, 1629 tests passed (308+410+911), 0 gpu-hotpath violations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-11 22:05:01 +01:00

jgrusewski

33cc076eb3

refactor(ml): move compute primitives to ml-core (5b)

Move optimizers, gradient_accumulation, gradient_utils, cuda_compat,
tensor_ops, and gpu (device config, capabilities, memory profiling) to
ml-core. These are shared compute primitives used by all models.

Also commit module files for core types (common, config, error, model,
traits, types) that were moved from ml to ml-core in task 5a but left
staged without being committed.

Notable changes:
- resolve_batch_size() stays in ml (new batch_size_resolver module)
  because it depends on memory_optimization::auto_batch_size which
  has not yet moved to ml-core
- FactoredAction legacy bridge converted from inherent impl to
  extension trait (FactoredActionLegacy) since FactoredAction is now
  defined in ml-core, not ml
- candle-optimisers added to ml-core dependencies (needed by Adam)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-08 15:13:41 +01:00

3 Commits