Commit Graph

6 Commits

Author SHA1 Message Date
jgrusewski
038a8db16f infra: add feature-cache-pvc volume to all training wrapper templates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 23:11:05 +02:00
jgrusewski
ec88b6aaa4 fix: add volumes to all train-* templates for compile-and-train compat
templateRef doesn't inherit workflow-level volumes from the referenced
template. All 3 train wrappers (DQN, PPO, supervised) now define
git-ssh-key, training-data, cargo-target-cuda volumes and workspace
volumeClaimTemplate locally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:20:06 +02:00
jgrusewski
cbce684546 refactor: align PPO + supervised templates with compile-and-train
Migrate from training-pipeline (requires pre-built binary-tag) to
compile-and-train (compiles from source). All 3 train templates now
use the same pattern: thin wrapper → compile-and-train → pipeline.

Consistent params: commit-sha + git-branch instead of binary-tag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:17:23 +02:00
jgrusewski
8889961837 fix: YAML indentation in train-supervised-template (same bug as PPO)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:10:32 +02:00
jgrusewski
ae5ae056d6 fix: add gpu-test network policy label to all GPU workflow templates
5 templates were missing app.kubernetes.io/component=gpu-test label,
causing MinIO log archival to fail (port 9000 blocked by network policy).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 02:08:50 +01:00
jgrusewski
6ba52425ea feat(infra): Argo workflow templates, drop cuDNN, GPU hotpath fixes
- Add compile-and-deploy, train-dqn/ppo/supervised WorkflowTemplates
- Add Argo Events (EventSource, Sensor, Service) for webhook triggers
- Add NetworkPolicy for compile-and-deploy pods (MinIO/DNS/API egress)
- Add convenience scripts: argo-compile-deploy.sh, argo-train.sh
- Drop cuDNN feature flags from all 9 ML crates (zero conv ops in codebase)
- Switch training runtime base to nvidia/cuda:12.9.1-runtime (saves ~800MB)
- Delete unused selective_scan.cu (16KB, zero Rust callers)
- Fix GPU hotpath violations in ml-core (NVTX, gradient utils, capabilities)
- Fix clippy warnings in ml-dqn (VarMap backticks, const fn)
- Add DQN GPU smoketest, backtest evaluator signal adapter fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 01:44:03 +01:00