Extend ml_training_service to dispatch GPU training jobs as K8s batch/v1
Jobs, collect results via a Rust sidecar uploader, and support model
promotion with operator approval via fxt CLI.
- K8s dispatcher creates Jobs on gpu-training pool with native sidecar
- training_uploader crate: watches DONE/FAILED marker, uploads to S3,
reports completion via ReportJobCompletion gRPC
- PromotionManager compares metrics, queues better models for approval
- 4 new proto RPCs: ReportJobCompletion, ListPendingPromotions,
ApprovePromotion, RejectPromotion
- fxt commands: train start, model list/approve/reject
- Training binaries write DONE/FAILED markers + metrics.json
- Dockerfile, K8s job template, and CI pipeline updated
- StartTraining gracefully falls back to in-process when outside K8s
- 27 new tests (16 service + 11 promotion), 141 total service tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0f9d756caa