Extend ml_training_service to dispatch GPU training jobs as K8s batch/v1 Jobs, collect results via a Rust sidecar uploader, and support model promotion with operator approval via fxt CLI. - K8s dispatcher creates Jobs on gpu-training pool with native sidecar - training_uploader crate: watches DONE/FAILED marker, uploads to S3, reports completion via ReportJobCompletion gRPC - PromotionManager compares metrics, queues better models for approval - 4 new proto RPCs: ReportJobCompletion, ListPendingPromotions, ApprovePromotion, RejectPromotion - fxt commands: train start, model list/approve/reject - Training binaries write DONE/FAILED markers + metrics.json - Dockerfile, K8s job template, and CI pipeline updated - StartTraining gracefully falls back to in-process when outside K8s - 27 new tests (16 service + 11 promotion), 141 total service tests pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
17 lines
601 B
Rust
17 lines
601 B
Rust
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
tonic_prost_build::configure()
|
|
.build_server(false)
|
|
.build_client(true)
|
|
.compile_well_known_types(true)
|
|
.extern_path(".google.protobuf", "::prost_types")
|
|
.client_mod_attribute(".", "#[allow(unused_qualifications)]")
|
|
.compile_protos(
|
|
&["../../services/ml_training_service/proto/ml_training.proto"],
|
|
&["../../services/ml_training_service/proto"],
|
|
)?;
|
|
|
|
println!("cargo:rerun-if-changed=../../services/ml_training_service/proto/ml_training.proto");
|
|
|
|
Ok(())
|
|
}
|