ci: increase compile-training pod timeout to 20 min
When all 8 per-service compile jobs fire simultaneously (e.g. on Cargo.toml change), the single POP2-32C-128G node is fully loaded. Training compile pod can't schedule until service jobs finish. KUBERNETES_POLL_TIMEOUT=1200 gives it time to wait. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -599,6 +599,9 @@ compile-training:
|
||||
# CUDA stubs from ci-builder image — compiles CUDA kernels without GPU hardware.
|
||||
# L4 GPU = compute capability 8.9 → candle-kernels generates sm_89 PTX
|
||||
CUDA_COMPUTE_CAP: "89"
|
||||
# When all 8 service jobs fire (Cargo.toml change), the node is fully loaded.
|
||||
# Increase pod scheduling timeout so training waits for service jobs to finish.
|
||||
KUBERNETES_POLL_TIMEOUT: "1200"
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
|
||||
changes:
|
||||
|
||||
Reference in New Issue
Block a user