From 0d630c799e7c4b6d048f01c44775c404426d0fd1 Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Tue, 28 Apr 2026 22:56:31 +0200 Subject: [PATCH] fix(argo): nsys upload to foxhunt-training-results bucket (real bucket name) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The original target 'foxhunt-training-artifacts' (cloned from train-multi-seed-template) does not exist on the cluster MinIO. Available production buckets: foxhunt-models, foxhunt-training-data, foxhunt-training-results, foxhunt-binaries, foxhunt-backups, foxhunt-gitlab-{artifacts,packages,registry}. foxhunt-training-results is the right home for profiling artefacts (model evaluation outputs already land there). Upload path is now foxhunt-training-results/profiles/smoke//profile-.nsys-rep. Note: train-multi-seed-template has the same bucket-name typo — production --profile uploads have been silently failing. Out of scope for this commit, separate fix needed there. --- infra/k8s/argo/nsys-test-template.yaml | 4 ++-- scripts/argo-nsys.sh | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/infra/k8s/argo/nsys-test-template.yaml b/infra/k8s/argo/nsys-test-template.yaml index 05fc21f84..b6bb82cbc 100644 --- a/infra/k8s/argo/nsys-test-template.yaml +++ b/infra/k8s/argo/nsys-test-template.yaml @@ -2,7 +2,7 @@ # # One-shot Nsight Systems profiling run on L40S. Wraps a smoke test under # `nsys profile` and uploads the .nsys-rep to MinIO at -# foxhunt-training-artifacts/profiles/smoke//. +# foxhunt-training-results/profiles/smoke//. # # Mirrors the nsys integration in train-multi-seed-template (Plan 5 Task 3, # A.4.1) but for smoke tests. Use this for rapid kernel-timing iteration @@ -296,7 +296,7 @@ spec: "${MINIO_ACCESS_KEY:-}" "${MINIO_SECRET_KEY:-}" 2>/dev/null || true # Smoke profiles go to a separate prefix to keep the # production training profile bucket clean. - UPLOAD_PATH="foxhunt/foxhunt-training-artifacts/profiles/smoke/${SHA}/profile-${POD_NAME}.nsys-rep" + UPLOAD_PATH="foxhunt/foxhunt-training-results/profiles/smoke/${SHA}/profile-${POD_NAME}.nsys-rep" if "$MC_BIN" cp "$NSYS_OUT" "$UPLOAD_PATH"; then echo "=== nsys profile uploaded to ${UPLOAD_PATH} ===" echo "" diff --git a/scripts/argo-nsys.sh b/scripts/argo-nsys.sh index f6295cb10..1516c25b8 100755 --- a/scripts/argo-nsys.sh +++ b/scripts/argo-nsys.sh @@ -2,10 +2,10 @@ # Run a smoke test under Nsight Systems profiling on L40S via Argo Workflows. # # Output: .nsys-rep uploaded to MinIO at -# foxhunt-training-artifacts/profiles/smoke//profile-.nsys-rep +# foxhunt-training-results/profiles/smoke//profile-.nsys-rep # # Download + open: -# mc cp foxhunt/foxhunt-training-artifacts/profiles/smoke//profile.nsys-rep . +# mc cp foxhunt/foxhunt-training-results/profiles/smoke//profile.nsys-rep . # nsys-ui profile.nsys-rep # # Why L40S not local: the laptop GPU has no nsys-ui counters bound and