gitadmin/foxhunt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(argo): nsys upload to foxhunt-training-results bucket (real bucket name)

Browse Source 
The original target 'foxhunt-training-artifacts' (cloned from
train-multi-seed-template) does not exist on the cluster MinIO.
Available production buckets: foxhunt-models, foxhunt-training-data,
foxhunt-training-results, foxhunt-binaries, foxhunt-backups,
foxhunt-gitlab-{artifacts,packages,registry}.

foxhunt-training-results is the right home for profiling artefacts
(model evaluation outputs already land there). Upload path is now
foxhunt-training-results/profiles/smoke/<short-sha>/profile-<pod>.nsys-rep.

Note: train-multi-seed-template has the same bucket-name typo —
production --profile uploads have been silently failing. Out of scope
for this commit, separate fix needed there.
This commit is contained in:
jgrusewski
2026-04-28 22:56:31 +02:00
parent 3c6470926f
commit 0d630c799e
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
infra/k8s/argo/nsys-test-template.yaml
View File

@@ -2,7 +2,7 @@
#
# One-shot Nsight Systems profiling run on L40S. Wraps a smoke test under
# `nsys profile` and uploads the .nsys-rep to MinIO at
# foxhunt-training-artifacts/profiles/smoke/<short-sha>/.
# foxhunt-training-results/profiles/smoke/<short-sha>/.
#
# Mirrors the nsys integration in train-multi-seed-template (Plan 5 Task 3,
# A.4.1) but for smoke tests. Use this for rapid kernel-timing iteration
@@ -296,7 +296,7 @@ spec:
"${MINIO_ACCESS_KEY:-}" "${MINIO_SECRET_KEY:-}" 2>/dev/null || true
# Smoke profiles go to a separate prefix to keep the
# production training profile bucket clean.
UPLOAD_PATH="foxhunt/foxhunt-training-artifacts/profiles/smoke/${SHA}/profile-${POD_NAME}.nsys-rep"
UPLOAD_PATH="foxhunt/foxhunt-training-results/profiles/smoke/${SHA}/profile-${POD_NAME}.nsys-rep"
if "$MC_BIN" cp "$NSYS_OUT" "$UPLOAD_PATH"; then
echo "=== nsys profile uploaded to ${UPLOAD_PATH} ==="
echo ""

4
scripts/argo-nsys.sh
View File

@@ -2,10 +2,10 @@
# Run a smoke test under Nsight Systems profiling on L40S via Argo Workflows.
#
# Output: .nsys-rep uploaded to MinIO at
# foxhunt-training-artifacts/profiles/smoke/<short-sha>/profile-<pod>.nsys-rep
# foxhunt-training-results/profiles/smoke/<short-sha>/profile-<pod>.nsys-rep
#
# Download + open:
# mc cp foxhunt/foxhunt-training-artifacts/profiles/smoke/<sha>/profile.nsys-rep .
# mc cp foxhunt/foxhunt-training-results/profiles/smoke/<sha>/profile.nsys-rep .
# nsys-ui profile.nsys-rep
#
# Why L40S not local: the laptop GPU has no nsys-ui counters bound and