27f536ba6e
Adds a no-wrapper L40S smoke runner alongside the nsys/sanitizer templates. Same compile + checkout + data-mount layout, no profiler binary, no sanitizer instrumentation, no MinIO artefact upload — just a plain test execution against the full training-data PVC. Use case: validate a smoke test passes on the real 27-month dataset when local data is too short (laptop only has 1 quarter of MBP-10/ trades, fxcache truncates below the 10-month minimum). - infra/k8s/argo/smoke-test-template.yaml: WorkflowTemplate smoke-test, entrypoint smoke-run, default test multi_fold_convergence::test_multi_fold_convergence. - infra/k8s/argo/kustomization.yaml: register the new template. - infra/k8s/argo/argo-workflow-netpol.yaml: extend the sanitizer/nsys NetworkPolicy podSelector to include smoke-test (identical egress requirements: git fetch, ci-builder pull, training-data PVC). - scripts/argo-smoke.sh: thin wrapper mirroring argo-nsys.sh / argo-sanitizer.sh (--multi-fold default, --test, --ref, --watch). Verified: kustomize dry-run clean, kubectl apply -k creates the template + reconfigures the netpol live in foxhunt namespace.
82 lines
2.8 KiB
Bash
Executable File
82 lines
2.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Run a smoke test on L40S via Argo Workflows — no profiler, no sanitizer.
|
|
#
|
|
# Use this when you want to validate that a smoke test passes against the
|
|
# full training-data PVC (27 months of OHLCV + MBP-10 + trades) without
|
|
# nsys profile overhead or sanitizer instrumentation. Iteration cycle is
|
|
# faster than argo-nsys.sh / argo-sanitizer.sh because there's no artefact
|
|
# upload step.
|
|
#
|
|
# Why L40S not local: the laptop has only 1 quarter of MBP-10/trades data,
|
|
# which truncates the fxcache below the 10-month minimum that the
|
|
# multi_fold_convergence smoke needs.
|
|
#
|
|
# Usage:
|
|
# ./scripts/argo-smoke.sh # multi_fold_convergence (default)
|
|
# ./scripts/argo-smoke.sh --test magnitude_distribution # different test
|
|
# ./scripts/argo-smoke.sh --ref <sha> # specific commit
|
|
# ./scripts/argo-smoke.sh --watch # follow logs
|
|
#
|
|
# Requires: argo CLI
|
|
|
|
set -euo pipefail
|
|
|
|
TEST_NAME="multi_fold_convergence::test_multi_fold_convergence"
|
|
COMMIT_REF="HEAD"
|
|
GPU_POOL=""
|
|
WATCH=false
|
|
|
|
usage() {
|
|
cat <<EOF
|
|
Usage: $(basename "$0") [OPTIONS]
|
|
|
|
Plain smoke-test runner on L40S (no profiler, no sanitizer).
|
|
|
|
Options:
|
|
--test <name> Smoke test name (default: $TEST_NAME)
|
|
Common values:
|
|
multi_fold_convergence::test_multi_fold_convergence
|
|
magnitude_distribution::test_magnitude_distribution
|
|
performance::test_real_data_single_epoch
|
|
iqn_quantile_monotonicity::iqn_multi_quantile_heads_produce_monotonic_estimates
|
|
--multi-fold Shortcut: --test multi_fold_convergence::test_multi_fold_convergence
|
|
(default — exercises fold-boundary code paths)
|
|
--ref <ref> Git ref to test (default: HEAD)
|
|
--gpu-pool <pool> GPU node pool (default: ci-training-l40s)
|
|
--watch Follow workflow logs
|
|
-h, --help Show this help
|
|
EOF
|
|
exit 0
|
|
}
|
|
|
|
while [[ $# -gt 0 ]]; do
|
|
case $1 in
|
|
--test) TEST_NAME="$2"; shift 2 ;;
|
|
--multi-fold) TEST_NAME="multi_fold_convergence::test_multi_fold_convergence"; shift ;;
|
|
--ref) COMMIT_REF="$2"; shift 2 ;;
|
|
--gpu-pool) GPU_POOL="$2"; shift 2 ;;
|
|
--watch) WATCH=true; shift ;;
|
|
-h|--help) usage ;;
|
|
*) echo "Unknown option: $1"; usage ;;
|
|
esac
|
|
done
|
|
|
|
if [ "$COMMIT_REF" = "HEAD" ]; then
|
|
COMMIT_REF=$(git rev-parse HEAD)
|
|
fi
|
|
|
|
CMD="argo submit -n foxhunt --from=wftmpl/smoke-test"
|
|
CMD="$CMD -p commit-ref=$COMMIT_REF"
|
|
CMD="$CMD -p test-name=$TEST_NAME"
|
|
[[ -n "$GPU_POOL" ]] && CMD="$CMD -p gpu-pool=$GPU_POOL"
|
|
|
|
if $WATCH; then
|
|
CMD="$CMD --watch"
|
|
fi
|
|
|
|
echo "Submitting smoke workflow..."
|
|
echo " commit: $(echo "$COMMIT_REF" | cut -c1-8)"
|
|
echo " test: $TEST_NAME"
|
|
echo ""
|
|
eval "$CMD"
|