Three fixes validated by 20/20 hyperopt trials on H100 (zero OOM): 1. Workspace default-features: ml-core, ml-dqn, ml-ppo, ml-supervised workspace deps now have default-features=false. Prevents cudarc (which requires nvcc) from leaking into CPU service builds via Cargo feature unification. CI compile-services was failing with "Failed to execute nvcc: No such file or directory" (exit 101). 2. BF16 comparison fix: Candle's gt()/le() don't support BF16 operands. Cast ADX/CUSUM features to F32 before threshold comparison in regime classification. Previous approach (cast threshold to BF16) failed due to Candle broadcast_as reverting dtype. 3. CI pipeline: expand ML change detection to all 14 sub-crates, add component:compile labels for sccache network policy matching, bump training runtime to CUDA 12.6 + Ubuntu 24.04 (glibc 2.39). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
28 lines
885 B
Docker
28 lines
885 B
Docker
# Generic GPU runtime base image for Foxhunt training pods
|
|
# Contains: CUDA 12.4 + cuDNN + NVRTC + rclone
|
|
# Binaries are fetched by initContainer at pod startup, not baked into this image.
|
|
# Rebuild: only on CUDA version bump
|
|
|
|
FROM nvidia/cuda:12.6.3-cudnn-runtime-ubuntu24.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
ca-certificates \
|
|
libssl3t64 \
|
|
curl \
|
|
unzip \
|
|
cuda-nvrtc-12-6 \
|
|
&& curl -fsSL https://downloads.rclone.org/v1.69.1/rclone-v1.69.1-linux-amd64.zip -o /tmp/rclone.zip \
|
|
&& unzip -j /tmp/rclone.zip '*/rclone' -d /usr/local/bin/ \
|
|
&& rm /tmp/rclone.zip \
|
|
&& apt-get purge -y unzip \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN usermod -l foxhunt ubuntu && groupmod -n foxhunt ubuntu
|
|
|
|
ENV NVIDIA_VISIBLE_DEVICES=all
|
|
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
|
|
|
USER foxhunt
|