c731bef759
RL models (DQN/PPO) only need basic CUDA runtime (~2GB), while supervised models need cuDNN (~4GB). Splitting saves ~2GB pull time per RL job. Rename the L4 GPU pool from ci-compile to ci-rl to reflect its actual use. Add DaemonSet image pre-puller to cache training images on GPU nodes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
45 lines
1.4 KiB
Docker
45 lines
1.4 KiB
Docker
# RL training runtime image — DQN + PPO (no cuDNN needed, ~2GB)
|
|
# Binaries are compiled in CI compile-training job with --features ml/cuda
|
|
# Usage: kaniko --context dir://build-out/training --dockerfile Dockerfile.training-rl
|
|
# Run: docker run --gpus all training-rl train_baseline_rl --model dqn [args...]
|
|
|
|
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
ca-certificates \
|
|
libssl3 \
|
|
curl \
|
|
unzip \
|
|
&& curl -fsSL https://downloads.rclone.org/v1.69.1/rclone-v1.69.1-linux-amd64.zip -o /tmp/rclone.zip \
|
|
&& unzip -j /tmp/rclone.zip '*/rclone' -d /usr/local/bin/ \
|
|
&& chmod +x /usr/local/bin/rclone \
|
|
&& rm /tmp/rclone.zip \
|
|
&& apt-get purge -y unzip \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN groupadd -g 1000 foxhunt \
|
|
&& useradd -u 1000 -g foxhunt -m -s /bin/false foxhunt
|
|
|
|
COPY train_baseline_rl \
|
|
evaluate_baseline \
|
|
hyperopt_baseline_rl \
|
|
training_uploader \
|
|
/usr/local/bin/
|
|
|
|
RUN chmod +x /usr/local/bin/train_baseline_rl \
|
|
/usr/local/bin/evaluate_baseline \
|
|
/usr/local/bin/hyperopt_baseline_rl \
|
|
/usr/local/bin/training_uploader
|
|
|
|
# CUDA runtime environment
|
|
ENV NVIDIA_VISIBLE_DEVICES=all
|
|
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
|
|
|
|
USER foxhunt
|
|
|
|
WORKDIR /data
|
|
|
|
CMD ["echo", "Specify training command via K8s Job args"]
|