a72c5743fe
wild (https://github.com/davidlattimore/wild) is a Rust-native linker, typically 10-30% faster than mold on large release-LTO links. We have ~5 service binaries each doing release-LTO link, so the saving is meaningful: ~30-90 sec wall-time on a fresh compile. Approach (CI-only swap, zero local-dev impact): 1. Dockerfile.ci-builder-cpu installs wild 0.8.0 alongside mold (both linkers present; revert path is trivial). 2. .cargo/config.toml keeps `-fuse-ld=mold` as the file default so `cargo build` works locally without requiring wild on PATH. 3. compile-and-deploy-template.yaml's compile-services script does an in-place sed substitution `mold -> wild` immediately before `cargo build`, gated on `command -v wild` so a missing binary silently falls back to mold instead of failing the build. Why sed-in-script over RUSTFLAGS env var: RUSTFLAGS env var REPLACES the entire target.<triple>.rustflags array (per cargo docs precedence: env > target > build, mutually exclusive — they do NOT merge), which would silently drop our existing -Wl,-z,relro/--as-needed/target-cpu flags. Sed swap edits one token while preserving everything else. Validation: - python3 yaml.safe_load_all parses the template - cargo check --workspace --release --locked succeeds locally (still using mold per the unchanged config.toml default) - Dockerfile syntax visually verified; wild tarball URL confirmed live via curl https://api.github.com/repos/davidlattimore/wild/releases/latest Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
94 lines
4.9 KiB
Docker
94 lines
4.9 KiB
Docker
# CUDA-free CI builder image for Foxhunt service compilation
|
|
# ~2-3GB vs ~8GB for the CUDA devel variant (Dockerfile.ci-builder)
|
|
# Contains: Rust 1.89, protoc, sccache, OpenTofu, Terragrunt, kubectl, git, OpenSSL, clang, mold, wild, lld, make,
|
|
# CUDA nvcc/cudart-dev/cublas-dev (for cudarc .cubin precompilation in ml-* crate builds)
|
|
# Base: Ubuntu 24.04 (glibc 2.39) — matches local dev and training runtime
|
|
# Build: docker build -f infra/docker/Dockerfile.ci-builder-cpu -t foxhunt-ci-builder-cpu .
|
|
# Push: docker tag foxhunt-ci-builder-cpu gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/ci-builder-cpu:latest
|
|
# docker push gitlab-registry.foxhunt.svc.cluster.local:5000/root/foxhunt/ci-builder-cpu:latest
|
|
# Note: rclone is NOT installed here — the only Argo step that uses it
|
|
# (build-web-dashboard) runs in node:22-alpine and `apk add`s rclone itself.
|
|
# terragrunt/tofu/kubectl ARE kept because terragrunt-apply and
|
|
# apply-argo-templates in ci-pipeline-template.yaml run in this image.
|
|
|
|
FROM ubuntu:24.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
git \
|
|
openssh-client \
|
|
curl \
|
|
ca-certificates \
|
|
pkg-config \
|
|
libssl-dev \
|
|
lld \
|
|
clang \
|
|
make \
|
|
perl \
|
|
build-essential \
|
|
libfontconfig1-dev \
|
|
unzip \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Install Rust 1.89 stable (no longer inheriting from rust: base image)
|
|
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.89.0
|
|
ENV PATH="/root/.cargo/bin:${PATH}"
|
|
|
|
# mold linker — 2-5x faster than lld for final link step on large Rust binaries
|
|
RUN curl -fsSL https://github.com/rui314/mold/releases/download/v2.35.1/mold-2.35.1-x86_64-linux.tar.gz \
|
|
| tar xz -C /usr/local --strip-components=1 \
|
|
&& chmod +x /usr/local/bin/mold
|
|
|
|
# wild linker — Rust-native, typically 10-30% faster than mold on link.
|
|
# Drop-in replacement; selected per-build via RUSTFLAGS=-Clink-arg=-fuse-ld=wild
|
|
# (CI sets that override). Local dev keeps mold via .cargo/config.toml.
|
|
# Single static musl binary; mold stays as fallback if wild ever regresses.
|
|
RUN curl -fsSL https://github.com/wild-linker/wild/releases/download/0.8.0/wild-linker-0.8.0-x86_64-unknown-linux-musl.tar.gz \
|
|
| tar xz -C /tmp \
|
|
&& mv /tmp/wild-linker-0.8.0-x86_64-unknown-linux-musl/wild /usr/local/bin/wild \
|
|
&& chmod +x /usr/local/bin/wild \
|
|
&& rm -rf /tmp/wild-linker-0.8.0-x86_64-unknown-linux-musl
|
|
|
|
# protoc 28.3 (proto3 optional support, matches local dev)
|
|
RUN curl -fsSL https://github.com/protocolbuffers/protobuf/releases/download/v28.3/protoc-28.3-linux-x86_64.zip -o /tmp/protoc.zip \
|
|
&& unzip -o /tmp/protoc.zip -d /usr/local bin/protoc 'include/*' \
|
|
&& rm /tmp/protoc.zip \
|
|
&& chmod +x /usr/local/bin/protoc
|
|
|
|
# sccache for local PVC-backed build caching
|
|
RUN curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz \
|
|
| tar xz -C /usr/local/bin --strip-components=1 sccache-v0.10.0-x86_64-unknown-linux-musl/sccache \
|
|
&& chmod +x /usr/local/bin/sccache
|
|
|
|
# clippy pre-installed
|
|
RUN rustup component add clippy
|
|
|
|
# OpenTofu 1.9.0 (Terraform-compatible IaC, used by Terragrunt for infra CI)
|
|
RUN curl -fsSL https://github.com/opentofu/opentofu/releases/download/v1.9.0/tofu_1.9.0_linux_amd64.zip -o /tmp/tofu.zip \
|
|
&& echo "638dd3fb9ecfa6fd9f54a0024b195b12b407c51ccee6f83b18a75a8be79f8214 /tmp/tofu.zip" | sha256sum -c - \
|
|
&& unzip -o /tmp/tofu.zip -d /usr/local/bin tofu \
|
|
&& rm /tmp/tofu.zip \
|
|
&& chmod +x /usr/local/bin/tofu
|
|
|
|
# Terragrunt 0.77.12 (DRY Terraform/OpenTofu wrapper)
|
|
RUN curl -fsSL https://github.com/gruntwork-io/terragrunt/releases/download/v0.77.12/terragrunt_linux_amd64 -o /usr/local/bin/terragrunt \
|
|
&& echo "7f6592fc7faa55b75d7a50ac6a485c48be502486ea61680836f19677b6e8e1b9 /usr/local/bin/terragrunt" | sha256sum -c - \
|
|
&& chmod +x /usr/local/bin/terragrunt
|
|
|
|
# kubectl 1.31.4 (CI deploy + argo template self-apply)
|
|
RUN curl -fsSL "https://dl.k8s.io/release/v1.31.4/bin/linux/amd64/kubectl" -o /usr/local/bin/kubectl \
|
|
&& chmod +x /usr/local/bin/kubectl
|
|
|
|
# CUDA nvcc compiler only (no runtime/driver) — needed for cudarc .cubin precompilation
|
|
# ~400MB vs ~6GB for the full cuda-devel image
|
|
RUN curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb -o /tmp/cuda-keyring.deb \
|
|
&& dpkg -i /tmp/cuda-keyring.deb && rm /tmp/cuda-keyring.deb \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends cuda-nvcc-12-6 cuda-cudart-dev-12-6 libcublas-dev-12-6 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
ENV PATH="/usr/local/cuda-12.6/bin:${PATH}"
|
|
|
|
# Verify
|
|
RUN rustc --version && cargo --version && git --version && protoc --version && sccache --version && make --version && mold --version && wild --version && tofu --version && terragrunt --version && kubectl version --client && nvcc --version
|