dbae70e811
The biggest unexplored win against CI cold-start: ship a Docker image
that carries a fully-built /cargo-target-prebuilt/ snapshot of the
workspace's third-party dependency rlibs, and rsync it into the
cargo-target-cpu PVC as the first step of compile-services.
Cold-start compile path before this commit (fresh node, fresh PVC):
1. cargo resolves Cargo.lock ~10s
2. cargo refreshes registry index ~30s
3. cargo compiles ~1500 dep crates ~3-5 min
4. cargo compiles workspace members ~1-2 min
Total cold compile: ~5-8 min
After this commit:
1. seed-deps-cache initContainer pull image ~30-60s (in-cluster registry)
2. rsync /cargo-target-prebuilt/ into PVC ~30-60s (~6-8 GB local)
3. cargo delta-compiles workspace members ~1-2 min
Total cold compile: ~2-4 min
Net saving: 2-4 min per cold-cache compile pod.
Steady-state (warm PVC, stamp file present): the seed initContainer
exits in ~50ms, no rsync.
Files:
- infra/docker/Dockerfile.ci-deps-cache:
Multi-stage. Stage 1 atop ci-builder-cpu, runs
`cargo build --locked --release --workspace || true` (|| true so
transient errors don't kill the cache build), then snapshots
target/release/{deps,.fingerprint,build} into /cargo-target-prebuilt.
Stage 2 thin Ubuntu + rsync, COPY's the snapshot. Default CMD just
prints the contents — actual entrypoint set by initContainer spec.
Used the "build everything" approach over cargo-chef-style stubbing
because it has zero special-case logic for build.rs / examples /
weird workspace shapes; image is ~1-2 GB bigger as a result, but
pulls fast from in-cluster registry.
- infra/k8s/argo/refresh-deps-cache-template.yaml:
WorkflowTemplate that drives a Kaniko build of the Dockerfile and
pushes to gitlab-registry/.../ci-builder-cpu-with-deps:nightly.
Includes a CronWorkflow that runs nightly at 03:00 UTC, suspended
by default (enable manually after first successful run validation).
Mirrors the existing build-ci-image-template.yaml pattern.
- infra/k8s/argo/compile-and-deploy-template.yaml:
Adds a `seed-deps-cache` initContainer to compile-services. Uses
rsync --ignore-existing (don't clobber newer artifacts the PVC may
already have from a prior build) and a stamp file
/cargo-target/cpu_deps_v${DEPS_VERSION}.stamp to short-circuit
re-seeding on warm pods. Tolerates a missing
/cargo-target-prebuilt dir (image not yet published) — emits a
WARN and continues without the seed.
- infra/k8s/argo/kustomization.yaml:
Registers the new template so kustomize/argo deploys it.
- infra/k8s/argo/README-deps-cache.md:
How to refresh manually, enable the cron, bump DEPS_VERSION,
debug a slow seed, and remove this when no longer needed.
Validation:
- python3 yaml.safe_load_all parses both compile-and-deploy and
refresh-deps-cache templates (1 + 2 docs respectively)
- cargo check --workspace --release --locked passes locally
- Dockerfile syntax visually inspected; the multi-stage COPY pattern
matches existing infra/docker/Dockerfile.* conventions
- Argo template structure mirrors build-ci-image-template.yaml which
is known-working in this cluster
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foxhunt
Production HFT trading system in Rust.
Architecture
The workspace contains 32 crates organized as follows:
Core Libraries (16)
| Crate | Purpose |
|---|---|
trading_engine |
Order processing, FIX 4.4, IB TWS, SIMD, RDTSC timing |
risk |
VaR, Kelly, circuit breakers, kill switches, compliance |
risk-data |
Risk data types and shared structures |
trading-data |
Trading data types |
ml |
DQN Rainbow, PPO, TFT, Mamba2, ensemble inference |
ml-data |
ML data types and feature definitions |
data |
Market data ingestion and storage |
backtesting |
Replay engine, strategy tester |
adaptive-strategy |
Ensemble execution, microstructure analysis |
common |
Shared types, resilience, error handling |
storage |
S3 and local model storage |
model_loader |
Model serialization and loading |
market-data |
Market data feed handlers |
database |
PostgreSQL access layer (SQLx) |
config |
Configuration management |
tli |
CLI commands and tooling |
Services (8)
| Service | Purpose |
|---|---|
backtesting_service |
gRPC backtesting service |
broker_gateway_service |
FIX routing, broker connectivity |
trading_service |
Core trading operations |
ml_training_service |
Model training orchestration |
data_acquisition_service |
Market data acquisition |
trading_agent_service |
Autonomous trading agents |
api_gateway |
gRPC API gateway with auth |
web-gateway |
Axum REST + WebSocket gateway |
Frontend
web-dashboard/ -- React 19 + TypeScript + Vite + TradingView charts.
Building
# Check compilation (no PostgreSQL required)
SQLX_OFFLINE=true cargo check --workspace
# Run tests for a specific crate
SQLX_OFFLINE=true cargo test -p <crate> --lib
# Clippy
SQLX_OFFLINE=true cargo clippy --workspace
ML Models
Four production model architectures on Candle v0.9.1 with CUDA:
- DQN Rainbow -- Deep Q-Network with prioritized replay, dueling heads, noisy nets
- PPO -- Proximal Policy Optimization with GAE and LSTM policies
- TFT -- Temporal Fusion Transformer for multi-horizon forecasting
- Mamba2 -- State space model for sequence prediction
Each model has a standalone trainer and a UnifiedTrainable adapter for the hyperopt pipeline.
Infrastructure
- Git: Gitea at
git.fxhnt.ai(Tailscale-only), Scaleway DEV1-S - Observability: OpenTelemetry OTLP (env
OTEL_EXPORTER_OTLP_ENDPOINT) - Database: PostgreSQL with SQLx offline mode for CI
License
Proprietary. All rights reserved.