980f5d33c1
Add GrpcMetricsLayer from common::metrics to every gRPC service's Server::builder() chain, enabling automatic Prometheus instrumentation (grpc_server_started_total, grpc_server_handled_total, grpc_server_handling_seconds) for all RPC handlers. Services wired: - trading-service - api-gateway - ml-training-service - backtesting-service - broker-gateway - data-acquisition-service - trading-agent-service Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ml_training_service
Model training orchestration and lifecycle management for the Foxhunt HFT trading system. Manages training jobs for DQN, PPO, TFT, Mamba2, TLOB, and Liquid models with progress tracking, resource allocation, and model artifact storage.
Building
# Default (minimal features)
cargo build --release -p ml_training_service
# With GPU acceleration (requires CUDA)
cargo build --release -p ml_training_service --features gpu
# With mock training data (testing only, bypasses database)
cargo build --release -p ml_training_service --features mock-data
Features
| Feature | Default | Description |
|---|---|---|
minimal |
Yes | Minimal ML feature set for financial models |
gpu |
No | SIMD GPU acceleration (requires CUDA) |
debug |
No | Additional debug logging |
mock-data |
No | Use mock training data instead of PostgreSQL |
Configuration
The gRPC listen port is set via the GRPC_PORT environment variable. Prometheus metrics are exposed on port 9094.
PostgreSQL (via sqlx) is used for job metadata, training history, and state management. Set the connection string with DATABASE_URL.
Running
GRPC_PORT=50053 DATABASE_URL="postgresql://user:pass@localhost:5432/foxhunt_training" \
./target/release/ml_training_service serve
Testing
# Unit tests (offline, no database required)
SQLX_OFFLINE=true cargo test -p ml_training_service --lib
# Integration tests (requires running PostgreSQL)
cargo test -p ml_training_service