foxhunt/ml

ml Crate

The ml crate provides the core machine learning capabilities for the Foxhunt High-Frequency Trading (HFT) System. It encompasses a suite of advanced models for sequence prediction, reinforcement learning, and time series analysis, optimized for low-latency inference and robust model management within a high-frequency trading environment.

Features

• Advanced Model Suite: Implementation of cutting-edge ML models tailored for HFT.
• Low-Latency Inference: Highly optimized inference engine designed for real-time market data processing.
• GPU Acceleration: Leverages CUDA/cuDNN for high-performance, GPU-accelerated model inference.
• Dynamic Model Management: Supports hot-swapping and versioning of models for seamless updates.
• Cloud-Native Storage: S3-based model storage and caching for reliable and scalable deployment.
• Experimentation & Monitoring: Built-in support for A/B testing and performance monitoring of deployed models.

Models Implemented

This crate includes specialized implementations of various machine learning models, each optimized for specific HFT challenges:

• MAMBA-2 State Space Models: Efficient sequence prediction, crucial for forecasting market movements, order flow, or short-term price trajectories in dynamic HFT scenarios.
• Deep Q-Learning (DQN): A reinforcement learning algorithm for discovering and executing optimal trading strategies, learning directly from market rewards and penalties.
• Proximal Policy Optimization (PPO) with GAE: A robust policy gradient reinforcement learning method, often employed for more complex, continuous action spaces in trading agents, offering stable and efficient learning.
• Temporal Fusion Transformer (TFT): An advanced transformer-based architecture for multivariate time series forecasting, adept at handling complex temporal dependencies and integrating exogenous variables for precise price or volume prediction.
• Liquid Networks: Biologically inspired neural networks offering high adaptability and robustness to changing data distributions, making them suitable for the non-stationary and volatile nature of financial markets.
• Transformer-based Order Book (TLOB) Analysis: Utilizes transformer architectures to process granular, high-dimensional order book data, identifying intricate patterns and predicting short-term price movements, liquidity shifts, or order imbalances.

Architecture

The ml crate is designed with the following key architectural components to ensure performance, reliability, and maintainability:

• Inference Bridge: A dedicated, low-latency communication channel facilitating seamless prediction delivery from ML models to the core trading_engine.
• Model Registry: A centralized service for managing, versioning, and deploying ML models. It supports hot-swapping, allowing new model versions to be deployed without service interruption.
• Performance Monitoring & Distillation: Real-time tracking of model efficacy, latency, and resource utilization. Includes mechanisms for model distillation to create smaller, faster models suitable for extreme low-latency environments.
• Ensemble Methods: Integrates capabilities for combining predictions from multiple models, often incorporating confidence scoring, to enhance overall prediction robustness and accuracy.

Usage

To use the ml crate, you'll typically interact with the ModelRegistry to load models and then use the InferenceEngine trait to make predictions.

use ml::{InferenceEngine, ModelRegistry};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize your application configuration
    let config = /* Your application configuration object */;

    // Instantiate the ModelRegistry
    let registry = ModelRegistry::new(config).await?;

    // Load a specific model by its identifier and version
    let model = registry.load_model("mamba2-v1.2.3").await?;

    // Prepare the current market state or features for inference
    let market_state = /* Your current market state object */;

    // Run inference using the loaded model
    let prediction = model.predict(&market_state).await?;

    println!("Inference result: {:?}", prediction);

    Ok(())
}

Testing

To run the tests for the ml crate, use the standard Cargo test command:

cargo test --package ml

Documentation

Comprehensive API documentation for the ml crate can be found on docs.rs/ml.