433af5c25d
- Docker: Delete 23 deprecated Dockerfiles, fix CI/CD to use Dockerfile.foxhunt-build - Config: Remove 36 .env files, keep 4 essential, delete config/environments/ - Docs: Archive 614 Wave D files to docs/archive/wave_d/, 95% reduction in root - Scripts: Delete 56 deprecated scripts, keep 58 production-critical (49% reduction) - Python: Organize 37 scripts into scripts/python/ subdirectories, delete ml/python/ - Build: Remove 1GB artifacts, delete old venvs, clean Python cache from git - Migrations: Delete deprecated directory (4,432 lines), remove duplicate database/migrations/ - Infrastructure: Delete deployment/ (61 files), docs/scripts/ (8 files) Total impact: ~2,500 files cleaned, 750MB+ space freed, zero production impact All deleted scripts backed up to archives. runpod/ and tests/runpod/ preserved. data_acquisition_service retained per user request.
Database Crate
Overview
The database crate manages the persistent storage layer for the Foxhunt HFT system, primarily utilizing PostgreSQL. It handles schema definitions, migrations, and provides utilities for storing and querying critical trading data, including time-series market data and audit logs.
Features
- PostgreSQL Schema & Migrations: Defines database schemas for trading events, market data, and user configurations, managed via an integrated migration system.
- Event Streaming & Audit Log: Provides interfaces for recording and querying all significant system events, ensuring a comprehensive audit trail for compliance and post-trade analysis.
- Optimized Time-Series Storage: Implements efficient storage and indexing strategies for high-volume, time-series market data.
- Query Utilities: Offers a set of helper functions and ORM-like abstractions for common data retrieval and manipulation tasks.
- Connection Pooling: Manages database connections efficiently using a connection pool to minimize overhead and improve throughput.
- Data Archiving & Retention: Includes mechanisms for managing data lifecycle, such as archiving old data or implementing retention policies.
Usage
use database::models::{TradeEvent, NewTradeEvent};
use database::connection::establish_connection;
use common::types::{InstrumentId, Price, Quantity};
use chrono::Utc;
// This would typically come from a connection pool
let mut conn = establish_connection().expect("Failed to connect to database");
let new_trade = NewTradeEvent {
timestamp: Utc::now(),
instrument_id: InstrumentId::new("ETHUSD".to_string()),
price: Price::new(3000.50),
quantity: Quantity::new(1.2),
side: "BUY".to_string(),
// ... other fields
};
// Example: Insert a new trade event
// let inserted_trade = database::crud::create_trade_event(&mut conn, new_trade)
// .expect("Failed to insert trade event");
// println!("Inserted trade: {:?}", inserted_trade);
Testing
cargo test --package database
Documentation
Detailed API documentation is available at docs.rs/database.