Implement comprehensive Runpod deployment with S3 volume mount architecture for FP32 ML model training on Tesla V100 GPUs. ## Infrastructure Components ### Deployment Scripts (scripts/) - runpod_deploy.sh: Master deployment orchestrator (8-step workflow) - runpod_upload.sh: S3 upload for binaries and test data - upload_env_to_runpod.sh: Secure .env credentials upload - runpod_deploy_test.sh: Prerequisites validation ### Docker Configuration - Dockerfile.runpod: Multi-stage CUDA 12.1 runtime (~2GB, no binaries) - entrypoint.sh: Volume verification and training execution - Architecture: Volume mount (NO S3 downloads in pods) ### S3 Configuration - Bucket: se3zdnb5o4 (Iceland region: eur-is-1) - Endpoint: https://s3api-eur-is-1.runpod.io - Structure: binaries/, test_data/, models/, .env ### OpenTofu Infrastructure (terraform/runpod/) - main.tf: Pod and volume resources - variables.tf: Configuration variables - outputs.tf: Pod connection info - Security: NO credentials in state (uses volume .env) ## Deployment Assets Uploaded ### Training Binaries (77MB) - train_tft_parquet (23M) - TFT-225 features - train_mamba2_parquet (22M) - MAMBA-2 state space - train_dqn (22M) - Deep Q-Network - train_ppo (13M) - Proximal Policy Optimization ### Test Data (13.8 MB) - 9 Parquet files: ES.FUT, NQ.FUT, 6E.FUT, ZN.FUT (180-day datasets) ### Credentials - .env file (1.5 KB, private access, chmod 600) ## Documentation ### Deployment Guides - RUNPOD_DEPLOYMENT_READY_SUMMARY.md: Complete deployment status - RUNPOD_VOLUME_DEPLOYMENT_GUIDE.md: Step-by-step guide (42KB) - RUNPOD_DEPLOYMENT_QUICK_START.md: Quick reference - RUNPOD_UPLOAD_GUIDE.md: S3 upload instructions - RUNPOD_VOLUME_CONFIGURATION_COMPLETE.md: S3 setup report - RUNPOD_S3_PARQUET_UPLOAD_REPORT.md: Data upload verification ### Architecture Documentation - RUNPOD_VOLUME_MOUNT_ARCHITECTURE.md: Volume mount design - RUNPOD_S3_ARCHITECTURE_DIAGRAM.txt: S3 API vs filesystem access - DOCKERFILE_RUNPOD_FINAL_SUMMARY.md: Docker image specification ### Decision Documentation - RUNPOD_DEPLOYMENT_CHECKLIST.md: Go/no-go decision matrix (27KB) - RUNPOD_DEPLOYMENT_DECISION_TREE.md: Decision workflow - FP32_RUNPOD_DEPLOYMENT_READY.md: FP32 deployment readiness ## QAT Enhancements ### Core QAT Infrastructure - ml/src/memory_optimization/qat.rs: Enhanced QAT observer (+226 lines) - ml/src/memory_optimization/auto_batch_size.rs: OOM recovery (+84 lines) - ml/src/tft/qat_tft.rs: QAT TFT wrapper (+154 lines) - ml/src/trainers/tft.rs: QAT training integration (+433 lines) - ml/src/qat_metrics_exporter.rs: NEW - QAT metrics export ### QAT Testing - ml/tests/qat_integration_tests.rs: NEW - Integration test suite - ml/tests/qat_gradient_clipping_test.rs: NEW - Gradient clipping tests - ml/tests/qat_device_consistency_test.rs: Device mismatch tests (+205 lines) - ml/tests/qat_accuracy_validation_test.rs: Accuracy validation - ml/tests/qat_tft_integration_test.rs: TFT QAT integration ### QAT Documentation - ml/docs/QAT_GUIDE.md: Comprehensive QAT guide (+616 lines) - ml/docs/QAT_GRADIENT_CHECKPOINTING_WORKAROUND.md: NEW - Workaround guide - QAT_BLOCKERS_ROOT_CAUSE_ANALYSIS.md: P0 blocker analysis (44KB) - QAT_ACCURACY_VALIDATION_REPORT.md: Accuracy comparison - QAT_GRADIENT_CLIPPING_VALIDATION_REPORT.md: Clipping validation ### QAT Monitoring - config/grafana/dashboards/qat-training-metrics.json: NEW - Grafana dashboard ## AWS CLI Configuration ### Credentials Setup - ~/.aws/credentials: Runpod profile configured - Access Key: user_2xxA3XcIFj16yfL3aBon9niiSpr - Secret Key: (from RUNPOD_S3_SECRET) - ~/.aws/config: Iceland region (eur-is-1) ## Production Readiness ### FP32 Models: ✅ READY FOR DEPLOYMENT - DQN: 15-20s training, ~6MB GPU memory - PPO: 7-10s training, ~145MB GPU memory - MAMBA-2: 2-3 min training, ~164MB GPU memory - TFT-225: 3-5 min training, ~500MB GPU memory - Total GPU Budget: 815MB (fits on 4GB+ Tesla V100) ### QAT Models: 🔴 BLOCKED - 24 tests implemented but DO NOT COMPILE (11 errors) - 3 P0 blockers: device mismatch, gradient checkpointing, OOM recovery - Timeline: 1-2 weeks to fix (13h P0 fixes + validation) ### Wave D Features: ✅ OPERATIONAL - 225 features fully integrated - Feature extraction: 5.10μs/bar (196x faster than target) - Wave D backtest: Sharpe 2.00, Win Rate 60%, Drawdown 15% - Database migration 045: Applied cleanly, zero conflicts ## Cost Analysis ### One-Time Setup - Network Volume: $4/month (50GB SSD) - Upload costs: FREE (S3 API included) ### Per Training Run (TFT-225) - GPU: Tesla V100-PCIE-16GB @ $0.29/hr - Training Time: ~4 hours - Cost per run: $1.16 ### Monthly (20 Training Runs) - Storage: $4.00/month - Training: $23.20/month (20 runs × $1.16) - Total: $27.20/month ## Security ### Credentials Management - ✅ NO credentials in Docker image - ✅ NO credentials in Terraform state - ✅ .env gitignored and not committed - ✅ .env file private on S3 (HTTP 401 on public access) - ✅ Docker Hub repository PRIVATE (jgrusewski/foxhunt) ### Access Control - S3 API: Local client uploads only - Volume mount: Pod filesystem access only - Authentication: AWS CLI with Runpod profile required ## Next Steps 1. ✅ COMPLETE: Build Docker image 2. ⏳ PENDING: Push to Docker Hub 3. ⏳ PENDING: Deploy pod via Runpod console 4. ⏳ PENDING: Validate training on Tesla V100 ## Performance Targets - Build time: 5-10 min - Upload time: ~20 sec (90MB total) - Pod startup: ~30 sec - Training time: 3-5 min (TFT-225) - Total deployment: ~40 min from start to first training run ## Test Status - FP32 tests: 597/608 passing (98.2%) - QAT tests: 0/24 passing (compilation errors) - Overall: 2,062/2,086 passing (98.8% excluding QAT) 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1043 lines
29 KiB
Rust
1043 lines
29 KiB
Rust
//! Market Data Edge Case Tests
|
|
//!
|
|
//! Comprehensive edge case coverage for market data handling, parsing, validation,
|
|
//! and real-time updates in the TLI. Tests cover:
|
|
//! - Price parsing (scientific notation, extremes, NaN/Inf)
|
|
//! - Symbol validation (invalid chars, empty, Unicode)
|
|
//! - Timestamp handling (future times, epoch boundaries)
|
|
//! - Update rate limiting and high-frequency scenarios
|
|
//! - Data staleness detection
|
|
//! - Order book edge cases (empty, single level, 100+ levels)
|
|
//! - Trade history extremes (zero trades, 10K+ trades)
|
|
//! - Connection interruptions and reconnection
|
|
|
|
// Suppress false-positive unused_crate_dependencies warnings
|
|
// dev-dependencies are shared across ALL test targets in the crate
|
|
// This test may not use all deps, but they are required by other integration tests
|
|
#![allow(unused_crate_dependencies)]
|
|
|
|
use chrono::{DateTime, Utc};
|
|
use std::collections::VecDeque;
|
|
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
|
use tokio::sync::mpsc;
|
|
use tokio::time::{sleep, timeout};
|
|
|
|
use adaptive_strategy::microstructure::OrderLevel;
|
|
use tli::dashboard::events::MarketDataDisplayEvent;
|
|
use tli::types::{current_unix_nanos, unix_nanos_to_system_time, validate_price, validate_symbol};
|
|
|
|
// ============================================================================
|
|
// HELPER FUNCTIONS AND TYPES
|
|
// ============================================================================
|
|
|
|
/// Parse price string that may contain scientific notation or extreme values
|
|
fn parse_price_string(price_str: &str) -> Result<f64, String> {
|
|
price_str
|
|
.parse::<f64>()
|
|
.map_err(|e| format!("Failed to parse price: {}", e))
|
|
}
|
|
|
|
/// Create test market data event
|
|
fn create_market_data(symbol: &str, price: f64, timestamp: i64) -> MarketDataDisplayEvent {
|
|
MarketDataDisplayEvent {
|
|
symbol: symbol.to_string(),
|
|
price,
|
|
volume: 1000,
|
|
timestamp,
|
|
bid: Some(price - 0.01),
|
|
ask: Some(price + 0.01),
|
|
change: Some(0.0),
|
|
change_percent: Some(0.0),
|
|
}
|
|
}
|
|
|
|
/// Create order book snapshot
|
|
fn create_order_book_snapshot(
|
|
bids: Vec<(f64, f64, u32)>,
|
|
asks: Vec<(f64, f64, u32)>,
|
|
) -> OrderBookSnapshot {
|
|
OrderBookSnapshot {
|
|
timestamp: Utc::now(),
|
|
bids: bids
|
|
.into_iter()
|
|
.map(|(price, size, count)| OrderLevel {
|
|
price,
|
|
quantity: size,
|
|
order_count: count,
|
|
timestamp: Utc::now(),
|
|
})
|
|
.collect(),
|
|
asks: asks
|
|
.into_iter()
|
|
.map(|(price, size, count)| OrderLevel {
|
|
price,
|
|
quantity: size,
|
|
order_count: count,
|
|
timestamp: Utc::now(),
|
|
})
|
|
.collect(),
|
|
spread: 0.0,
|
|
}
|
|
}
|
|
|
|
/// Order book snapshot structure for tests
|
|
#[derive(Debug, Clone)]
|
|
struct OrderBookSnapshot {
|
|
timestamp: DateTime<Utc>,
|
|
bids: Vec<OrderLevel>,
|
|
asks: Vec<OrderLevel>,
|
|
spread: f64,
|
|
}
|
|
|
|
impl OrderBookSnapshot {
|
|
fn calculate_spread(&mut self) {
|
|
if let (Some(best_bid), Some(best_ask)) = (self.bids.first(), self.asks.first()) {
|
|
self.spread = best_ask.price - best_bid.price;
|
|
}
|
|
}
|
|
|
|
fn mid_price(&self) -> Option<f64> {
|
|
if let (Some(best_bid), Some(best_ask)) = (self.bids.first(), self.asks.first()) {
|
|
Some((best_bid.price + best_ask.price) / 2.0)
|
|
} else {
|
|
None
|
|
}
|
|
}
|
|
}
|
|
|
|
// ============================================================================
|
|
// PRICE PARSING TESTS (15 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_scientific_notation_positive() {
|
|
let price_str = "1.23e5";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert_eq!(parsed, 123000.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_scientific_notation_negative_exponent() {
|
|
let price_str = "1.5e-3";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!((parsed - 0.0015).abs() < 1e-10);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_very_small_value() {
|
|
let price_str = "0.00000001";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!((parsed - 1e-8).abs() < 1e-15);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_very_large_value() {
|
|
let price_str = "999999999.99";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!((parsed - 999999999.99).abs() < 1e-2);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_zero_value() {
|
|
let price_str = "0.0";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert_eq!(parsed, 0.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_nan_string() {
|
|
let price_str = "NaN";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!(parsed.is_nan());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_infinity_positive() {
|
|
let price_str = "inf";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!(parsed.is_infinite() && parsed.is_sign_positive());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_parsing_infinity_negative() {
|
|
let price_str = "-inf";
|
|
let parsed = parse_price_string(price_str).unwrap();
|
|
assert!(parsed.is_infinite() && parsed.is_sign_negative());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_rejects_nan() {
|
|
let result = validate_price(f64::NAN);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_rejects_infinity() {
|
|
let result = validate_price(f64::INFINITY);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_rejects_negative_infinity() {
|
|
let result = validate_price(f64::NEG_INFINITY);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_rejects_zero() {
|
|
let result = validate_price(0.0);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_rejects_negative() {
|
|
let result = validate_price(-100.0);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_accepts_max_f64() {
|
|
let result = validate_price(f64::MAX);
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_price_validation_accepts_min_positive() {
|
|
let result = validate_price(f64::MIN_POSITIVE);
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
// ============================================================================
|
|
// SYMBOL VALIDATION TESTS (15 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_empty_string() {
|
|
let result = validate_symbol("");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_too_long() {
|
|
let symbol = "A".repeat(21);
|
|
let result = validate_symbol(&symbol);
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_max_length() {
|
|
let symbol = "A".repeat(20);
|
|
let result = validate_symbol(&symbol);
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_special_characters_slash() {
|
|
let result = validate_symbol("BTC/USD");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_special_characters_space() {
|
|
let result = validate_symbol("BTC USD");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_unicode_emoji() {
|
|
let result = validate_symbol("BTC📈USD");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_unicode_chinese() {
|
|
let result = validate_symbol("比特币");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_allowed_dot() {
|
|
let result = validate_symbol("BTC.USD");
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_allowed_dash() {
|
|
let result = validate_symbol("BTC-USD");
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_allowed_underscore() {
|
|
let result = validate_symbol("BTC_USD");
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_numeric_only() {
|
|
let result = validate_symbol("12345");
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_alphanumeric() {
|
|
let result = validate_symbol("ES50");
|
|
assert!(result.is_ok());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_whitespace_only() {
|
|
let result = validate_symbol(" ");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_leading_trailing_spaces() {
|
|
let result = validate_symbol(" BTC ");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_symbol_validation_null_byte() {
|
|
let result = validate_symbol("BTC\0USD");
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
// ============================================================================
|
|
// TIMESTAMP HANDLING TESTS (10 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_epoch_zero() {
|
|
let system_time = unix_nanos_to_system_time(0);
|
|
assert_eq!(system_time, UNIX_EPOCH);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_negative_value() {
|
|
let system_time = unix_nanos_to_system_time(-1000);
|
|
assert_eq!(system_time, UNIX_EPOCH);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_future_far() {
|
|
// Year 2100 (approximated)
|
|
let future_nanos = 4_102_444_800_000_000_000i64;
|
|
let system_time = unix_nanos_to_system_time(future_nanos);
|
|
assert!(system_time > SystemTime::now());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_y2k38_boundary() {
|
|
// 2038-01-19 03:14:07 UTC (32-bit signed int overflow)
|
|
let y2k38_nanos = 2_147_483_647i64 * 1_000_000_000;
|
|
let system_time = unix_nanos_to_system_time(y2k38_nanos);
|
|
assert!(system_time > UNIX_EPOCH);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_max_i64() {
|
|
let system_time = unix_nanos_to_system_time(i64::MAX);
|
|
assert!(system_time > UNIX_EPOCH);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_current_time_roundtrip() {
|
|
let now_nanos = current_unix_nanos();
|
|
let system_time = unix_nanos_to_system_time(now_nanos);
|
|
let diff = SystemTime::now()
|
|
.duration_since(system_time)
|
|
.unwrap_or_default();
|
|
// Allow up to 10ms difference for test execution time
|
|
assert!(diff < Duration::from_millis(10));
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_millisecond_precision() {
|
|
let base_nanos = 1_000_000_000i64; // 1 second
|
|
let millis_nanos = base_nanos + 500_000_000; // +500ms
|
|
let system_time = unix_nanos_to_system_time(millis_nanos);
|
|
let duration = system_time.duration_since(UNIX_EPOCH).unwrap();
|
|
assert_eq!(duration.as_millis(), 1500);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_microsecond_precision() {
|
|
let base_nanos = 1_000_000_000i64; // 1 second
|
|
let micros_nanos = base_nanos + 123_000_000; // +123ms = 123000μs
|
|
let system_time = unix_nanos_to_system_time(micros_nanos);
|
|
let duration = system_time.duration_since(UNIX_EPOCH).unwrap();
|
|
assert_eq!(duration.as_micros(), 1_123_000);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_ordering() {
|
|
let ts1 = current_unix_nanos();
|
|
sleep(Duration::from_millis(1)).await;
|
|
let ts2 = current_unix_nanos();
|
|
assert!(ts2 > ts1);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_timestamp_market_data_stale_detection() {
|
|
let old_timestamp = current_unix_nanos() - 5_000_000_000; // 5 seconds ago
|
|
let current = current_unix_nanos();
|
|
let staleness_threshold = 1_000_000_000; // 1 second
|
|
let is_stale = (current - old_timestamp) > staleness_threshold;
|
|
assert!(is_stale);
|
|
}
|
|
|
|
// ============================================================================
|
|
// UPDATE RATE LIMITING TESTS (15 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_high_frequency_updates_1000_per_second() {
|
|
let (tx, mut rx) = mpsc::channel(1000);
|
|
|
|
// Send 1000 updates rapidly
|
|
tokio::spawn(async move {
|
|
for i in 0..1000 {
|
|
let data = create_market_data("BTC/USD", 50000.0 + i as f64, current_unix_nanos());
|
|
let _ = tx.send(data).await;
|
|
}
|
|
});
|
|
|
|
// Count received updates
|
|
let mut count = 0;
|
|
while let Ok(result) = timeout(Duration::from_millis(100), rx.recv()).await {
|
|
if result.is_some() {
|
|
count += 1;
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
|
|
// Should receive at least 990 (allow 1% loss)
|
|
assert!(count >= 990, "Only received {} out of 1000 updates", count);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_burst_updates_handling() {
|
|
let (tx, mut rx) = mpsc::channel(100);
|
|
|
|
// Send burst of 100 updates
|
|
for i in 0..100 {
|
|
let data = create_market_data("ETH/USD", 3000.0 + i as f64, current_unix_nanos());
|
|
tx.send(data).await.unwrap();
|
|
}
|
|
|
|
// Verify all received
|
|
let mut count = 0;
|
|
while rx.try_recv().is_ok() {
|
|
count += 1;
|
|
}
|
|
assert_eq!(count, 100);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_channel_saturation_backpressure() {
|
|
let (tx, mut rx) = mpsc::channel(10); // Small buffer
|
|
|
|
// Send more than buffer size without draining
|
|
let mut send_count = 0;
|
|
for i in 0..20 {
|
|
let data = create_market_data("AAPL", 150.0 + i as f64, current_unix_nanos());
|
|
if tx.try_send(data).is_ok() {
|
|
send_count += 1;
|
|
}
|
|
}
|
|
|
|
// Should saturate at buffer size
|
|
assert!(send_count <= 10);
|
|
|
|
// Drain and verify
|
|
let mut recv_count = 0;
|
|
while rx.try_recv().is_ok() {
|
|
recv_count += 1;
|
|
}
|
|
assert_eq!(recv_count, send_count);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_delayed_updates_ordering() {
|
|
let (tx, mut rx) = mpsc::channel(100);
|
|
|
|
// Send updates with delays
|
|
tokio::spawn(async move {
|
|
for i in 0..10 {
|
|
let data = create_market_data("TSLA", 800.0 + i as f64, current_unix_nanos());
|
|
tx.send(data).await.unwrap();
|
|
sleep(Duration::from_millis(5)).await;
|
|
}
|
|
});
|
|
|
|
// Verify ordering
|
|
let mut prev_price = 0.0;
|
|
let mut ordered = true;
|
|
while let Ok(Some(data)) = timeout(Duration::from_millis(100), rx.recv()).await {
|
|
if data.price < prev_price {
|
|
ordered = false;
|
|
break;
|
|
}
|
|
prev_price = data.price;
|
|
}
|
|
assert!(ordered);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_update_rate_calculation() {
|
|
let (tx, mut rx) = mpsc::channel(100);
|
|
let start = SystemTime::now();
|
|
|
|
// Send 50 updates over 100ms
|
|
tokio::spawn(async move {
|
|
for i in 0..50 {
|
|
let data = create_market_data("SPY", 420.0 + i as f64, current_unix_nanos());
|
|
let _ = tx.send(data).await;
|
|
sleep(Duration::from_micros(2000)).await;
|
|
}
|
|
});
|
|
|
|
let mut count = 0;
|
|
while let Ok(result) = timeout(Duration::from_millis(200), rx.recv()).await {
|
|
if result.is_some() {
|
|
count += 1;
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
|
|
let elapsed = start.elapsed().unwrap();
|
|
let rate = (count as f64 / elapsed.as_secs_f64()) as u32;
|
|
// Should be roughly 500 updates/sec
|
|
assert!(rate >= 400 && rate <= 600);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_update_deduplication() {
|
|
let mut last_price = 0.0;
|
|
let mut last_timestamp = 0i64;
|
|
|
|
for i in 0..10 {
|
|
let data = create_market_data("BTC/USD", 50000.0, current_unix_nanos());
|
|
|
|
// Only process if different from last
|
|
if data.price != last_price || data.timestamp != last_timestamp {
|
|
last_price = data.price;
|
|
last_timestamp = data.timestamp;
|
|
}
|
|
}
|
|
|
|
// Should have deduplicated (only first update processed)
|
|
assert_eq!(last_price, 50000.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_update_buffer_overflow() {
|
|
let mut buffer: VecDeque<MarketDataDisplayEvent> = VecDeque::with_capacity(10);
|
|
|
|
// Add more than capacity
|
|
for i in 0..20 {
|
|
let data = create_market_data("QQQ", 350.0 + i as f64, current_unix_nanos());
|
|
if buffer.len() >= 10 {
|
|
buffer.pop_front();
|
|
}
|
|
buffer.push_back(data);
|
|
}
|
|
|
|
assert_eq!(buffer.len(), 10);
|
|
// Should have latest 10 updates (prices 360-369)
|
|
assert_eq!(buffer.back().unwrap().price, 369.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_concurrent_symbol_updates() {
|
|
let (tx, mut rx) = mpsc::channel(100);
|
|
let symbols = vec!["AAPL", "TSLA", "SPY", "QQQ", "NVDA"];
|
|
|
|
// Send updates for multiple symbols concurrently
|
|
for symbol in symbols.clone() {
|
|
let tx_clone = tx.clone();
|
|
tokio::spawn(async move {
|
|
for i in 0..10 {
|
|
let data = create_market_data(symbol, 100.0 + i as f64, current_unix_nanos());
|
|
let _ = tx_clone.send(data).await;
|
|
}
|
|
});
|
|
}
|
|
drop(tx);
|
|
|
|
// Collect all updates
|
|
let mut updates = Vec::new();
|
|
while let Some(data) = rx.recv().await {
|
|
updates.push(data);
|
|
}
|
|
|
|
// Should receive all 50 updates
|
|
assert_eq!(updates.len(), 50);
|
|
|
|
// Verify all symbols present
|
|
for symbol in symbols {
|
|
let count = updates.iter().filter(|u| u.symbol == symbol).count();
|
|
assert_eq!(count, 10);
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_update_latency_tracking() {
|
|
let mut latencies = Vec::new();
|
|
|
|
for _ in 0..10 {
|
|
let sent_time = current_unix_nanos();
|
|
sleep(Duration::from_micros(100)).await;
|
|
let recv_time = current_unix_nanos();
|
|
latencies.push(recv_time - sent_time);
|
|
}
|
|
|
|
let avg_latency = latencies.iter().sum::<i64>() / latencies.len() as i64;
|
|
// Should be around 100μs (100,000 nanos)
|
|
assert!(avg_latency > 50_000 && avg_latency < 200_000);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_missed_update_detection() {
|
|
let mut sequence_numbers = Vec::new();
|
|
|
|
// Simulate missing sequence number
|
|
for i in 0..10 {
|
|
if i != 5 {
|
|
// Skip 5
|
|
sequence_numbers.push(i);
|
|
}
|
|
}
|
|
|
|
// Check for gaps
|
|
let mut has_gap = false;
|
|
for window in sequence_numbers.windows(2) {
|
|
if window[1] - window[0] > 1 {
|
|
has_gap = true;
|
|
break;
|
|
}
|
|
}
|
|
|
|
assert!(has_gap);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_update_compression() {
|
|
// Test that identical updates can be compressed
|
|
let updates = vec![
|
|
create_market_data("BTC/USD", 50000.0, current_unix_nanos()),
|
|
create_market_data("BTC/USD", 50000.0, current_unix_nanos()),
|
|
create_market_data("BTC/USD", 50001.0, current_unix_nanos()),
|
|
create_market_data("BTC/USD", 50001.0, current_unix_nanos()),
|
|
];
|
|
|
|
// Compress consecutive duplicates
|
|
let mut compressed = Vec::new();
|
|
let mut last_price = -1.0;
|
|
for update in updates {
|
|
if update.price != last_price {
|
|
compressed.push(update.clone());
|
|
last_price = update.price;
|
|
}
|
|
}
|
|
|
|
assert_eq!(compressed.len(), 2); // Only 2 unique prices
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_rate_limiter_token_bucket() {
|
|
// Simple token bucket rate limiter simulation
|
|
let mut tokens = 100.0;
|
|
let max_tokens = 100.0;
|
|
let refill_rate = 10.0; // tokens per second
|
|
|
|
let mut accepted = 0;
|
|
let mut rejected = 0;
|
|
|
|
for _ in 0..150 {
|
|
if tokens >= 1.0 {
|
|
tokens -= 1.0;
|
|
accepted += 1;
|
|
} else {
|
|
rejected += 1;
|
|
}
|
|
// Simulate refill
|
|
tokens = f64::min(tokens + refill_rate / 150.0, max_tokens as f64);
|
|
}
|
|
|
|
// Should accept roughly 100-110 requests
|
|
assert!(accepted >= 100 && accepted <= 120);
|
|
assert!(rejected >= 30 && rejected <= 50);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_sliding_window_rate_limiter() {
|
|
let window_size = Duration::from_millis(100);
|
|
let max_requests = 10;
|
|
let mut timestamps = Vec::new();
|
|
|
|
let start = SystemTime::now();
|
|
for i in 0..20 {
|
|
let now = SystemTime::now();
|
|
// Remove timestamps outside window
|
|
timestamps.retain(|&ts: &SystemTime| now.duration_since(ts).unwrap() < window_size);
|
|
|
|
if timestamps.len() < max_requests {
|
|
timestamps.push(now);
|
|
// Request accepted
|
|
} else {
|
|
// Request rejected
|
|
}
|
|
|
|
if i < 10 {
|
|
sleep(Duration::from_millis(5)).await;
|
|
}
|
|
}
|
|
|
|
// First 10 should be accepted, rest rejected
|
|
assert!(timestamps.len() <= max_requests);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_adaptive_rate_limiting() {
|
|
let mut rate_limit = 100; // Initial limit
|
|
let mut errors = 0;
|
|
|
|
for i in 0..200 {
|
|
if i % rate_limit == 0 {
|
|
// Simulate checking if we hit rate limit
|
|
if errors > 5 {
|
|
// Reduce rate if too many errors
|
|
rate_limit = (rate_limit as f64 * 0.8) as usize;
|
|
errors = 0;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Rate limit should adapt
|
|
assert!(rate_limit < 100);
|
|
}
|
|
|
|
// ============================================================================
|
|
// ORDER BOOK EDGE CASES (15 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_empty_levels() {
|
|
let mut book = create_order_book_snapshot(vec![], vec![]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.bids.len(), 0);
|
|
assert_eq!(book.asks.len(), 0);
|
|
assert_eq!(book.mid_price(), None);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_only_bids() {
|
|
let mut book = create_order_book_snapshot(vec![(100.0, 50.0, 1)], vec![]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.bids.len(), 1);
|
|
assert_eq!(book.asks.len(), 0);
|
|
assert_eq!(book.mid_price(), None);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_only_asks() {
|
|
let mut book = create_order_book_snapshot(vec![], vec![(100.0, 50.0, 1)]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.bids.len(), 0);
|
|
assert_eq!(book.asks.len(), 1);
|
|
assert_eq!(book.mid_price(), None);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_single_level_each_side() {
|
|
let mut book = create_order_book_snapshot(vec![(99.0, 100.0, 1)], vec![(100.0, 100.0, 1)]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.spread, 1.0);
|
|
assert!(book.mid_price().is_some());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_100_levels() {
|
|
let bids: Vec<_> = (0..100)
|
|
.map(|i| (100.0 - i as f64 * 0.01, 100.0, 1))
|
|
.collect();
|
|
let asks: Vec<_> = (0..100)
|
|
.map(|i| (100.01 + i as f64 * 0.01, 100.0, 1))
|
|
.collect();
|
|
|
|
let book = create_order_book_snapshot(bids, asks);
|
|
|
|
assert_eq!(book.bids.len(), 100);
|
|
assert_eq!(book.asks.len(), 100);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_zero_size_levels() {
|
|
let mut book = create_order_book_snapshot(vec![(100.0, 0.0, 1)], vec![(101.0, 0.0, 1)]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.bids[0].quantity, 0.0);
|
|
assert_eq!(book.asks[0].quantity, 0.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_very_large_size() {
|
|
let large_size = 1_000_000_000.0;
|
|
let book =
|
|
create_order_book_snapshot(vec![(100.0, large_size, 1)], vec![(101.0, large_size, 1)]);
|
|
|
|
assert!(book.bids[0].quantity > 0.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_very_tight_spread() {
|
|
let mut book = create_order_book_snapshot(vec![(100.000, 100.0, 1)], vec![(100.001, 100.0, 1)]);
|
|
book.calculate_spread();
|
|
|
|
assert!(book.spread < 1.0);
|
|
assert!(book.spread > 0.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_very_wide_spread() {
|
|
let mut book = create_order_book_snapshot(vec![(50.0, 100.0, 1)], vec![(100.0, 100.0, 1)]);
|
|
book.calculate_spread();
|
|
|
|
assert_eq!(book.spread, 50.0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_crossed_book() {
|
|
// Invalid state: best bid > best ask
|
|
let book = create_order_book_snapshot(vec![(101.0, 100.0, 1)], vec![(100.0, 100.0, 1)]);
|
|
|
|
// Should still calculate (negative spread indicates crossed book)
|
|
let bid_price = book.bids[0].price;
|
|
let ask_price = book.asks[0].price;
|
|
assert!(bid_price > ask_price);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_duplicate_price_levels() {
|
|
let bids = vec![(100.0, 50.0, 1), (100.0, 30.0, 2), (99.0, 100.0, 1)];
|
|
let book = create_order_book_snapshot(bids, vec![]);
|
|
|
|
// Should handle duplicates (may aggregate or keep separate)
|
|
assert!(book.bids.len() >= 2);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_out_of_order_levels() {
|
|
let unsorted_bids = vec![(99.0, 100.0, 1), (100.0, 100.0, 1), (98.0, 100.0, 1)];
|
|
let book = create_order_book_snapshot(unsorted_bids, vec![]);
|
|
|
|
// Should work regardless of input order
|
|
assert_eq!(book.bids.len(), 3);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_zero_count() {
|
|
let book = create_order_book_snapshot(vec![(100.0, 100.0, 0)], vec![]);
|
|
|
|
assert_eq!(book.bids[0].order_count, 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_large_count() {
|
|
let book = create_order_book_snapshot(vec![(100.0, 100.0, 10000)], vec![]);
|
|
|
|
assert_eq!(book.bids[0].order_count, 10000);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_order_book_snapshot_update() {
|
|
let mut book = create_order_book_snapshot(vec![(100.0, 100.0, 1)], vec![(101.0, 100.0, 1)]);
|
|
let old_timestamp = book.timestamp;
|
|
|
|
sleep(Duration::from_millis(1)).await;
|
|
|
|
// Update snapshot
|
|
book.timestamp = Utc::now();
|
|
assert!(book.timestamp > old_timestamp);
|
|
}
|
|
|
|
// ============================================================================
|
|
// TRADE HISTORY TESTS (5 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_trade_history_zero_trades() {
|
|
let trades: Vec<MarketDataDisplayEvent> = vec![];
|
|
assert_eq!(trades.len(), 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_trade_history_zero_volume() {
|
|
let trade = MarketDataDisplayEvent {
|
|
symbol: "BTC/USD".to_string(),
|
|
price: 50000.0,
|
|
volume: 0,
|
|
timestamp: current_unix_nanos(),
|
|
bid: Some(49999.0),
|
|
ask: Some(50001.0),
|
|
change: Some(0.0),
|
|
change_percent: Some(0.0),
|
|
};
|
|
|
|
assert_eq!(trade.volume, 0);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_trade_history_10k_trades() {
|
|
let mut trades = Vec::with_capacity(10000);
|
|
|
|
for i in 0..10000 {
|
|
let trade = create_market_data("ETH/USD", 3000.0 + i as f64 * 0.01, current_unix_nanos());
|
|
trades.push(trade);
|
|
}
|
|
|
|
assert_eq!(trades.len(), 10000);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_trade_history_memory_efficiency() {
|
|
// Test circular buffer for trade history
|
|
let capacity = 1000;
|
|
let mut trades: VecDeque<MarketDataDisplayEvent> = VecDeque::with_capacity(capacity);
|
|
|
|
for i in 0..5000 {
|
|
if trades.len() >= capacity {
|
|
trades.pop_front();
|
|
}
|
|
let trade = create_market_data("SPY", 420.0 + i as f64 * 0.01, current_unix_nanos());
|
|
trades.push_back(trade);
|
|
}
|
|
|
|
assert_eq!(trades.len(), capacity);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_trade_history_time_range_filter() {
|
|
let now = current_unix_nanos();
|
|
let hour_ago = now - 3_600_000_000_000; // 1 hour in nanos
|
|
|
|
let mut trades = vec![
|
|
create_market_data("BTC/USD", 50000.0, hour_ago),
|
|
create_market_data("BTC/USD", 50100.0, hour_ago + 1_800_000_000_000), // 30 min ago
|
|
create_market_data("BTC/USD", 50200.0, now),
|
|
];
|
|
|
|
// Filter trades from last 45 minutes
|
|
let threshold = now - 2_700_000_000_000; // 45 min
|
|
trades.retain(|t| t.timestamp >= threshold);
|
|
|
|
assert_eq!(trades.len(), 2);
|
|
}
|
|
|
|
// ============================================================================
|
|
// CONNECTION INTERRUPTION TESTS (5 tests)
|
|
// ============================================================================
|
|
|
|
#[tokio::test]
|
|
async fn test_connection_loss_recovery() {
|
|
let (tx, mut rx) = mpsc::channel(10);
|
|
|
|
// Send some data
|
|
for i in 0..5 {
|
|
tx.send(create_market_data(
|
|
"BTC/USD",
|
|
50000.0 + i as f64,
|
|
current_unix_nanos(),
|
|
))
|
|
.await
|
|
.unwrap();
|
|
}
|
|
|
|
// Simulate disconnection (drop sender)
|
|
drop(tx);
|
|
|
|
// Verify channel closed
|
|
assert!(rx.recv().await.is_some()); // Drain existing
|
|
while rx.try_recv().is_ok() {} // Drain all
|
|
assert!(rx.recv().await.is_none()); // Channel closed
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_reconnection_backoff() {
|
|
let retry_delays = vec![100, 200, 400, 800, 1600]; // Exponential backoff in ms
|
|
let mut total_delay = 0;
|
|
|
|
for delay in retry_delays {
|
|
total_delay += delay;
|
|
// Simulate retry delay
|
|
sleep(Duration::from_millis(delay as u64)).await;
|
|
}
|
|
|
|
// Total delay should be sum of all retries
|
|
assert_eq!(total_delay, 3100);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_connection_heartbeat_timeout() {
|
|
let timeout_duration = Duration::from_millis(50);
|
|
let (tx, mut rx) = mpsc::channel(10);
|
|
|
|
// Send heartbeat
|
|
tx.send(create_market_data("HEARTBEAT", 0.0, current_unix_nanos()))
|
|
.await
|
|
.unwrap();
|
|
|
|
// Wait for timeout
|
|
let result = timeout(timeout_duration, rx.recv()).await;
|
|
assert!(result.is_ok()); // Should receive heartbeat
|
|
|
|
// Wait again without sending
|
|
let result = timeout(timeout_duration, rx.recv()).await;
|
|
assert!(result.is_err()); // Should timeout
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_missed_updates_during_disconnect() {
|
|
let mut sequence = 0u64;
|
|
let mut expected = 0u64;
|
|
let mut missed = 0u64;
|
|
|
|
// Simulate sequence with gap
|
|
let sequences = vec![0, 1, 2, 3, 7, 8, 9]; // Missing 4, 5, 6
|
|
|
|
for seq in sequences {
|
|
sequence = seq;
|
|
if sequence != expected {
|
|
missed += sequence - expected;
|
|
}
|
|
expected = sequence + 1;
|
|
}
|
|
|
|
assert_eq!(missed, 3);
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_connection_backfill_request() {
|
|
// Simulate requesting backfill for missed data
|
|
let last_received_sequence = 100u64;
|
|
let current_sequence = 150u64;
|
|
|
|
let backfill_needed = current_sequence > last_received_sequence + 1;
|
|
let backfill_count = if backfill_needed {
|
|
current_sequence - last_received_sequence - 1
|
|
} else {
|
|
0
|
|
};
|
|
|
|
assert!(backfill_needed);
|
|
assert_eq!(backfill_count, 49);
|
|
}
|