BREAKING CHANGES: - Removed orphaned dqn.rs monolithic trainer (4,975 lines) - Removed orphaned dqn_ensemble.rs module (816 lines) - Removed orphaned tft.rs and tft_complete_int8_integration_test.rs - TFT trainer split into modular directory structure DQN Module Refactoring: - Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs) - Fixed hyperopt 39D search space (continuous params only) - Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions - use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues) Clean Module Structure: - ml/src/trainers/dqn/ directory with proper mod.rs exports - ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs - All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness Documentation: - Added comprehensive docs in docs/codebase-cleanup/ - ADR-001 for DQN refactoring decisions - Rainbow DQN component matrix and quick reference guides Build Status: Compiles with zero errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
6.6 KiB
6.6 KiB
Test Fixtures Migration Guide
Quick reference for migrating test files to use the new test_common crate.
Quick Start
1. Add Dependency
In your test crate's Cargo.toml:
[dev-dependencies]
test_common = { path = "../../tests/test_common" }
2. Common Migrations
Database Setup
Before:
async fn setup_test_db() -> PgPool {
let database_url = std::env::var("DATABASE_URL")
.unwrap_or_else(|_| "postgresql://foxhunt:foxhunt_dev_password@localhost:5432/foxhunt".to_string());
PgPool::connect(&database_url).await.unwrap()
}
#[tokio::test]
async fn my_test() {
let pool = setup_test_db().await;
// ...
}
After:
use test_common::TestDb;
#[tokio::test]
async fn my_test() {
let db = TestDb::with_migrations().await;
let pool = db.pool();
// ...
}
Mock Orders
Before:
fn create_mock_order() -> Order {
Order {
id: uuid::Uuid::new_v4().to_string(),
symbol: "AAPL".to_string(),
quantity: 100.0,
price: Some(150.0),
side: Side::Buy,
order_type: OrderType::Limit,
status: OrderStatus::Pending,
timestamp: Utc::now(),
filled_quantity: 0.0,
average_fill_price: None,
}
}
After:
use test_common::MockOrderBuilder;
let order = MockOrderBuilder::new()
.symbol("AAPL")
.buy()
.quantity(100.0)
.limit_price(150.0)
.build();
OHLCV Bars
Before:
fn create_mock_bars(count: usize) -> Vec<OHLCVBar> {
let mut bars = Vec::with_capacity(count);
let base_price = 100.0;
let base_time = chrono::Utc::now();
for i in 0..count {
let change = ((i as f64 / 100.0).sin() * 0.02);
let price = base_price * (1.0 + change);
bars.push(OHLCVBar {
timestamp: base_time + chrono::Duration::minutes((i * 5) as i64),
open: price,
high: price * 1.01,
low: price * 0.99,
close: price,
volume: 1000 + (i * 10) as u64,
});
}
bars
}
After:
use test_common::generate_ohlcv_bars;
let bars = generate_ohlcv_bars(100, 100.0);
Crisis Scenarios (VaR Testing)
Before:
let crisis_returns = vec![
dec!(-0.08), dec!(-0.10), dec!(-0.07), dec!(-0.12),
// ... 16 more lines
];
After:
use test_common::fixtures::market_data::generate_crisis_returns;
let crisis_returns = generate_crisis_returns();
Migration Checklist
For each test file:
- Add
test_commontoCargo.toml - Replace
setup_test_db()withTestDb - Replace
create_mock_order()withMockOrderBuilder - Replace
create_mock_bars()withgenerate_ohlcv_bars() - Replace crisis scenarios with
generate_*_returns() - Replace custom assertions with
test_common::assert_*!() - Remove old helper functions
- Run tests:
cargo test - Count LOC saved
Examples by Test Type
ML Tests
// Before: 60+ LOC of setup
async fn setup_test_orchestrator() -> Arc<TrainingOrchestrator> {
let ml_config = MLConfig::default();
let db_config = create_test_database_config().await;
// ... 50 more lines
}
// After: 8 LOC
use test_common::{TestDb, fixtures::config::MLTestConfig};
let db = TestDb::with_migrations().await;
let config = MLTestConfig::default_test();
Risk Tests
// Before: 40+ LOC
fn create_crisis_scenario() -> Vec<Decimal> {
vec![
dec!(-0.08), dec!(-0.10), // ... 18 more
]
}
fn assert_var_valid(returns: &[Decimal], var: Decimal) {
let exceedances = returns.iter()
.filter(|&&r| r.abs() > var)
.count();
assert!(exceedances <= 2);
}
// After: 5 LOC
use test_common::{
fixtures::market_data::generate_crisis_returns,
assertions::assert_var_exceedances,
};
let returns = generate_crisis_returns();
assert_var_exceedances(&returns, var, 0.99, 0.1);
Integration Tests
// Before: 80+ LOC of database + orchestrator setup
// After: 15 LOC
use test_common::{TestDb, generate_ohlcv_bars, MockOrderBuilder};
#[tokio::test]
async fn end_to_end_workflow() {
let db = TestDb::with_migrations().await;
let bars = generate_ohlcv_bars(100, 150.0);
let order = MockOrderBuilder::new()
.symbol("AAPL")
.buy()
.build();
// Test logic...
}
Common Patterns
Builder Pattern
All builders follow the same pattern:
let result = Builder::new()
.property1(value1)
.property2(value2)
.build();
Assertions
Domain-specific assertions:
// Percentage tolerance
assert_approx_eq!(actual, expected, 0.02); // 2% tolerance
// OHLC validation
assert_ohlc_valid(open, high, low, close);
// VaR exceedances
assert_var_exceedances(&returns, var, confidence, tolerance);
// P&L validation
assert_pnl(entry_price, current_price, quantity, expected, tolerance);
Troubleshooting
Import Errors
// If you get "can't find crate test_common"
// Check Cargo.toml has correct path:
[dev-dependencies]
test_common = { path = "../../tests/test_common" }
Type Mismatches
// MockOrder vs actual Order type
// The mock is for testing only, map to your domain type:
let mock = MockOrderBuilder::new().build();
let domain_order = Order {
id: mock.id,
symbol: Symbol::from_str(&mock.symbol).unwrap(),
// ... map other fields
};
Database Connection Issues
// If tests fail with connection errors:
// 1. Ensure DATABASE_URL is set
// 2. Ensure PostgreSQL is running
// 3. Check connection pool limits
// Each TestDb creates isolated schema
let db = TestDb::with_migrations().await;
// Schema is automatically cleaned up on drop
Migration Progress Tracking
Create a tracking document:
## Test Migration Progress
### ML Training Service (18 files)
- [ ] checkpoint_manager_tests.rs (~30 LOC saved)
- [ ] job_tracker_test.rs (~25 LOC saved)
- [ ] integration_tests.rs (~40 LOC saved)
... (15 more)
### Risk Tests (18 files)
- [ ] risk_comprehensive_tests.rs (~80 LOC saved)
- [ ] var_calculations_tests.rs (~60 LOC saved)
... (16 more)
Total Estimated Savings: XXX LOC
Best Practices
- Migrate by directory: Complete one directory before moving to next
- Run tests frequently: After each migration
- Commit often: One file or small group per commit
- Document savings: Track LOC removed in commit messages
- Update imports: Remove unused imports after migration
Need Help?
- See
/tests/test_common/README.mdfor full documentation - Check
/docs/test_fixtures_consolidation_report.mdfor analysis - Review existing tests in
test_common/src/*/testsfor examples