61801cfd06
**Wave D Phase 6 - Technical Debt Cleanup (Agent C6)** ## Changes - Identified deprecated code patterns across codebase - Analyzed mock repository usage (strategically retained per AGENT_M13) - Documented deprecation cleanup strategy - Prepared deprecation removal todos ## Analysis Results - Mock structs: RETAINED (strategic testing infrastructure) - Never-read fields: 2 instances in backtesting_service - Dead code warnings: 35 total across workspace - databento_old references: None found in active code ## Status - ✅ Deprecation analysis complete - ⏳ Cleanup execution pending user confirmation - 📊 Test impact assessment ready 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
477 lines
24 KiB
Markdown
477 lines
24 KiB
Markdown
# Backtesting Service Architecture Diagrams
|
|
|
|
## 1. Overall Architecture
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────────────┐
|
|
│ │
|
|
│ SERVICE LAYER (Business Logic) │
|
|
│ │
|
|
│ ┌─────────────────────┐ ┌──────────────────────┐ ┌──────────────┐ │
|
|
│ │ BacktestingService │ │ StrategyEngine │ │ Wave │ │
|
|
│ │ Impl │ │ │ │ Comparison │ │
|
|
│ └──────────┬──────────┘ └──────────┬───────────┘ └──────┬───────┘ │
|
|
│ │ │ │ │
|
|
│ ▼ ▼ ▼ │
|
|
│ ┌──────────────────────────────────────────────────────────────┐ │
|
|
│ │ repositories: Arc<dyn BacktestingRepositories> │ │
|
|
│ │ [Depends on trait abstraction, not concrete types] │ │
|
|
│ └───────────────────────────┬────────────────────────────────┘ │
|
|
│ │ │
|
|
│ │ Injected at construction time │
|
|
└──────────────────────────────┼──────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────────┐
|
|
│ │
|
|
│ TRAIT LAYER (Contracts) │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ pub trait BacktestingRepositories: Send + Sync │ │
|
|
│ │ { │ │
|
|
│ │ fn market_data(&self) -> &dyn MarketDataRepository; │ │
|
|
│ │ fn trading(&self) -> &dyn TradingRepository; │ │
|
|
│ │ fn news(&self) -> &dyn NewsRepository; │ │
|
|
│ │ fn mock() -> Self where Self: Sized; │ │
|
|
│ │ } │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ Individual Trait Contracts: │
|
|
│ ┌──────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
|
│ │ MarketDataRepo │ │ TradingRepo │ │ NewsRepository │ │
|
|
│ ├──────────────────┤ ├─────────────────┤ ├─────────────────┤ │
|
|
│ │ • load_hist_data │ │ • save_results │ │ • load_events │ │
|
|
│ │ • check_avail │ │ • load_results │ │ • get_sentiment │ │
|
|
│ │ │ │ • create_record │ │ │ │
|
|
│ │ │ │ • update_status │ │ │ │
|
|
│ │ │ │ • list_backtests│ │ │ │
|
|
│ │ │ │ • store_timeseries │ │
|
|
│ └──────────────────┘ └─────────────────┘ └─────────────────┘ │
|
|
└──────────────────────────────────────────────────────────────────────────┘
|
|
▲
|
|
│ Implemented by
|
|
│
|
|
┌──────────────────────────────────────────────────────────────────────────┐
|
|
│ │
|
|
│ DATA LAYER (Implementations) │
|
|
│ │
|
|
│ COMPOSITE IMPLEMENTATION: │
|
|
│ ┌────────────────────────────────────────────────────────┐ │
|
|
│ │ pub struct DefaultRepositories { │ │
|
|
│ │ market_data: Box<dyn MarketDataRepository>, │ │
|
|
│ │ trading: Box<dyn TradingRepository>, │ │
|
|
│ │ news: Box<dyn NewsRepository>, │ │
|
|
│ │ } │ │
|
|
│ └────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
│ MARKET DATA IMPLEMENTATIONS: │
|
|
│ ┌──────────────────────┐ ┌────────────────┐ ┌──────────────┐ │
|
|
│ │ DataProvider Market │ │ Dbn Market │ │ Mock │ │
|
|
│ │ DataRepository │ │ DataRepository │ │ MarketData │ │
|
|
│ │ (Databento API) │ │ (Local DBN) │ │ Repository │ │
|
|
│ └──────────────────────┘ └────────────────┘ └──────────────┘ │
|
|
│ │
|
|
│ TRADING IMPLEMENTATIONS: │
|
|
│ ┌──────────────────────┐ ┌──────────────────────────────┐ │
|
|
│ │ StorageManager │ │ Mock Trading │ │
|
|
│ │ TradingRepository │ │ Repository │ │
|
|
│ │ (PostgreSQL) │ │ (In-memory) │ │
|
|
│ └──────────────────────┘ └──────────────────────────────┘ │
|
|
│ │
|
|
│ NEWS IMPLEMENTATIONS: │
|
|
│ ┌──────────────────────┐ ┌──────────────────────────────┐ │
|
|
│ │ BenzingaNews │ │ Mock News │ │
|
|
│ │ Repository │ │ Repository │ │
|
|
│ │ (Benzinga API) │ │ (In-memory) │ │
|
|
│ └──────────────────────┘ └──────────────────────────────┘ │
|
|
│ │
|
|
└──────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Dependency Injection Flow
|
|
|
|
```
|
|
┌────────────────────────────────────────────────────────────────┐
|
|
│ main.rs - Service Initialization │
|
|
└────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────┐
|
|
│ create_repositories() │
|
|
│ [Factory Function] │
|
|
└──────────────┬──────────────────────┘
|
|
│
|
|
┌─────────────┴─────────────┐
|
|
│ │
|
|
▼ ▼
|
|
USE_DBN_DATA USE_DBN_DATA
|
|
== true == false
|
|
│ │
|
|
▼ ▼
|
|
DbnMarketData DataProvider
|
|
Repository MarketDataRepository
|
|
│ │
|
|
└─────────────┬─────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────┐
|
|
│ StorageManagerTrading │
|
|
│ Repository (always) │
|
|
└──────────────┬───────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────┐
|
|
│ BenzingaNewsRepository │
|
|
│ (always) │
|
|
└──────────────┬───────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────────────────┐
|
|
│ DefaultRepositories │
|
|
│ { │
|
|
│ market_data: Box<dyn ...>, │
|
|
│ trading: Box<dyn ...>, │
|
|
│ news: Box<dyn ...>, │
|
|
│ } │
|
|
└──────────────┬───────────────────────────┘
|
|
│
|
|
▼
|
|
┌────────────────────────────────────────────┐
|
|
│ Arc::new(DefaultRepositories { ... }) │
|
|
│ [Thread-safe shared reference] │
|
|
└────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌────────────────────────────────────────────┐
|
|
│ BacktestingServiceImpl::new( │
|
|
│ repositories: Arc<dyn ...> │
|
|
│ ) │
|
|
│ [Dependency Injection] │
|
|
└────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────┴─────────────┐
|
|
│ │
|
|
▼ ▼
|
|
StrategyEngine::new() Service ready
|
|
[Also receives repos] for gRPC
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Repository Interface Hierarchy
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ BacktestingRepositories (Composite) │
|
|
│ │
|
|
│ Methods: │
|
|
│ • market_data() -> &dyn MarketDataRepository │
|
|
│ • trading() -> &dyn TradingRepository │
|
|
│ • news() -> &dyn NewsRepository │
|
|
│ • mock() -> Self │
|
|
└──────────────────────┬──────────────────────────────────────┘
|
|
│
|
|
┌───────────────┼───────────────┐
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌─────────────────┐ ┌─────────────┐ ┌──────────────┐
|
|
│ MarketData │ │ Trading │ │ News │
|
|
│ Repository │ │ Repository │ │ Repository │
|
|
├─────────────────┤ ├─────────────┤ ├──────────────┤
|
|
│ • load_hist │ │ • save │ │ • load_events│
|
|
│ • check_avail │ │ • load │ │ • sentiment │
|
|
│ │ │ • create │ │ │
|
|
│ │ │ • update │ │ │
|
|
│ │ │ • list │ │ │
|
|
│ │ │ • store_ts │ │ │
|
|
└────────┬────────┘ └─────┬───────┘ └──────┬───────┘
|
|
│ │ │
|
|
┌───┴────────┐ ┌───┴───┐ ┌───┴─────┐
|
|
│ │ │ │ │ │
|
|
▼ ▼ ▼ ▼ ▼ ▼
|
|
DataProv DbnRepo Storage Mock Benzinga Mock
|
|
impl impl impl impl impl impl
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Test Setup Flow
|
|
|
|
```
|
|
PRODUCTION PATH:
|
|
────────────────
|
|
|
|
let storage = Arc::new(StorageManager::new(&config).await?);
|
|
│
|
|
▼
|
|
let repos = Arc::new(create_repositories(storage).await?);
|
|
│
|
|
▼
|
|
[Factory selects impls based on env]
|
|
│
|
|
▼
|
|
let service = BacktestingServiceImpl::new(repos, None).await?;
|
|
│
|
|
▼
|
|
[Service ready with real data sources]
|
|
|
|
|
|
TESTING PATH:
|
|
──────────────
|
|
|
|
let repos = Arc::new(DefaultRepositories::mock());
|
|
│
|
|
▼
|
|
[Mocks all three repositories]
|
|
│
|
|
▼
|
|
let backtest = WaveComparisonBacktest::new(repos, 100_000.0);
|
|
│
|
|
▼
|
|
[Backtest engine with mock data sources]
|
|
```
|
|
|
|
---
|
|
|
|
## 5. File Organization Map
|
|
|
|
```
|
|
backtesting_service/src/
|
|
│
|
|
├─ repositories.rs (302 lines)
|
|
│ │
|
|
│ ├─ trait BacktestingRepositories [1-49]
|
|
│ │ ├─ fn market_data()
|
|
│ │ ├─ fn trading()
|
|
│ │ ├─ fn news()
|
|
│ │ └─ fn mock()
|
|
│ │
|
|
│ ├─ trait MarketDataRepository [18-45]
|
|
│ ├─ trait TradingRepository [52-108]
|
|
│ ├─ trait NewsRepository [115-132]
|
|
│ │
|
|
│ ├─ struct DefaultRepositories [156-163]
|
|
│ │ impl BacktestingRepositories for DefaultRepositories
|
|
│ │
|
|
│ └─ Mocks [189-302]
|
|
│ ├─ MockMarketDataRepository [191-212]
|
|
│ ├─ MockTradingRepository [215-277]
|
|
│ └─ MockNewsRepository [280-301]
|
|
│
|
|
├─ repository_impl.rs (366 lines)
|
|
│ │
|
|
│ ├─ struct DataProviderMarketDataRepository [24-40]
|
|
│ │ └─ impl MarketDataRepository [42-105]
|
|
│ │
|
|
│ ├─ struct StorageManagerTradingRepository [108-117]
|
|
│ │ └─ impl TradingRepository [120-200]
|
|
│ │
|
|
│ ├─ struct BenzingaNewsRepository [203-216]
|
|
│ │ └─ impl NewsRepository [219-286]
|
|
│ │
|
|
│ └─ fn create_repositories() [297-365]
|
|
│
|
|
├─ dbn_repository.rs (150+ lines)
|
|
│ │
|
|
│ └─ struct DbnMarketDataRepository [49-55]
|
|
│ ├─ fn new()
|
|
│ ├─ fn new_with_mappings()
|
|
│ └─ impl MarketDataRepository
|
|
│
|
|
├─ service.rs (main service)
|
|
│ │
|
|
│ └─ struct BacktestingServiceImpl
|
|
│ ├─ repositories: Arc<dyn BacktestingRepositories>
|
|
│ └─ impl BacktestingService
|
|
│
|
|
├─ wave_comparison.rs (comparison engine)
|
|
│ │
|
|
│ └─ struct WaveComparisonBacktest
|
|
│ ├─ repositories: Arc<dyn BacktestingRepositories>
|
|
│ └─ async fn run_comparison()
|
|
│
|
|
├─ main.rs (service initialization)
|
|
│ │
|
|
│ ├─ create_repositories() call
|
|
│ ├─ BacktestingServiceImpl::new()
|
|
│ └─ gRPC server setup
|
|
│
|
|
└─ lib.rs (public API)
|
|
│
|
|
├─ pub mod repositories
|
|
├─ pub mod repository_impl
|
|
└─ pub mod wave_comparison
|
|
```
|
|
|
|
---
|
|
|
|
## 6. Implementation Selection Logic
|
|
|
|
```
|
|
CREATE_REPOSITORIES LOGIC:
|
|
──────────────────────────
|
|
|
|
┌─ Check environment: USE_DBN_DATA
|
|
│
|
|
├─ If USE_DBN_DATA == "true"
|
|
│ │
|
|
│ ├─ Parse DBN_SYMBOL_MAPPINGS env var
|
|
│ │ └─ Format: "ES.FUT:/path/to/es.dbn,NQ.FUT:/path/to/nq.dbn"
|
|
│ │
|
|
│ ├─ Parse DBN_SYMBOL_MAP env var (optional)
|
|
│ │ └─ Format: "BTC/USD:ES.FUT" (for remapping)
|
|
│ │
|
|
│ └─ Create DbnMarketDataRepository with mappings
|
|
│
|
|
├─ Else (default)
|
|
│ │
|
|
│ └─ Create DataProviderMarketDataRepository (Databento API)
|
|
│
|
|
├─ Always create StorageManagerTradingRepository (PostgreSQL)
|
|
│
|
|
├─ Always create BenzingaNewsRepository (Benzinga API)
|
|
│
|
|
└─ Combine into DefaultRepositories
|
|
└─ Return Arc<DefaultRepositories>
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Type Relationships
|
|
|
|
```
|
|
Concrete Types Trait Types Usage
|
|
────────────────── ───────────── ──────
|
|
|
|
DataProviderMarket ──▶ MarketDataRepository ──▶ BacktestingService
|
|
DataRepository WaveComparison
|
|
StrategyEngine
|
|
|
|
DbnMarketData ──▶ MarketDataRepository
|
|
Repository
|
|
|
|
StorageManagerTrad ──▶ TradingRepository
|
|
ingRepository
|
|
|
|
BenzingaNews ──▶ NewsRepository
|
|
Repository
|
|
|
|
Mock* (3 types) ──▶ Individual traits
|
|
|
|
DefaultRepositories ──▶ BacktestingRepositories ──▶ Service Layer
|
|
(composite) └─▶ All sub-traits
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Error Propagation
|
|
|
|
```
|
|
Service Layer
|
|
├─ BacktestingServiceImpl::method()
|
|
│ ├─ repositories.market_data()
|
|
│ │ ├─ returns Result<Vec<MarketData>>
|
|
│ │ ├─? Error
|
|
│ │ │ └─ Wrapped in anyhow::Error
|
|
│ │ │ └─ .context("Failed to load market data")
|
|
│ │ │ └─ Propagated to caller
|
|
│ │ │ └─ gRPC error response
|
|
│ │ │
|
|
│ │ └─ Ok
|
|
│ │ └─ Process data
|
|
│ │ └─ Continue
|
|
│
|
|
Data Layer
|
|
├─ Concrete Repository Implementation
|
|
│ ├─ DataProviderMarketDataRepository::load_historical_data()
|
|
│ │ ├─ Call Databento API
|
|
│ │ ├─? Network error
|
|
│ │ │ └─ Return Err(anyhow::anyhow!("API error"))
|
|
│ │ │
|
|
│ │ └─ Parse response
|
|
│ │ ├─? Parse error
|
|
│ │ │ └─ Return Err(e)
|
|
│ │ │
|
|
│ │ └─ Ok(Vec<MarketData>)
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Concurrency Model
|
|
|
|
```
|
|
Request from gRPC Client
|
|
│
|
|
▼
|
|
BacktestingServiceImpl::run_backtest()
|
|
│
|
|
├─ Arc::clone(repositories)
|
|
│ └─ Atomic reference count increment
|
|
│
|
|
├─ Spawn task 1: Load market data
|
|
│ └─ repositories.market_data().load_historical_data()
|
|
│ └─ May be API or file-based
|
|
│ └─ Fully async (no blocking)
|
|
│
|
|
├─ Spawn task 2: Load trading history
|
|
│ └─ repositories.trading().load_backtest_results()
|
|
│ └─ Async database query
|
|
│
|
|
├─ Spawn task 3: Load news
|
|
│ └─ repositories.news().load_news_events()
|
|
│ └─ Async API call
|
|
│
|
|
└─ Wait for all tasks (tokio::join_all)
|
|
└─ Continue processing
|
|
|
|
Repository Implementations
|
|
├─ Send + Sync
|
|
└─ All methods are async
|
|
└─ No blocking calls
|
|
└─ Can be called from any async task
|
|
└─ Safe to share via Arc<dyn Trait>
|
|
```
|
|
|
|
---
|
|
|
|
## 10. Mock vs Real Decision Tree
|
|
|
|
```
|
|
Start
|
|
│
|
|
├─ Question: Do you need real data?
|
|
│
|
|
├─ NO (unit/example test)
|
|
│ │
|
|
│ └─ Use DefaultRepositories::mock()
|
|
│ └─ All mocks in-memory
|
|
│ └─ Instant results
|
|
│ └─ No external dependencies
|
|
│ └─ Example: wave_comparison.rs example
|
|
│
|
|
├─ YES (integration/E2E test)
|
|
│ │
|
|
│ ├─ Question: Use Databento API or local files?
|
|
│ │
|
|
│ ├─ API (production-like)
|
|
│ │ │
|
|
│ │ └─ Unset USE_DBN_DATA
|
|
│ │ └─ create_repositories() will use
|
|
│ │ DataProviderMarketDataRepository
|
|
│ │
|
|
│ ├─ Local Files (fast repeatable)
|
|
│ │ │
|
|
│ │ └─ Set USE_DBN_DATA=true
|
|
│ │ └─ Set DBN_SYMBOL_MAPPINGS
|
|
│ │ └─ create_repositories() will use
|
|
│ │ DbnMarketDataRepository
|
|
│ │ └─ Example: backtesting_service tests
|
|
│ │
|
|
│ └─ Both trading and news from providers
|
|
│ └─ StorageManagerTradingRepository (PostgreSQL)
|
|
│ └─ BenzingaNewsRepository (Benzinga API)
|
|
```
|
|
|