foxhunt/docs/MOCK_REPOSITORY_REFERENCE.md

# Mock Repository Reference Guide

**Purpose**: Quick reference for understanding backtesting service mock repositories

**Audience**: Developers working on backtesting service

**Last Updated**: 2025-10-18

---

## At a Glance

```
┌────────────────────────────────────────────────────────────┐
│            Backtesting Service - Repository Layer          │
├─────────────────────────┬──────────────────────────────────┤
│ PRODUCTION CODE         │ TEST CODE                        │
│                         │                                  │
│ main.rs (line 133)      │ strategy_engine_tests.rs         │
│   ↓                     │   ↓                              │
│ create_repositories()   │ MockBacktestingRepositories      │
│   ↓                     │   ├─ MockMarketDataRepository    │
│ ┌─ Databento API        │   ├─ MockTradingRepository       │
│ └─ DBN Files            │   └─ MockNewsRepository          │
│ ┌─ PostgreSQL           │                                  │
│ └─ Benzinga API         │ +19 other test files             │
└────────────────────────────────────────────────────────────┘
```

---

## Mock Hierarchy

### Level 1: Repository Trait Definitions
**File**: `src/repositories.rs` (lines 1-187)

```rust
#[async_trait]
pub trait MarketDataRepository: Send + Sync {
    async fn load_historical_data(...) -> Result<Vec<MarketData>>;
    async fn check_data_availability(...) -> Result<HashMap<String, bool>>;
}

#[async_trait]
pub trait TradingRepository: Send + Sync {
    async fn save_backtest_results(...) -> Result<()>;
    async fn load_backtest_results(...) -> Result<(Vec<BacktestTrade>, PerformanceMetrics)>;
    // ... 5 more methods
}

#[async_trait]
pub trait NewsRepository: Send + Sync {
    async fn load_news_events(...) -> Result<Vec<NewsEvent>>;
    async fn get_sentiment_data(...) -> Result<HashMap<String, f64>>;
}
```

### Level 2: Production Mock Stubs
**File**: `src/repositories.rs` (lines 188-302)

These are used **only in production code when DefaultRepositories::mock() is called** (rare):

```rust
pub struct MockMarketDataRepository;

#[async_trait]
impl MarketDataRepository for MockMarketDataRepository {
    async fn load_historical_data(...) -> Result<Vec<MarketData>> {
        Ok(vec![])  // Empty - no data loading
    }
    // ...
}
```

**Usage in production code**: 2 places (wave_comparison.rs for metric validation)

### Level 3: Test Helper Mocks
**File**: `tests/mock_repositories.rs` (lines 1-441)

These are more sophisticated mocks used in actual tests:

```rust
pub struct MockMarketDataRepository {
    pub data: Arc<RwLock<Vec<MarketData>>>,  // Can hold test data
}

impl MockMarketDataRepository {
    pub fn new() -> Self { /* ... */ }
    pub fn with_data(data: Vec<MarketData>) -> Self { /* ... */ }
}

#[async_trait]
impl MarketDataRepository for MockMarketDataRepository {
    async fn load_historical_data(
        &self,
        symbols: &[String],
        start_time: i64,
        end_time: i64,
    ) -> Result<Vec<MarketData>> {
        // Actually filters and returns test data!
        let data = self.data.read().await;
        let filtered: Vec<MarketData> = data
            .iter()
            .filter(|d| {
                symbols.contains(&d.symbol)
                    && d.timestamp.timestamp_nanos_opt().unwrap_or(0) >= start_time
                    && d.timestamp.timestamp_nanos_opt().unwrap_or(0) <= end_time
            })
            .cloned()
            .collect();
        Ok(filtered)
    }
}
```

**Usage**: All unit/integration tests

### Level 4: Real Implementations
**File**: `src/repository_impl.rs` (lines 1-365)

```rust
pub struct DataProviderMarketDataRepository {
    databento_provider: Arc<DatabentoHistoricalProvider>,
}

#[async_trait]
impl MarketDataRepository for DataProviderMarketDataRepository {
    async fn load_historical_data(...) -> Result<Vec<MarketData>> {
        // Actually loads from Databento API
        let market_events = self.databento_provider.fetch(...).await?;
        // Convert and return real data
        Ok(all_market_data)
    }
}
```

**Also in this file**:
- `StorageManagerTradingRepository` - Uses PostgreSQL
- `BenzingaNewsRepository` - Uses Benzinga API
- `create_repositories()` - Factory function

**Usage**: Production code (main.rs), ML backtesting (ml_strategy_engine.rs)

---

## When to Use Which Mock

### Use Production Mock Stubs (src/repositories.rs)

**Scenario**: Only when testing metric calculations that don't depend on data loading

```rust
// Example: wave_comparison.rs
let backtest = WaveComparisonBacktest::new(
    Arc::new(DefaultRepositories::mock()),  // Empty mocks OK here
    100000.0,
);
let improvements = backtest.calculate_improvements(...);
// Only calculates percentages, doesn't load data
```

**Characteristics**:
- No data loading needed
- Only calculation logic
- Can use empty implementations

### Use Test Helper Mocks (tests/mock_repositories.rs)

**Scenario**: Testing strategy logic, portfolio management, order execution

```rust
// Example: strategy_engine_tests.rs
let market_data = generate_sample_market_data(
    "AAPL",
    100,
    100.0,
    0.05,
);

let market_data_repo = Box::new(
    MockMarketDataRepository::with_data(market_data)
);

let engine = StrategyEngine::new(&config, repositories).await?;
let trades = engine.execute_backtest(&context).await?;

// Assert specific behavior based on deterministic data
assert_eq!(trades.len(), expected_count);
```

**Characteristics**:
- Hold test data
- Stateful (can track trades, metrics)
- Deterministic data patterns
- Fast execution (no I/O)

### Use Real Implementations (src/repository_impl.rs)

**Scenario**: Production, integration tests with real data

```rust
// Example: main.rs
let repositories = Arc::new(
    create_repositories(storage_manager)
        .await?
);
// Uses Databento API or DBN files (based on USE_DBN_DATA env var)
```

**Or in integration tests**:

```rust
// Example: dbn_integration_tests.rs
let repo = DbnMarketDataRepository::new(file_mapping).await?;
let data = repo.load_historical_data(&symbols, start_time, end_time).await?;
// Assert real market patterns
```

---

## Key Functions

### Test Data Generators

**File**: `tests/mock_repositories.rs`

```rust
/// Generate deterministic sample market data
pub fn generate_sample_market_data(
    symbol: &str,
    num_points: usize,
    start_price: f64,
    volatility: f64,
) -> Vec<MarketData> {
    // Creates sine-wave price pattern
    // Ensures price oscillates through levels multiple times
}

/// Generate sample news events
pub fn generate_sample_news_events(
    symbols: &[String],
    num_events: usize,
) -> Vec<NewsEvent> {
    // Creates random news with sentiment scores
}

/// Create DBN-based repository for real data
pub async fn create_dbn_repository() -> Result<Box<dyn MarketDataRepository>> {
    use backtesting_service::dbn_repository::DbnMarketDataRepository;
    let mut file_mapping = HashMap::new();
    file_mapping.insert("ES.FUT".to_string(), get_dbn_test_file_path());
    let repo = DbnMarketDataRepository::new(file_mapping).await?;
    Ok(Box::new(repo))
}
```

### Repository Factory

**File**: `src/repository_impl.rs`

```rust
pub async fn create_repositories(
    storage_manager: Arc<StorageManager>,
) -> Result<DefaultRepositories> {
    // Environment-controlled selection:
    // USE_DBN_DATA=true  → DbnMarketDataRepository (test_data/)
    // USE_DBN_DATA=false → DataProviderMarketDataRepository (Databento API)
    
    let market_data: Box<dyn MarketDataRepository> = if use_dbn_data {
        // Load from local DBN files
        let file_mapping = HashMap::new();
        file_mapping.insert("ES.FUT".to_string(), "test_data/real/databento/ES.FUT_ohlcv-1m_2024-01-02.dbn");
        Box::new(DbnMarketDataRepository::new(file_mapping).await?)
    } else {
        // Use Databento API
        Box::new(DataProviderMarketDataRepository::new().await?)
    };
    
    let trading = Box::new(StorageManagerTradingRepository::new(storage_manager));
    let news = Box::new(BenzingaNewsRepository::new().await?);
    
    Ok(DefaultRepositories { market_data, trading, news })
}
```

---

## Usage Statistics

### Mock Occurrences

| Mock Type | Count | Primary Files |
|---|---|---|
| `MockMarketDataRepository` | 59 | strategy_engine_tests.rs (15+), others (44+) |
| `MockTradingRepository` | 61 | strategy_engine_tests.rs (15+), others (46+) |
| `MockNewsRepository` | 54 | strategy_engine_tests.rs (10+), others (44+) |
| **Total** | **174** | 8 test files |

### Real Repository Occurrences

| Type | Count | Primary Usage |
|---|---|---|
| `DbnMarketDataRepository` | 24+ | dbn_repository.rs tests, create_repositories() |
| `DataProviderMarketDataRepository` | 12+ | repository_impl.rs, tests |
| `StorageManagerTradingRepository` | 12+ | repository_impl.rs, main.rs, ml_strategy_engine.rs |
| `BenzingaNewsRepository` | 12+ | repository_impl.rs, create_repositories() |
| **Total** | **67** | Production code, integration tests |

---

## Testing Strategy

### Fast Path (< 1 second)
```
Unit Tests → Mock Repositories → In-memory operations → Fast ✓
```

Typical test:
```rust
#[tokio::test]
async fn test_portfolio_initialization() -> Result<()> {
    let market_data_repo = Box::new(MockMarketDataRepository::new());
    let engine = StrategyEngine::new(&config, repositories).await?;
    let trades = engine.execute_backtest(&context).await?;
    assert_eq!(trades.len(), 0);
}
```

### Real Data Path (1-10 seconds)
```
Integration Tests → DbnMarketDataRepository → File I/O → Real data ✓
```

Typical test:
```rust
#[tokio::test]
async fn test_dbn_data_loading() -> Result<()> {
    let repo = DbnMarketDataRepository::new(file_mapping).await?;
    let data = repo.load_historical_data(&symbols, start_time, end_time).await?;
    assert!(data.len() > 0);
}
```

---

## Environment Variables

### For Testing

```bash
# Use local DBN files instead of API
export USE_DBN_DATA=true

# Specify DBN file paths (comma-separated symbol:path pairs)
export DBN_SYMBOL_MAPPINGS="ES.FUT:test_data/real/databento/ES.FUT_ohlcv-1m_2024-01-02.dbn,NQ.FUT:test_data/real/databento/NQ.FUT_ohlcv-1m_2024-01-02.dbn"

# Optional: Symbol mapping for test compatibility
export DBN_SYMBOL_MAP="BTC/USD:ES.FUT,ETH/USD:NQ.FUT"
```

### For Production

```bash
# Use Databento API (default)
export USE_DBN_DATA=false  # or unset

# Databento credentials
export DATABENTO_API_KEY=your_key_here

# Database
export DATABASE_URL=postgresql://user:pass@localhost/foxhunt

# Other services
export BENZINGA_API_KEY=your_key_here
```

---

## Common Patterns

### Pattern 1: Quick Unit Test with Mocks

```rust
#[tokio::test]
async fn test_strategy_logic() -> Result<()> {
    // 1. Create test data
    let market_data = generate_sample_market_data("AAPL", 100, 100.0, 0.05);
    
    // 2. Create mock repositories
    let market_data_repo = Box::new(
        MockMarketDataRepository::with_data(market_data)
    );
    let trading_repo = Box::new(MockTradingRepository::new());
    let news_repo = Box::new(MockNewsRepository::new());
    
    // 3. Combine into repositories
    let repositories = Arc::new(MockBacktestingRepositories::new(
        market_data_repo,
        trading_repo,
        news_repo,
    ));
    
    // 4. Test business logic
    let engine = StrategyEngine::new(&config, repositories).await?;
    let trades = engine.execute_backtest(&context).await?;
    
    // 5. Assert results
    assert!(trades.len() > 0);
    Ok(())
}
```

### Pattern 2: Integration Test with Real Data

```rust
#[tokio::test]
async fn test_with_real_dbn_data() -> Result<()> {
    // 1. Load real data
    let mut file_mapping = HashMap::new();
    file_mapping.insert("ES.FUT".to_string(), "test_data/real/databento/ES.FUT_ohlcv-1m_2024-01-02.dbn".to_string());
    let repo = DbnMarketDataRepository::new(file_mapping).await?;
    
    // 2. Use real repository directly
    let data = repo.load_historical_data(&["ES.FUT".to_string()], start_time, end_time).await?;
    
    // 3. Assert real market patterns
    assert!(data.len() > 0);
    // Validate prices, timestamps, etc.
    Ok(())
}
```

### Pattern 3: Production Code (No Mocks)

```rust
// In main.rs
let repositories = Arc::new(
    create_repositories(storage_manager)
        .await?
);

let service = BacktestingServiceImpl::new(repositories, Some(Arc::clone(&model_cache)))
    .await?;

// repositories uses real implementations, never mocks
```

---

## Troubleshooting

### "Mock returned empty data"
**Issue**: Test expecting data but mock returns empty Vec
**Solution**: Use `MockMarketDataRepository::with_data()` instead of `new()`

```rust
// Wrong
let repo = Box::new(MockMarketDataRepository::new());

// Correct
let data = generate_sample_market_data("AAPL", 100, 100.0, 0.05);
let repo = Box::new(MockMarketDataRepository::with_data(data));
```

### "Tests running slow"
**Issue**: Using real repositories (DBN files) in unit tests
**Solution**: Use mocks for fast tests, real data only for integration tests

### "Production using mocks"
**Issue**: main.rs calling `DefaultRepositories::mock()`
**Solution**: Should never happen. Always call `create_repositories()` instead

---

## Best Practices

1. **Use mocks for unit tests** - They're fast, deterministic, and isolated
2. **Use real data for integration tests** - Validate actual patterns and edge cases
3. **Never use mocks in production code** - Only in tests and optional validation (wave_comparison.rs)
4. **Use environment variables for control** - USE_DBN_DATA, DBN_SYMBOL_MAPPINGS
5. **Generate realistic test data** - Use `generate_sample_market_data()` for deterministic patterns
6. **Keep mock implementations simple** - Easy to reason about in tests

---

**Related Documentation**:
- AGENT_M1_MOCK_USAGE_ANALYSIS.md - Detailed analysis report
- AGENT_M1_QUICK_SUMMARY.md - 60-second summary
- services/backtesting_service/src/repositories.rs - Trait definitions
- services/backtesting_service/tests/mock_repositories.rs - Mock implementations