e8a68ee39f
- Created data/examples/download_ml_training_data.rs using reqwest + Databento HTTP API - Downloaded 90 days × 4 symbols (ES.FUT, NQ.FUT, ZN.FUT, 6E.FUT) - Files saved to test_data/real/databento/ml_training/ - Total: 360 files, 15 MB compressed DBN format - Used existing Rust pattern from download_nq_fut.rs - API key loaded from .env file - 100% success rate (360/360 files) - Ready for ML training benchmarks Next: Create simplified training benchmark for RTX 3050 Ti GPU measurements
65 lines
2.4 KiB
Markdown
65 lines
2.4 KiB
Markdown
# DbnMarketDataRepository - Advanced Queries Quick Reference
|
|
|
|
## Quick Method Reference
|
|
|
|
| Method | Purpose | Performance | Usage |
|
|
|--------|---------|-------------|-------|
|
|
| `load_by_time_range()` | DateTime filtering | <10ms | `repo.load_by_time_range(&symbols, start, end).await?` |
|
|
| `load_with_volume_filter()` | Liquidity filtering | <10ms + O(n) | `repo.load_with_volume_filter(&symbols, min_vol, start, end).await?` |
|
|
| `load_regime_samples()` | Regime-specific data | <10ms + O(n) | `repo.load_regime_samples("trending", 50, &symbols).await?` |
|
|
| `get_date_range()` | Available dates | <10ms | `let (first, last) = repo.get_date_range("ES.FUT").await?` |
|
|
| `resample_bars()` | Timeframe aggregation | O(n) | `repo.resample_bars(&bars, 5)?` |
|
|
| `calculate_rolling_stats()` | Moving windows | O(n*w) | `repo.calculate_rolling_stats(&bars, 20)` |
|
|
| `generate_summary_stats()` | Comprehensive stats | O(n) | `repo.generate_summary_stats(&bars)` |
|
|
|
|
## Regime Types
|
|
|
|
| Regime | Threshold | Description |
|
|
|--------|-----------|-------------|
|
|
| `"trending"` | >0.5% range | High directional movement |
|
|
| `"ranging"` | <0.2% range | Sideways consolidation |
|
|
| `"volatile"` | >0.8% range + volume | Explosive moves |
|
|
| `"stable"` | <0.15% range | Minimal volatility |
|
|
|
|
## Common Patterns
|
|
|
|
### Filter by Liquidity
|
|
```rust
|
|
let min_volume = Decimal::new(100, 0);
|
|
let bars = repo.load_with_volume_filter(&symbols, min_volume, start, end).await?;
|
|
```
|
|
|
|
### Test Regime Strategy
|
|
```rust
|
|
let samples = repo.load_regime_samples("volatile", 50, &symbols).await?;
|
|
let results = strategy.backtest(&samples).await?;
|
|
```
|
|
|
|
### Multi-Timeframe Analysis
|
|
```rust
|
|
let bars_1m = repo.load_historical_data(&symbols, start, end).await?;
|
|
let bars_5m = repo.resample_bars(&bars_1m, 5)?;
|
|
let bars_15m = repo.resample_bars(&bars_1m, 15)?;
|
|
```
|
|
|
|
### Calculate Statistics
|
|
```rust
|
|
let stats = repo.generate_summary_stats(&bars);
|
|
println!("Mean: {:.2}", stats["mean_close"]);
|
|
println!("Volatility: {:.2}%", stats["std_close"] / stats["mean_close"] * 100.0);
|
|
```
|
|
|
|
## Performance Targets
|
|
|
|
- **Time Range Query**: <10ms ✅
|
|
- **Volume Filter**: <10ms + O(n) filter ✅
|
|
- **Regime Samples**: <10ms + O(n) filter ✅
|
|
- **Resampling**: O(n) single pass ✅
|
|
- **Statistics**: O(n) single pass ✅
|
|
|
|
## See Also
|
|
|
|
- `DBN_REPOSITORY_USAGE.md` - Detailed usage examples
|
|
- `AGENT_17_SUMMARY.md` - Implementation summary
|
|
- `dbn_data_source.rs` - Underlying DBN loader
|