feat(ml): WAVE 29 DQN Codebase Cleanup & Refactoring Campaign

BREAKING CHANGES:
- Removed orphaned dqn.rs monolithic trainer (4,975 lines)
- Removed orphaned dqn_ensemble.rs module (816 lines)
- Removed orphaned tft.rs and tft_complete_int8_integration_test.rs
- TFT trainer split into modular directory structure

DQN Module Refactoring:
- Split trainers/dqn.rs into modular structure (config.rs, statistics.rs, trainer.rs)
- Fixed hyperopt 39D search space (continuous params only)
- Boolean flags (use_dueling, use_double_dqn, use_per, use_noisy_nets) are now FIXED architectural decisions
- use_distributional defaults to false (Candle BUG #36 - scatter_add gradient issues)

Clean Module Structure:
- ml/src/trainers/dqn/ directory with proper mod.rs exports
- ml/src/trainers/tft/ directory with config.rs, types.rs, model.rs, trainer.rs, tests.rs
- All P0 features validated: TD-error clamping, batch diversity, LR scheduler, priority staleness

Documentation:
- Added comprehensive docs in docs/codebase-cleanup/
- ADR-001 for DQN refactoring decisions
- Rainbow DQN component matrix and quick reference guides

Build Status: Compiles with zero errors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2025-11-27 23:46:13 +01:00
parent 2c1acda2f3
commit 2df1ea92e1
763 changed files with 247870 additions and 1714 deletions

View File

@@ -0,0 +1,778 @@
# DQN Stress Testing Implementation Analysis
**Analysis Date:** 2025-11-27
**Evaluated Against:** 2025 Industry Standards for Financial ML Stress Testing
**File Analyzed:** `/home/jgrusewski/Work/foxhunt/ml/src/dqn/stress_testing.rs` (521 lines)
---
## Executive Summary
The DQN stress testing framework provides a **solid foundation** but falls short of 2025 institutional standards in several critical areas. While basic scenario coverage exists, the implementation lacks:
- **Real market data integration** (currently uses simulated metrics)
- **Advanced liquidity stress testing** (spread widening only)
- **Tail risk modeling** (EVT, copulas, fat-tailed distributions)
- **Multi-regime correlation breakdown scenarios**
- **Recovery path validation** (time-to-recovery metrics incomplete)
**Overall Grade:** C+ (Functional but incomplete for production deployment)
---
## 1. Scenario Generation (Current: Basic, Target: Advanced)
### Current Implementation ✅
**Strengths:**
- **8 predefined scenarios** covering standard stress events:
```rust
flash_crash_scenario() // -10% in 5 minutes
liquidity_crisis_scenario() // 50x spread widening
vix_spike_scenario() // 5x volatility
trending_market_scenario() // +8% strong trend
whipsaw_scenario() // rapid reversals
gap_risk_scenario() // -7% gap down
correlation_breakdown_scenario() // multi-asset stress
multi_asset_stress_scenario() // combined stress (-12%)
```
- **Parameterized design** allows custom scenarios:
```rust
pub struct StressScenario {
price_shock_pct: f64,
volatility_multiplier: f64,
spread_multiplier: f64,
duration_steps: usize,
max_drawdown_threshold: f64,
min_action_diversity: f64,
}
```
- **Synthetic data generation** (`apply_stress()` method):
```rust
fn apply_stress(&self, scenario: &StressScenario) -> Result<Vec<f64>> {
// Gradual shock over first 20% of duration
// Random walk with scaled volatility
// Returns stressed price series
}
```
### Critical Gaps ❌
**1. No Historical Scenario Replay**
- Missing ability to replay actual historical crises:
- Flash Crash (May 6, 2010)
- COVID-19 crash (March 2020)
- Silicon Valley Bank collapse (March 2023)
- FTX collapse (November 2022)
**2025 Standard:** Historical scenario libraries with tick-level data
**2. No Monte Carlo Scenario Generation**
- No stochastic scenario generation using:
- Jump-diffusion processes
- Regime-switching models
- GARCH-based volatility paths
- Copula-based multi-asset scenarios
**2025 Standard:** 10,000+ Monte Carlo paths per scenario
**3. No Conditional Scenarios**
- Missing conditional stress tests:
- "Given VIX > 40, what if correlation → 1.0?"
- "Given position size > 80%, what if liquidity dries up?"
- "Given trending regime, what if sudden reversal?"
**2025 Standard:** Bayesian scenario trees with conditional probabilities
**4. Limited Severity Levels**
- Only single severity per scenario
- No stress ladder: mild → moderate → severe → extreme
**2025 Standard:** 4-5 severity levels per scenario with probability weights
---
## 2. Market Regime Stress Tests (Current: Basic, Target: Advanced)
### Current Implementation ⚠️
**Present but Incomplete:**
- **Basic regime scenarios**:
- `trending_market_scenario()` - +8% trend
- `whipsaw_scenario()` - rapid reversals
- `vix_spike_scenario()` - volatility regime
- **Regime-conditional architecture exists** (separate file):
```rust
// From ml/src/dqn/regime_conditional.rs
pub enum RegimeType {
Trending, // High ADX, directional moves
Ranging, // Low ADX, mean reversion
Volatile, // High entropy, unpredictable
}
```
### Critical Gaps ❌
**1. No Regime Transition Stress**
- Missing tests for regime shifts:
- Trending → Volatile (breakdown)
- Ranging → Trending (breakout)
- Volatile → Ranging (normalization)
**2025 Standard:** Hidden Markov Model regime transitions with transition probabilities
**2. No Regime-Specific Thresholds**
- All scenarios use same thresholds regardless of regime:
```rust
// Current: Same threshold for all regimes
max_drawdown_threshold: 20.0,
// 2025 Standard: Regime-adaptive thresholds
// Trending: 15% drawdown acceptable
// Ranging: 8% drawdown max
// Volatile: 25% drawdown expected
```
**3. No Multi-Regime Scenarios**
- No tests combining multiple regimes:
- Morning trending + afternoon whipsaw
- Pre-market gap + intraday ranging
- Volatile open + trending close
**2025 Standard:** Sequential regime chains with realistic durations
**4. No Regime Detection Under Stress**
- No validation that regime classifier remains accurate during stress
- Risk: Model may misclassify regime during crisis → wrong action selection
**2025 Standard:** Regime classification robustness tests (accuracy > 80% even at 3σ events)
---
## 3. Liquidity Stress Tests (Current: Primitive, Target: Advanced)
### Current Implementation ⚠️
**Basic Coverage:**
- **Spread widening** simulation:
```rust
liquidity_crisis_scenario() {
spread_multiplier: 50.0, // 50x normal spread
price_shock_pct: -2.0,
duration_steps: 600,
}
```
- **Circuit breaker integration** (separate module):
```rust
// From ml/src/dqn/circuit_breaker.rs
pub struct CircuitBreaker {
failure_threshold: usize,
timeout_duration: Duration,
// Halts trading during consecutive failures
}
```
### Critical Gaps ❌
**1. No Market Depth Modeling**
- Missing order book depth simulation:
- No bid/ask queue sizes
- No level 2 liquidity curves
- No market depth degradation
**2025 Standard:** Full L2 order book replay with depth-dependent slippage
**Example Missing:**
```rust
// 2025 Standard
pub struct LiquidityStressParams {
depth_levels: Vec<(Price, Quantity)>, // Bid/ask ladder
depth_decay_rate: f64, // How fast depth evaporates
replenishment_rate: f64, // How fast it recovers
toxic_flow_intensity: f64, // Informed trader activity
}
```
**2. No Market Impact Modeling**
- Missing dynamic market impact:
- Temporary impact (recovers over time)
- Permanent impact (price dislocation)
- No Kyle's Lambda or Almgren-Chriss models
**Current Risk:** DQN may learn to trade large sizes without penalty
**2025 Standard:** Square-root market impact + transient impact decay
**3. No Liquidity Recovery Modeling**
- No simulation of liquidity returning after stress
- Recovery metrics incomplete:
```rust
// Current: Simple heuristic
let recovery_steps = if max_drawdown < 15.0 {
Some(scenario.duration_steps / 2)
} else {
None
};
// 2025 Standard: Exponential recovery with half-life
// recovery_time = f(shock_magnitude, market_regime, time_of_day)
```
**4. No Fragmentation Scenarios**
- Missing multi-venue liquidity tests:
- Primary exchange vs dark pools
- Cross-venue arbitrage during stress
- Smart order routing under fragmentation
**2025 Standard:** Multi-venue liquidity aggregation with venue-specific stress
**5. No Adverse Selection**
- No toxic flow detection or informed trader scenarios
- DQN may learn exploitable patterns without adversarial testing
**2025 Standard:** Adversarial agents with information asymmetry
---
## 4. Correlation Breakdown Tests (Current: Minimal, Target: Comprehensive)
### Current Implementation ⚠️
**Nominal Coverage:**
- **Single scenario** for correlation breakdown:
```rust
correlation_breakdown_scenario() {
name: "Correlation Breakdown",
price_shock_pct: -6.0,
volatility_multiplier: 3.5,
spread_multiplier: 7.0,
duration_steps: 900,
}
```
- **Multi-asset infrastructure exists** (separate module):
```rust
// From ml/src/dqn/multi_asset.rs
pub struct MultiAssetPortfolio {
correlation_matrix: Array2<f64>, // NxN correlation matrix
positions: HashMap<String, Position>,
}
```
### Critical Gaps ❌
**1. No Dynamic Correlation Modeling**
- Correlation matrix is **static** (identity matrix by default)
- No time-varying correlations (DCC-GARCH, EWMA)
- No correlation regime shifts
**2025 Standard:** Dynamic Conditional Correlation (DCC) with regime-dependent correlations
**Example Missing:**
```rust
// 2025 Standard
pub struct CorrelationStressParams {
normal_correlation: Array2<f64>, // Calm markets: ρ ≈ 0.3
stress_correlation: Array2<f64>, // Panic: ρ ≈ 0.9
transition_speed: f64, // How fast correlation converges
asymmetry: bool, // Downside correlation > upside
}
```
**2. No Tail Dependence Modeling**
- Missing copula-based tail correlation:
- Assets can be uncorrelated normally but correlated in tails
- Critical for portfolio VaR estimation
**Current Risk:** Portfolio diversification may fail during crisis
**2025 Standard:** t-copulas or Archimedean copulas for tail dependence
**3. No Cross-Asset Contagion**
- No stress propagation across asset classes:
- Equity crash → credit spread widening
- FX volatility → commodity dislocation
- Liquidity contagion across markets
**2025 Standard:** Multi-layer correlation networks with shock propagation
**4. No Factor Model Stress**
- Missing factor-based stress tests:
- What if market beta → 1.5 (all stocks move together)?
- What if size factor reverses?
- What if momentum crashes?
**2025 Standard:** Barra-style factor risk model with factor shocks
---
## 5. Recovery Testing (Current: Incomplete, Target: Rigorous)
### Current Implementation ⚠️
**Basic Metrics:**
- **Recovery steps calculation**:
```rust
pub struct StressResult {
recovery_steps: Option<usize>, // Time to 95% recovery
// ...
}
```
- **Simple heuristic**:
```rust
let recovery_steps = if max_drawdown < 15.0 {
Some(scenario.duration_steps / 2)
} else {
None
};
```
### Critical Gaps ❌
**1. No Recovery Path Validation**
- Missing time-series validation:
- Is recovery monotonic or oscillating?
- Are there secondary drawdowns?
- What's the recovery volatility?
**2025 Standard:** Full recovery trajectory analysis with confidence bands
**2. No Adaptive Recovery Modeling**
- Recovery time should depend on:
- Market regime during recovery
- Volatility levels
- Liquidity conditions
- Position size
**Current:** Fixed heuristic (duration / 2)
**2025 Standard:** Empirically-calibrated recovery models from historical data
**3. No Scarring Effects**
- Missing tests for permanent damage:
- Reduced risk appetite after stress
- Lower position sizes
- Increased epsilon (more exploration)
**2025 Standard:** Agent behavior shift metrics (pre-stress vs post-stress)
**4. No Multiple Recovery Scenarios**
- Only tests single recovery path
- Missing:
- V-shaped recovery (fast bounce)
- U-shaped recovery (slow normalization)
- W-shaped recovery (double dip)
- L-shaped recovery (no recovery)
**2025 Standard:** Recovery pattern classification with probability distribution
**5. No Compounding Stress**
- What if second shock occurs during recovery?
- No tests for clustered volatility (GARCH effects)
**2025 Standard:** Compound stress scenarios with recovery interruption
---
## 6. Integration with Risk Framework
### Current Capabilities ✅
**Well-Integrated:**
1. **Circuit Breaker** (`ml/src/dqn/circuit_breaker.rs`)
- Halts trading after consecutive failures
- Cooldown periods
- Half-open testing state
2. **Risk-Adjusted Rewards** (referenced in code)
- Sharpe ratio integration
- Drawdown penalties
- VaR-based risk metrics
3. **Action Masking** (risk constraints)
- Position limits
- Drawdown limits
- Compliance rules
### Missing Integrations ❌
**1. No VaR Backtesting**
- Missing Kupiec POF test (Proportion of Failures)
- No Christoffersen test (clustering of violations)
- No traffic light system (Basel III)
**2025 Standard:** Automated VaR model validation with regulatory tests
**2. No Expected Shortfall (ES) Calculation**
- VaR tells "how often" but not "how bad"
- ES = average loss beyond VaR threshold
**2025 Standard:** ES calculation for all scenarios (Basel III requirement)
**3. No Stress VaR**
- Missing stressed VaR (using crisis-period calibration)
- No incremental VaR (marginal contribution analysis)
**2025 Standard:** Stressed VaR + Incremental VaR for position sizing
---
## 7. Code Quality Assessment
### Strengths ✅
1. **Clean Architecture**
- Clear separation of concerns
- Builder pattern for scenarios
- Composable stress tests
2. **Good Documentation**
- Comprehensive module docs
- Example usage provided
- Clear parameter explanations
3. **Type Safety**
- Strong typing throughout
- Serde serialization for results
- Result types for error handling
4. **Testability**
- Unit tests for scenario definitions
- Validation tests for thresholds
- Smoke tests for basic functionality
### Weaknesses ❌
1. **Simulated Metrics** (Lines 153-170)
```rust
// NOTE: Full DQN training integration would go here
// For now, we simulate metrics based on scenario severity
let severity_factor = (scenario.price_shock_pct.abs() +
scenario.volatility_multiplier) / 15.0;
```
**Impact:** Not actually running DQN through stress scenarios
2. **Hardcoded Parameters**
- Base price = 4000.0 (line 299)
- Volatility = 0.02 (line 312)
- Recovery heuristic = duration / 2 (line 166)
3. **No Parallelization**
- Scenarios run sequentially (`run_all_scenarios()`)
- Could leverage Rayon for parallel execution
4. **Limited Metrics**
- Missing key metrics:
- Maximum adverse excursion (MAE)
- Time underwater
- Ulcer index
- Calmar ratio
- Recovery factor
---
## 8. Recommended Improvements (Prioritized)
### Tier 1: Critical (Must-Have for Production) 🔴
1. **Integrate Real DQN Execution** (Effort: 3 days)
```rust
// Replace simulation with actual DQN rollout
pub fn run_scenario(&mut self, scenario: &StressScenario) -> Result<StressResult> {
let stressed_data = self.apply_stress(scenario)?;
// ACTUAL DQN evaluation on stressed data
let mut cumulative_reward = 0.0;
let mut max_drawdown = 0.0;
let mut actions = Vec::new();
for (i, price) in stressed_data.iter().enumerate() {
let state = self.build_state(i, price, &stressed_data);
let action = self.trainer.select_action(&state)?;
let reward = self.trainer.step(action, &state)?;
actions.push(action);
cumulative_reward += reward;
max_drawdown = max_drawdown.max(compute_drawdown(...));
}
// Compute real metrics from actual execution
}
```
2. **Add Historical Scenario Replay** (Effort: 5 days)
```rust
pub fn load_historical_scenario(
scenario_name: &str, // "flash_crash_2010", "covid_march_2020"
data_path: &Path,
) -> Result<StressScenario> {
// Load tick-level historical data
// Replay exact price/volume/spread dynamics
}
```
3. **Implement Market Depth Modeling** (Effort: 4 days)
```rust
pub struct LiquidityProfile {
bid_depth: Vec<(Price, Quantity)>,
ask_depth: Vec<(Price, Quantity)>,
depth_decay_halflife: Duration,
replenishment_rate: f64,
}
pub fn apply_liquidity_stress(
&mut self,
profile: &LiquidityProfile,
order_size: Quantity,
) -> (Price, Slippage, MarketImpact) {
// Walk the book, compute slippage
}
```
4. **Add Expected Shortfall** (Effort: 1 day)
```rust
pub struct StressResult {
// Existing fields...
pub expected_shortfall: f64, // CVaR / ES
pub var_95: f64,
pub var_99: f64,
pub worst_1pct_avg: f64,
}
```
### Tier 2: Important (Needed for Robustness) 🟡
5. **Dynamic Correlation Modeling** (Effort: 3 days)
```rust
pub struct DynamicCorrelationModel {
ewma_lambda: f64,
correlation_regime: RegimeType,
stress_correlation_matrix: Array2<f64>,
normal_correlation_matrix: Array2<f64>,
}
```
6. **Monte Carlo Scenario Generation** (Effort: 4 days)
```rust
pub fn generate_monte_carlo_scenarios(
num_scenarios: usize,
process: StochasticProcess, // JumpDiffusion, GARCH, etc.
) -> Vec<StressScenario> {
// Generate 10,000+ scenarios
}
```
7. **Recovery Path Validation** (Effort: 2 days)
```rust
pub struct RecoveryMetrics {
recovery_time: Option<Duration>,
recovery_pattern: RecoveryPattern, // V, U, W, L
secondary_drawdowns: Vec<f64>,
recovery_volatility: f64,
}
```
8. **Regime Transition Stress** (Effort: 3 days)
```rust
pub fn regime_transition_scenario(
from: RegimeType,
to: RegimeType,
transition_speed: Duration,
) -> StressScenario {
// Model regime shifts
}
```
### Tier 3: Enhancement (Nice-to-Have) 🟢
9. **Parallel Execution** (Effort: 1 day)
```rust
use rayon::prelude::*;
pub fn run_all_scenarios_parallel(&mut self) -> Vec<StressResult> {
self.scenarios
.par_iter()
.map(|scenario| self.run_scenario(scenario))
.collect()
}
```
10. **Adversarial Agents** (Effort: 5 days)
```rust
pub struct AdversarialScenario {
informed_trader_intensity: f64,
front_running_probability: f64,
spoofing_rate: f64,
}
```
11. **Multi-Venue Fragmentation** (Effort: 4 days)
```rust
pub struct VenueStressParams {
venues: Vec<TradingVenue>,
venue_correlations: Array2<f64>,
cross_venue_arb_delay: Duration,
}
```
---
## 9. Comparison with Industry Standards
| Feature | Current Status | 2025 Standard | Gap |
|---------|---------------|---------------|-----|
| **Scenario Coverage** | 8 predefined | 50+ historical + Monte Carlo | ❌ Large |
| **Severity Levels** | 1 per scenario | 4-5 levels per scenario | ❌ Large |
| **Market Depth** | None | Full L2 order book | ❌ Critical |
| **Market Impact** | None | Kyle's Lambda + Almgren-Chriss | ❌ Critical |
| **Correlation Dynamics** | Static | DCC-GARCH | ❌ Large |
| **Tail Dependence** | None | Copula-based | ❌ Large |
| **Recovery Modeling** | Simple heuristic | Empirical calibration | ❌ Moderate |
| **VaR Backtesting** | None | Kupiec + Christoffersen | ❌ Critical |
| **Expected Shortfall** | None | Full ES calculation | ❌ Critical |
| **Regime Transitions** | None | HMM-based | ❌ Moderate |
| **Execution** | Simulated | Real DQN rollout | ❌ Critical |
| **Parallelization** | Sequential | Rayon parallel | ⚠️ Minor |
---
## 10. Risk Assessment
### Production Deployment Risks 🔴
**If deployed as-is, the following risks exist:**
1. **False Confidence** (Severity: HIGH)
- Simulated metrics may not reflect actual DQN behavior
- Could pass stress tests in simulation but fail in production
2. **Liquidity Blindness** (Severity: HIGH)
- No market impact modeling → DQN may learn unrealistic strategies
- Large orders may cause unmodeled slippage
3. **Correlation Failure** (Severity: MEDIUM)
- Static correlations → diversification may fail in crisis
- Portfolio VaR underestimated
4. **Incomplete Recovery** (Severity: MEDIUM)
- Simple recovery heuristic may miss scarring effects
- Agent may not adapt after extreme stress
5. **Regulatory Risk** (Severity: HIGH for regulated firms)
- Missing Basel III requirements (ES, Stressed VaR)
- No VaR backtesting → cannot validate risk model
---
## 11. Suggested Next Steps
### Phase 1: Make It Real (2 weeks)
1. Integrate actual DQN execution into stress scenarios
2. Add market depth and slippage modeling
3. Implement Expected Shortfall calculation
### Phase 2: Add Realism (2 weeks)
4. Load historical crisis scenarios
5. Implement dynamic correlation modeling
6. Add regime transition stress tests
### Phase 3: Robustness (1 week)
7. Implement recovery path validation
8. Add VaR backtesting (Kupiec/Christoffersen)
9. Parallelize scenario execution
### Phase 4: Advanced Features (2 weeks)
10. Monte Carlo scenario generation
11. Adversarial agent testing
12. Multi-venue fragmentation modeling
---
## 12. Conclusion
The current DQN stress testing implementation provides a **good architectural foundation** with clean code and extensible design. However, it requires **significant enhancements** to meet 2025 institutional standards:
**Key Strengths:**
- ✅ Well-structured code with clear separation of concerns
- ✅ Comprehensive scenario coverage for basic stress events
- ✅ Integration with circuit breaker and risk management
- ✅ Good documentation and testability
**Critical Weaknesses:**
- ❌ **Simulated metrics instead of real DQN execution**
- ❌ **No market depth or liquidity modeling**
- ❌ **Missing Expected Shortfall and VaR backtesting**
- ❌ **Static correlations without tail dependence**
- ❌ **Incomplete recovery modeling**
**Recommended Action:** Prioritize **Tier 1 improvements** (real DQN execution, market depth, ES calculation) before considering production deployment.
**Estimated Effort:** 4-6 weeks of focused development to reach production-grade status.
---
## Appendix A: Code Examples from Analysis
### Example 1: Current Simulated Metrics (Lines 153-170)
```rust
// Apply stress scenario
let _stressed_data = self.apply_stress(scenario)?;
// Simulate stress test (NOTE: Full DQN training integration would go here)
// For now, we simulate metrics based on scenario severity
let severity_factor = (scenario.price_shock_pct.abs() + scenario.volatility_multiplier) / 15.0;
// Simulate portfolio performance under stress
let initial_portfolio = 100_000.0;
let max_drawdown = scenario.price_shock_pct.abs() * (1.0 + severity_factor * 0.5);
let final_portfolio = initial_portfolio * (1.0 + scenario.price_shock_pct / 100.0 * 0.8);
// Simulate action diversity (reduces under extreme stress)
let action_diversity = (70.0 - severity_factor * 15.0).max(20.0);
```
### Example 2: Synthetic Stress Data Generation (Lines 294-319)
```rust
fn apply_stress(&self, scenario: &StressScenario) -> Result<Vec<f64>> {
let mut stressed_prices = Vec::new();
let base_price = 4000.0; // ES futures typical price
for step in 0..scenario.duration_steps {
let progress = step as f64 / scenario.duration_steps as f64;
// Apply price shock (gradual over first 20% of duration)
let shock_factor = if progress < 0.2 {
1.0 + (scenario.price_shock_pct / 100.0) * (progress / 0.2)
} else {
1.0 + (scenario.price_shock_pct / 100.0)
};
// Add volatility (random walk scaled by volatility multiplier)
let volatility = 0.02 * scenario.volatility_multiplier * (rand::random::<f64>() - 0.5);
let price = base_price * shock_factor * (1.0 + volatility);
stressed_prices.push(price);
}
Ok(stressed_prices)
}
```
### Example 3: Recovery Heuristic (Line 166)
```rust
let recovery_steps = if max_drawdown < 15.0 {
Some(scenario.duration_steps / 2)
} else {
None
};
```
---
**Analyst:** Code Analyzer Agent
**Review Status:** Complete
**Confidence Level:** High (based on comprehensive codebase analysis)