Stashed changes had real safetensors weight loading via VarMap::load.
Audit branch was conservative (validate-only). Kept the real loading
and simplified Mamba2 to use the same VarMap::load pattern as TFT.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
from_checkpoint was creating fresh models with random weights and setting
is_trained=true, allowing trading on noise. Now validates checkpoint exists
and does NOT set is_trained=true when weights are random.
Mamba2 predict errors are now wrapped as MLError::InferenceError so the
fallback manager can degrade model health. Both TFT and Mamba2 predict
refuse to run on untrained models, and is_ready() reflects trained state.
Also fixes TFT input_dim mismatch (16 vs 5+10+16=31).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Low-confidence ensemble votes now forced to Hold (was letting weak
signals through to order generation in both vote paths)
- Inference latency tracking uses EMA (alpha=0.1) instead of
overwriting with the latest sample
- retrain_model returns Status::Unimplemented instead of faking
success with a random job_id
- get_model_performance returns honest zeroes instead of hardcoded
fake metrics (accuracy=0.85 etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was only checking quantity limit and VaR. Now runs all 5 risk checks:
1. Kill switch / circuit breaker (via TradingServiceKillSwitch)
2. Max order size + position limits (via RiskRepository.get_risk_limits)
3. Daily loss / drawdown limit (via RiskRepository.get_risk_metrics)
4. Leverage limit (via config + RiskRepository.get_risk_metrics)
5. VaR limit (via RiskEngine.check_var_limit, existing)
Violations accumulate rather than short-circuit so callers see all
failures at once. Added 6 tests verifying violation type coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The update_position method previously acquired a read lock to get the
Arc<AtomicPosition>, dropped it, then called update_with_execution
outside any lock. Two concurrent fills for the same position could both
load the same old_quantity via Acquire, compute independent new quantities,
and the last Release store would silently discard the other fill — causing
the position to show e.g. 10 shares when it should show 20.
Now holds the positions write lock across the entire get-or-create +
update_with_execution sequence, serializing concurrent fills per position.
The validate_position_update call remains outside the lock to minimize
hold time (it does its own async reads and risk checks).
Also removes excessive per-step RDTSC latency tracking from the critical
path (total latency tracking is preserved) and adds a multi-threaded
regression test that spawns 10 concurrent +1 fills and asserts the final
quantity equals 10.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clone_for_async() was creating independent AtomicU64 instances for
message_count, drop_count, last_heartbeat, and reconnect_attempts
instead of sharing the originals. This meant spawned tasks incremented
their own counters while get_stats() read the original's (always 0),
and the heartbeat monitor watched a counter never updated by the
connection task.
Also fixes last_heartbeat initializing to 0, which caused the first
heartbeat check to compute a huge elapsed time and immediately trigger
a false "connection appears dead" alert.
Changes:
- Change 4 struct fields from AtomicU64 to Arc<AtomicU64>
- Initialize last_heartbeat to HardwareTimestamp::now() instead of 0
- clone_for_async() now clones the Arcs (shared counters)
- Add test verifying counters are shared between original and clone
- Add test verifying heartbeat initialized to current time
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Casting negative f64 to u64 saturates to 0, corrupting avg_price and all
downstream PnL calculations. Add price_to_fixed_checked() that rejects
non-finite, zero, and negative prices with PositionError::InvalidPrice.
The critical update_with_execution path now uses the checked variant.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Spawn a background tokio task that feeds synthetic OHLCV bars into the
ensemble coordinator's per-symbol feature extractors. The extractors
require ~51 bars of warmup before they can produce real 51-dim feature
vectors; without this feed they remain cold and return zeros.
The task uses a random-walk price generator with realistic base prices
for the four baseline futures symbols (ES, NQ, ZN, 6E) and configurable
interval (MARKET_FEED_INTERVAL_MS, default 1s) and symbol list
(MARKET_FEED_SYMBOLS). In production this will be replaced by a
Databento or exchange data feed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register TGGN, TLOB, KAN, xLSTM, and Diffusion inference adapters
alongside the existing DQN, PPO, TFT, Mamba2, and Liquid-CfC.
Rebalance weights to 0.10 each (equal weighting across 10 models).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use tokio::task::block_in_place for safe sync→async bridging
- Replace count(*) with count(1) in division context (QuestDB parser bug)
- Add coalesce() for stddev NULL handling (single-row edge case)
- Consolidate integration tests into single lifecycle test with DROP+CREATE
- All 4 QuestDB tests pass against real QuestDB 8.2.3
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace stub metrics provider with real QuestDBMetricsProvider that
queries QuestDB via PostgreSQL wire protocol (port 8812) for rolling
model Sharpe, win rates, confidence buckets, and ensemble metrics.
- Add QuestDB service to docker-compose.yml (8.2.3, ports 9009/8812/9003)
- Create questdb_metrics.rs with 5 SQL queries and graceful fallback
- Wire QuestDBMetricsProvider into main.rs (replaces StubMetricsProvider)
- Fix unused Instant import in feedback_loop.rs (#[cfg(test)] scope)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FeedbackLoop runs as a background tokio task, periodically running
weight and gate optimization cycles with kill switch monitoring and
retraining trigger detection. Wired into main.rs with a stub
MetricsProvider until QuestDB is deployed.
7 new tests covering kill switch, freeze/unfreeze, retraining triggers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gate optimizer adjusts conviction gate thresholds based on win-rate per
confidence bucket with cooldown and kill switch safety rails. Model
registry provides lifecycle management (Candidate → Staging → Production
→ Archived) with InMemoryModelRegistry for testing. P&L attribution
decomposes realized trade P&L into per-model contributions using signal
alignment.
24 new tests across 3 modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7-gate conviction system, autonomous feedback loop with kill switch,
and Rust-native model registry — backed by PostgreSQL + QuestDB.
Resolve merge conflicts in hyperopt/adapters/mod.rs and enhanced_ml.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 16 unit tests for Enhanced ML service components:
- EnsembleConfig::default() values (min_models, thresholds, voting)
- FeaturePreprocessor::classify_feature_type() for price, volume,
technical, sentiment, and unknown features
- FeaturePreprocessor normalization (z-score, tanh fallback, zero std_dev)
- FeaturePreprocessor default stats validation
- RuntimeModelInfo creation for all 5 model types (DQN/PPO/TFT/Mamba/LNN)
- ModelPerformanceMetrics::default() zero initialization
- FeatureNormStats volatility default bounds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 25 unit tests for the RiskServiceImpl pure functions:
- Parametric VaR fallback formula (notional * 0.02)
- Equal contribution percentage for N symbols (including empty)
- Drawdown computation (empty, positive PnL, negative, mixed)
- Returns from executions (empty, single, sorted, zero-price filtering)
- Volatility (empty, single, constant, known series)
- Sharpe ratio (insufficient data, zero vol, positive returns)
- Sortino ratio (insufficient data, no downside, mixed)
- VaR square-root-of-time scaling (1d→5d→30d)
- Concentration risk level thresholds
- Risk constants validation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add defensive checks to FeaturePreprocessor::normalize():
- Return 0.0 with warning log for NaN/Inf input values
- Clamp z-score output to [-10, 10] to prevent extreme values
- Add 4 unit tests covering NaN, Inf, -Inf, and extreme value clamping
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move position fetching before VaR calculation in both get_va_r and
get_risk_metrics so the portfolio notional is computed from real
position data (sum of |quantity * avg_price|) instead of the fake
confidence_level * 1_000_000.0 placeholder. Falls back to 100_000.0
when the portfolio is empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Closes the integration gap between Liquid CfC and DQN/PPO by adding:
- LiquidTrainer with gRPC progress callbacks, early stopping, and
checkpoint management (ml/src/trainers/liquid.rs)
- LiquidModel wrapper in enhanced_ml.rs for hot-loading from safetensors
- Ensemble weight rebalance: DQN 0.25, PPO 0.25, TFT 0.20, Mamba2 0.15,
Liquid-CfC 0.15
Verified: 81 liquid unit tests + 3 integration tests + 211 trading_service
tests pass, 0 compile errors across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add load_from_safetensors() to DQNAgent for weight loading via VarMap
- Update RealDQNModel::from_checkpoint to try safetensors first, fall back to JSON
- Replace std::mem::forget(prediction_shutdown_tx) with proper Vec-based storage
that sends shutdown signal and drops senders during graceful shutdown
- Wire order_manager.get_open_orders() into emergency_stop response so callers
see which orders were active when the kill switch engaged
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded position_size: 1000.0 with actual position quantities
from fetch_positions(). Compute contribution_pct as marginal VaR ratio
(symbol_var / portfolio_var * 100) instead of naive equal-weight split.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tasks 8-14 production hardening batch:
- Populate Order/Position/Execution proto messages from JSON payload in event
stream converters instead of returning None (Task 9)
- Compute max_drawdown from cumulative PnL samples in A/B testing pipeline
instead of hardcoded 0.0 (Task 11)
- Document feature pipeline integration blockers with detailed roadmap
comments in state.rs and trading.rs (Task 8)
- Document realized PnL gap: TradingPosition lacks the field, repository
has async method incompatible with Iterator::map (Task 10)
- Document ML order quantity gap in api_gateway proxy: MlOrderResponse
proto lacks quantity field (Task 12)
- Document per-symbol weight tracking roadmap in ensemble_coordinator (Task 13)
- Document OHLCV bar pipeline upgrade roadmap in state.rs (Task 14)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move identical TLS type definitions from 4 service crates into
common/src/tls.rs, eliminating ~435 lines of duplicated code.
Each service retains its own TlsConfig struct and validation logic
(async vs sync, delegated vs monolithic) while sharing the type
definitions. Services re-export the types for backward compatibility.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removed duplicate DQNConfig from agent.rs (13 fields, pre-Rainbow with
f64 gamma/epsilon) and adaptive-strategy stub (unit struct). Canonical
definition in dqn/dqn.rs now has 51 fields covering full Rainbow DQN
plus agent-level trading parameters (minimum_profit_factor, weight_decay).
Key changes:
- agent.rs imports DQNConfig from dqn.rs instead of defining its own
- Fixed f32/f64 type mismatches (epsilon_start/end/decay cast to f64
where QNetworkConfig expects f64)
- Renamed replay_buffer_size -> replay_buffer_capacity across all callers
- Updated 13 files across ml, adaptive-strategy, and trading_service
- All 2009 ml tests pass, 0 clippy warnings in modified files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace two .expect() calls on HashMap lookups with safe if-let pattern
to comply with deny(clippy::expect_used) rule.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire Mamba2 model type into the gRPC model loading match in enhanced_ml.rs.
All four model types (DQN, PPO, TFT, Mamba2) now have real loading paths.
The wildcard arm now returns invalid_argument instead of unimplemented.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Status::unimplemented() with real streaming implementations:
- stream_va_r_updates: periodic VaR recalculation with change detection
- stream_risk_alerts: kill switch status + VaR breach monitoring
- stream_system_status: real-time health, CPU/memory/disk metrics
- stream_metrics: trading performance counters (connections, requests, latency)
- stream_alerts: operational alerts for health/CPU/memory/disk anomalies
All use tokio::time::interval with graceful client disconnect handling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire EnsembleCoordinator with 4 candle-backed inference adapters:
- DQN (0.30 weight), PPO (0.30), TFT (0.20), Mamba2 (0.20)
Add InferenceAdapterBridge to convert ModelInferenceAdapter→MLModel trait.
Models initialize with random weights; production checkpoints hot-loaded
via ModelRegistry shadow-buffer swap. PredictionGenerationLoop now active.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Compute real values for 5 previously-zero risk metrics:
- current_drawdown: from unrealized PnL vs market value
- volatility: annualized std dev from execution price returns
- sharpe_ratio: excess return / volatility * sqrt(252)
- sortino_ratio: excess return / downside deviation * sqrt(252)
- position_risks: per-position VaR contribution and concentration
Wire get_position_risk() to real positions with filtering.
beta/alpha remain 0.0 (requires benchmark data integration).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded 0.0 system metrics with live values from sysinfo
crate (already in dependencies). Uses Arc<RwLock<System>> for thread-
safe refreshing. Network I/O left at 0.0 (requires sustained sampling).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Compliance: return Uuid::nil() + warn! when features disabled instead
of random untraceable UUIDs (SOX, MiFID II, position monitoring, best
execution analysis)
- Audit queue: change fallback path from /tmp/ to /var/lib/foxhunt/,
upgrade fallback log from info to warn for alerting visibility
- Enhanced ML: wire get_ensemble_vote gRPC handler to real ensemble
coordinator instead of hardcoded [0.1, 0.2, -0.05, 0.8] features
- Paper trading: change stub confidence from 0.5 to 0.0 (below 0.6
threshold, preventing accidental orders) and add warn! log
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- WebSocket stream bridges: replace permanent pending() stalls with
actual gRPC streaming subscriptions and exponential backoff reconnection
- Risk metrics: replace fake sharpe_ratio=1.5, sortino_ratio=2.0 with 0.0
(not-yet-computed) and wire circuit breaker status to real kill switch
- ML orders: use actual ensemble prediction direction and confidence
instead of hardcoded BUY at 0.65 confidence
- Fallback signal: return Err instead of fabricated Hold at 0.60 confidence
to prevent accidental trades when ML pipeline is disconnected
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete 22 orphaned files (.backup, .broken_backup, .old, .rej, .disabled)
- Remove duplicate KillSwitch stub from risk_engine.rs, use AtomicKillSwitch
- Deduplicate UnixSocketKillSwitch via re-export from unix_socket module
- Rename StreamingConfig → EventStreamingConfig to resolve naming collision
- Guard MockTradingRepository behind #[cfg(test)] in trading_service
- Replace adaptive-strategy EnsembleConfig with re-export from ml crate
- Merge error_recovery.rs fields into canonical RetryConfig (circuit breaker,
jitter, HFT precision mode) and delete the 328-line dead module
- Replace local 3-variant RiskError with risk::error::RiskError import
- Fix all RetryConfig struct literals with ..Default::default()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- emergency_stop: delegates to TradingServiceKillSwitch.emergency_shutdown()
which activates the global AtomicKillSwitch; returns Status::unavailable
when kill_switch_system is None rather than silently succeeding
- validate_order: reads max_order_quantity from config repository (falls back
to 1_000_000); additionally calls RiskEngine.check_var_limit() for VaR
validation when symbol and price are provided
- get_va_r: uses RiskEngine.calculate_marginal_var() for real VaR with a
parametric fallback; per-symbol marginal VaRs computed individually
- get_risk_metrics: derives portfolio_var_1d from RiskEngine; scales to 5d
and 30d via sqrt-of-time rule; remaining fields (Sharpe, beta, alpha,
current_drawdown) keep placeholder values with explicit TODO comments
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace hardcoded uptime (3600s) with real elapsed time tracked via
Instant::now() in MonitoringServiceImpl constructor. Surface live kill
switch state (active, emergency, unhealthy) as critical_issues strings
in GetSystemStatusResponse. Set sysinfo-dependent CPU/memory/disk/network
metrics to 0.0 with TODO comments pending sysinfo crate integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the hardcoded [0.5, 0.6, 0.7, 0.8, 0.9] mock in
`extract_features_for_symbol` with a real attempt to retrieve
market ticks via `market_data_repository.get_latest_prices`.
Derives a 51-dim feature vector (price mean/std/CV, momentum,
volume mean, tick count) from live tick data, padding unused
dimensions with zeros. Falls back to a zero-filled 51-dim
vector with a WARN log on any failure or empty response, ensuring
the ensemble prediction path is never blocked by missing data.
Also replaces the silent `info!` in `get_fallback_trading_signal`
with a WARN-level message that makes the mock-signal boundary
clearly visible in production logs and adds a TODO pointing to
real DQN epoch-30 checkpoint inference.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- allocation.rs: get_asset_volatilities now uses flat 0.20 (20% annual vol)
instead of index-scaled 0.15+i*0.05; get_covariance_matrix diagonal is
0.04 (0.20^2) instead of 0.0225; get_ml_predictions uses 0.0 (neutral)
instead of 0.05+i*0.02. All three emit tracing::warn so mock state is
visible in production logs.
- training.rs: train_all emits tracing::warn that it is using a placeholder
loop and directs callers to use model-specific trainers for production.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- trading_engine/src/types/metrics.rs: Replace 4 panic!() in Lazy statics
with eprintln + std::process::abort() (avoids clippy::panic lint)
- trading_service: Fix test compilation after PaperTradingExecutor::new()
return type change to Result
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
