The rate limiter trusted the client-controlled X-Forwarded-For header
to identify clients. An attacker could rotate this header value on every
request to bypass rate limits entirely.
Now uses ConnectInfo<SocketAddr> (the actual TCP connection address) as
the rate limiting key. Server is configured with
into_make_service_with_connect_info to populate this.
X-Forwarded-For and X-Real-Ip headers are no longer consulted.
Falls back to "unknown" if ConnectInfo is unavailable (tests).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_historical_returns was returning a hardcoded 20-value vector of fake
returns for ALL symbols, causing Kelly criterion to compute position sizes
based on fabricated data. Now returns an error explaining that no market
data feed is connected.
calculate_position_size catches the Kelly/PPO sizing errors and falls
back to standard fixed-fraction sizing with a warning log, instead of
propagating the error to callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The VaR engine's get_symbol_volatility used hardcoded per-asset-class
annual volatility values (e.g. 25% for equities, 80% for crypto) without
any indication to operators that real market data was not being used.
Now emits a tracing::warn on every static volatility lookup so operators
see it in logs. Also adds a volatility_overrides HashMap<String, f64>
field on VarEngine for manual per-symbol overrides until a real market
data feed is integrated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
reconnect() previously slept 500ms then unconditionally set
SessionState::Active, faking a successful reconnection regardless of
whether any broker or infrastructure was actually reachable.
Now performs a DB health check (SELECT 1) after the sleep. If the
check fails, state is set to Disconnected and an error is returned.
Active is only set when the health check passes. The DB ping is the
best available liveness probe until a real FIX/cTrader session is
wired into SessionRecovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A single CTRADER_LIVE=true env var was enough to route real money orders
through the broker. Now requires CTRADER_LIVE_CONFIRMED=I_UNDERSTAND_REAL_MONEY
alongside it, preventing accidental live trading from copy-pasted configs.
Also adds prominent warning log when live mode activates and info log
for demo mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both route_order and cancel_order used let _ = to discard DB update
errors after successful broker operations. This means an order could be
live at the broker while the database still shows PENDING_SUBMIT or
CANCEL_PENDING, with no log entry to alert operators.
Replaced with if let Err(db_err) that logs at error level with CRITICAL
prefix, client_order_id, and broker_order_id for manual reconciliation.
The RPC still returns success since the broker operation completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous (quantity * 100_000.0) as i64 silently truncated
fractional lots and could overflow on extreme values. Added
convert_quantity_to_volume() that validates the result is finite,
non-negative, and within i64 range, using round() instead of
truncation. Includes 7 unit tests covering normal values, fractional
rounding, overflow, negative, NaN, infinity, and zero edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
from_checkpoint was creating fresh models with random weights and setting
is_trained=true, allowing trading on noise. Now validates checkpoint exists
and does NOT set is_trained=true when weights are random.
Mamba2 predict errors are now wrapped as MLError::InferenceError so the
fallback manager can degrade model health. Both TFT and Mamba2 predict
refuse to run on untrained models, and is_ready() reflects trained state.
Also fixes TFT input_dim mismatch (16 vs 5+10+16=31).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
try_write() is non-blocking and silently drops the CTraderClient when
the lock is contended. This means a successfully connected broker
client could be lost without any error. Changed to write().await which
guarantees the client is stored. Made function async and updated the
call site in main.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Low-confidence ensemble votes now forced to Hold (was letting weak
signals through to order generation in both vote paths)
- Inference latency tracking uses EMA (alpha=0.1) instead of
overwriting with the latest sample
- retrain_model returns Status::Unimplemented instead of faking
success with a random job_id
- get_model_performance returns honest zeroes instead of hardcoded
fake metrics (accuracy=0.85 etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use chrono-tz America/New_York for correct EST/EDT trading session
boundaries (was hardcoded UTC-5, off by 1h during daylight saving)
- Normalize effective weights to sum to 1.0 before signal aggregation
(raw weights could sum to anything, biasing the ensemble)
- Skip adapter predictions that return NaN/Inf direction or confidence
instead of letting them poison the weighted average
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NaN initial features propagate through the PPO policy network and
produce garbage position sizing on the very first inference call.
Replace with 0.0 so the first prediction is safe (neutral).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was only checking quantity limit and VaR. Now runs all 5 risk checks:
1. Kill switch / circuit breaker (via TradingServiceKillSwitch)
2. Max order size + position limits (via RiskRepository.get_risk_limits)
3. Daily loss / drawdown limit (via RiskRepository.get_risk_metrics)
4. Leverage limit (via config + RiskRepository.get_risk_metrics)
5. VaR limit (via RiskEngine.check_var_limit, existing)
Violations accumulate rather than short-circuit so callers see all
failures at once. Added 6 tests verifying violation type coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VaR, drawdown, and leverage checks in check_risk_limits() only logged
warnings and returned Ok(()). Now they return errors to block order
generation when limits are breached. Also removes unused warn import.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Linear extrapolation above 0.99 gave z=2.48 for 99.9% (correct: 3.09).
VaR was understated by ~25% at high confidence levels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unlimited concurrent probes could overwhelm recovering services.
Added probe_in_flight guard to ensure only one request probes the
recovering service at a time in HalfOpen state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-account circuit breaker and daily loss tracking was never applied to
real accounts because check_order() hardcoded "default". Added account_id
field to OrderInfo with backward-compatible None fallback. Updated all
construction sites across risk crate tests and integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Position and leverage checks returned Approved by default when
broker_account_service was None. Now returns ServiceUnavailable
error to prevent unvalidated orders from passing through.
Added 3 tests verifying the fail-safe behavior for both
check_position_limits and check_leverage_limits, plus the
public check_order integration path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Duplicate execution reports could double fill_quantity and corrupt
average price. Added guards in process_execution to reject fills on
already-Filled orders and to reject fills that would exceed order qty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
validate_order and add_order were separate operations. The gap between
them allowed duplicate order IDs to both pass validation because
validate_order only took a read lock. New validate_and_add_order method
holds a single write lock for the entire check-and-insert operation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The update_position method previously acquired a read lock to get the
Arc<AtomicPosition>, dropped it, then called update_with_execution
outside any lock. Two concurrent fills for the same position could both
load the same old_quantity via Acquire, compute independent new quantities,
and the last Release store would silently discard the other fill — causing
the position to show e.g. 10 shares when it should show 20.
Now holds the positions write lock across the entire get-or-create +
update_with_execution sequence, serializing concurrent fills per position.
The validate_position_update call remains outside the lock to minimize
hold time (it does its own async reads and risk checks).
Also removes excessive per-step RDTSC latency tracking from the critical
path (total latency tracking is preserved) and adds a multi-threaded
regression test that spawns 10 concurrent +1 fills and asserts the final
quantity equals 10.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old code computed `_execution_value = quantity * price` but
discarded it (underscore prefix). Only commission was subtracted,
so the account cash balance never reflected the cost of buying or
the proceeds from selling.
Now `update_from_execution` matches on the new `execution.side` field:
- Buy → cash -= execution_value + commission
- Sell → cash += execution_value − commission
Test assertions updated to expect the corrected balances.
Addresses audit item H2 (account never deducting trade value).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ExecutionResult.executed_quantity is always a positive magnitude, so the
old `is_buy = executed_quantity > ZERO` check was always true — every
fill was treated as a buy regardless of order side.
Add an explicit `pub side: OrderSide` field to ExecutionResult and use
`execution.side == OrderSide::Buy` in PositionManager. All construction
sites (source, tests, benchmarks) updated; sell-side tests now use
positive quantities with `side: OrderSide::Sell` instead of the former
negative-quantity hack.
Addresses audit item H4 (position direction always buy).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clone_for_async() was creating independent AtomicU64 instances for
message_count, drop_count, last_heartbeat, and reconnect_attempts
instead of sharing the originals. This meant spawned tasks incremented
their own counters while get_stats() read the original's (always 0),
and the heartbeat monitor watched a counter never updated by the
connection task.
Also fixes last_heartbeat initializing to 0, which caused the first
heartbeat check to compute a huge elapsed time and immediately trigger
a false "connection appears dead" alert.
Changes:
- Change 4 struct fields from AtomicU64 to Arc<AtomicU64>
- Initialize last_heartbeat to HardwareTimestamp::now() instead of 0
- clone_for_async() now clones the Arcs (shared counters)
- Add test verifying counters are shared between original and clone
- Add test verifying heartbeat initialized to current time
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Casting negative f64 to u64 saturates to 0, corrupting avg_price and all
downstream PnL calculations. Add price_to_fixed_checked() that rejects
non-finite, zero, and negative prices with PositionError::InvalidPrice.
The critical update_with_execution path now uses the checked variant.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The get_orders() status filter used `matches!(order.status, _status)` which
creates a new wildcard binding instead of comparing against the captured
`status` variable. This caused every order to match regardless of filter,
meaning get_orders(Some(Filled)) returned ALL orders — inflating exposure
calculations. Replaced with direct equality comparison `order.status == *status`.
Added test_get_orders_status_filter_only_returns_matching to prevent regression.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ubuntu 22.04's protobuf-compiler (v3.12) lacks proto3 optional field
support. Install protoc v25.1 from GitHub releases instead, which
matches the protoc version available in Debian bookworm-based images
used by the other 5 services.
Validated: docker build completes successfully, binary starts with
ml_training_service --help.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 service Dockerfiles were missing newly-added workspace crates
(web-gateway, ctrader-openapi, foxhunt-deploy, broker_gateway_service),
causing cargo workspace resolution failures during Docker builds.
Changes across all Dockerfiles:
- Add COPY directives for web-gateway, ctrader-openapi, foxhunt-deploy,
broker_gateway_service (new workspace members since Dockerfiles written)
- Add SQLX_OFFLINE=true env and .sqlx cache copy where missing
- Add perl and make system deps (needed for OpenSSL build from source)
- Remove COPY migrations (dir excluded by .dockerignore, not needed)
- Expand broker_gateway_service from 3-crate to full workspace copy
Validated: docker build --check passes all 6, cargo check -p passes all 6.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Essential workflow fixes:
- ci.yml: add SQLX_OFFLINE=true, replace fictitious cargo subcommands
(test-unit, ci-lint, audit-deps, etc.) with real cargo commands,
remove broken 9-way test matrix, remove instrument-coverage RUSTFLAGS
- test.yml: add SQLX_OFFLINE=true to build-check job, fix conflicting
clippy flags (-D and -W on clippy::all), update JWT secret to 32+ chars
- compilation-guard.yml: add SQLX_OFFLINE=true, remove references to
non-existent paths (services/trading-engine, crates/common/types),
remove dangerous auto-commit-to-main, remove MIRI on missing packages
Deleted duplicates (justification):
- comprehensive_testing.yml: duplicate of comprehensive-testing.yml
(same purpose, underscore vs hyphen naming)
- production-deploy.yml: duplicate of production-deployment.yml
(both named "Production Deployment Pipeline", this one has stale paths)
- coverage-fixed.yml: duplicate of coverage.yml
(uses deprecated actions-rs/toolchain@v1 and actions/cache@v3)
- performance.yml: duplicate of benchmark_regression.yml
(both "Performance Regression Detection", this one less mature)
- quality-baseline.json: not a workflow, stale fake data (999 warnings)
- quality-metrics.json: not a workflow, stale fake data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Proves the critical trading pipeline path works end-to-end:
OHLCV -> features -> ensemble -> prediction. Uses all 10 real candle
inference adapters (DQN, PPO, TFT, Mamba2, Liquid-CfC, TGGN, TLOB,
KAN, xLSTM, Diffusion) with random weights, 60 synthetic OHLCV bars,
and asserts confidence/action/model-count invariants. No external
dependencies (no DB, no Docker).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Spawn a background tokio task that feeds synthetic OHLCV bars into the
ensemble coordinator's per-symbol feature extractors. The extractors
require ~51 bars of warmup before they can produce real 51-dim feature
vectors; without this feed they remain cold and return zeros.
The task uses a random-walk price generator with realistic base prices
for the four baseline futures symbols (ES, NQ, ZN, 6E) and configurable
interval (MARKET_FEED_INTERVAL_MS, default 1s) and symbol list
(MARKET_FEED_SYMBOLS). In production this will be replaced by a
Databento or exchange data feed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register TGGN, TLOB, KAN, xLSTM, and Diffusion inference adapters
alongside the existing DQN, PPO, TFT, Mamba2, and Liquid-CfC.
Rebalance weights to 0.10 each (equal weighting across 10 models).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sequence-buffered adapter with 3-layer MLP projection
(flat_dim -> hidden -> hidden/2 -> 1). Ring buffer collects
seq_len feature vectors, returns neutral prediction until full,
then flattens and projects through the MLP with sigmoid output
mapping to direction [-1,1] and confidence [0,1].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove panic!() calls from test_futures_baseline_micro_mapping,
use map()+Some() pattern consistent with rest of test suite
- Make DatasetSpec::from_universe() accept a name parameter instead
of hardcoding "futures-baseline"
- Add doc comment to UniverseConfigMeta explaining dead_code fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Detailed TDD implementation plan for wiring all 10 ML models into
the ensemble coordinator. Covers 5 new inference adapters (KAN, TGGN,
xLSTM, TLOB, Diffusion), main.rs registration with equal weights,
market data wiring, E2E integration test, Docker/CI validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire existing components into end-to-end pipeline: 5 missing inference
adapters, 10-model registration, market data feeding, integration test,
Docker build validation, and CI pipeline triage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove duplicate AssetClass enum from data_pipeline module and replace
with a re-export from asset_selection, which is now the canonical
location. This ensures data_pipeline::AssetClass and
asset_selection::AssetClass are the same type, enabling direct
comparison and preventing subtle type mismatch bugs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
10-task plan: unify AssetClass, add trading_symbol field, futures_baseline()
preset, TOML config loading, DatasetSpec::from_universe() wiring, data
reorganization into cache structure, stale data cleanup, manifest generation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
