- Low-confidence ensemble votes now forced to Hold (was letting weak
signals through to order generation in both vote paths)
- Inference latency tracking uses EMA (alpha=0.1) instead of
overwriting with the latest sample
- retrain_model returns Status::Unimplemented instead of faking
success with a random job_id
- get_model_performance returns honest zeroes instead of hardcoded
fake metrics (accuracy=0.85 etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use chrono-tz America/New_York for correct EST/EDT trading session
boundaries (was hardcoded UTC-5, off by 1h during daylight saving)
- Normalize effective weights to sum to 1.0 before signal aggregation
(raw weights could sum to anything, biasing the ensemble)
- Skip adapter predictions that return NaN/Inf direction or confidence
instead of letting them poison the weighted average
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NaN initial features propagate through the PPO policy network and
produce garbage position sizing on the very first inference call.
Replace with 0.0 so the first prediction is safe (neutral).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was only checking quantity limit and VaR. Now runs all 5 risk checks:
1. Kill switch / circuit breaker (via TradingServiceKillSwitch)
2. Max order size + position limits (via RiskRepository.get_risk_limits)
3. Daily loss / drawdown limit (via RiskRepository.get_risk_metrics)
4. Leverage limit (via config + RiskRepository.get_risk_metrics)
5. VaR limit (via RiskEngine.check_var_limit, existing)
Violations accumulate rather than short-circuit so callers see all
failures at once. Added 6 tests verifying violation type coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VaR, drawdown, and leverage checks in check_risk_limits() only logged
warnings and returned Ok(()). Now they return errors to block order
generation when limits are breached. Also removes unused warn import.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Linear extrapolation above 0.99 gave z=2.48 for 99.9% (correct: 3.09).
VaR was understated by ~25% at high confidence levels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unlimited concurrent probes could overwhelm recovering services.
Added probe_in_flight guard to ensure only one request probes the
recovering service at a time in HalfOpen state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-account circuit breaker and daily loss tracking was never applied to
real accounts because check_order() hardcoded "default". Added account_id
field to OrderInfo with backward-compatible None fallback. Updated all
construction sites across risk crate tests and integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Position and leverage checks returned Approved by default when
broker_account_service was None. Now returns ServiceUnavailable
error to prevent unvalidated orders from passing through.
Added 3 tests verifying the fail-safe behavior for both
check_position_limits and check_leverage_limits, plus the
public check_order integration path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Duplicate execution reports could double fill_quantity and corrupt
average price. Added guards in process_execution to reject fills on
already-Filled orders and to reject fills that would exceed order qty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
validate_order and add_order were separate operations. The gap between
them allowed duplicate order IDs to both pass validation because
validate_order only took a read lock. New validate_and_add_order method
holds a single write lock for the entire check-and-insert operation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The update_position method previously acquired a read lock to get the
Arc<AtomicPosition>, dropped it, then called update_with_execution
outside any lock. Two concurrent fills for the same position could both
load the same old_quantity via Acquire, compute independent new quantities,
and the last Release store would silently discard the other fill — causing
the position to show e.g. 10 shares when it should show 20.
Now holds the positions write lock across the entire get-or-create +
update_with_execution sequence, serializing concurrent fills per position.
The validate_position_update call remains outside the lock to minimize
hold time (it does its own async reads and risk checks).
Also removes excessive per-step RDTSC latency tracking from the critical
path (total latency tracking is preserved) and adds a multi-threaded
regression test that spawns 10 concurrent +1 fills and asserts the final
quantity equals 10.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old code computed `_execution_value = quantity * price` but
discarded it (underscore prefix). Only commission was subtracted,
so the account cash balance never reflected the cost of buying or
the proceeds from selling.
Now `update_from_execution` matches on the new `execution.side` field:
- Buy → cash -= execution_value + commission
- Sell → cash += execution_value − commission
Test assertions updated to expect the corrected balances.
Addresses audit item H2 (account never deducting trade value).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ExecutionResult.executed_quantity is always a positive magnitude, so the
old `is_buy = executed_quantity > ZERO` check was always true — every
fill was treated as a buy regardless of order side.
Add an explicit `pub side: OrderSide` field to ExecutionResult and use
`execution.side == OrderSide::Buy` in PositionManager. All construction
sites (source, tests, benchmarks) updated; sell-side tests now use
positive quantities with `side: OrderSide::Sell` instead of the former
negative-quantity hack.
Addresses audit item H4 (position direction always buy).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clone_for_async() was creating independent AtomicU64 instances for
message_count, drop_count, last_heartbeat, and reconnect_attempts
instead of sharing the originals. This meant spawned tasks incremented
their own counters while get_stats() read the original's (always 0),
and the heartbeat monitor watched a counter never updated by the
connection task.
Also fixes last_heartbeat initializing to 0, which caused the first
heartbeat check to compute a huge elapsed time and immediately trigger
a false "connection appears dead" alert.
Changes:
- Change 4 struct fields from AtomicU64 to Arc<AtomicU64>
- Initialize last_heartbeat to HardwareTimestamp::now() instead of 0
- clone_for_async() now clones the Arcs (shared counters)
- Add test verifying counters are shared between original and clone
- Add test verifying heartbeat initialized to current time
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Casting negative f64 to u64 saturates to 0, corrupting avg_price and all
downstream PnL calculations. Add price_to_fixed_checked() that rejects
non-finite, zero, and negative prices with PositionError::InvalidPrice.
The critical update_with_execution path now uses the checked variant.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The get_orders() status filter used `matches!(order.status, _status)` which
creates a new wildcard binding instead of comparing against the captured
`status` variable. This caused every order to match regardless of filter,
meaning get_orders(Some(Filled)) returned ALL orders — inflating exposure
calculations. Replaced with direct equality comparison `order.status == *status`.
Added test_get_orders_status_filter_only_returns_matching to prevent regression.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ubuntu 22.04's protobuf-compiler (v3.12) lacks proto3 optional field
support. Install protoc v25.1 from GitHub releases instead, which
matches the protoc version available in Debian bookworm-based images
used by the other 5 services.
Validated: docker build completes successfully, binary starts with
ml_training_service --help.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 service Dockerfiles were missing newly-added workspace crates
(web-gateway, ctrader-openapi, foxhunt-deploy, broker_gateway_service),
causing cargo workspace resolution failures during Docker builds.
Changes across all Dockerfiles:
- Add COPY directives for web-gateway, ctrader-openapi, foxhunt-deploy,
broker_gateway_service (new workspace members since Dockerfiles written)
- Add SQLX_OFFLINE=true env and .sqlx cache copy where missing
- Add perl and make system deps (needed for OpenSSL build from source)
- Remove COPY migrations (dir excluded by .dockerignore, not needed)
- Expand broker_gateway_service from 3-crate to full workspace copy
Validated: docker build --check passes all 6, cargo check -p passes all 6.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Essential workflow fixes:
- ci.yml: add SQLX_OFFLINE=true, replace fictitious cargo subcommands
(test-unit, ci-lint, audit-deps, etc.) with real cargo commands,
remove broken 9-way test matrix, remove instrument-coverage RUSTFLAGS
- test.yml: add SQLX_OFFLINE=true to build-check job, fix conflicting
clippy flags (-D and -W on clippy::all), update JWT secret to 32+ chars
- compilation-guard.yml: add SQLX_OFFLINE=true, remove references to
non-existent paths (services/trading-engine, crates/common/types),
remove dangerous auto-commit-to-main, remove MIRI on missing packages
Deleted duplicates (justification):
- comprehensive_testing.yml: duplicate of comprehensive-testing.yml
(same purpose, underscore vs hyphen naming)
- production-deploy.yml: duplicate of production-deployment.yml
(both named "Production Deployment Pipeline", this one has stale paths)
- coverage-fixed.yml: duplicate of coverage.yml
(uses deprecated actions-rs/toolchain@v1 and actions/cache@v3)
- performance.yml: duplicate of benchmark_regression.yml
(both "Performance Regression Detection", this one less mature)
- quality-baseline.json: not a workflow, stale fake data (999 warnings)
- quality-metrics.json: not a workflow, stale fake data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Proves the critical trading pipeline path works end-to-end:
OHLCV -> features -> ensemble -> prediction. Uses all 10 real candle
inference adapters (DQN, PPO, TFT, Mamba2, Liquid-CfC, TGGN, TLOB,
KAN, xLSTM, Diffusion) with random weights, 60 synthetic OHLCV bars,
and asserts confidence/action/model-count invariants. No external
dependencies (no DB, no Docker).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Spawn a background tokio task that feeds synthetic OHLCV bars into the
ensemble coordinator's per-symbol feature extractors. The extractors
require ~51 bars of warmup before they can produce real 51-dim feature
vectors; without this feed they remain cold and return zeros.
The task uses a random-walk price generator with realistic base prices
for the four baseline futures symbols (ES, NQ, ZN, 6E) and configurable
interval (MARKET_FEED_INTERVAL_MS, default 1s) and symbol list
(MARKET_FEED_SYMBOLS). In production this will be replaced by a
Databento or exchange data feed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register TGGN, TLOB, KAN, xLSTM, and Diffusion inference adapters
alongside the existing DQN, PPO, TFT, Mamba2, and Liquid-CfC.
Rebalance weights to 0.10 each (equal weighting across 10 models).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sequence-buffered adapter with 3-layer MLP projection
(flat_dim -> hidden -> hidden/2 -> 1). Ring buffer collects
seq_len feature vectors, returns neutral prediction until full,
then flattens and projects through the MLP with sigmoid output
mapping to direction [-1,1] and confidence [0,1].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove panic!() calls from test_futures_baseline_micro_mapping,
use map()+Some() pattern consistent with rest of test suite
- Make DatasetSpec::from_universe() accept a name parameter instead
of hardcoding "futures-baseline"
- Add doc comment to UniverseConfigMeta explaining dead_code fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Detailed TDD implementation plan for wiring all 10 ML models into
the ensemble coordinator. Covers 5 new inference adapters (KAN, TGGN,
xLSTM, TLOB, Diffusion), main.rs registration with equal weights,
market data wiring, E2E integration test, Docker/CI validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire existing components into end-to-end pipeline: 5 missing inference
adapters, 10-model registration, market data feeding, integration test,
Docker build validation, and CI pipeline triage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove duplicate AssetClass enum from data_pipeline module and replace
with a re-export from asset_selection, which is now the canonical
location. This ensures data_pipeline::AssetClass and
asset_selection::AssetClass are the same type, enabling direct
comparison and preventing subtle type mismatch bugs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
10-task plan: unify AssetClass, add trading_symbol field, futures_baseline()
preset, TOML config loading, DatasetSpec::from_universe() wiring, data
reorganization into cache structure, stale data cleanup, manifest generation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Define futures baseline universe (ES, NQ, ZN, 6E) with micro
contract mapping for limited capital. Reorganize scattered test
data into cache structure. Download spec for 730 days of OHLCV-1m.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use tokio::task::block_in_place for safe sync→async bridging
- Replace count(*) with count(1) in division context (QuestDB parser bug)
- Add coalesce() for stddev NULL handling (single-row edge case)
- Consolidate integration tests into single lifecycle test with DROP+CREATE
- All 4 QuestDB tests pass against real QuestDB 8.2.3
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace stub metrics provider with real QuestDBMetricsProvider that
queries QuestDB via PostgreSQL wire protocol (port 8812) for rolling
model Sharpe, win rates, confidence buckets, and ensemble metrics.
- Add QuestDB service to docker-compose.yml (8.2.3, ports 9009/8812/9003)
- Create questdb_metrics.rs with 5 SQL queries and graceful fallback
- Wire QuestDBMetricsProvider into main.rs (replaces StubMetricsProvider)
- Fix unused Instant import in feedback_loop.rs (#[cfg(test)] scope)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FeedbackLoop runs as a background tokio task, periodically running
weight and gate optimization cycles with kill switch monitoring and
retraining trigger detection. Wired into main.rs with a stub
MetricsProvider until QuestDB is deployed.
7 new tests covering kill switch, freeze/unfreeze, retraining triggers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gate optimizer adjusts conviction gate thresholds based on win-rate per
confidence bucket with cooldown and kill switch safety rails. Model
registry provides lifecycle management (Candidate → Staging → Production
→ Archived) with InMemoryModelRegistry for testing. P&L attribution
decomposes realized trade P&L into per-model contributions using signal
alignment.
24 new tests across 3 modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
