- Remove !.env.runpod exclusion from .gitignore (file says "DO NOT COMMIT")
- Untrack .claude/ralph-loop.local.md (session-local state)
- .env.runpod.template remains tracked as the safe reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
.claude/settings.local.json was already in .gitignore but still tracked.
.serena/project.yml is IDE-specific config that shouldn't be versioned.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Shared DBN loading, timestamp dedup, and spread cost helpers used by
train_baseline, evaluate_baseline, and hyperopt_baseline extracted to
ml/examples/baseline_common/mod.rs to eliminate ~190 lines of duplication.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three critical PPO fixes:
1. Add PPO::act_with_log_prob() returning (action, log_prob, value).
The existing act() discarded the policy log-probability, making
PPO importance sampling use wrong ratios during training.
2. Cap trajectory length with --max-steps-per-epoch in train_baseline
PPO path. DQN already had this limit; PPO iterated all features
(~500K per fold), causing OOM on 4GB GPU.
3. Replace random actions and fake log_prob/value in hyperopt PPO
adapter with real agent.act_with_log_prob() calls. Trajectories
now reflect actual policy behavior for meaningful hyperopt.
Also adds warmup offset alignment to PPO trajectory collection
(matching the DQN fix) and fixes .unwrap() in test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ignore ML checkpoints, trained model safetensors, stray ml/ml/ dir,
and .claude/worktrees/. Clean up duplicate hive-mind-prompt entries.
Add 17 design/implementation plan docs from 2026-02-20 to 2026-02-22.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stashed changes had real safetensors weight loading via VarMap::load.
Audit branch was conservative (validate-only). Kept the real loading
and simplified Mamba2 to use the same VarMap::load pattern as TFT.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The rate limiter trusted the client-controlled X-Forwarded-For header
to identify clients. An attacker could rotate this header value on every
request to bypass rate limits entirely.
Now uses ConnectInfo<SocketAddr> (the actual TCP connection address) as
the rate limiting key. Server is configured with
into_make_service_with_connect_info to populate this.
X-Forwarded-For and X-Real-Ip headers are no longer consulted.
Falls back to "unknown" if ConnectInfo is unavailable (tests).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_historical_returns was returning a hardcoded 20-value vector of fake
returns for ALL symbols, causing Kelly criterion to compute position sizes
based on fabricated data. Now returns an error explaining that no market
data feed is connected.
calculate_position_size catches the Kelly/PPO sizing errors and falls
back to standard fixed-fraction sizing with a warning log, instead of
propagating the error to callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The VaR engine's get_symbol_volatility used hardcoded per-asset-class
annual volatility values (e.g. 25% for equities, 80% for crypto) without
any indication to operators that real market data was not being used.
Now emits a tracing::warn on every static volatility lookup so operators
see it in logs. Also adds a volatility_overrides HashMap<String, f64>
field on VarEngine for manual per-symbol overrides until a real market
data feed is integrated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
reconnect() previously slept 500ms then unconditionally set
SessionState::Active, faking a successful reconnection regardless of
whether any broker or infrastructure was actually reachable.
Now performs a DB health check (SELECT 1) after the sleep. If the
check fails, state is set to Disconnected and an error is returned.
Active is only set when the health check passes. The DB ping is the
best available liveness probe until a real FIX/cTrader session is
wired into SessionRecovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A single CTRADER_LIVE=true env var was enough to route real money orders
through the broker. Now requires CTRADER_LIVE_CONFIRMED=I_UNDERSTAND_REAL_MONEY
alongside it, preventing accidental live trading from copy-pasted configs.
Also adds prominent warning log when live mode activates and info log
for demo mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both route_order and cancel_order used let _ = to discard DB update
errors after successful broker operations. This means an order could be
live at the broker while the database still shows PENDING_SUBMIT or
CANCEL_PENDING, with no log entry to alert operators.
Replaced with if let Err(db_err) that logs at error level with CRITICAL
prefix, client_order_id, and broker_order_id for manual reconciliation.
The RPC still returns success since the broker operation completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous (quantity * 100_000.0) as i64 silently truncated
fractional lots and could overflow on extreme values. Added
convert_quantity_to_volume() that validates the result is finite,
non-negative, and within i64 range, using round() instead of
truncation. Includes 7 unit tests covering normal values, fractional
rounding, overflow, negative, NaN, infinity, and zero edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
from_checkpoint was creating fresh models with random weights and setting
is_trained=true, allowing trading on noise. Now validates checkpoint exists
and does NOT set is_trained=true when weights are random.
Mamba2 predict errors are now wrapped as MLError::InferenceError so the
fallback manager can degrade model health. Both TFT and Mamba2 predict
refuse to run on untrained models, and is_ready() reflects trained state.
Also fixes TFT input_dim mismatch (16 vs 5+10+16=31).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
try_write() is non-blocking and silently drops the CTraderClient when
the lock is contended. This means a successfully connected broker
client could be lost without any error. Changed to write().await which
guarantees the client is stored. Made function async and updated the
call site in main.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Low-confidence ensemble votes now forced to Hold (was letting weak
signals through to order generation in both vote paths)
- Inference latency tracking uses EMA (alpha=0.1) instead of
overwriting with the latest sample
- retrain_model returns Status::Unimplemented instead of faking
success with a random job_id
- get_model_performance returns honest zeroes instead of hardcoded
fake metrics (accuracy=0.85 etc.)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use chrono-tz America/New_York for correct EST/EDT trading session
boundaries (was hardcoded UTC-5, off by 1h during daylight saving)
- Normalize effective weights to sum to 1.0 before signal aggregation
(raw weights could sum to anything, biasing the ensemble)
- Skip adapter predictions that return NaN/Inf direction or confidence
instead of letting them poison the weighted average
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NaN initial features propagate through the PPO policy network and
produce garbage position sizing on the very first inference call.
Replace with 0.0 so the first prediction is safe (neutral).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was only checking quantity limit and VaR. Now runs all 5 risk checks:
1. Kill switch / circuit breaker (via TradingServiceKillSwitch)
2. Max order size + position limits (via RiskRepository.get_risk_limits)
3. Daily loss / drawdown limit (via RiskRepository.get_risk_metrics)
4. Leverage limit (via config + RiskRepository.get_risk_metrics)
5. VaR limit (via RiskEngine.check_var_limit, existing)
Violations accumulate rather than short-circuit so callers see all
failures at once. Added 6 tests verifying violation type coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VaR, drawdown, and leverage checks in check_risk_limits() only logged
warnings and returned Ok(()). Now they return errors to block order
generation when limits are breached. Also removes unused warn import.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Linear extrapolation above 0.99 gave z=2.48 for 99.9% (correct: 3.09).
VaR was understated by ~25% at high confidence levels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unlimited concurrent probes could overwhelm recovering services.
Added probe_in_flight guard to ensure only one request probes the
recovering service at a time in HalfOpen state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per-account circuit breaker and daily loss tracking was never applied to
real accounts because check_order() hardcoded "default". Added account_id
field to OrderInfo with backward-compatible None fallback. Updated all
construction sites across risk crate tests and integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Position and leverage checks returned Approved by default when
broker_account_service was None. Now returns ServiceUnavailable
error to prevent unvalidated orders from passing through.
Added 3 tests verifying the fail-safe behavior for both
check_position_limits and check_leverage_limits, plus the
public check_order integration path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Duplicate execution reports could double fill_quantity and corrupt
average price. Added guards in process_execution to reject fills on
already-Filled orders and to reject fills that would exceed order qty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix zstd decoder in train_baseline.rs and evaluate_baseline.rs (same
pattern as hyperopt adapters — branch on .dbn.zst extension). Add CLI
flags for walk-forward config (train/val/test/step months), learning
rate, and max-steps-per-epoch to make pipeline validation feasible.
Pipeline validated end-to-end: hyperopt (5 trials, best Sharpe 2.37) →
walk-forward training (4 folds, 6/1/1 month windows on ES.FUT) →
evaluation (4 fold test sets, checkpoints + norm stats saved).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PPO hyperopt: Split DATA 80/20 for train/val (not trajectory count).
Previously indexed training_data[..num_trajectories] which was only
16 samples from 56K — causing num_batches=0 and NaN from 0/0 division.
Now properly splits data array and uses min(64, num_train) for batch
episodes with ceil division for batch count.
DQN: Fix hardcoded "45 actions" in diagnostic log (now uses actual
q_vec.len()). Replace indexed[..top_n] slice with .take(top_n)
iterator to prevent potential slice panic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
validate_order and add_order were separate operations. The gap between
them allowed duplicate order IDs to both pass validation because
validate_order only took a read lock. New validate_and_add_order method
holds a single write lock for the entire check-and-insert operation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The update_position method previously acquired a read lock to get the
Arc<AtomicPosition>, dropped it, then called update_with_execution
outside any lock. Two concurrent fills for the same position could both
load the same old_quantity via Acquire, compute independent new quantities,
and the last Release store would silently discard the other fill — causing
the position to show e.g. 10 shares when it should show 20.
Now holds the positions write lock across the entire get-or-create +
update_with_execution sequence, serializing concurrent fills per position.
The validate_position_update call remains outside the lock to minimize
hold time (it does its own async reads and risk checks).
Also removes excessive per-step RDTSC latency tracking from the critical
path (total latency tracking is preserved) and adds a multi-threaded
regression test that spawns 10 concurrent +1 fills and asserts the final
quantity equals 10.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Databento data pipeline stored files as .dbn.zst in symbol subdirectories
(e.g., ES.FUT/ES.FUT_2024-Q1.dbn.zst), but the hyperopt adapters and DQN
data loader only matched .dbn extension and used flat directory listing.
Three fixes:
- Match both .dbn and .dbn.zst file extensions in collect_dbn_files_recursive
- Use recursive directory traversal instead of flat read_dir
- Branch on extension to use Decoder::from_zstd_file() for .dbn.zst files
(from_file() doesn't handle zstd decompression)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old code computed `_execution_value = quantity * price` but
discarded it (underscore prefix). Only commission was subtracted,
so the account cash balance never reflected the cost of buying or
the proceeds from selling.
Now `update_from_execution` matches on the new `execution.side` field:
- Buy → cash -= execution_value + commission
- Sell → cash += execution_value − commission
Test assertions updated to expect the corrected balances.
Addresses audit item H2 (account never deducting trade value).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ExecutionResult.executed_quantity is always a positive magnitude, so the
old `is_buy = executed_quantity > ZERO` check was always true — every
fill was treated as a buy regardless of order side.
Add an explicit `pub side: OrderSide` field to ExecutionResult and use
`execution.side == OrderSide::Buy` in PositionManager. All construction
sites (source, tests, benchmarks) updated; sell-side tests now use
positive quantities with `side: OrderSide::Sell` instead of the former
negative-quantity hack.
Addresses audit item H4 (position direction always buy).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clone_for_async() was creating independent AtomicU64 instances for
message_count, drop_count, last_heartbeat, and reconnect_attempts
instead of sharing the originals. This meant spawned tasks incremented
their own counters while get_stats() read the original's (always 0),
and the heartbeat monitor watched a counter never updated by the
connection task.
Also fixes last_heartbeat initializing to 0, which caused the first
heartbeat check to compute a huge elapsed time and immediately trigger
a false "connection appears dead" alert.
Changes:
- Change 4 struct fields from AtomicU64 to Arc<AtomicU64>
- Initialize last_heartbeat to HardwareTimestamp::now() instead of 0
- clone_for_async() now clones the Arcs (shared counters)
- Add test verifying counters are shared between original and clone
- Add test verifying heartbeat initialized to current time
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 symbols (ES, NQ, ZN, 6E) x 9 quarters (2024-Q1 through 2026-Q1),
723 days of 1-minute OHLCV bars from GLBX.MDP3 dataset.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The download binary was using the default SType::RawSymbol which
returned empty DBN files (95 bytes) for parent symbols like ES.FUT.
Adding SType::Parent resolves the symbol mapping. Also adjusted end
date to 2026-02-22 (latest available data).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Casting negative f64 to u64 saturates to 0, corrupting avg_price and all
downstream PnL calculations. Add price_to_fixed_checked() that rejects
non-finite, zero, and negative prices with PositionError::InvalidPrice.
The critical update_with_execution path now uses the checked variant.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The get_orders() status filter used `matches!(order.status, _status)` which
creates a new wildcard binding instead of comparing against the captured
`status` variable. This caused every order to match regardless of filter,
meaning get_orders(Some(Filled)) returned ALL orders — inflating exposure
calculations. Replaced with direct equality comparison `order.status == *status`.
Added test_get_orders_status_filter_only_returns_matching to prevent regression.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add #![allow(unused_crate_dependencies)] for consistency with other ml tests
- Extract EXPECTED_FEATURE_DIM constant (replaces magic number 51)
- Add validated_folds counter to prevent vacuous pass in walk-forward test
- Replace unwrap_or_else(panic) with match pattern
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three integration tests validating the full pipeline end-to-end:
1. Feature extraction: 90-day synthetic bars -> 51-dim features, no NaN/Inf
2. Walk-forward with normalization: 24-month bars -> windows -> NormStats from train only -> normalized mean ~0
3. No lookahead bias: train timestamps < val start, val timestamps < test start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Loads trained DQN/PPO checkpoints, runs greedy inference on walk-forward
test windows, computes Sharpe ratio, max drawdown, win rate, profit factor,
and total return per fold. Outputs a JSON evaluation report with aggregate
metrics and sanity checks (beats-random, action diversity, fold consistency).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ml/examples/train_baseline.rs that trains DQN and PPO models using
expanding walk-forward windows on real Databento OHLCV data.
Features:
- CLI args via clap (--model, --epochs, --batch-size, --data-dir, etc.)
- Recursive .dbn.zst file discovery and OHLCV bar loading
- 51-dim feature extraction via extract_ml_features()
- Walk-forward window generation with NormStats per fold
- DQN training loop with epsilon-greedy, experience replay, early stopping
- PPO training loop with GAE, trajectory collection, early stopping
- PnL-based reward (BUY/SELL/HOLD)
- Safetensors checkpoint saving per fold
- NormStats JSON export for evaluation reproducibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements expanding-window walk-forward cross-validation for time-series
ML models, preventing lookahead bias by always evaluating on unseen future
data. Includes z-score NormStats computed from training splits only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds `ml/examples/download_baseline.rs` that downloads 730 days of
Databento OHLCV-1m data in quarterly chunks for 4 CME futures symbols.
Features:
- Reads universe config from TOML (symbols, date range, dataset)
- Splits date range into calendar-quarter chunks (~90 days each)
- Resume support: skips existing non-empty files
- Uses `get_range_to_file` for streaming writes to .dbn.zst
- Dry-run mode with cost estimate ($0.12/symbol/day)
- Confirmation prompt (skippable with --yes)
- Per-file progress with timing and byte counts
- Failure-tolerant: logs errors and continues
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
