Production trading must never rely on simulated predictions. The ensemble
coordinator now requires real model adapters and returns errors when none
are registered or all fail inference, instead of silently falling back to
mock/simulated predictions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace empty placeholder safetensors file with actual model weight
serialization using the VarMap, matching the DQN trainable adapter pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace 6 unwrap() calls with safe error handling in DQN IQN code:
- Production: 3 unwrap() on iqn_network/iqn_target_network replaced with
ok_or_else returning MLError::ModelError for clear diagnostics
- Tests: 2 result.unwrap() replaced with ?, 2 DQN::new().unwrap() replaced
with ? after changing test signatures to return anyhow::Result<()>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add #[allow(unsafe_code)] on each unsafe impl Send/Sync for the four
inference adapters (DQN, PPO, TFT, Mamba2). The SAFETY comments explain
why these are sound — Mutex provides exclusive access.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PaperBroker: simulated fills with configurable slippage and commission.
PnLTracker: rolling Sharpe ratio (252-day window), max drawdown, cumulative return.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CampaignResults with serde serialization. run_campaign() orchestrates
ArgminOptimizer for DQN hyperopt, saves best_params.json and
campaign_summary.json to timestamped results directory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Lightweight synchronous coordinator that wraps N ModelInferenceAdapter
instances. Aggregates predictions via confidence-weighted voting with
graceful degradation for unready or failing models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN adapter: save/load/predict round-trip validation
- Coordinator: full ensemble with real DQN adapter integration test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace mock predictions with real model inference when adapters are
registered. Falls back to mock predictions for models without adapters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements TftInferenceAdapter that wraps the Temporal Fusion Transformer
for ensemble prediction. Unlike single-step models (DQN, PPO), TFT requires
a sequence of observations before running inference. The adapter maintains
an internal VecDeque buffer that collects sequence_length feature vectors,
then constructs static/historical/future tensors for the TFT forward pass.
Median quantile is mapped to directional signal via smooth saturating
function; IQR provides confidence estimation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement ModelInferenceAdapter for DQN (Q-value argmax -> direction + softmax confidence)
and PPO (probability-weighted action values -> direction + max prob confidence) with
zero-padding for mismatched feature dimensions and 6 passing unit tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ObjectiveMode enum (EpisodeReward/Sharpe) to DQNTrainer and a
TwoPhaseObjective trait + optimize_two_phase() method to ArgminOptimizer.
Phase A optimizes episode reward for fast convergence, Phase B (pending
model Clone support) refines with Sharpe ratio. This keeps the static
extract_objective trait method untouched by separating objective switching
into instance-level state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add load_ofi_features() helper to both DQN and PPO trainer adapters
that loads MBP10 snapshots from a sibling mbp10/ directory, computes
8-slot OFI feature vectors via OFICalculator, and overlays them onto
positions 43-50 of the training feature arrays. Gracefully falls back
to zero-padded features when MBP10 data is not available.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add use_qr_dqn, num_quantiles, qr_kappa fields to DQNHyperparameters
and pass them from DQNParams (hyperopt) through to DQNConfig (agent),
replacing hardcoded IQN values in the trainer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add num_quantiles and qr_kappa to the continuous search space for
Quantile Regression DQN, replacing the disabled C51 distributional RL
(BUG #36). QR-DQN is enabled by default with 32 quantiles and kappa=1.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace flat read_dir() with recursive collect_dbn_files_recursive() in
both DQN and PPO hyperopt adapters so .dbn files inside symbol
subdirectories (6E.FUT/, ES.FUT/, NQ.FUT/, ZN.FUT/) are discovered
automatically.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces the TODO stub with a working Hyperband implementation that
applies Successive Halving pruning only at rung epochs (max_resource/eta^k).
Between rungs, trials always continue. Adds two tests verifying pruning
at rung epochs and non-pruning between rungs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single-sample latency measurement was flaky under concurrent test load.
Using median eliminates outlier sensitivity (127μs median vs 1154μs spike).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implement load_from_dbn() for both PPO and DQN hyperopt adapters
using dbn::DbnDecoder (same pattern as RealDataLoader)
- Fix PPO feature extraction: [f64;51] → [f32;54] with zero-padded
portfolio state (was silently dropping all samples due to len==54 check)
- Add NaN/Inf → 1e6 penalty in optimizer for non-finite objectives
- Fix partial_cmp().unwrap() panic when comparing NaN objectives
- Add ensemble real-model validation test (DQN + PPO trained on
real 6E.FUT data, predictions aggregated through ensemble)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create PpoStrategy and PpoLstmStrategy implementing ValidatableStrategy
to run PPO through walk-forward validation with DSR, PBO, and permutation
tests. Both variants validated on real 6E.FUT data (29,937 bars, 15 folds).
Key implementation detail: LSTM hidden states are detached from the
computation graph after each step to prevent stack overflow from
unbounded graph growth across 30k+ sequential forward passes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add accumulation_steps config to PPOConfig with gradient accumulation
in update_mlp() using existing accumulate_grads/scale_grads utilities
- Add clip_epsilon_high: Option<f32> for asymmetric PPO clipping to
prevent entropy collapse during long training
- Rename WorkingPPO → PPO for consistency with DQN naming convention
- Add pub type WorkingPPO = PPO for backward compatibility
- Fix PPOConfig struct literals in trading_service and hyperopt adapter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Apply max_grad_norm clipping in update_mlp() and update_lstm() by splitting
backward_step into backward + norm computation + conditional scaling.
Add from_varbuilder() to LSTMPolicyNetwork and LSTMValueNetwork for
checkpoint deserialization. Remove TODO early-return error in load_checkpoint()
and add LSTM branching for actor/critic loading.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unnecessary unsqueeze(1) in FlowPolicy log_prob and entropy
(shapes should be [batch_size], not [batch_size, 1])
- Update flow_policy tests to expect correct [batch_size] shape
- Simplify kelly position sizing test with helper function
- Clean up example files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Bug #42 sign inversion diagnostic was logging thousands of lines to
stdout on every training run. Now gated behind RUST_LOG=debug level.
Removed unused expected_position variable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The default DQN agent (RegimeConditionalDQN with 3 regime heads) was
silently failing all gradient accumulation calls because only the
Standard DQN variant was supported. All training steps errored out and
loss was recorded as 0.0.
Changes:
- Add compute_gradients() to RegimeConditionalDQN: classifies batch by
regime, routes sub-batches to heads, merges gradient stores (no key
collisions since heads have independent parameters)
- Add apply_accumulated_gradients(): applies to heads with initialized
optimizers, skips uninitialized heads (no training data for that regime)
- Add optimizer_vars(): returns combined vars from all heads
- Update DQNAgentType dispatch to delegate to RegimeConditional
- Re-export GradientResult from dqn module
- Fix batch size validation tests for AutoBatchSizer clamping behavior
Verified: convergence test shows 4.8% relative difference between
accumulated (16x4) and direct (64x1) training paths (threshold: 30%).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQNTrainer::new() now clamps oversized batch sizes to the safe GPU limit
instead of rejecting them. Updated test_batch_size_validation and
test_gpu_batch_limit_230_enforced to assert is_ok() with clamping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the previous implementation that ran N independent optimizer steps
with true gradient accumulation: compute gradients for each mini-batch,
accumulate them across steps, average, then apply a single optimizer step.
This simulates training with effective batch size of accumulation_steps *
batch_size while keeping memory usage at batch_size. Also adds PER priority
updates from accumulated TD errors and replay buffer stepping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract target network update logic into private update_target_networks()
helper to avoid duplication between train_step() and the new gradient
accumulation methods. Add three new public methods:
- compute_gradients(): forward+backward without optimizer step
- apply_accumulated_gradients(): apply pre-computed grads + target update
- optimizer_vars(): expose optimizer variables for accumulation utilities
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the ~700-line train_step into two methods to prepare for gradient
accumulation support. compute_loss_internal handles batch sampling,
tensor creation, forward pass (IQN/C51/standard), entropy and CQL
regularization, returning a ComputeLossResult with the loss tensor still
in the computation graph. train_step now calls compute_loss_internal,
then performs backward pass, optimizer step, and all bookkeeping
(PER updates, diagnostics, target network updates). No behavioral
changes -- identical results to the original monolithic method.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OOM recovery loop halves `current_batch_size` on each retry, but
`train_step_single_batch()` and `train_step_with_accumulation()` both
called `agent.train_step(None)`, which makes the agent sample from its
replay buffer at the original `config.batch_size` -- ignoring the
reduction entirely. This meant OOM retries would always fail with the
same allocation size.
Fix: when `current_batch_size < hyperparams.batch_size`, manually
sample from the replay buffer at the reduced size and pass the explicit
batch via `agent.train_step(Some(experiences))`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace misleading doc comments and TODOs that described "true gradient
accumulation" with accurate documentation. The method runs N sequential
mini-batch training steps (each with its own optimizer.step()), not
gradient accumulation in the strict sense. Also switch debug log
messages from self.hyperparams.batch_size to self.current_batch_size
so they reflect the dynamically-adjusted batch size after any OOM
recovery halving.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wraps the train_step() dispatch in a retry loop (up to 3 attempts) that
detects CUDA OOM errors via string matching on the anyhow error chain and
halves the tracked batch size on each retry. Adds a current_batch_size
field to DQNTrainer for monitoring the effective batch size.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the static MAX_BATCH_SIZE=230 hard-coded cap with dynamic GPU memory
detection via AutoBatchSizer. When nvidia-smi is available, the trainer queries
actual free GPU memory and calculates an optimal batch size. When unavailable
(CI environments), it falls back to the previous static cap of 230.
Behavioral change: instead of returning an error when batch_size exceeds the
limit, the trainer now clamps it down to the safe maximum. This is more
user-friendly and enables OOM recovery in a later task.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Assertions verified on real 6E.FUT data:
1. All 20 epochs complete (no premature early stopping)
2. Loss decreases >5% (gradient flow works) — actual: 22.4%
3. All per-epoch losses are finite (no NaN/Inf)
4. Q-value divergence (model develops action preferences)
5. Checkpoint round-trip (save/load weight integrity)
6. Epsilon decayed below 0.5 (exploration schedule ran)
Also fixes divide-by-zero in triple_barrier.rs:103 when
entry_price_cents is zero (guard in both barrier tracker
and trainer caller).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add public accessor methods for smoke test verification:
- loss_history() — per-epoch training loss
- val_loss_history() — per-epoch validation loss
- get_agent_epsilon() — current epsilon from DQN agent
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
