Multi-window backtest: splits validation data into 3 non-overlapping
windows and aggregates with mean(Sharpe) - 0.5*std(Sharpe), penalizing
inconsistency and reducing overfit to a single data segment.
Top-K ensemble: hyperopt now emits top_k_params (top 5 trials) in JSON
output. train_baseline_rl gains --ensemble-top-k flag to train multiple
models per fold from different hyperopt configs, saving checkpoints as
dqn_ensemble_{k}_fold_{n}.safetensors.
Workflow template: adds ensemble-top-k parameter (default 5) and passes
--ensemble-top-k to the train-best step.
2720 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 root causes of 45-action DQN collapsing to 1-6 actions:
1. Batch epsilon ignoring noisy_epsilon_floor: select_actions_batch()
and select_actions_batch_gpu() used get_epsilon() which returns 0.0
with noisy nets — zero random exploration in the training path.
Added get_effective_epsilon() that respects noisy_epsilon_floor.
2. Entropy coefficient too weak: default 0.05 with bounds (0.01, 0.2)
produced max ~0.19 bonus vs TD loss of 4+. Bumped default to 0.1,
widened bounds to (0.05, 0.5) for effective anti-collapse.
3. count_bonus_coefficient not in search space: was hardcoded at 0.1
in from_continuous(). Promoted to 31st search dimension with bounds
(0.05, 1.0) so PSO/TPE can optimize exploration strength.
4. Diversity penalty too coarse: objective had <10 unique actions
short-circuit but nothing for 10-20. Added graduated penalty that
linearly ramps from 3.0 (10 actions) to 0.0 (20 actions).
Also fixes pre-existing clippy impl_trait_in_params in optimizer.rs.
2720 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire record_hyperopt_trial_duration, set_hyperopt_best_objective,
set_hyperopt_trial_best_loss, and set_hyperopt_elapsed into all three
optimizer paths (PSO sequential, PSO parallel, TPE). These metrics were
registered but never called during the optimization loop, causing
"Best Objective Over Time", "Trial Duration", and "Elapsed Time"
Grafana panels to show "No data".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ephemeral Argo workflow pods terminate after training completes, causing
Prometheus to lose all scraped metrics. Add push_to_gateway() to POST
final metrics to the existing pushgateway service so they persist on the
Grafana training dashboard after pod completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The batch training path (select_actions_batch → forward()) does NOT
increment DQN::total_steps. Only select_action() does. When
warmup_steps > 0, train_step() checks total_steps < warmup_steps
and returns (0.0, 0.0) — zero loss, zero gradients. The model
never trained; "results" were random initialization Q-values.
Fix: force warmup_steps=0 in hyperopt adapter and remove
warmup_ratio from the 31D→30D search space (saves a dimension
for TPE/PSO effectiveness).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace Silverman's bandwidth (h = 1.06σn^(-1/5)) with Scott's rule
(h = 0.7σn^(-1/(d+4))) for tighter kernels in high-D parameter spaces
- Add best-trial injection: always evaluate EI at best known point plus
5 small perturbations (±5%), preventing optimizer from forgetting peaks
- Scale n_candidates dynamically: max(256, 8*n_dims) instead of fixed 100
- Reduce gamma from 0.25 to 0.15 when trials < 50 for tighter exploitation
- Wire model_name through PSO/TPE paths for per-trial Prometheus metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-export HardwareTimestamp through data crate instead of ml/risk
depending directly on trading_engine. Reduces coupling between
the ML pipeline and the trading engine.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two issues causing all 20 hyperopt trials to have identical f64::MAX
objective (TPE optimizer blind):
1. Val-loss plateau early stopping fired at epoch 5-6 of every 8-epoch
trial (plateau_window=5 too aggressive for short runs). Disabled
early_stopping_enabled for hyperopt; gradient-collapse patience
still active as safety net.
2. Penalty metrics used f64::MAX for gradient_norm/q_value_std which
produced ~3.6e+308 objective. Changed to 100.0 so TPE can still
differentiate between early-stopped trials by other metric fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Hyperopt: catch early stopping errors and return penalty metrics
instead of aborting entire run. Trials that stop early are scored
as poor (objective=-500) so optimizer avoids those configs.
- DaemonSet: detect GPU nodes (NVIDIA conf.d overlay) and create
v3-compatible registry config drop-in for containerd v2.1+.
Old grpc.v1.cri.registry path is silently ignored by containerd v2.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes from deep investigation audit (LOW/MEDIUM priority):
1. CVaR penalty: hard cliff (0 or 10) → smooth ramp with gradient signal
for PSO. Formula: min(10, max(0, -cvar-0.05)*200).
2. Clip outliers leakage: data_loading.rs now computes clip bounds from
training portion only (first 80%), then applies to full series.
Log returns and windowed normalize are causal (no leakage).
3. Noisy sigma scheduler: hyperopt now matches conservative() defaults
(enabled, initial=0.8, final=0.4) so hyperopt-found params
generalize to train_best without scheduler mismatch.
4. evaluate_supervised.rs: NormStats fallback from test data (leakage)
replaced with bail! matching evaluate_baseline.rs behavior.
5. Doc comments: stale 27D references updated to 31D (4 locations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 bugs found by deep investigation agents:
1. HIGH: minimum_profit_factor (search dim 30) was never forwarded from
DQNHyperparameters to DQNConfig — trainer hardcoded 1.5, making the
entire dimension wasted. Added field to DQNHyperparameters, wired
through trainer.rs.
2. HIGH: Backtest EvaluationEngine used hardcoded $10K initial capital
while training used $35K (self.initial_capital). Returns/Sharpe were
3.5x distorted. Now uses self.initial_capital.
3. MEDIUM: calculate_hft_activity_score_wave10 multiplied already-100x
buy_pct/sell_pct by 100 again, making the diversity penalty threshold
(15%) unreachable (values were ~2700). Removed double multiplication.
4. MEDIUM: Sortino ratio returned 0.0 for all-positive returns (no
downside deviation), penalizing perfect strategies in the 40%-weighted
composite score. Now returns 100.0 (capped) when mean return > 0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Degenerate trials with <10/45 unique actions now short-circuit the objective
to a graduated penalty (8.0 for 1 action → 0 for 10+), ignoring composite
score entirely. Previously, phantom Sharpe=2317 drove objective to -369k,
trapping TPE in the low-temp region.
Combined with temp floor raise (0.1→0.5 from prior commit), this eliminates
both the supply (no low temps) and demand (no reward) for degenerate trials.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt TPE was stuck chasing phantom Sharpe=2317 from degenerate trials
with 2/45 unique actions (temp 0.15-0.20). Two fixes:
1. Raise eval_softmax_temp bounds from [0.1, 2.0] to [0.5, 2.0] — temps
below 0.3 consistently produce 1-3/45 actions regardless of model quality
2. Add diversity_penalty to objective: 50*(1 - unique/10) for <10/45 actions,
so degenerate trials score badly even if they accidentally reach the TPE
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrite batch_hierarchical_softmax_actions to keep all sampling on GPU:
reshape [N,45]→[N,5,9], max-pool→[N,5] exposure Q, Gumbel-max over
exposures, parallel Gumbel-max over all sub-actions, gather chosen
exposure's sub-action. Only CPU transfer: N flat indices.
Add imagePullPolicy: Always to compile-services and compile-training
containers so rebuilt builder images are never stale-cached.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Flat Gumbel-max over 45 actions can collapse to a single exposure
bucket (e.g., 2/45 unique actions all in Long100) even with adequate
temperature, because order/urgency diversity doesn't produce trades.
Two-stage sampling: first Gumbel-max over 5 exposure levels (max Q
per level), then Gumbel-max over 9 order×urgency combos within the
chosen exposure. Guarantees exposure-level diversity proportional to
actual Q-value differences.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Raise eval_softmax_temp lower bound from 0.01 to 0.1 (log scale) to
prevent near-greedy collapse in hyperopt walk-forward eval. Temps below
0.05 produced degenerate 1-trade trials even with softmax sampling.
- Mount MinIO CA cert in CI compile pods and append to system trust store
so sccache S3 backend can verify MinIO's self-signed TLS certificate.
- Add openssh-client to ci-builder-cpu Dockerfile (missing, broke git
clone over SSH).
- Bump MinIO memory limits from 512Mi to 2Gi (OOMKilled under load).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add device_pool to DQN/PPO hyperopt trainers for round-robin GPU assignment
per trial. Binary detects all CUDA devices, scales VRAM budget by GPU count.
Single-GPU: no behavior change (pool of 1).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
T=0.1 was too conservative for degenerate configs — trials with uniform
Q-values still collapsed to 1 unique action. Making temperature a
hyperopt parameter lets TPE learn the optimal exploration-exploitation
balance per config. Range [0.01, 2.0] log-scale.
Also sets training-workflow default gpu-pool to ci-training-h100.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN hyperopt walk-forward eval collapsed to 1 trade when Q-values were
nearly uniform (greedy argmax always picked the same action). Replace
batch_greedy_actions with batch_softmax_actions using the Gumbel-max
trick (argmax(Q/T + Gumbel(0,1))) — fully GPU-resident, no CPU
softmax/sampling. Temperature 0.1: nearly greedy when Q-values are
separated, diverse when uniform. Logs unique_actions/45 per backtest.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace legacy Buy/Sell/Hold collapse with exposure-aware evaluation.
Long100→Long50 now generates a partial-close trade instead of being
a no-op. Combined with graduated trade penalty, PSO now gets gradient
signal across the entire 26D search space.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add calculate_trade_insufficiency_penalty() that produces monotonically
decreasing penalties for low trade counts (10.0 at 0 trades, 0.0 at 100+),
giving PSO gradient signal across the degenerate plateau where all trials
produce the same 1.25 objective.
Wire the penalty into extract_objective() with short-circuit for <10 trades
(skip composite score since it's all zeros anyway).
Include 4 unit tests verifying zero/one/graduated/no-plateau properties.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add factored evaluation support to EvaluationEngine with continuous
exposure tracking (-1.0 to +1.0), partial position changes, and
order-type-aware transaction costs for the 45-action factored space.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When use_noisy_nets=true (the conservative() default), epsilon never
decayed from 1.0 because (1) the trainer skipped update_epsilon() and
(2) DQNAgentType::set_epsilon() was a no-op for RegimeConditional agents.
This caused ALL training actions to be random — Q-values were learned but
never used for action selection.
Fix: set stored epsilon to 0.0 at training start when noisy nets are on.
Exploration is provided by NoisyLinear weight perturbation + the separate
noisy_epsilon_floor (5% safety floor for 45-action spaces).
Also fixes:
- Zstd-compressed .dbn file detection via magic bytes (0x28B52FFD)
- Test data path resolution using ancestors().find() for workspace root
- Test assertions updated for epsilon < 0.01 with noisy nets
2698 lib tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add live training metrics monitor CLI command (streaming & one-shot) using
the monitoring gRPC service. Update DQN tests to match post-fix defaults:
IQN disabled, CQL alpha=0.1, v_min/v_max widened, 26D search space.
- train.rs: `fxt train monitor [--once] [--model X] [--interval N]`
- Rewrite gradient collapse test for BF16 mixed precision awareness
- Update inference test config to match trainer defaults (IQN off, CQL on)
- Update production smoke test for 26D parameter space
- Add dqn_action_collapse_fix_test.rs verifying all 6 root cause fixes
- Add planning docs for monitoring service and epoch financial metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Expand search space from 25D to 26D with cql_alpha (0.0-0.5)
- v_min bounds: (-3,-1) → (-15,-3), v_max: (1,3) → (3,15)
- Default use_qr_dqn: true → false (IQN disabled)
- Wire cql_alpha into DQNHyperparameters construction
- Update all 18 adapter tests for new dimensions and ranges
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hold penalty was divided by 1000 (producing ~0.00001), negligible vs
transaction costs (0.05-0.15%). Now uses hold_penalty_weight directly
from hyperopt (0.01-2.0 range).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Delete stub rainbow_agent.rs (fake select_action, hardcoded train loss)
and promote rainbow_agent_impl.rs to rainbow_agent.rs. Unify on the
real 8-field RainbowAgentMetrics from rainbow_config.rs, replacing the
4-field stub version (epsilon→exploration_rate, average_loss→current_loss).
5 files changed: -667/+447 lines, 2698+32 tests pass, 0 clippy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change PercentileScaler, RewardNormalizer, CompositeReward, and
PPORewardShaper public APIs from f64 to f32 — matching what callers
actually pass. Internal EMA/percentile math stays f64 for precision.
Keep data_loader SMA/EMA/MACD accumulators in f64 throughout (was
f64→f32→f64 per step), cast to f32 only at output boundary.
Remove dead _transition computation in rainbow_agent_impl and unused
bigdecimal deps from broker_gateway_service and trading_service.
2704 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
On Ampere+ GPUs the training dtype is BF16. Network forward passes
with BF16 inputs return BF16 tensors, but all loss-path arithmetic
(rewards, dones, gamma, PER weights, Huber constants, atoms, clip
epsilon) was created as F32 from Tensor::from_vec. Candle forbids
mixed-dtype binary ops, causing "dtype mismatch in {mul,sub,add}"
on every training step.
DQN fixes (7 mismatch sites):
- Cast current_q_values to F32 immediately after forward pass
- Cast all target network outputs (standard, dueling, C51, IQN) to F32
- Keep rewards/dones/gamma/weights/atoms/half/delta as F32 (remove
.to_dtype(training_dtype) casts — now use DType::F32 sentinel)
- Entropy regularization and CQL paths now match (F32 + F32)
PPO fixes (3 mismatch sites):
- Use DType::F32 for clip epsilon one_tensor (was training_dtype BF16)
- Cast critic.forward() output to F32 in both MLP and LSTM value loss
- Cast target_returns to F32 for symlog/normalization path
Infra: revert runner-h100 from SXM2 (zero Scaleway quota) back to
ci-training-h100 PCIe pool.
2704 ml lib tests pass, 260 PPO tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BUG #41 kept forward pass in F32 for autograd, but the target-side
tensors (reward, gamma, done, next_q) were cast to BF16 via `dtype`.
The `state_action_values.sub(&target_q_values)` then hit F32-vs-BF16
mismatch on Ampere+ GPUs, causing every training step to fail silently.
Fix: `.to_dtype(state_action_values.dtype())` on the detached target.
Safe because target is detached (no autograd graph to break).
Also: H100 runner → SXM2 pool, GPU availability checker script.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Same class of bug as the DQN fix (4c88498b): network outputs were
being cast to F32 mid-pipeline, defeating tensor-core acceleration
on H100/L40S. Now the full training loop stays in training_dtype()
(BF16 on CUDA Ampere+, F32 on CPU), with F32 casts only at scalar
extraction boundaries (to_scalar, to_vec1).
Files fixed:
- ppo.rs: Actor/Critic forward, act_with_log_prob, compute_losses,
update_mlp, LSTM recurrent loop, predict method
- lstm_networks.rs: removed F32 output casts from both networks
- continuous_ppo.rs: one_tensor + scalar extractions
- hidden_state_manager.rs: zeros/ones use training_dtype()
- flow_policy/mod.rs: log_det accumulators + dummy log_std
- ensemble/adapters/ppo.rs + dqn.rs: F32 cast at extraction
2704 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove premature F32 output casts from network forward passes
(Sequential, NetworkLayers, DistributionalDueling) that were negating
tensor-core acceleration. Instead, cast F32 input tensors (rewards,
dones, gamma, weights, atoms) to training_dtype() at the boundary
and only escape to F32 at scalar extraction points (loss value,
TD errors, logging vectors).
465 DQN tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NoisyLinear layers have different parameter names (sigma_weight/sigma_bias)
than standard Linear layers. Hardcoding use_noisy_nets=false caused
checkpoint loading to silently fail when the training run used noisy nets,
resulting in empty folds in the eval report.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
num_atoms, dueling_hidden_dim, v_min/v_max, gamma were using defaults
instead of hyperopt values — causing tensor shape mismatch on checkpoint
load (e.g. output layer 45×200=9000 vs 45×51=2295). Also fixed use_iqn
to read from use_qr_dqn key (matching trainer's field mapping).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt found hold_penalty_weight=1.818, curiosity_weight=0.402, etc.
but walk-forward training used tiny defaults (0.01, 0.1), causing the
agent to learn "holding is optimal" → 0 trades across all 50 epochs.
Now passes: hold_penalty_weight, max_position_absolute, huber_delta,
entropy_coefficient, curiosity_weight, weight_decay, kelly_fractional,
kelly_max_fraction from hyperopt JSON to DQNHyperparameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Walk-forward DQN training hardcoded epsilon_start=1.0 with noisy_nets=true,
forcing pure random exploration for all 50 epochs (0 trades, 0 Sharpe).
Now reads epsilon, PER, dueling, distributional, noisy nets, and QR-DQN
params from the hyperopt JSON. When noisy_nets=true, defaults epsilon to
0.05 instead of 1.0.
Evaluator now falls back to highest dqn_fold{N}_epoch{E}.safetensors when
_best.safetensors doesn't exist (early stopping saves epoch checkpoints).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Added select_action_inference(&self) — pure greedy forward pass with
no exploration mutations (epsilon, noisy nets, count bonus, step counter
are all skipped). Returns (FactoredAction, f32) where confidence is
softmax probability of the selected action, clamped [0.5, 0.95].
Also added select_action_with_confidence(&mut self) for training path
with the same confidence computation.
Production wrapper now uses read lock (was write lock), enabling
concurrent predictions. Confidence is real softmax probability
instead of hardcoded 0.85.
323 trading_service tests, 15 DQN core tests, 0 clippy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
egobox is not a dependency (replaced by argmin PSO long ago).
The egobox_tuner.rs module was pure dead code — optimize_mamba2()
just returned a deprecation error. Deleted the module, its 26 tests,
and the integration test. Cleaned up stale egobox references in
ParameterSpace trait docs.
174 hyperopt tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN: disable all exploration for production (warmup=0, epsilon=0,
noisy_nets=false, count_bonus=false), read feature_count from
checkpoint metadata instead of hardcoding 54, add loaded guard
and NaN check on predict output.
PPO: fix confidence formula — use act_with_log_prob() instead of
act() which returns value_estimate not log_prob, add loaded guard
and NaN check, remove WorkingPPO alias.
Liquid: add loaded flag for is_ready()/predict() guards.
Remove EgoboxOptimizer/EgoboxOptimizerBuilder backward-compat
aliases — replaced with canonical ArgminOptimizer everywhere.
323 trading_service tests, 200 hyperopt tests, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire LR scheduler to actually update the Adam optimizer (was logged but
never applied). Add update_learning_rate to DQN, RegimeConditionalDQN,
and DQNAgentType so decay_factor propagates through all agent variants.
Fix train/eval parity: CLI defaults 51→54 features, 3→45 actions;
DQN eval always uses 3-layer hidden_dims; PPO eval uses 5-layer value
network matching trainer. enhanced_ml.rs hardcoded config updated from
state_dim=16/num_actions=3 to 54/45.
Fix Candle F32/BF16 traps: replace `Tensor * 0.5` (f64 literal) with
broadcast_mul(Tensor::full(0.5_f32)) in quantile_regression.rs (2x),
dqn.rs Huber loss, and IQN gamma multiplication. Prevents panics on
Ampere+ BF16 GPUs.
Add urgency_weight() multiplier to training cost model in reward.rs
and portfolio_tracker.rs — urgency dimension (Patient/Normal/Aggressive)
now affects learned value function, matching evaluate_baseline.rs.
Fix equity tracking: additive (equity += ret) → multiplicative
(equity *= 1.0 + ret) in compute_metrics. Fix total return calc.
Fix PER beta annealing: epochs*70 → epochs*1000 (~1015 actual steps
per epoch for 130k bars / batch 128).
Fix silent target network freeze: mutex lock failure now propagates
error instead of silently skipping weight update.
Make save_checkpoint atomic (write .tmp then rename). Make NormStats
write atomic with error logging instead of silent discard.
Convert EnsembleConfig::new assert! → Result<Self, MLError> with
14 call site updates. Fix hyperopt result serialization to warn
instead of silently dropping to Value::Null.
Fix clippy MSRV mismatch: clippy.toml 1.75 → 1.85 matching Cargo.toml.
2732 tests pass, 0 clippy warnings across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQNAgent::load_from_safetensors: validate architecture hash before
loading weights (was bypassing validation entirely - P0 production gap)
- DQNEnsemble::save_to_directory: embed architecture metadata via
safetensors::serialize_to_file instead of bare VarMap::save (save/load
round-trip was guaranteed to fail)
- architecture_hash: hash hidden_dims.len() before values to prevent
theoretical collision between different-length configs
- train_baseline_rl: NormStats write now atomic (write .tmp then rename)
with cleanup guard on rename failure; serialization error is now loud
(error! + return None) instead of silently discarded
- train_baseline_rl: checkpoint rename failure cleans up orphaned .tmp
- hyperopt_baseline_rl: fix clippy single_match_else (match on equality
check -> if/else)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace `let _ = sync_tensor.to_vec0::<f32>()` with `drop()` to
satisfy clippy's let_underscore_must_use lint on the Result return.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
