Sharpe-based early stopping kills every hyperopt trial at epoch 4
because compute_epoch_financials() is deterministic (greedy argmax on
fixed validation data) — the model doesn't change enough in 8 short
epochs to shift any argmax decisions, making Sharpe bit-identical
across epochs and triggering plateau detection immediately.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multi-window backtest: splits validation data into 3 non-overlapping
windows and aggregates with mean(Sharpe) - 0.5*std(Sharpe), penalizing
inconsistency and reducing overfit to a single data segment.
Top-K ensemble: hyperopt now emits top_k_params (top 5 trials) in JSON
output. train_baseline_rl gains --ensemble-top-k flag to train multiple
models per fold from different hyperopt configs, saving checkpoints as
dqn_ensemble_{k}_fold_{n}.safetensors.
Workflow template: adds ensemble-top-k parameter (default 5) and passes
--ensemble-top-k to the train-best step.
2720 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 root causes of 45-action DQN collapsing to 1-6 actions:
1. Batch epsilon ignoring noisy_epsilon_floor: select_actions_batch()
and select_actions_batch_gpu() used get_epsilon() which returns 0.0
with noisy nets — zero random exploration in the training path.
Added get_effective_epsilon() that respects noisy_epsilon_floor.
2. Entropy coefficient too weak: default 0.05 with bounds (0.01, 0.2)
produced max ~0.19 bonus vs TD loss of 4+. Bumped default to 0.1,
widened bounds to (0.05, 0.5) for effective anti-collapse.
3. count_bonus_coefficient not in search space: was hardcoded at 0.1
in from_continuous(). Promoted to 31st search dimension with bounds
(0.05, 1.0) so PSO/TPE can optimize exploration strength.
4. Diversity penalty too coarse: objective had <10 unique actions
short-circuit but nothing for 10-20. Added graduated penalty that
linearly ramps from 3.0 (10 actions) to 0.0 (20 actions).
Also fixes pre-existing clippy impl_trait_in_params in optimizer.rs.
2720 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire record_hyperopt_trial_duration, set_hyperopt_best_objective,
set_hyperopt_trial_best_loss, and set_hyperopt_elapsed into all three
optimizer paths (PSO sequential, PSO parallel, TPE). These metrics were
registered but never called during the optimization loop, causing
"Best Objective Over Time", "Trial Duration", and "Elapsed Time"
Grafana panels to show "No data".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The batch training path (select_actions_batch → forward()) does NOT
increment DQN::total_steps. Only select_action() does. When
warmup_steps > 0, train_step() checks total_steps < warmup_steps
and returns (0.0, 0.0) — zero loss, zero gradients. The model
never trained; "results" were random initialization Q-values.
Fix: force warmup_steps=0 in hyperopt adapter and remove
warmup_ratio from the 31D→30D search space (saves a dimension
for TPE/PSO effectiveness).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace Silverman's bandwidth (h = 1.06σn^(-1/5)) with Scott's rule
(h = 0.7σn^(-1/(d+4))) for tighter kernels in high-D parameter spaces
- Add best-trial injection: always evaluate EI at best known point plus
5 small perturbations (±5%), preventing optimizer from forgetting peaks
- Scale n_candidates dynamically: max(256, 8*n_dims) instead of fixed 100
- Reduce gamma from 0.25 to 0.15 when trials < 50 for tighter exploitation
- Wire model_name through PSO/TPE paths for per-trial Prometheus metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two issues causing all 20 hyperopt trials to have identical f64::MAX
objective (TPE optimizer blind):
1. Val-loss plateau early stopping fired at epoch 5-6 of every 8-epoch
trial (plateau_window=5 too aggressive for short runs). Disabled
early_stopping_enabled for hyperopt; gradient-collapse patience
still active as safety net.
2. Penalty metrics used f64::MAX for gradient_norm/q_value_std which
produced ~3.6e+308 objective. Changed to 100.0 so TPE can still
differentiate between early-stopped trials by other metric fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Hyperopt: catch early stopping errors and return penalty metrics
instead of aborting entire run. Trials that stop early are scored
as poor (objective=-500) so optimizer avoids those configs.
- DaemonSet: detect GPU nodes (NVIDIA conf.d overlay) and create
v3-compatible registry config drop-in for containerd v2.1+.
Old grpc.v1.cri.registry path is silently ignored by containerd v2.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes from deep investigation audit (LOW/MEDIUM priority):
1. CVaR penalty: hard cliff (0 or 10) → smooth ramp with gradient signal
for PSO. Formula: min(10, max(0, -cvar-0.05)*200).
2. Clip outliers leakage: data_loading.rs now computes clip bounds from
training portion only (first 80%), then applies to full series.
Log returns and windowed normalize are causal (no leakage).
3. Noisy sigma scheduler: hyperopt now matches conservative() defaults
(enabled, initial=0.8, final=0.4) so hyperopt-found params
generalize to train_best without scheduler mismatch.
4. evaluate_supervised.rs: NormStats fallback from test data (leakage)
replaced with bail! matching evaluate_baseline.rs behavior.
5. Doc comments: stale 27D references updated to 31D (4 locations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 bugs found by deep investigation agents:
1. HIGH: minimum_profit_factor (search dim 30) was never forwarded from
DQNHyperparameters to DQNConfig — trainer hardcoded 1.5, making the
entire dimension wasted. Added field to DQNHyperparameters, wired
through trainer.rs.
2. HIGH: Backtest EvaluationEngine used hardcoded $10K initial capital
while training used $35K (self.initial_capital). Returns/Sharpe were
3.5x distorted. Now uses self.initial_capital.
3. MEDIUM: calculate_hft_activity_score_wave10 multiplied already-100x
buy_pct/sell_pct by 100 again, making the diversity penalty threshold
(15%) unreachable (values were ~2700). Removed double multiplication.
4. MEDIUM: Sortino ratio returned 0.0 for all-positive returns (no
downside deviation), penalizing perfect strategies in the 40%-weighted
composite score. Now returns 100.0 (capped) when mean return > 0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Degenerate trials with <10/45 unique actions now short-circuit the objective
to a graduated penalty (8.0 for 1 action → 0 for 10+), ignoring composite
score entirely. Previously, phantom Sharpe=2317 drove objective to -369k,
trapping TPE in the low-temp region.
Combined with temp floor raise (0.1→0.5 from prior commit), this eliminates
both the supply (no low temps) and demand (no reward) for degenerate trials.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt TPE was stuck chasing phantom Sharpe=2317 from degenerate trials
with 2/45 unique actions (temp 0.15-0.20). Two fixes:
1. Raise eval_softmax_temp bounds from [0.1, 2.0] to [0.5, 2.0] — temps
below 0.3 consistently produce 1-3/45 actions regardless of model quality
2. Add diversity_penalty to objective: 50*(1 - unique/10) for <10/45 actions,
so degenerate trials score badly even if they accidentally reach the TPE
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Flat Gumbel-max over 45 actions can collapse to a single exposure
bucket (e.g., 2/45 unique actions all in Long100) even with adequate
temperature, because order/urgency diversity doesn't produce trades.
Two-stage sampling: first Gumbel-max over 5 exposure levels (max Q
per level), then Gumbel-max over 9 order×urgency combos within the
chosen exposure. Guarantees exposure-level diversity proportional to
actual Q-value differences.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Raise eval_softmax_temp lower bound from 0.01 to 0.1 (log scale) to
prevent near-greedy collapse in hyperopt walk-forward eval. Temps below
0.05 produced degenerate 1-trade trials even with softmax sampling.
- Mount MinIO CA cert in CI compile pods and append to system trust store
so sccache S3 backend can verify MinIO's self-signed TLS certificate.
- Add openssh-client to ci-builder-cpu Dockerfile (missing, broke git
clone over SSH).
- Bump MinIO memory limits from 512Mi to 2Gi (OOMKilled under load).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add device_pool to DQN/PPO hyperopt trainers for round-robin GPU assignment
per trial. Binary detects all CUDA devices, scales VRAM budget by GPU count.
Single-GPU: no behavior change (pool of 1).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
T=0.1 was too conservative for degenerate configs — trials with uniform
Q-values still collapsed to 1 unique action. Making temperature a
hyperopt parameter lets TPE learn the optimal exploration-exploitation
balance per config. Range [0.01, 2.0] log-scale.
Also sets training-workflow default gpu-pool to ci-training-h100.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN hyperopt walk-forward eval collapsed to 1 trade when Q-values were
nearly uniform (greedy argmax always picked the same action). Replace
batch_greedy_actions with batch_softmax_actions using the Gumbel-max
trick (argmax(Q/T + Gumbel(0,1))) — fully GPU-resident, no CPU
softmax/sampling. Temperature 0.1: nearly greedy when Q-values are
separated, diverse when uniform. Logs unique_actions/45 per backtest.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace legacy Buy/Sell/Hold collapse with exposure-aware evaluation.
Long100→Long50 now generates a partial-close trade instead of being
a no-op. Combined with graduated trade penalty, PSO now gets gradient
signal across the entire 26D search space.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add calculate_trade_insufficiency_penalty() that produces monotonically
decreasing penalties for low trade counts (10.0 at 0 trades, 0.0 at 100+),
giving PSO gradient signal across the degenerate plateau where all trials
produce the same 1.25 objective.
Wire the penalty into extract_objective() with short-circuit for <10 trades
(skip composite score since it's all zeros anyway).
Include 4 unit tests verifying zero/one/graduated/no-plateau properties.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Expand search space from 25D to 26D with cql_alpha (0.0-0.5)
- v_min bounds: (-3,-1) → (-15,-3), v_max: (1,3) → (3,15)
- Default use_qr_dqn: true → false (IQN disabled)
- Wire cql_alpha into DQNHyperparameters construction
- Update all 18 adapter tests for new dimensions and ranges
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change PercentileScaler, RewardNormalizer, CompositeReward, and
PPORewardShaper public APIs from f64 to f32 — matching what callers
actually pass. Internal EMA/percentile math stays f64 for precision.
Keep data_loader SMA/EMA/MACD accumulators in f64 throughout (was
f64→f32→f64 per step), cast to f32 only at output boundary.
Remove dead _transition computation in rainbow_agent_impl and unused
bigdecimal deps from broker_gateway_service and trading_service.
2704 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
egobox is not a dependency (replaced by argmin PSO long ago).
The egobox_tuner.rs module was pure dead code — optimize_mamba2()
just returned a deprecation error. Deleted the module, its 26 tests,
and the integration test. Cleaned up stale egobox references in
ParameterSpace trait docs.
174 hyperopt tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN: disable all exploration for production (warmup=0, epsilon=0,
noisy_nets=false, count_bonus=false), read feature_count from
checkpoint metadata instead of hardcoding 54, add loaded guard
and NaN check on predict output.
PPO: fix confidence formula — use act_with_log_prob() instead of
act() which returns value_estimate not log_prob, add loaded guard
and NaN check, remove WorkingPPO alias.
Liquid: add loaded flag for is_ready()/predict() guards.
Remove EgoboxOptimizer/EgoboxOptimizerBuilder backward-compat
aliases — replaced with canonical ArgminOptimizer everywhere.
323 trading_service tests, 200 hyperopt tests, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace `let _ = sync_tensor.to_vec0::<f32>()` with `drop()` to
satisfy clippy's let_underscore_must_use lint on the Result return.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Atomic checkpoint writes: write to .tmp then fs::rename() (POSIX atomic)
- Remove redundant thread::sleep(50ms) after CUDA tensor readback sync
- NormStats: bail on missing file instead of computing from test data (lookahead bias)
- q_value_std: warn when missing from training metrics instead of silent 0.0 fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrate three standalone PPO modules into the hyperopt training loop:
- PPORewardShaper: hold penalty + rolling Sharpe + diversity bonus per step
- CompositeReward: risk-adjusted reward (downside dev + differential return)
- TrajectoryReplayBuffer: ExO-PPO M=4 rollouts with IS-weighted replay
Also fixes GAE hardcoded gamma=0.99/lambda=0.95 — now uses hyperopt params.
2726 tests pass, 0 clippy errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrate all PPO improvement modules into the core training paths:
- Symlog value predictions in compute_value_loss (MLP + LSTM)
- Adaptive entropy auto-tuning replaces fixed entropy_coeff
- Percentile P5/P95 advantage scaling for heavy-tailed returns
- DAPO clip_epsilon_high wired in all 7 PPOConfig construction sites
- Shape mismatch fix in adaptive_entropy (unsqueeze scalar)
2726 tests pass, 0 clippy errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change --optimizer default from "pso" to "tpe" since TPE has better
sample efficiency in 25D spaces. Fix clippy let_underscore_must_use
on CUDA sync tensor readback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix 20 exotic parameters to validated defaults, keeping only the 25
that genuinely affect trading performance. This makes both PSO and
TPE dramatically more effective at exploring the parameter space.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add optimize_with_tpe() function that uses Tree-Parzen Estimator for
Bayesian hyperparameter optimization. Add --optimizer=tpe CLI flag
to hyperopt_baseline_rl binary (default: pso for backward compat).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Force CUDA synchronization between trials by creating a tiny tensor and
reading it back (forces cudaDeviceSynchronize). This ensures GPU memory
from dropped models is actually freed before the next trial starts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tighten max_position_absolute search range from [4,8] to [1,4] to
prevent catastrophic leverage during hyperopt. Add drawdown circuit
breaker to PortfolioTracker that force-closes positions at >20%
drawdown and refuses new trades while flat.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a standalone TPE optimizer for Bayesian hyperparameter optimization that
replaces PSO by modeling good/bad trial distributions with per-dimension
kernel density estimates. Includes Latin Hypercube Sampling for initial
exploration, Silverman bandwidth selection, log-sum-exp numerical stability,
and JSON-based trial persistence. 15 tests covering splits, bounds, KDE
density, convergence on 1D/2D objectives, and history serialization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-bar GPU forward passes with chunked batch inference (1024 bars
per chunk). This reduces ~90K individual CUDA kernel launches to ~88 batched
forward passes — ~1000× fewer GPU round-trips.
Changes:
- Add DQN::batch_greedy_actions(&self) for immutable batched forward+argmax
- Add RegimeConditionalDQN::batch_greedy_actions with per-regime-head batching
- Add DQNTrainer::convert_to_state_vec (CPU-only, skips GPU tensor allocation)
- Add PortfolioTracker::set_position_direct for backtest state sync
- Rewrite hyperopt backtest loop: chunked batching with portfolio state
updates between chunks (1024-bar granularity ≈ 17h of 1-min data)
Portfolio features (last 3 of 54 dims) are refreshed between chunks via
set_portfolio_for_backtest(), keeping position/PnL/exposure accurate at
chunk boundaries while batching inference within each chunk.
2640 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split model overhead into two constants: MODEL_OVERHEAD_MB (pure
model weights for batch-size capping) and TRIAL_VRAM_MB (total
per-trial VRAM for concurrent hyperopt planning). DQN trials
empirically consume ~7 GB each on L40S (model + GPU replay buffer +
experience collector + CUDA allocations + fragmentation), not the
200 MB previously estimated. This caused plan_hyperopt to compute
128 concurrent trials instead of the actual 5, inflating PSO
particles from 20→128 and total trials from 20→384 via .max()
instead of .min(), guaranteeing a 4h timeout kill.
Fix auto-scaling to: (1) match particles to GPU concurrency for
maximum hardware utilization on any node, (2) cap particles at
max_trials to never inflate the trial budget, (3) never auto-inflate
the total trial count.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DQN: wire SAC-style entropy regularizer into TD loss, fix diversity
penalty normalization from ln(3) to ln(45), add epsilon floor (0.05)
for noisy nets, enable sigma scheduling (0.8→0.4 with 0.3 floor),
add UCB count-based exploration bonus, expand hyperopt search space
from 43D to 45D.
PPO: replace fabricated entropy metric (value_loss×0.5) with real
Shannon entropy from action distribution, fix EntropyRegularizer
normalization from ln(3) to ln(45), implement Monte Carlo entropy
estimate for flow policy (was returning zeros), fix hyperopt VRAM
bound from 3 to 45 actions.
Monitoring: add normalized action entropy and diversity Prometheus
gauges, add exploration diagnostics logging per epoch.
2503 ml tests pass, 277 common tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PerformanceMetrics::from_trades() already returns win_rate as percentage
(e.g. 57.78), but TRIAL_SUMMARY and backtest details log lines multiplied
by 100 again, producing values like 5778%. Remove the extra * 100.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enable curiosity module by default (weight 0.0→0.1) to satisfy the
three-way gate (dueling + target + curiosity) that was blocking the
GPU experience collector CUDA kernel. This eliminates the 30-40% CPU
experience collection phase that was the main GPU idle bottleneck.
Additional changes:
- BatchSample API: train_step/compute_gradients now accept
Option<BatchSample> instead of Option<Vec<Experience>>, preserving
PER importance-sampling weights and indices through pre-sampling.
- Async PER pre-sampling: train_step_single_batch and
train_step_with_accumulation now pre-sample from the replay buffer
using a READ lock before acquiring the WRITE lock for GPU training.
This separates CPU sampling (~250μs) from GPU forward/backward (~3ms).
- Delete dead EpochPrefetcher: the binary (train_baseline_rl.rs) already
implements fold prefetching with background thread + mpsc channel +
GPU double-buffering, making the trainer's EpochPrefetcher redundant.
- Hyperopt DQN bounds: curiosity_weight min 0.0→0.01 so PSO can never
fully disable curiosity (which would re-gate the GPU collector).
10 files changed, -195 net lines. 2497 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace &Device::Cpu with &self.device for input and target tensor
creation in Mamba2 hyperopt adapter. Avoids unnecessary CPU→GPU
transfer during hyperparameter optimization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt adapter and PPO benchmark still had num_actions: 3, which would
produce misconfigured PPO models when used with the 45-action FactoredAction
sampling path. Found by spec compliance review.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
