Noisy nets are now always on (mandatory feature since config unification).
Epsilon start always uses the noisy-nets-aware default (0.05).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove use_double_dqn, use_dueling, use_per, use_branching,
use_distributional, use_noisy_nets, use_huber_loss, and use_cql
from DQNConfig, DQNHyperparameters, and DqnParams structs.
These features are always enabled (Rainbow DQN standard). The boolean
flags were dead code — every constructor set them to true, and the
only code paths that set them to false were in tests that disabled
features for simplicity. With the fields removed, the features are
unconditionally active, eliminating ~490 lines of dead configuration.
Key changes:
- Struct field declarations removed from 3 core config structs
- Conditional branches (if use_X { ... } else { ... }) simplified:
dueling/branching/PER network creation is now unconditional
- Checkpoint metadata hardcodes "true" for backward compatibility
- Hyperopt search space index 11 (use_branching) fixed at 1.0
- TOML/YAML config files cleaned of removed fields
- Tests that toggled these flags updated or rewritten
45 files changed, -487 net lines. Zero new test failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
use_qr_dqn was a hacky toggle that gated the IQN dual-head behind
a num_atoms threshold. IQN is now always enabled when iqn_lambda > 0
(default 0.25). C51 remains the main loss; IQN is the auxiliary head
for CVaR risk quantification.
Removed use_qr_dqn from: DQNParams, DQNHyperparameters, fused_training,
hyperopt adapter, all tests, all examples.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Created config/gpu/{default,rtx3050,h100,a100}.toml with all GPU-specific
parameters: batch_size, num_atoms, buffer_size, hidden_dim_base,
replay_buffer_vram_fraction, gpu_n_episodes, gpu_timesteps_per_episode,
cuda_stack_bytes.
GpuProfile::load() auto-detects GPU by device name, falls back to
embedded defaults (include_str!). Override via FOXHUNT_GPU_PROFILE env.
Removed dead code:
- detect_vram_mb(), vram_scaled_hidden_dims(), vram_scaled_base_dim(),
resolve_hidden_dim_base() + 18 tests for these functions
All callers updated: train_baseline_rl, DQNTrainer constructor,
PPO trainer, smoke tests, pipeline tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace scattered VRAM-based if/else chains with a declarative TOML profile
system. GPU profiles (rtx3050, a100, h100, default) are selected by device
name and embedded at compile time via include_str! for zero-filesystem
fallback in CI/containers, with filesystem and env var overrides.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- DQN CLI --max-steps-per-epoch was only wired to PPO (bug: 1394 steps ran instead of 10)
- Added per-substep timing: sample/fused/guard breakdown per epoch
- Phase 1a results: PER sampling now 0.1ms/step (was ~100ms with CPU roundtrips)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GpuTrainResult::gpu_tensor_to_cuda_slice: implement via CudaSlice::clone()
(was TODO returning hard error, blocking training guard loss readback)
- DQNAgentType::primary_dqn_mut(): new method returning &mut DQN for both
Standard and RegimeConditional variants
- FusedTrainingCtx: replace as_standard_mut() with primary_dqn_mut() at
all 3 call sites (EMA target update, IQN EMA, VarStore sync)
- fused_post_step/fused_post_step_no_ema/fused_post_step_gpu_bookkeeping:
delegate to primary_dqn_mut() instead of returning error for RegimeConditional
- train_baseline_rl: add error chain logging for fold failures
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Query CU_DEVICE_ATTRIBUTE_MAX_SHARED_MEMORY_PER_BLOCK_OPTIN at runtime
instead of name-based heuristics for shared memory tile sizing
- Cap shmem at 48KB on consumer GPUs (< 164KB hardware max) — A100/H100
with 164KB+ use full opt-in, RTX 3050/4090 use safe 48KB default
- Dynamic C51 atom scaling: <8GB→11, <16GB→21, >=16GB→51 atoms
- Dynamic CUDA stack: 16KB for 11 atoms, 32KB for 21, 64KB for 51
- VRAM-aware GPU experience episodes: 16 episodes on <8GB (was 256)
- Bypass CUDA Graph when shmem > 48KB (direct kernel launch fallback)
- Remove .max(51) on num_atoms_max — use actual configured atom count
- Shared query_max_shmem_bytes() used by both trainer and collector
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The fused DQN training kernel allocates DIST_SIZE(HIDDEN)*NUM_ATOMS
per-thread arrays on CUDA stack. With 51 atoms this needs ~52KB/thread
which exceeds stack limits on RTX 3050 (4GB). Scale atoms dynamically:
<8GB→11, <16GB→21, >=16GB→51 (matches smoke_test_real_data pattern).
Note: experience collector still compiles with atoms_max=51 hardcoded
(separate issue — needs collector kernel dim parameterization).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Cast input to weight dtype in DQN residual, rmsnorm, noisy_layers
- Set use_gpu=true in QNetworkConfig defaults and all config sites
- Resolve BF16 boundary mismatches in attention, curiosity, branching,
distributional_dueling across ml-dqn
- GPU-resident regime ops with BF16 boundary casts, eliminate .expect() in CUDA paths
- Eliminate all Device::Cpu fallbacks — GPU-only across 10 ML crates
- PPO: cast logits to F32 before softmax, cast batch tensors to training dtype
- Gradient collapse detection for RegimeConditionalDQN
- Wire halt_grad_collapse from CUDA guard kernel to halt training
- Dead neuron detection uses active network VarMap + squeeze factored readback
- Increment gradient_logging_step in GPU PER path
- Gradient collapse warmup guards use original buffer_size
- Cap training steps per epoch + tracing migration
- Replace Tensor::all() with sum_all() for pinned Candle compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CUBLAS_WORKSPACE_CONFIG and NVIDIA_TF32_OVERRIDE are set once at
startup before any threads or CUDA work begins. The unsafe block is
correct but triggers -W unsafe-code. Adding #[allow(unsafe_code)] at
the call site silences the warning while keeping the safety comment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- gradient_utils: Add TensorId-based fallback when Var identity mismatch
causes 0/N vars to match GradStore (Candle Adam clones Arcs). Fallback
computes norm AND clips via insert_id. Throttled warning (1st + every
1000th). 7 unit tests including mismatch-actually-clips.
- monitoring: Track full 45-action factored space (5 exposure × 3 order
× 3 urgency). Fix validate_rewards false alarm on GPU path where single
aggregated mean_reward per epoch gives N=1 → std=0.
- trainer: GPU experience collection routes exposure actions through
route_action() for factored tracking instead of exposure-only counts.
Applied in both per-step and epoch-summary paths.
- train_baseline_rl: Auto-detect VRAM <8GB → disable GPU replay buffer
to prevent OOM on RTX 3050 Ti class GPUs.
- smoke_test_real_data: E2E DQN training test with 6 assertions (epoch
completion, loss decrease, finite losses, Q-value divergence, 45-action
space, finite gradient norms).
Validated: 1642 tests pass (ml=915, ml-core=311, ml-dqn=416), 0 clippy
warnings, baseline RL trains 10 epochs on CUDA with Sharpe +5.45.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN hyperopt adapter: remove static VRAM gate for GPU experience
collector and GPU PER — dynamic scaling handles constraints at
runtime with graceful CPU fallback on init failure
- Remove is_parquet_file branching from hyperopt eval path (all data
loads via DBN pipeline now)
- Update training example CLI args for consistency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand FeatureVector from 40 to 42 dimensions by including ADX(14) at
index 40 and CUSUM direction at index 41 from the existing CPU feature
extraction pipeline. This eliminates proxy-based regime classification
and enables GPU-native regime detection via tensor narrow/comparison ops.
Key changes:
- extraction.rs: wire RegimeADXFeatures + RegimeCUSUMFeatures into
extract_current_features_v2(), output 42 features per bar
- regime_conditional.rs: classify_regime_masks_gpu() creates per-regime
mask tensors entirely on GPU (ADX > 0.25 = trending, |CUSUM| > 0.7 =
volatile, else ranging). Zero CPU roundtrip in training hot path.
- trainer.rs/config.rs: state_dim 43→45 (no OFI), 51→53 (with OFI),
aligned dims unchanged (48/56). GPU batch insertion for all 3 heads.
- CUDA header: MARKET_DIM 40→42
- walk_forward.rs: FEATURE_DIM 40→42
- 42 files updated, all [f64;40]→[f64;42] propagated across workspace
Test results: ml=874/0, ml-dqn=354/0, ml-features=282/0, ml-core=274/0
Real data GPU smoke tests: 7/7 passed (OHLCV + OFI + trade enrichment)
Hyperopt baseline RL: 2 trials completed on local RTX 3050 Ti
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add QuestDB ILP sink for training metrics, update Prometheus scrape
configs, and fix network policies for monitoring stack connectivity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire 8 OFI features (OFI L1/L5, depth imbalance, VPIN, Kyle's lambda,
bid/ask slopes, trade imbalance) through the DQN training pipeline:
- Add mbp10_data_dir config field to DQNHyperparameters
- Dynamic state_dim: 43 (no OFI) or 51 (with OFI) based on config
- Compute OFI per bar during data loading, store on trainer
- Pass OFI features through regime_features slot in TradingState
- Configurable MBP-10 path with recursive .dbn/.dbn.zst discovery
- Add zstd auto-detection to DbnParser::parse_mbp10_file()
- Add --mbp10-data-dir CLI flag to train_baseline_rl
- Fix hardcoded [f64; 51] → FeatureVector51 ([f64; 40]) across
examples, walk_forward, GPU memory profile, and test fixtures
- Fix stale state_dim=51 in dqn_config_2025() and DQN tests
2747 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Critical bug: all 3 DQN action selection methods (select_action,
select_action_with_confidence, select_action_inference) used
FactoredAction::from_index() which maps indices 0-4 to exposure_idx=0
(Short100) via division by 9. This is the root cause of action
diversity collapse during both training and production inference.
Fix: ExposureLevel::from_index() + OrderRouter::route_default() in all
DQN paths. Also fixes hyperopt objective thresholds (<10 → <3 for
5-action degenerate detection), stale defaults/comments, integration
test configs.
Files: dqn.rs (3 methods), trainer.rs (validation + select_action),
hyperopt/adapters/dqn.rs (thresholds), dqn_model.rs (comments),
train_baseline_rl.rs (default), reward.rs (comment),
dqn_integration.rs + ensemble_integration.rs (num_actions).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Multi-window backtest: splits validation data into 3 non-overlapping
windows and aggregates with mean(Sharpe) - 0.5*std(Sharpe), penalizing
inconsistency and reducing overfit to a single data segment.
Top-K ensemble: hyperopt now emits top_k_params (top 5 trials) in JSON
output. train_baseline_rl gains --ensemble-top-k flag to train multiple
models per fold from different hyperopt configs, saving checkpoints as
dqn_ensemble_{k}_fold_{n}.safetensors.
Workflow template: adds ensemble-top-k parameter (default 5) and passes
--ensemble-top-k to the train-best step.
2720 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ephemeral Argo workflow pods terminate after training completes, causing
Prometheus to lose all scraped metrics. Add push_to_gateway() to POST
final metrics to the existing pushgateway service so they persist on the
Grafana training dashboard after pod completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt found hold_penalty_weight=1.818, curiosity_weight=0.402, etc.
but walk-forward training used tiny defaults (0.01, 0.1), causing the
agent to learn "holding is optimal" → 0 trades across all 50 epochs.
Now passes: hold_penalty_weight, max_position_absolute, huber_delta,
entropy_coefficient, curiosity_weight, weight_decay, kelly_fractional,
kelly_max_fraction from hyperopt JSON to DQNHyperparameters.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Walk-forward DQN training hardcoded epsilon_start=1.0 with noisy_nets=true,
forcing pure random exploration for all 50 epochs (0 trades, 0 Sharpe).
Now reads epsilon, PER, dueling, distributional, noisy nets, and QR-DQN
params from the hyperopt JSON. When noisy_nets=true, defaults epsilon to
0.05 instead of 1.0.
Evaluator now falls back to highest dqn_fold{N}_epoch{E}.safetensors when
_best.safetensors doesn't exist (early stopping saves epoch checkpoints).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire LR scheduler to actually update the Adam optimizer (was logged but
never applied). Add update_learning_rate to DQN, RegimeConditionalDQN,
and DQNAgentType so decay_factor propagates through all agent variants.
Fix train/eval parity: CLI defaults 51→54 features, 3→45 actions;
DQN eval always uses 3-layer hidden_dims; PPO eval uses 5-layer value
network matching trainer. enhanced_ml.rs hardcoded config updated from
state_dim=16/num_actions=3 to 54/45.
Fix Candle F32/BF16 traps: replace `Tensor * 0.5` (f64 literal) with
broadcast_mul(Tensor::full(0.5_f32)) in quantile_regression.rs (2x),
dqn.rs Huber loss, and IQN gamma multiplication. Prevents panics on
Ampere+ BF16 GPUs.
Add urgency_weight() multiplier to training cost model in reward.rs
and portfolio_tracker.rs — urgency dimension (Patient/Normal/Aggressive)
now affects learned value function, matching evaluate_baseline.rs.
Fix equity tracking: additive (equity += ret) → multiplicative
(equity *= 1.0 + ret) in compute_metrics. Fix total return calc.
Fix PER beta annealing: epochs*70 → epochs*1000 (~1015 actual steps
per epoch for 130k bars / batch 128).
Fix silent target network freeze: mutex lock failure now propagates
error instead of silently skipping weight update.
Make save_checkpoint atomic (write .tmp then rename). Make NormStats
write atomic with error logging instead of silent discard.
Convert EnsembleConfig::new assert! → Result<Self, MLError> with
14 call site updates. Fix hyperopt result serialization to warn
instead of silently dropping to Value::Null.
Fix clippy MSRV mismatch: clippy.toml 1.75 → 1.85 matching Cargo.toml.
2732 tests pass, 0 clippy warnings across workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQNAgent::load_from_safetensors: validate architecture hash before
loading weights (was bypassing validation entirely - P0 production gap)
- DQNEnsemble::save_to_directory: embed architecture metadata via
safetensors::serialize_to_file instead of bare VarMap::save (save/load
round-trip was guaranteed to fail)
- architecture_hash: hash hidden_dims.len() before values to prevent
theoretical collision between different-length configs
- train_baseline_rl: NormStats write now atomic (write .tmp then rename)
with cleanup guard on rename failure; serialization error is now loud
(error! + return None) instead of silently discarded
- train_baseline_rl: checkpoint rename failure cleans up orphaned .tmp
- hyperopt_baseline_rl: fix clippy single_match_else (match on equality
check -> if/else)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Atomic checkpoint writes: write to .tmp then fs::rename() (POSIX atomic)
- Remove redundant thread::sleep(50ms) after CUDA tensor readback sync
- NormStats: bail on missing file instead of computing from test data (lookahead bias)
- q_value_std: warn when missing from training metrics instead of silent 0.0 fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The batch span processor needs a tokio runtime for gRPC transport and
periodic flush. Async services already have one via #[tokio::main], but
sync training binaries (hyperopt, train, evaluate) don't.
Previous approach (making binaries async with #[tokio::main]) caused
"Cannot start a runtime from within a runtime" panics because the ML
crate's internal code creates its own tokio runtimes for block_on().
New approach: build_otel_tracer() detects runtime context via
Handle::try_current(). If absent, it creates a dedicated 1-worker
multi-thread runtime stored in a process-lifetime OnceLock. The worker
thread actively polls the OTLP batch export task.
Reverts training binaries to sync fn main() so internal runtime creation
(hyperopt adapters, DQN/PPO trainers) continues working as before.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 training binaries (hyperopt_baseline_rl, hyperopt_baseline_supervised,
train_baseline_rl, train_baseline_supervised, evaluate_baseline,
evaluate_supervised) used sync fn main() but the OTLP batch exporter
requires a tokio runtime (tonic/hyper-util gRPC transport). This caused
an immediate panic on CI when OTEL_EXPORTER_OTLP_ENDPOINT was set.
Fix: #[tokio::main(flavor = "current_thread")] on all 6 binaries.
Also fix pre-existing clippy warnings (shadow, let_underscore_must_use,
doc_markdown, cognitive_complexity, integer_division, unsafe_code).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN PER: defer td_errors to_vec1() after loss.to_scalar() — piggyback on
existing pipeline flush instead of forcing premature GPU→CPU stall
- PPO trajectories: capacity-hint Vec allocations, extend_flat_states methods,
states_flat field on TrajectoryBatch for zero-copy GPU upload
- TGGN validate(): batch N per-sample losses on GPU → single to_scalar() sync
(was N GPU→CPU syncs)
- Liquid backward(): batch grad-norm per-param sqr().sum_all() on GPU → single
to_scalar() sync (was N GPU→CPU syncs per optimizer step)
- Liquid validate(): same N→1 GPU sync reduction as TGGN
- DQN trainer: restore EpochPrefetcher/DoubleBufferedLoader API (wrongly deleted)
- train_baseline_rl: wire DoubleBuffer GPU pre-upload — after CPU prefetch
completes, immediately upload next fold to GPU via DqnGpuData::upload() so
next fold starts with data already resident on GPU
2478 tests pass, 0 clippy warnings, 0 compile errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace tracing_subscriber::fmt() with init_observability() which adds
JSON structured logging + optional OTLP export to Tempo. When
OTEL_EXPORTER_OTLP_ENDPOINT env var is unset, falls back to fmt-only.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Emit set_epoch, set_epoch_loss, set_validation_loss, set_iteration_seconds
after each DQN/PPO fold completes. Feeds training cockpit Grafana dashboard.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously train_baseline_rl.rs only passed commission (tx_cost_bps) to
DQN/PPO trainers, ignoring bid-ask spread slippage. Now computes per-fold
average spread via spread_cost_bps() (same as evaluate_baseline) and passes
total cost (commission + spread) to both trainers.
Removes #[allow(dead_code)] — function is now used by all 4 example binaries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace raw DQN::new() + manual training loop in the walk-forward
training binary with DQNTrainer, which automatically activates:
- Mixed precision (BF16/F16 auto-detected from GPU)
- Dynamic batch sizing (AutoBatchSizer + HardwareBudget)
- Gradient accumulation
- Full Rainbow DQN (PER, dueling, C51, noisy nets, n-step)
- Regime-conditional Q-networks
- Portfolio tracking, Kelly sizing, entropy regularization
The walk-forward fold structure (data loading, feature extraction,
window generation, normalization) stays in the binary — only per-fold
training delegates to DQNTrainer::train_with_preloaded_data().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire BF16/FP16 mixed precision end-to-end for DQN and PPO with auto-detection
from GPU name (Ampere+ → BF16, Volta/Turing → FP16). Add hidden_dim_base to
hyperopt and wire through training/eval binaries. Reduce GPU sync points: make
DQN NaN checks periodic (every 100 steps), replace PPO GAE GPU round-trip with
pure CPU implementation. Cache training data across hyperopt trials for all 10
models via Arc. Batch DQN experience storage (128x fewer lock acquisitions).
Correct VRAM constants and batch bounds for all 9 supervised model adapters.
28 files changed, +1207/-208 lines. 2418 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Prometheus training metrics from example-local baseline_common/
to common::metrics::{server,training_metrics} following the existing
grpc_metrics.rs pattern. Fix 29 let_underscore_must_use clippy errors
in push_metrics.rs, 3 shadow lint errors in training binaries, and
demote gradient clipping log from warn to debug.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create shared baseline_common/metrics.rs module that registers all 18
dashboard-expected metrics (11 gauges, 5 counters, 2 histograms) and
spawns a lightweight HTTP metrics server on port 9094.
Instrument train_baseline_supervised and train_baseline_rl with:
- Epoch progress, training/validation loss gauges
- Checkpoint save timing, size, and failure counters
- NaN/gradient explosion detection counters
- Data loading latency histograms
- Active workers lifecycle (1 on start, 0 on exit)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend ml_training_service to dispatch GPU training jobs as K8s batch/v1
Jobs, collect results via a Rust sidecar uploader, and support model
promotion with operator approval via fxt CLI.
- K8s dispatcher creates Jobs on gpu-training pool with native sidecar
- training_uploader crate: watches DONE/FAILED marker, uploads to S3,
reports completion via ReportJobCompletion gRPC
- PromotionManager compares metrics, queues better models for approval
- 4 new proto RPCs: ReportJobCompletion, ListPendingPromotions,
ApprovePromotion, RejectPromotion
- fxt commands: train start, model list/approve/reject
- Training binaries write DONE/FAILED markers + metrics.json
- Dockerfile, K8s job template, and CI pipeline updated
- StartTraining gracefully falls back to in-process when outside K8s
- 27 new tests (16 service + 11 promotion), 141 total service tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a parent/child GitLab CI pipeline for ML model training:
- Generator script produces per-model hyperopt/train/evaluate jobs
- Parent pipeline (.gitlab-ci-training.yml) with manual trigger
- NFS-backed ReadWriteMany PVC for shared training outputs
- Hyperopt params wired into training binaries (DQN, PPO, TFT, Mamba2)
- Shared DBN loader eliminates duplicate code across hyperopt adapters
- Supervised hyperopt unified to DBN data (was parquet-only)
Pipeline: hyperopt (4 models) → train (10 models) → evaluate ensemble
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete 22 dead/placeholder/broken example files (-3,489 lines code)
- Delete 4 tracked CSV files (-1.1M lines, were accidentally committed)
- Move baseline training data default from data/cache/ to test_data/
- Update 5 unified binary defaults, gitignore, k8s upload comment, docs
- Consolidate all training data under test_data/futures-baseline/
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
