Bring Branching Dueling Q-Network (Tavakoli 2018) to full Rainbow parity
with the existing GPU hotpath. 3 independent advantage heads (exposure=5,
order=3, urgency=3) decompose the 45-action space into learnable branches.
H1 - CUDA fallback: gate GpuExperienceCollector when use_branching=true
(fused kernel hardcodes NUM_ACTIONS=5, incompatible with 45 factored)
H2 - Per-branch C51 distributional: each branch outputs [batch, n_d, atoms]
log-softmax, loss = avg of D cross-entropies vs projected Bellman target
M1 - NoisyNet: MaybeNoisyLinear enum in branch heads, reset_noise/disable_noise
wired through select_action, compute_loss, and set_eval_mode
M2 - Regime-conditional IS weights: Trending=1.2, Ranging=0.8, Volatile=0.6
applied to branching loss via ADX/CUSUM features at state[40:41]
M3 - State dim alignment: align_dim_for_tensor_cores() in from_dqn_params()
for H100 HMMA dispatch (8-byte alignment)
L1 - Fill simulator: splitmix64 replaces golden ratio hash (chi-squared tested)
L2 - Hyperopt 29D: branch_hidden_dim [64,256] added to PSO search space
Config plumbing: branch_hidden_dim, v_min/v_max/num_atoms, use_distributional,
use_noisy, noisy_sigma_init all flow from DQNConfig → BranchingConfig.
10 files, +3207/-125 lines, 33 branching tests + 387 ml-dqn + 284 ml-core pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
training_dtype() now returns BF16 on all CUDA devices, enabling full
tensor-core utilization. F32 is used only at boundaries (scalar
extraction, loss computation, softmax). VRAM estimator updated to
account for BF16 byte sizes, C51/QR atoms, dueling streams, NoisyNet
param doubling, and GPU PER buffer pre-allocation.
Changes:
- mixed_precision.rs: training_dtype() returns BF16 on CUDA
- curiosity.rs: F32 cast before scalar extraction
- network.rs: F32 output at NetworkLayers forward boundary
- traits.rs: estimate_trial_vram_mb_full() with BF16-aware sizing
- dqn.rs adapter: uses full estimator with worst-case architecture
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expand FeatureVector from 40 to 42 dimensions by including ADX(14) at
index 40 and CUSUM direction at index 41 from the existing CPU feature
extraction pipeline. This eliminates proxy-based regime classification
and enables GPU-native regime detection via tensor narrow/comparison ops.
Key changes:
- extraction.rs: wire RegimeADXFeatures + RegimeCUSUMFeatures into
extract_current_features_v2(), output 42 features per bar
- regime_conditional.rs: classify_regime_masks_gpu() creates per-regime
mask tensors entirely on GPU (ADX > 0.25 = trending, |CUSUM| > 0.7 =
volatile, else ranging). Zero CPU roundtrip in training hot path.
- trainer.rs/config.rs: state_dim 43→45 (no OFI), 51→53 (with OFI),
aligned dims unchanged (48/56). GPU batch insertion for all 3 heads.
- CUDA header: MARKET_DIM 40→42
- walk_forward.rs: FEATURE_DIM 40→42
- 42 files updated, all [f64;40]→[f64;42] propagated across workspace
Test results: ml=874/0, ml-dqn=354/0, ml-features=282/0, ml-core=274/0
Real data GPU smoke tests: 7/7 passed (OHLCV + OFI + trade enrichment)
Hyperopt baseline RL: 2 trials completed on local RTX 3050 Ti
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Four monitoring functions (estimate_avg_q_value_with_early_stopping,
collect_qvalue_statistics, compute_q_gap_for_epoch, compute_per_action_q_values)
iterated over batch_sample.experiences to build CPU tensors for forward passes.
When GPU PER (GpuPrioritized) is active, experiences is always vec![] — all data
lives on GPU tensors in gpu_batch. This created zero-element tensors with non-zero
shapes, triggering CUBLAS_STATUS_INVALID_VALUE on the next forward pass.
Fix: all four functions now check for gpu_batch.states and use it directly,
falling back to CPU experiences only for non-GPU buffers. Also removes debug
eprintln probes, wires new_on_device for DQN/RegimeConditionalDQN construction,
and adds GPU-aware smoke tests (33 pass, 874 total, 0 failures).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Eliminates the CPU roundtrip in PER binary search: previously the
cumsum tensor (~400KB for 100K buffer) was downloaded to CPU, searched
in a loop, then indices uploaded back. Now a CUDA kernel runs one
thread per target with O(log n) binary search — zero DMA.
- Add searchsorted_kernel.cu (40-line CUDA binary search kernel)
- Implement SearchSorted as Candle CustomOp2 with CPU fallback
- Wire into sample_proportional() and sample_rank_based()
- Fix all clippy warnings in gpu_replay_buffer.rs (const fn, shadow,
doc backticks, to_owned, div_ceil, module_name_repetitions)
- Fix wildcard_enum_match_arm in mixed_precision.rs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace expensive CPU↔GPU data transfers with GPU-native operations:
- check_gradients_finite: sum_all() scalar readback (4B per tensor)
replaces flatten_all()+to_vec1() that copied entire gradients to CPU
(up to 200MB per step across 9 call sites in DQN+PPO)
- Huber loss: affine() replaces 3× Tensor::from_vec(vec![const; N])
CPU Vec allocations in the inner loss computation loop
- update_priorities_gpu: per-index slice_scatter (~1KB DMA) replaces
full-buffer to_vec1()+from_vec() roundtrip (1.2MB DMA per step)
- PER sampling: single from_vec + GPU to_dtype replaces duplicate
from_vec calls and CPU type conversion roundtrips
- RegimeConditional: log warning on silent GPU batch drops
822 tests pass (350 ml-dqn, 274 ml-core, 198 ml-ppo), 0 clippy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes for GPU PER hot path:
1. IQN quantile loss used empty CPU weights Vec instead of GPU-resident
weights_tensor_cached — caused CUDA_ERROR_ILLEGAL_ADDRESS from
uninitialized GPU memory. Now uses cached GPU tensor matching C51
and standard DQN paths.
2. GpuPrioritized add()/add_batch() replaced with StagedGpuBuffer:
add() stages on CPU (Vec::push, zero GPU ops), sample() batch-flushes
staging→GPU in one DMA before sampling. Production path (insert_batch_tensors)
bypasses staging entirely — GPU→GPU with zero CPU.
3. All 9 ML sub-crates default to cuda feature so `cargo test -p ml-dqn`
exercises GPU code paths on CUDA workstations. CI service crates use
default-features=false, unaffected.
Test results: 350 passed (was 343+7 failed), 0 failed, 1 ignored.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Wire GpuReplayBuffer into DQN constructor with OOM fallback to CPU PER
- Fix P0: GPU PER priorities never updated in single-batch train_step()
(result.indices was empty CPU Vec; GPU tensors td_errors_gpu/indices_gpu
were silently dropped)
- Fix IS-weight ordering: apply weights AFTER Huber loss, not before
(weighting before nonlinear Huber shifts quadratic/linear regime boundary)
- Fix IQN CVaR: add .contiguous() before sort_last_dim (341/341 tests pass)
- Defer loss scalar readback to after backward pass (piggyback on grad flush)
- Add next_states to ExperienceBatch with episode-aware shift computation
- Add insert_batch_tensors() for direct GPU tensor insertion into replay buffer
- Make Q-value estimation periodic (every 50 steps) to reduce forward passes
- Fix CPU fallback path types (u8 action, i32 fixed-point reward, timestamp)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Plan to reduce ml crate from 91K to ~12K LOC by extracting trainers
into model sub-crates, hyperopt adapters into ml-hyperopt, and
infrastructure into existing sub-crates. ml becomes an orchestration
layer owning inference, model factory, and training pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ml-supervised, ml-ensemble, ml-labeling, ml-explainability, ml-hyperopt
all had default = ["cuda"] which pulled cudarc into the CPU services
build, causing compile-services to fail with "nvcc not found".
Changed all to default = [] and wired ml/Cargo.toml cuda feature to
propagate to all 8 sub-crates (was only 3: ml-core, ml-dqn, ml-ppo).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a `source` template variable to the training dashboard that controls
which Prometheus job labels are queried:
- Live: only `training-pods` (running pods scraped directly)
- CI History: only `.*_baseline.*` (completed CI runs via pushgateway)
- All: both sources combined
All 37 panel queries now use `job=~"$source"` for consistent filtering.
Active Workers panel stays hardcoded to `job="training-pods"` since it
only makes sense for live pods. Template variable queries (model/fold
dropdowns) also respect the source selector.
Default is "Live" — dashboard shows only the currently running training
session with no pushgateway stale data contamination.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stat panels (Active Workers, Current Epoch, Trial Progress, Elapsed, Best
Objective) showed conflicting values from pushgateway stale gauges overlapping
with live training pod metrics. Scoped all stat/gauge panels and template
variable queries to job="training-pods" so the dashboard reflects only the
currently running training session. Time-series panels remain unfiltered via
$model template variable scoping — dropdown only lists live models, so
historical pushgateway data is naturally excluded without explicit filtering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove cuda from default features in ml-core, ml-dqn, ml-ppo, ml
- Propagate cuda feature from ml → ml-core/ml-dqn/ml-ppo
- CI compile-training already uses --features ml/cuda explicitly
- Fix MaxDD log format: {:.1}% → {:.3}% (was rounding 0.033% to 0.0%)
- Suppress unused_labels/unused_variables warnings for cfg(cuda) code
- Add CALLBACK_ENDPOINT env to ml-training-service deployment
- Fix Grafana active_workers query to use sum() with fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prometheus was configured to scrape pods via prometheus.io/scrape
annotations, but all services and training jobs used gitlab.com/
prefix from legacy GitLab-managed Prometheus — causing Prometheus
to never discover any foxhunt pods. This resulted in stale/missing
metrics on the Grafana training dashboard.
12 files updated across services/, gpu-overlays/, training/, and
monitoring/ (node-exporter, dcgm-exporter cleanup).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The nodeSelector pointed to non-existent 'ci-training' pool. The actual
Scaleway pool is 'ci-training-h100', matching the CI pipeline template.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU kernel produced raw percentage PnL rewards (~3.5e-4) which were
~10,000x smaller than Q-values (~7.0), making rewards invisible in the
Bellman backup. Q-values froze at initialization.
Port the CPU RewardNormalizer algorithm (EMA mean/variance, z-score
normalization, [-3,3] clamp) directly into the CUDA kernel hot path
as per-thread register state. Each thread maintains its own running
mean/variance with configurable decay rate (reward_norm_alpha, default
0.01 = ~100-step window). EMA resets on episode boundaries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
21 sub-crates extracted from the 260K-line ml monolith, reducing it to ~90K.
Deletes ~7.5K lines of dead code. Zero compilation errors, zero warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused struct fields, dead methods, unreachable code across
ml-dqn, ml-supervised, ml-features, ml-ppo, ml-checkpoint, ml-labeling,
ml-observability, and ml-universe. Gate test-only infra behind #[cfg(test)].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move UnifiedTrainable trait, TrainingMetrics, CheckpointMetadata, and
checkpoint helpers from ml to ml-core (zero new dependencies — ml-core
already had candle-core + serde_json)
- Wrap Mamba2SSM in Mamba2TrainableAdapter to satisfy orphan rule (trait
in ml-core, type in ml-supervised — all other 9 models already used
wrapper pattern)
- Make Mamba2SSM::validate() pub for cross-crate adapter access
- Delete 5 permanently disabled deployment modules (cfg(any()) — never
compiled): registry, hot_swap, validation, monitoring, endpoints
(-5,842 lines)
- Delete 2 undeclared dead files in training/: dqn_trainer.rs,
transformer_trainer.rs (-138 lines)
- Fix pre-existing compute_loss test shape mismatch in mamba adapter
UnifiedTrainable in ml-core unblocks future trainers/ extraction (17.8K
lines) since model-specific trainers can now depend on ml-core for the
trait without pulling in the full ml monolith.
14 files changed, +128 -6,497 (net -6,369 lines)
Tests: 274 ml-core + 948 ml = 1,222 passed, 0 failed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move TFT, Mamba-2, Liquid, TGGN, TLOB, KAN, xLSTM, and Diffusion model
implementations to ml-supervised. Bridge files (UnifiedTrainable adapters,
Checkpointable impls) stay in ml. Delete AsyncDataLoader (replaced by
StreamingDbnLoader + simple .chunks() batching). Remove empty ml-infra
scaffold — the remaining ml modules are too tightly coupled for clean
extraction, so ml stays as the orchestration facade.
- ml-supervised: 234 tests, 0 failures
- ml: 1687 tests, 0 failures
- Workspace: 0 compilation errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 24 PPO source files + flow_policy/ from ml into standalone
ml-ppo crate. The ml crate's ppo module is now a thin re-export layer
(`pub use ml_ppo::*`) plus two bridge files (trainable_adapter.rs,
stress_testing.rs) that depend on ml-internal types.
Key changes:
- ml-ppo: 25 modules (incl flow_policy subdir), 198 tests, standalone
- ml: depends on ml-ppo, re-exports via ppo/mod.rs
- Import rewrites: crate::common::action → ml_core::action_space,
crate::dqn::{mixed_precision,xavier_init} → ml_core::*,
crate::gradient_accumulation → ml_core::gradient_accumulation
- ml tests: 1929 pass (down from 2127 — 198 moved to ml-ppo)
- Workspace: 0 errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 53 DQN source files + gpu_replay_buffer from ml into standalone
ml-dqn crate. The ml crate's dqn module is now a thin re-export layer
(`pub use ml_dqn::*`) plus two bridge files (trainable_adapter.rs,
stress_testing.rs) that depend on ml-internal types.
Key changes:
- ml-dqn: 45 modules, 334 tests, compiles standalone
- ml: depends on ml-dqn, re-exports via dqn/mod.rs
- DQN.config: pub(crate) → pub for cross-crate access
- DQNAgent checkpoint methods moved into ml-dqn
- gpu_replay_buffer moved from cuda_pipeline/ to ml-dqn
- 8 dead-code files removed (never declared in mod.rs)
Net: -30,648 lines from ml crate. Workspace: 0 errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Sharpe ratio metrics and SIMD performance module to ml-core.
These only depend on MLError which is already in ml-core.
Add approx = "0.5" to ml-core dev-dependencies for Sharpe tests.
Skip observability/ (needs prometheus dep — stays in ml).
Test counts: 271 ml-core + 2487 ml = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 6 shared modules (~2.3K lines) from ml to ml-core:
- trading_action.rs (TradingAction enum)
- action_space.rs (action masking, re-exports)
- xavier_init.rs (Xavier/Glorot weight initialization)
- mixed_precision.rs (AMP utilities, FP16/BF16)
- order_router.rs (deterministic order routing)
- portfolio_tracker.rs (portfolio state tracking)
With TradingAction now in ml-core alongside FactoredAction, the
FactoredActionExt extension trait is eliminated entirely —
from_trading_action()/to_trading_action() become inherent methods
on FactoredAction. This removes the need for `use FactoredActionExt`
imports in reward.rs, gae.rs, and trajectories.rs.
Test counts: 250 ml-core + 2508 ml = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move memory_optimization/ (4.6K lines, 7 files) from ml to ml-core
- Move batch_size_resolver.rs to ml-core (now both deps in same crate)
- Add tempfile to ml-core dev-dependencies (qat tests)
- Re-export both modules from ml facade
- Keep checkpoint/ in ml (model_implementations.rs has model-specific deps)
185 ml-core tests + 2573 ml tests = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move safety/ module (6.5K lines) from ml to ml-core (self-contained)
- Keep security/ in ml (depends on ensemble::EnsembleDecision)
- Move From<ProductionTrainingError> for MLSafetyError to ml (cross-crate)
- Rename Real-prefixed inference types to proper names:
RealInferenceError → InferenceError
RealInferenceConfig → InferenceConfig
RealPredictionResult → InferencePrediction
RealNeuralNetwork → NeuralNetwork
RealMLInferenceEngine → MLInferenceEngine
- Re-export safety from ml facade for backward compatibility
129 ml-core tests + 2629 ml tests = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move optimizers, gradient_accumulation, gradient_utils, cuda_compat,
tensor_ops, and gpu (device config, capabilities, memory profiling) to
ml-core. These are shared compute primitives used by all models.
Also commit module files for core types (common, config, error, model,
traits, types) that were moved from ml to ml-core in task 5a but left
staged without being committed.
Notable changes:
- resolve_batch_size() stays in ml (new batch_size_resolver module)
because it depends on memory_optimization::auto_batch_size which
has not yet moved to ml-core
- FactoredAction legacy bridge converted from inherent impl to
extension trait (FactoredActionLegacy) since FactoredAction is now
defined in ml-core, not ml
- candle-optimisers added to ml-core dependencies (needed by Adam)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all inline type definitions from ml/src/lib.rs to ml-core:
MLError, MLResult, Trade, MarketRegime, HealthStatus, Features,
MLModel trait, ModelRegistry, ParallelExecutor, LatencyOptimizer,
TrainingMetrics, ValidationMetrics, InferenceResult, ModelMetadata.
Dedup: consolidate ConfigError{reason}/ConfigurationError(msg) into
single ConfigError(String) tuple variant (was 2 variants, 174 refs).
Cleanup: convert create_hft_* free functions to associated methods
(HFTPerformanceProfile::ultra_low_latency(), ParallelExecutor::hft()).
ml facade re-exports via `pub use ml_core::*`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move mixed_precision, xavier_init, portfolio_tracker, action_space,
order_router from dqn/ to ml/src/ root. Extract TradingAction enum
from dqn/agent.rs to standalone trading_action module. Re-export
from dqn/mod.rs for backward compatibility.
These modules are shared infrastructure used by PPO, trainers,
hyperopt, and ensemble — not DQN-specific.
Test results: 2757 passed, 0 failed, 25 ignored (unchanged).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 5 empty sub-crates (ml-core, ml-dqn, ml-ppo, ml-supervised, ml-infra)
to workspace. Modules will be moved from monolithic ml crate in subsequent
tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split ml-rl into ml-dqn and ml-ppo for 3-way parallel compilation
and better sccache hit rate. Extract TradingAction to ml-core to
enable full DQN/PPO independence.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the monolithic ml crate (260K lines, 55s compile) into 5 crates:
ml-core, ml-rl, ml-supervised, ml-infra, ml (facade).
15-task plan with full module inventory and import migration guide.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three bugs in the GPU experience collection hot path:
1. gpu_batch_to_experiences() had hardcoded state_dim=43 but the CUDA kernel
outputs states at the ALIGNED dimension (56 with OFI, 48 without). After
sample 0, every replay buffer entry had corrupted state vectors — the
network was learning from garbage data.
2. GPU path never called monitor.track_reward(), so mean_reward was always
reported as 0.0 in epoch logs despite the agent generating real rewards.
3. Action tracking was double-counted (direct array write + track_action_by_exposure),
inflating diversity metrics by 2x. Consolidated into single bounded call.
Also adds missing app.kubernetes.io/component label to job-template.yaml
so Prometheus training-pods scrape job discovers training pods.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PSO can now tune sharpe_weight in [0.0, 0.5] instead of hardcoded 0.3.
This lets hyperopt discover the optimal Sharpe ratio blending weight
in the composite reward signal per-symbol.
Also updates docstring to reflect current C1-C4 search space state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU experience kernel had hardcoded STATE_DIM=54, MARKET_DIM=51,
NUM_ATOMS_MAX=51 but the host-side feature buffer uploads 40 features
per bar and networks use state_dim=56 with up to 100 atoms. Past ~702K
bars the stride mismatch caused out-of-bounds GPU reads → ILLEGAL_ADDRESS.
All dimension constants now use #ifndef guards so the NVRTC JIT compiler
receives the actual values via #define injection — zero runtime overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt varies hidden_dim_base (256–1024) but the CUDA experience
kernel hardcoded SHARED_H1=256, SHARED_H2=256. When hyperopt picked
larger dims, the kernel did matvec with wrong strides → illegal memory
access → training crash on first epoch.
Fix: wrap CUDA #defines in #ifndef guards and inject actual dims from
the dueling network config at NVRTC compile time. The kernel is now
specialized per-trial with the exact weight matrix dimensions — zero
runtime overhead, no dimension mismatch possible.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
from_dqn_params() was stripping the last hidden_dim, creating only 1
shared layer (shared_0) when hidden_dims=[256,256]. The CUDA experience
kernel hardcodes 2 shared layers and gpu_weights.rs expects shared_1.*
weight keys. This caused "Missing weight: shared_1.weight" every epoch,
forcing CPU fallback for all experience collection on H100.
Fix: use all hidden_dims as shared layers. Default [256,256] now creates
shared_0 + shared_1, matching the CUDA kernel architecture exactly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The DQN hyperopt adapter has an explicit trades_data_dir field from CLI
args, but load_ofi_features() was only using the derived sibling path.
Now uses the explicit field when available, falling back to derivation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OFI calculator's compute_ofi_from_file() never called feed_trade(),
leaving 3/8 features (VPIN, Kyle's Lambda, trade_imbalance) at zero in
production. Added compute_ofi_with_trades() that interleaves trades by
timestamp into the MBP-10 streaming callback. Updated DQN and PPO
hyperopt adapters to derive trades_dir as sibling of dbn_data_dir.
Smoke test validates: without trades VPIN=0/5000, with trades VPIN=5000/5000.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
