Move 53 DQN source files + gpu_replay_buffer from ml into standalone
ml-dqn crate. The ml crate's dqn module is now a thin re-export layer
(`pub use ml_dqn::*`) plus two bridge files (trainable_adapter.rs,
stress_testing.rs) that depend on ml-internal types.
Key changes:
- ml-dqn: 45 modules, 334 tests, compiles standalone
- ml: depends on ml-dqn, re-exports via dqn/mod.rs
- DQN.config: pub(crate) → pub for cross-crate access
- DQNAgent checkpoint methods moved into ml-dqn
- gpu_replay_buffer moved from cuda_pipeline/ to ml-dqn
- 8 dead-code files removed (never declared in mod.rs)
Net: -30,648 lines from ml crate. Workspace: 0 errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Sharpe ratio metrics and SIMD performance module to ml-core.
These only depend on MLError which is already in ml-core.
Add approx = "0.5" to ml-core dev-dependencies for Sharpe tests.
Skip observability/ (needs prometheus dep — stays in ml).
Test counts: 271 ml-core + 2487 ml = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 6 shared modules (~2.3K lines) from ml to ml-core:
- trading_action.rs (TradingAction enum)
- action_space.rs (action masking, re-exports)
- xavier_init.rs (Xavier/Glorot weight initialization)
- mixed_precision.rs (AMP utilities, FP16/BF16)
- order_router.rs (deterministic order routing)
- portfolio_tracker.rs (portfolio state tracking)
With TradingAction now in ml-core alongside FactoredAction, the
FactoredActionExt extension trait is eliminated entirely —
from_trading_action()/to_trading_action() become inherent methods
on FactoredAction. This removes the need for `use FactoredActionExt`
imports in reward.rs, gae.rs, and trajectories.rs.
Test counts: 250 ml-core + 2508 ml = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move memory_optimization/ (4.6K lines, 7 files) from ml to ml-core
- Move batch_size_resolver.rs to ml-core (now both deps in same crate)
- Add tempfile to ml-core dev-dependencies (qat tests)
- Re-export both modules from ml facade
- Keep checkpoint/ in ml (model_implementations.rs has model-specific deps)
185 ml-core tests + 2573 ml tests = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move safety/ module (6.5K lines) from ml to ml-core (self-contained)
- Keep security/ in ml (depends on ensemble::EnsembleDecision)
- Move From<ProductionTrainingError> for MLSafetyError to ml (cross-crate)
- Rename Real-prefixed inference types to proper names:
RealInferenceError → InferenceError
RealInferenceConfig → InferenceConfig
RealPredictionResult → InferencePrediction
RealNeuralNetwork → NeuralNetwork
RealMLInferenceEngine → MLInferenceEngine
- Re-export safety from ml facade for backward compatibility
129 ml-core tests + 2629 ml tests = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move optimizers, gradient_accumulation, gradient_utils, cuda_compat,
tensor_ops, and gpu (device config, capabilities, memory profiling) to
ml-core. These are shared compute primitives used by all models.
Also commit module files for core types (common, config, error, model,
traits, types) that were moved from ml to ml-core in task 5a but left
staged without being committed.
Notable changes:
- resolve_batch_size() stays in ml (new batch_size_resolver module)
because it depends on memory_optimization::auto_batch_size which
has not yet moved to ml-core
- FactoredAction legacy bridge converted from inherent impl to
extension trait (FactoredActionLegacy) since FactoredAction is now
defined in ml-core, not ml
- candle-optimisers added to ml-core dependencies (needed by Adam)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all inline type definitions from ml/src/lib.rs to ml-core:
MLError, MLResult, Trade, MarketRegime, HealthStatus, Features,
MLModel trait, ModelRegistry, ParallelExecutor, LatencyOptimizer,
TrainingMetrics, ValidationMetrics, InferenceResult, ModelMetadata.
Dedup: consolidate ConfigError{reason}/ConfigurationError(msg) into
single ConfigError(String) tuple variant (was 2 variants, 174 refs).
Cleanup: convert create_hft_* free functions to associated methods
(HFTPerformanceProfile::ultra_low_latency(), ParallelExecutor::hft()).
ml facade re-exports via `pub use ml_core::*`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move mixed_precision, xavier_init, portfolio_tracker, action_space,
order_router from dqn/ to ml/src/ root. Extract TradingAction enum
from dqn/agent.rs to standalone trading_action module. Re-export
from dqn/mod.rs for backward compatibility.
These modules are shared infrastructure used by PPO, trainers,
hyperopt, and ensemble — not DQN-specific.
Test results: 2757 passed, 0 failed, 25 ignored (unchanged).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 5 empty sub-crates (ml-core, ml-dqn, ml-ppo, ml-supervised, ml-infra)
to workspace. Modules will be moved from monolithic ml crate in subsequent
tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split ml-rl into ml-dqn and ml-ppo for 3-way parallel compilation
and better sccache hit rate. Extract TradingAction to ml-core to
enable full DQN/PPO independence.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the monolithic ml crate (260K lines, 55s compile) into 5 crates:
ml-core, ml-rl, ml-supervised, ml-infra, ml (facade).
15-task plan with full module inventory and import migration guide.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three bugs in the GPU experience collection hot path:
1. gpu_batch_to_experiences() had hardcoded state_dim=43 but the CUDA kernel
outputs states at the ALIGNED dimension (56 with OFI, 48 without). After
sample 0, every replay buffer entry had corrupted state vectors — the
network was learning from garbage data.
2. GPU path never called monitor.track_reward(), so mean_reward was always
reported as 0.0 in epoch logs despite the agent generating real rewards.
3. Action tracking was double-counted (direct array write + track_action_by_exposure),
inflating diversity metrics by 2x. Consolidated into single bounded call.
Also adds missing app.kubernetes.io/component label to job-template.yaml
so Prometheus training-pods scrape job discovers training pods.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PSO can now tune sharpe_weight in [0.0, 0.5] instead of hardcoded 0.3.
This lets hyperopt discover the optimal Sharpe ratio blending weight
in the composite reward signal per-symbol.
Also updates docstring to reflect current C1-C4 search space state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU experience kernel had hardcoded STATE_DIM=54, MARKET_DIM=51,
NUM_ATOMS_MAX=51 but the host-side feature buffer uploads 40 features
per bar and networks use state_dim=56 with up to 100 atoms. Past ~702K
bars the stride mismatch caused out-of-bounds GPU reads → ILLEGAL_ADDRESS.
All dimension constants now use #ifndef guards so the NVRTC JIT compiler
receives the actual values via #define injection — zero runtime overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt varies hidden_dim_base (256–1024) but the CUDA experience
kernel hardcoded SHARED_H1=256, SHARED_H2=256. When hyperopt picked
larger dims, the kernel did matvec with wrong strides → illegal memory
access → training crash on first epoch.
Fix: wrap CUDA #defines in #ifndef guards and inject actual dims from
the dueling network config at NVRTC compile time. The kernel is now
specialized per-trial with the exact weight matrix dimensions — zero
runtime overhead, no dimension mismatch possible.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
from_dqn_params() was stripping the last hidden_dim, creating only 1
shared layer (shared_0) when hidden_dims=[256,256]. The CUDA experience
kernel hardcodes 2 shared layers and gpu_weights.rs expects shared_1.*
weight keys. This caused "Missing weight: shared_1.weight" every epoch,
forcing CPU fallback for all experience collection on H100.
Fix: use all hidden_dims as shared layers. Default [256,256] now creates
shared_0 + shared_1, matching the CUDA kernel architecture exactly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The DQN hyperopt adapter has an explicit trades_data_dir field from CLI
args, but load_ofi_features() was only using the derived sibling path.
Now uses the explicit field when available, falling back to derivation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OFI calculator's compute_ofi_from_file() never called feed_trade(),
leaving 3/8 features (VPIN, Kyle's Lambda, trade_imbalance) at zero in
production. Added compute_ofi_with_trades() that interleaves trades by
timestamp into the MBP-10 streaming callback. Updated DQN and PPO
hyperopt adapters to derive trades_dir as sibling of dbn_data_dir.
Smoke test validates: without trades VPIN=0/5000, with trades VPIN=5000/5000.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests require a CUDA GPU and are #[ignore]d by default. Run with:
cargo test -p ml --test gpu_kernel_parity_test -- --ignored
Covers:
- Standard dueling forward: finite Q-values, valid action range [0,4]
- C51 distributional forward: Q-values bounded by atom support [-25,25]
- Candle vs kernel Q-value parity: argmax action consistency
- NoisyNet exploration: noise injection produces action diversity
- Weight extraction roundtrip: VarMap → CudaSlice → sync
- Distributional weight shapes: value_out [51,128], advantage_out [255,128]
- RMSNorm gamma extraction: initialized to 1.0
- Repeated kernel launches: 5 consecutive runs all produce finite output
All 8 tests pass on RTX 3050 Ti (4.49s total).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Port factorized Gaussian noise exploration and 51-atom categorical value
distribution to the CUDA experience collection kernel, fixing a correctness
bug where GPU Q-values were wrong when use_distributional=true (production
default) due to misinterpreted distributional weight shapes.
- D5: NoisyNet factorized noise (Box-Muller + f(x)=sign(x)*sqrt(|x|)) on
all 6 dueling layers, online network only — target stays deterministic
- D6: C51 distributional dueling forward with per-action atom softmax,
RMSNorm after shared/value/advantage layers, correct [51,128]/[255,128]
weight interpretation
- RmsNormWeightSet extraction and post-epoch sync (GPU-to-GPU)
- Fix get_effective_epsilon() to report actual 2% noisy floor instead of 0.0
- Proportional diversity entropy penalty (continuous gradient vs cliff)
- 2758 tests passing, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU experience collector was falling back to CPU because it required a
curiosity VarMap, but curiosity_weight=0.0 means the module is never
created. Fix: make curiosity optional (CuriosityWeightSet::zeros() for
GPU buffers, curiosity_scale=0.0 in kernel config). Also require
explicit binary-tag SHA in Argo training workflow (no "latest" fallback).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The walk-forward backtest in the DQN hyperopt adapter constructed batch
tensors with raw state_dim (51/43) but the model expects aligned
state_dim (56/48). This caused shape mismatch errors during backtest
evaluation: matmul [1024, 51] vs [56, 1024].
Fix: use align_dim_for_tensor_cores() and zero-pad each state vector
before tensor construction, matching the same pattern used in
compute_loss_internal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Additional padding in get_q_values and convert_to_state — these are
currently unused in the training pipeline but would crash if called.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Validation, action selection, and Q-value monitoring paths were using
raw state dimensions (51) while the model expected aligned dims (56).
Training epoch 1 passed because the GPU pipeline pads correctly, but
validation crashed: shape mismatch [1000,51] vs [56,1024].
Fixed: validation batch, select_actions_batch CPU fallback,
estimate_avg_q_value, compute_q_gap — all now zero-pad to aligned dim.
Also fixed model size estimation log to show aligned dims.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add QuestDB ILP sink for training metrics, update Prometheus scrape
configs, and fix network policies for monitoring stack connectivity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move BF16 tensor core alignment from per-forward-pass allocation to
data pipeline boundaries. On H100, state_dim 43→48 and 51→56 (8-aligned)
so cuBLAS dispatches HMMA instructions instead of falling back to scalar FMA.
Architecture:
- Trainer computes aligned state_dim at source (align_dim_for_tensor_cores)
- GPU path: DqnGpuData.pad_state_tensor() pads once at upload boundary
- CPU path: train_batch() fold zero-pads Experience.state vectors
- Networks receive pre-aligned tensors — zero per-step overhead
All state_dim defaults updated to aligned values (43→48, 51→56).
Removed pad_to_aligned() from all network forward() methods.
2758 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
cuBLAS requires M/N/K dimensions to be multiples of 8 for BF16 tensor
core dispatch. state_dim=51 (with OFI) caused silent fallback to scalar
FMA ops, leaving tensor cores at 0.1% utilization despite BF16 enabled.
Pad state_dim to next 8-multiple (51→56, 43→48) at network construction
and zero-pad input tensors in forward pass across all DQN network types:
Sequential, DuelingQNetwork, DistributionalDuelingQNetwork, NetworkLayers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix NaN detection: replace non-existent isnan() with ne(&self) (NaN≠NaN)
- C51 gradient strip: copy().detach() replaces to_vec2→from_vec GPU→CPU→GPU roundtrip
- GPU batch fast path: skip CPU fold + 5× from_vec when GpuBatch available
- Action validation: GPU clamp() replaces CPU loop in GPU batch path
- PER TD errors: keep as GPU Tensor when GpuPrioritized replay active
- PER indices: use GpuBatch.indices directly instead of CPU Vec→Tensor
- IS weights: cache GPU tensor from GpuBatch, reuse in both loss paths
- Logging: Q-values 10→500 steps, diagnostics 100→1000 steps
2758 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both adapters now call load_ofi_features_parallel from mbp10_loader
instead of duplicating the rayon + streaming logic inline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three fixes:
- Streaming MBP-10 parser (parse_mbp10_streaming): computes OFI inline
during decode — eliminates Vec<Mbp10Snapshot> allocation (41.9M clones)
- Parallel file processing: rayon par_iter across 9 MBP-10 files
(778s sequential → ~90s expected on H100 24-core)
- Fix hardcoded state_dim=54 in walk-forward backtest tensor creation
that caused panic "range end index 55296 out of range for slice of
length 52224" — now uses dynamic state_dim (43 or 51 with OFI)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New compile-and-train-template.yaml: compile + GPU warmup in parallel,
then fetch-binary → hyperopt → train-best → evaluate → upload-results
- Compile step outputs SHA tag via Argo output parameter, fetch-binary
uses it directly (no 'latest' package indirection)
- Remove trading-service from CI deploy step — must be explicitly enabled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: preload_data() called loader.ofi_features.take() on the internal
DQN trainer, but load_training_data() only loads OHLCV bars — it never
populates ofi_features. The OFI loading is done by the hyperopt adapter's
own load_ofi_features() method.
Fixes:
- preload_data() now calls self.load_ofi_features() directly
- load_ofi_features() uses self.mbp10_data_dir when set (was hardcoded ../mbp10)
- input_dim pre-computation uses OFI-aware size (51 when enabled, 43 otherwise)
- CI 'latest' package: delete-then-upload to avoid GitLab duplicate file issue
- Warn when mbp10_data_dir is set but no OFI features loaded
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename node-dns-fix → node-bootstrap, fix nvidia gate race condition
(always create conf.d/ and write 50-registry.toml, don't gate on
99-nvidia.toml which doesn't exist on fresh autoscaled GPU nodes)
- Update busybox 1.36 → 1.37
- Fix fetch-binary: use PRIVATE-TOKEN/gitlab-pat (not DEPLOY-TOKEN)
- Fix upload-results: use gitlab-pat (gitlab-ci-token didn't exist)
- Add 'latest' rolling package version in both compile-services and
compile-training (re-upload after CalVer upload)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Service pods' initContainers need egress to gitlab-webservice-default:8181
to fetch release binaries from the Generic Package Registry. Without this,
curl hangs for 2+ minutes then times out (default-deny-all blocks it).
Covers all app.kubernetes.io/part-of=foxhunt pods in one policy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Argo Workflows can't resolve {{tasks.X.outputs}} inside template
bodies — only in DAG task arguments. Pass tag via arguments →
inputs.parameters in compile-services, compile-training,
upload-release, and deploy-services templates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The hyperopt preload_data() created a loader with default hyperparams
(mbp10_data_dir=None), so MBP-10/trades data was never loaded. Each
trial then set mbp10_data_dir → state_dim=51, but the preloaded data
had no OFI features → shape mismatch [128,43] vs [51,1024].
- Pass mbp10_data_dir/trades_data_dir to preload hyperparams
- Extract ofi_features from loader after preload
- Store as preloaded_ofi_features: Option<Arc<Vec<[f64;8]>>>
- Inject into each trial's DQNTrainer before training
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace MinIO binary distribution with GitLab Generic Package Registry.
Every code push to main auto-creates a CalVer tag (vYYYY.MM.N),
compiles, uploads binaries to GitLab packages, creates a Release
with auto-generated notes, and deploys via deployment patching.
- New CI templates: create-tag, upload-release
- Modified: compile-services/training upload to GitLab packages
- Modified: deploy-services patches FOXHUNT_RELEASE on deployments
- All 7 service initContainers fetch from GitLab (curl, deploy token)
- Training job-template binary fetch from GitLab (data stays MinIO)
- MinIO retains: sccache, training data, model checkpoints
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU path (build_batch_states / build_state_tensor) only concatenated
market[40] + portfolio[3] = 43 dims, while state_dim was set to 51 when
OFI was enabled. This caused shape mismatch [128,43] vs [51,1024].
- Add ofi_features: Option<Tensor> field to DqnGpuData
- Add upload_ofi() method for MBP-10 order book features (8 dims/bar)
- Concatenate OFI in build_batch_states: [count,40]+[count,3]+[count,8]=[count,51]
- Concatenate OFI in build_state_tensor: [1,40]+[1,3]+[1,8]=[1,51]
- Wire trainer to call upload_ofi() after GPU data upload
- Fix CPU path to zero-pad regime_features when OFI enabled but data missing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The training-data PVC already has all data (OHLCV, MBP-10, trades).
Mount it read-only at /data in hyperopt/train/evaluate steps instead
of rcloning ~50GB from MinIO every step.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CI pipeline: add gpu-warmup step parallel to compile-training,
triggers GPU node autoscale so it's ready when training starts
- Training workflow: add mbp10-data-dir and trades-data-dir parameters,
download MBP-10 + trades data in all steps (hyperopt, train, evaluate)
- Use parameterized paths consistently (no hardcoded /tmp paths)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add --mbp10-data-dir and --trades-data-dir CLI args to
hyperopt_baseline_rl binary so hyperopt trials can use real
order book and trade data for VPIN/Kyle's Lambda features.
- DQNTrainer: add mbp10_data_dir/trades_data_dir fields + with_ofi_data_dirs() builder
- DQNHyperparameters: pipe through from trainer instead of hardcoded None
- download-trades-job: fix nodeSelector to ci-compile-cpu (platform pool full)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Expand training-data-pvc from 10Gi → 200Gi (OHLCV + MBP-10 + headroom)
- Add data-sync-job: rclone sync from MinIO → PVC (delta-only, fast repeats)
- Add sync-training-data init container to training job template
(auto-syncs data from MinIO before each training run)
- Add training-job + data-sync-job NetworkPolicies (MinIO, Tempo, Pushgateway)
- Add app.kubernetes.io/part-of: foxhunt to training pod template
- download_baseline: add --parallel N flag for concurrent Databento downloads
- download_baseline: delete local files after MinIO upload (saves scratch space)
- Bump download job scratch volume to 100Gi
Architecture: Databento → MinIO (source of truth) → PVC (local cache).
First sync pulls everything; subsequent runs only sync deltas (seconds).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The download job pod was missing app.kubernetes.io/part-of: foxhunt,
so the default-deny-all egress policy blocked it and MinIO's ingress
policy rejected it. Also removed hostname pinning (was a misdiagnosis).
Added data-download-job NetworkPolicy allowing egress to MinIO:9000
and external HTTPS:443 (Databento API).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire 8 OFI features (OFI L1/L5, depth imbalance, VPIN, Kyle's lambda,
bid/ask slopes, trade imbalance) through the DQN training pipeline:
- Add mbp10_data_dir config field to DQNHyperparameters
- Dynamic state_dim: 43 (no OFI) or 51 (with OFI) based on config
- Compute OFI per bar during data loading, store on trainer
- Pass OFI features through regime_features slot in TradingState
- Configurable MBP-10 path with recursive .dbn/.dbn.zst discovery
- Add zstd auto-detection to DbnParser::parse_mbp10_file()
- Add --mbp10-data-dir CLI flag to train_baseline_rl
- Fix hardcoded [f64; 51] → FeatureVector51 ([f64; 40]) across
examples, walk_forward, GPU memory profile, and test fixtures
- Fix stale state_dim=51 in dqn_config_2025() and DQN tests
2747 tests pass, 0 failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- download_baseline now reads schema from universe TOML config
(was hardcoded to ohlcv-1m, now supports mbp-10 and other schemas)
- Add --rclone-dest flag for direct upload to MinIO after each download
- Add config/universe-es-mbp10.toml: ES.FUT MBP-10, same date range
- Add infra/k8s/training/download-mbp10-job.yaml: K8s Job to download
ES.FUT MBP-10 data in-cluster and upload to MinIO
Usage:
kubectl apply -f infra/k8s/training/download-mbp10-job.yaml
Prereqs: databento-credentials secret, download_baseline binary in MinIO
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
