Commit Graph

17 Commits

Author SHA1 Message Date
jgrusewski
09c515e3e9 fix(clippy): ZERO errors across entire workspace — CI ready
Final 31 ml crate fixes: unsafe_code allows, unused vars prefixed,
boolean simplification, dead code removal, integer suffix, drop cleanup.

cargo fix auto-removed ~30 unused imports from ml crate.

Total clippy cleanup: 278 errors → 0 across all ML crates.
Full workspace: `cargo clippy --workspace --lib -- -D warnings` = 0 errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 01:04:09 +01:00
jgrusewski
daf771c38d audit: annotate all remaining to_vec/memcpy_dtoh — categorized 158 sites
Every to_vec()/memcpy_dtoh across 48 files audited and annotated:
- ~100 false positives: Rust slice .to_vec() (cpu-side, never touches GPU)
- ~25 gpu-exit: legitimate scalar readbacks (loss, grad_norm, epoch state)
- ~20 test-only readbacks: gated by #[cfg(test)] scope
- ~10 cpu-side uploads: .to_vec() before from_vec() GPU upload
- ~3 checkpoint exports: export_to_host at epoch boundary

Annotations use inline comments: // cpu-side, // gpu-exit:, // test-only

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 19:36:41 +01:00
jgrusewski
42e6e1053f fix(ml): validation, inference, hyperopt, PPO — GPU-native APIs
- validation/ppo_adapter.rs: complete rewrite, host-side PPO validation
- inference.rs: Arc<CudaStream> + CudaBlas + ActivationKernels
- hyperopt/adapters: 8 adapters migrated to UnifiedTrainable forward_loss
- trainers/ppo.rs: GPU collector VarStore, checkpoint, weight sync
- ppo/trainable_adapter.rs: PPO::new() constructor
- trainers/tft/trainer.rs: GpuTensor↔StreamTensor conversion helpers

334→65 errors (81% reduction). Remaining: StreamTensor vs GpuTensor
type mismatches in ensemble/trainable adapters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:48:09 +01:00
jgrusewski
c89a1dbf29 fix(ml): migrate remaining files — hyperopt, validation, data_loaders, inference
25+ files fixed:
- hyperopt/adapters: duplicate Arc imports, Device→MlDevice
- validation: regime_analysis rewritten for GpuTensor, ppo_adapter NativeDType
- data_loaders: all 3 loaders migrated to GpuTensor::from_host
- tft/training: MlDevice, GpuAdamW, StreamTensor, CPU loss tracking
- training_pipeline: AdamW→GpuAdamW, NativeDevice fixes
- features/multi_timeframe: removed to_candle_tensor
- inference: ModelForward trait replaces candle_nn::Module
- lib.rs: removed cuda_compat re-export
- ppo/mod.rs: fixed re-export

~95 errors remain in: inference.rs, flash_attention, validation/ppo_adapter,
hyperopt adapter method signatures (blocked on trainable adapter trait).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 09:57:24 +01:00
jgrusewski
0d62bdba2e refactor(ml): eliminate candle from all 104 src files
Zero candle_core/candle_nn imports in ml/src/. Three-agent parallel migration:

- cuda_pipeline/ (19 files): Tensor→CudaSlice, Device→Arc<CudaStream>,
  VarMap→GpuVarStore, cudarc import path fixed
- trainers/ + adapters (45 files): DQN/PPO/TFT trainers, 10 ensemble
  adapters, 11 hyperopt adapters — all migrated to MlDevice, GpuTensor,
  GpuVarStore, GpuAdamW
- model dirs + infra (40 files): 10 trainable adapters, preprocessing,
  inference, transformers, validation, benchmarks

61 test/example files still reference candle — next commit.
candle-nn still in Cargo.toml (needed by tests until migrated).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 00:24:35 +01:00
jgrusewski
6efb78ba9c feat(cuda): pure-CUDA backtest forward, eliminate Candle dispatch in hyperopt DQN
Replace closure-based evaluate() with evaluate_dqn_graphed() for non-OFI
walk-forward backtest path. Extracts DuelingWeightSet from VarMap (branching
or standard dueling) and runs hand-written warp-cooperative CUDA forward
kernel with CUDA Graph capture — zero Candle dispatch overhead per step.

Key changes:
- GpuBacktestEvaluator::stream() getter for weight extraction on eval stream
- DQNAgentType::is_using_branching() / network_dims() for CUDA kernel config
- Hyperopt evaluate_gpu() non-OFI path: extract_dueling_weights_branching()
  → evaluate_dqn_graphed() (CUDA Graph accelerated)
- OFI path: retains Candle closure for state permutation (gather kernel
  layout mismatch — future CUDA permutation kernel)
- 66+ GPU hot-path violations hardened to hard errors across DQN/PPO/supervised
- Stripped all gpu-ok suppression comments
- Proper #[cfg(feature = "cuda")] gating for CUDA-only code paths

77 files, 0 errors, 0 warnings across workspace.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 02:49:25 +01:00
jgrusewski
e0e1e2fff3 fix(cuda): eliminate CPU fallbacks in all 10 supervised/RL model adapters
Convert Device::Cpu fallback patterns to hard errors across all
hyperopt adapters and the Mamba2 trainer. On H100, CUDA must be
available — silent CPU fallback runs at 1/10th throughput.

Models hardened: TFT, Mamba2, TGGN, TLOB, Liquid, KAN, xLSTM,
Diffusion, ContinuousPPO (hyperopt adapters) + Mamba2 (trainer).

Pattern: Device::new_cuda(0).unwrap_or_else(|e| { warn!(...); Cpu })
      →  Device::new_cuda(0).map_err(|e| MLError::ConfigError(...))?

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 00:23:19 +01:00
jgrusewski
4709ca8bc2 feat(dqn): enable Branching DQN with 45 factored actions (5×3×3)
Restore 45-action factored space via Branching DQN (Tavakoli 2018),
outputting 11 Q-values (5+3+3) instead of 45. This was reduced to 5
exposure-only actions during debugging and was never intended as permanent.

- Enable use_branching: true by default in DQNConfig and DQNHyperparameters
- Add branching paths to select_action_with_confidence and select_action_inference
- Update agent.rs select_action_factored for branching-aware selection
- Expand CountBonus to per-branch tracking with bonuses_branched()
- Add order_type + urgency distribution tracking in monitoring
- Add DQN_ORDER_ACTIONS=3, DQN_URGENCY_ACTIONS=3, DQN_TOTAL_ACTIONS=45 to CUDA header
- Fix 7 pre-existing clippy doc_markdown errors in regime_conditional.rs
- Fix pre-existing cognitive_complexity in replay_buffer_type.rs (extract helpers)
- Fix flaky GPU test OOM under parallel execution (CPU fallback + test VRAM safety)
- Delete unused flash_attention submodules (block_sparse, causal_masking, etc.)
- Add GPU hot-path guard scripts and ensemble/hyperopt adapter improvements

Tests: ml-dqn 416/0, ml 905/0, clippy 0 errors on both crates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:00:13 +01:00
jgrusewski
e690b1c60e feat(hyperopt): add signal threshold params + backtest fitness to all 8 supervised adapters
All supervised hyperopt adapters (TFT, Mamba2, Liquid, KAN, xLSTM, TGGN,
TLOB, Diffusion) now include:
- signal_high_bps and signal_low_bps in ParameterSpace (tunable thresholds)
- backtest_sharpe and backtest_trades fields in Metrics structs
- extract_objective prefers GPU backtest fitness over val_loss when available
- All tests updated (101 passing)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 15:59:38 +01:00
jgrusewski
a5ea8713b6 feat(dqn): Branching DQN + KernelWeightPack + metrics propagation
Branching DQN (Tavakoli et al., 2018):
- 3 independent advantage heads (exposure=5, order=3, urgency=3) in CUDA kernel
- `use_branching` as hyperopt-tunable parameter (index 11 in 29D space)
- Branch weight pointers packed into KernelWeightPack struct

KernelWeightPack refactor:
- 87→38 CUDA kernel params by packing 48 weight pointers into 384-byte #[repr(C)] struct
- UNPACK_WEIGHT_PTRS macro in CUDA header for clean kernel-side access
- Single .arg(&weight_pack) replaces 48 individual .arg() calls

Hyperopt metrics in JSON output:
- Added `metrics: Option<serde_json::Value>` to TrialResult<P>
- `extract_metrics()` trait method with default None (DQN overrides)
- JSON output now includes per-trial backtest metrics (Sharpe, Sortino,
  Calmar, Omega, drawdown, win_rate, trades) + top-level best_metrics

Prometheus backtest gauges (Rust-side, no CUDA):
- 8 new gauges: foxhunt_hyperopt_best_{sharpe,sortino,calmar,omega,
  max_drawdown_pct,win_rate,total_trades,total_return_pct}

Dynamic episode scaling:
- Removed hardcoded 256 episode cap on small GPUs
- VRAM budget calculation in optimal_n_episodes() handles scaling naturally
- Removed stale .min(256) in PPO trainer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 00:46:53 +01:00
jgrusewski
7d1b0f232f refactor(ml): move core types to ml-core + dedup MLError (5a)
Move all inline type definitions from ml/src/lib.rs to ml-core:
MLError, MLResult, Trade, MarketRegime, HealthStatus, Features,
MLModel trait, ModelRegistry, ParallelExecutor, LatencyOptimizer,
TrainingMetrics, ValidationMetrics, InferenceResult, ModelMetadata.

Dedup: consolidate ConfigError{reason}/ConfigurationError(msg) into
single ConfigError(String) tuple variant (was 2 variants, 174 refs).

Cleanup: convert create_hft_* free functions to associated methods
(HFTPerformanceProfile::ultra_low_latency(), ParallelExecutor::hft()).

ml facade re-exports via `pub use ml_core::*`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:13:41 +01:00
jgrusewski
2aedc2ae1a feat(ml): comprehensive GPU saturation audit — 58 fixes across all 10 models
Phase 1 — Fix broken models (P0):
- Diffusion: wire optimizer_step to actually apply gradients (was no-op)
- TLOB: connect forward pass to projection layers (was Tensor::zeros)
- Mamba2: F64→F32 migration across 5 files (~30x faster on L40S tensor cores)

Phase 2 — Eliminate hot-path GPU sync stalls:
- Mamba2: keep dt on GPU in discretize_ssm (4 functions, no CPU round-trip)
- TFT: gate attention weight logging to eval only (8 syncs/forward eliminated)
- Mamba2: defer loss scalar after backward (pipeline stall removed)
- Mamba2: delete dead gradient clipping (4N wasted GPU syncs removed)

Phase 3 — Enable BF16 for supervised models:
- Flip mixed_precision defaults to true in 4 config locations
- Fix cuda_layer_norm to support BF16/F16 via F32 intermediate

Phase 4 — Raise hyperopt bounds for datacenter GPUs:
- 7 adapters with VRAM-aware tiers (TFT, Liquid, TGGN, KAN, xLSTM,
  Diffusion, TLOB) — L40S gets full hidden_dim range
- Fix L40S tier boundary (was excluded at <48000, now >=40000)

Phase 5 — Update memory estimates:
- 10 param_count estimates updated (DQN 200K→12M, TFT 2M→50M, etc.)
- Fix power-of-two rounding (was wasting up to 49% of budget)
- Correct MODEL_OVERHEAD_MB in DQN/PPO/TFT adapters

Phase 6 — Fix per-epoch CPU bottlenecks:
- PPO: deduplicate double advantage normalization (correctness fix)
- PPO: GPU tensor reward normalization + explained variance
- Fuse per-parameter grad norm to single GPU sync (xLSTM, KAN, TGGN)

Phase 7 — Data pipeline:
- GpuBufferPool: use from_slice (eliminate staging buffer copy)

Phase 8 — Correctness:
- TFT: remove broken .detach() in forward_checkpointed (restore gradients)
- Update stale RTX 3050 Ti doc references

33 files changed, 2451 tests pass, 0 clippy warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 15:41:23 +01:00
jgrusewski
4a3f387860 fix(ml): Arc<Vec<OHLCVBar>> → Arc<[OHLCVBar]> in hyperopt adapters
Fixes clippy::rc_buffer lint in 8 hyperopt adapter files.
Arc<[T]> avoids double indirection vs Arc<Vec<T>>.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 20:59:21 +01:00
jgrusewski
2b2ff4ffa5 feat(ml): maximize GPU utilization — BF16 mixed precision, dynamic sizing, sync reduction
Wire BF16/FP16 mixed precision end-to-end for DQN and PPO with auto-detection
from GPU name (Ampere+ → BF16, Volta/Turing → FP16). Add hidden_dim_base to
hyperopt and wire through training/eval binaries. Reduce GPU sync points: make
DQN NaN checks periodic (every 100 steps), replace PPO GAE GPU round-trip with
pure CPU implementation. Cache training data across hyperopt trials for all 10
models via Arc. Batch DQN experience storage (128x fewer lock acquisitions).
Correct VRAM constants and batch bounds for all 9 supervised model adapters.

28 files changed, +1207/-208 lines. 2418 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 17:54:25 +01:00
jgrusewski
b4239e8c43 feat(hyperopt): add VRAM-aware batch_size scaling to 9 existing adapters
Override continuous_bounds_for() in all adapters that already have batch_size:
- TFT (idx 1): 150MB overhead, 0.05 MB/sample, cap 2048
- Mamba2 (idx 1): 200MB overhead, 0.03 MB/sample, cap 2048
- TGGN (idx 7): 100MB overhead, 0.08 MB/sample, cap 1024
- TLOB (idx 6): 120MB overhead, 0.10 MB/sample, cap 1024
- KAN (idx 7): 80MB overhead, 0.04 MB/sample, cap 1024
- xLSTM (idx 6): 180MB overhead, 0.06 MB/sample, cap 1024
- Diffusion (idx 6): 250MB overhead, 0.12 MB/sample, cap 512
- DQN (idx 1): 300MB overhead, 0.02 MB/sample, cap 4096
- ContinuousPPO (idx 9): 250MB overhead, 0.03 MB/sample, cap 4096

On CPU-only, all adapters fall back to existing static bounds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 19:19:25 +01:00
jgrusewski
b09ebfc3cb feat(ml): add HyperparameterOptimizable trainers for all 8 supervised models
Extend hyperopt infrastructure to support all 10 ML models (DQN, PPO +
8 supervised). Previously only TFT and Mamba2 had hyperopt trainers.

- Add HyperparameterOptimizable impl for Liquid, TGGN, TLOB, KAN, xLSTM, Diffusion
- Create shared_data.rs with common data prep utilities (build_flat_pairs,
  build_sequence_pairs, write_trial_result_json)
- Extend hyperopt_baseline_supervised binary to dispatch all 8 models
  (individual, "both" for tft+mamba2, "all" for all 8)
- Add CI jobs: 7 train-validate + 10 hyperopt jobs for all models
- Fix DiffusionMetrics NaN default, XLSTMMetrics serde, safe indexing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 23:47:52 +01:00
jgrusewski
9c3d741a08 refactor: restructure repo — crates/, bin/, testing/ layout
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.

Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 11:56:00 +01:00