Commit Graph

7 Commits

Author SHA1 Message Date
jgrusewski
533249eb91 fix(ml): wire spread_cost_bps into RL training — commission + spread = total tx cost
Previously train_baseline_rl.rs only passed commission (tx_cost_bps) to
DQN/PPO trainers, ignoring bid-ask spread slippage. Now computes per-fold
average spread via spread_cost_bps() (same as evaluate_baseline) and passes
total cost (commission + spread) to both trainers.

Removes #[allow(dead_code)] — function is now used by all 4 example binaries.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 22:10:43 +01:00
jgrusewski
f975c9cf71 fix(ml): suppress dead_code warning on spread_cost_bps (used by other examples)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 22:02:30 +01:00
jgrusewski
afd85b2f8f chore: clean up examples, update ML binaries and risk tests
- Delete 14 unused example files (-3,543 lines): config, adaptive-strategy,
  data, storage, trading_engine, api_gateway, backtesting, trading_service, chaos
- Update ML training/eval binaries: improved CLI args, completion tracking,
  CUDA test cleanup, hyperopt enhancements
- Fix KAN network and TFT module adjustments
- Update risk test assertions for consistency
- Fix backtesting repositories and promotion manager
- Update .serena project config and Cargo dependencies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 01:33:18 +01:00
jgrusewski
265bd2441c fix(ml,ci): zero-dim guards on all 10 models, eliminate warnings, unblock CI parallelism
- Add dimension validation in DQN, PPO, Mamba2, TGGN, TLOB, Liquid,
  KAN, xLSTM, Diffusion constructors (fail-fast on zero-dim inputs
  that would cause CUDA_ERROR_INVALID_VALUE at runtime)
- Add num_unknown_features > 0 guard to TFT (temporal input required)
- Fix 12 dead-code/unused warnings in test compilation
- Remove opt-level=3 and codegen-units=1 from target rustflags
  (was forcing O3 + single-thread codegen on dev/test builds)
- Remove hardcoded jobs=16 cap (cargo now auto-detects CPU count)
- Switch linker to clang+lld (2-5x faster linking)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 17:38:43 +01:00
jgrusewski
0f9d756caa feat: on-demand training dispatch via K8s Jobs with sidecar uploader
Extend ml_training_service to dispatch GPU training jobs as K8s batch/v1
Jobs, collect results via a Rust sidecar uploader, and support model
promotion with operator approval via fxt CLI.

- K8s dispatcher creates Jobs on gpu-training pool with native sidecar
- training_uploader crate: watches DONE/FAILED marker, uploads to S3,
  reports completion via ReportJobCompletion gRPC
- PromotionManager compares metrics, queues better models for approval
- 4 new proto RPCs: ReportJobCompletion, ListPendingPromotions,
  ApprovePromotion, RejectPromotion
- fxt commands: train start, model list/approve/reject
- Training binaries write DONE/FAILED markers + metrics.json
- Dockerfile, K8s job template, and CI pipeline updated
- StartTraining gracefully falls back to in-process when outside K8s
- 27 new tests (16 service + 11 promotion), 141 total service tests pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:43:17 +01:00
jgrusewski
6e339316cf feat(ml): add manually-triggered GitLab CI training pipeline
Adds a parent/child GitLab CI pipeline for ML model training:

- Generator script produces per-model hyperopt/train/evaluate jobs
- Parent pipeline (.gitlab-ci-training.yml) with manual trigger
- NFS-backed ReadWriteMany PVC for shared training outputs
- Hyperopt params wired into training binaries (DQN, PPO, TFT, Mamba2)
- Shared DBN loader eliminates duplicate code across hyperopt adapters
- Supervised hyperopt unified to DBN data (was parquet-only)

Pipeline: hyperopt (4 models) → train (10 models) → evaluate ensemble

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 09:04:58 +01:00
jgrusewski
9c3d741a08 refactor: restructure repo — crates/, bin/, testing/ layout
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.

Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 11:56:00 +01:00