Remove dependencies declared in 5 service Cargo.toml files that no
source code in those services references (verified by grepping for
use statements). Reduces dep-graph fan-out and unnecessary recompiles.
services/api/Cargo.toml −16 deps (async-trait, bytes,
const-oid, hdrhistogram,
hex, http-body, hyper,
hyper-util, num-traits,
rust_decimal, tokio-stream,
tower-layer, tower-service,
tracing-subscriber,
trading_engine, zeroize.
+ json feature added to
reqwest since trading_engine
was enabling it transitively)
services/trading_service/Cargo.toml −11 deps
services/backtesting_service/Cargo.toml −17 deps
services/trading_agent_service/Cargo.toml −6 deps
services/ml_training_service/Cargo.toml −4 deps
False positives kept (cargo-machete misses these because they're only
referenced in tonic-generated proto code, not in hand-written src):
- prost (`::prost::Message` derive in build.rs-generated code)
- tonic-prost (`tonic_prost::ProstCodec::default()` in generated tonic
clients/servers)
cargo check --workspace passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migrations:
- 001_trading_events.sql, 003_audit_system.sql: the hard-coded
node_id literals (`trading-node-01`, `audit-node-01`,
`ml-node-01`, `system-node-01`, `change-tracker-01`) are
overridden per-deployment by later migrations rather than read
from the environment. Describe that in the inline comment.
- 004_compliance_views.sql: `generate_compliance_report` is a log
stub — actual report generation is performed by the compliance
service. Say so explicitly.
services:
- ml_training_service/tests/orchestrator_225_features_test.rs: the
empty `#[ignore]`d placeholder for the 225-feature orchestrator
loader has been removed; it held no assertions and only tracked
a TODO (feedback_no_stubs.md).
- trading_agent_service/src/service.rs: portfolio volatility uses
the diagonal-only approximation because cross-asset return
correlations are not maintained in this service. Document that.
- trading_service/src/services/risk.rs: `get_risk_metrics` uses
`calculate_marginal_var` + asset-class fallback; describe why
`calculate_comprehensive_var` is not wired at this boundary.
- trading_service/tests/auth_comprehensive.rs: delete the entire
commented-out legacy BackupCodeValidator test block — the old
`generate_backup_codes` / `store_backup_code` /
`verify_backup_code` surface no longer exists, and MFA
integration tests already cover the new API.
testing:
- harness/grpc_clients.rs: no BacktestingServiceClient proto
exists; reword the stale TODO import line.
- chaos/*: reword the family of "TODO: Implement ..." stubs as
"Currently a no-op / synthetic result" descriptions so readers
know exactly how much of the chaos framework is live.
- compliance_automation_tests.rs: delete the file; it was a giant
/* ... */ block referencing a nonexistent compliance module
and was not wired into any Cargo target.
- framework.rs: describe why `setup()` uses `println!` instead of
`tracing_subscriber` (tracing_subscriber is not a dep of this
integration crate).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove use_double_dqn, use_dueling, use_per, use_branching,
use_distributional, use_noisy_nets, use_huber_loss, and use_cql
from DQNConfig, DQNHyperparameters, and DqnParams structs.
These features are always enabled (Rainbow DQN standard). The boolean
flags were dead code — every constructor set them to true, and the
only code paths that set them to false were in tests that disabled
features for simplicity. With the fields removed, the features are
unconditionally active, eliminating ~490 lines of dead configuration.
Key changes:
- Struct field declarations removed from 3 core config structs
- Conditional branches (if use_X { ... } else { ... }) simplified:
dueling/branching/PER network creation is now unconditional
- Checkpoint metadata hardcodes "true" for backward compatibility
- Hyperopt search space index 11 (use_branching) fixed at 1.0
- TOML/YAML config files cleaned of removed fields
- Tests that toggled these flags updated or rewritten
45 files changed, -487 net lines. Zero new test failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Final cleanup:
- 61 test files + 5 example files: candle imports replaced
- 8 testing/integration files: migrated to cudarc/ml-core types
- 3 services/trading_service test files: migrated
- Root Cargo.toml: candle-core, candle-nn removed from [workspace.dependencies]
- crates/ml/Cargo.toml: candle-nn dependency removed
- testing/e2e/Cargo.toml: candle-core dependency removed
Zero active candle_core/candle_nn/candle_optimisers code references remain.
Zero candle dependency declarations in any Cargo.toml.
Remaining "candle" strings are exclusively in doc comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
containerd on Kapsule nodes uses host DNS which can't resolve
.svc.cluster.local names. Switch all image references from
gitlab-registry.foxhunt.svc.cluster.local:5000 to localhost:30500
(NodePort on the GitLab registry). Also make training runtime image
configurable via TRAINING_RUNTIME_IMAGE env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- JWT issuer now foxhunt-api across all 16 files (services, tests, config, docker-compose)
- Remove serde alias api_gateway_url from FxtConfig (no backwards compat)
- Remove api_gateway CLI alias from e2e orchestrator
- All services must deploy simultaneously for JWT validation to match
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename 7 service binaries from snake_case to kebab-case to match
K8s deployment names. Update Cargo.toml package/bin names, K8s
manifest S3 paths and commands, and cross-crate dependency keys.
- api_gateway → api-gateway
- trading_service → trading-service
- broker_gateway_service → broker-gateway
- ml_training_service → ml-training-service
- backtesting_service → backtesting-service
- trading_agent_service → trading-agent-service
- data_acquisition_service → data-acquisition-service
broker-gateway gets an explicit [lib] name = "broker_gateway_service"
since its new package name maps to broker_gateway (not the original
broker_gateway_service used in source code). All other services map
correctly with Rust's automatic hyphen-to-underscore conversion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Delete per-service proto/ directories. All 7 services now compile
from the single canonical proto/ at workspace root.
- trading_service: 5 protos + ml_training client -> ../../proto/
- ml_training_service: ml_training server + fxt_trading client -> ../../proto/
- broker_gateway_service: broker_gateway -> ../../proto/
- trading_agent_service: trading_agent + ml client -> ../../proto/
- data_acquisition_service: data_acquisition -> ../../proto/
- api_gateway: 8 proto compilations -> ../../proto/
- monitoring_service: keeps local proto (3 training RPCs only, deleted in Task 5)
Added proto/fxt_trading.proto (fat-client proto, package foxhunt.tli)
separate from proto/trading.proto (backend, package trading) since the
API gateway needs both for protocol translation.
Added unimplemented stubs for merged monitoring.proto RPCs:
- trading_service: 3 training RPCs (served by monitoring_service)
- api_gateway monitoring_proxy: 13 system health RPCs (Task 4)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- trade_ml.rs: Replace 3 mock data fallbacks (submit, predictions,
performance) with proper error propagation. Commands now fail
honestly when the API Gateway is unreachable instead of silently
returning fake data. Mark 3 integration tests as #[ignore].
- monitoring_service: Add tonic-health with set_serving for
MonitoringServiceServer. Enables grpc_health_probe readiness checks.
- ml_training_service: Add tonic-health with set_serving for
MlTrainingServiceServer. Wired into both TLS and non-TLS paths.
- data_acquisition_service: Add tonic-health with set_serving for
DataAcquisitionServiceServer.
- ml/cuda_streams: Fix pre-existing unused variable clippy warning.
All 8 services now have standard gRPC health checking enabled.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- ml_training_service: handlers delegate to promotion_manager
- api_gateway: proxy pass-through for both new RPCs
- Proto messages added to service-side ml_training.proto
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unnecessary async from init_observability -- body was fully sync.
Change otlp_endpoint from &str to Option<&str> -- when None, OTLP layer
is skipped (fmt-only mode). Update all 8 service callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create 3 missing service READMEs (api_gateway, data_acquisition_service,
trading_agent_service). Create bin/fxt/README.md. Create
testing/service-integration/README.md. Update existing service and testing
READMEs to standard template. Delete 4 subdirectory READMEs from
testing/integration/ and testing/e2e/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove stale test reports, quick-start guides, benchmark analyses,
profiling reports, and tool artifacts from across the workspace.
Keeps only root README.md per crate/service.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch all MinIO communication from HTTP to HTTPS across the entire stack:
- MinIO deployment: mount TLS secret, tcpSocket probes, HTTPS init-buckets
- 11 service YAMLs: HTTPS rclone endpoint + CA cert volume mount
- Training job template + train.sh: HTTPS for fetch-binaries and uploader
- CI pipeline (.gitlab-ci.yml): all 7 rclone exports use HTTPS + CA cert
- GitLab runner values: minio-ca-cert ConfigMap volume for CI pods
- Rust: CA cert loading via MINIO_CA_CERT_PATH in training_uploader,
storage backend, data_acquisition uploader, and k8s_dispatcher
- Cert generation script (infra/scripts/generate-minio-tls.sh)
Fixes training uploader S3 upload failures caused by with_allow_http(false)
connecting to an HTTP endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add tracing::instrument(skip_all) to gRPC handlers across all services
for distributed trace spans via OTLP. Pairs with Prometheus scrape
annotations from previous commit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add --hyperopt, --trials, and --parallel flags to `fxt train start` so
users can dispatch PSO hyperparameter optimization jobs alongside
regular training. The mode=hyperopt tag propagates through the service
layer which selects the correct binary (hyperopt_baseline_rl for DQN/PPO,
hyperopt_baseline_supervised for all others) and builds hyperopt-specific
CLI arguments via a new extra_args field on TrainingJobParams.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Align test assertions with the production job spec changes from b5fca4d1:
- Binary paths use /binaries/ prefix
- Args use --symbol=X format
- Config uses runtime_image and binaries_bucket fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace separate training/uploader images with single runtime_image
- Add fetch-binaries initContainer (rclone from S3 binaries bucket)
- Switch to emptyDir for output and binaries (no output PVC needed)
- Add Cilium CNI toleration for fresh scale-from-zero nodes
- Extract symbol from file_path last component in data_source
- Increase active_deadline_seconds to 6 hours for hyperopt jobs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace stub approve_promotion and reject_promotion gRPC handlers with
real implementations that call PromotionManager.approve() and .reject().
- approve_promotion: removes from pending, promotes to active model map
- reject_promotion: removes from pending with operator-supplied reason
- Input validation: empty model_id returns INVALID_ARGUMENT
- Error handling: nonexistent model_id returns success=false with message
- Add 4 tests covering approve, reject, and not-found error paths
- Add #[cfg(test)] active_models_for_test() accessor on PromotionManager
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the stub list_pending_promotions gRPC handler with a real
implementation that queries PromotionManager::list_pending() and maps
each PendingModel to the proto PendingPromotion message. Adds a unit
test verifying the field mapping from domain type to proto type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the stub report_job_completion gRPC handler with a real
implementation that:
- Validates job_id as UUID upfront
- On failure: updates DB status to Failed (best-effort), returns "failed"
- On success: updates DB status to Completed, looks up model_type and
symbol from child_jobs table, registers with PromotionManager, and
returns the actual promotion status string
Also adds JobSpawner::get_job_by_id() to fetch individual child jobs
from PostgreSQL, and two new tests covering promotion status mapping
and the PromotionManager integration path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Background tokio task polls JobSpawner.get_next_pending_job() every 5s,
marks found jobs as Running, builds TrainingJobParams, and dispatches
to K8s via K8sDispatcher. On dispatch failure the job is marked Failed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add JobSpawner as the highest-priority dispatch path in start_training.
When job_spawner is available, training requests are persisted to
PostgreSQL via spawn_batch() and a batch ID is returned immediately.
The queue consumer (Task 3) will later poll for pending jobs and
dispatch them to K8s.
Priority order: JobSpawner (DB) → K8sDispatcher (direct) → orchestrator (in-process).
Also adds test_start_training_model_binary_mapping unit test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire JobSpawner into the gRPC service struct so that training jobs can
be persisted to PostgreSQL before being dispatched to K8s. This is the
first step toward a durable job queue that survives pod restarts.
- Add `job_spawner: Option<Arc<JobSpawner>>` to MLTrainingServiceImpl
- Extend `new()` constructor to accept the spawner parameter
- Add `DatabaseManager::pg_pool()` accessor for cheap PgPool cloning
- Construct JobSpawner in main.rs and pass `Some(job_spawner)` to service
- Add compile-time test verifying the field exists
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend ml_training_service to dispatch GPU training jobs as K8s batch/v1
Jobs, collect results via a Rust sidecar uploader, and support model
promotion with operator approval via fxt CLI.
- K8s dispatcher creates Jobs on gpu-training pool with native sidecar
- training_uploader crate: watches DONE/FAILED marker, uploads to S3,
reports completion via ReportJobCompletion gRPC
- PromotionManager compares metrics, queues better models for approval
- 4 new proto RPCs: ReportJobCompletion, ListPendingPromotions,
ApprovePromotion, RejectPromotion
- fxt commands: train start, model list/approve/reject
- Training binaries write DONE/FAILED markers + metrics.json
- Dockerfile, K8s job template, and CI pipeline updated
- StartTraining gracefully falls back to in-process when outside K8s
- 27 new tests (16 service + 11 promotion), 141 total service tests pass
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 17 library crates into crates/, CLI binary into bin/fxt,
consolidate 10 test crates into testing/, split config crate
from deployment config files.
Root directory reduced from 38+ to ~17 directories.
All Cargo.toml paths and build.rs proto refs updated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace _param suppression pattern with actual usage across 21 files:
- adaptive-strategy: wire EpistemicConfig/AleatoricConfig into
UncertaintyQuantifier, KellyConfig into DrawdownTracker,
TLOBConfig into TLOBTransformer
- trading_engine/compliance: store config in 26 compliance structs
(audit_trails, best_execution, sox, iso27001, transaction_reporting,
compliance_reporting, automated_reporting) with public accessors
- fxt: store Channel in LoginClient, ConnectionConfig in ConnectionManager
- ml: remove unused path param from ReplayBuffer::new(), wire
Mamba2Config.target_latency_us into HardwareOptimizer
- services: store TrainingConfig in GpuConfigManager, symbol in
TechnicalIndicatorCalculator
- database: change let _result to let _ (intentional discard)
- trading_engine/brokers: store BrokerConnectorConfig in BrokerConnector
Result: 0 warnings across all 37+ workspace crates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Strip all 413 #[allow(dead_code)] annotations from 139 files and remove
the actual dead code they were suppressing: unused struct fields (and their
constructor sites), unused methods/functions, and entire dead structs.
Key removals:
- trading_engine compliance: ~50 dead structs/fields across audit, reporting, SOX modules
- trading_service: dead execution engine fields, broker routing, paper trading methods
- ml_training_service: dead TLS validation (~340 lines), GPU state, monitoring fields
- backtesting_service: dead model cache, TLS validation, TradeSignal fields
- risk: dead VaR engine fields, safety coordinator fields, position tracker fields
- adaptive-strategy: dead ensemble methods, regime detection, sizing functions
147 files changed, -4264 net lines. Workspace compiles with 0 errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ubuntu 22.04's protobuf-compiler (v3.12) lacks proto3 optional field
support. Install protoc v25.1 from GitHub releases instead, which
matches the protoc version available in Debian bookworm-based images
used by the other 5 services.
Validated: docker build completes successfully, binary starts with
ml_training_service --help.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 service Dockerfiles were missing newly-added workspace crates
(web-gateway, ctrader-openapi, foxhunt-deploy, broker_gateway_service),
causing cargo workspace resolution failures during Docker builds.
Changes across all Dockerfiles:
- Add COPY directives for web-gateway, ctrader-openapi, foxhunt-deploy,
broker_gateway_service (new workspace members since Dockerfiles written)
- Add SQLX_OFFLINE=true env and .sqlx cache copy where missing
- Add perl and make system deps (needed for OpenSSL build from source)
- Remove COPY migrations (dir excluded by .dockerignore, not needed)
- Expand broker_gateway_service from 3-crate to full workspace copy
Validated: docker build --check passes all 6, cargo check -p passes all 6.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate 4 separate ModelType enum definitions (ml 15 variants,
model_loader 7, campaign 2, job_spawner 4) into a single canonical
definition in common/src/model_types.rs with the union of all variants
and all methods (file_extension, as_str, to_db_string, weight, from_str,
Display).
- ml/src/lib.rs: replace 15-variant enum with re-export
- model_loader/src/lib.rs: replace 7-variant enum with re-export,
update PascalCase names (Dqn->DQN, Tft->TFT, etc)
- ml/hyperopt/campaign.rs: replace 2-variant enum with re-export
- services/ml_training_service/job_spawner.rs: replace 4-variant enum
with re-export, MAMBA2->MAMBA
- Remove orphan impl ToString in ml/observability/metrics.rs (Display
now provided by canonical type)
- Update backtesting_service and model_loader tests for new names
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move identical TLS type definitions from 4 service crates into
common/src/tls.rs, eliminating ~435 lines of duplicated code.
Each service retains its own TlsConfig struct and validation logic
(async vs sync, delegated vs monolithic) while sharing the type
definitions. Services re-export the types for backward compatibility.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removed 3 duplicate ModelType enums (model_loader, hyperopt campaign,
job_spawner). Canonical definition in ml/src/lib.rs with 15 variants.
model_loader and job_spawner now re-export from ml. Added as_str(),
s3_prefix(), Display, to_db_string(), and weight() to canonical enum.
Replaced conflicting ToString impl with Display. Fixed variant name
mismatches (Dqn->DQN, Mamba2->MAMBA, Liquid->LNN, TlobTransformer->TLOB).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
