ml::validation is gated behind validation-mod, but dqn/config.rs:177
references crate::validation::RegimeMetrics unconditionally. Both
services pull in dqn::config via the trainer pipeline, so they need
the gate opened to compile. Cheaper than refactoring 700+ refs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
default-features which included cuda; my prior commit dropped both
default-features=false AND added only `financial`, breaking compile.
ml's source references cudarc unconditionally (cudarc::driver::CudaSlice
etc. are not behind #[cfg(feature = "cuda")]). With `cuda` feature off,
cudarc is not pulled in, leading to 722 unresolved-module errors.
Add `cuda` explicitly to the features list. The 8 leaf sub-crates
(ml-backtesting, ml-paper-trading, etc.) stay dropped — the win on
service compile time is preserved.
Real fix is to gate cudarc references in crates/ml/src/ behind
`#[cfg(feature = "cuda")]`, but that's a separate refactor.
Two-part fix that finally removes ~35% of ml-* sub-crates from
service-binary build closures:
1. crates/ml/Cargo.toml: 8 leaf-level ml-* sub-crates become optional
behind feature flags (one feature per `pub mod` in lib.rs):
ml-backtesting ↔ feature `backtest-mod` (ml::backtesting)
ml-paper-trading ↔ feature `paper-trading-mod` (ml::paper_trading)
ml-stress-testing ↔ feature `stress-testing-mod` (ml::stress_testing)
ml-explainability ↔ feature `explainability-mod` (ml::explainability)
ml-universe ↔ feature `universe-mod` (ml::universe)
ml-regime-detection ↔ feature `regime-detection-mod`(ml::regime_detection)
ml-validation ↔ feature `validation-mod` (ml::validation)
ml-data-validation ↔ feature `data-validation-mod` (ml::data_validation)
Aggregated into the umbrella `full-stack` feature, which is in
`default = [...]` so existing `ml.workspace = true` callers see
identical behavior (testing/integration, crates/backtesting).
2. services/trading_service + services/backtesting_service: switched
from `ml.workspace = true` to explicit `path = "../../crates/ml"`
form with `default-features = false`. This is necessary because
cargo 1.89 silently ignores `default-features = false` when
combined with `workspace = true` (a known cargo limitation).
Path form bypasses the workspace dep and applies the opt-out.
Verified by `cargo tree -p X | grep ml-* | wc -l`:
trading-service 23 → 16 (-30%)
backtesting-service 23 → 16 (-30%)
Source-grep verified neither service references any of the 8 gated
modules (`use ml::backtesting`, `use ml::explainability`, etc. all
zero hits in src/). The only `ml::explainability` mention in
trading-service is a string in an error log — not a code path.
ml-training-service and trading-agent-service were already on path
form with default-features = false; they're unaffected here.
cargo check --workspace passes.
Remove dependencies declared in 5 service Cargo.toml files that no
source code in those services references (verified by grepping for
use statements). Reduces dep-graph fan-out and unnecessary recompiles.
services/api/Cargo.toml −16 deps (async-trait, bytes,
const-oid, hdrhistogram,
hex, http-body, hyper,
hyper-util, num-traits,
rust_decimal, tokio-stream,
tower-layer, tower-service,
tracing-subscriber,
trading_engine, zeroize.
+ json feature added to
reqwest since trading_engine
was enabling it transitively)
services/trading_service/Cargo.toml −11 deps
services/backtesting_service/Cargo.toml −17 deps
services/trading_agent_service/Cargo.toml −6 deps
services/ml_training_service/Cargo.toml −4 deps
False positives kept (cargo-machete misses these because they're only
referenced in tonic-generated proto code, not in hand-written src):
- prost (`::prost::Message` derive in build.rs-generated code)
- tonic-prost (`tonic_prost::ProstCodec::default()` in generated tonic
clients/servers)
cargo check --workspace passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Atomic removal of 5 boolean feature flags from DQNConfig per
`feedback_no_feature_flags.md`, all of which gated dead or redundant code.
(use_iqn already removed in da632446c.)
- `use_dueling`, `use_distributional`, `use_noisy_nets`, `use_branching`:
pure-cosmetic always-on flags. Only metadata strings remained.
4 `if true /* always on */` blocks in dqn.rs collapsed to live arms;
dead else arms (legacy non-branching code paths, 5-flat ExposureLevel)
deleted.
- `use_count_bonus`: redundant with `count_bonus_coefficient` (the live
numeric kill-switch). Recording made unconditional; coefficient is the
single dial.
- `use_cvar_action_selection`: dead post-use_iqn cleanup (only consumers
were inside the deleted IQN arms). cvar_alpha retained for cuda_pipeline
CVaR loss usage.
count_bonus.rs API tightened: `bonuses_branched_f32(&self,
exp_out: &mut [f32], ord_out: &mut [f32], urg_out: &mut [f32])` write-into
API alongside the existing Vec<f64> form (kept for tests). Production
DQN::get_count_bonuses_branched() returns fixed `([f32; 4], [f32; 3],
[f32; 3])` arrays — allocation-free, f32 throughout, fed directly to GPU
action selector via stack-allocated buffers.
Test 0.F outputs bit-identical vs pre-cleanup (sigma_C51, argmax,
Thompson counts) — confirms legacy use_* arms were unreachable as
expected.
`docs/dqn-wire-up-audit.md` updated per Invariant 7.
Precondition for the MoE regime redesign per
`docs/superpowers/specs/2026-04-27-moe-regime-redesign-design.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`use_iqn` is exactly the `use_/enable_` boolean banned by
`feedback_no_feature_flags.md`. It gated dead code: production training
runs IQN unconditionally through `cuda_pipeline/gpu_iqn_head.rs` +
`iqn_dual_head_kernel`, properly wired into the branching architecture
with the `FIXED_TAUS` 5-quantile schedule. Nothing in the cuda_pipeline
production path ever read `use_iqn` or `iqn_network`.
The legacy `iqn_network: Option<QuantileNetwork>` field on `DQN` was a
parallel CPU-side network from a pre-branching era, structurally
unreachable in production: every consumer was gated behind the
`if true /* use_branching: always on */` arm at `q_values_for_batch`,
so the IQN else-if at L1822 was dead code. Training optimised
`iqn_network` parameters in isolation; inference never read them. That's
the train/inference mismatch the L283 comment ("IQN trains base
q_network but inference uses dist_dueling network (zero gradients)")
was working around by **disabling** the feature instead of fixing the
inference path. Per `feedback_no_quickfixes.md` + `feedback_no_hiding.md`
the fix is to remove the dead path entirely.
Strip:
- `DQNConfig::use_iqn` field + parses + checkpoint hash + metadata.
`dqn.use_iqn` / `dqn.iqn_embedding_dim` / `dqn.iqn_num_quantiles` /
`dqn.iqn_kappa` from older checkpoints are silently dropped on load
(same pattern used for `use_dueling`). `iqn_lambda` stays — the
cuda_pipeline dual head consumes it as the IQN aux loss weight.
- 3 vestigial config fields (`iqn_num_quantiles`, `iqn_kappa`,
`iqn_embedding_dim`) — never read in production; kernel-side macros
(`IQN_NUM_QUANTILES = 5`, embed_dim 64, kappa 1.0) are the actual
config.
- 4 default builders (`Default`, `aggressive`, `conservative`,
`emergency_safe_defaults`) drop the 4 IQN-related fields each.
- `DQN::iqn_network` field + initialisation block in `new_with_stream`.
- 6 conditional gates in `select_action`, `select_action_with_confidence`,
`select_action_inference`, `q_values_for_batch` — all collapse to the
live (branching or standard-Q) arm.
- `DQN::get_state_embedding` (only consumed by deleted IQN paths).
- The entire `crates/ml-dqn/src/quantile_regression.rs` module (392 LOC)
+ its 2 lib.rs exports. Nothing outside `ml-dqn` ever imported it
(the `quantile_huber_loss` reference in `gpu_iqn_head.rs` is a CUDA
kernel name string, unrelated to this Rust module).
Downstream call sites:
- `crates/ml/src/trainers/dqn/{config,fused_training,trainer/constructor}.rs`:
drop `iqn_num_quantiles` / `iqn_embedding_dim` / `iqn_kappa` references
off `DQNConfig`; substitute kernel-fixed literals (64, 1.0) where
`GpuDqnTrainConfig` / `GpuIqnConfig` still expect them.
- `crates/ml/examples/evaluate_baseline.rs`: drop two `iqn_num_quantiles`
hyperparam reads (their `..DQNConfig::default()` fallbacks now stand
alone).
- `crates/ml/tests/dqn_action_collapse_fix_test.rs`: drop the
`assert!(!config.use_iqn, "...gradient dead zone")` and the explicit
`config.use_iqn = false` setter; the dead-zone pathology is now
structurally impossible.
- `crates/ml/tests/dqn_inference_test.rs`: drop `config.use_iqn = false`.
- `services/trading_service/src/services/dqn_model.rs`: drop the
`iqn={}` debug-log field.
- `crates/ml/src/trainers/dqn/distributional_q_tests.rs`: ship Test 0.F
(Plan A Task 8 #186) — converged-checkpoint extraction harness +
`MappedF32Buffer` (mapped pinned f32 mirror) + `compute_sigma_c51_test`
kernel handle. The structural assertions panic on the local 5-epoch
smoke checkpoint as designed (under-converged: sigma_C51 spread ~1.4%
across directions, P(active)=0.4330 >= 0.20). Docstring rewritten to
drop `use_iqn=false` framing and the legacy "Tier-B-prime" caveat;
Tier-A version is GPU-integration-only per
`feedback_no_cpu_forwards.md` (CPU is read-only).
`docs/dqn-wire-up-audit.md` updated per Invariant 7.
Build: `cargo check --workspace --tests` clean (0 errors).
Test: `cargo test -p ml --lib distributional_q_tests::test_0f
-- --ignored --nocapture` produces bit-identical sigma_C51 /
argmax / Thompson values vs pre-removal — confirms branching
forward path is the same code post-cleanup as pre-cleanup
(legacy `use_iqn` arms were unreachable, as expected).
Net: 12 files, +458 / -785 lines (327 LOC deleted).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the convergence guardrail: every per-epoch HEALTH_DIAG metric is
checked against the bands in config/metric-bands.toml; N consecutive
warn-band epochs emit a tracing::warn; 2N consecutive error-band epochs
return Err(CommonError::RegressionDetected{...}) cleanly from the
training loop, which propagates to the train_baseline_rl subprocess
exit code (no libc::raise — clean Rust error path).
Wire-points:
- New module: crates/ml/src/trainers/dqn/trainer/monitoring.rs
- MetricBands {warn_low, warn_high, error_low, error_high}
- BandSettings {consecutive_epochs_for_warn, consecutive_epochs_for_error}
- MetricBandsRegistry: load_from_toml + update_and_check
- TerminationReason {RegressionWarn, RegressionError}
- NaN treated as out-of-band (consecutive++; never resets streak)
- Unknown metrics return None (silent OK per Invariant 7 audit)
- crates/common/src/error.rs: new CommonError::RegressionDetected variant
carrying {metric, value, band, consecutive}
- crates/ml/src/trainers/dqn/trainer/constructor.rs: load
config/metric-bands.toml at trainer init; warn-only on missing file
(backward compat for environments without the config)
- crates/ml/src/trainers/dqn/trainer/training_loop.rs: harvest per-epoch
metrics (parallel emit alongside HEALTH_DIAG), feed each through
registry.update_and_check; on Some(TerminationReason::RegressionError)
emit final HEALTH_DIAG[N]: TERMINATED_BY_REGRESSION line and return Err
- services/trading_service/src/error.rs: minimal handler for the new
CommonError variant (existing pattern)
Validation:
- 8 unit tests in monitoring::tests pass (band logic, NaN, warn-only
behaviour, error-streak threshold, unknown-metric, invalid TOML)
- regression_detection GPU smoke (3.19s): trainer with intentionally
narrow train_loss error band [0, 1e-9] self-terminates at epoch 5
after 6 consecutive error-band epochs; final HEALTH_DIAG line emits
TERMINATED_BY_REGRESSION with metric/value/consecutive/band fields
- multi_fold_convergence smoke (650s, --release): all 3 folds train
to completion, all 3 checkpoints saved, no false-positive
termination on the populated metric bands. Per-fold best train
Sharpe: F0=-9.7831 (bit-baseline), F1=25.8272, F2=39.2687. F1/F2
on the lower end of observed noise distribution
({74.56, 61.10, 71.53, 25.83} for F1; {88.20, 61.57, 65.96, 39.27}
for F2) but training healthy throughout: aux clauses fire every
epoch, sharpe_ema recovers from F0 collapse (-9.78 → +14.8 by start
of F2), no regression detection trips.
config/metric-bands.toml populated for the metrics emitted by
HEALTH_DIAG today (avg_q_value, train_loss, val_sharpe, train_sharpe,
aux_next_bar_mse, aux_regime_ce, isv_* slot EMAs, sharpe_ema, etc.).
Bands derived from current cleanroom smoke + permissive defaults
where only one sample exists; populate-metric-bands-from-runs.py will
tighten them after Plan 5 Task 5's multi-seed pass produces real
distributions.
Constraints honoured: GPU-only in hot path (band check is CPU-side
post-HEALTH_DIAG, off the captured graph); no atomicAdd; no stubs;
no // ok: band-aids; no tuned constants beyond the toml-loaded bands;
no .unwrap() introduced; cargo check clean at 11 warnings (workspace
baseline preserved, plus ml-dqn pre-existing 1 warning).
Audit doc: new row added documenting monitoring.rs module, the
CommonError variant, the training_loop wire-point, and the design
choice that band-checks run AFTER HEALTH_DIAG emit (not before) so
the diag log already reflects the metric values that triggered any
termination.
Plan 5 T1 (multi-seed harness) landed at c6634254e+47c8b783c; T2
(this) gives the regression hard-stop that the multi-seed final
pass (T5) consumes to bail out early on bad seeds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Branch worktree-agent-aadd27f0, commit 9b27428de. Replaces the stub
{value:0.5, confidence:0.0} with dispatch via Arc<dyn
ModelInferenceAdapter>, wrapping any of the 10 concrete per-model
adapters in crates/ml/src/ensemble/adapters/. prediction_value
derived from inner direction∈[-1,1] via (d+1)/2 bullish probability.
confidence forwarded from inner adapter (softmax-max / quantile IQR
per model), clamped [0,1]. Returns Err (not fake 0-confidence) when
inner is_ready=false, inner predict fails, output non-finite, or
features empty.
build_production_strategy signature now Vec<(String, Arc<...>)>; sole
caller ml_strategy_engine.rs passes empty vec (not 10 ghost adapters)
with a note to wire real from_checkpoint when backtesting-side
checkpoint loading is ready.
# Conflicts:
# crates/ml/src/ensemble/model_adapter.rs
The adapter's `predict` returned
`{ prediction_value: 0.5, confidence: 0.0 }` as a "neutral" stub.
Downstream ensemble filtering dropped any 0-confidence prediction,
so the adapter silently contributed nothing -- a stub that survived
only because the caller discarded it.
Now: `EnsembleModelAdapter` holds an
`Arc<dyn ModelInferenceAdapter>` and dispatches `predict` to the
wrapped per-model GPU adapter (`DqnInferenceAdapter`,
`PpoInferenceAdapter`, `TftInferenceAdapter`, `Mamba2InferenceAdapter`,
`LiquidInferenceAdapter`, `KanInferenceAdapter`,
`XlstmInferenceAdapter`, `TggnInferenceAdapter`,
`TlobInferenceAdapter`, `DiffusionInferenceAdapter`). The inner
adapter already runs a full forward pass on its model and returns a
normalized `(direction, confidence)` pair; the bridge maps
`direction in [-1, 1]` -> `prediction_value in [0, 1]` via
`(direction + 1) / 2` to match `MLPrediction`'s bullish-probability
contract (> 0.5 = bullish), clamps confidence to [0, 1], and
propagates `metadata.latency_us` as the inference latency.
The bridge returns `Err` (never a faked 0-confidence success) when
the inner model reports `is_ready() == false`, the inner `predict`
fails, the output is non-finite, or the feature slice is empty.
`build_production_strategy` no longer fabricates ten zero-confidence
ghost adapters. It now accepts
`Vec<(String, Arc<dyn ModelInferenceAdapter>)>` -- the caller owns
real model construction (checkpoint loading, device selection). The
only current caller (backtesting service `MLPoweredStrategy::new`)
passes an empty vec; that yields an empty ensemble, which is an
honest "no models loaded" signal rather than ten stubs that exist
only to be filtered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
services/trading_service/src/services/risk.rs compute_sharpe_ratio /
compute_sortino_ratio previously returned `0.0` as an
"insufficient-samples" fallback. Plain zero is indistinguishable
from a legitimate "Sharpe happened to equal 0" result; callers
gating on `sharpe > threshold` silently pass/fail on the fallback,
and downstream monitoring logs it as a real value.
Now: returns `Option<f64>` — `Some(value)` for valid samples,
`None` when:
* returns.len() < MIN_RETURN_OBSERVATIONS (threshold at line 30
unchanged — only the return-shape changes)
* daily_std < 1e-12 (degenerate zero-volatility series; ratio
mathematically undefined, not zero)
* downside_count < 2 (Sortino only; too few sub-risk-free
observations to estimate downside variance)
* downside_dev < 1e-12 (Sortino only)
Production caller (get_risk_metrics, the only call site) updated:
* On Some/Some: debug-log as before
* On either None: WARN-log with `[insufficient-samples]` tag,
formatting each ratio as "None" or "{:.4}" so operators can
distinguish missing data from a real zero
* Protobuf RiskMetrics { sharpe_ratio, sortino_ratio } is a
non-optional `double` in risk.proto — export `f64::NAN` rather
than a fake 0, so consumers can detect the undefined case
without silently failing threshold gates.
Tests: each function now has happy-path `Some(_)`,
insufficient-samples `None`, and empty-input `None` cases;
existing `_zero_volatility` / `_no_downside` tests re-asserted
against `None` (previously asserted against 0.0).
compute_volatility and compute_current_drawdown retain `f64`
return — their 0.0 outputs are legitimate (no position → 0%
drawdown; constant returns → 0% volatility).
Callers touched:
- services/trading_service/src/services/risk.rs:570-596
(get_risk_metrics — only production caller)
- services/trading_service/src/services/risk.rs:1585-1671
(tests)
Cross-file grep (compute_sharpe|compute_sortino|RiskServiceImpl::)
confirms no other callers in crates/, services/, bin/.
position_manager.rs line 772 references the function in a comment
only.
Compile: SQLX_OFFLINE=true CARGO_INCREMENTAL=0 cargo check
--workspace --tests — clean on risk.rs (pre-existing ml-crate
warnings unchanged).
Tests: cargo test -p trading-service --lib risk — 59 passed,
0 failed, including 2 new empty-input tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
migrations:
- 001_trading_events.sql, 003_audit_system.sql: the hard-coded
node_id literals (`trading-node-01`, `audit-node-01`,
`ml-node-01`, `system-node-01`, `change-tracker-01`) are
overridden per-deployment by later migrations rather than read
from the environment. Describe that in the inline comment.
- 004_compliance_views.sql: `generate_compliance_report` is a log
stub — actual report generation is performed by the compliance
service. Say so explicitly.
services:
- ml_training_service/tests/orchestrator_225_features_test.rs: the
empty `#[ignore]`d placeholder for the 225-feature orchestrator
loader has been removed; it held no assertions and only tracked
a TODO (feedback_no_stubs.md).
- trading_agent_service/src/service.rs: portfolio volatility uses
the diagonal-only approximation because cross-asset return
correlations are not maintained in this service. Document that.
- trading_service/src/services/risk.rs: `get_risk_metrics` uses
`calculate_marginal_var` + asset-class fallback; describe why
`calculate_comprehensive_var` is not wired at this boundary.
- trading_service/tests/auth_comprehensive.rs: delete the entire
commented-out legacy BackupCodeValidator test block — the old
`generate_backup_codes` / `store_backup_code` /
`verify_backup_code` surface no longer exists, and MFA
integration tests already cover the new API.
testing:
- harness/grpc_clients.rs: no BacktestingServiceClient proto
exists; reword the stale TODO import line.
- chaos/*: reword the family of "TODO: Implement ..." stubs as
"Currently a no-op / synthetic result" descriptions so readers
know exactly how much of the chaos framework is live.
- compliance_automation_tests.rs: delete the file; it was a giant
/* ... */ block referencing a nonexistent compliance module
and was not wired into any Cargo target.
- framework.rs: describe why `setup()` uses `println!` instead of
`tracing_subscriber` (tracing_subscriber is not a dep of this
integration crate).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove pub state_dim field from DQNConfig and GpuReplayBufferConfig; remove the
state_dim field from GpuExperienceCollector. Replace all reads with
ml_core::state_layout::STATE_DIM (and STATE_DIM_PADDED for cuBLAS-padded
strides). Checkpoint loading now validates saved state_dim against the
constant and hard-errors on mismatch. GpuAttentionConfig.state_dim is a
distinct attention-feature dim and is left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OFICalculator + MicrostructureState fed per-tick. Bar close snapshots
and broadcasts 20 features via tokio::watch channel. Intermediate
snapshots available for speculative compute between bars.
Databento live client integration deferred (requires crate dep).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove use_double_dqn, use_dueling, use_per, use_branching,
use_distributional, use_noisy_nets, use_huber_loss, and use_cql
from DQNConfig, DQNHyperparameters, and DqnParams structs.
These features are always enabled (Rainbow DQN standard). The boolean
flags were dead code — every constructor set them to true, and the
only code paths that set them to false were in tests that disabled
features for simplicity. With the fields removed, the features are
unconditionally active, eliminating ~490 lines of dead configuration.
Key changes:
- Struct field declarations removed from 3 core config structs
- Conditional branches (if use_X { ... } else { ... }) simplified:
dueling/branching/PER network creation is now unconditional
- Checkpoint metadata hardcodes "true" for backward compatibility
- Hyperopt search space index 11 (use_branching) fixed at 1.0
- TOML/YAML config files cleaned of removed fields
- Tests that toggled these flags updated or rewritten
45 files changed, -487 net lines. Zero new test failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Final cleanup:
- 61 test files + 5 example files: candle imports replaced
- 8 testing/integration files: migrated to cudarc/ml-core types
- 3 services/trading_service test files: migrated
- Root Cargo.toml: candle-core, candle-nn removed from [workspace.dependencies]
- crates/ml/Cargo.toml: candle-nn dependency removed
- testing/e2e/Cargo.toml: candle-core dependency removed
Zero active candle_core/candle_nn/candle_optimisers code references remain.
Zero candle dependency declarations in any Cargo.toml.
Remaining "candle" strings are exclusively in doc comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove ALL #[cfg(feature = "cuda")] guards (~400+ occurrences)
- Remove ALL #[cfg_attr(not(feature = "cuda"), ignore)] test annotations (~250)
- Make cuda default feature in 9 ML crates (ml, ml-core, ml-dqn, ml-ppo, etc.)
- Convert nvrtc JIT compilation to precompiled nvcc (searchsorted, prefix_sum)
- Move compile_ptx_for_device() to ml-core for shared access
- Delete dead CPU code: multi_step.rs, self_supervised_pretraining.rs,
training_guard_gpu_tests.rs, CPU PER buffer paths, CPU Q-diagnostics
- Replace unwrap_or(Device::Cpu) with hard errors everywhere
- Remove dead is_cuda() else branches in DQN/PPO/hyperopt trainers
- Change config defaults from "cpu" to "cuda" (rainbow, tlob, pipeline)
- Port IQL value network to GPU kernel (5 CUDA entry points)
- Port HER goal relabeling to GPU kernel (warp-per-sample)
- Wire DSR GPU-to-CPU sync in training loop
- cfg!(feature = "cuda") → true in inference_validator
Zero warnings, zero errors across entire workspace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ProductionFeatureExtractorAdapter was changed to produce 42 features
(40 base + 2 regime) but three test sites and the FEATURE_NAMES constant
still expected 51 (42 + 1 volatility_regime + 8 OFI placeholders).
- backtesting: strategy_runner test assertions 51→42
- trading-service: ensemble_coordinator test assertions 51→42
- trading-service: FEATURE_NAMES_51 → FEATURE_NAMES_42 (drop OFI
placeholders and volatility_regime, matching extraction.rs v2 layout)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all inline type definitions from ml/src/lib.rs to ml-core:
MLError, MLResult, Trade, MarketRegime, HealthStatus, Features,
MLModel trait, ModelRegistry, ParallelExecutor, LatencyOptimizer,
TrainingMetrics, ValidationMetrics, InferenceResult, ModelMetadata.
Dedup: consolidate ConfigError{reason}/ConfigurationError(msg) into
single ConfigError(String) tuple variant (was 2 variants, 174 refs).
Cleanup: convert create_hft_* free functions to associated methods
(HFTPerformanceProfile::ultra_low_latency(), ParallelExecutor::hft()).
ml facade re-exports via `pub use ml_core::*`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Critical bug: all 3 DQN action selection methods (select_action,
select_action_with_confidence, select_action_inference) used
FactoredAction::from_index() which maps indices 0-4 to exposure_idx=0
(Short100) via division by 9. This is the root cause of action
diversity collapse during both training and production inference.
Fix: ExposureLevel::from_index() + OrderRouter::route_default() in all
DQN paths. Also fixes hyperopt objective thresholds (<10 → <3 for
5-action degenerate detection), stale defaults/comments, integration
test configs.
Files: dqn.rs (3 methods), trainer.rs (validation + select_action),
hyperopt/adapters/dqn.rs (thresholds), dqn_model.rs (comments),
train_baseline_rl.rs (default), reward.rs (comment),
dqn_integration.rs + ensemble_integration.rs (num_actions).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add api_uptime_seconds gauge to API service (5s update interval),
matching the pattern used by trading/backtesting/ml-training services
- Remove obsolete Web Gateway Uptime and Active WebSocket Connections
panels from cockpit (web-gateway is not deployed)
- Add API Gateway Uptime (api_uptime_seconds) and API Auth Rate
(api_auth_requests_total) panels in their place
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The dashboard's Config page was returning [unimplemented] because
trading-service had no ConfigService implementation. The API gateway's
ConfigServiceProxy forwards config RPCs to trading-service which now
handles them via direct SQL against the `configuration` table.
Implements GetConfiguration, UpdateConfiguration, DeleteConfiguration,
and ListCategories. Remaining 8 RPCs return UNIMPLEMENTED until needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
containerd on Kapsule nodes uses host DNS which can't resolve
.svc.cluster.local names. Switch all image references from
gitlab-registry.foxhunt.svc.cluster.local:5000 to localhost:30500
(NodePort on the GitLab registry). Also make training runtime image
configurable via TRAINING_RUNTIME_IMAGE env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace simulated login in fxt CLI with real AuthServiceClient gRPC call
- Add auth proto to fxt build.rs compile list and lib.rs proto module
- Add api.access permission to JWT claims issued by AuthGrpcService
- Remove orphaned ci-sensor.yaml (replaced by cluster-applied version)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- JWT issuer now foxhunt-api across all 16 files (services, tests, config, docker-compose)
- Remove serde alias api_gateway_url from FxtConfig (no backwards compat)
- Remove api_gateway CLI alias from e2e orchestrator
- All services must deploy simultaneously for JWT validation to match
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend (Rust):
- Create proto/auth.proto with Login and RefreshToken RPCs
- Implement AuthGrpcService in services/api with JWT issuance
- Register as unauthenticated service in main.rs (pre-auth)
- Update JWT issuer from foxhunt-api-gateway to foxhunt-api
Dashboard (TypeScript):
- Install @connectrpc/connect-web + @bufbuild/protobuf
- Generate TypeScript proto clients via buf (src/gen/)
- Create grpc.ts transport with JWT interceptor
- Rewrite all useApi hooks to typed gRPC calls
- Replace WebSocket with useGrpcStream server-streaming hooks
- Migrate all 6 pages + 6 components to grpc-web
- Delete api.ts, websocket.ts, useWebSocket.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copied from api_gateway, removed REST handlers (port 8080),
added tonic-web + CORS for grpc-web browser access.
Binary renamed: api-gateway → api
Changes:
- Package name: api-gateway → api
- Deleted src/handlers/ (REST ML endpoints on port 8080)
- Added tonic-web 0.13 + tower-http CORS layer
- Server::builder().accept_http1(true) for grpc-web
- CORS_ORIGINS env var (default http://localhost:5173)
- Metrics server on port 9091 (axum) preserved
- All 95 lib tests pass, 0 clippy warnings
- Added services/api to workspace members
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename 7 service binaries from snake_case to kebab-case to match
K8s deployment names. Update Cargo.toml package/bin names, K8s
manifest S3 paths and commands, and cross-crate dependency keys.
- api_gateway → api-gateway
- trading_service → trading-service
- broker_gateway_service → broker-gateway
- ml_training_service → ml-training-service
- backtesting_service → backtesting-service
- trading_agent_service → trading-agent-service
- data_acquisition_service → data-acquisition-service
broker-gateway gets an explicit [lib] name = "broker_gateway_service"
since its new package name maps to broker_gateway (not the original
broker_gateway_service used in source code). All other services map
correctly with Rust's automatic hyphen-to-underscore conversion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change PercentileScaler, RewardNormalizer, CompositeReward, and
PPORewardShaper public APIs from f64 to f32 — matching what callers
actually pass. Internal EMA/percentile math stays f64 for precision.
Keep data_loader SMA/EMA/MACD accumulators in f64 throughout (was
f64→f32→f64 per step), cast to f32 only at output boundary.
Remove dead _transition computation in rainbow_agent_impl and unused
bigdecimal deps from broker_gateway_service and trading_service.
2704 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
kube-rs Client::try_default() uses rustls for in-cluster TLS but no
CryptoProvider was installed, causing a panic on first pod list call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- StreamSystemStatus now queries real Prometheus data (was all zeros)
- StreamAlerts returns honest unimplemented (was sending empty events)
- Training epoch shows "N" not "N/0" (max_epochs not in proto)
- GPU power shows "300W" not "300W / 0W" when limit unavailable
- Services cockpit shows error reason when service is DOWN
- System status polling detects staleness after 3 consecutive failures
- Training cockpit header shows active K8s job count
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
config_service.proto (package foxhunt.config, 4 RPCs) was a subset of
config.proto (package config, 12 RPCs). Migrated api_gateway to implement
ConfigService from config.proto directly. Removed dead config_service()
method from fxt client.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add kube 3.0 + k8s-openapi 0.27 dependencies and create pods_handler.rs
that queries the Kubernetes API for pod status. Replace the UNIMPLEMENTED
stub in monitoring_handler.rs with a real streaming implementation that
polls pods every N seconds and maps them to PodInfo proto messages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add query_cluster_resources() for real CPU/RAM from Prometheus
- Wire into build_response() replacing hardcoded zeros
- Fix NetworkPolicy: add ports 8080/9100/9400 for KSM/node-exporter/dcgm
- Add CiliumNetworkPolicy for hostNetwork pod scraping
- Add SubscribeClusterPods stub to unblock proto addition from Task 5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
