Collect loss tensors on GPU during accumulation loop and perform a
single mean+to_scalar readback at the accumulation boundary instead
of N per-step CPU readbacks. Falls back to CPU sum when GPU tensors
are unavailable. Also adds missing loss_tensor_gpu field to
RegimeConditionalDQN's GradientResult construction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GradientResult and ComputeLossResult gain td_errors_gpu/indices_gpu
Option<Tensor> fields for GPU-resident TD errors. Trainer accumulates
GPU tensors and calls update_priorities_gpu() when available, falling
back to CPU path otherwise. DqnAgent delegates via memory().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New variant delegates to GpuReplayBuffer for all operations. Sample
returns BatchSample with gpu_batch populated (empty CPU vecs). Added
update_priorities_gpu() for tensor-based priority updates, as_gpu_buffer()
for direct access from trainer. Existing tests unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Proportional: cumsum + binary search over alpha-weighted priorities.
Rank-based: sort descending, rank probabilities 1/rank^alpha, cumsum sample.
Priority update: |td_error|^alpha + epsilon scatter into buffer.
All methods return GpuBatch with IS weights normalized by max.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Slice-scatter based insertion handles wrap-around by splitting into
tail + head copies. New experiences get max_priority. Tests verify
correctness at capacity boundary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-allocates all experience tensors on device at init (~47 MB for 100K
buffer). Ring buffer cursor and beta annealing state tracked CPU-side.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU-resident batch tensors allow compute_gradients() to skip CPU→GPU
transfer when sampling from the GPU replay buffer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PerformanceMetrics::from_trades() already returns win_rate as percentage
(e.g. 57.78), but TRIAL_SUMMARY and backtest details log lines multiplied
by 100 again, producing values like 5778%. Remove the extra * 100.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN/PPO trainers now record epoch, loss, val_loss, batch/s every epoch
- CI training jobs get Prometheus scrape annotations via runner overrides
- Allow Prometheus (foxhunt namespace) to scrape foxhunt-ci pods
- Skip no-model metrics in monitoring service session grouping
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GPU experience collector gate checked only `dqn.dueling_q_network`
(plain dueling), but with both `use_dueling: true` AND `use_distributional: true`
(the defaults), DQN creates hybrid `dist_dueling_q_network` instead, leaving
the plain dueling fields as None. This meant the GPU collector never initialized
despite curiosity being enabled.
Fix: Add else-if fallback to check `dist_dueling_q_network`/`dist_dueling_target_network`
when plain dueling fields are None. Same fix applied to the weight sync site.
Also fix test_train_with_empty_data_completes_gracefully: reduce to 5 epochs with
early stopping disabled. The debug-mode async state machine is large enough that
empty-data epochs run ~180ms each (vs ~3ms in release), triggering both plateau
and patience-based early stopping. The test purpose is crash-freedom, not timing.
2497 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enable curiosity module by default (weight 0.0→0.1) to satisfy the
three-way gate (dueling + target + curiosity) that was blocking the
GPU experience collector CUDA kernel. This eliminates the 30-40% CPU
experience collection phase that was the main GPU idle bottleneck.
Additional changes:
- BatchSample API: train_step/compute_gradients now accept
Option<BatchSample> instead of Option<Vec<Experience>>, preserving
PER importance-sampling weights and indices through pre-sampling.
- Async PER pre-sampling: train_step_single_batch and
train_step_with_accumulation now pre-sample from the replay buffer
using a READ lock before acquiring the WRITE lock for GPU training.
This separates CPU sampling (~250μs) from GPU forward/backward (~3ms).
- Delete dead EpochPrefetcher: the binary (train_baseline_rl.rs) already
implements fold prefetching with background thread + mpsc channel +
GPU double-buffering, making the trainer's EpochPrefetcher redundant.
- Hyperopt DQN bounds: curiosity_weight min 0.0→0.01 so PSO can never
fully disable curiosity (which would re-gate the GPU collector).
10 files changed, -195 net lines. 2497 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The auto-detect heuristic used cpus/2 ("smart cap"), designed for
CPU-bound workloads. DQN/PPO trials are GPU-bound — each rayon
thread submits CUDA kernels and waits on cudaDeviceSynchronize(),
using minimal CPU. cpus-1 is the correct cap.
On L40S-1-48G (8 vCPU): 3 threads → 7 threads (2.3× more trials).
Also bumps CI CPU limit 7500m→8000m to expose all 8 cores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The batch span processor needs a tokio runtime for gRPC transport and
periodic flush. Async services already have one via #[tokio::main], but
sync training binaries (hyperopt, train, evaluate) don't.
Previous approach (making binaries async with #[tokio::main]) caused
"Cannot start a runtime from within a runtime" panics because the ML
crate's internal code creates its own tokio runtimes for block_on().
New approach: build_otel_tracer() detects runtime context via
Handle::try_current(). If absent, it creates a dedicated 1-worker
multi-thread runtime stored in a process-lifetime OnceLock. The worker
thread actively polls the OTLP batch export task.
Reverts training binaries to sync fn main() so internal runtime creation
(hyperopt adapters, DQN/PPO trainers) continues working as before.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 training binaries (hyperopt_baseline_rl, hyperopt_baseline_supervised,
train_baseline_rl, train_baseline_supervised, evaluate_baseline,
evaluate_supervised) used sync fn main() but the OTLP batch exporter
requires a tokio runtime (tonic/hyper-util gRPC transport). This caused
an immediate panic on CI when OTEL_EXPORTER_OTLP_ENDPOINT was set.
Fix: #[tokio::main(flavor = "current_thread")] on all 6 binaries.
Also fix pre-existing clippy warnings (shadow, let_underscore_must_use,
doc_markdown, cognitive_complexity, integer_division, unsafe_code).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace CPU-side argmax (to_vec1 + iter enumerate max_by) with
GPU-native argmax(D). Single u32 scalar extraction instead of
full Q-value vector transfer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace &Device::Cpu with &self.device for input and target tensor
creation in Mamba2 hyperopt adapter. Avoids unnecessary CPU→GPU
transfer during hyperparameter optimization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stream-aware ensemble that runs models on separate CUDA streams
for true GPU-level parallelism. Falls back to rayon on CPU.
Uses CudaStreamPool for synchronization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CUDA stream pool with CPU no-op fallback. Foundation for
StreamAwareEnsemble that runs models on separate CUDA streams.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use predict_raw() to collect raw GPU tensors from adapters. Stack,
sigmoid, weighted-sum on GPU before single extraction. Falls back
to CPU path for adapters without tensor output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Override predict_raw() in TGGN, TLOB, KAN, xLSTM, Diffusion adapters
to return raw GPU tensors. Enables GPU-side ensemble aggregation
instead of per-model CPU extraction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace clamped.to_vec1() CPU loop with GPU-native floor/ceil/frac
operations. Removes 3 tensor transfers per forward pass (N floats
down + 3*N up). All index computation now stays on GPU.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stack var_returns, var_residuals, mean_reward, var_reward into single
tensor before extraction. Uses broadcast_sub for GPU-native mean
centering instead of scalar round-trip.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backward-compatible trait extension. Default predict_raw() wraps
predict() result with tensor: None. Adapters can override to return
raw GPU tensors for GPU-side ensemble aggregation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-param gradient norm extraction loop with batched
Tensor::stack pattern. Single GPU→CPU sync instead of one per
parameter tensor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-param calculate_gradient_norm() loop with batched
Tensor::stack pattern. Single GPU→CPU sync instead of one per
parameter tensor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-batch loss.to_vec0() in TFT training/validation loops
with GPU tensor accumulation. Single extraction per epoch + NaN guard
every 100 batches.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-batch .to_dtype(F64).to_scalar() in Liquid train/validate
with GPU tensor accumulation. Single extraction per epoch + NaN guard
every 100 batches. Removes ~100-500 GPU→CPU syncs per epoch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-batch loss.to_scalar() in TLOB train_epoch/validate_epoch
with GPU tensor accumulation. Single extraction per epoch + NaN guard
every 100 batches.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both DQN and PPO eval paths used old 3-action index matching (0=Buy,
1=Sell, 2=Hold). Now uses FactoredAction.target_exposure() for
exposure-weighted returns and order-type-specific transaction costs.
PPO path had .to_int() which doesn't exist on FactoredAction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
compute_reward_pnl() took raw action_idx (0=Buy,1=Sell,2=Hold) — wrong
with 45-action FactoredAction encoding. Now takes &FactoredAction and
uses order-type-specific transaction costs. Position tracking uses
action.exposure instead of old 3-way index match.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hyperopt adapter and PPO benchmark still had num_actions: 3, which would
produce misconfigured PPO models when used with the 45-action FactoredAction
sampling path. Found by spec compliance review.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use a purely linear network (no ReLU) for the completeness axiom test.
IG on linear functions is mathematically exact, so the test is
deterministic regardless of random weight initialization. Tighten
tolerance from 20% to 1% (f32 rounding only). Keep ReLU network in
the basic test for non-linear verification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- trading_service: PPO predict() used TradingAction match but act() now
returns FactoredAction. Use target_exposure() mapped to 0-1 range.
- IG completeness axiom test: relax tolerance from 5% to 20% (random
weights with ReLU non-linearity and trapezoidal rule can exceed 5%).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was using direction=idx/15 (3 groups of 15: Buy/Sell/Hold) — incompatible
with DQN's FactoredAction encoding. Now uses exposure_idx=idx/9 (5 groups
of 9: Short100/Short50/Flat/Long50/Long100) matching FactoredAction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sample_action(), act(), act_with_log_prob(), greedy_action() now return
FactoredAction instead of TradingAction. Fixes the architectural disconnect
where num_actions=45 output neurons were sampled through a 3-action bottleneck.
TrajectoryStep.action and TrajectoryBatch.actions now use FactoredAction.
Added FactoredAction::from_legacy() for backward compatibility in tests.
Updated all PPO consumers: trainers/ppo.rs, hyperopt/adapters/ppo.rs,
validation/ppo_adapter.rs, benchmark/ppo_benchmark.rs.
2487 tests pass, 0 clippy warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidates ExposureLevel, Urgency, FactoredAction from dqn/action_space.rs
and ppo/factored_action.rs into common/action.rs. Both dqn:: and ppo::
re-export for backward compatibility. Deletes ppo/factored_action.rs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
calculate_feature_importance() returned hardcoded fabricated scores with
wrong feature names on every inference. Replaced with honest empty map.
Real importance is computed on-demand via integrated gradients.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements IntegratedGradients using Candle autograd (Var::from_tensor +
backward). Computes attributions by integrating input gradients along
interpolation path from baseline to input.
Verified via completeness axiom test: sum(attributions) ≈ F(x) - F(baseline).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The real_ prefix was misleading — there is no fake data loader.
Mechanical rename across 18 source files, no logic changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DQN PER: defer td_errors to_vec1() after loss.to_scalar() — piggyback on
existing pipeline flush instead of forcing premature GPU→CPU stall
- PPO trajectories: capacity-hint Vec allocations, extend_flat_states methods,
states_flat field on TrajectoryBatch for zero-copy GPU upload
- TGGN validate(): batch N per-sample losses on GPU → single to_scalar() sync
(was N GPU→CPU syncs)
- Liquid backward(): batch grad-norm per-param sqr().sum_all() on GPU → single
to_scalar() sync (was N GPU→CPU syncs per optimizer step)
- Liquid validate(): same N→1 GPU sync reduction as TGGN
- DQN trainer: restore EpochPrefetcher/DoubleBufferedLoader API (wrongly deleted)
- train_baseline_rl: wire DoubleBuffer GPU pre-upload — after CPU prefetch
completes, immediately upload next fold to GPU via DqnGpuData::upload() so
next fold starts with data already resident on GPU
2478 tests pass, 0 clippy warnings, 0 compile errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
