The kernel was updated with portfolio_states, min_hold_bars, max_position
params (Task 3 of trade lifecycle fix) but the backtest evaluator's launch
site wasn't updated. It passed 10 args to a 13-param kernel, causing
SIGSEGV from reading garbage for the missing params.
Fix: pass NULL/0/0.0 for the 3 new params (hold enforcement disabled
during evaluation for now — will be enabled in eval cleanup plan).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extend build.rs to compile all 25 .cu kernel files to cubins via nvcc.
Common header prepended to 24 kernels; experience_kernels.cu is standalone.
Total cubin size ~1.1 MB embedded in binary.
- cargo:rerun-if-changed tracks all .cu and .cuh files
- CUDA_COMPUTE_CAP env var selects target arch (sm_86, sm_90)
- Graceful skip when nvcc unavailable (non-CUDA builds)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The reversal P&L block (Kelly stats, entry_price reset) ran BEFORE the
hold enforcement guard. If hold_time < min_hold_bars cancelled a reversal,
the P&L was already booked for a trade that didn't execute, corrupting
portfolio_states. This caused SIGSEGV in the hyperopt path where the
corrupted state triggered invalid memory access in subsequent kernels.
Fix: moved the reversal P&L block from line 751 to after line 862
(after hold enforcement resolves final reversing_trade value).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trade count dropped from 77% turnover (620K/800K) to 16.5% (527/3200)
with min_hold_bars=3. The hold enforcement is working correctly in the
standard training path. SIGSEGV in hyperopt path is a separate issue
(likely buffer sizing in the hyperopt adapter's experience collector).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
With v_range ±240 (vs old ±2), C51 distributional loss is naturally
~100-400 instead of ~3-6. Updated smoke test assertion accordingly.
Trade lifecycle fix validated: 467 trades (was 620K) with min_hold_bars=3.
PF=1.87, WinRate=49.7%, MaxDD=41.7% — model learns multi-bar swing trading.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add build.rs that compiles .cu kernels to cubins at cargo build time via nvcc.
Replace OnceLock<Ptx> + compile_ptx_for_device() in gpu_action_selector.rs with
include_bytes! + Ptx::from_binary() loading from the precompiled cubin.
- build.rs gracefully skips when nvcc is not available (non-CUDA builds)
- CUDA_COMPUTE_CAP env var controls target GPU arch (86=RTX 3050, 90=H100)
- common_device_functions.cuh prepended to kernel sources at build time
- Zero runtime compiler invocations for epsilon_greedy kernel
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Integer parameter (rounded from continuous). Lets PSO discover
optimal hold duration for ES futures regime characteristics.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
During min_hold period, forces exposure to current position index.
Covers both greedy (Q-value) and random (epsilon) paths.
Portfolio state already GPU-resident — zero-copy parameter pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add min_hold_bars to DQNHyperparameters (usize, default 5), ExperienceSection
in training_profile, and both TOML configs (smoketest=3, production=5).
Wired through apply_to() so TOML overrides land correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added BARS_PER_DAY, BARS_PER_YEAR, ANNUALIZATION_FACTOR to common::thresholds::time
- Added bars_per_day field to DQNHyperparameters (default 390.0, configurable)
- compute_epoch_financials() now takes bars_per_day parameter from hyperparams
- coordinator_extended.rs + ab_testing.rs use centralized ANNUALIZATION_FACTOR
- When switching to tick data, change one constant in common/thresholds.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Compounding per-bar returns over 10K+ bars still produces extreme numbers
(+1.8M% return, 2687% MaxDD). Replaced with:
- Return: mean_per_bar × bars_per_year (annualized, no compounding)
- MaxDD: windowed over last 10K bars AND capped at 1.0 (100%)
These are monitoring metrics — the hyperopt objective uses the backtest
evaluator which already has correct windowed metrics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Training epoch metrics compound step_returns multiplicatively. With 2M+
bars per epoch on H100, this produces Return=+2.1e24% and MaxDD=99.5% —
meaningless numbers that corrupt risk monitoring and early stopping.
Fixed: total_return and max_drawdown now use only the last 10K bars
(~25 trading days), matching the backtest evaluator's window cap.
Sharpe/Sortino are unaffected (use arithmetic mean/std, not compounding).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
dqn-smoke: Walk-forward validation DQN uses raw price returns (~0.001),
not production reward_scale=10. Set v_min/v_max to ±10 for the small
16-dim 3-action test network (was inheriting ±240 from production default).
dqn-early-stop: Gradient collapse threshold = lr × multiplier must exceed
the actual gradient norm to trigger collapse. With v_range ±240, gradient
norms reach 100-10000 (was ~0.5-2.0 with old ±2.0 range). Updated
multiplier from 1e9 to 1e12 to guarantee threshold (10000) > grad norm.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Noisy nets are now always on (mandatory feature since config unification).
Epsilon start always uses the noisy-nets-aware default (0.05).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- dqn_action_collapse_fix_test: search space grew from 39D to 45D
(added c51_warmup, her_ratio, curiosity, cvar, dt_pretrain). Updated
assertions to use >= 39 and dynamic index lookup for cql_alpha.
- training_stability: Q-value assertion widened from 2.5 to 300.0
to match v_range ±240 (computed from reward_scale=10, gamma=0.95).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds mbp10_data_dir and trades_data_dir to CampaignConfig, passed to
DQNTrainer via with_ofi_data_dirs(). Enables OFI features (VPIN,
Kyle's Lambda, etc.) in hyperopt campaigns. dqn_full() now defaults
to 50 epochs/trial for baseline runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: backtest windows of 300K bars produced ±billions% returns via
multiplicative compounding, and sqrt(252) annualization was wrong for
1-minute bars.
Fixes:
- Window size capped to 10K bars (~25 trading days), evenly distributed
across the full validation set (was clustered in first 6%)
- Annualization: configurable bars_per_day field in GpuBacktestConfig
(default 390.0 for 1-min), produces sqrt(98280) ≈ 313.5
- tanh normalization recalibrated: Sharpe/5, Sortino/8 (was /2, /3)
- CVaR threshold scaled to per-bar: 0.003 with slope 1400 (was 0.05/200)
- VaR/CVaR strided sampling covers full window (was first 4096 only)
- financials.rs + ab_testing.rs: sqrt(252) → sqrt(98280) for consistency
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hybrid approach: each of 256 threads exports boundary metadata (first/last
action, prefix/suffix return, interior trade count). Thread 0 stitches
boundaries in O(256) to produce exact trade count and win rate.
Fixes: trades spanning chunk boundaries were fragmented (returns lost,
counts incorrect). Now every trade's cumulative return is tracked exactly
regardless of which thread chunks it spans.
7 boundary values stored in s_sorted[stride..8*stride] — zero extra
shared memory (reuses sort scratch between equity reduction and bitonic sort).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
s_max_dd, s_trades, s_wins parallel reductions were made redundant by
the exact sequential drawdown scan and upcoming boundary stitching.
Reduces shared memory from 25KB to 22KB and removes 3 dead ops/bar
from the per-thread loop. Trade count/win rate temporarily zeroed —
restored by boundary stitching in next commit.
The stress tester creates a SECOND DQNTrainer for validation, which
OOMs on 4GB GPUs. Changed from fatal error (?) to warning + skip.
Hyperopt now successfully trains on RTX 3050 — 2 trials × 10 epochs
completed in 19 minutes. Backtest metrics still show extreme returns
(multiplicative compounding over 895K bars) — needs separate fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DQNTrainer adapter in hyperopt silently returns penalty (1000000)
because build_hyperparams() fails with the cleaned config struct.
Needs: update DQNParams→DQNHyperparameters mapping for removed use_
booleans and new reward_scale field.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 7 new fields to DQNHyperparameters (fill_ioc_fill_prob,
fill_limit_fill_min, fill_limit_fill_max, fill_spread_cost_frac,
fill_spread_capture_frac, q_clip_min, q_clip_max) and wire them
through training_profile.rs into ExperienceCollectorConfig construction
in training_loop.rs. Previously these 7 values were hardcoded at the
construction site; now they flow from TOML [experience.fill_simulation]
and [risk] sections. Default values match the prior hardcoded constants
so existing behavior is unchanged.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove use_double_dqn, use_dueling, use_per, use_branching,
use_distributional, use_noisy_nets, use_huber_loss, and use_cql
from DQNConfig, DQNHyperparameters, and DqnParams structs.
These features are always enabled (Rainbow DQN standard). The boolean
flags were dead code — every constructor set them to true, and the
only code paths that set them to false were in tests that disabled
features for simplicity. With the fields removed, the features are
unconditionally active, eliminating ~490 lines of dead configuration.
Key changes:
- Struct field declarations removed from 3 core config structs
- Conditional branches (if use_X { ... } else { ... }) simplified:
dueling/branching/PER network creation is now unconditional
- Checkpoint metadata hardcodes "true" for backward compatibility
- Hyperopt search space index 11 (use_branching) fixed at 1.0
- TOML/YAML config files cleaned of removed fields
- Tests that toggled these flags updated or rewritten
45 files changed, -487 net lines. Zero new test failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Five bugs in the GPU backtest metrics kernel produced impossible results
(17,975% return, 100% MaxDD on every trial):
1. Additive return accumulation (local_cum += r) replaced with
multiplicative compounding (local_cum *= 1+r, init 1.0)
2. Absolute drawdown (peak - current) replaced with fractional
drawdown ((peak - current) / peak)
3. Strided bar processing (thread sees every Nth bar) replaced with
consecutive chunked processing for correct drawdown tracking
4. Per-bar win counting replaced with per-trade win/loss tracking
5. Total return now computed via multiplicative equity product
reduction across threads (s_sorted[0] - 1.0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Five reward computation fixes in experience_env_step CUDA kernel:
1. Replace CUSUM vol proxy with ATR(14): CUSUM at feature[41] is a binary
direction indicator [-1,1,0], NOT volatility. When CUSUM≈0, vol_proxy
became 0.0001 causing 10000x reward amplification. ATR(14) at feature[9]
is actual realized volatility — reverse the safe_normalize encoding
(ln(atr)+7)/16 to recover atr_pct = exp(norm*16-7) / price.
2. Move loss aversion BEFORE squash: previously applied after hard clamp,
creating asymmetric [-15, +10] range making expected reward negative
even for fair strategies. Now applied pre-squash for smooth asymmetry.
3. Replace hard clamp with tanh soft squash: fmaxf(-10, fminf(10, reward))
destroyed tail information (1% and 5% wins both → 10.0). tanh preserves
that larger wins produce proportionally larger rewards.
4. Remove turnover penalty: the 0.05*|delta|/max_position penalty double-
counted transaction costs already deducted from cash via Almgren-Chriss
impact model at line ~679, over-penalizing necessary rebalancing.
5. Clarify CUSUM spread_scale usage: CUSUM at feature[41] is correctly used
as market-stress proxy for spread widening in tx cost computation — this
is distinct from the (now-fixed) vol proxy for reward normalization.
Also: annotate min_hold_bars=5 as hyperopt candidate.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DBN loader was producing 2-element targets [close, next_close] which got
zero-padded to 4 elements. Now produces full 4-element layout matching the
kernel's expected format: [0:1]=network input, [2:3]=raw prices for portfolio sim.
Kernel reads tgt[2:3] for raw_close/raw_next (restored to original design).
All other kernels (DT, PPO, expert demos) also read tgt[2:3] correctly.
Stale feature cache invalidated by this data format change.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ROOT CAUSE of 0% win rate: The experience kernel read raw_close from tgt[2] and
raw_next from tgt[3], but the DBN data loader produces 2-element target vectors
[current_close, next_close] which get zero-padded to 4 elements. So tgt[2:3]=0,
triggering the degenerate price guard (raw_close=1.0, raw_next=1.0), making
every trade's P&L exactly zero → classified as loss.
Fix: read tgt[0] and tgt[1] which contain the actual raw prices.
Result: wins=264/594 (44.4% win rate), PF improving 0.44→0.88→0.98 over 3 epochs.
Also: segment-based trade detection correctly counts reversals,
old_pos_pnl uses saved pre-update position for correct P&L computation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Trade detection now counts reversals (S100→L50) as completed segments
- old_pos_pnl saved before position update for correct reversal P&L
- realized_pnl writeback uses old position PnL on reversal bars
- 0% win rate persists — needs deeper investigation (likely tx cost interaction)
WIP: The trade_return formula produces correct sign for raw market moves,
but every trade still shows as a loss. Suspect tx costs on both entry AND
exit of each reversal segment exceed the 1-bar price movement.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Task 9: Document ensemble consensus limitation — ensemble heads live on
FusedTrainingCtx (training-only), not accessible at inference time.
The ensemble already provides value via diversity gradient during training.
Task 10: Add three-phase training pipeline to the epoch loop:
- Phase 1 (Behavioral Cloning): C51 warmup + expert demos (already existed)
- Phase 2 (Full-Stack RL): all features, expert decay (already existed)
- Phase 3 (Refinement, last 20%): force expert_ratio=0, shrink-and-perturb
at phase boundary for plasticity consolidation
Task 11: Expand hyperopt search space from 41D to 45D with 4 new dims:
- her_ratio [0.0, 0.5]: HER relabeling ratio (was hardcoded 0.0)
- curiosity_weight [0.0, 0.2]: intrinsic reward weight (was fixed 0.0)
- use_cvar_action_selection [0.0, 1.0]: risk-aware IQN action scoring
- cvar_alpha [0.01, 0.2]: CVaR confidence level
All wired through build_hyperparams to DQNHyperparameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Task 7: Wire ensemble diversity gradient through value head into shared trunk.
The KL gradient kernel was computing d_logits but gradient flow stopped there.
Now: d_logits → cuBLAS backward W_v2 → ReLU mask → backward W_v1 → d_h_s2 →
trunk backward (layers 2,1) → SAXPY(diversity_weight) into grad_buf.
Uses launch_dx_only for value head layers (skip dW/db — only d_h_s2 needed).
graph_adam sees combined C51 + IQN + ensemble diversity in single update.
Task 8: Decision Transformer Phase 1 already fully implemented. DT runs before
main training loop when dt_pretrain_epochs > 0: builds trajectories from GPU
data, runs pretrain_step per epoch/batch, logs loss. Verified and confirmed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace reward v4 (pure mark-to-market return) with reward v5, a two-component
trade-aware hybrid that separates dense per-bar signal from sparse trade-exit signal:
Dense (every bar, weight 0.1): raw_pnl / equity when in a trade, zero when flat.
Keeps gradients flowing without overwhelming the sparse trade completion signal.
Sparse (at trade exit, weight 2.0): trade_return * patience_multiplier where
patience = sqrt(hold_time / expected_hold). Regime-adaptive expected hold via
ADX: trending (ADX>30) = 20 bars, ranging (ADX<20) = 8 bars, default = 12 bars.
Dynamic trailing stop: regime-adaptive trail distance (0.5% base, widens with
volatility via CUSUM and trend via ADX). Activates when trade is profitable and
held > 2 bars. Locks in profits by forcing exit when unrealized P&L drops below
the trailing floor.
Also fixes kernel signature mismatch: removes 7 old reward v2 parameters
(w_dsr, w_pnl, w_dd, w_idle, dd_threshold, time_decay_rate, eta) that were
already removed from the Rust launcher in a prior commit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the 8-component dense shaping + sparse trade completion with:
reward_t = (equity_t - equity_{t-1}) / equity_{t-1}
- Losing bars get NEGATIVE reward (every bar, not just exit)
- Flat bars get ZERO reward
- Trade entry: tx_cost hits cash → immediate negative reward
- Loss aversion: losses weighted 1.5x (prospect theory)
- No more DSR, no dense shaping, no hold penalty
NOTE: v_range needs recalibration for percentage returns (~0.001/bar)
instead of the old reward scale (~0.01-2.0). Current v_range=42 is
1000x too wide for the new reward magnitude.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Risk management (CVaR, conviction, Kelly) re-enabled as ENVIRONMENT PHYSICS:
- Agent observes scaling via portfolio state features
- Learns to account for risk limits in its policy
- No longer destroys credit assignment (scaling is physics, not action override)
- Kelly uses half-Kelly (0.5x) for safety, activates after 20 trades
2. Backtest tx_cost now uses training's transaction_cost_multiplier from hyperopt
(was hardcoded 0.1 bps — 17x lower than training). Training and eval see same costs.
3. Backtest env tx_cost formula expanded to match training:
- Square-root market impact (Almgren-Chriss)
- Order-type premiums (Market=0, IoC=+2bps, LimitMaker=-5bps)
Result: first POSITIVE Sharpe (+0.0838) in project history. 134K trades.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
