Commit Graph

187 Commits

Author SHA1 Message Date
jgrusewski
f4ea0f6d13 test(policy-quality): Task 0.16 surrogate_noise_check smoke (mandatory gate)
Adds evaluate_baseline CLI args:
  --surrogate-mode=off|random
  --surrogate-seed=<u64>
  --surrogate-marginals=<path.json>
  --emit-action-marginals
  --emit-pooled-sharpe

Random-action surrogate samples actions from marginal distribution (or
uniform fallback) using a seeded RNG, bypassing the model entirely.
Surrogate mode forces the CPU DQN path so action selection can be
overridden (GPU path would need invasive kernel changes and defeats
the bypass-the-model sanity check).

Flat action-index counts are tracked in the CPU DQN path only; the
ACTION_MARGINALS: line emits those as a JSON distribution, or emits
an honest stub marker when only the GPU path was exercised. Pooled
Sharpe is computed from concatenated per-fold returns (CPU path) or
falls back to mean-of-fold-Sharpes with a warning.

Smoke test runs 30 surrogate seeds + 1 trained run via evaluate_baseline
subprocess, asserts trained pooled Sharpe exceeds the 95th percentile of
surrogate Sharpes. Test is #[ignore]'d and requires a trained checkpoint
at /workspace/output/dqn_fold0_best.safetensors (Phase 3 deliverable);
FOXHUNT_SURROGATE_CKPT env var overrides for local testing. Uses the
compiled target/release/examples/evaluate_baseline if present, otherwise
falls back to cargo run.

Will pass once Phase 3 produces a checkpoint.
2026-04-21 23:22:27 +02:00
jgrusewski
4e1a937cec feat+test(policy-quality): Task 0.15 — multi_fold_convergence smoke + --max-folds
Adds --max-folds CLI flag to train_baseline_rl (0 = no cap). When set,
truncates the generated walk-forward fold list to the first N folds.
Used by the new multi_fold_convergence smoke test to run a 3-fold × 20-epoch
local variant of the Phase 3 L40S 6-fold × 50-epoch validation gate
(~5 min on RTX 3050 vs ~1 hour on L40S).

Test asserts:
  * train_baseline_rl subprocess exits 0 (no NaN/Inf)
  * ≥ 2/3 folds produce dqn_fold{N}_best.safetensors checkpoint
    (checkpoint is only written when Best Sharpe improves during the
    fold, so presence = policy learned something on that window)

Per plan Task 0.15 at docs/superpowers/plans/2026-04-21-policy-quality.md.
2026-04-21 21:37:01 +02:00
jgrusewski
f988ca384c fix(train): emit per-fold norm_stats.json for evaluate_baseline
evaluate_baseline looks for <output_dir>/norm_stats_fold{N}.json per fold
(matching the convention written by train_baseline_supervised.rs:812).
train_baseline_rl never produced this file — the fxcache has a single
<hex_key>.norm_stats.json written by precompute_features, which RL
training uses implicitly for pre-normalized features but never copies to
the per-fold paths evaluate wants. Result (L40S train-7r9zf):

  Error: NormStats not found at /workspace/output/norm_stats_fold0.json
  - cannot evaluate without training-set statistics (computing from test
    data would introduce lookahead bias). Run training first to generate
    this file.
  WARN: Evaluation failed, continuing

Evaluate fails gracefully but no report is produced.

Fix: new helper `fxcache::norm_stats_path_for_key(data_dir, override, cache_key)`
returns the canonical <hex_key>.norm_stats.json path that sits next to
the .fxcache file. train_baseline_rl resolves this once after the fxcache
load (falling back to None if the key is zero — i.e., DBN fallback path
with no precomputed stats) and copies it into the output dir as
norm_stats_fold{N}.json for every fold processed.

The RL-side fxcache norm stats are fold-independent (one z-score per
cache key, applied to all walk-forward windows), so the per-fold copies
are intentional duplicates of the same file — this matches evaluate's
per-fold lookup semantics without diverging from supervised convention.

Closes task #32.
2026-04-21 18:09:18 +02:00
jgrusewski
882497caa4 fix: OFI_DIAG reads new canonical indices [42..62), remove state_dim from evaluate_baseline
- OFI_DIAG now reads positions [42..62) using OFI_START constant instead of
  hardcoded [66..74). Verified: raw_mean=0.0891, delta_mean=-0.0370,
  book_agg=0.4500, log_dur=-0.2303 (was all zeros before).
- Removed 3 stale state_dim field initializers from evaluate_baseline.rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 15:33:42 +02:00
jgrusewski
e41c4909f1 fix: validate OFI data content, not just has_ofi flag — v2 cache had all-zero OFI
The v2 fxcache on PVC passed has_ofi=true validation (mbp10_dir was present
when built) but contained all-zero OFI data. The old binary set the flag
based on directory existence, not actual computed values.

Now counts non-zero OFI rows before accepting a cache — forces rebuild
when MBP-10 data is available but OFI content is all zeros.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 12:08:11 +02:00
jgrusewski
4ce2fb7d4b fix: make OFI unconditional, fix *8→*20 dimension bug, remove 882 lines dead code
OFI (Order Flow Imbalance) features are mandatory — the model is worthless
without MBP-10 order book data. Every silent fallback that degraded to zeros
has been converted to a hard error.

Critical fixes:
- build_batch_states used *8 instead of *20 for OFI dimensions — every
  walk-forward backtest was reading corrupted OFI features from adjacent memory
- precompute_features early exit skipped cache rebuild when stale v2/v3
  cache existed with has_ofi=false — now validates has_ofi before skipping
- 14 silent OFI fallback paths converted to hard errors across data loading,
  training loop, experience collector, state construction, metrics, hyperopt

Dead code removed (-751 lines):
- DoubleBufferedLoader (superseded by init_from_fxcache)
- GpuBufferPool (superseded by init_from_fxcache)
- DqnGpuData::upload legacy method (no OFI support)
- CPU training fallback path (CUDA always required)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 11:50:26 +02:00
jgrusewski
429967dcd1 feat: precompute OFI deltas + book aggression + bar duration in fxcache
FXCACHE_VERSION 3→4. Three precomputed features added to OFI region:
- ofi[8..16): temporal deltas (ofi[bar] - ofi[bar-1]) for 8 features
- ofi[16]: book aggression (MBP-10 10-level center-of-mass asymmetry)
- ofi[17]: log bar duration (imbalance bar formation time, normalized)

Previously 10 of 18 OFI embed MLP inputs were zero. Now all 18 have
real data: raw_ofi(8) + delta_ofi(8) + book_aggression(1) + log_duration(1).

Added read_state_sample() diagnostic for GPU state verification.
Legacy v3 fxcache handled by zeroing new slots (v2→v4 graceful upgrade).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 08:11:03 +02:00
jgrusewski
063fd27166 feat: target_dim 4→6 + spec v5 with pearls (bar duration, book CoM, retrospective hold)
target_dim expansion: adds raw_open (OHLCV) and mid_price_open
(MBP-10 midpoint at bar formation) to fxcache targets. FXCACHE_VERSION
2→3 for auto-rebuild. Legacy v2 files handled with close-price fallback.

Spec v5 adds 3 pearls:
- Bar duration encoding in Mamba2 (continuous-time SSM awareness)
- Order book center of mass from all 10 MBP-10 levels (aggression signal)
- Retrospective hold quality bonus (teaches exit timing)

Plus: Hold action (4th direction), DSR Sharpe EMA fix, counterfactual
magnitude/order sign fix, MFT mid-price mark-to-market.

OFI embed MLP now 18→10 (was 16→8). Mamba2 width SH2+10 (was SH2+8).
Attention width D+10 (was D+8).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 23:47:04 +02:00
jgrusewski
64e6353a5d fix: IQN num_quantiles 64→32 in all binaries + production config
Default was hardcoded as 64 in train_baseline_rl.rs and evaluate_baseline.rs,
overriding the config default of 32. Added num_quantiles=32 to production
config so it's explicit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 13:06:58 +02:00
jgrusewski
108bb63fce feat: cost-driven hold timing — replace min_hold_bars with learned cost signals
Removed: enforce_hold(), min_hold_bars from config/kernels/backtest.
Added: holding_cost_rate (inventory penalty), churn_threshold_bars +
churn_penalty_scale (graduated flip penalty) to reward in
experience_env_step and backtest kernels.

The model learns optimal hold timing from cost signals:
- Per-trade tx cost prevents churning (existing)
- Inventory penalty makes large positions expensive to hold
- Churn penalty graduates cost for rapid flips
- Temporal attention learns when holding cost > expected profit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 01:04:11 +02:00
jgrusewski
30e94446cd fix: compute bars_per_day from fxcache timestamps — correct annualization
bars_per_day was hardcoded 390 (1-minute candles) but we use MBP-10
imbalance bars (~7500/day). All Sharpe/Sortino/return annualization
was off by sqrt(7500/390) ≈ 4.4×.

Now computed from actual fxcache timestamps:
  bars_per_day = total_bars / (trading_days from timestamp span)

Propagates to: val_Sharpe, train_Sharpe, financials, backtest evaluator.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 20:46:04 +02:00
jgrusewski
d78fe9d4a8 fix: fxcache version display + export FXCACHE_VERSION + cleanup dead code
precompute_features prints Format: f32 (v2), OFI_DIM=20 (was hardcoded v1).
FXCACHE_VERSION exported as pub const for external use.
graph_adam_m/v/step removed (dead buffers). q_mean_ema_ptr removed
from c51_grad_kernel (ISV replaced Q-drift penalty).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 02:12:33 +02:00
jgrusewski
8b68954ded feat(tick): precompute pipeline wires MicrostructureState — 20 OFI features
12 new tick-level features computed per bar from MBP-10 snapshots:
OFI trajectory, realized variance, Hawkes intensity, book pressure,
spread dynamics, aggression, queue depletion, order count flux,
intra-bar momentum, regime score, OFI acceleration, toxicity gradient.
Combined with existing 8 OFI into [f64; 20] per bar in fxcache.

Also fixes pre-existing bug: no-MBP-10 fallback was [0.0; 8] not [0.0; 20].

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 00:29:03 +02:00
jgrusewski
81771d7920 feat(tick): OFI_DIM 8→20 — atomic update across 15 files
fxcache: OFI_DIM=20 (pub const), RECORD_F64_COUNT=66, 272 bytes/bar.
constructor: state_dim 74→86 (aligned 88) with OFI.
experience collector: ofi_dim detection 8→20.
All [f64; 8] → [f64; 20]. Existing 8 features preserved at 0-7.
New 12 features zero-padded until precompute is extended.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 00:21:15 +02:00
jgrusewski
c74a687ea8 feat: position-gated episodes + 5000-bar limit — close 45x training/val gap
Episode done flag: timer-based -> position-gated (trade complete = done).
V(flat)=0 is correct terminal anchor. Soft reset keeps equity on
trade completion; hard reset only on data-end or capital breach.
H100: 100 bars -> 5000 bars, gpu_n_episodes -> 1024.
ExperienceProfile gains optional gpu_n_episodes field.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 21:56:44 +02:00
jgrusewski
61f1b0a1d2 fix: evaluate_baseline compile — make q_provider Optional, remove dead evaluate_dqn path
evaluate_dqn_graphed now takes Option<&mut dyn QValueProvider>.
Training eval passes Some(fused_ctx), standalone eval uses closure path.
Removed dead evaluate_dqn non-graphed branch from evaluate_baseline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:05:01 +02:00
jgrusewski
31c610b69a WIP: training quality fixes + VarStore elimination (INCOMPLETE — needs xavier_init_params_buf)
Phase 1 — Training quality (14 structural bugs):
- Reward: sqrt micro-reward scaling, remove sparse normalization, Kelly penalty,
  additive OFI, magnitude-scaled capital floor penalty
- Gradient: CQL 25→10%, IQN 60→75%, MSE magnitude 4×, 3-phase counterfactual,
  tau_anneal 100K→500K, epsilon floor 2%, PopArt fold reset, dd_threshold 0.5%
- Monitoring: per-branch Q-value instrumentation
- Hold: min_hold_bars 5→1

Phase 2 — VarStore elimination:
- Deleted copy_weights_from from all network types
- Deleted to_varstore from DuelingQNetwork + DistributionalDuelingQNetwork
- Deleted branching_to_varstore, 7 extract/sync VarStore functions
- Deleted update_target_networks, legacy CPU training paths
- Deleted iqn_target_network, target_network fields
- Deleted gpu_kernel_parity_test.rs (705 lines)
- Refactored GpuExperienceCollector to single flat buffer
- Added weight_sets_from_branching() zero-copy replacement
- EMA tau=1.0 init for target_params_buf

KNOWN ISSUE: params_buf is zero-initialized but from_flat_buffer() creates
weight set pointers into it BEFORE flatten_online_weights runs. The online
forward reads zeros → NaN. Needs xavier_init_params_buf() at construction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:03:05 +02:00
jgrusewski
9f18772527 fix: update all test/example files for VarStore removal + 7-level exposure
Add to_varstore() compatibility shims on DuelingQNetwork and
DistributionalDuelingQNetwork so test/example code can rebuild a
GpuVarStore snapshot when needed. Delete dead tests that referenced
removed DQNAgent, PrioritizedReplayBuffer, and ReplayBufferType.
Fix action index references (action_19/21 -> action_28/30) and
type annotation issues (sin ambiguity, remainder operator).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 15:21:21 +02:00
jgrusewski
1c3eb3e331 refactor: eliminate GpuVarStore — DuelingWeightSet is zero-copy pointer view into params_buf
DuelingWeightSet and BranchingWeightSet fields changed from owned
CudaSlice<f32> to raw u64 device pointers + usize element counts.
This enables zero-copy views directly into the flat params_buf for
the training hot path (no D2D copies, no shadow allocations).

Key changes:
- Weight sets store u64 + usize pairs instead of CudaSlice<f32>
- from_flat_buffer() creates zero-copy views into params_buf
- from_slices() creates pointer views from owned CudaSlice allocations
- DuelingWeightBacking/BranchingWeightBacking hold owned CudaSlice
  arrays for callers that need independent allocations (experience
  collector, ensemble heads, tests)
- flatten/unflatten become no-ops when weight sets point into params_buf
- extract functions return (Backing, WeightSet) tuples
- KernelWeightPack::build() uses u64 fields directly (no raw_device_ptr)
- All sync functions updated for pointer-based signatures
- Ensemble clone functions return backing + pointer view pairs
- gradient_budget tests updated (7/7 pass)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 14:02:55 +02:00
jgrusewski
448b61d095 refactor: collapse 9-level to 7-level ExposureLevel — eliminate degenerate Flat variants
The 4-branch DQN (direction x magnitude) had 3 degenerate variants
(Short25, Flat, Long25) that all mapped to 0.0 target exposure when
direction=Flat, causing 82% Flat collapse. Collapse these into a
single Flat variant, giving 7 levels (ShortSmall/Half/Full, Flat,
LongSmall/Half/Full) and 63 total factored actions (7x3x3).

- ExposureLevel enum: 9 variants -> 7 (add direction/magnitude/from_dir_mag)
- FactoredAction: 81 -> 63 total actions, from_index/to_index updated
- DQN epsilon-greedy: use from_dir_mag() instead of dir*3+mag indexing
- DQN config: num_actions default 9 -> 7
- PPO action space: 45 -> 63 actions, action masking updated
- Signal adapter CUDA kernel: 5-bin -> 7-bin exposure aggregation
- All tests updated for new variant names and index ranges

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:54:09 +02:00
jgrusewski
21abd2b0f9 fix: evaluate_baseline bf16→f32 remnants, delete old fxcache files
- Fix f32::from_f32/f32::ZERO (nonexistent) in evaluate_baseline.rs
- Eliminate redundant Vec copies (upload source data directly)
- Fix comments referencing __nv_bfloat16 in reductions.rs, training_guard_kernel.cu
- Delete 3 old bf16-format .fxcache files from test_data/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:57:36 +02:00
jgrusewski
4842a04011 refactor: eliminate ALL bf16 from entire workspace — pure f32/TF32 pipeline
Complete bf16 elimination across all crates (ml, ml-core, ml-dqn, ml-ppo,
ml-supervised). Zero half::bf16, __nv_bfloat16, or CudaSlice<half::bf16>
references remain (verified by grep).

CUDA: All 60+ .cu kernels and .cuh headers converted to native float.
  - Half-precision intrinsics (__hmul, __hadd, __hdiv) → float operators
  - atomicAddBF16 → native atomicAdd(float)
  - bf16 wrapper functions → f32 identity passthroughs

Rust: All CudaSlice<half::bf16> → CudaSlice<f32> across 90+ files.
  - htod_f32_to_bf16/dtoh_bf16_to_f32 → htod_f32/dtoh_f32 (direct, no conversion)
  - Deleted bf16 mirror infrastructure (DuelingWeightSetBf16, alloc_bf16_mirror, etc.)
  - Renamed params_bf16→params_flat, d_value_logits_bf16→d_value_logits, etc.
  - Fixed .to_f32() sed damage on Decimal::to_f32() and rng.f32()

FxCache: Single f32 disk format (was bf16/f64 dual-version).
  - Deleted --bf16 CLI flag from precompute_features
  - PVC cache files need regeneration via precompute_features

TF32 tensor cores activated via cublasLtMatmul CUBLAS_COMPUTE_32F — no
explicit TF32 types needed. Storage is pure f32 everywhere.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:52:21 +02:00
jgrusewski
00bbe76b05 refactor: fxcache single f32 format — delete bf16 version, remove --bf16 CLI flag
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 18:45:49 +02:00
jgrusewski
a2370d7f33 fix: evaluate_baseline example bf16→f32 closure signatures
The evaluate closures now receive &CudaSlice<f32> from the backtest
evaluator. Updated all 3 closures (DQN, PPO, supervised) to match.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:02:13 +02:00
jgrusewski
fea6371af7 fix(data): filter front-month PER FILE, not globally — each quarterly DBN has different contract
Global filter selected only ONE quarter's instrument_id, producing 3 months
of data instead of 2+ years. Now filters within each file so quarterly
rolls are handled correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 21:27:54 +02:00
jgrusewski
ced178812a feat(data): precompute + data_loading use trades volume bars instead of broken OHLCV
Replace extract_ohlcv_bars_from_dbn() with trades-based volume bar pipeline
(load_trades_sync -> filter_front_month -> build_volume_bars) in both the
precompute binary and the runtime fallback path, eliminating fake price jumps
from interleaved contract months in OHLCV DBN files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:36:05 +02:00
jgrusewski
bacaf2765a feat: leverage-based position cap (replaces hardcoded max_position)
max_position_absolute is now computed from max_leverage:
  max_position = floor(capital * max_leverage / (price * multiplier))

- Added max_leverage config field (default: 5.0)
- compute_max_position() derives position from leverage + median price
- Hyperopt risk_intensity scales max_leverage (not position directly)
- Updated all TOML configs: dqn-production, dqn-smoketest, dqn-localdev
- Hyperopt search space: max_leverage = [2.0, 10.0] (replaces [1.0, 4.0] contracts)
- GpuBacktestConfig wired with max_leverage for consistent eval

With $35K capital, ES at $5K, multiplier=50:
  5× leverage → floor(35000*5/250000) = 0.7 → 1 contract (safe)
  Old default 2.0 contracts → 14× leverage (dangerous)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 16:24:20 +02:00
jgrusewski
9c081ac7ce refactor: CLI binaries use shared fxcache::discover_and_load()
Replace 40-line inline fxcache discovery block in train_baseline_rl.rs
with a single call to ml::fxcache::discover_and_load(). precompute_features.rs
already uses correct symbol/data_source args — no changes needed there.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 10:05:26 +02:00
jgrusewski
8b0122a5ac feat: has_ofi flag in fxcache header (explicit, no zero-detection)
Adds `has_ofi: bool` to `FxCacheData` and the fxcache binary header
(stored in `reserved[0]`). Propagates through load_fxcache, write_fxcache,
all callers (precompute_features, train_baseline_rl, hyperopt dqn adapter),
existing roundtrip tests, and adds two new has_ofi-specific roundtrip tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 09:48:38 +02:00
jgrusewski
fccfe1b0d6 feat: cache key includes symbol + data_source (prevents cross-instrument collisions)
Different instruments (ES.FUT vs NQ.FUT) and data source modes (ohlcv vs mbp10)
now produce distinct .fxcache keys, preventing silent overwrites and wrong-bar-type
loads at cache lookup time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 09:41:53 +02:00
jgrusewski
74c62eb5dc fix: fxcache key uses data_dir (not symbol_dir) to match precompute
The cache key in train_baseline_rl was computed from data_dir/symbol
(e.g. test_data/futures-baseline/ES.FUT) but precompute_features uses
data_dir (test_data/futures-baseline). Different paths → different keys
→ cache miss → OFI=false.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 01:26:01 +02:00
jgrusewski
b1fa34c5eb feat: --feature-cache-dir CLI arg for training binaries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 22:59:08 +02:00
jgrusewski
047c01dcfa fix: precompute skips if cache exists + integrated into compile-and-train
- precompute_features early-exits if {cache_key}.fxcache already exists
  (cache key = SHA256 of filenames + sizes, only changes when data changes)
- compile-and-train copies binaries to PVC /data/bin/ after compile
- precompute step added to compile-and-train DAG (runs after compile,
  before training, parallel with GPU warmup)
- precompute template reads binary from PVC (no GitLab download)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:43:41 +02:00
jgrusewski
b7840fc978 fix: filter DBN files by --symbol in precompute (was mixing ES+NQ+ZN+6E)
collect_dbn_files_filtered() checks for symbol subdirectory first,
falls back to filename matching. Precompute now loads only the
target symbol's data instead of all 4 instruments mixed together.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 18:20:42 +02:00
jgrusewski
9f7c14978f feat: z-score normalization at precompute time + remove per-fold normalization
Normalize features once in precompute_features (single source of truth).
NormStats saved alongside .fxcache for inference denormalization.
Removed per-fold NormStats computation from train_baseline_rl, hyperopt
adapter, and smoketests — fxcache is pre-normalized, consumers use as-is.

NOTE: features still show raw price values (max=18000+) because
test_data/futures-baseline contains multiple symbols (ES, NQ, ZN, 6E)
mixed into one dataset. The feature extraction pipeline needs
investigation — log returns between different symbols produce garbage.
This is tracked as a separate task.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 17:48:26 +02:00
jgrusewski
a448e03e64 fix: 6 CUDA OOB bugs + remove 1228 lines of dead training path
Root cause of NaN at step 22: cuBLAS out-of-bounds reads in the GPU
experience collector and training forward pass, caused by buffer
dimension mismatches that produced garbage states → garbage Q-values →
NaN loss.

Bugs fixed:
1. GpuDqnTrainConfig::default() had bottleneck_dim=2, market_dim=42 —
   collector computed s1_input_dim=40 instead of state_dim=80, cuBLAS
   read 2x past W1 weight buffer (10KB OOB per SGEMM)
2. batch_states allocated with state_dim=80 but cuBLAS ldb=128 (CUTLASS
   K-tile alignment) — 48 bf16/f32 values read past end per row
3. Validation state tensor had 59 elements/row (42+3+8+6) instead of
   128 (state_dim_padded) — missing portfolio/MTF features + padding
4. CUTLASS K-tile overread on all hidden activation buffers (K=64 but
   tile=128) — added 128-element padding to 14 bf16 buffers
5. init_from_fxcache never set self.ofi_features — collector OFI buffer
   was 1-element placeholder, state_gather kernel read OOB
6. collect_gpu_experiences_slices missing *2 counterfactual multiplier —
   half of collected experiences never inserted into PER

Dead code removed (~1228 lines):
- train_with_data_full_loop (old Vec<(FeatureVector, Vec<f64>)> loop)
- init_gpu_data, init_gpu_raw_buffers, collect_gpu_experiences,
  run_training_steps (old helpers only called by above)
- train_with_preloaded_data, train_with_shared_data (old hyperopt APIs)
- train() and train_walk_forward() now convert to fixed-size arrays
  and route through init_from_fxcache + train_fold_from_slices

Other fixes:
- precompute_features defaults output-dir to sibling of data-dir
- workspace_root() + feature_cache_dir() helpers for reliable path
  resolution (works from any cwd, with/without CARGO_MANIFEST_DIR)
- Production smoketest uses fxcache (no slow DBN parsing)
- compute-sanitizer: 0 errors (was 2539)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:22:46 +02:00
jgrusewski
95fc94ae9c fix: precommit review — ensemble reuse, keep gpu_data across folds, eliminate Vec<f64>
Fix 1 (HIGH): Ensemble trainers were created + data uploaded inside the
fold loop, causing redundant GPU uploads per fold per ensemble member.
Moved creation + init_from_fxcache before the fold loop; inside the loop
we now reuse trainers via set_training_range + reset_for_fold.

Fix 2 (MEDIUM): reset_for_fold cleared gpu_data = None, forcing re-upload
on every fold even though the full dataset was already GPU-resident via
init_from_fxcache. Removed the clearing — data stays on GPU across folds.

Fix 3 (LOW): train_fold_from_slices allocated a Vec<f64> per bar via
t.to_vec(). Added train_with_data_full_loop_slices that accepts
&[([f64; 42], [f64; 4])] — both are Copy stack types, zero heap alloc
per element. Added matching collect_gpu_experiences_slices and
run_training_steps_slices helpers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 23:21:45 +02:00
jgrusewski
3278cec29b perf: PPO zero-copy fold loop — train_from_slices + delete train_ppo_fold
Add PpoTrainer::train_from_slices(&[[f64; 42]]) that converts to
Vec<Vec<f32>> internally (PPO train() API requires ownership). Replace
the PPO fold loop in train_baseline_rl.rs: eliminates the OHLCVBar
construction shim and calls train_from_slices directly on fxcache
feature slices. Delete the standalone train_ppo_fold function entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 22:54:00 +02:00
jgrusewski
8d5af977f7 feat: train_fold_from_slices — trainer accepts contiguous feature/target slices
Adds DQNTrainer::train_fold_from_slices(&[[f64;42]], &[[f64;4]]) so the
fold loop in train_baseline_rl passes raw fxcache slices directly, without
the caller constructing any Vec<(FeatureVector, Vec<f64>)>. Removes
features_to_trainer_format_fast helper (no longer needed) and updates
train_dqn_fold to call the new method.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 22:50:11 +02:00
jgrusewski
5c985df147 perf: zero-copy fold loop — fxcache to GPU once, index slices per fold
Rewrites the train_baseline_rl fold loop to eliminate per-fold waste:

- Data loading: fxcache -> fold index ranges from timestamps (no bar
  reconstruction). DBN fallback builds FxCacheData in-place.
- DQN trainer created ONCE before fold loop, fxcache uploaded to GPU
  ONCE via init_from_fxcache. Each fold uses set_training_range +
  set_val_data_from_slices + reset_for_fold instead of recreating.
- Tokio runtime created ONCE (not per fold).
- Hyperparams construction extracted to build_dqn_hyperparams().
- Deleted: prepare_fold_data, FoldData type, prefetch thread,
  DoubleBufferedLoader GPU staging, features_to_trainer_format (old).
- Added: generate_walk_forward_indices_from_timestamps (i64 ns
  timestamps, O(log n) partition_point, no OHLCVBar dependency).
- Added: features_to_trainer_format_fast (fxcache targets directly).
- PPO compatibility preserved: constructs minimal OHLCVBars from
  fxcache targets for train_ppo_fold (Task 6 will refactor).
- Ensemble mode preserved with per-member trainers for k>0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 22:46:30 +02:00
jgrusewski
f5b21406a8 fix: restore batch_size to GPU profiles — profile defaults, not auto-scaling
Auto-scaling returned 8192 on 4GB RTX 3050 (should be 64), causing
17s/epoch instead of 0.28s. The VRAM math didn't account for IQN (1.1GB),
attention, IQL, replay buffer (70% VRAM).

Profile-tested values:
- RTX 3050: 64 (4GB, minimal)
- A100: 2048 (40-80GB)
- H100: 8192 (80GB, production)
- Default: 256 (conservative)

Auto-scaling remains as fallback for batch_size=0 (unknown GPU).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:26:24 +02:00
jgrusewski
f11c263700 chore: remove per-step debug eprints, keep per-epoch only
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 18:26:03 +02:00
jgrusewski
1f43d8c3a6 debug: add fxcache key debug + fix PREFETCH_K hang
- Print fxcache lookup key to diagnose cache miss on H100
- PREFETCH_K=16 (was usize::MAX causing 8M PER samples in one shot)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 18:06:36 +02:00
jgrusewski
b7735f2e9e debug: add eprintln traces for H100 hang — fxcache, experience collection, training phases 2026-04-02 16:58:16 +02:00
jgrusewski
9aad6ff60e refactor: remove max_training_steps_per_epoch — always train full dataset
Epoch duration self-balances: bigger GPU → bigger auto-scaled batch →
fewer steps per epoch. The manual cap created 7 different values
(0, 8, 64, 100, 200, 300, 2000) across configs/tests/examples, making
behavior inconsistent between environments.

Removed from: DQNHyperparameters, training profiles (smoketest,
localdev, production), CLI args, Argo templates, hyperopt adapter,
all test overrides, supervised example.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:41:16 +02:00
jgrusewski
d353af98e9 chore: clean up dead batch_size override code in train_baseline_rl
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:34:24 +02:00
jgrusewski
1321e11fb2 refactor: remove batch_size CLI override from train_baseline_rl
All VRAM-derived parameters (batch_size, gpu_n_episodes, buffer_size)
are auto-scaled — CLI overrides bypass this and cause inconsistent
behavior between local testing and production.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:33:00 +02:00
jgrusewski
5546a45bc3 refactor: remove gpu_n_episodes override — auto-scale from VRAM everywhere
gpu_n_episodes was manually overridden in GPU profiles, training configs,
test files, and hyperopt — all set to 0 or small fixed values that
bypassed the auto-scaling logic, causing a div-by-zero crash in
train_baseline_rl.

Now: single auto-scaling path via optimal_n_episodes() from VRAM/SM
count. No manual override field. Cap at 16384 (consistent with
AutoBatchSizer's 8192 cap pattern). Floor at 32 for small GPUs.

Removed gpu_n_episodes from:
- DQNHyperparameters, PpoHyperparameters structs
- All 4 GPU profiles (rtx3050, h100, a100, default)
- Training profiles (smoketest, localdev)
- ExperienceProfile struct + serde
- Hyperopt adapter
- All test overrides

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 14:21:45 +02:00
jgrusewski
5b06484da7 fix: 11 H100 training bugs — Sharpe per-trade, batch autosizing, PER/HER capacity, Q-clip
Sharpe calculation:
- Use per-trade returns (sum_returns/sum_sq_returns) instead of per-bar
  step_returns. Per-bar Sharpe collapsed variance → bogus 19.75 with PF=0.03.
- Annualize by sqrt(trades_per_year) not sqrt(bars_per_year).

Batch sizing:
- Cap auto-computed batch_size at 8192 (VRAM ceiling of 2M is OOM limit, not
  optimal RL batch).
- Add VRAM floor: batch_size < ceiling/4096 gets scaled UP (128 → 512 on H100).
- Only let hyperopt override batch_size when explicitly non-zero — preserve
  profile's batch_size=0 auto-compute sentinel.

Replay buffer:
- Divide per_max_memory_bytes by 3.0 for regime heads (PER budget was 3x too
  large, causing OOM cascade 74M → 37M → 18M → 9M → 4.6M).
- HER buffer uses original_buffer_size (pre-autosizer), not inflated 74M.

Q-value clipping:
- Wire hyperparams.q_clip_min/max to DQNConfig (was hardcoded ±500, production
  TOML has ±50). Prevents Q-value overestimation ratio of 94.6x at epoch 2.

Training stability:
- Anti-LR warmup: skip first 5 epochs (early Sharpe unreliable from random
  policy). Prevents bogus 3x LR boost at epoch 2.
- min_replay_size from profile (1000), not hyperopt batch_size (128).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 00:54:12 +02:00
jgrusewski
f9609baff6 fix: train-baseline-rl compiles in-cluster, requires fxcache
- DAG: compile (ci-compile-cpu) + gpu-warmup → train (H100)
- Compiles from source targeting compute cap 90
- fxcache REQUIRED — fails if no .fxcache found
- Error chains printed with {:#} for full cause visibility
- Uses cargo-target-pvc for compile cache + sccache

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 20:37:23 +02:00