Files
foxhunt/docs
jgrusewski db9936b9ff fix(data): align fxcache data_source + normalise DBN fallback
Two-part fix for a class of bugs causing un-normalised features to
silently flow into training.

(1) data_source mismatch between precompute writer and trainer reader.

  precompute_features.rs:214,633 hardcoded "ohlcv"
  train_baseline_rl.rs:582       hardcoded "ohlcv"
  config/training/dqn-production.toml: data_source = "mbp10"

  Production runs the trainer with the production profile (data_source
  = mbp10), but the actual cache lookup hardcoded "ohlcv". Smoke worked
  by accident (smoke profile is also "ohlcv"). Any future profile with
  a different data_source silently mismatches → cache MISS → DBN
  fallback path.

  Both call sites now hardcode "mbp10" (the canonical production data
  source per CLAUDE.md). precompute_features adds a `--data-source`
  CLI override for the rare case a smoke flow needs to regenerate
  the local "ohlcv" fxcache; default is "mbp10".

(2) DBN-fallback path didn't normalise features.

  precompute_features.rs:629 applies NormStats::normalize_batch on the
  canonical fxcache write path. The fallback in train_baseline_rl.rs
  (cache-miss → load DBN files → extract features → upload to GPU) did
  NOT normalise. Any cache miss (data_source drift, schema-hash mismatch,
  missing file) silently uploaded RAW features. Raw close prices
  (~$5180 ES futures) flowed into next_states[:, 0]; the aux next-bar
  head's label_scale EMA latched onto raw-price magnitude (~5443 vs
  expected ~1.0 z-score); the shared trunk learned to predict next-bar
  prices; the policy effectively traded with future-price knowledge →
  train-h5gxb epoch-0 Sharpe = 141 with 0.32% max-drawdown over 214k
  bars (impossibly good = oracle leak).

  DBN fallback now applies the same z-score normalisation unconditionally
  as defence-in-depth, so a future cache-miss cannot reintroduce raw
  values into training.

Audit entry updated.
2026-04-27 14:21:42 +02:00
..