Files
foxhunt/docs
jgrusewski cb69e410ea fix(fxcache): track precompute_features.rs in FEATURE_SCHEMA_HASH
build.rs::emit_feature_schema_hash only hashed
src/features/extraction.rs, src/fxcache.rs, and
../ml-core/src/state_layout.rs. The z-score normalization step lives
in examples/precompute_features.rs:625-631 (added 2026-04-03 in
9f7c14978f) and was NOT covered by the hash. A fxcache written by an
older precompute build (raw features, no normalization) silently
passed today's validate() because every other field matched.

Empirical impact (L40S Argo train-f8h6q, 2026-04-27):

  - PVC fxcache: stale, written pre-normalization → feature column 0
    contains RAW CLOSE PRICES (~$5180 ES futures) instead of z-
    normalized log-returns
  - aux head reads next_states[:, 0] as its next-bar regression
    label (gpu_dqn_trainer.rs:7758-7789)
  - EMA label_scale climbed to 5420 (vs smoke 0.05) → shared trunk
    learned to predict next-bar prices → policy effectively traded
    with future-bar information
  - epoch-0 Sharpe = 141.99 with 0.32% max-drawdown over 214k bars
    — physically impossible; clear future-leak signature

Fix adds examples/precompute_features.rs to schema_sources. New hash
invalidates the stale PVC cache. Argo's ensure-fxcache step has a
regenerate-on-failure branch (infra/k8s/argo/train-template.yaml:
372-383) that auto-regens with current normalized precompute.

Generalises beyond this incident: any future change to feature
normalization, target ordering, or precompute post-processing now
bumps the hash and forces fxcache regen.

Audit entry updated.
2026-04-27 12:53:39 +02:00
..