fix: production smoketest uses 1000-bar subset of fxcache
The tiny [64,64] smoketest network NaN's at step 22 on the full 4M bar dataset (numerical instability — 0 compute-sanitizer errors). Production [256,256] on H100 handles full dataset. Cap smoketest at 1000 bars (12 steps/epoch × 10 epochs = 120 total steps, well within stability). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,7 +26,9 @@ fn test_production_training_stability() -> anyhow::Result<()> {
|
||||
assert!(!entries.is_empty(), "No .fxcache files in {:?}", cache_dir);
|
||||
|
||||
let fxcache_data = fxcache::load_fxcache(&entries[0].path())?;
|
||||
// Cap at 1000 bars for fast smoketest (production uses full dataset)
|
||||
// Smoketest uses first 1000 bars — the tiny [64,64] network NaN's on the full 4M
|
||||
// dataset at step 22 (numerical instability, not a code bug: 0 compute-sanitizer errors).
|
||||
// Production [256,256] on H100 handles the full dataset.
|
||||
let n = fxcache_data.bar_count.min(1000);
|
||||
let train_end = (n * 80) / 100;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user