The GPU PER replay buffer had a hardcoded 4 GB MAX_BYTES limit that
rejected the auto-sizer's 10M-entry proposal on H100 (80 GB VRAM).
Now per_max_buffer_bytes() computes 20% of total VRAM (min 1 GB) and
flows through OptimalReplayConfig → DQNConfig → GpuReplayBufferConfig
so both subsystems agree on the budget.
Also fixes misleading regime detection log (indices 211/219 → 40/41)
and renames dqn_config_2025 → dqn_default_config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents experience collector output tensors from starving backward
graph and GPU PER on VRAM-constrained GPUs like RTX 3050 Ti 4GB.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
free_memory_mb is sampled after model/optimizer load, so subtract only
backward-graph + fragmentation + data upload headroom (400 MB), not the
full model/PER/data stack. Increase collector share to 50%.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Staging/GPU replay buffer: truncate batch when it exceeds ring buffer
capacity (CPU fallback can flush 100K+ experiences at once)
- GpuExperienceCollector: compute shmem tile rows dynamically to stay
under 48 KB default limit instead of hardcoded 64-row constant
- Auto batch size: subtract concurrent VRAM consumers (~530 MB model +
optimizer + PER + data) before budgeting experience collector at 40%
- Hyperopt DQN adapter: GPU PER always on, include replay buffer in
VRAM estimate for small GPUs (was excluded assuming CPU-only PER)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU experience collector:
- Add GpuHardwareInfo with SM count detection from device name lookup
(H100=132, A100=108, L40S=142, RTX 4090=128, etc.)
- optimal_n_episodes() scales to GPU: sm_count × 2 warps × 32 threads,
capped by 15% free VRAM budget, 256-aligned for block scheduling
- Remove hardcoded .min(256) cap in trainer — auto-scales from 128 to 8192
- MAX_EPISODES_LIMIT raised from 256 to 4096
Numerical stability:
- BF16 cross-entropy epsilon: 1e-8 → 1e-4 (below BF16 precision floor
1e-8 rounds to zero, making log-stabilization a no-op → -inf → NaN)
- Add log_probs.clamp(-20, 0) guard against -inf × 0 = NaN in loss
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move memory_optimization/ (4.6K lines, 7 files) from ml to ml-core
- Move batch_size_resolver.rs to ml-core (now both deps in same crate)
- Add tempfile to ml-core dev-dependencies (qat tests)
- Re-export both modules from ml facade
- Keep checkpoint/ in ml (model_implementations.rs has model-specific deps)
185 ml-core tests + 2573 ml tests = 2758 total, 0 failures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
