perf: batch_size 16384 → 8192 — 2x faster GEMMs, more frequent updates

At 16384, each GEMM saturates 97% of 132 SMs — no room for multi-stream
parallelism. At 8192, GEMMs use ~50% SMs, allowing branch streams and
aux parallelism to fill idle SMs. 2x more steps/epoch but each ~2x faster.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
jgrusewski
2026-04-19 09:01:34 +02:00
parent 4d76d22d24
commit c2edb273ca

View File

@@ -1,6 +1,6 @@
# H100 PCIe/SXM (80GB VRAM) -- full production
[training]
batch_size = 16384
batch_size = 8192
num_atoms = 52
buffer_size = 500000
hidden_dim_base = 256