cublasSgemm was running on pure f32 CUDA cores (~60 TFLOPS on H100),
causing 15× slowdown vs the old bf16 tensor core path (77s vs 5s/epoch).
cublasGemmEx with CUBLAS_COMPUTE_32F_FAST_TF32 + CUBLAS_GEMM_DEFAULT_TENSOR_OP
uses TF32 tensor cores (~500 TFLOPS on H100) — 8× faster than pure f32.
The earlier NOT_SUPPORTED error was caused by bf16 weight pointers feeding
f32 GEMMs (stride mismatch), not API incompatibility. Now that all pointers
are f32, TF32 GemmEx works on both RTX 3050 (SM86) and H100 (SM90).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>