feat(ml): comprehensive GPU saturation audit — 58 fixes across all 10 models

Phase 1 — Fix broken models (P0):
- Diffusion: wire optimizer_step to actually apply gradients (was no-op)
- TLOB: connect forward pass to projection layers (was Tensor::zeros)
- Mamba2: F64→F32 migration across 5 files (~30x faster on L40S tensor cores)

Phase 2 — Eliminate hot-path GPU sync stalls:
- Mamba2: keep dt on GPU in discretize_ssm (4 functions, no CPU round-trip)
- TFT: gate attention weight logging to eval only (8 syncs/forward eliminated)
- Mamba2: defer loss scalar after backward (pipeline stall removed)
- Mamba2: delete dead gradient clipping (4N wasted GPU syncs removed)

Phase 3 — Enable BF16 for supervised models:
- Flip mixed_precision defaults to true in 4 config locations
- Fix cuda_layer_norm to support BF16/F16 via F32 intermediate

Phase 4 — Raise hyperopt bounds for datacenter GPUs:
- 7 adapters with VRAM-aware tiers (TFT, Liquid, TGGN, KAN, xLSTM,
  Diffusion, TLOB) — L40S gets full hidden_dim range
- Fix L40S tier boundary (was excluded at <48000, now >=40000)

Phase 5 — Update memory estimates:
- 10 param_count estimates updated (DQN 200K→12M, TFT 2M→50M, etc.)
- Fix power-of-two rounding (was wasting up to 49% of budget)
- Correct MODEL_OVERHEAD_MB in DQN/PPO/TFT adapters

Phase 6 — Fix per-epoch CPU bottlenecks:
- PPO: deduplicate double advantage normalization (correctness fix)
- PPO: GPU tensor reward normalization + explained variance
- Fuse per-parameter grad norm to single GPU sync (xLSTM, KAN, TGGN)

Phase 7 — Data pipeline:
- GpuBufferPool: use from_slice (eliminate staging buffer copy)

Phase 8 — Correctness:
- TFT: remove broken .detach() in forward_checkpointed (restore gradients)
- Update stale RTX 3050 Ti doc references

33 files changed, 2451 tests pass, 0 clippy warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

crates/ml/src/memory_optimization/auto_batch_size.rs

@@ -290,8 +290,12 @@ impl AutoBatchSizer {
             .max(config.min_batch_size)
             .min(config.max_batch_size);
 
-        // Round down to nearest power of 2 for better GPU utilization
-        let batch_size = optimal_batch_size.next_power_of_two() / 2;
+        // Round down to nearest power of 2 for GPU alignment
+        let batch_size = if optimal_batch_size == 0 {
+            0
+        } else {
+            1_usize << (usize::BITS - 1 - optimal_batch_size.leading_zeros())
+        };
         let final_batch_size = batch_size
             .max(config.min_batch_size)
             .min(config.max_batch_size);
@@ -543,10 +547,10 @@ mod tests {
         // Clamped to max_batch_size: 256 (but then rounded down)
         // Final: 128 (nearest power of 2 ≤ 256)
 
-        // With new calculation, INT8 should still get 64-128 batch size
+        // With corrected power-of-2 rounding (floor, not halving), INT8 should get 128-256
         assert!(
-            batch_size >= 64 && batch_size <= 128,
-            "INT8 batch_size should be 64-128 on RTX 3050 Ti, got {}",
+            batch_size >= 128 && batch_size <= 256,
+            "INT8 batch_size should be 128-256 on RTX 3050 Ti, got {}",
             batch_size
         );
     }
@@ -573,8 +577,8 @@ mod tests {
 
         // With 15GB free, should calculate large batch size but clamp to max_batch_size
         // The actual calculation will produce a very large number (>150K samples)
-        // which rounds down to 128 after power-of-2 rounding and max_batch_size clamping
-        assert_eq!(batch_size, 128);
+        // which clamps to 256 (max_batch_size) then floors to 256 (already power-of-2)
+        assert_eq!(batch_size, 256);
     }
 
     #[test]
@@ -718,12 +722,12 @@ mod tests {
             batch_size_int8
         );
 
-        // Verify FP32 gets reasonable small batch size (1-32)
-        // Note: 80MB base model is small enough that batch_size=32 can fit
-        // Real TFT-225 (125MB base → 500MB FP32) would get much smaller batch size
+        // Verify FP32 gets reasonable batch size (1-64)
+        // Note: 80MB base model is small enough that batch_size=64 can fit
+        // Real TFT-225 (125MB base -> 500MB FP32) would get much smaller batch size
         assert!(
-            batch_size_fp32 >= 1 && batch_size_fp32 <= 32,
-            "FP32 batch_size should be 1-32 on 4GB GPU with small model, got {}",
+            batch_size_fp32 >= 1 && batch_size_fp32 <= 64,
+            "FP32 batch_size should be 1-64 on 4GB GPU with small model, got {}",
             batch_size_fp32
         );