foxhunt/docs/archive/wave_d/reports/MIMALLOC_QUICK_REFERENCE.md

Mimalloc Allocator Quick Reference

Last Updated: 2025-10-25 Status: ✅ Active in all production training binaries

---

What is Mimalloc?

mimalloc (pronounced "me-malloc") is a high-performance memory allocator developed by Microsoft Research. It provides:

• 10-25% faster memory allocation vs. system allocator
• Lower memory fragmentation (better memory utilization)
• Better cache locality (improved CPU cache hit rate)
• Thread-local caching (reduced lock contention)

---

Quick Start

1. Build with Mimalloc

# All ML training binaries
cargo build --release -p ml --examples --features mimalloc-allocator

# Specific binary
cargo build --release -p ml --example train_tft_parquet --features "cuda,mimalloc-allocator"

2. Verify Allocator is Active

# Run any training binary and check logs
./target/release/examples/train_tft_parquet --epochs 1 2>&1 | head -5

# Expected output:
# 🚀 Using mimalloc allocator for improved performance

3. Compare Performance

# WITHOUT mimalloc (system allocator)
time cargo run -p ml --example train_tft_parquet --release -- --epochs 3

# WITH mimalloc
time cargo run -p ml --example train_tft_parquet --release --features mimalloc-allocator -- --epochs 3

# Expected: 8-15% faster with mimalloc

---

Training Binaries with Mimalloc

Binary	Mimalloc Support	Build Command
train_dqn	✅ Active	cargo build --release -p ml --example train_dqn --features mimalloc-allocator
train_ppo	✅ Active	cargo build --release -p ml --example train_ppo --features mimalloc-allocator
train_tft_parquet	✅ Active	cargo build --release -p ml --example train_tft_parquet --features mimalloc-allocator
train_mamba2_parquet	✅ Active	cargo build --release -p ml --example train_mamba2_parquet --features mimalloc-allocator

---

Implementation Pattern

Standard Pattern (All Training Binaries)

// Use mimalloc allocator for 10-25% performance improvement
#[cfg(feature = "mimalloc-allocator")]
use mimalloc::MiMalloc;

#[cfg(feature = "mimalloc-allocator")]
#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;

fn main() {
    #[cfg(feature = "mimalloc-allocator")]
    println!("🚀 Using mimalloc allocator for improved performance");

    #[cfg(not(feature = "mimalloc-allocator"))]
    println!("ℹ️  Using system allocator (consider --features mimalloc-allocator for 10-25% speedup)");

    // ... rest of training code
}

Cargo.toml Configuration

[features]
mimalloc-allocator = ["mimalloc"]  # Fast memory allocator

[dependencies]
mimalloc = { version = "0.1", optional = true }

---

Performance Expectations

Training Speed Improvements

Model	Workload Type	Expected Speedup
DQN	Mixed (data + GPU)	8-12%
PPO	Mixed (rollouts + GPU)	10-15%
TFT	GPU-heavy	3-7%
MAMBA-2	GPU-heavy	5-10%

Where Mimalloc Helps Most

1. Data Loading (Parquet, DBN files): 15-25% faster
2. Feature Extraction (225 features): 10-20% faster
3. Batch Preparation: 10-15% faster
4. Checkpoint Saving: 5-10% faster
5. GPU Tensor Allocation: 3-5% faster

Where Mimalloc Helps Less

1. Pure GPU Compute (matrix multiplication): <2% improvement
2. Attention Operations (TFT, MAMBA-2): <3% improvement
3. SSM State Updates (MAMBA-2): <5% improvement

Key Insight: Mimalloc speeds up CPU-bound memory operations. GPU-bound workloads see minimal improvement.

---

Runpod Deployment

1. Build Binaries with Mimalloc

# Build all training binaries (one-time)
cargo build --release -p ml --examples --features "cuda,mimalloc-allocator"

# Verify build
ls -lh target/release/examples/train_*

2. Upload to Runpod Network Volume

# Upload binaries to /runpod-volume/binaries/
# (via SSH, web UI, or Runpod file manager)

# Example structure:
/runpod-volume/binaries/
├── train_dqn                 (21 MB, mimalloc enabled)
├── train_tft_parquet         (21 MB, mimalloc enabled)
└── train_mamba2_parquet      (20 MB, mimalloc enabled)

3. Deploy to Runpod

# Deploy via Runpod console (no rebuild needed)
# Image: jgrusewski/foxhunt:latest
# Mount: /runpod-volume → Runpod Network Volume
# Env: BINARY_NAME=train_tft_parquet
# Args: --parquet-file /runpod-volume/test_data/ES_FUT_180d.parquet --epochs 50

Note: Binaries are pre-built with mimalloc. No Docker image rebuild required.

---

Troubleshooting

Issue: "Using system allocator" log message

Cause: Binary not built with mimalloc-allocator feature

Fix:

# Rebuild with feature flag
cargo build --release -p ml --examples --features mimalloc-allocator

# Verify log output
./target/release/examples/train_tft_parquet 2>&1 | head -1
# Expected: "🚀 Using mimalloc allocator for improved performance"

---

Issue: Binary crashes with "segmentation fault"

Cause: Rare allocator conflict with system libraries

Fix 1: Verify CUDA compatibility

# Check CUDA version
nvcc --version
nvidia-smi

# Ensure CUDA 13.0 compatible allocator
cargo clean && cargo build --release -p ml --examples --features "cuda,mimalloc-allocator"

Fix 2: Use jemalloc instead (alternative allocator)

# Edit ml/Cargo.toml
# Replace: mimalloc = { version = "0.1", optional = true }
# With:    jemallocator = { version = "0.5", optional = true }

# Rebuild
cargo build --release -p ml --examples --features jemalloc-allocator

---

Issue: No performance improvement observed

Cause 1: Workload is GPU-bound (expected behavior)

• Solution: Accept 2-5% improvement (mimalloc helps with data loading only)

Cause 2: Small dataset (allocator overhead dominates)

• Solution: Test with larger Parquet files (180+ days)

Cause 3: Profiling overhead (perf, valgrind)

• Solution: Disable profiling tools during benchmarks

---

Benchmarking

Simple Benchmark

# Run benchmark script (automated)
./benchmark_mimalloc.sh

# Expected output:
# System Allocator Duration: 180s
# Mimalloc Duration:         165s
# Performance Improvement:   8.3%
# Speedup Ratio:             1.09x

Manual Benchmark

# 1. WITHOUT mimalloc
time cargo run -p ml --example train_tft_parquet --release --features cuda -- \
  --parquet-file test_data/ES_FUT_small.parquet \
  --epochs 3 \
  --batch-size 16 \
  --use-gpu

# 2. WITH mimalloc
time cargo run -p ml --example train_tft_parquet --release --features "cuda,mimalloc-allocator" -- \
  --parquet-file test_data/ES_FUT_small.parquet \
  --epochs 3 \
  --batch-size 16 \
  --use-gpu

# 3. Compare times
# Expected: 5-15% faster with mimalloc

---

Advanced Usage

Profile Memory Allocation Patterns

# Install profiling tools
sudo apt-get install -y valgrind perf

# Profile allocations
cargo build --release -p ml --example train_tft_parquet --features "cuda,mimalloc-allocator"

# Run with massif (heap profiler)
valgrind --tool=massif ./target/release/examples/train_tft_parquet --epochs 1

# Analyze results
ms_print massif.out.*

Compare Allocators

# 1. System allocator (glibc malloc)
cargo build --release -p ml --example train_tft_parquet --features cuda
time ./target/release/examples/train_tft_parquet --epochs 3

# 2. Mimalloc
cargo build --release -p ml --example train_tft_parquet --features "cuda,mimalloc-allocator"
time ./target/release/examples/train_tft_parquet --epochs 3

# 3. Jemalloc (alternative)
# Edit Cargo.toml to add jemalloc feature
cargo build --release -p ml --example train_tft_parquet --features "cuda,jemalloc-allocator"
time ./target/release/examples/train_tft_parquet --epochs 3

---

FAQ

Q: Does mimalloc work with CUDA?

A: Yes, mimalloc is CPU-only and complements CUDA GPU memory. They don't conflict.

Q: Can I use mimalloc in Docker?

A: Yes, mimalloc works in Docker containers (static linking, no extra dependencies).

Q: Is mimalloc thread-safe?

A: Yes, mimalloc uses thread-local caching for high concurrency.

Q: Does mimalloc reduce GPU memory usage?

A: No, mimalloc only affects CPU memory. GPU memory is managed by CUDA allocator.

Q: Should I always use mimalloc?

A: Yes, for production builds. Disable only for debugging (valgrind compatibility).

Q: How much memory overhead does mimalloc add?

A: ~50 KB per thread (negligible for ML workloads).

---

References

• Mimalloc Paper: ISMM 2019
• Mimalloc GitHub: microsoft/mimalloc
• Rust Bindings: mimalloc crate
• Performance Benchmarks: mimalloc-bench

---

Validation Status

✅ All training binaries verified with mimalloc (2025-10-25) ✅ Runtime logs confirm allocator active ✅ Binary analysis confirms static linking ✅ Ready for Runpod deployment

See MIMALLOC_ALLOCATOR_VALIDATION_REPORT.md for full validation details.