# GPU Memory Profile Report - RTX 3050 Ti (4GB VRAM)

**Generated**: 2025-10-14 19:38:02 UTC
**GPU**: NVIDIA GeForce RTX 3050 Ti Laptop
**VRAM**: 4096 MB total, 3669 MB free at start

---

## Executive Summary

This report profiles GPU VRAM usage for all ML models using direct `nvidia-smi` measurements.

- **DQN**: 135.0 MB peak VRAM, batch size 64 (training), batch size 128 (inference) - ✅ Safe
- **PPO**: 135.0 MB peak VRAM, batch size 64 (training), batch size 128 (inference) - ✅ Safe
- **MAMBA-2**: 167.0 MB peak VRAM, batch size 32 (training), batch size 64 (inference) - ✅ Safe
- **TFT**: 167.0 MB peak VRAM, batch size 8 (training), batch size 16 (inference) - ✅ Safe
- **Liquid NN**: 167.0 MB peak VRAM, batch size 64 (training), batch size 128 (inference) - ✅ Safe

---

## Detailed Model Profiles

### DQN

- **Parameters**: 83717
- **Base VRAM**: 103.0 MB
- **Peak VRAM**: 135.0 MB
- **Status**: ✅ Safe
- **Max Safe Batch Size**: 512
- **Training Batch Size**: 64
- **Inference Batch Size**: 128

#### Batch Size Tests

| Batch Size | VRAM (MB) | Status |
|------------|-----------|--------|
| 1 | 135.0 (3%) | ✅ Success |
| 8 | 135.0 (3%) | ✅ Success |
| 16 | 135.0 (3%) | ✅ Success |
| 32 | 135.0 (3%) | ✅ Success |
| 64 | 135.0 (3%) | ✅ Success |
| 128 | 135.0 (3%) | ✅ Success |
| 256 | 135.0 (3%) | ✅ Success |
| 512 | 135.0 (3%) | ✅ Success |

### PPO

- **Parameters**: 165376
- **Base VRAM**: 135.0 MB
- **Peak VRAM**: 135.0 MB
- **Status**: ✅ Safe
- **Max Safe Batch Size**: 256
- **Training Batch Size**: 64
- **Inference Batch Size**: 128

#### Batch Size Tests

| Batch Size | VRAM (MB) | Status |
|------------|-----------|--------|
| 1 | 135.0 (3%) | ✅ Success |
| 8 | 135.0 (3%) | ✅ Success |
| 16 | 135.0 (3%) | ✅ Success |
| 32 | 135.0 (3%) | ✅ Success |
| 64 | 135.0 (3%) | ✅ Success |
| 128 | 135.0 (3%) | ✅ Success |
| 256 | 135.0 (3%) | ✅ Success |

### MAMBA-2

- **Parameters**: 786432
- **Base VRAM**: 135.0 MB
- **Peak VRAM**: 167.0 MB
- **Status**: ✅ Safe
- **Max Safe Batch Size**: 64
- **Training Batch Size**: 32
- **Inference Batch Size**: 64

#### Batch Size Tests

| Batch Size | VRAM (MB) | Status |
|------------|-----------|--------|
| 1 | 135.0 (3%) | ✅ Success |
| 4 | 135.0 (3%) | ✅ Success |
| 8 | 135.0 (3%) | ✅ Success |
| 16 | 135.0 (3%) | ✅ Success |
| 32 | 135.0 (3%) | ✅ Success |
| 64 | 167.0 (4%) | ✅ Success |

### TFT

- **Parameters**: 6291456
- **Base VRAM**: 167.0 MB
- **Peak VRAM**: 167.0 MB
- **Status**: ✅ Safe
- **Max Safe Batch Size**: 32
- **Training Batch Size**: 8
- **Inference Batch Size**: 16

#### Batch Size Tests

| Batch Size | VRAM (MB) | Status |
|------------|-----------|--------|
| 1 | 167.0 (4%) | ✅ Success |
| 2 | 167.0 (4%) | ✅ Success |
| 4 | 167.0 (4%) | ✅ Success |
| 8 | 167.0 (4%) | ✅ Success |
| 16 | 167.0 (4%) | ✅ Success |
| 32 | 167.0 (4%) | ✅ Success |

### Liquid NN

- **Parameters**: 83456
- **Base VRAM**: 167.0 MB
- **Peak VRAM**: 167.0 MB
- **Status**: ✅ Safe
- **Max Safe Batch Size**: 256
- **Training Batch Size**: 64
- **Inference Batch Size**: 128

#### Batch Size Tests

| Batch Size | VRAM (MB) | Status |
|------------|-----------|--------|
| 1 | 167.0 (4%) | ✅ Success |
| 8 | 167.0 (4%) | ✅ Success |
| 16 | 167.0 (4%) | ✅ Success |
| 32 | 167.0 (4%) | ✅ Success |
| 64 | 167.0 (4%) | ✅ Success |
| 128 | 167.0 (4%) | ✅ Success |
| 256 | 167.0 (4%) | ✅ Success |

---

## Memory Budget Allocation

### Training (Single Model)

| Model | Peak VRAM | Training Batch | Status |
|-------|-----------|----------------|--------|
| DQN | 135.0 MB | 64 | ✅ |
| PPO | 135.0 MB | 64 | ✅ |
| MAMBA-2 | 167.0 MB | 32 | ✅ |
| TFT | 167.0 MB | 8 | ✅ |
| Liquid NN | 167.0 MB | 64 | ✅ |

### Inference (Multi-Model Ensemble)

- **Total VRAM for all models**: 707.0 MB
- **Available VRAM**: 4096.0 MB
- **Can load all models**: ✅ Yes


---

## Recommendations

### Training

1. **Train one model at a time** - Use recommended batch sizes above
2. **Monitor VRAM** - Run `watch -n1 nvidia-smi` during training
3. **Use gradient accumulation** for TFT model (small batch size)
4. **Enable mixed precision (FP16)** to reduce VRAM by ~40%
5. **Clear CUDA cache** between model switches: `torch.cuda.empty_cache()`

### Inference

1. **All models can be loaded simultaneously** for ensemble inference
2. **Use batch inference** with recommended batch sizes

---

## Expected vs Actual VRAM Usage

| Model | Expected Range (MB) | Actual (MB) | Status |
|-------|---------------------|-------------|--------|
| DQN | 50-150 | 135.0 | ✅ Within range |
| PPO | 50-200 | 135.0 | ✅ Within range |
| MAMBA-2 | 150-500 | 167.0 | ✅ Within range |
| TFT | 1500-2500 | 167.0 | ⚠️ Lower |
| Liquid NN | 100-300 | 167.0 | ✅ Within range |

---

**Agent**: 133 (GPU Memory Profiling)
**Command**: `cargo run -p ml --example gpu_memory_benchmark --release --features cuda`