# MAMBA-2 TDD Quick Start Guide

**Created**: 2025-10-14
**Purpose**: Fast reference for MAMBA-2 debugging with TDD tests

---

## 🚀 Quick Commands

### Run All Tests (1 minute)
```bash
cargo test --release -p ml --test e2e_mamba2_training -- --nocapture
```

### Run Single Test (5 seconds)
```bash
cargo test --release -p ml --test e2e_mamba2_training test_mamba2_simple_forward_pass -- --nocapture
```

### Debug with Backtrace
```bash
RUST_BACKTRACE=1 cargo test --release -p ml --test e2e_mamba2_training -- --nocapture
```

---

## 🐛 Current Bug: Dtype Mismatch

**Error**:
```
Error: Model error: Candle error: unexpected dtype, expected: F64, got: F32
```

**Fix** (Option 1 - Recommended):
```rust
// In /home/jgrusewski/Work/foxhunt/ml/tests/e2e_mamba2_training.rs
// Line 68: Change F32 to F64
let input = Tensor::randn(0f64, 1.0, (batch_size, seq_len, config.d_model), &device)?;
                    ^^^^ Change from 0f32 to 0f64
```

**Apply to All Tests**:
Search for: `Tensor::randn(0f32,`
Replace with: `Tensor::randn(0f64,`

**Count**: ~10 occurrences

---

## 📊 Test Suite Overview

| Test Name | Duration | Purpose | Status |
|-----------|----------|---------|--------|
| `test_mamba2_simple_forward_pass` | 5s | Basic forward pass | ❌ Dtype error |
| `test_mamba2_batch_shapes` | 15s | Batch size validation | ⏸️ Blocked |
| `test_mamba2_cuda_device` | 5s | CUDA verification | ⏸️ Blocked |
| `test_mamba2_sequence_lengths` | 15s | Sequence handling | ⏸️ Blocked |
| `test_mamba2_gradient_flow` | 5s | Loss computation | ⏸️ Blocked |
| `test_mamba2_training_loop_simple` | 10s | Multi-batch training | ⏸️ Blocked |
| `test_mamba2_config_variations` | 20s | Config flexibility | ⏸️ Blocked |

**Total Duration**: ~75 seconds (when all pass)

---

## 🔧 Debugging Workflow

### Step 1: Run Test
```bash
cargo test --release -p ml --test e2e_mamba2_training test_mamba2_simple_forward_pass -- --nocapture
```

**Output**:
```
🧪 E2E Test: MAMBA-2 Simple Forward Pass
  Device: Cuda(CudaDevice(DeviceId(1)))
  Config: d_model=256, layers=2
  Model created
  Input shape: [8, 60, 256]
Error: Model error: Candle error: unexpected dtype, expected: F64, got: F32
```

### Step 2: Fix Code
See "Current Bug" section above

### Step 3: Rerun Test
Same command as Step 1 (5 seconds)

### Step 4: Repeat
Until all tests pass

---

## 📁 File Locations

**Test File**:
```
/home/jgrusewski/Work/foxhunt/ml/tests/e2e_mamba2_training.rs
```

**Model Code**:
```
/home/jgrusewski/Work/foxhunt/ml/src/mamba/mod.rs
```

**Full Documentation**:
```
/home/jgrusewski/Work/foxhunt/AGENT_146_MAMBA2_TDD_TEST.md
```

---

## 🎯 Success Criteria

✅ All 7 tests pass
✅ Test duration <60 seconds total
✅ No compilation warnings
✅ Clear output for each test

---

## 🚨 Common Errors

### Error 1: Dtype Mismatch
**Symptom**: "expected: F64, got: F32"
**Fix**: Change `Tensor::randn(0f32,` to `Tensor::randn(0f64,`

### Error 2: Shape Mismatch
**Symptom**: "dimension mismatch" or "shape error"
**Fix**: Check tensor dimensions match config (batch, seq, d_model)

### Error 3: CUDA OOM
**Symptom**: "out of memory"
**Fix**: Reduce batch size or num_layers in test config

---

## 📈 Performance Comparison

| Metric | Full Training | TDD Tests | Speedup |
|--------|--------------|-----------|---------|
| First error | 84s | 36s | 2.3x |
| Per iteration | 80s | 5s | 16x |
| 10 iterations | 800s | 50s | 16x |

---

## 🎓 Why TDD is Faster

**Traditional Approach**:
```
Build (77s) → Run training → Wait for crash (3s) → Debug → Repeat
= 80 seconds per cycle
```

**TDD Approach**:
```
Run test (5s) → See failure → Fix → Rerun test (5s)
= 5 seconds per cycle
```

**Benefit**: Catch errors in seconds, not minutes

---

## 💡 Tips

1. **Run tests before full training** - Catch 99% of bugs in <1 minute
2. **Use `--nocapture` flag** - See detailed output for debugging
3. **Start with simple tests** - Fix basic issues before complex ones
4. **Watch for dtype mismatches** - Common issue with candle tensors
5. **Check GPU memory** - Use `nvidia-smi` if tests hang

---

## 📞 Quick Help

**Test hanging?**
- Check `nvidia-smi` for GPU memory
- Reduce batch size in config
- Kill with Ctrl+C and reduce num_layers

**Compilation errors?**
- Check for missing fields in `Mamba2Config`
- Verify all imports are correct
- Run `cargo clean` and rebuild

**Test passing but training fails?**
- Increase test complexity (more epochs, larger batches)
- Add real data loading tests
- Check for differences in config between test and training

---

**Last Updated**: 2025-10-14
**Next Action**: Fix dtype mismatch (5 minutes), verify all tests pass