foxhunt/AGENT_146_QUICK_START.md

MAMBA-2 TDD Quick Start Guide

Created: 2025-10-14 Purpose: Fast reference for MAMBA-2 debugging with TDD tests

---

🚀 Quick Commands

Run All Tests (1 minute)

cargo test --release -p ml --test e2e_mamba2_training -- --nocapture

Run Single Test (5 seconds)

cargo test --release -p ml --test e2e_mamba2_training test_mamba2_simple_forward_pass -- --nocapture

Debug with Backtrace

RUST_BACKTRACE=1 cargo test --release -p ml --test e2e_mamba2_training -- --nocapture

---

🐛 Current Bug: Dtype Mismatch

Error:

Error: Model error: Candle error: unexpected dtype, expected: F64, got: F32

Fix (Option 1 - Recommended):

// In /home/jgrusewski/Work/foxhunt/ml/tests/e2e_mamba2_training.rs
// Line 68: Change F32 to F64
let input = Tensor::randn(0f64, 1.0, (batch_size, seq_len, config.d_model), &device)?;
                    ^^^^ Change from 0f32 to 0f64

Apply to All Tests: Search for: Tensor::randn(0f32, Replace with: Tensor::randn(0f64,

Count: ~10 occurrences

---

📊 Test Suite Overview

Test Name	Duration	Purpose	Status
test_mamba2_simple_forward_pass	5s	Basic forward pass	❌ Dtype error
test_mamba2_batch_shapes	15s	Batch size validation	⏸️ Blocked
test_mamba2_cuda_device	5s	CUDA verification	⏸️ Blocked
test_mamba2_sequence_lengths	15s	Sequence handling	⏸️ Blocked
test_mamba2_gradient_flow	5s	Loss computation	⏸️ Blocked
test_mamba2_training_loop_simple	10s	Multi-batch training	⏸️ Blocked
test_mamba2_config_variations	20s	Config flexibility	⏸️ Blocked

Total Duration: ~75 seconds (when all pass)

---

🔧 Debugging Workflow

Step 1: Run Test

cargo test --release -p ml --test e2e_mamba2_training test_mamba2_simple_forward_pass -- --nocapture

Output:

🧪 E2E Test: MAMBA-2 Simple Forward Pass
  Device: Cuda(CudaDevice(DeviceId(1)))
  Config: d_model=256, layers=2
  Model created
  Input shape: [8, 60, 256]
Error: Model error: Candle error: unexpected dtype, expected: F64, got: F32

Step 2: Fix Code

See "Current Bug" section above

Step 3: Rerun Test

Same command as Step 1 (5 seconds)

Step 4: Repeat

Until all tests pass

---

📁 File Locations

Test File:

/home/jgrusewski/Work/foxhunt/ml/tests/e2e_mamba2_training.rs

Model Code:

/home/jgrusewski/Work/foxhunt/ml/src/mamba/mod.rs

Full Documentation:

/home/jgrusewski/Work/foxhunt/AGENT_146_MAMBA2_TDD_TEST.md

---

🎯 Success Criteria

✅ All 7 tests pass ✅ Test duration <60 seconds total ✅ No compilation warnings ✅ Clear output for each test

---

🚨 Common Errors

Error 1: Dtype Mismatch

Symptom: "expected: F64, got: F32" Fix: Change Tensor::randn(0f32, to Tensor::randn(0f64,

Error 2: Shape Mismatch

Symptom: "dimension mismatch" or "shape error" Fix: Check tensor dimensions match config (batch, seq, d_model)

Error 3: CUDA OOM

Symptom: "out of memory" Fix: Reduce batch size or num_layers in test config

---

📈 Performance Comparison

Metric	Full Training	TDD Tests	Speedup
First error	84s	36s	2.3x
Per iteration	80s	5s	16x
10 iterations	800s	50s	16x

---

🎓 Why TDD is Faster

Traditional Approach:

Build (77s) → Run training → Wait for crash (3s) → Debug → Repeat
= 80 seconds per cycle

TDD Approach:

Run test (5s) → See failure → Fix → Rerun test (5s)
= 5 seconds per cycle

Benefit: Catch errors in seconds, not minutes

---

💡 Tips

1. Run tests before full training - Catch 99% of bugs in <1 minute
2. Use --nocapture flag - See detailed output for debugging
3. Start with simple tests - Fix basic issues before complex ones
4. Watch for dtype mismatches - Common issue with candle tensors
5. Check GPU memory - Use nvidia-smi if tests hang

---

📞 Quick Help

Test hanging?

• Check nvidia-smi for GPU memory
• Reduce batch size in config
• Kill with Ctrl+C and reduce num_layers

Compilation errors?

• Check for missing fields in Mamba2Config
• Verify all imports are correct
• Run cargo clean and rebuild

Test passing but training fails?

• Increase test complexity (more epochs, larger batches)
• Add real data loading tests
• Check for differences in config between test and training

---

Last Updated: 2025-10-14 Next Action: Fix dtype mismatch (5 minutes), verify all tests pass