foxhunt/docs/archive/ml_models/MAMBA2_TUNING_QUICKSTART.md

MAMBA-2 Hyperparameter Tuning - Quick Start Guide

Status: ✅ READY TO EXECUTE Duration: 6-10 hours (40 trials) GPU: RTX 3050 Ti (4GB VRAM) Created: 2025-10-14

---

🚀 Quick Commands

1. Start MAMBA-2 Tuning (Recommended)

# Login first
tli login

# Start 40-trial tuning with live monitoring
tli tune start \
  --model MAMBA_2 \
  --trials 40 \
  --watch

Expected Output:

Job ID: 8a7b9c3d-4e5f-6a1b-2c3d-4e5f6a7b8c9d
Model: MAMBA_2
Trials: 40
Status: Running
Estimated time: 6-10 hours

[Trial 1/40] lr=0.0001, batch=32, state=16, hidden=256, sharpe=1.42
[Trial 2/40] lr=0.001, batch=16, state=32, hidden=512, sharpe=1.38 (PRUNED)
[Trial 3/40] lr=0.0001, batch=32, state=16, hidden=256, sharpe=1.68 ⭐
...

---

2. Monitor Progress

# Check status anytime
tli tune status --job-id 8a7b9c3d-4e5f-6a1b-2c3d-4e5f6a7b8c9d

Output:

Job ID: 8a7b9c3d-4e5f-6a1b-2c3d-4e5f6a7b8c9d
Status: Running
Progress: 23/40 trials (57.5%)
Best Sharpe: 1.72
Elapsed: 4h 15m
Estimated remaining: 3h 10m

---

3. Get Best Hyperparameters

# After completion (or anytime during run)
tli tune best --job-id 8a7b9c3d-4e5f-6a1b-2c3d-4e5f6a7b8c9d

Example Output:

Best Trial: 23
Sharpe Ratio: 1.78
Hyperparameters:
  learning_rate: 0.0001
  batch_size: 32
  hidden_dim: 256
  state_size: 16
  num_layers: 4
  expansion_factor: 2
  dropout: 0.15
  dt_min: 0.001
  dt_max: 0.08
  use_ssd: true
  use_selective_state: true
  hardware_aware: true
  grad_clip: 1.25
  weight_decay: 0.0005
  warmup_steps: 800

---

4. Stop Tuning (Optional)

# Cancel running job if needed
tli tune stop --job-id 8a7b9c3d-4e5f-6a1b-2c3d-4e5f6a7b8c9d

Note: Partial results are saved and can be retrieved with tune best

---

📊 Search Space Summary

MAMBA-2 tuning optimizes 14 hyperparameters across 40 trials:

Core Architecture (State-Space Dynamics)

• learning_rate: [0.00001, 0.0001, 0.001]
• batch_size: [16, 32, 64] (memory-constrained)
• hidden_dim: [128, 256, 512] (d_model)
• state_size: [8, 16, 32] (d_state - critical for dynamics)
• num_layers: [2, 4, 8]
• expansion_factor: [2, 4]

State-Space Dynamics

• dt_min: [0.0001, 0.01] (log scale)
• dt_max: [0.01, 1.0] (log scale)

Architecture Features

• use_ssd: [true, false] (Structured State Duality)
• use_selective_state: [true, false] (context-aware transitions)
• hardware_aware: [true, false] (RTX 3050 Ti optimizations)

Regularization

• dropout: [0.0, 0.3]
• grad_clip: [0.5, 2.0] (critical for state-space stability)
• weight_decay: [0.0001, 0.01] (log scale)
• warmup_steps: [100, 2000]

Total search space: ~50,000+ unique configurations

---

⏱️ Time Estimates

Stage	Duration	Activity
Trial 1-5	60-90 min	Baseline establishment (no pruning)
Trial 6-40	4-8 hours	Optuna TPE optimization with MedianPruner
Total	6-10 hours	Full 40-trial study

MedianPruner Savings

• 30-50% time reduction by stopping unpromising trials early
• Prunes trials with poor Sharpe ratio trends at epoch 10-30
• Estimated 16/40 trials will be pruned (saves ~2 hours)

---

🎯 Expected Performance

Baseline (DQN/PPO from prior tuning)

DQN:  Sharpe 1.50, Win Rate 52%, Max DD -15%
PPO:  Sharpe 1.30, Win Rate 50%, Max DD -18%

MAMBA-2 Expected Outcome (40 trials)

Conservative:

Sharpe: 1.60-1.80  (10-20% improvement)
Win Rate: 53-56%
Max Drawdown: -12-14%
Inference: <100μs
VRAM: ~2.2GB

Optimistic (if state-space excels at HFT patterns):

Sharpe: 1.90-2.20  (30-50% improvement)
Win Rate: 57-62%
Max Drawdown: -10-12%
Inference: <80μs
VRAM: ~2.2GB

---

🔬 Key Research Questions

1. State Size vs Performance: Does larger state size (32) justify memory cost?
2. Memory-Accuracy Tradeoff: hidden_dim=512 vs hidden_dim=256?
3. Time-Step Dynamics: Optimal dt_min/dt_max for HFT tick capture?
4. Advanced Features: Do use_ssd and use_selective_state provide lift?

Hypothesis: State size 16-32, hidden_dim 256, use_selective_state=true likely optimal

---

📁 Output Artifacts

MinIO Storage

s3://foxhunt-ml-models/mamba2/tuning_jobs/{job_id}/
├── optuna_study.db           # JournalStorage for crash recovery
├── trial_results.json         # All 40 trial results
├── best_checkpoint.safetensors
└── analysis/
    ├── sharpe_vs_state_size.png
    ├── memory_vs_accuracy.png
    └── feature_importance.png

Local Analysis

# Download tuning results
tli tune best --job-id <uuid> --export mamba2_best.yaml

# Use best config for final training
tli train \
  --model MAMBA_2 \
  --config mamba2_best.yaml \
  --epochs 500 \
  --symbols ES.FUT,NQ.FUT,ZN.FUT,6E.FUT

---

🔒 GPU Memory Safety

VRAM Monitoring Strategy

• Pre-trial estimation: Check if config exceeds 3.5GB
• Auto-skip: Dangerous configs (hidden_dim=512, state_size=32, batch_size=64)
• Graceful fallback: CPU inference if GPU OOM

Safe Configurations

hidden_dim=256, state_size=16, batch_size=32: 2.2GB (optimal)
hidden_dim=128, state_size=8, batch_size=64:  1.4GB (conservative)

Risky Configurations (will be skipped)

hidden_dim=512, state_size=32, batch_size=64: 4.8GB (OOM risk)

---

✅ Success Criteria

Must-Have (Critical)

• ✅ Complete 40 trials without crashes
• ✅ Sharpe ratio > 1.50 (match DQN baseline)
• ✅ Inference latency < 200μs
• ✅ VRAM usage < 3.5GB
• ✅ No training instability

Should-Have (Important)

• ✅ Sharpe ratio > 1.60 (10%+ improvement)
• ✅ MedianPruner saves 30%+ time
• ✅ State-space features (use_ssd, use_selective_state) provide lift
• ✅ Clear hyperparameter trends

Nice-to-Have (Aspirational)

• ✅ Sharpe ratio > 1.80 (20%+ improvement)
• ✅ Inference latency < 100μs
• ✅ Win rate > 55%
• ✅ Max drawdown < -12%

---

🐛 Troubleshooting

Issue: OOM Errors

Solution: Batch size auto-reduced to 16, or trial skipped

Issue: Training Instability (NaN loss)

Solution: Gradient clipping [0.5, 2.0], warmup steps [100, 2000]

Issue: Poor Sharpe Ratio (<1.50)

Investigation:

• Check data quality (665K samples across ES/NQ/ZN/6E)
• Verify feature engineering (16 features + 10 indicators)
• Inspect state-space dynamics (dt_min, dt_max)

Issue: Slow Tuning (>12 hours)

Optimization:

• Increase MedianPruner aggressiveness (n_warmup_steps: 5)
• Reduce trials to 30
• Use cloud GPU (A100: 8x speedup)

---

📞 Next Steps After Tuning

1. Extract Best Config (5 minutes):

   tli tune best --job-id <uuid> > mamba2_best.yaml
   

2. Train Final Model (2-3 days):

   tli train --model MAMBA_2 --config mamba2_best.yaml --epochs 500
   

3. Backtest & Validate (1 day):

   tli backtest \
     --model MAMBA_2 \
     --checkpoint mamba2_final.safetensors \
     --start-date 2024-10-01 \
     --end-date 2024-11-01
   

4. Compare with DQN/PPO (1 hour):

   • Sharpe ratio comparison
   • Win rate analysis
   • Drawdown risk assessment

5. Production Deployment Decision:

   • If Sharpe > 1.70: Deploy to production ensemble
   • If Sharpe 1.50-1.70: Use as diversification model
   • If Sharpe < 1.50: Investigate failure modes

---

📚 References

• Full Technical Report: /home/jgrusewski/Work/foxhunt/MAMBA2_HYPERPARAMETER_TUNING_REPORT.md
• Tuning Config: /home/jgrusewski/Work/foxhunt/services/ml_training_service/tuning_config.yaml
• MAMBA-2 Trainer: /home/jgrusewski/Work/foxhunt/ml/src/trainers/mamba2.rs
• Optuna Integration: /home/jgrusewski/Work/foxhunt/OPTUNA_QUICKSTART.md

---

🚀 Ready to Start?

# Login
tli login

# Start MAMBA-2 tuning (6-10 hours)
tli tune start --model MAMBA_2 --trials 40 --watch

# Let it run overnight, check progress in the morning
tli tune status --job-id <uuid>

# Get best hyperparameters
tli tune best --job-id <uuid>

Expected Completion: Tomorrow morning (overnight run) Best Sharpe Estimate: 1.60-1.80 (conservative), 1.90-2.20 (optimistic)

---

Document Version: 1.0 Status: ✅ READY TO EXECUTE Last Updated: 2025-10-14