# MAMBA-2 Parquet Training Example

This example demonstrates how to train the MAMBA-2 State Space Model using Parquet market data files.

## Quick Start

```bash
# Default: 200 epochs using ES.FUT data
cargo run -p ml --example train_mamba2_parquet --release

# Custom Parquet file and epochs
cargo run -p ml --example train_mamba2_parquet --release -- \
    --parquet-file test_data/NQ_FUT_180d.parquet \
    --epochs 50

# Custom lookback window (sequence length)
cargo run -p ml --example train_mamba2_parquet --release -- \
    --parquet-file test_data/ES_FUT_180d.parquet \
    --lookback-window 120 \
    --epochs 100
```

## Available Arguments

- `--parquet-file <path>`: Path to Parquet file (default: test_data/ES_FUT_180d.parquet)
- `--epochs <n>`: Number of training epochs (default: 200)
- `--lookback-window <n>`: Sequence length/lookback window (default: 60)
- `--batch-size <n>`: Batch size for training (default: 32)
- `--learning-rate <f>`: Learning rate (default: 0.0001)
- `--hidden-dim <n>`: Hidden dimension (default: 225, Wave D feature count)
- `--state-dim <n>`: SSM state dimension (default: 16)
--output-dir <path>`: Output directory for checkpoints (default: ml/checkpoints/mamba2_parquet)

## Available Parquet Files

- `test_data/ES_FUT_180d.parquet` - E-mini S&P 500 (180 days)
- `test_data/NQ_FUT_180d.parquet` - E-mini NASDAQ (180 days)
- `test_data/6E_FUT_180d.parquet` - Euro FX (180 days)
- `test_data/ZN_FUT_90d.parquet` - 10-Year T-Note (90 days)

## Output

Training outputs are saved to `ml/checkpoints/mamba2_parquet/`:

- `best_model_epoch_*.ckpt` - Best model based on validation loss
- `checkpoint_epoch_*.ckpt` - Periodic checkpoints (every 10 epochs)
- `final_model.ckpt` - Final model after training
- `training_losses.csv` - Training/validation loss curves
- `training_metrics.json` - Summary metrics and configuration

## Features

- **225 Wave D Features**: Includes 201 Wave C features + 24 Wave D regime features
- **GPU Training**: CUDA acceleration (RTX 3050 Ti optimized)
- **Early Stopping**: Automatic stopping after 20 epochs of no improvement
- **Checkpointing**: Saves best models and periodic snapshots
- **Monitoring**: Real-time loss, perplexity, and training speed metrics

## Expected Training Time

- 50 epochs: ~30-45 minutes (pilot run)
- 200 epochs: ~2-3 hours (full training)
- GPU utilization: ~60-70% (memory-bound on RTX 3050 Ti)

## Requirements

- CUDA GPU (required - no CPU fallback)
- Minimum 50 bars in Parquet file (for feature warmup period)
- ~4GB VRAM available

## Data Format

Parquet files should contain OHLCV market data with these columns:
- `timestamp_ns`: Nanosecond timestamp
- `open`, `high`, `low`: Price data (optional, defaults to `price`)
- `price`: Close price (required)
- `quantity`: Volume (optional, defaults to 0)
- `symbol`, `venue`, `event_type`, `sequence`: Metadata columns

See `data/src/providers/databento/dbn_to_parquet_converter.rs` for conversion utilities.