foxhunt/data/README.md

# Interactive Brokers TWS/Gateway Integration

This implementation provides a production-ready integration with Interactive Brokers Trading Workstation (TWS) and IB Gateway for algorithmic trading applications.

## Features

- **Real TWS Socket Connections**: Direct TCP connections to TWS (port 7497) or Gateway (port 4001)
- **Binary Message Protocol**: Native TWS API message encoding/decoding
- **Client ID Management**: Proper TWS session management with client ID tracking
- **Request ID Tracking**: Asynchronous request/response correlation
- **Order Management**: Complete order lifecycle (submit, cancel, status, executions)
- **Market Data**: Real-time market data subscriptions and tick handling
- **Account Information**: Account updates and position tracking
- **Connection Management**: Robust connection state management with reconnection logic
- **Error Handling**: Comprehensive error handling and recovery mechanisms

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Trading Application                      │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│                 BrokerAdapter Trait                        │
│  ┌─────────────────────────────────────────────────────┐    │
│  │          InteractiveBrokersAdapter              │    │
│  │  ┌─────────────────────────────────────────────┐    │    │
│  │  │             TWS Message Codec               │    │    │
│  │  │  ┌─────────────────────────────────────┐    │    │    │
│  │  │  │        TCP Socket Connection        │    │    │    │
│  │  │  └─────────────────┬───────────────────┘    │    │    │
│  │  └────────────────────┼────────────────────────┘    │    │
│  └───────────────────────┼─────────────────────────────┘    │
└──────────────────────────┼──────────────────────────────────┘
                           │
┌──────────────────────────▼──────────────────────────────────┐
│              Interactive Brokers TWS/Gateway               │
│                     (localhost:7497/4001)                  │
└─────────────────────────────────────────────────────────────┘
```

## Prerequisites

### TWS/Gateway Setup

1. **Install Interactive Brokers TWS or Gateway**
   - Download from [Interactive Brokers website](https://www.interactivebrokers.com/en/trading/tws.php)
   - Install and configure with your IB account

2. **Enable API Connections**
   - Open TWS/Gateway
   - Go to File → Global Configuration → API → Settings
   - Enable "Enable ActiveX and Socket Clients"
   - Set "Socket Port" to 7497 (paper trading) or 7496 (live trading)
   - For Gateway, use port 4001
   - Enable "Download open orders on connection"
   - Set "Master API client ID" (optional)
   - Click "Apply" and "OK"

3. **Configure Trusted IPs**
   - In API settings, add 127.0.0.1 to trusted IPs
   - For production, configure appropriate IP restrictions

### Rust Dependencies

Add to your `Cargo.toml`:

```toml
[dependencies]
tokio = { version = "1.0", features = ["full"] }
async-trait = "0.1"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
chrono = { version = "0.4", features = ["serde"] }
tracing = "0.1"
uuid = { version = "1.0", features = ["v4"] }
types = { path = "../types" } # Your types crate
```

## Quick Start

### Basic Connection

```rust
use data::brokers::{InteractiveBrokersAdapter, IBConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    // Configure connection
    let config = IBConfig {
        host: "127.0.0.1".to_string(),
        port: 7497, // Paper trading port
        client_id: 1,
        account_id: "DU123456".to_string(),
        connection_timeout: 30,
        heartbeat_interval: 30,
        max_reconnect_attempts: 5,
        request_timeout: 10,
    };

    // Create and connect adapter
    let mut adapter = InteractiveBrokersAdapter::new(config);
    adapter.connect().await?;

    println!("Connected to TWS!");

    // Disconnect when done
    adapter.disconnect().await?;
    Ok(())
}
```

### Order Submission

```rust
use types::prelude::*;

// Create a market order
let order = Order {
    id: OrderId::new(),
    symbol: Symbol::from_str("AAPL"),
    side: Side::Buy,
    quantity: Quantity::new(100.0)?,
    order_type: OrderType::Market,
    price: None,
    stop_price: None,
    time_in_force: TimeInForce::Day,
    created_at: chrono::Utc::now(),
    updated_at: chrono::Utc::now(),
    filled_quantity: Quantity::ZERO,
    status: OrderStatus::New,
    metadata: std::collections::HashMap::new(),
};

// Submit to TWS
let tws_order_id = adapter.submit_order(&order).await?;
println!("Order submitted with TWS ID: {}", tws_order_id);
```

### Market Data Subscription

```rust
// Subscribe to market data
let symbol = Symbol::from_str("AAPL");
let request_id = adapter.request_market_data(&symbol).await?;

// Start message processing to receive data
let adapter_arc = std::sync::Arc::new(adapter);
let process_handle = {
    let adapter = adapter_arc.clone();
    tokio::spawn(async move {
        adapter.process_messages().await
    })
};

// Let it run for 30 seconds
tokio::time::sleep(tokio::time::Duration::from_secs(30)).await;

// Cancel subscription and stop processing
adapter_arc.cancel_market_data(request_id).await?;
process_handle.abort();
```

## Configuration

### Environment Variables

The adapter supports configuration via environment variables:

```bash
export IB_TWS_HOST=127.0.0.1
export IB_TWS_PORT=7497
export IB_CLIENT_ID=1
export IB_ACCOUNT_ID=DU123456
```

### Configuration File

Create a JSON configuration file:

```json
{
    "host": "127.0.0.1",
    "port": 7497,
    "client_id": 1,
    "account_id": "DU123456",
    "connection_timeout": 30,
    "heartbeat_interval": 30,
    "max_reconnect_attempts": 5,
    "request_timeout": 10
}
```

Load with:

```rust
let config: IBConfig = serde_json::from_str(&config_json)?;
let adapter = InteractiveBrokersAdapter::new(config);
```

## Port Configuration

| Environment | TWS Port | Gateway Port | Description |
|-------------|----------|--------------|-------------|
| Paper Trading | 7497 | 4001 | Safe for testing |
| Live Trading | 7496 | 4002 | Real money - use with caution |

**Important**: Always start with paper trading (port 7497) for development and testing.

## Message Processing

The adapter uses asynchronous message processing to handle incoming TWS messages:

```rust
// Start message processing loop
let adapter_arc = std::sync::Arc::new(adapter);
let process_handle = {
    let adapter = adapter_arc.clone();
    tokio::spawn(async move {
        if let Err(e) = adapter.process_messages().await {
            eprintln!("Message processing error: {}", e);
        }
    })
};

// Your trading logic here...

// Stop processing when done
process_handle.abort();
```

## Error Handling

The adapter provides comprehensive error handling:

```rust
match adapter.connect().await {
    Ok(()) => println!("Connected successfully"),
    Err(e) => {
        eprintln!("Connection failed: {}", e);
        // Handle connection error
    }
}
```

Common errors:
- **Connection timeout**: TWS/Gateway not running or not configured for API
- **Authentication failed**: Invalid client ID or account
- **Port in use**: Another client connected with same client ID
- **Permission denied**: API not enabled in TWS settings

## Performance Considerations

### Low Latency Settings

1. **TCP Socket Optimization**:
   - The adapter automatically sets `TCP_NODELAY` for minimal latency
   - Uses direct binary protocol communication

2. **Message Processing**:
   - Asynchronous message handling prevents blocking
   - Efficient binary message encoding/decoding

3. **Connection Management**:
   - Persistent connections minimize connection overhead
   - Automatic reconnection with exponential backoff

### Memory Usage

- Request tracking maintains minimal state
- Message buffers are efficiently managed
- Order mapping uses memory-efficient data structures

## Security Considerations

1. **Network Security**:
   - Use localhost connections when possible
   - Configure TWS IP restrictions appropriately
   - Use VPN for remote connections

2. **API Security**:
   - Rotate client IDs periodically
   - Monitor API usage and connections
   - Implement proper authentication in production

3. **Account Security**:
   - Use paper trading accounts for development
   - Implement position and risk limits
   - Monitor all trading activity

## Troubleshooting

### Connection Issues

1. **"Connection refused"**:
   - Verify TWS/Gateway is running
   - Check port configuration (7497 vs 7496 vs 4001)
   - Ensure API is enabled in TWS settings

2. **"Authentication failed"**:
   - Verify client ID is not already in use
   - Check account ID matches TWS account
   - Ensure API connections are enabled

3. **"Connection timeout"**:
   - Increase connection timeout in config
   - Check network connectivity
   - Verify firewall settings

### Message Processing Issues

1. **"No market data"**:
   - Verify market data subscriptions in TWS
   - Check market hours
   - Ensure symbols are valid

2. **"Order rejected"**:
   - Check account permissions
   - Verify order parameters
   - Check position limits

### Debugging

Enable debug logging:

```rust
use tracing_subscriber;

tracing_subscriber::fmt::init();
```

This will show detailed connection and message information.

## Testing

Run the included examples:

```bash
# Basic connection test
cargo run --example basic_connection

# Order submission test  
cargo run --example order_submission

# Market data test
cargo run --example market_data

# Comprehensive workflow test
cargo run --example comprehensive_trading
```

## Production Deployment

### Pre-Production Checklist

- [ ] Test with paper trading account extensively
- [ ] Validate all order types and scenarios
- [ ] Test reconnection logic
- [ ] Verify error handling
- [ ] Load test with expected message volume
- [ ] Security review and IP restrictions
- [ ] Monitoring and alerting setup

### Production Configuration

```rust
let config = IBConfig {
    host: "127.0.0.1".to_string(),
    port: 7496, // Live trading port
    client_id: 2, // Use different client ID for production
    account_id: "U123456".to_string(), // Live account
    connection_timeout: 15, // Shorter timeout for production
    heartbeat_interval: 10, // More frequent heartbeats
    max_reconnect_attempts: 10, // More retry attempts
    request_timeout: 5, // Faster request timeout
};
```

### Monitoring

Implement monitoring for:
- Connection status
- Message processing latency
- Order submission/execution rates
- Error rates and types
- Account balance and positions

## Support

For issues related to:
- **TWS/Gateway setup**: Consult Interactive Brokers documentation
- **API permissions**: Contact Interactive Brokers support
- **Integration issues**: Check this documentation and examples
- **Performance optimization**: Review configuration and architecture

## License

This implementation is provided as-is for educational and development purposes. Ensure compliance with Interactive Brokers terms of service and applicable regulations when using in production.