foxhunt/SYMBOL_MIGRATION_GUIDE.md

Symbol Type Migration Guide

Last Updated: 2025-10-16 Audience: Foxhunt service developers Status: Production guidelines for Symbol type usage

---

Quick Reference

When to Use Symbol vs String

Context	Use Type	Example
API Boundaries (gRPC, REST)	Symbol	Validate at entry point
Database Columns	VARCHAR(50) → Symbol	Use .as_str() in queries
Internal Logic	String or Symbol	Either works, prefer Symbol
CLI Arguments	String	Convert to Symbol after parsing
Proto Definitions	string	Codegen handles conversion
Test Fixtures	Symbol::from("ES.FUT")	Use From<&str> trait

---

1. The Symbol Type

1.1 What is Symbol?

// Location: common/src/types.rs:3568
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[cfg_attr(feature = "database", derive(sqlx::Type))]
pub struct Symbol {
    value: String,  // Private field for encapsulation
}

Purpose: Type-safe wrapper around String for trading instrument identifiers with validation.

Features:

• ✅ Validation: Rejects empty/invalid symbols
• ✅ Type Safety: Prevents mixing with other string types
• ✅ Zero-Cost: No runtime overhead vs String
• ✅ Database: Works seamlessly with SQLX
• ✅ Serialization: Transparent JSON/Binary encoding

1.2 Validation Rules

// Valid symbols
Symbol::new_validated("ES.FUT".to_string()).unwrap();   // ✅ Futures
Symbol::new_validated("AAPL".to_string()).unwrap();     // ✅ Equity
Symbol::new_validated("BTC-USD".to_string()).unwrap();  // ✅ Crypto
Symbol::new_validated("EUR_USD".to_string()).unwrap();  // ✅ Forex

// Invalid symbols
Symbol::new_validated("".to_string()).unwrap_err();           // ❌ Empty
Symbol::new_validated("ES FUT".to_string()).unwrap_err();     // ❌ Space
Symbol::new_validated("ES/FUT".to_string()).unwrap_err();     // ❌ Slash
Symbol::new_validated("A".repeat(51)).unwrap_err();           // ❌ Too long (>50 chars)

Validation Checklist:

• ✅ Length: 1-50 characters
• ✅ Charset: Alphanumeric + dot (.) + hyphen (-) + underscore (_)
• ✅ No whitespace (leading, trailing, or internal)
• ✅ Case-sensitive (ES.FUT ≠ es.fut)

---

2. Creating Symbols

2.1 Permissive Constructor (Internal Use)

use common::types::Symbol;

// No validation - use for trusted sources
let symbol = Symbol::new("ES.FUT".to_string());
let symbol = Symbol::from("ES.FUT");  // Shorthand via From trait

When to use: Internal logic where symbols are already validated.

2.2 Validated Constructor (API Boundaries)

use common::types::Symbol;

// With validation - use for user input
match Symbol::new_validated(user_input) {
    Ok(symbol) => {
        // Symbol is valid, proceed with business logic
        process_order(symbol);
    }
    Err(e) => {
        // Symbol is invalid, return error to user
        return Err(format!("Invalid symbol: {}", e));
    }
}

When to use: API handlers, CLI parsing, external data ingestion.

2.3 From Trait Conversions

// String → Symbol (zero-cost)
let s = "ES.FUT".to_string();
let symbol: Symbol = s.into();

// &str → Symbol (allocates String)
let symbol: Symbol = "ES.FUT".into();

// Symbol → String (clones inner value)
let symbol = Symbol::from("ES.FUT");
let s: String = symbol.into();

---

3. Using Symbols

3.1 String Operations

let symbol = Symbol::from("ES.FUT");

// Get as string slice (zero-cost)
let s: &str = symbol.as_str();       // Primary method
let s: &str = symbol.value();        // Alias
let s: &str = symbol.as_ref();       // Via AsRef trait

// Get as owned String (clones)
let s: String = symbol.to_string();  // Via Display trait
let s: String = symbol.into();       // Via Into trait

// String manipulation
let upper = symbol.to_uppercase();   // "ES.FUT"
let replaced = symbol.replace(".", "_");  // "ES_FUT"
let contains = symbol.contains("ES");     // true

3.2 Comparison Operations

let symbol = Symbol::from("ES.FUT");

// Compare with other Symbols
assert_eq!(symbol, Symbol::from("ES.FUT"));

// Compare with &str (no allocation)
assert_eq!(symbol, "ES.FUT");
assert_ne!(symbol, "NQ.FUT");

// Compare with String
let s = "ES.FUT".to_string();
assert_eq!(symbol, s);

// Pattern matching
match symbol.as_str() {
    "ES.FUT" => println!("E-mini S&P 500"),
    "NQ.FUT" => println!("Nasdaq futures"),
    _ => println!("Unknown symbol"),
}

3.3 Special Helpers

// Check if empty
let symbol = Symbol::default();  // Empty symbol
assert!(symbol.is_empty());

// Create "NONE" sentinel value
let none = Symbol::none();
assert_eq!(none, "NONE");

// Get as bytes (for hashing, binary protocols)
let bytes = symbol.as_bytes();

---

4. Database Integration

4.1 SQLX Queries

use common::types::Symbol;
use sqlx::PgPool;

// SELECT query
pub async fn get_order(pool: &PgPool, symbol: &Symbol) -> Result<Order> {
    let order = sqlx::query_as!(
        Order,
        r#"
        SELECT order_id, symbol, quantity, price
        FROM orders
        WHERE symbol = $1
        "#,
        symbol.as_str()  // Use .as_str() for query parameters
    )
    .fetch_one(pool)
    .await?;

    Ok(order)
}

// INSERT query
pub async fn insert_order(pool: &PgPool, order: &Order) -> Result<()> {
    sqlx::query!(
        r#"
        INSERT INTO orders (order_id, symbol, quantity, price)
        VALUES ($1, $2, $3, $4)
        "#,
        order.id,
        order.symbol.as_str(),  // Convert Symbol to &str
        order.quantity,
        order.price
    )
    .execute(pool)
    .await?;

    Ok(())
}

4.2 Database Schema

-- Standard symbol column definition
CREATE TABLE orders (
    order_id UUID PRIMARY KEY,
    symbol VARCHAR(50) NOT NULL,
    quantity DECIMAL(20, 8) NOT NULL,
    price DECIMAL(20, 8),
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Add validation constraints (optional)
ALTER TABLE orders
ADD CONSTRAINT symbol_length CHECK (LENGTH(symbol) BETWEEN 1 AND 50);

ALTER TABLE orders
ADD CONSTRAINT symbol_charset CHECK (symbol ~ '^[A-Za-z0-9._-]+$');

-- Index for symbol lookups
CREATE INDEX idx_orders_symbol ON orders(symbol);

---

5. gRPC/Proto Integration

5.1 Proto Definition

// File: trading.proto
message SubmitOrderRequest {
  string symbol = 1;  // ✅ Use string type in proto (not Symbol)
  double quantity = 2;
  double price = 3;
}

Rule: Always use string in proto definitions. Rust codegen handles conversion.

5.2 gRPC Handler (Server)

use common::types::Symbol;
use tonic::{Request, Response, Status};

pub async fn submit_order(
    &self,
    request: Request<SubmitOrderRequest>,
) -> Result<Response<SubmitOrderResponse>, Status> {
    let req = request.into_inner();

    // Validate symbol at API boundary
    let symbol = Symbol::new_validated(req.symbol)
        .map_err(|e| {
            Status::invalid_argument(format!("Invalid symbol: {}", e))
        })?;

    // Use validated Symbol in business logic
    let order = self.service.create_order(symbol, req.quantity, req.price).await
        .map_err(|e| Status::internal(e.to_string()))?;

    Ok(Response::new(SubmitOrderResponse {
        order_id: order.id,
        symbol: order.symbol.to_string(),  // Convert back to String for proto
        ...
    }))
}

5.3 gRPC Client

use common::types::Symbol;

pub async fn submit_order(&mut self, symbol: Symbol) -> Result<String> {
    let request = SubmitOrderRequest {
        symbol: symbol.to_string(),  // Convert Symbol → String for proto
        quantity: 100.0,
        price: 4500.0,
    };

    let response = self.client.submit_order(request).await?;

    Ok(response.into_inner().order_id)
}

---

6. Serialization Examples

6.1 JSON Serialization

use serde::{Deserialize, Serialize};
use common::types::Symbol;

#[derive(Serialize, Deserialize)]
struct Order {
    order_id: String,
    symbol: Symbol,  // Serializes as JSON string
    quantity: f64,
}

// Serialize to JSON
let order = Order {
    order_id: "ord_123".to_string(),
    symbol: Symbol::from("ES.FUT"),
    quantity: 100.0,
};

let json = serde_json::to_string(&order)?;
// Output: {"order_id":"ord_123","symbol":"ES.FUT","quantity":100.0}

// Deserialize from JSON
let json = r#"{"order_id":"ord_123","symbol":"ES.FUT","quantity":100.0}"#;
let order: Order = serde_json::from_str(json)?;
assert_eq!(order.symbol, "ES.FUT");

6.2 Binary Serialization (Bincode)

use bincode;
use common::types::Symbol;

let symbol = Symbol::from("ES.FUT");

// Serialize to bytes
let bytes = bincode::serialize(&symbol)?;

// Deserialize from bytes
let symbol2: Symbol = bincode::deserialize(&bytes)?;
assert_eq!(symbol, symbol2);

---

7. Common Patterns

7.1 API Boundary Pattern

// API Gateway handler
pub async fn handle_request(
    symbol_str: String,  // Raw input from user
) -> Result<Response> {
    // 1. Validate at boundary
    let symbol = Symbol::new_validated(symbol_str)
        .map_err(|e| ApiError::InvalidSymbol(e.to_string()))?;

    // 2. Pass validated Symbol to business logic
    let result = business_logic(symbol).await?;

    // 3. Convert back to String for response
    Ok(Response {
        symbol: result.symbol.to_string(),
        ...
    })
}

7.2 Service Layer Pattern

// Business logic accepts Symbol (already validated)
pub async fn create_order(
    &self,
    symbol: Symbol,  // Type-safe, validated input
    quantity: f64,
    price: f64,
) -> Result<Order> {
    // Use symbol directly (no validation needed)
    let order = Order {
        id: Uuid::new_v4().to_string(),
        symbol,  // Direct assignment
        quantity,
        price,
        status: OrderStatus::New,
    };

    // Persist to database
    self.repository.insert_order(&order).await?;

    Ok(order)
}

7.3 Test Fixture Pattern

#[cfg(test)]
mod tests {
    use common::types::Symbol;

    fn create_test_symbol() -> Symbol {
        Symbol::from("TEST.SYM")  // Use From trait for brevity
    }

    #[test]
    fn test_order_creation() {
        let symbol = create_test_symbol();
        let order = Order::new(symbol, 100.0, 4500.0);

        assert_eq!(order.symbol, "TEST.SYM");
    }

    #[test]
    fn test_with_real_symbols() {
        // Test with actual DBN symbols
        let symbols = vec![
            Symbol::from("ES.FUT"),
            Symbol::from("NQ.FUT"),
            Symbol::from("ZN.FUT"),
        ];

        for symbol in symbols {
            assert!(symbol.as_str().ends_with(".FUT"));
        }
    }
}

---

8. Performance Considerations

8.1 Zero-Cost Abstractions

// String storage
let s = "ES.FUT".to_string();  // Heap: 6 bytes, Stack: 24 bytes

// Symbol storage
let sym = Symbol::from("ES.FUT");  // Heap: 6 bytes, Stack: 24 bytes

// Same memory layout! Symbol is a newtype wrapper around String.

8.2 Conversion Costs

Operation	Cost	Notes
String → Symbol	Zero	Moves pointer, no reallocation
Symbol.as_str()	Zero	Inlined by compiler
Symbol.to_string()	Clone	Allocates new String
Symbol::new_validated()	~50ns	Length + charset validation

8.3 Best Practices

// ✅ GOOD: Pass by reference
fn process_order(symbol: &Symbol) {
    // Use .as_str() for string operations
    if symbol.as_str().starts_with("ES") {
        // Process futures order
    }
}

// ✅ GOOD: Take ownership if needed
fn store_order(symbol: Symbol) -> Order {
    Order { symbol, ... }  // Moves Symbol into Order
}

// ❌ BAD: Unnecessary clones
fn process_order(symbol: Symbol) {
    let s = symbol.clone();  // Unnecessary clone
    // Use symbol directly or pass by reference
}

// ❌ BAD: Repeated to_string() calls
fn format_order(symbol: &Symbol) -> String {
    format!("{} {} {}", symbol.to_string(), symbol.to_string(), symbol.to_string())
    // Use symbol directly (implements Display)
    format!("{} {} {}", symbol, symbol, symbol)  // Better
}

---

9. Migration Checklist

9.1 For New Code

• Use Symbol type in struct fields (not String)
• Validate symbols at API boundaries with new_validated()
• Use .as_str() for database queries
• Use .to_string() only when returning to external APIs
• Write tests with real symbols (ES.FUT, NQ.FUT, etc.)

9.2 For Existing Code

• Identify symbol: String fields in structs
• Replace with symbol: Symbol
• Update constructors to accept Symbol
• Update database queries to use .as_str()
• Update gRPC handlers to validate at boundaries
• Run tests to verify no regressions

9.3 Validation Checklist

When adding Symbol validation:

• Check input is not empty
• Check length is 1-50 characters
• Check charset is alphanumeric + . + - + _
• Check no leading/trailing whitespace
• Return clear error message on failure
• Test with real market symbols (ES.FUT, AAPL, BTC-USD)

---

10. Troubleshooting

10.1 Compilation Errors

Error: cannot convert String to Symbol

// ❌ Wrong
let symbol: Symbol = "ES.FUT".to_string();

// ✅ Correct
let symbol: Symbol = Symbol::from("ES.FUT");
let symbol = Symbol::from("ES.FUT".to_string());

Error: expected &str, found Symbol

// ❌ Wrong
my_function(symbol);  // Expected &str

// ✅ Correct
my_function(symbol.as_str());

Error: Symbol does not implement Display

// Symbol DOES implement Display - check imports
use std::fmt;

// ✅ Should work
println!("Symbol: {}", symbol);

10.2 Runtime Errors

Error: ValidationError: Symbol cannot be empty

// User provided empty string
let result = Symbol::new_validated("".to_string());
// Handle error gracefully
match result {
    Ok(sym) => process_symbol(sym),
    Err(e) => return Err(ApiError::InvalidInput(e.to_string())),
}

Error: ValidationError: Symbol contains invalid characters

// User provided symbol with spaces
let result = Symbol::new_validated("ES FUT".to_string());
// Suggest correct format to user
match result {
    Err(e) => {
        return Err(ApiError::InvalidInput(
            format!("{} - Use 'ES.FUT' format", e)
        ));
    }
    Ok(sym) => process_symbol(sym),
}

10.3 Database Errors

Error: symbol value too long for type character varying(20)

-- Database column is too small
ALTER TABLE orders ALTER COLUMN symbol TYPE VARCHAR(50);

-- Or use TEXT for unlimited length (less efficient)
ALTER TABLE orders ALTER COLUMN symbol TYPE TEXT;

Error: invalid input syntax for type character varying

// Ensure symbol.as_str() is used, not symbol directly
sqlx::query!("INSERT INTO orders (symbol) VALUES ($1)", symbol.as_str())

---

11. Examples by Service

11.1 Trading Service

// File: services/trading_service/src/lib.rs
use common::types::Symbol;

pub struct TradingService {
    db_pool: PgPool,
}

impl TradingService {
    pub async fn submit_order(
        &self,
        symbol: Symbol,  // Accept validated Symbol
        side: OrderSide,
        quantity: f64,
    ) -> Result<Order> {
        // Use symbol directly (no validation needed)
        let order = Order {
            id: Uuid::new_v4().to_string(),
            symbol,
            side,
            quantity,
            status: OrderStatus::New,
            created_at: Utc::now(),
        };

        // Persist to database
        sqlx::query!(
            "INSERT INTO orders (id, symbol, side, quantity, status) VALUES ($1, $2, $3, $4, $5)",
            order.id,
            order.symbol.as_str(),  // Convert to &str for DB
            order.side as i32,
            order.quantity,
            order.status as i32
        )
        .execute(&self.db_pool)
        .await?;

        Ok(order)
    }
}

11.2 Risk Service

// File: risk/src/position_tracker.rs
use common::types::Symbol;
use std::collections::HashMap;

pub struct PositionTracker {
    positions: HashMap<Symbol, Position>,  // Use Symbol as HashMap key
}

impl PositionTracker {
    pub fn update_position(&mut self, symbol: Symbol, quantity: f64) {
        self.positions
            .entry(symbol)  // Symbol implements Hash + Eq
            .and_modify(|pos| pos.quantity += quantity)
            .or_insert(Position { quantity, ..Default::default() });
    }

    pub fn get_position(&self, symbol: &Symbol) -> Option<&Position> {
        self.positions.get(symbol)  // HashMap lookup by Symbol
    }
}

11.3 ML Service

// File: ml/src/ensemble/decision.rs
use common::types::Symbol;

pub struct EnsemblePredictor {
    models: Vec<Box<dyn MLModel>>,
}

impl EnsemblePredictor {
    pub async fn predict(&self, symbol: Symbol) -> Result<Prediction> {
        // Generate predictions from all models
        let predictions = futures::future::try_join_all(
            self.models.iter().map(|model| model.predict(&symbol))
        ).await?;

        // Ensemble voting
        let action = self.vote(predictions);

        Ok(Prediction {
            symbol,  // Move Symbol into result
            action,
            confidence: 0.85,
            timestamp: Utc::now(),
        })
    }
}

---

12. FAQ

Q: Why not just use String everywhere?

A: Symbol provides:

• Type safety (prevents mixing with other string types)
• Validation (rejects invalid symbols at API boundaries)
• Documentation (self-documenting code)
• Zero-cost abstraction (no performance penalty)

Q: When should I use new() vs new_validated()?

A: Use new_validated() at API boundaries (user input, external data). Use new() for internal logic where symbols are already validated.

Q: Does Symbol work with HashMap/HashSet?

A: Yes! Symbol implements Hash and Eq, so it works as a HashMap key or HashSet member.

Q: Can I compare Symbol with &str directly?

A: Yes! Symbol implements PartialEq<&str>, so symbol == "ES.FUT" works without conversion.

Q: What's the performance impact of Symbol vs String?

A: Zero. Symbol is a newtype wrapper around String with identical memory layout and zero-cost abstractions.

Q: How do I test with real market symbols?

A: Use the From trait: Symbol::from("ES.FUT") in tests. See real DBN symbols in test_data/real/dbn/.

Q: Should I migrate existing String fields to Symbol?

A: Not necessarily. Prioritize API boundaries and new code. Internal String usage is fine if it works.

Q: Can I use Symbol in proto definitions?

A: No. Proto definitions must use string type. Rust codegen handles conversion to/from Symbol at API boundaries.

---

13. References

• Symbol Implementation: /home/jgrusewski/Work/foxhunt/common/src/types.rs:3568
• Symbol Tests: /home/jgrusewski/Work/foxhunt/common/src/types.rs:4638-4695
• Database Schemas: /home/jgrusewski/Work/foxhunt/migrations/
• Proto Definitions: /home/jgrusewski/Work/foxhunt/tli/proto/trading.proto
• Real Market Data: /home/jgrusewski/Work/foxhunt/test_data/real/dbn/

---

Last Updated: 2025-10-16 Version: 1.0 Status: Production-ready guidelines