docs: numeric type standardization design (i64/6dp financial types)
Consolidate competing Price/Quantity implementations into canonical i64 fixed-point types in common/. Adds Money and Ratio newtypes. Phased migration plan across 5 phases using parallel worktrees. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
228
docs/plans/2026-02-28-numeric-type-standardization-design.md
Normal file
228
docs/plans/2026-02-28-numeric-type-standardization-design.md
Normal file
@@ -0,0 +1,228 @@
|
||||
# Numeric Type Standardization Design
|
||||
|
||||
**Date**: 2026-02-28
|
||||
**Status**: Approved
|
||||
**Goal**: Eliminate precision-lossy f64 casting by standardizing on domain-typed i64 fixed-point wrappers for all financial values
|
||||
|
||||
## Problem
|
||||
|
||||
- 23,730 `as` casts across 1,044 files — many are f32↔f64 conversions at ML↔trading boundaries
|
||||
- 11,000+ raw f64 occurrences for prices, quantities, P&L — no type safety between different financial quantities
|
||||
- Two competing implementations: `common::Price` (u64, 8dp) vs `trading_engine::IntegerPrice` (i64, 6dp)
|
||||
- Neither widely adopted — raw `price: f64` still dominates (88+ field occurrences)
|
||||
- Proactive hardening: prevent precision bugs before they affect P&L or risk calculations
|
||||
|
||||
## Decision
|
||||
|
||||
**Approach C: Domain-typed wrappers with i64 fixed-point internals (6 decimal places)**
|
||||
|
||||
Rejected alternatives:
|
||||
- **rust_decimal::Decimal everywhere**: 3-10x slower than f64, no sqrt/ln/exp, kills SIMD optimizations in trading_engine
|
||||
- **Fixed-point tick integers**: Correct for HFT but requires per-instrument tick tables and is too large an architectural change
|
||||
- **u64 Price (current common::Price)**: Cannot represent negative values (P&L, unrealized losses)
|
||||
|
||||
## Canonical Type Set
|
||||
|
||||
All types live in `crates/common/src/` and are re-exported from `common::`.
|
||||
|
||||
### Price (i64, scale 1_000_000)
|
||||
|
||||
Represents: bid, ask, mid, last, VWAP, limit price, stop price, fill price, OHLCV values.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct Price(i64);
|
||||
|
||||
impl Price {
|
||||
pub const ZERO: Self = Self(0);
|
||||
pub const SCALE: i64 = 1_000_000;
|
||||
|
||||
pub fn from_f64(value: f64) -> Result<Self, CommonTypeError>;
|
||||
pub fn to_f64(self) -> f64;
|
||||
pub fn to_f32(self) -> f32; // ML boundary only
|
||||
pub fn from_f32(value: f32) -> Result<Self, CommonTypeError>;
|
||||
pub fn to_decimal(self) -> Decimal;
|
||||
pub fn from_decimal(decimal: Decimal) -> Self;
|
||||
pub fn abs(self) -> Self;
|
||||
pub fn sqrt(self) -> Self; // For volatility calculations on price diffs
|
||||
}
|
||||
```
|
||||
|
||||
6 decimal places sufficient for: ES (0.25), NQ (0.25), 6E (0.00005), ZN (1/64=0.015625), FX spot (0.00001), equities ($0.0001).
|
||||
|
||||
### Quantity (i64, scale 1_000_000)
|
||||
|
||||
Represents: contracts, shares, lots, position sizes. Signed to represent short positions.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct Quantity(i64);
|
||||
```
|
||||
|
||||
### Money (i64, scale 1_000_000)
|
||||
|
||||
Represents: P&L, margin, notional value, fees, commissions, account balance. Always signed.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct Money(i64);
|
||||
```
|
||||
|
||||
### Ratio (f64 newtype)
|
||||
|
||||
Represents: returns, percentages, volatility, Sharpe ratio, correlation, Kelly fraction — continuous mathematical quantities where IEEE 754 is correct.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, PartialOrd, Serialize, Deserialize)]
|
||||
pub struct Ratio(f64);
|
||||
|
||||
impl Ratio {
|
||||
pub const ZERO: Self = Self(0.0);
|
||||
pub const ONE: Self = Self(1.0);
|
||||
|
||||
pub fn new(value: f64) -> Result<Self, CommonTypeError>; // rejects NaN/Inf
|
||||
pub fn to_f64(self) -> f64;
|
||||
pub fn to_f32(self) -> f32; // ML boundary
|
||||
pub fn abs(self) -> Self;
|
||||
pub fn sqrt(self) -> Self;
|
||||
pub fn ln(self) -> Self;
|
||||
pub fn exp(self) -> Self;
|
||||
pub fn powi(self, n: i32) -> Self;
|
||||
pub fn is_positive(self) -> bool;
|
||||
pub fn is_negative(self) -> bool;
|
||||
pub fn clamp(self, min: f64, max: f64) -> Self;
|
||||
}
|
||||
```
|
||||
|
||||
Benefits of newtype over alias:
|
||||
- **NaN/Inf rejection at construction** — invalid values caught at boundaries, not mid-calculation
|
||||
- **Type safety** — can't accidentally assign a Ratio to a Price field
|
||||
- **Audit trail** — all f64→Ratio conversions go through `Ratio::new()` which validates finiteness
|
||||
- Implements standard math ops (Add, Sub, Mul, Div, Neg) delegating to f64 arithmetic
|
||||
|
||||
## Arithmetic
|
||||
|
||||
All types use saturating arithmetic (no panics on overflow):
|
||||
- `Price + Price → Price` (saturating_add)
|
||||
- `Price - Price → Price` (saturating_sub)
|
||||
- `Price * Quantity → Money` (cross-type multiplication)
|
||||
- `Money / Quantity → Price` (cross-type division)
|
||||
- `Price * i64 → Price` (scaling)
|
||||
- `Price / i64 → Price` (splitting)
|
||||
- Division by zero returns `Self(0)` (consistent with existing IntegerPrice behavior)
|
||||
|
||||
## Type Boundaries
|
||||
|
||||
```
|
||||
┌─────────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
||||
│ FINANCIAL DOMAIN │ │ ML DOMAIN │ │ ANALYTICS │
|
||||
│ │ │ │ │ │
|
||||
│ Price(i64) │────▶│ f32 tensors │ │ Ratio (f64) │
|
||||
│ Quantity(i64) │ │ (Candle GPU) │────▶│ (returns, vol, │
|
||||
│ Money(i64) │◀────│ predictions f32 │ │ Sharpe, stats) │
|
||||
│ │ │ │ │ │
|
||||
└─────────────────────┘ └──────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
### Conversion rules
|
||||
|
||||
- `Price → f64`: explicit `.to_f64()` only
|
||||
- `f64 → Price`: explicit `Price::from_f64()` (fallible, rounds to scale)
|
||||
- `Price → f32`: explicit `.to_f32()` (ML tensor input)
|
||||
- `f32 → Price`: explicit `Price::from_f32()` (ML prediction output)
|
||||
- **No `impl From<f64> for Price`** — all conversions are explicit and auditable
|
||||
- **No implicit casting** — compiler enforces type boundaries
|
||||
|
||||
### What stays raw f64/f32
|
||||
|
||||
- ML weights, gradients, losses, learning rates, activations (Candle tensor internals)
|
||||
- Raw sensor/timing data (RDTSC timestamps, latency measurements)
|
||||
|
||||
### What becomes Ratio
|
||||
|
||||
- Returns, log-returns, percentage changes
|
||||
- Volatility, implied vol, realized vol
|
||||
- Sharpe ratio, Sortino, Calmar, information ratio
|
||||
- Correlation coefficients, beta, R-squared
|
||||
- Kelly fraction, position sizing multipliers
|
||||
- Feature importance scores, confidence scores
|
||||
- Any continuous mathematical quantity that is not a financial amount
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Bottom-up, dependency-driven, parallelized via git worktrees.
|
||||
|
||||
### Phase 1: Foundation (1 worktree)
|
||||
|
||||
**Crate**: `common/`
|
||||
|
||||
- Consolidate `Price`/`Quantity` into i64/6dp (adopt IntegerPrice's API)
|
||||
- Delete the old u64 `Price` type
|
||||
- Add `Money` type (i64/6dp)
|
||||
- Add `type Ratio = f64`
|
||||
- Implement all arithmetic ops (Add, Sub, Mul, Div, Display, FromStr, Serialize)
|
||||
- Implement cross-type ops (`Price * Quantity → Money`)
|
||||
- Add Decimal interop (to_decimal, from_decimal)
|
||||
- Update all `common/` internal usage
|
||||
- All downstream crates will fail to compile — that's expected and intentional
|
||||
|
||||
### Phase 2: Core crates (3 parallel worktrees)
|
||||
|
||||
**Crates**: `trading_engine/`, `risk/`, `data/`
|
||||
|
||||
- `trading_engine/`: delete `types/financial.rs` (IntegerPrice/IntegerQuantity/IntegerMoney), use `common::Price`/`Quantity`/`Money` everywhere
|
||||
- `risk/`: replace f64 price/pnl/margin fields with Price/Money in risk_engine.rs, position_tracker.rs, risk_types.rs, VaR calculations
|
||||
- `data/`: replace f64 in OHLCV structs, PricePoint, PriceBounds, validation, feature extraction
|
||||
|
||||
### Phase 3: ML boundary (1 worktree)
|
||||
|
||||
**Crate**: `ml/`
|
||||
|
||||
- Replace f64 price fields in labeling (PricePoint, triple_barrier) with Price
|
||||
- Replace f64 price fields in features (price_features, microstructure) with Price
|
||||
- Add `.to_f32()` / `Price::from_f32()` at Candle tensor boundaries
|
||||
- Keep f32/f64 for weights, gradients, losses, learning rates (these are NOT prices)
|
||||
- Keep f64 for ensemble scores, confidence, feature importance (these are Ratios)
|
||||
|
||||
### Phase 4: Services (4 parallel worktrees)
|
||||
|
||||
**Crates**: `trading_service/`, `backtesting_service/`, `trading_agent_service/`, remaining services
|
||||
|
||||
- Replace f64 price/qty/pnl fields in each service
|
||||
- Update proto type conversions (f64 ↔ Price at gRPC boundary)
|
||||
- Update sqlx queries (Decimal ↔ Price at DB boundary)
|
||||
|
||||
### Phase 5: Peripheral (2 parallel worktrees)
|
||||
|
||||
**Crates**: `bin/fxt/`, `web-gateway/`, `testing/`
|
||||
|
||||
- Update proto-generated types
|
||||
- Update API request/response types
|
||||
- Update test fixtures and builders
|
||||
|
||||
### Per-crate migration checklist
|
||||
|
||||
1. Replace `price: f64` fields → `price: Price`
|
||||
2. Replace `quantity: f64` fields → `quantity: Quantity`
|
||||
3. Replace P&L/margin/notional f64 fields → `Money`
|
||||
4. Replace `as f64` / `as f32` casts → `.to_f64()` / `.to_f32()`
|
||||
5. Run `SQLX_OFFLINE=true cargo check -p <crate>`
|
||||
6. Fix compilation errors
|
||||
7. Run `SQLX_OFFLINE=true cargo test -p <crate> --lib`
|
||||
8. Run `cargo clippy -p <crate>`
|
||||
|
||||
## Scope Exclusions
|
||||
|
||||
- No changes to Candle internals (f32 tensors stay f32)
|
||||
- No changes to ML training loops (weights/gradients/losses stay f64/f32)
|
||||
- Ratio is a proper f64 newtype with NaN/Inf rejection at construction
|
||||
- No per-instrument tick size enforcement (future enhancement)
|
||||
- No bigdecimal removal (deferred — only used in sqlx features)
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
- **Phased rollout**: each phase is independently compilable and testable
|
||||
- **Worktree isolation**: parallel work doesn't conflict
|
||||
- **4,500+ existing tests**: catch regressions immediately
|
||||
- **Clippy deny rules**: `clippy::as_conversions` already denied — new types enforce this
|
||||
- **Saturating arithmetic**: no panics on overflow (consistent with existing IntegerPrice)
|
||||
Reference in New Issue
Block a user