foxhunt/docs/plans/2026-03-01-codebase-consolidation-design.md

Codebase Consolidation & DRY Cleanup — Design

Date: 2026-03-01 Branch: feature/codebase-consolidation Estimated savings: ~13,100 lines

Context

Following the ML inference cleanup (-8,281 lines), a comprehensive codebase audit revealed 4 major cleanup areas across 543K LOC (crates+services). The ml crate alone is 247K LOC but is well-structured; the biggest opportunities are in shared infrastructure, compliance boilerplate, and service-level glue code.

Area 1: DRY Violations (~600 lines saved)

Problem

Three type definitions exist as exact duplicates in multiple crates:

Type	Canonical Location	Duplicate Location
RetryStrategy enum	common/src/error.rs:159	trading_engine/src/types/error.rs:18
ErrorSeverity enum	common/src/error.rs:132	data/src/error.rs:441, database/src/error.rs:119
RetryConfig struct	common/src/resilience/retry.rs:40	storage/src/model_helpers.rs:141

Design

1. Keep canonical definitions in common/
2. Delete duplicates from trading_engine, data, database, storage
3. Re-export from common crate root if not already
4. Update all use statements in consuming code
5. Verify with cargo check --workspace

Risk

Low. These are exact duplicates with identical APIs. The only risk is missed import sites, which the compiler will catch.

Area 2: common/src/types.rs Split (structural, ~4,909 LOC)

Problem

common/src/types.rs is a 4,909-line god file containing 6+ unrelated domains: type aliases, service types, market data types, order types, connection types, error types, and currency types. This makes the file hard to navigate and violates the project's 500-line-per-file convention.

Design

Split into focused modules under common/src/types/:

Module	Content	Est. LOC
aliases.rs	AsyncResult, SharedHashMap, PositionMap, etc.	~80
service.rs	ServiceId, ServiceStatus, ConfigVersion	~120
market_data.rs	MarketDataEvent, QuoteEvent, TradeEvent, BarEvent, orderbook types	~1,500
order.rs	OrderType, OrderSide, OrderStatus, OrderEvent, BrokerType	~400
connection.rs	ConnectionInfo, ConnectionEvent, ConnectionStatus, ResourceLimits	~200
error.rs	CommonTypeError + trait impls	~200
currency.rs	Currency enum + Display/Default	~100
mod.rs	pub use re-exports for backward compatibility	~30

Backward Compatibility

types/mod.rs will pub use everything, so existing use common::types::* and use common::types::QuoteEvent imports continue working unchanged.

Risk

Low. Pure structural refactor with re-exports. No API changes.

Area 3: trading_engine/compliance/ Audit (~11,000 lines saved)

Problem

The compliance directory contains 16,068 LOC across 9 files:

• iso27001_compliance.rs (3,217) — ISMS scope, organization info, facilities
• compliance_reporting.rs (2,406) — Report generation
• sox_compliance.rs (2,262) — Sarbanes-Oxley controls
• audit_trails.rs (1,847) — Audit trail storage
• transaction_reporting.rs (1,714) — MiFID/Dodd-Frank TCA
• automated_reporting.rs (1,628) — Automated report scheduling
• best_execution.rs (1,101) — 29 struct/enums for best execution analysis
• mod.rs (1,107) — Module orchestration
• regulatory_api.rs (786) — API endpoints

Much of this appears to be enterprise scaffolding (ISO27001 OrganizationInfo, Location, FacilityType, ContactInfo, ISMSScope) that is unlikely to be called from production code paths in an HFT futures trading system.

Design

1. Audit usage: grep for each public type/function to find callers outside tests
2. Categorize: active (called in production) vs dormant (tests-only or zero callers)
3. Delete dormant code: Remove types and functions with zero production callers
4. Consolidate active code: Merge remaining functions into fewer, smaller files
5. Keep essentials: audit trail logging, basic compliance checks that risk/ references

Risk

Medium. Need to verify each type's usage carefully. Some compliance types may be referenced transitively through trait bounds. The #[allow(unused_variables)] at the top of best_execution.rs is a red flag suggesting it may be entirely unused.

Area 4: trading_service/enhanced_ml.rs Cleanup (~1,500 lines saved)

Problem

At 3,027 LOC, this file implements the ML gRPC service with:

• Runtime model tracking (RuntimeModelInfo) duplicating ml::ModelMetadata
• Model loading code that should delegate to EnsembleModelAdapter
• Known Status::unavailable stubs for retrain and feature_importance
• PPO/TFT inference code with hardcoded fallbacks

Design

1. Audit for legacy patterns matching deleted MLFeatureExtractor/SimpleDQNAdapter
2. Consolidate RuntimeModelInfo with existing ml::ModelMetadata types
3. Simplify model loading to use ml::ensemble::build_production_strategy()
4. Remove dead code paths that duplicate what EnsembleModelAdapter now handles
5. Keep the gRPC service interface — just slim the implementation

Risk

Medium. This file implements active gRPC endpoints. Changes must preserve the proto contract. Focus on internal simplification, not API changes.

Execution Order

1. DRY violations (creates canonical shared types)
2. types.rs split (structural, no deps on other areas)
3. compliance/ audit (largest deletion target)
4. enhanced_ml.rs cleanup (builds on Area 1 patterns)

Verification

After each area:

• SQLX_OFFLINE=true cargo check --workspace — zero errors
• SQLX_OFFLINE=true cargo clippy -p <affected_crates> --lib -- -D warnings
• SQLX_OFFLINE=true cargo test -p <affected_crates> --lib
• git diff --stat main — track cumulative line savings

Final target: -13,000+ net lines deleted while preserving all production functionality.