# ML Crate Split Phase 2 — Domain-Aligned Extraction

**Date**: 2026-03-08
**Approach**: B (domain-aligned extraction, ml becomes orchestration layer)
**Target**: 91K LOC → ~12K LOC (87% reduction)

## Context

Phase 1 (`feature/ml-crate-split`, merged as `121962b7`) extracted types and shared
infrastructure into 23 sub-crates. The heavy implementation code (trainers, hyperopt
adapters, CUDA pipeline, model architectures) remained in the main `ml` crate.

## Extraction Groups

### Group 1: Trainers → model sub-crates (18K LOC)

| Source | Destination |
|--------|-------------|
| `trainers/dqn/` (trainer, config, data_loading) | `ml-dqn` |
| `trainers/tft/` | `ml-supervised` |
| `trainers/ppo.rs` | `ml-ppo` |
| `trainers/online_learning.rs` | `ml-core` |
| `trainers/mod.rs` (registry glue) | stays in `ml` |

### Group 2: Hyperopt adapters → ml-hyperopt (12.7K LOC)

Move all 12 adapters + campaign/shared_data/tests into `ml-hyperopt`.
`ml-hyperopt` gains deps on `ml-dqn`, `ml-ppo`, `ml-supervised`.

### Group 3: Model impl remnants → model sub-crates (~5.7K LOC)

| Source | Destination |
|--------|-------------|
| `ml/src/dqn/` | merge into `ml-dqn` |
| `ml/src/tft/` (training, trainable_adapter) | merge into `ml-supervised` |
| `ml/src/ppo/` | merge into `ml-ppo` |
| `ml/src/liquid/`, `tgnn/`, `kan/`, `xlstm/`, `diffusion/`, `mamba/`, `tlob/` | merge into `ml-supervised` |

### Group 4: Infrastructure → existing sub-crates (~11K LOC)

| Source | Destination |
|--------|-------------|
| `cuda_pipeline/` (4K) | `ml-core` |
| `data_loaders/` (4.4K) + `data_pipeline/` (1.5K) + `training/` (1.2K) | `ml-data` |
| `features/` remaining (4.3K) | merge into `ml-features` |
| `ensemble/` remaining (3.4K) | merge into `ml-ensemble` |
| `flash_attention/` (1.2K) | `ml-core` |
| `microstructure/` (682) | `ml-features` |

### Group 5: New sub-crates (8.3K LOC)

| Source | Destination |
|--------|-------------|
| `benchmark/` (6K) | new `ml-benchmark` |
| `deployment/` (2.3K) | new `ml-deployment` |

### Group 6: Small modules → existing homes

- `model_registry/` + `registry/` (1.1K) → `ml-core`
- Thin re-export stubs (backtesting, validation, etc.) → deleted, direct imports

## What stays in ml (~12K LOC — orchestration layer)

- `lib.rs` — re-exports, From impls, prelude (~500)
- `integration/` — inference engine, performance monitor (3.7K)
- `inference.rs` + `inference_validator.rs` — inference orchestration (1.8K)
- `model_factory.rs` — model instantiation (~500)
- `training_pipeline.rs` — training orchestration (847)
- `portfolio_transformer.rs` — portfolio transformer (814)
- `preprocessing.rs` — data preprocessing (754)
- `examples.rs` — usage examples (892)
- `bridge.rs` — type system bridge (~400)
- `trainers/mod.rs` — trainer registry/factory (~500)
- `batch_processing.rs`, `walk_forward.rs`, `data_loader.rs` — misc glue (~600)

## Dependency Graph (post-split)

```
ml-core (foundation)
  ↑
ml-dqn, ml-ppo, ml-supervised (models + trainers)
  ↑
ml-hyperopt (optimizer framework + all model adapters)
  ↑
ml (orchestration: inference, factory, pipeline)
```

## Execution Strategy

- Work in a git worktree for isolation
- Swarm of parallel agents per extraction group
- Each group: move code, update imports, fix Cargo.toml deps, run clippy+tests
- Groups 1-3 are interdependent (trainer moves affect hyperopt adapter imports)
- Groups 4-6 are independent of each other and of 1-3
- Final: update all downstream crates (services, binaries) that import from `ml`