docs: add ML crate split phase 2 design (domain-aligned extraction)

Plan to reduce ml crate from 91K to ~12K LOC by extracting trainers
into model sub-crates, hyperopt adapters into ml-hyperopt, and
infrastructure into existing sub-crates. ml becomes an orchestration
layer owning inference, model factory, and training pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs/plans/2026-03-08-ml-crate-split-phase2-design.md

@@ -0,0 +1,95 @@
+# ML Crate Split Phase 2 — Domain-Aligned Extraction
+
+**Date**: 2026-03-08
+**Approach**: B (domain-aligned extraction, ml becomes orchestration layer)
+**Target**: 91K LOC → ~12K LOC (87% reduction)
+
+## Context
+
+Phase 1 (`feature/ml-crate-split`, merged as `121962b7`) extracted types and shared
+infrastructure into 23 sub-crates. The heavy implementation code (trainers, hyperopt
+adapters, CUDA pipeline, model architectures) remained in the main `ml` crate.
+
+## Extraction Groups
+
+### Group 1: Trainers → model sub-crates (18K LOC)
+
+| Source | Destination |
+|--------|-------------|
+| `trainers/dqn/` (trainer, config, data_loading) | `ml-dqn` |
+| `trainers/tft/` | `ml-supervised` |
+| `trainers/ppo.rs` | `ml-ppo` |
+| `trainers/online_learning.rs` | `ml-core` |
+| `trainers/mod.rs` (registry glue) | stays in `ml` |
+
+### Group 2: Hyperopt adapters → ml-hyperopt (12.7K LOC)
+
+Move all 12 adapters + campaign/shared_data/tests into `ml-hyperopt`.
+`ml-hyperopt` gains deps on `ml-dqn`, `ml-ppo`, `ml-supervised`.
+
+### Group 3: Model impl remnants → model sub-crates (~5.7K LOC)
+
+| Source | Destination |
+|--------|-------------|
+| `ml/src/dqn/` | merge into `ml-dqn` |
+| `ml/src/tft/` (training, trainable_adapter) | merge into `ml-supervised` |
+| `ml/src/ppo/` | merge into `ml-ppo` |
+| `ml/src/liquid/`, `tgnn/`, `kan/`, `xlstm/`, `diffusion/`, `mamba/`, `tlob/` | merge into `ml-supervised` |
+
+### Group 4: Infrastructure → existing sub-crates (~11K LOC)
+
+| Source | Destination |
+|--------|-------------|
+| `cuda_pipeline/` (4K) | `ml-core` |
+| `data_loaders/` (4.4K) + `data_pipeline/` (1.5K) + `training/` (1.2K) | `ml-data` |
+| `features/` remaining (4.3K) | merge into `ml-features` |
+| `ensemble/` remaining (3.4K) | merge into `ml-ensemble` |
+| `flash_attention/` (1.2K) | `ml-core` |
+| `microstructure/` (682) | `ml-features` |
+
+### Group 5: New sub-crates (8.3K LOC)
+
+| Source | Destination |
+|--------|-------------|
+| `benchmark/` (6K) | new `ml-benchmark` |
+| `deployment/` (2.3K) | new `ml-deployment` |
+
+### Group 6: Small modules → existing homes
+
+- `model_registry/` + `registry/` (1.1K) → `ml-core`
+- Thin re-export stubs (backtesting, validation, etc.) → deleted, direct imports
+
+## What stays in ml (~12K LOC — orchestration layer)
+
+- `lib.rs` — re-exports, From impls, prelude (~500)
+- `integration/` — inference engine, performance monitor (3.7K)
+- `inference.rs` + `inference_validator.rs` — inference orchestration (1.8K)
+- `model_factory.rs` — model instantiation (~500)
+- `training_pipeline.rs` — training orchestration (847)
+- `portfolio_transformer.rs` — portfolio transformer (814)
+- `preprocessing.rs` — data preprocessing (754)
+- `examples.rs` — usage examples (892)
+- `bridge.rs` — type system bridge (~400)
+- `trainers/mod.rs` — trainer registry/factory (~500)
+- `batch_processing.rs`, `walk_forward.rs`, `data_loader.rs` — misc glue (~600)
+
+## Dependency Graph (post-split)
+
+```
+ml-core (foundation)
+  ↑
+ml-dqn, ml-ppo, ml-supervised (models + trainers)
+  ↑
+ml-hyperopt (optimizer framework + all model adapters)
+  ↑
+ml (orchestration: inference, factory, pipeline)
+```
+
+## Execution Strategy
+
+- Work in a git worktree for isolation
+- Swarm of parallel agents per extraction group
+- Each group: move code, update imports, fix Cargo.toml deps, run clippy+tests
+- Groups 1-3 are interdependent (trainer moves affect hyperopt adapter imports)
+- Groups 4-6 are independent of each other and of 1-3
+- Final: update all downstream crates (services, binaries) that import from `ml`