# Candle Hard Refactor — Complete Elimination

> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Remove candle-core, candle-nn, and candle-optimisers from the entire workspace. Replace with cudarc + cuda_autograd (GpuTensor, GpuLinear, GpuVarStore, GpuAdamW) which already exist in ml-core.

**Architecture:** Hard refactor — no bridge types, no compat shims. Delete candle from Cargo.toml, fix every compile error bottom-up (ml-core → ml-dqn → ml-ppo → ml-supervised → ml → services). Each crate is one task. Safetensors stays (direct `safetensors` crate, not candle's wrapper).

**Tech Stack:** Rust, cudarc 0.19 (direct dep, NOT via candle re-export), safetensors 0.7, cuda_autograd module

---

## Scope

- **1,369 candle references** across **271 files**
- **12 Cargo.toml** files with candle deps
- **88 files** using VarMap/VarBuilder
- **96 Linear layers** across 3 sub-crates
- **4 Module impls** to replace
- **~20 checkpoint save/load sites** using candle safetensors

## Replacement Map

| Candle Type | Replacement | Location |
|-------------|-------------|----------|
| `candle_core::Device` | `MlDevice` enum (Cpu, Cuda { device, stream }) | `ml-core/src/device.rs` (new) |
| `candle_core::DType` | Delete. BF16 unconditional on CUDA, F32 at boundaries | — |
| `candle_core::Tensor` | `GpuTensor` (ml-core cuda_autograd) | Already exists |
| `candle_core::Result<T>` / `candle_core::Error` | `Result<T, MLError>` | 68 occurrences in 20 files |
| `candle_nn::Linear` | `GpuLinear` (ml-core cuda_autograd) | Already exists |
| `candle_nn::linear()` constructor | `GpuVarStore::add_linear(name, in, out)` | Already exists |
| `candle_nn::VarMap` | `GpuVarStore` (ml-core cuda_autograd) | Already exists |
| `candle_nn::VarBuilder` | `GpuVarStore::add_linear()` factory | Already exists |
| `candle_nn::Module` trait | Direct `forward()` method on each struct | Inline, delete trait bound |
| `candle_nn::LSTM` / `LSTMConfig` / `LSTMState` | `CudaLSTM` | Already in `ml-ppo/src/cuda_nn/lstm.rs` |
| `candle_nn::rnn::RNN` trait | Direct `step()` method on CudaLSTM | Inline |
| `candle_nn::Dropout` | Pass-through (inference) or CUDA kernel | Create in cuda_autograd or inline |
| `candle_nn::LayerNorm` | GPU LayerNorm kernel | Create in cuda_autograd |
| `candle_nn::Activation` | `ActivationKernels` (cuda_autograd) | Already exists |
| `candle_nn::Init` | `cuda_autograd::init::{xavier_uniform, kaiming_uniform}` | Already exists |
| `candle_nn::Optimizer` trait | Delete trait bound, use `GpuAdamW` directly | — |
| `candle_optimisers::Adam` / `ParamsAdam` | `GpuAdamW` / `AdamWConfig` | Already exists |
| `candle_optimisers::Decay` | LR schedule field on `AdamWConfig` | Extend AdamWConfig |
| `candle_core::safetensors::*` | `safetensors` crate direct | Add dep |
| `Tensor::zeros/ones/randn` | `GpuTensor::zeros()`, init module | Already exists |
| `candle_core::cuda_backend::cudarc` re-export | `use cudarc;` direct import | 8 files in cuda_autograd |
| `candle_core::DeviceLocation` | Delete (only used in conversion code) | — |

## Pre-requisite: cudarc Import Path Fix

**CRITICAL**: Before removing candle from any Cargo.toml, all `use candle_core::cuda_backend::cudarc;` imports (8 files in cuda_autograd + native_types.rs) must change to `use cudarc;` (direct dependency). Without this, cuda_autograd itself won't compile.

## Feature Flag Rewrite

Every crate's `Cargo.toml` `[features]` section must change:
```toml
# BEFORE:
cuda = ["candle-core/cuda", "candle-nn/cuda", "cudarc"]

# AFTER:
cuda = ["cudarc"]
```

This applies to: ml-core, ml-dqn, ml-ppo, ml-supervised, ml-ensemble, ml-explainability, ml-hyperopt, ml-labeling.

## File Structure (new/modified)

### New files
- `crates/ml-core/src/device.rs` — `MlDevice` enum replacing candle Device
- `crates/ml-core/src/checkpoint.rs` — safetensors save/load for `GpuVarStore`
- `crates/ml-core/src/cuda_autograd/dropout.rs` — GPU dropout (pass-through at inference, mask at training)
- `crates/ml-core/src/cuda_autograd/layer_norm.rs` — GPU LayerNorm kernel

### Modified files (by crate, compile order)
1. `crates/ml-core/` — 23 files (foundation, must be first)
2. `crates/ml-dqn/` — 26 src + 1 test file
3. `crates/ml-ppo/` — 25 files
4. `crates/ml-supervised/` — 13 files
5. `crates/ml-ensemble/` — 4 files
6. `crates/ml-explainability/` — 1 file
7. `crates/ml-labeling/` — 1 file
8. `crates/ml-hyperopt/` — 1 file (Cargo.toml only)
9. `crates/ml/` — 104 src + 56 test + 5 example files
10. `services/trading_service/` — 3 test files
11. `testing/` — 9 files (integration + e2e)
12. Root `Cargo.toml` — remove workspace candle deps

---

## Task 0: Pre-requisite Fixes (before any Cargo.toml changes)

- [ ] **Step 1: Fix cudarc import path in cuda_autograd**

Replace in 8 files (`crates/ml-core/src/cuda_autograd/*.rs` + `native_types.rs`):
```rust
// BEFORE:
use candle_core::cuda_backend::cudarc;
// AFTER:
use cudarc;
```

- [ ] **Step 2: Create `crates/ml-core/src/cuda_autograd/dropout.rs`**

Minimal GPU dropout: identity at inference, random mask at training.

- [ ] **Step 3: Create `crates/ml-core/src/cuda_autograd/layer_norm.rs`**

GPU LayerNorm: mean/variance reduction kernel + scale/shift.

- [ ] **Step 4: Extend `AdamWConfig` with LR decay schedule**

Replace `candle_optimisers::Decay` with a `lr_schedule: Option<LrSchedule>` field.

- [ ] **Step 5: Update cuda_autograd/mod.rs exports**
- [ ] **Step 6: Compile check: `SQLX_OFFLINE=true cargo check -p ml-core`**
- [ ] **Step 7: Commit**

---

## Task 1: ml-core Candle Removal

**Files:**
- Create: `crates/ml-core/src/device.rs`
- Create: `crates/ml-core/src/checkpoint.rs`
- Modify: `crates/ml-core/src/lib.rs` — update re-exports, remove candle prelude
- Modify: `crates/ml-core/Cargo.toml` — remove candle-core, candle-nn, candle-optimisers; add safetensors; rewrite `cuda` feature
- Modify: 23 `.rs` files in ml-core that import candle

- [ ] **Step 1: Create `crates/ml-core/src/device.rs` — MlDevice enum**
- [ ] **Step 2: Create `crates/ml-core/src/checkpoint.rs` — safetensors for GpuVarStore**
- [ ] **Step 3: Update Cargo.toml — remove candle, add safetensors, rewrite features**
- [ ] **Step 4: Update lib.rs — remove candle re-exports from prelude, add MlDevice**
- [ ] **Step 5: Sweep 23 .rs files — replace all candle imports**

Key files:
- `cuda_compat.rs` (35 refs) — likely deletable (was candle compat layer)
- `tensor_ops.rs` → GpuTensor ops
- `training.rs` → GpuVarStore in UnifiedTrainable trait, `Result<_, MLError>` not `candle_core::Result`
- `xavier_init.rs` → cuda_autograd::init (delete candle_nn::Init usage)
- `optimizers/adam.rs` → GpuAdamW (delete candle_nn::Optimizer trait)
- `safety/*.rs` → GpuTensor validation
- `gpu/mod.rs` → MlDevice
- `memory_optimization/*.rs` → simplify (DType gone)

- [ ] **Step 6: Compile check: `SQLX_OFFLINE=true cargo check -p ml-core`**
- [ ] **Step 7: Commit**

---

## Task 2: ml-dqn (26 src + 1 test)

**Files:**
- Modify: `crates/ml-dqn/Cargo.toml` — remove candle deps, rewrite features
- Modify: 26 src `.rs` files + `tests/gpu_smoketest.rs`

### Key replacements:

| File | Candle refs | Key changes |
|------|------------|-------------|
| `dqn.rs` | 37 | Tensor→GpuTensor, Device→MlDevice, VarMap→GpuVarStore, Module→forward(), Decay→AdamWConfig |
| `network.rs` | ~15 | nn::Linear→GpuLinear, Module impl→forward(), VarBuilder→GpuVarStore, Dropout→cuda_autograd |
| `noisy_layers.rs` | 31 | NoisyLinear→GpuLinear + noise buffer, Module→forward() |
| `branching.rs` | 25 | BranchingDuelingQNetwork layers→GpuLinear, Dropout→cuda_autograd |
| `agent.rs` | 14 | Device→MlDevice, Tensor→GpuTensor, Decay→AdamWConfig |
| `gpu_replay_buffer.rs` | ~8 | Remove remaining Tensor wrappers (d2t_*/w_* already migrated to Result) |
| `distributional*.rs` | ~25 | C51/IQN layers→GpuLinear |
| `rainbow_network.rs` | 15 | Module impl→forward() |
| Other 15 files | 1-8 each | Mechanical import swaps, `candle_core::Result` → `Result<_, MLError>` |

- [ ] **Step 1: Update Cargo.toml + feature flags**
- [ ] **Step 2: Migrate network.rs (QNetwork: Linear→GpuLinear, Module→forward())**
- [ ] **Step 3: Migrate noisy_layers.rs (NoisyLinear)**
- [ ] **Step 4: Migrate branching.rs (BranchingDuelingQNetwork)**
- [ ] **Step 5: Migrate distributional.rs + distributional_dueling.rs**
- [ ] **Step 6: Migrate rainbow_network.rs**
- [ ] **Step 7: Migrate dqn.rs (DQNAgent — largest)**
- [ ] **Step 8: Migrate agent.rs, attention.rs, residual.rs, curiosity.rs**
- [ ] **Step 9: Migrate remaining files + tests/gpu_smoketest.rs**
- [ ] **Step 10: Compile check: `SQLX_OFFLINE=true cargo check -p ml-dqn`**
- [ ] **Step 11: Commit**

---

## Task 3: ml-ppo (25 files)

**Files:**
- Modify: `crates/ml-ppo/Cargo.toml` — remove candle deps
- Modify: 25 `.rs` files

### Key replacements:

| File | Key changes |
|------|-------------|
| `ppo.rs` (29 refs) | PPOAgent: VarMap→GpuVarStore, checkpoint→safetensors, Optimizer trait→GpuAdamW |
| `lstm_networks.rs` | **candle_nn::LSTM→CudaLSTM** (from cuda_nn/lstm.rs), RNN trait→step() |
| `continuous_policy.rs` (17 refs) | FlowPolicy layers→GpuLinear |
| `continuous_ppo.rs` (9 refs) | ContinuousPPO→GpuLinear + GpuAdamW |
| `continuous_demo.rs` (7 refs) | Demo networks→GpuLinear |
| `hidden_state_manager.rs` (9 refs) | LSTMState→CudaLSTM state |
| `flow_policy/*.rs` | Coupling layers→GpuLinear |
| `cuda_nn/*.rs` | Already cudarc-native, remove Tensor bridges |
| `action_space.rs`, `action_masking.rs`, `continuous_action_masking.rs` | Tensor→GpuTensor |
| `adaptive_entropy.rs` | candle_nn::Init→cuda_autograd::init, Optimizer→GpuAdamW |
| `trajectories.rs`, `symlog.rs` | Tensor→GpuTensor |

- [ ] **Step 1: Update Cargo.toml + features**
- [ ] **Step 2: Migrate ppo.rs (PPOAgent — checkpoint save/load critical)**
- [ ] **Step 3: Migrate lstm_networks.rs (LSTM→CudaLSTM — hardest file)**
- [ ] **Step 4: Migrate continuous_policy.rs, continuous_ppo.rs, continuous_demo.rs**
- [ ] **Step 5: Migrate hidden_state_manager.rs, adaptive_entropy.rs**
- [ ] **Step 6: Migrate flow_policy/, cuda_nn/ bridges**
- [ ] **Step 7: Migrate remaining (trajectories, action_masking, symlog, action_space)**
- [ ] **Step 8: Compile check: `SQLX_OFFLINE=true cargo check -p ml-ppo`**
- [ ] **Step 9: Commit**

---

## Task 4: ml-supervised (13 files)

**Files:**
- Modify: `crates/ml-supervised/Cargo.toml` — remove candle deps
- Modify: 13 `.rs` files

### Key replacements by model:

| Model | Files | Key changes |
|-------|-------|-------------|
| TFT | 5 | GatedResidualNetwork→GpuLinear, LSTMEncoder→GpuLinear, TemporalAttention→GpuLinear |
| Mamba2 | 5 | SSD layer→GpuLinear + custom kernels, selective state |
| Liquid CfC | 3 | CfC dynamics→GpuLinear (already partially migrated) |
| gpu_tensor.rs | 1 | Remove `from_candle_tensor()` / `to_candle_tensor()` bridge methods |

- [ ] **Step 1: Update Cargo.toml + features**
- [ ] **Step 2: Migrate TFT**
- [ ] **Step 3: Migrate Mamba2**
- [ ] **Step 4: Migrate Liquid CfC**
- [ ] **Step 5: Remove candle bridges from gpu_tensor.rs**
- [ ] **Step 6: Compile check: `SQLX_OFFLINE=true cargo check -p ml-supervised`**
- [ ] **Step 7: Commit**

---

## Task 5: Thin crates (ml-ensemble, ml-explainability, ml-labeling, ml-hyperopt)

- [ ] **Step 1: Update all 4 Cargo.toml files + feature flags**
- [ ] **Step 2: ml-ensemble — cuda_streams.rs, inference_adapter.rs, inference_ensemble.rs, stream_ensemble.rs**
- [ ] **Step 3: ml-explainability — integrated_gradients.rs**
- [ ] **Step 4: ml-labeling — gpu_acceleration.rs**
- [ ] **Step 5: Compile check all 4 crates**
- [ ] **Step 6: Commit**

---

## Task 6: ml crate — src/ (104 files)

**Files:**
- Modify: `crates/ml/Cargo.toml` — remove candle-nn dep
- Modify: 104 `.rs` source files

### 6a: cuda_pipeline/ (14 files, partially migrated)
- [ ] Complete gpu_weights.rs, signal_adapter.rs, gpu_ppo_collector.rs
- [ ] Remove remaining Tensor/Device/DType refs from all 14 files

### 6b: trainers/ (~20 files)
- [ ] trainers/dqn/ (config.rs, trainer/, data_loading.rs, fused_training.rs, smoke_tests/)
- [ ] trainers/ppo.rs, trainers/tft/, trainers/tlob.rs
- [ ] trainers/liquid.rs, trainers/mamba2.rs, trainers/online_learning.rs, trainers/mod.rs

### 6c: ensemble/adapters/ + hyperopt/adapters/ (22 files)
- [ ] All 10 ensemble adapters
- [ ] All 11 hyperopt adapters + shared_data.rs

### 6d: model directories (~15 files)
- [ ] dqn/, ppo/, tft/, liquid/, mamba/, tgnn/, tlob/, kan/, xlstm/, diffusion/

### 6e: infrastructure (~30 files)
- [ ] preprocessing.rs, inference.rs, inference_validator.rs
- [ ] transformers/ (attention, features, financial_transformer, hft_transformer, benchmarks)
- [ ] validation/ (adapters, harness, ppo_adapter, regime_analysis)
- [ ] benchmark/, data_loaders/, flash_attention/, training_pipeline.rs, portfolio_transformer.rs (LayerNorm→cuda_autograd)
- [ ] benchmarks.rs, features/mod.rs, features/multi_timeframe.rs, data_pipeline/

- [ ] **Compile check: `SQLX_OFFLINE=true cargo check -p ml`**
- [ ] **Commit**

---

## Task 7: ml crate — tests/ + examples/ (56 test + 5 example files)

Mechanical: replace `use candle_core::{Device, Tensor, DType}` with ml-core types.

- [ ] **Step 1: Batch-replace imports in all 56 test files**
- [ ] **Step 2: Update 5 example binaries**
- [ ] **Step 3: Compile check: `SQLX_OFFLINE=true cargo check -p ml --tests --examples`**
- [ ] **Step 4: Commit**

---

## Task 8: Services + Testing + Cleanup

**Files:**
- Modify: `services/trading_service/tests/` (3 files)
- Modify: `testing/integration/` (9 files including lib.rs, gpu/mod.rs, 7 GPU tests)
- Modify: `testing/e2e/Cargo.toml` + any `.rs` files with candle imports
- Modify: Root `Cargo.toml` — delete candle workspace deps

- [ ] **Step 1: Fix trading_service test files**
- [ ] **Step 2: Fix testing/integration/ files (9 files)**
- [ ] **Step 3: Fix testing/e2e/ (Cargo.toml + source files)**
- [ ] **Step 4: Remove candle-core, candle-nn from root Cargo.toml [workspace.dependencies]**
- [ ] **Step 5: Full workspace compile: `SQLX_OFFLINE=true cargo check --workspace`**
- [ ] **Step 6: Full workspace clippy: `SQLX_OFFLINE=true cargo clippy --workspace`**
- [ ] **Step 7: Commit**

---

## Task 9: Verify + Final

- [ ] **Step 1: Run ml-core tests: `SQLX_OFFLINE=true cargo test -p ml-core --lib`**
- [ ] **Step 2: Run ml-dqn tests: `SQLX_OFFLINE=true cargo test -p ml-dqn --lib`**
- [ ] **Step 3: Run ml-ppo tests: `SQLX_OFFLINE=true cargo test -p ml-ppo --lib`**
- [ ] **Step 4: Verify no candle references remain:**

```bash
rg "candle_core|candle_nn|candle_optimisers" --type rust
# Expected: 0 matches
```

- [ ] **Step 5: Verify candle not in any Cargo.toml:**

```bash
grep -r "candle" */Cargo.toml crates/*/Cargo.toml services/*/Cargo.toml testing/*/Cargo.toml
# Expected: 0 matches
```

- [ ] **Step 6: Final commit + tag**

Note: Checkpoint format is greenfield — no backward compat needed. GpuVarStore safetensors is the canonical format going forward.

---

## Execution Strategy

This plan has **10 tasks** (0-9). Task 0 is a pre-req that must finish first.

**Recommended: 3-agent pipeline after Task 0+1**
- Agent A: Tasks 0→1→2 (pre-req + ml-core + ml-dqn) — critical path foundation
- Agent B: Tasks 3→4 (ml-ppo + ml-supervised) — starts after Task 1 completes
- Agent C: Tasks 5→6→7→8 (thin crates + ml + services) — starts after Tasks 2+3+4 complete
- Task 9 runs after all agents finish

```
Task 0 → Task 1 ──→ Task 2 ──────────────────→ Task 6 → Task 7 → Task 8 → Task 9
              └──→ Task 3 → Task 4 ──→ Task 5 ─┘
```

Tasks 1→2→6→7→8→9 are the **critical path**.
Tasks 3+4 can run **in parallel** with Task 2 (only depend on Task 1).
Task 5 can start after Task 1 (thin crates only depend on ml-core).
Task 6 must wait for Tasks 2, 3, 4, 5 (ml crate depends on all sub-crates).

**Estimated: 4-6 hours with 3 parallel agents, 8-12 hours single agent.**