foxhunt/docs/superpowers/plans/2026-03-17-candle-hard-refactor.md

Candle Hard Refactor — Complete Elimination

For agentic workers: REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Remove candle-core, candle-nn, and candle-optimisers from the entire workspace. Replace with cudarc + cuda_autograd (GpuTensor, GpuLinear, GpuVarStore, GpuAdamW) which already exist in ml-core.

Architecture: Hard refactor — no bridge types, no compat shims. Delete candle from Cargo.toml, fix every compile error bottom-up (ml-core → ml-dqn → ml-ppo → ml-supervised → ml → services). Each crate is one task. Safetensors stays (direct safetensors crate, not candle's wrapper).

Tech Stack: Rust, cudarc 0.19 (direct dep, NOT via candle re-export), safetensors 0.7, cuda_autograd module

---

Scope

• 1,369 candle references across 271 files
• 12 Cargo.toml files with candle deps
• 88 files using VarMap/VarBuilder
• 96 Linear layers across 3 sub-crates
• 4 Module impls to replace
• ~20 checkpoint save/load sites using candle safetensors

Replacement Map

Candle Type	Replacement	Location
candle_core::Device	MlDevice enum (Cpu, Cuda { device, stream })	ml-core/src/device.rs (new)
candle_core::DType	Delete. BF16 unconditional on CUDA, F32 at boundaries	—
candle_core::Tensor	GpuTensor (ml-core cuda_autograd)	Already exists
candle_core::Result<T> / candle_core::Error	Result<T, MLError>	68 occurrences in 20 files
candle_nn::Linear	GpuLinear (ml-core cuda_autograd)	Already exists
candle_nn::linear() constructor	GpuVarStore::add_linear(name, in, out)	Already exists
candle_nn::VarMap	GpuVarStore (ml-core cuda_autograd)	Already exists
candle_nn::VarBuilder	GpuVarStore::add_linear() factory	Already exists
candle_nn::Module trait	Direct forward() method on each struct	Inline, delete trait bound
candle_nn::LSTM / LSTMConfig / LSTMState	CudaLSTM	Already in ml-ppo/src/cuda_nn/lstm.rs
candle_nn::rnn::RNN trait	Direct step() method on CudaLSTM	Inline
candle_nn::Dropout	Pass-through (inference) or CUDA kernel	Create in cuda_autograd or inline
candle_nn::LayerNorm	GPU LayerNorm kernel	Create in cuda_autograd
candle_nn::Activation	ActivationKernels (cuda_autograd)	Already exists
candle_nn::Init	cuda_autograd::init::{xavier_uniform, kaiming_uniform}	Already exists
candle_nn::Optimizer trait	Delete trait bound, use GpuAdamW directly	—
candle_optimisers::Adam / ParamsAdam	GpuAdamW / AdamWConfig	Already exists
candle_optimisers::Decay	LR schedule field on AdamWConfig	Extend AdamWConfig
candle_core::safetensors::*	safetensors crate direct	Add dep
Tensor::zeros/ones/randn	GpuTensor::zeros(), init module	Already exists
candle_core::cuda_backend::cudarc re-export	use cudarc; direct import	8 files in cuda_autograd
candle_core::DeviceLocation	Delete (only used in conversion code)	—

Pre-requisite: cudarc Import Path Fix

CRITICAL: Before removing candle from any Cargo.toml, all use candle_core::cuda_backend::cudarc; imports (8 files in cuda_autograd + native_types.rs) must change to use cudarc; (direct dependency). Without this, cuda_autograd itself won't compile.

Feature Flag Rewrite

Every crate's Cargo.toml [features] section must change:

# BEFORE:
cuda = ["candle-core/cuda", "candle-nn/cuda", "cudarc"]

# AFTER:
cuda = ["cudarc"]

This applies to: ml-core, ml-dqn, ml-ppo, ml-supervised, ml-ensemble, ml-explainability, ml-hyperopt, ml-labeling.

File Structure (new/modified)

New files

• crates/ml-core/src/device.rs — MlDevice enum replacing candle Device
• crates/ml-core/src/checkpoint.rs — safetensors save/load for GpuVarStore
• crates/ml-core/src/cuda_autograd/dropout.rs — GPU dropout (pass-through at inference, mask at training)
• crates/ml-core/src/cuda_autograd/layer_norm.rs — GPU LayerNorm kernel

Modified files (by crate, compile order)

1. crates/ml-core/ — 23 files (foundation, must be first)
2. crates/ml-dqn/ — 26 src + 1 test file
3. crates/ml-ppo/ — 25 files
4. crates/ml-supervised/ — 13 files
5. crates/ml-ensemble/ — 4 files
6. crates/ml-explainability/ — 1 file
7. crates/ml-labeling/ — 1 file
8. crates/ml-hyperopt/ — 1 file (Cargo.toml only)
9. crates/ml/ — 104 src + 56 test + 5 example files
10. services/trading_service/ — 3 test files
11. testing/ — 9 files (integration + e2e)
12. Root Cargo.toml — remove workspace candle deps

---

Task 0: Pre-requisite Fixes (before any Cargo.toml changes)

• Step 1: Fix cudarc import path in cuda_autograd

Replace in 8 files (crates/ml-core/src/cuda_autograd/*.rs + native_types.rs):

// BEFORE:
use candle_core::cuda_backend::cudarc;
// AFTER:
use cudarc;

• Step 2: Create crates/ml-core/src/cuda_autograd/dropout.rs

Minimal GPU dropout: identity at inference, random mask at training.

• Step 3: Create crates/ml-core/src/cuda_autograd/layer_norm.rs

GPU LayerNorm: mean/variance reduction kernel + scale/shift.

• Step 4: Extend AdamWConfig with LR decay schedule

Replace candle_optimisers::Decay with a lr_schedule: Option<LrSchedule> field.

• Step 5: Update cuda_autograd/mod.rs exports
• Step 6: Compile check: SQLX_OFFLINE=true cargo check -p ml-core
• Step 7: Commit

---

Task 1: ml-core Candle Removal

Files:

• Create: crates/ml-core/src/device.rs

• Create: crates/ml-core/src/checkpoint.rs

• Modify: crates/ml-core/src/lib.rs — update re-exports, remove candle prelude

• Modify: crates/ml-core/Cargo.toml — remove candle-core, candle-nn, candle-optimisers; add safetensors; rewrite cuda feature

• Modify: 23 .rs files in ml-core that import candle

• Step 1: Create crates/ml-core/src/device.rs — MlDevice enum

• Step 2: Create crates/ml-core/src/checkpoint.rs — safetensors for GpuVarStore

• Step 3: Update Cargo.toml — remove candle, add safetensors, rewrite features

• Step 4: Update lib.rs — remove candle re-exports from prelude, add MlDevice

• Step 5: Sweep 23 .rs files — replace all candle imports

Key files:

• cuda_compat.rs (35 refs) — likely deletable (was candle compat layer)

• tensor_ops.rs → GpuTensor ops

• training.rs → GpuVarStore in UnifiedTrainable trait, Result<_, MLError> not candle_core::Result

• xavier_init.rs → cuda_autograd::init (delete candle_nn::Init usage)

• optimizers/adam.rs → GpuAdamW (delete candle_nn::Optimizer trait)

• safety/*.rs → GpuTensor validation

• gpu/mod.rs → MlDevice

• memory_optimization/*.rs → simplify (DType gone)

• Step 6: Compile check: SQLX_OFFLINE=true cargo check -p ml-core

• Step 7: Commit

---

Task 2: ml-dqn (26 src + 1 test)

Files:

• Modify: crates/ml-dqn/Cargo.toml — remove candle deps, rewrite features
• Modify: 26 src .rs files + tests/gpu_smoketest.rs

Key replacements:

File	Candle refs	Key changes
dqn.rs	37	Tensor→GpuTensor, Device→MlDevice, VarMap→GpuVarStore, Module→forward(), Decay→AdamWConfig
network.rs	~15	nn::Linear→GpuLinear, Module impl→forward(), VarBuilder→GpuVarStore, Dropout→cuda_autograd
noisy_layers.rs	31	NoisyLinear→GpuLinear + noise buffer, Module→forward()
branching.rs	25	BranchingDuelingQNetwork layers→GpuLinear, Dropout→cuda_autograd
agent.rs	14	Device→MlDevice, Tensor→GpuTensor, Decay→AdamWConfig
gpu_replay_buffer.rs	~8	Remove remaining Tensor wrappers (d2t_/w_ already migrated to Result)
distributional*.rs	~25	C51/IQN layers→GpuLinear
rainbow_network.rs	15	Module impl→forward()
Other 15 files	1-8 each	Mechanical import swaps, candle_core::Result → Result<_, MLError>

• Step 1: Update Cargo.toml + feature flags
• Step 2: Migrate network.rs (QNetwork: Linear→GpuLinear, Module→forward())
• Step 3: Migrate noisy_layers.rs (NoisyLinear)
• Step 4: Migrate branching.rs (BranchingDuelingQNetwork)
• Step 5: Migrate distributional.rs + distributional_dueling.rs
• Step 6: Migrate rainbow_network.rs
• Step 7: Migrate dqn.rs (DQNAgent — largest)
• Step 8: Migrate agent.rs, attention.rs, residual.rs, curiosity.rs
• Step 9: Migrate remaining files + tests/gpu_smoketest.rs
• Step 10: Compile check: SQLX_OFFLINE=true cargo check -p ml-dqn
• Step 11: Commit

---

Task 3: ml-ppo (25 files)

Files:

• Modify: crates/ml-ppo/Cargo.toml — remove candle deps
• Modify: 25 .rs files

Key replacements:

File	Key changes
ppo.rs (29 refs)	PPOAgent: VarMap→GpuVarStore, checkpoint→safetensors, Optimizer trait→GpuAdamW
lstm_networks.rs	candle_nn::LSTM→CudaLSTM (from cuda_nn/lstm.rs), RNN trait→step()
continuous_policy.rs (17 refs)	FlowPolicy layers→GpuLinear
continuous_ppo.rs (9 refs)	ContinuousPPO→GpuLinear + GpuAdamW
continuous_demo.rs (7 refs)	Demo networks→GpuLinear
hidden_state_manager.rs (9 refs)	LSTMState→CudaLSTM state
flow_policy/*.rs	Coupling layers→GpuLinear
cuda_nn/*.rs	Already cudarc-native, remove Tensor bridges
action_space.rs, action_masking.rs, continuous_action_masking.rs	Tensor→GpuTensor
adaptive_entropy.rs	candle_nn::Init→cuda_autograd::init, Optimizer→GpuAdamW
trajectories.rs, symlog.rs	Tensor→GpuTensor

• Step 1: Update Cargo.toml + features
• Step 2: Migrate ppo.rs (PPOAgent — checkpoint save/load critical)
• Step 3: Migrate lstm_networks.rs (LSTM→CudaLSTM — hardest file)
• Step 4: Migrate continuous_policy.rs, continuous_ppo.rs, continuous_demo.rs
• Step 5: Migrate hidden_state_manager.rs, adaptive_entropy.rs
• Step 6: Migrate flow_policy/, cuda_nn/ bridges
• Step 7: Migrate remaining (trajectories, action_masking, symlog, action_space)
• Step 8: Compile check: SQLX_OFFLINE=true cargo check -p ml-ppo
• Step 9: Commit

---

Task 4: ml-supervised (13 files)

Files:

• Modify: crates/ml-supervised/Cargo.toml — remove candle deps
• Modify: 13 .rs files

Key replacements by model:

Model	Files	Key changes
TFT	5	GatedResidualNetwork→GpuLinear, LSTMEncoder→GpuLinear, TemporalAttention→GpuLinear
Mamba2	5	SSD layer→GpuLinear + custom kernels, selective state
Liquid CfC	3	CfC dynamics→GpuLinear (already partially migrated)
gpu_tensor.rs	1	Remove from_candle_tensor() / to_candle_tensor() bridge methods

• Step 1: Update Cargo.toml + features
• Step 2: Migrate TFT
• Step 3: Migrate Mamba2
• Step 4: Migrate Liquid CfC
• Step 5: Remove candle bridges from gpu_tensor.rs
• Step 6: Compile check: SQLX_OFFLINE=true cargo check -p ml-supervised
• Step 7: Commit

---

Task 5: Thin crates (ml-ensemble, ml-explainability, ml-labeling, ml-hyperopt)

• Step 1: Update all 4 Cargo.toml files + feature flags
• Step 2: ml-ensemble — cuda_streams.rs, inference_adapter.rs, inference_ensemble.rs, stream_ensemble.rs
• Step 3: ml-explainability — integrated_gradients.rs
• Step 4: ml-labeling — gpu_acceleration.rs
• Step 5: Compile check all 4 crates
• Step 6: Commit

---

Task 6: ml crate — src/ (104 files)

Files:

• Modify: crates/ml/Cargo.toml — remove candle-nn dep
• Modify: 104 .rs source files

6a: cuda_pipeline/ (14 files, partially migrated)

• Complete gpu_weights.rs, signal_adapter.rs, gpu_ppo_collector.rs
• Remove remaining Tensor/Device/DType refs from all 14 files

6b: trainers/ (~20 files)

• trainers/dqn/ (config.rs, trainer/, data_loading.rs, fused_training.rs, smoke_tests/)
• trainers/ppo.rs, trainers/tft/, trainers/tlob.rs
• trainers/liquid.rs, trainers/mamba2.rs, trainers/online_learning.rs, trainers/mod.rs

6c: ensemble/adapters/ + hyperopt/adapters/ (22 files)

• All 10 ensemble adapters
• All 11 hyperopt adapters + shared_data.rs

6d: model directories (~15 files)

• dqn/, ppo/, tft/, liquid/, mamba/, tgnn/, tlob/, kan/, xlstm/, diffusion/

6e: infrastructure (~30 files)

• preprocessing.rs, inference.rs, inference_validator.rs

• transformers/ (attention, features, financial_transformer, hft_transformer, benchmarks)

• validation/ (adapters, harness, ppo_adapter, regime_analysis)

• benchmark/, data_loaders/, flash_attention/, training_pipeline.rs, portfolio_transformer.rs (LayerNorm→cuda_autograd)

• benchmarks.rs, features/mod.rs, features/multi_timeframe.rs, data_pipeline/

• Compile check: SQLX_OFFLINE=true cargo check -p ml

• Commit

---

Task 7: ml crate — tests/ + examples/ (56 test + 5 example files)

Mechanical: replace use candle_core::{Device, Tensor, DType} with ml-core types.

• Step 1: Batch-replace imports in all 56 test files
• Step 2: Update 5 example binaries
• Step 3: Compile check: SQLX_OFFLINE=true cargo check -p ml --tests --examples
• Step 4: Commit

---

Task 8: Services + Testing + Cleanup

Files:

• Modify: services/trading_service/tests/ (3 files)

• Modify: testing/integration/ (9 files including lib.rs, gpu/mod.rs, 7 GPU tests)

• Modify: testing/e2e/Cargo.toml + any .rs files with candle imports

• Modify: Root Cargo.toml — delete candle workspace deps

• Step 1: Fix trading_service test files

• Step 2: Fix testing/integration/ files (9 files)

• Step 3: Fix testing/e2e/ (Cargo.toml + source files)

• Step 4: Remove candle-core, candle-nn from root Cargo.toml [workspace.dependencies]

• Step 5: Full workspace compile: SQLX_OFFLINE=true cargo check --workspace

• Step 6: Full workspace clippy: SQLX_OFFLINE=true cargo clippy --workspace

• Step 7: Commit

---

Task 9: Verify + Final

• Step 1: Run ml-core tests: SQLX_OFFLINE=true cargo test -p ml-core --lib
• Step 2: Run ml-dqn tests: SQLX_OFFLINE=true cargo test -p ml-dqn --lib
• Step 3: Run ml-ppo tests: SQLX_OFFLINE=true cargo test -p ml-ppo --lib
• Step 4: Verify no candle references remain:

rg "candle_core|candle_nn|candle_optimisers" --type rust
# Expected: 0 matches

• Step 5: Verify candle not in any Cargo.toml:

grep -r "candle" */Cargo.toml crates/*/Cargo.toml services/*/Cargo.toml testing/*/Cargo.toml
# Expected: 0 matches

• Step 6: Final commit + tag

Note: Checkpoint format is greenfield — no backward compat needed. GpuVarStore safetensors is the canonical format going forward.

---

Execution Strategy

This plan has 10 tasks (0-9). Task 0 is a pre-req that must finish first.

Recommended: 3-agent pipeline after Task 0+1

• Agent A: Tasks 0→1→2 (pre-req + ml-core + ml-dqn) — critical path foundation
• Agent B: Tasks 3→4 (ml-ppo + ml-supervised) — starts after Task 1 completes
• Agent C: Tasks 5→6→7→8 (thin crates + ml + services) — starts after Tasks 2+3+4 complete
• Task 9 runs after all agents finish

Task 0 → Task 1 ──→ Task 2 ──────────────────→ Task 6 → Task 7 → Task 8 → Task 9
              └──→ Task 3 → Task 4 ──→ Task 5 ─┘

Tasks 1→2→6→7→8→9 are the critical path. Tasks 3+4 can run in parallel with Task 2 (only depend on Task 1). Task 5 can start after Task 1 (thin crates only depend on ml-core). Task 6 must wait for Tasks 2, 3, 4, 5 (ml crate depends on all sub-crates).

Estimated: 4-6 hours with 3 parallel agents, 8-12 hours single agent.