Detailed step-by-step plan covering: S3 bucket creation, CI builder
updates, 3 base image Dockerfiles, CI pipeline rewiring, 8 cache PVCs,
all service deployment YAML updates, training job template, and cleanup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace per-service Docker images with generic base images + S3-based binary
distribution. CI compiles and uploads stripped binaries to S3. Pods fetch
binaries via initContainer with PVC cache fallback for trading resilience.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single monolithic kernel with in-kernel GAE, full portfolio simulation,
shared device functions header, 5-layer critic matching trainer exactly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Full design for monolithic CUDA kernel that runs the entire DQN
experience collection pipeline on GPU: Dueling Q-network inference
(online + target), epsilon-greedy action selection, portfolio
simulation, barrier tracking, diversity entropy, curiosity inference,
and reward combination. 128 parallel episodes × 500 timesteps =
64,000 experiences per kernel launch with zero CPU roundtrips.
Key decisions:
- Approach A (monolithic kernel) for speed + accuracy
- Parallel episode architecture (N threads × L timesteps)
- Q-network + curiosity inference in raw CUDA (matrix-vector multiply)
- Curiosity training stays on Candle (periodic batch updates)
- TD error pre-computed for priority replay
- Weight sync after gradient updates (~1.2MB upload)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
12-worktree phased migration: foundation types → core crates →
ML boundary → services → peripheral → cleanup. Full code for
Phase 1 (Price/Quantity/Money/Ratio in common/financial_types),
migration patterns for Phases 2-6.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate competing Price/Quantity implementations into canonical
i64 fixed-point types in common/. Adds Money and Ratio newtypes.
Phased migration plan across 5 phases using parallel worktrees.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all `.to_string().parse::<f32/f64>()` patterns with
`num_traits::ToPrimitive` methods (`.to_f32()`, `.to_f64()`).
Each string roundtrip heap-allocated per conversion — fatal in
DQN hot loop (300K+ bars × epochs). Decimal stays as canonical
financial type; conversions happen at GPU/float boundaries only.
Also fixes blocking_read() in async context (risk_integration.rs).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Route DQN/PPO to cheaper L4 (CPU-bound), supervised models to L40S
(GPU-bound). Separate CI templates with tuned resource requests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5-phase agent loop (diagnose → plan → implement → verify → deliver)
with GitLab issues as durable communication/handoff layer. OpenHands
on Kapsule K8s runtime, LiteLLM proxy routing to Claude Opus/Sonnet
and Scaleway devstral. Rust dispatcher service. Auto-merge for CI
failure fixes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
10 tasks across 2 workstreams (NAT + Metrics), mostly parallelizable.
Includes Terraform modules, K8s manifests, Helm values, and runbook.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers two workstreams:
1. Scaleway Public Gateway for NAT (all pools private-only)
2. Scaleway Cockpit + Alloy + DCGM Exporter for full metrics
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
K8s Job dispatch from ml_training_service with Rust sidecar uploader,
S3 artifact storage, and model promotion with fxt CLI approval gate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a parent/child GitLab CI pipeline for ML model training:
- Generator script produces per-model hyperopt/train/evaluate jobs
- Parent pipeline (.gitlab-ci-training.yml) with manual trigger
- NFS-backed ReadWriteMany PVC for shared training outputs
- Hyperopt params wired into training binaries (DQN, PPO, TFT, Mamba2)
- Shared DBN loader eliminates duplicate code across hyperopt adapters
- Supervised hyperopt unified to DBN data (was parquet-only)
Pipeline: hyperopt (4 models) → train (10 models) → evaluate ensemble
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The check stage ran cargo check + clippy (~6min) before tests, but
the test stage compiles the same workspace anyway. Skip check with
an echo pass-through to save ~6min per pipeline. Easy to uncomment
when clippy enforcement is needed (e.g., before releases).
Also adds sccache stats to the test job for cache monitoring.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate 21 training examples into 2 binaries:
- train_baseline_rl (DQN, PPO with RL walk-forward)
- train_baseline_supervised (8 models via UnifiedTrainable)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers tfstate migration nl-ams→fr-par, resource import for
drift gaps (sccache bucket, foxhunt-ci registry, DNS records),
and GitLab CI/CD pipeline (plan on MR, apply on merge, weekly
drift detection).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Terraform pool, PVC, Dockerfile, devcontainer.json, pod template,
CI job, setup script, infra apply, GitLab badge. TDD-style steps
with exact file paths and commands.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DevPod on Kapsule with GP1-L (16 vCPU, 64GB) autoscaling to zero,
persistent PVCs for Claude Code + cargo cache, "Open in DevPod"
GitLab badge, CI-built devcontainer image.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers: in-cluster GitLab CE with external Postgres/Redis, dedicated
ci-build node pool (scale-to-zero), Kaniko image builds, GitLab
built-in registry, and 4-phase migration plan with rollback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix container registry region nl-ams → fr-par across all CI jobs
- Add sccache build-args to training image build (no-op fallback for local)
- Add rclone to training Docker runtime for S3 output sync
- Update train.sh: S3 sync on Job completion via rclone env-var config,
--run-id tracking, evaluate preset for walk-forward evaluation
- Add s3-credentials Secret template (.example, apply via kubectl)
- Add design doc and implementation plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaced imperative shell scripts with declarative OpenTofu modules
and Terragrunt DRY configuration. 16 tasks across 4 batches.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
14 tasks across 5 batches: cluster foundation, CI/CD,
paper trading deployment, GPU training, and verification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 18 findings now have resolution status:
- 8 fixed with real implementations
- 3 fixed with honest errors (instead of fake data)
- 4 deferred (need infrastructure)
- 3 acceptable as-is
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8 critical stubs found in risk-data, adaptive-strategy, trading_service,
and ml/integration. 3 crate groups clean (infra, frontend, CLI).
Priority: VPIN calculator, portfolio positions, enhanced_ml endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved design for full-codebase audit using 6 parallel agents
scanning 14 stub patterns, plus lint prevention in pre-commit hook.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ignore ML checkpoints, trained model safetensors, stray ml/ml/ dir,
and .claude/worktrees/. Clean up duplicate hive-mind-prompt entries.
Add 17 design/implementation plan docs from 2026-02-20 to 2026-02-22.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Detailed TDD implementation plan for wiring all 10 ML models into
the ensemble coordinator. Covers 5 new inference adapters (KAN, TGGN,
xLSTM, TLOB, Diffusion), main.rs registration with equal weights,
market data wiring, E2E integration test, Docker/CI validation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire existing components into end-to-end pipeline: 5 missing inference
adapters, 10-model registration, market data feeding, integration test,
Docker build validation, and CI pipeline triage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
