12fdd18223
Deleted: - DQN::compute_loss_internal (280 lines) — old Candle forward+loss - DQN::train_step (55 lines) — old Candle training step - DQN::compute_gradients (47 lines) — old gradient accumulation - ComputeLossResult struct — only used by deleted functions - RegimeConditionalDQN::train_step (65 lines) — old dispatch - RegimeConditionalDQN::train_step_gpu_regime (100 lines) — old GPU path - RegimeConditionalDQN::compute_gradients_gpu (130 lines) — old regime gradients - RegimeConditionalDQN::compute_gradients (92 lines) — old dispatch - DQNAgentType::train_step dispatch — dead - DQNAgentType::compute_gradients dispatch — dead - GpuDqnTrainer::upload_batch (71 lines) — old CPU→GPU upload - train_step.rs (500 lines) — entire module including ensure_fused_ctx - dqn_benchmark.rs — used old train_step - examples.rs — used old train_step - validation/adapters.rs (289 lines) — used old train_step - dqn/trainable_adapter.rs — used old train_step - gpu_smoketest.rs — tested old train_step - Gradient accumulation path in training_loop.rs (144 lines) - IQN d_h_s2().clone() → raw pointer (zero alloc) - Causal intervention format! string alloc removed - Dead HER relabel functions (320 lines) Kept: - ensure_fused_ctx logic inlined into training_loop.rs - set_noise_sigma_scale re-added to RegimeConditionalDQN Fixed: - GpuReplayBuffer max_batch_size wired from batch_size parameter (was hardcoded 1024, blocking batch_size=8192) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
128 lines
4.7 KiB
Markdown
128 lines
4.7 KiB
Markdown
# Foxhunt — HFT Trading System
|
|
|
|
## Behavioral Rules
|
|
|
|
- Do what has been asked; nothing more, nothing less
|
|
- NEVER create files unless absolutely necessary
|
|
- ALWAYS prefer editing an existing file to creating a new one
|
|
- NEVER proactively create documentation files (*.md) unless explicitly requested
|
|
- NEVER save working files, text/mds, or tests to the root folder
|
|
- ALWAYS read a file before editing it
|
|
- NEVER commit secrets, credentials, or .env files
|
|
|
|
## Project Overview
|
|
|
|
Rust-based high-frequency trading system with ML-driven strategies (DQN, PPO, TFT, Mamba2), deployed on Kubernetes (Scaleway Kapsule) via Argo Workflows.
|
|
|
|
### Tech Stack
|
|
|
|
| Component | Technology |
|
|
|-----------|------------|
|
|
| Language | Rust 1.85+, Edition 2021 |
|
|
| Async | Tokio 1.40 (multi-threaded) |
|
|
| RPC | Tonic 0.14 (gRPC), tonic-web (browser gateway) |
|
|
| Database | PostgreSQL 16 (SQLx, offline mode), Redis 7, QuestDB 8.2.3 |
|
|
| ML | DQN, PPO, TFT, Mamba2 (Rust inference) |
|
|
| Frontend | React 19, TypeScript, Tailwind CSS, Vite (`web-dashboard/`) |
|
|
| Orchestration | Kubernetes + Argo Workflows |
|
|
| Build Cache | sccache (MinIO S3 backend) |
|
|
| GPU | CUDA 12.4, L40S (48GB), H100 (80GB) |
|
|
|
|
## Project Structure
|
|
|
|
| Directory | Purpose |
|
|
|-----------|---------|
|
|
| `crates/` (40 crates) | Core libs: trading engine, ML models, data, backtesting, risk mgmt |
|
|
| `services/` (8 services) | Microservices: API, Trading, ML Training, Data Acquisition, Backtesting, Broker Gateway, Trading Agent |
|
|
| `bin/fxt/` | CLI binary |
|
|
| `web-dashboard/` | React dashboard (gRPC-web) |
|
|
| `testing/` | Integration, e2e, load, stress tests |
|
|
| `infra/k8s/` | K8s manifests, Argo Workflows, network policies, monitoring |
|
|
| `infra/docker/` | Dockerfiles (ci-builder, runtime, training-runtime) |
|
|
| `scripts/` | Build, deployment, health-check, GPU scripts |
|
|
| `migrations/` | PostgreSQL migrations |
|
|
| `proto/` | Protocol Buffer definitions |
|
|
| `config/` | Configuration files and templates |
|
|
|
|
## Build & Test
|
|
|
|
```bash
|
|
# Build
|
|
cargo build --release
|
|
cargo build --profile=dev-release # Fast iteration (thin LTO)
|
|
cargo build -p <service-name> --release # Single service
|
|
|
|
# Test
|
|
cargo test --workspace
|
|
cargo test -p <crate-name>
|
|
cargo test --test integration
|
|
|
|
# Benchmarks
|
|
cargo bench --bench performance_regression
|
|
cargo bench --bench trading_latency
|
|
cargo bench --bench database_performance
|
|
```
|
|
|
|
- `SQLX_OFFLINE=true` — compiled SQL queries (no live DB needed for build)
|
|
- `SCCACHE_BUCKET=foxhunt-sccache` — shared compile cache
|
|
|
|
## Deployment (Argo Workflows)
|
|
|
|
```bash
|
|
./scripts/argo-compile-deploy.sh [--branch main] [--commit <sha>]
|
|
./scripts/argo-train.sh [--model dqn|ppo|supervised]
|
|
```
|
|
|
|
- **Node pools**: `ci-compile-cpu` (compile/test), `ci-training` (L40S GPU), `ci-training-h100` (H100 GPU)
|
|
- **Training data**: PVC at `/mnt/training-data/futures-baseline`
|
|
- **Docker**: Kaniko push to Scaleway Container Registry
|
|
- **Legacy CI**: `.gitlab-ci.yml` exists but disabled (`when: never`), Argo is primary
|
|
|
|
## File Organization Rules
|
|
|
|
- NEVER save to root folder
|
|
- `/src`, `/crates` — Rust source code
|
|
- `/tests`, `/testing` — test files
|
|
- `/docs` — documentation
|
|
- `/config` — configuration
|
|
- `/scripts` — utility scripts
|
|
- `/infra` — infrastructure (K8s, Docker)
|
|
|
|
## Security Rules
|
|
|
|
- NEVER hardcode API keys, secrets, or credentials
|
|
- NEVER commit .env files or trading credentials
|
|
- Always validate input at system boundaries
|
|
- Sanitize file paths to prevent directory traversal
|
|
- Broker credentials (AMP Futures/CQG FIX 4.4) are K8s secrets — never log them
|
|
|
|
## Concurrency: 1 MESSAGE = ALL RELATED OPERATIONS
|
|
|
|
- All operations MUST be concurrent/parallel in a single message
|
|
- Use Claude Code's Task tool for spawning agents
|
|
- ALWAYS batch ALL todos in ONE TodoWrite call
|
|
- ALWAYS spawn ALL agents in ONE message via Task tool
|
|
- ALWAYS batch ALL file reads/writes/edits in ONE message
|
|
- ALWAYS batch ALL Bash commands in ONE message
|
|
|
|
## RuFlo V3 Integration
|
|
|
|
Hooks are configured in `.claude/settings.json` and run local scripts from `.claude/helpers/`. All hooks have timeouts — no network calls on the hot path.
|
|
|
|
### CLI Commands
|
|
|
|
```bash
|
|
npx ruflo@latest status # Check status
|
|
npx ruflo@latest hooks metrics # View learning metrics
|
|
npx ruflo@latest hooks pretrain # Bootstrap from repository
|
|
npx ruflo@latest hooks build-agents # Generate agent configs
|
|
```
|
|
|
|
## important-instruction-reminders
|
|
|
|
Do what has been asked; nothing more, nothing less.
|
|
NEVER create files unless they're absolutely necessary for achieving your goal.
|
|
ALWAYS prefer editing an existing file to creating a new one.
|
|
NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly requested by the User.
|
|
Never save working files, text/mds and tests to the root folder.
|