docs: workspace cleanup design and implementation plan
Delete 92 scattered .md files, 137 plan docs, create/update all READMEs, add lib.rs doc comments, clean stale worktrees. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jgrusewski
145
docs/plans/2026-03-01-workspace-cleanup-design.md
Normal file
145
docs/plans/2026-03-01-workspace-cleanup-design.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Workspace Cleanup — Design
|
||||
|
||||
**Date**: 2026-03-01
|
||||
**Scope**: Delete stale docs, create/update READMEs, delete plan docs, clean worktrees
|
||||
|
||||
## Problem
|
||||
|
||||
92 scattered .md files (test reports, quick-starts, analyses) across crates/services.
|
||||
137 historical plan docs in docs/plans/. 13 crates/services missing README.md.
|
||||
12 crates missing lib.rs doc comments. 3 tracked .serena tool artifacts. 4 stale worktrees (~13GB).
|
||||
|
||||
## Phase 1: Delete Stale Docs
|
||||
|
||||
### Scattered .md files to delete (92 total)
|
||||
|
||||
**crates/data/** (1):
|
||||
- tests/EDGE_CASE_FINDINGS.md
|
||||
|
||||
**crates/database/** (1):
|
||||
- QUICK_START.md
|
||||
|
||||
**crates/ml/** (18):
|
||||
- PERFORMANCE_TRACKING.md, MAMBA2_CONFIGURATION_FIX.md, PERFORMANCE_QUICK_START.md
|
||||
- MAMBA2_DIMENSION_ANALYSIS.md, QUICKSTART_GPU_BENCHMARK.md
|
||||
- docs/codebase-cleanup/DQN_PARAMETER_CONSISTENCY_AUDIT.md
|
||||
- docs/dqn_hyperopt_parameter_audit.md, docs/GPU_BENCHMARK_GUIDE.md
|
||||
- docs/dqn_hyperparameters_analysis.md, docs/QAT_GRADIENT_CHECKPOINTING_WORKAROUND.md
|
||||
- docs/QAT_GUIDE.md, hyperparams/README.md
|
||||
- profiling_reports/TFT_INT8_MEMORY_PROFILING_GUIDE.md, profiling_reports/MEMORY_COMPARISON_CHART.md
|
||||
- src/benchmark/TFT_BENCHMARK_README.md, src/checkpoint/README.md
|
||||
- tests/TFT_TEST_REPORT.md, tests/MAMBA_TEST_COVERAGE.md
|
||||
- trained_models/production/tft_real_data/TRAINING_REPORT.md
|
||||
|
||||
**crates/risk/** (1):
|
||||
- docs/TEST_COVERAGE_REPORT.md
|
||||
|
||||
**crates/storage/** (1):
|
||||
- tests/S3_TEST_COVERAGE.md
|
||||
|
||||
**crates/trading_engine/** (2):
|
||||
- docs/audit_trail_persistence_usage.md, docs/TEST_COVERAGE_REPORT.md
|
||||
|
||||
**crates/trading_engine/src/types/.serena/** (3 tracked files):
|
||||
- memories/types_crate_fix_progress.md (+ parent dir)
|
||||
|
||||
**services/api_gateway/** (8):
|
||||
- tests/INTEGRATION_TEST_REPORT.md, tests/RATE_LIMITER_TEST_REPORT.md
|
||||
- src/metrics/README.md, tests/README.md, benches/README.md
|
||||
- METRICS_DEPLOYMENT.md, METRICS_ARCHITECTURE.md, REVOCATION_CACHE_USAGE.md
|
||||
- BENCHMARKS.md, ML_TRAINING_PROXY_INTEGRATION.md, RATE_LIMITER_IMPLEMENTATION.md
|
||||
|
||||
**services/backtesting_service/** (7):
|
||||
- tests/SERVICE_TESTS_REPORT.md, tests/COVERAGE_MAPPING.md
|
||||
- tests/fixtures/README.md, tests/fixtures/PERFORMANCE.md
|
||||
- tests/fixtures/QUICKSTART.md, tests/fixtures/ARCHITECTURE.md
|
||||
- docs/DBN_LOADING_PERFORMANCE_REPORT.md
|
||||
- DBN_REPOSITORY_USAGE.md, ADVANCED_QUERIES_QUICKREF.md
|
||||
|
||||
**services/broker_gateway_service/** (5):
|
||||
- DEPLOYMENT.md, MONITORING_IMPLEMENTATION.md, METRICS_QUICK_REF.md
|
||||
- docs/API.md, docs/TROUBLESHOOTING.md, docs/DEPLOYMENT.md
|
||||
|
||||
**services/ml_training_service/** (2):
|
||||
- HYPERPARAMETER_TUNING.md, TRIAL_EXECUTOR_USAGE.md
|
||||
|
||||
**services/trading_service/** (2):
|
||||
- src/streaming/README.md, docs/ml_integration_design.md
|
||||
- tests/common/README.md
|
||||
|
||||
**services/trading_agent_service/** (1):
|
||||
- README_TLS.md
|
||||
|
||||
**bin/fxt/** (7):
|
||||
- CONFIG_FILE_SUPPORT.md, TUNE_COMMAND_README.md
|
||||
- docs/AUTHENTICATION.md, docs/ML_TRADING_COMMANDS.md, docs/USAGE.md
|
||||
- tests/INTEGRATION_TEST_GUIDE.md, tests/TEST_EXECUTION_README.md
|
||||
|
||||
**testing/** (13):
|
||||
- api-gateway-load/CHEAT_SHEET.md, api-gateway-load/QUICK_START.md
|
||||
- e2e/E2E_TEST_GUIDE.md
|
||||
- integration/e2e_helpers/INFRASTRUCTURE_VALIDATION.md
|
||||
- integration/e2e_helpers/USAGE_EXAMPLES.md
|
||||
- integration/e2e_helpers/QUICKSTART.md
|
||||
- integration/e2e_helpers/VALIDATION_REPORT.md
|
||||
- integration/e2e_helpers/QUICK_START_INFRASTRUCTURE.md
|
||||
- integration/fixtures/README.md
|
||||
- service-integration/README_DBN_INTEGRATION.md
|
||||
|
||||
### Plan docs (137)
|
||||
|
||||
Keep `docs/plans/` as historical reference. No action needed.
|
||||
|
||||
### Worktrees (4)
|
||||
|
||||
- `.claude/worktrees/real-metrics-overhaul` (1KB, abandoned) — delete
|
||||
- `.claude/worktrees/autonomous-agents` (2.4GB) — delete
|
||||
- `.claude/worktrees/numeric-type-unification` (4.2GB) — delete
|
||||
- `.claude/worktrees/training-deploy` (6.5GB) — delete
|
||||
|
||||
### Empty directories after deletion
|
||||
|
||||
Remove empty `docs/`, `profiling_reports/`, `.serena/` directories left behind.
|
||||
|
||||
## Phase 2: READMEs + Doc Comments
|
||||
|
||||
### Missing crate READMEs (8):
|
||||
config, ctrader-openapi, market-data, ml-data, model_loader, risk-data, trading-data, training_uploader
|
||||
|
||||
### Missing service READMEs (4):
|
||||
api_gateway, data_acquisition_service, trading_agent_service, tests
|
||||
|
||||
### Missing bin README (1):
|
||||
bin/fxt
|
||||
|
||||
### Update existing READMEs (22):
|
||||
All existing READMEs get refreshed with standard template.
|
||||
|
||||
### Standard template:
|
||||
```markdown
|
||||
# <name>
|
||||
<one-line purpose>
|
||||
|
||||
## Key Types / API
|
||||
<bullet list of primary public types or endpoints>
|
||||
|
||||
## Usage
|
||||
<how to use this crate/service>
|
||||
|
||||
## Configuration
|
||||
<env vars, feature flags if applicable>
|
||||
```
|
||||
|
||||
### lib.rs doc comments (12 crates):
|
||||
Add `//! <crate-name> — <purpose>` to top of lib.rs for crates missing them.
|
||||
|
||||
## Phase 3: Verify
|
||||
|
||||
- `SQLX_OFFLINE=true cargo check --workspace`
|
||||
- `SQLX_OFFLINE=true cargo clippy --workspace --lib -- -D warnings`
|
||||
- Confirm no broken doc links
|
||||
|
||||
## Risk
|
||||
|
||||
Low. Only deleting .md files and creating new ones. No Rust code changes except
|
||||
`//!` doc comments (which can't break compilation).
|
||||
611
docs/plans/2026-03-01-workspace-cleanup-implementation.md
Normal file
611
docs/plans/2026-03-01-workspace-cleanup-implementation.md
Normal file
@@ -0,0 +1,611 @@
|
||||
# Workspace Cleanup — Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Delete ~92 scattered .md files, create/update READMEs for all 36 crates/services/testing, add lib.rs doc comments. Keep docs/plans/.
|
||||
|
||||
**Architecture:** Three phases: (1) bulk delete stale docs, (2) create/update READMEs with standard template, (3) add lib.rs doc comments. No Rust code changes except `//!` comments. Worktree cleanup is a separate non-git operation.
|
||||
|
||||
**Tech Stack:** git rm, file creation, `SQLX_OFFLINE=true cargo check --workspace`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Delete all scattered .md files
|
||||
|
||||
Delete every .md file that is NOT a root-level README.md for a crate/service/testing directory. Also delete `.serena` tool artifacts. Keep `docs/plans/`.
|
||||
|
||||
**Step 1: Delete scattered .md in crates/**
|
||||
|
||||
```bash
|
||||
cd /home/jgrusewski/Work/foxhunt
|
||||
git rm \
|
||||
crates/data/tests/EDGE_CASE_FINDINGS.md \
|
||||
crates/database/QUICK_START.md \
|
||||
crates/ml/PERFORMANCE_TRACKING.md \
|
||||
crates/ml/MAMBA2_CONFIGURATION_FIX.md \
|
||||
crates/ml/PERFORMANCE_QUICK_START.md \
|
||||
crates/ml/MAMBA2_DIMENSION_ANALYSIS.md \
|
||||
crates/ml/QUICKSTART_GPU_BENCHMARK.md \
|
||||
crates/ml/docs/codebase-cleanup/DQN_PARAMETER_CONSISTENCY_AUDIT.md \
|
||||
crates/ml/docs/dqn_hyperopt_parameter_audit.md \
|
||||
crates/ml/docs/GPU_BENCHMARK_GUIDE.md \
|
||||
crates/ml/docs/dqn_hyperparameters_analysis.md \
|
||||
crates/ml/docs/QAT_GRADIENT_CHECKPOINTING_WORKAROUND.md \
|
||||
crates/ml/docs/QAT_GUIDE.md \
|
||||
crates/ml/hyperparams/README.md \
|
||||
crates/ml/profiling_reports/TFT_INT8_MEMORY_PROFILING_GUIDE.md \
|
||||
crates/ml/profiling_reports/MEMORY_COMPARISON_CHART.md \
|
||||
crates/ml/src/benchmark/TFT_BENCHMARK_README.md \
|
||||
crates/ml/src/checkpoint/README.md \
|
||||
crates/ml/tests/TFT_TEST_REPORT.md \
|
||||
crates/ml/tests/MAMBA_TEST_COVERAGE.md \
|
||||
crates/ml/trained_models/production/tft_real_data/TRAINING_REPORT.md \
|
||||
crates/risk/docs/TEST_COVERAGE_REPORT.md \
|
||||
crates/storage/tests/S3_TEST_COVERAGE.md \
|
||||
crates/trading_engine/docs/audit_trail_persistence_usage.md \
|
||||
crates/trading_engine/docs/TEST_COVERAGE_REPORT.md
|
||||
```
|
||||
|
||||
**Step 2: Delete .serena tool artifacts**
|
||||
|
||||
```bash
|
||||
git rm -r crates/trading_engine/src/types/.serena/
|
||||
```
|
||||
|
||||
**Step 3: Delete scattered .md in services/**
|
||||
|
||||
```bash
|
||||
git rm \
|
||||
services/api_gateway/tests/INTEGRATION_TEST_REPORT.md \
|
||||
services/api_gateway/tests/RATE_LIMITER_TEST_REPORT.md \
|
||||
services/api_gateway/src/metrics/README.md \
|
||||
services/api_gateway/tests/README.md \
|
||||
services/api_gateway/benches/README.md \
|
||||
services/api_gateway/METRICS_DEPLOYMENT.md \
|
||||
services/api_gateway/METRICS_ARCHITECTURE.md \
|
||||
services/api_gateway/REVOCATION_CACHE_USAGE.md \
|
||||
services/api_gateway/BENCHMARKS.md \
|
||||
services/api_gateway/ML_TRAINING_PROXY_INTEGRATION.md \
|
||||
services/api_gateway/RATE_LIMITER_IMPLEMENTATION.md \
|
||||
services/backtesting_service/tests/SERVICE_TESTS_REPORT.md \
|
||||
services/backtesting_service/tests/COVERAGE_MAPPING.md \
|
||||
services/backtesting_service/tests/fixtures/README.md \
|
||||
services/backtesting_service/tests/fixtures/PERFORMANCE.md \
|
||||
services/backtesting_service/tests/fixtures/QUICKSTART.md \
|
||||
services/backtesting_service/tests/fixtures/ARCHITECTURE.md \
|
||||
services/backtesting_service/docs/DBN_LOADING_PERFORMANCE_REPORT.md \
|
||||
services/backtesting_service/DBN_REPOSITORY_USAGE.md \
|
||||
services/backtesting_service/ADVANCED_QUERIES_QUICKREF.md \
|
||||
services/broker_gateway_service/DEPLOYMENT.md \
|
||||
services/broker_gateway_service/MONITORING_IMPLEMENTATION.md \
|
||||
services/broker_gateway_service/METRICS_QUICK_REF.md \
|
||||
services/broker_gateway_service/docs/API.md \
|
||||
services/broker_gateway_service/docs/TROUBLESHOOTING.md \
|
||||
services/broker_gateway_service/docs/DEPLOYMENT.md \
|
||||
services/ml_training_service/HYPERPARAMETER_TUNING.md \
|
||||
services/ml_training_service/TRIAL_EXECUTOR_USAGE.md \
|
||||
services/ml_training_service/tests/fixtures/README.md \
|
||||
services/trading_service/src/streaming/README.md \
|
||||
services/trading_service/docs/ml_integration_design.md \
|
||||
services/trading_service/tests/common/README.md \
|
||||
services/trading_agent_service/README_TLS.md
|
||||
```
|
||||
|
||||
**Step 4: Delete scattered .md in bin/ and testing/**
|
||||
|
||||
```bash
|
||||
git rm \
|
||||
bin/fxt/CONFIG_FILE_SUPPORT.md \
|
||||
bin/fxt/TUNE_COMMAND_README.md \
|
||||
bin/fxt/docs/AUTHENTICATION.md \
|
||||
bin/fxt/docs/ML_TRADING_COMMANDS.md \
|
||||
bin/fxt/docs/USAGE.md \
|
||||
bin/fxt/tests/INTEGRATION_TEST_GUIDE.md \
|
||||
bin/fxt/tests/TEST_EXECUTION_README.md \
|
||||
testing/api-gateway-load/CHEAT_SHEET.md \
|
||||
testing/api-gateway-load/QUICK_START.md \
|
||||
testing/e2e/E2E_TEST_GUIDE.md \
|
||||
testing/integration/e2e_helpers/INFRASTRUCTURE_VALIDATION.md \
|
||||
testing/integration/e2e_helpers/USAGE_EXAMPLES.md \
|
||||
testing/integration/e2e_helpers/QUICKSTART.md \
|
||||
testing/integration/e2e_helpers/VALIDATION_REPORT.md \
|
||||
testing/integration/e2e_helpers/QUICK_START_INFRASTRUCTURE.md \
|
||||
testing/integration/fixtures/README.md \
|
||||
testing/service-integration/README_DBN_INTEGRATION.md
|
||||
```
|
||||
|
||||
**Step 5: Keep docs/plans/**
|
||||
|
||||
The 137 plan docs in `docs/plans/` are kept as historical reference. No action needed.
|
||||
|
||||
**Step 6: Remove empty directories left behind**
|
||||
|
||||
```bash
|
||||
# git auto-removes empty dirs on `git rm`, but check for any remaining
|
||||
find crates/ services/ bin/ testing/ -type d -empty -delete 2>/dev/null
|
||||
# Also remove docs/plans/ dir itself if empty
|
||||
rmdir docs/plans 2>/dev/null || true
|
||||
```
|
||||
|
||||
**Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "cleanup: delete 92 scattered .md files and .serena artifacts"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Remove stale git worktrees
|
||||
|
||||
Not a git-tracked change, but frees ~13GB disk.
|
||||
|
||||
**Step 1: List worktrees**
|
||||
|
||||
```bash
|
||||
git worktree list
|
||||
```
|
||||
|
||||
**Step 2: Remove all worktrees in .claude/worktrees/**
|
||||
|
||||
```bash
|
||||
git worktree remove --force .claude/worktrees/real-metrics-overhaul
|
||||
git worktree remove --force .claude/worktrees/autonomous-agents
|
||||
git worktree remove --force .claude/worktrees/numeric-type-unification
|
||||
git worktree remove --force .claude/worktrees/training-deploy
|
||||
```
|
||||
|
||||
**Step 3: Prune stale worktree references**
|
||||
|
||||
```bash
|
||||
git worktree prune
|
||||
```
|
||||
|
||||
**Step 4: Verify**
|
||||
|
||||
```bash
|
||||
git worktree list
|
||||
# Should show only main worktree
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Create/update crate READMEs (17 crates)
|
||||
|
||||
For each crate, create or update `README.md` using this template:
|
||||
|
||||
```markdown
|
||||
# <crate-name>
|
||||
|
||||
<Cargo.toml description>
|
||||
|
||||
## Key Types
|
||||
|
||||
- `TypeA` — brief description
|
||||
- `TypeB` — brief description
|
||||
|
||||
## Usage
|
||||
|
||||
Brief usage notes or example.
|
||||
```
|
||||
|
||||
**Step 1: Create 8 missing crate READMEs**
|
||||
|
||||
Create `README.md` in each of these directories with content based on their Cargo.toml description and key types:
|
||||
|
||||
**crates/config/README.md:**
|
||||
```markdown
|
||||
# config
|
||||
|
||||
Configuration management for Foxhunt HFT trading system.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `FoxhuntConfig` — top-level configuration struct
|
||||
- `MlConfig` — ML model and training configuration
|
||||
- `RiskConfig` — risk management parameters
|
||||
- `DatabaseConfig` — PostgreSQL connection settings
|
||||
- `StorageConfig` — S3/object storage configuration
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use config::FoxhuntConfig;
|
||||
let config = FoxhuntConfig::load("config/foxhunt.toml")?;
|
||||
```
|
||||
```
|
||||
|
||||
**crates/ctrader-openapi/README.md:**
|
||||
```markdown
|
||||
# ctrader-openapi
|
||||
|
||||
cTrader Open API client — Protobuf over TCP+TLS for order routing, account queries, and market data.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `CtraderConnection` — TLS-wrapped TCP connection to cTrader
|
||||
- `CtraderCodec` — Protobuf message framing
|
||||
- `RateLimiter` — request rate limiting
|
||||
- `AuthManager` — OAuth2 authentication flow
|
||||
|
||||
## Usage
|
||||
|
||||
Broker gateway integration for order routing via cTrader Open API protocol.
|
||||
```
|
||||
|
||||
**crates/market-data/README.md:**
|
||||
```markdown
|
||||
# market-data
|
||||
|
||||
Market data repository for the Foxhunt HFT trading system.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `MarketDataRepository` — PostgreSQL-backed market data storage
|
||||
- Price, orderbook, indicator, and model data access methods
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use market_data::MarketDataRepository;
|
||||
let repo = MarketDataRepository::new(pool).await?;
|
||||
```
|
||||
```
|
||||
|
||||
**crates/ml-data/README.md:**
|
||||
```markdown
|
||||
# ml-data
|
||||
|
||||
Production-ready ML data management for HFT trading systems.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `MlDataConfig` — ML data repository configuration
|
||||
- `FeaturesRepository` — feature storage and retrieval
|
||||
- `ModelsRepository` — model metadata and versioning
|
||||
- `PerformanceRepository` — model performance tracking
|
||||
- `TrainingRepository` — training run management
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use ml_data::{MlDataConfig, FeaturesRepository};
|
||||
```
|
||||
```
|
||||
|
||||
**crates/model_loader/README.md:**
|
||||
```markdown
|
||||
# model_loader
|
||||
|
||||
ML model loading and caching infrastructure.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `AsyncModelLoader` — async model loading with LRU cache
|
||||
- `LoadedModelInfo` — loaded model metadata and weights
|
||||
- `ModelType` — re-exported from common (10 model variants)
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use model_loader::AsyncModelLoader;
|
||||
let loader = AsyncModelLoader::new(cache_size);
|
||||
let model = loader.load("path/to/checkpoint").await?;
|
||||
```
|
||||
```
|
||||
|
||||
**crates/risk-data/README.md:**
|
||||
```markdown
|
||||
# risk-data
|
||||
|
||||
Risk data repository for high-frequency trading risk management.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `RiskDataConfig` — repository configuration
|
||||
- `VarRepository` — Value-at-Risk data storage
|
||||
- `LimitsRepository` — position and risk limit management
|
||||
- `ComplianceRepository` — compliance record storage
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use risk_data::{RiskDataConfig, VarRepository};
|
||||
```
|
||||
```
|
||||
|
||||
**crates/trading-data/README.md:**
|
||||
```markdown
|
||||
# trading-data
|
||||
|
||||
Trading data repository layer for the high-frequency trading system.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `PostgresOrderRepository` — PostgreSQL-backed order storage
|
||||
- `OrderRepository` trait — order CRUD operations
|
||||
- Position and execution tracking
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use trading_data::PostgresOrderRepository;
|
||||
let repo = PostgresOrderRepository::new(pool).await?;
|
||||
```
|
||||
```
|
||||
|
||||
**crates/training_uploader/README.md:**
|
||||
```markdown
|
||||
# training_uploader
|
||||
|
||||
K8s sidecar that uploads training artifacts to S3 and reports completion via gRPC.
|
||||
|
||||
## Usage
|
||||
|
||||
Deployed as a sidecar container alongside training jobs. Watches for training
|
||||
artifacts (checkpoints, metrics) and uploads them to the configured S3 bucket.
|
||||
|
||||
## Configuration
|
||||
|
||||
- `S3_BUCKET` — target bucket for artifacts
|
||||
- `TRAINING_SERVICE_URL` — gRPC endpoint for completion reporting
|
||||
```
|
||||
|
||||
**Step 2: Update 9 existing crate READMEs**
|
||||
|
||||
Read each existing README and update to the standard template format. Strip stale content, keep accurate descriptions. Existing READMEs to update:
|
||||
- `crates/backtesting/README.md`
|
||||
- `crates/common/README.md`
|
||||
- `crates/data/README.md`
|
||||
- `crates/database/README.md`
|
||||
- `crates/ml/README.md`
|
||||
- `crates/risk/README.md`
|
||||
- `crates/storage/README.md`
|
||||
- `crates/trading_engine/README.md`
|
||||
- `crates/web-gateway/README.md`
|
||||
|
||||
For each: read current content, trim to standard template (title, purpose, key types, usage), remove bloated sections.
|
||||
|
||||
**Step 3: Verify no compilation impact**
|
||||
|
||||
Run: `SQLX_OFFLINE=true cargo check --workspace`
|
||||
Expected: Clean (README changes can't break compilation)
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/*/README.md
|
||||
git commit -m "docs: create/update README.md for all 17 crates"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Create/update service, binary, and testing READMEs
|
||||
|
||||
**Step 1: Create 3 missing service READMEs**
|
||||
|
||||
**services/api_gateway/README.md:**
|
||||
```markdown
|
||||
# api_gateway
|
||||
|
||||
API Gateway service with 6-layer authentication, RBAC, rate limiting, and gRPC proxy routing.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `ApiGatewayService` — main service implementation
|
||||
- `RoleBasedAccessControl` — RBAC authorization
|
||||
- `RateLimiter` — 3-tier rate limiting (auth/trading/compute)
|
||||
- `CircuitBreaker` — upstream service protection
|
||||
|
||||
## Configuration
|
||||
|
||||
- `JWT_SECRET` — JWT signing secret (min 32 chars)
|
||||
- `UPSTREAM_TRADING_URL` — trading service gRPC endpoint
|
||||
- `UPSTREAM_ML_URL` — ML service gRPC endpoint
|
||||
|
||||
## Features
|
||||
|
||||
- `mfa` (default) — multi-factor authentication support
|
||||
- Build without: `cargo check -p api_gateway --no-default-features --features minimal`
|
||||
```
|
||||
|
||||
**services/data_acquisition_service/README.md:**
|
||||
```markdown
|
||||
# data_acquisition_service
|
||||
|
||||
Automated Databento data downloading, validation, and S3 upload for HFT trading.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `DataAcquisitionServiceImpl` — main gRPC service
|
||||
- `Downloader` — Databento API client
|
||||
- `Uploader` — S3 artifact upload
|
||||
- `Validator` — data quality checks
|
||||
|
||||
## Configuration
|
||||
|
||||
- `DATABENTO_API_KEY` — Databento API credentials
|
||||
- `S3_BUCKET` — target storage bucket
|
||||
```
|
||||
|
||||
**services/trading_agent_service/README.md:**
|
||||
```markdown
|
||||
# trading_agent_service
|
||||
|
||||
Portfolio management with autonomous universe selection, asset allocation, and order generation.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `TradingAgentServiceImpl` — main gRPC service
|
||||
- `AutonomousUniverseManager` — dynamic universe scaling
|
||||
- `AssetSelector` — instrument selection
|
||||
- `Allocator` — position sizing and allocation
|
||||
- `OrderGenerator` — order creation from signals
|
||||
|
||||
## Configuration
|
||||
|
||||
- `ML_SERVICE_URL` — ensemble ML service endpoint
|
||||
- `TRADING_SERVICE_URL` — order execution endpoint
|
||||
```
|
||||
|
||||
**Step 2: Update 4 existing service READMEs**
|
||||
|
||||
Read and update to standard template:
|
||||
- `services/backtesting_service/README.md`
|
||||
- `services/broker_gateway_service/README.md`
|
||||
- `services/ml_training_service/README.md`
|
||||
- `services/trading_service/README.md`
|
||||
|
||||
For `services/tests/` — this is a disabled test service, skip README creation.
|
||||
|
||||
**Step 3: Create bin/fxt/README.md**
|
||||
|
||||
```markdown
|
||||
# fxt
|
||||
|
||||
Foxhunt CLI — command-line interface for the Foxhunt HFT trading system.
|
||||
|
||||
## Commands
|
||||
|
||||
- `fxt auth` — authentication and token management
|
||||
- `fxt trade` — order submission and management
|
||||
- `fxt train` — model training job management
|
||||
- `fxt tune` — hyperparameter optimization
|
||||
- `fxt agent` — trading agent control
|
||||
- `fxt model` — model listing and inspection
|
||||
- `fxt backtest-ml` — ML backtesting workflows
|
||||
|
||||
## Configuration
|
||||
|
||||
- `FXT_CONFIG` — config file path (default: `~/.config/fxt/config.toml`)
|
||||
- `TRADING_SERVICE_URL` — gRPC endpoint for trading service
|
||||
```
|
||||
|
||||
**Step 4: Update 10 testing crate READMEs**
|
||||
|
||||
Read existing testing READMEs and update to standard template. For those with existing README.md, trim to standard format. Testing crates:
|
||||
- `testing/api-gateway-load/README.md` (exists)
|
||||
- `testing/e2e/README.md` (exists)
|
||||
- `testing/harness/` (no README — skip, not a published crate)
|
||||
- `testing/integration/README.md` (exists)
|
||||
- `testing/load/README.md` (exists)
|
||||
- `testing/service-integration/` (no README — create minimal)
|
||||
- `testing/service-load/README.md` (exists)
|
||||
- `testing/stress/` (no README — skip)
|
||||
- `testing/test-common/README.md` (exists)
|
||||
- `testing/vault-integration/` (no README — skip)
|
||||
|
||||
Also delete subdirectory READMEs that survived (chaos, smoke_tests, e2e_helpers — only keep root README per crate):
|
||||
- `testing/integration/chaos/README.md`
|
||||
- `testing/integration/e2e_helpers/README.md`
|
||||
- `testing/integration/smoke_tests/README.md`
|
||||
- `testing/e2e/integration/README.md`
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add services/*/README.md bin/fxt/README.md testing/*/README.md
|
||||
git rm testing/integration/chaos/README.md testing/integration/e2e_helpers/README.md testing/integration/smoke_tests/README.md testing/e2e/integration/README.md 2>/dev/null || true
|
||||
git commit -m "docs: create/update README.md for all services, CLI, and testing crates"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Add lib.rs doc comments to 7 crates
|
||||
|
||||
Add `//!` module-level doc comments to the top of `src/lib.rs` for crates that are missing them.
|
||||
|
||||
**Step 1: Add doc comments**
|
||||
|
||||
For each crate, add a single `//!` line after any `#![...]` attributes but before the first `use`/`mod` statement.
|
||||
|
||||
**crates/backtesting/src/lib.rs** — add after `#![allow(missing_docs)]`:
|
||||
```rust
|
||||
//! Backtesting engine for strategy evaluation with walk-forward analysis.
|
||||
```
|
||||
|
||||
**crates/ctrader-openapi/src/lib.rs** — add after `#![deny(...)]`:
|
||||
```rust
|
||||
//! cTrader Open API client — Protobuf over TCP+TLS for order routing and market data.
|
||||
```
|
||||
|
||||
**crates/data/src/lib.rs** — add after `#![allow(unused_extern_crates)]`:
|
||||
```rust
|
||||
//! Data processing and validation for the Foxhunt HFT trading system.
|
||||
```
|
||||
|
||||
**crates/ml/src/lib.rs** — add after `#![deny(...)]`:
|
||||
```rust
|
||||
//! Machine learning models and ensemble inference for HFT signal generation.
|
||||
```
|
||||
|
||||
**crates/risk/src/lib.rs** — add after the comment block:
|
||||
```rust
|
||||
//! Risk management — VaR, Kelly criterion, circuit breakers, and kill switches.
|
||||
```
|
||||
|
||||
**crates/trading_engine/src/lib.rs** — add after `#![deny(...)]`:
|
||||
```rust
|
||||
//! Trading engine — FIX 4.4, IB TWS, order management, and execution.
|
||||
```
|
||||
|
||||
**crates/training_uploader/src/main.rs** — add after `#![allow(...)]`:
|
||||
```rust
|
||||
//! K8s sidecar that uploads training artifacts to S3 and reports completion via gRPC.
|
||||
```
|
||||
|
||||
**Step 2: Verify compilation**
|
||||
|
||||
Run: `SQLX_OFFLINE=true cargo check --workspace`
|
||||
Expected: Clean (doc comments can't break compilation)
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/backtesting/src/lib.rs crates/ctrader-openapi/src/lib.rs crates/data/src/lib.rs crates/ml/src/lib.rs crates/risk/src/lib.rs crates/trading_engine/src/lib.rs crates/training_uploader/src/main.rs
|
||||
git commit -m "docs: add lib.rs doc comments to 7 crates"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Final verification
|
||||
|
||||
**Step 1: Workspace build**
|
||||
|
||||
Run: `SQLX_OFFLINE=true cargo check --workspace`
|
||||
Expected: zero errors
|
||||
|
||||
**Step 2: Clippy on workspace**
|
||||
|
||||
Run: `SQLX_OFFLINE=true cargo clippy --workspace --lib -- -D warnings`
|
||||
Expected: zero warnings
|
||||
|
||||
**Step 3: Verify no stray .md files remain**
|
||||
|
||||
```bash
|
||||
# List all non-README .md files in code directories (should be empty or near-empty)
|
||||
find crates/ services/ bin/ testing/ -name "*.md" ! -name "README.md" | sort
|
||||
```
|
||||
|
||||
Expected: empty output (all scattered docs deleted)
|
||||
|
||||
**Step 4: Verify every crate/service has README**
|
||||
|
||||
```bash
|
||||
for dir in crates/*/; do
|
||||
[ -f "$dir/README.md" ] && echo "OK $dir" || echo "MISSING $dir"
|
||||
done
|
||||
for dir in services/*/; do
|
||||
[ -f "$dir/README.md" ] && echo "OK $dir" || echo "MISSING $dir"
|
||||
done
|
||||
[ -f bin/fxt/README.md ] && echo "OK bin/fxt" || echo "MISSING bin/fxt"
|
||||
```
|
||||
|
||||
Expected: all OK (services/tests/ may be MISSING — that's acceptable, it's a disabled service)
|
||||
|
||||
**Step 5: Line count verification**
|
||||
|
||||
```bash
|
||||
git diff --stat HEAD~4
|
||||
# Should show significant net negative lines from doc deletion
|
||||
```
|
||||
Reference in New Issue
Block a user