Phase 1: Create services/api/ from api_gateway, add tonic-web, update refs Phase 2: Update K8s, CI, Docker, Prometheus, scripts, FXT CLI Phase 3: Migrate dashboard from REST+WS to grpc-web (@connectrpc) Phase 4: Final validation and cleanup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1145 lines
30 KiB
Markdown
1145 lines
30 KiB
Markdown
# Gateway Unification Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Unify `services/api_gateway/` and `crates/web-gateway/` into a single `services/api/` gRPC service with tonic-web for browser access, then migrate the React dashboard from REST+WebSocket to grpc-web.
|
|
|
|
**Architecture:** Move all api_gateway code into `services/api/`, add `tonic-web` + CORS for grpc-web browser support, delete both old gateways and the web-gateway entirely, update ~30 reference files across workspace. Dashboard migrated from REST+WS to `@connectrpc/connect-web` with server-streaming replacing WebSocket topics.
|
|
|
|
**Tech Stack:** Rust (tonic 0.14, tonic-web), React 19, TypeScript, @connectrpc/connect-web, @bufbuild/protobuf
|
|
|
|
---
|
|
|
|
## Phase 1: Create services/api/ (Backend)
|
|
|
|
### Task 1: Scaffold services/api/ directory from api_gateway
|
|
|
|
**Files:**
|
|
- Create: `services/api/Cargo.toml`
|
|
- Create: `services/api/build.rs`
|
|
- Create: `services/api/src/` (copy all from `services/api_gateway/src/`)
|
|
- Create: `services/api/tests/` (copy all from `services/api_gateway/tests/`)
|
|
- Create: `services/api/benches/` (copy all from `services/api_gateway/benches/`)
|
|
|
|
**Step 1: Copy api_gateway into services/api/**
|
|
|
|
```bash
|
|
cp -r services/api_gateway services/api
|
|
```
|
|
|
|
**Step 2: Update Cargo.toml for the new crate**
|
|
|
|
In `services/api/Cargo.toml`, change:
|
|
```toml
|
|
# OLD
|
|
[package]
|
|
name = "api-gateway"
|
|
description = "API Gateway Service with 6-layer authentication and request routing"
|
|
|
|
[[bin]]
|
|
name = "api-gateway"
|
|
path = "src/main.rs"
|
|
```
|
|
to:
|
|
```toml
|
|
# NEW
|
|
[package]
|
|
name = "api"
|
|
description = "Foxhunt API Service — unified gRPC gateway with tonic-web for browser access"
|
|
|
|
[[bin]]
|
|
name = "api"
|
|
path = "src/main.rs"
|
|
```
|
|
|
|
**Step 3: Add tonic-web and tower-http CORS dependencies**
|
|
|
|
Add to `services/api/Cargo.toml` `[dependencies]`:
|
|
```toml
|
|
tonic-web = "0.13"
|
|
tower-http = { workspace = true, features = ["cors"] }
|
|
```
|
|
|
|
Remove the `axum` dependency (was only for ML REST handlers on port 8080).
|
|
|
|
**Step 4: Update build.rs proto paths**
|
|
|
|
In `services/api/build.rs`, proto paths change from `../../proto/` to `../../proto/` — this stays the same since `services/api/` is at the same depth as `services/api_gateway/`. No change needed.
|
|
|
|
**Step 5: Verify proto paths are correct**
|
|
|
|
```bash
|
|
ls services/api/../../proto/fxt_trading.proto # Should resolve to proto/fxt_trading.proto
|
|
```
|
|
|
|
**Step 6: Commit**
|
|
|
|
```bash
|
|
git add services/api/
|
|
git commit -m "feat: scaffold services/api/ from api_gateway (copy)"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Remove REST handlers and add tonic-web
|
|
|
|
**Files:**
|
|
- Delete: `services/api/src/handlers/` (ML REST endpoints + auth middleware for REST)
|
|
- Modify: `services/api/src/lib.rs` — remove handler re-exports, remove `handlers` module
|
|
- Modify: `services/api/src/main.rs` — remove Axum REST server, add tonic-web + CORS layer
|
|
- Modify: `services/api/Cargo.toml` — remove `axum` dep
|
|
|
|
**Step 1: Delete the handlers directory**
|
|
|
|
```bash
|
|
rm -rf services/api/src/handlers
|
|
```
|
|
|
|
**Step 2: Update lib.rs — remove handler module and re-exports**
|
|
|
|
Remove these lines from `services/api/src/lib.rs`:
|
|
```rust
|
|
pub mod handlers;
|
|
```
|
|
and:
|
|
```rust
|
|
// Re-export REST API handlers
|
|
pub use handlers::{
|
|
jwt_auth_middleware, ml_router, AuthMiddlewareState, BatchPredictRequest, BatchPredictResponse,
|
|
HotSwapRequest, HotSwapResponse, MlHandlerState, ModelStatusResponse, PredictRequest,
|
|
PredictResponse,
|
|
};
|
|
```
|
|
|
|
**Step 3: Update main.rs — replace Axum REST server with tonic-web**
|
|
|
|
Find the section that starts the Axum REST server on port 8080 and remove it entirely. Then wrap the gRPC server with tonic-web:
|
|
|
|
Add imports:
|
|
```rust
|
|
use tonic_web::GrpcWebLayer;
|
|
use tower_http::cors::{AllowHeaders, AllowMethods, CorsLayer};
|
|
use http::Method;
|
|
```
|
|
|
|
Modify the `Server::builder()` call to add tonic-web and CORS:
|
|
```rust
|
|
let cors_origins = std::env::var("CORS_ORIGINS")
|
|
.unwrap_or_else(|_| "http://localhost:5173".to_string());
|
|
let origins: Vec<_> = cors_origins
|
|
.split(',')
|
|
.filter_map(|s| s.trim().parse().ok())
|
|
.collect();
|
|
|
|
let cors = CorsLayer::new()
|
|
.allow_origin(origins)
|
|
.allow_headers(AllowHeaders::any())
|
|
.allow_methods([Method::POST, Method::OPTIONS])
|
|
.expose_headers([
|
|
"grpc-status".parse().unwrap_or_else(|_| http::HeaderName::from_static("grpc-status")),
|
|
"grpc-message".parse().unwrap_or_else(|_| http::HeaderName::from_static("grpc-message")),
|
|
]);
|
|
|
|
Server::builder()
|
|
.accept_http1(true) // Required for grpc-web (HTTP/1.1)
|
|
.layer(cors)
|
|
.layer(GrpcWebLayer::new())
|
|
// ... existing .add_service() calls unchanged ...
|
|
.serve(addr)
|
|
.await?;
|
|
```
|
|
|
|
Also remove the `axum` import and REST server startup code.
|
|
|
|
**Step 4: Update main.rs observability service name**
|
|
|
|
Change `init_observability("api_gateway", ...)` to `init_observability("api", ...)`.
|
|
|
|
**Step 5: Update main.rs clap command name**
|
|
|
|
Change `#[command(name = "api_gateway", ...)]` to `#[command(name = "api", ...)]`.
|
|
|
|
**Step 6: Remove axum dep from Cargo.toml**
|
|
|
|
Remove:
|
|
```toml
|
|
axum = "0.7"
|
|
```
|
|
|
|
Add (if not already present):
|
|
```toml
|
|
http = "1.1" # Already present
|
|
```
|
|
|
|
**Step 7: Verify it compiles**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p api
|
|
```
|
|
|
|
**Step 8: Commit**
|
|
|
|
```bash
|
|
git add services/api/
|
|
git commit -m "feat(api): remove REST handlers, add tonic-web + CORS for grpc-web browser access"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Update lib.rs internal naming
|
|
|
|
**Files:**
|
|
- Modify: `services/api/src/lib.rs`
|
|
|
|
**Step 1: Update crate-level doc comment**
|
|
|
|
Change:
|
|
```rust
|
|
//! API Gateway for Foxhunt HFT Trading System
|
|
```
|
|
to:
|
|
```rust
|
|
//! Foxhunt API Service — unified gRPC gateway with tonic-web for browser access
|
|
```
|
|
|
|
**Step 2: Verify all imports still work**
|
|
|
|
The lib crate name changes from `api_gateway` to `api`. Search `services/api/src/` for any `use api_gateway::` and replace with `use api::`:
|
|
|
|
```bash
|
|
grep -r "api_gateway" services/api/src/ --include="*.rs"
|
|
```
|
|
|
|
Replace all occurrences of `use api_gateway::` with `use api::` in the src/ directory.
|
|
|
|
**Step 3: Update tests similarly**
|
|
|
|
```bash
|
|
grep -r "api_gateway" services/api/tests/ --include="*.rs"
|
|
```
|
|
|
|
Replace `use api_gateway::` with `use api::` in all test files.
|
|
|
|
**Step 4: Verify compilation**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p api
|
|
```
|
|
|
|
**Step 5: Run tests**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo test -p api --lib
|
|
```
|
|
|
|
**Step 6: Commit**
|
|
|
|
```bash
|
|
git add services/api/
|
|
git commit -m "refactor(api): rename crate references api_gateway → api"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Update workspace Cargo.toml
|
|
|
|
**Files:**
|
|
- Modify: `Cargo.toml` (workspace root)
|
|
|
|
**Step 1: Replace old members with new**
|
|
|
|
Find the `members` array and:
|
|
- Remove `"crates/web-gateway"`
|
|
- Remove `"services/api_gateway"`
|
|
- Add `"services/api"`
|
|
- Change `"testing/api-gateway-load"` to `"testing/api-load"`
|
|
|
|
**Step 2: Update workspace dependency if exists**
|
|
|
|
Search for `api-gateway` in the `[workspace.dependencies]` section. If present, rename to `api`.
|
|
|
|
**Step 3: Verify workspace resolves**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p api
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add Cargo.toml
|
|
git commit -m "chore: update workspace members — api_gateway+web-gateway → api"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Update trading_service dependency
|
|
|
|
**Files:**
|
|
- Modify: `services/trading_service/Cargo.toml:106`
|
|
|
|
**Step 1: Update path dependency**
|
|
|
|
Change:
|
|
```toml
|
|
api-gateway = { path = "../api_gateway" } # For auth tests - no cyclic dependency
|
|
```
|
|
to:
|
|
```toml
|
|
api = { path = "../api" } # For auth tests - no cyclic dependency
|
|
```
|
|
|
|
**Step 2: Update any import references in trading_service**
|
|
|
|
```bash
|
|
grep -r "api_gateway\|api-gateway" services/trading_service/ --include="*.rs"
|
|
```
|
|
|
|
Replace `use api_gateway::` with `use api::` in any files found.
|
|
|
|
**Step 3: Verify compilation**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p trading-service
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add services/trading_service/
|
|
git commit -m "refactor(trading-service): update dep api-gateway → api"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Rename testing/api-gateway-load → testing/api-load
|
|
|
|
**Files:**
|
|
- Rename: `testing/api-gateway-load/` → `testing/api-load/`
|
|
- Modify: `testing/api-load/Cargo.toml`
|
|
|
|
**Step 1: Move directory**
|
|
|
|
```bash
|
|
mv testing/api-gateway-load testing/api-load
|
|
```
|
|
|
|
**Step 2: Update Cargo.toml**
|
|
|
|
In `testing/api-load/Cargo.toml`, change:
|
|
```toml
|
|
name = "api_gateway_load_tests"
|
|
```
|
|
to:
|
|
```toml
|
|
name = "api_load_tests"
|
|
```
|
|
|
|
**Step 3: Verify compilation**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p api_load_tests
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add testing/api-load/ testing/api-gateway-load/
|
|
git commit -m "refactor: rename testing/api-gateway-load → testing/api-load"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: Update test crates referencing api_gateway
|
|
|
|
**Files:**
|
|
- Modify: `testing/service-integration/tests/common/auth_helpers.rs:235` — `get_api_gateway_addr()` → `get_api_addr()`
|
|
- Modify: `testing/service-integration/tests/backtesting_service_e2e.rs` — update imports
|
|
- Modify: `testing/service-integration/tests/service_health_resilience_e2e.rs` — update imports
|
|
- Modify: `testing/integration/fixtures/mod.rs` — update comment
|
|
- Modify: `testing/integration/smoke_tests/service_health.rs` — rename fn
|
|
- Modify: `testing/integration/smoke_tests/mod.rs:71` — rename fn
|
|
- Modify: `testing/integration/standalone/watch_tuning_progress_updated.rs` — rename param
|
|
|
|
**Step 1: Update auth_helpers.rs**
|
|
|
|
Rename `get_api_gateway_addr()` → `get_api_addr()` and update the env var it reads (likely `API_GATEWAY_URL` → keep same env var for backwards compat, or rename to `API_URL`).
|
|
|
|
**Step 2: Update all call sites**
|
|
|
|
Replace `get_api_gateway_addr` with `get_api_addr` across all test files. Replace `api_gateway_url` with `api_url` across smoke tests.
|
|
|
|
**Step 3: Run test crate compilation**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p service-integration-tests
|
|
SQLX_OFFLINE=true cargo check -p integration-tests
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add testing/
|
|
git commit -m "refactor(tests): rename api_gateway references → api"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: Delete old gateways
|
|
|
|
**Files:**
|
|
- Delete: `services/api_gateway/` (entire directory)
|
|
- Delete: `crates/web-gateway/` (entire directory)
|
|
|
|
**Step 1: Verify services/api/ compiles and tests pass**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check --workspace
|
|
SQLX_OFFLINE=true cargo test -p api --lib
|
|
```
|
|
|
|
**Step 2: Delete old directories**
|
|
|
|
```bash
|
|
rm -rf services/api_gateway
|
|
rm -rf crates/web-gateway
|
|
```
|
|
|
|
**Step 3: Verify workspace still builds**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check --workspace
|
|
```
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add -A
|
|
git commit -m "chore: delete services/api_gateway/ and crates/web-gateway/ (replaced by services/api/)"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 2: Update Infrastructure & Config
|
|
|
|
### Task 9: Update K8s manifests
|
|
|
|
**Files:**
|
|
- Modify: `infra/k8s/services/api-gateway.yaml` → rename to `infra/k8s/services/api.yaml`
|
|
- Delete: `infra/k8s/services/web-gateway.yaml`
|
|
- Modify: `infra/k8s/network-policies/api-gateway.yaml` → rename to `infra/k8s/network-policies/api.yaml`
|
|
- Delete: `infra/k8s/network-policies/web-gateway.yaml`
|
|
|
|
**Step 1: Rename api-gateway.yaml → api.yaml**
|
|
|
|
```bash
|
|
mv infra/k8s/services/api-gateway.yaml infra/k8s/services/api.yaml
|
|
mv infra/k8s/network-policies/api-gateway.yaml infra/k8s/network-policies/api.yaml
|
|
```
|
|
|
|
**Step 2: Update api.yaml deployment**
|
|
|
|
In `infra/k8s/services/api.yaml`, search-replace:
|
|
- `app: api-gateway` → `app: api`
|
|
- `name: api-gateway` → `name: api`
|
|
- Binary name in initContainer: `api-gateway` → `api`
|
|
- Service name references
|
|
|
|
Remove the web-gateway environment variable (`WEB_GATEWAY_URL` or similar) if present.
|
|
|
|
Add `CORS_ORIGINS` env var:
|
|
```yaml
|
|
- name: CORS_ORIGINS
|
|
value: "https://dashboard.fxhnt.ai,http://localhost:5173"
|
|
```
|
|
|
|
**Step 3: Update api.yaml network policy**
|
|
|
|
In `infra/k8s/network-policies/api.yaml`:
|
|
- Update `app: api-gateway` → `app: api` in podSelector
|
|
- Remove web-gateway egress rule (port 3000)
|
|
- Keep all other egress rules (backend services, DB, Redis)
|
|
|
|
**Step 4: Delete web-gateway manifests**
|
|
|
|
```bash
|
|
rm infra/k8s/services/web-gateway.yaml
|
|
rm infra/k8s/network-policies/web-gateway.yaml
|
|
```
|
|
|
|
**Step 5: Update any other K8s files referencing api-gateway or web-gateway**
|
|
|
|
```bash
|
|
grep -r "api-gateway\|web-gateway" infra/k8s/ --include="*.yaml"
|
|
```
|
|
|
|
Update references in other service network policies that allow traffic to/from api-gateway or web-gateway.
|
|
|
|
**Step 6: Commit**
|
|
|
|
```bash
|
|
git add infra/k8s/
|
|
git commit -m "infra(k8s): rename api-gateway → api, delete web-gateway manifests"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 10: Update CI pipeline (.gitlab-ci.yml)
|
|
|
|
**Files:**
|
|
- Modify: `.gitlab-ci.yml`
|
|
|
|
**Step 1: Rename compile job**
|
|
|
|
Change `compile-api-gateway` to `compile-api`:
|
|
- `SERVICE_PACKAGE: api-gateway` → `SERVICE_PACKAGE: api`
|
|
- Rules path: `services/api_gateway/**` → `services/api/**`
|
|
- Anchor: `&rules-api-gateway` → `&rules-api`
|
|
|
|
**Step 2: Delete web-gateway compile job**
|
|
|
|
Remove the entire `compile-web-gateway` job block (lines ~575-600).
|
|
|
|
**Step 3: Rename deploy job**
|
|
|
|
Change `deploy-api-gateway` to `deploy-api`:
|
|
- `needs: [{job: compile-api-gateway, ...}]` → `needs: [{job: compile-api, ...}]`
|
|
- `SERVICE_PACKAGE: api-gateway` → `SERVICE_PACKAGE: api`
|
|
- `rules: *rules-api-gateway` → `rules: *rules-api`
|
|
|
|
**Step 4: Delete web-gateway deploy job**
|
|
|
|
Remove the entire `deploy-web-gateway` job block (lines ~1689-1694).
|
|
|
|
**Step 5: Update JWT_SECRET comment**
|
|
|
|
Line 40: `# Stable JWT secret for api_gateway tests` → `# Stable JWT secret for api tests`
|
|
|
|
**Step 6: Commit**
|
|
|
|
```bash
|
|
git add .gitlab-ci.yml
|
|
git commit -m "ci: rename api-gateway → api, remove web-gateway jobs"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 11: Update Docker Compose
|
|
|
|
**Files:**
|
|
- Modify: `docker-compose.yml`
|
|
- Modify: `docker-compose.override.yml`
|
|
|
|
**Step 1: Rename api_gateway service in docker-compose.yml**
|
|
|
|
Change service name `api_gateway:` → `api:` and update:
|
|
- `container_name: foxhunt-api-gateway` → `container_name: foxhunt-api`
|
|
- `dockerfile: services/api_gateway/Dockerfile` → `dockerfile: services/api/Dockerfile`
|
|
- `JWT_ISSUER=foxhunt-api-gateway` → `JWT_ISSUER=foxhunt-api`
|
|
- Add `CORS_ORIGINS=http://localhost:5173`
|
|
|
|
Also update any other services referencing `api_gateway` hostname to `api`.
|
|
|
|
**Step 2: Remove web-gateway service if present**
|
|
|
|
Search for `web-gateway` or `web_gateway` service block and remove it.
|
|
|
|
**Step 3: Update docker-compose.override.yml**
|
|
|
|
Line 4: `api_gateway:` → `api:`
|
|
|
|
**Step 4: Update JWT_ISSUER in other services**
|
|
|
|
Lines 202, 258, 316 reference `JWT_ISSUER=foxhunt-api-gateway` — change to `foxhunt-api`.
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add docker-compose.yml docker-compose.override.yml
|
|
git commit -m "chore(docker): rename api_gateway → api, remove web-gateway"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 12: Update Prometheus config
|
|
|
|
**Files:**
|
|
- Modify: `config/prometheus/prometheus.yml`
|
|
- Modify: `config/prometheus/rules/production-alerts.yml`
|
|
- Modify: `config/prometheus/rules/service-health-alerts.yml`
|
|
|
|
**Step 1: Update scrape target**
|
|
|
|
In `prometheus.yml`:
|
|
```yaml
|
|
# OLD
|
|
- job_name: 'api_gateway'
|
|
static_configs:
|
|
- targets: ['api_gateway:9091']
|
|
# NEW
|
|
- job_name: 'api'
|
|
static_configs:
|
|
- targets: ['api:9091']
|
|
```
|
|
|
|
**Step 2: Update alert rules**
|
|
|
|
In `production-alerts.yml`, replace all `api_gateway` with `api` in:
|
|
- `job="api_gateway"` → `job="api"`
|
|
- `service: api_gateway` → `service: api`
|
|
- String literals in annotations
|
|
|
|
In `service-health-alerts.yml`:
|
|
- `job="api_gateway"` → `job="api"`
|
|
- `service: api_gateway` → `service: api`
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add config/prometheus/
|
|
git commit -m "config(prometheus): rename api_gateway → api in scrape targets and alert rules"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 13: Update scripts
|
|
|
|
**Files:**
|
|
- Modify: `infra/scripts/smoke-test.sh` — rename web-gateway checks to api
|
|
- Modify: `infra/scripts/build-and-push.sh` — rename api-gateway, remove web-gateway
|
|
- Modify: `scripts/generate_dev_certs.sh` — update DNS alt name
|
|
- Modify: `scripts/enforce_coverage.sh` — update path
|
|
|
|
**Step 1: Update smoke-test.sh**
|
|
|
|
Replace web-gateway pod/service checks with api service checks. The section `"Services (gRPC + web-gateway)"` becomes just `"Services (gRPC)"`. Remove web-gateway health check (HTTP /health on port 3000) — api uses gRPC health check.
|
|
|
|
**Step 2: Update build-and-push.sh**
|
|
|
|
Replace `api-gateway` in the services list. Remove web-gateway build section and `--skip-webgw` flag.
|
|
|
|
**Step 3: Update generate_dev_certs.sh**
|
|
|
|
Line 75: `DNS.5 = api-gateway.foxhunt.local` → `DNS.5 = api.foxhunt.local`
|
|
|
|
**Step 4: Update enforce_coverage.sh**
|
|
|
|
Line 32: `"services/api_gateway"` → `"services/api"`
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add infra/scripts/ scripts/
|
|
git commit -m "chore(scripts): rename api-gateway → api, remove web-gateway references"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 14: Update FXT CLI references
|
|
|
|
**Files:**
|
|
- Modify: `bin/fxt/src/config.rs` — rename `api_gateway_url` field to `api_url`
|
|
- Modify: `bin/fxt/src/commands/config.rs` — update field reference
|
|
- Modify: `bin/fxt/src/auth/login.rs` — rename `is_api_gateway_available()`
|
|
- Modify: `bin/fxt/src/auth/jwt_generator.rs` — update any references
|
|
- Modify: `bin/fxt/src/tui/data_fetcher.rs` — update test data `"api-gateway"` → `"api"`
|
|
- Modify: `bin/fxt/src/tui/event_loop.rs` — update service name
|
|
- Modify: `bin/fxt/src/tui/cockpits/pods.rs` — update doc comment
|
|
|
|
**Step 1: Rename config field**
|
|
|
|
In `bin/fxt/src/config.rs`:
|
|
- `pub api_gateway_url: String` → `pub api_url: String`
|
|
- `fn default_api_gateway_url()` → `fn default_api_url()`
|
|
- `#[serde(default = "default_api_gateway_url")]` → `#[serde(default = "default_api_url")]`
|
|
- Update all tests referencing this field
|
|
- Keep the serde alias for backwards compat: `#[serde(alias = "api_gateway_url")]`
|
|
|
|
**Step 2: Update CLI and auth references**
|
|
|
|
In `bin/fxt/src/auth/login.rs`:
|
|
- `is_api_gateway_available()` → `is_api_available()`
|
|
|
|
In `bin/fxt/src/commands/config.rs`:
|
|
- `config.api_gateway_url` → `config.api_url`
|
|
|
|
**Step 3: Update TUI test data**
|
|
|
|
In `bin/fxt/src/tui/data_fetcher.rs`:
|
|
- `"api-gateway"` → `"api"` in test helper and assertions
|
|
|
|
In `bin/fxt/src/tui/event_loop.rs`:
|
|
- `"api-gateway"` → `"api"` in service names
|
|
|
|
In `bin/fxt/src/tui/cockpits/pods.rs`:
|
|
- Update doc comment example: `api-gateway-7686d84bdd-zl4mn` → `api-7686d84bdd-zl4mn`
|
|
|
|
**Step 4: Verify compilation and tests**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check -p fxt
|
|
SQLX_OFFLINE=true cargo test -p fxt --lib
|
|
```
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add bin/fxt/
|
|
git commit -m "refactor(fxt): rename api_gateway → api in config, auth, and TUI"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 3: Dashboard Migration (REST+WS → grpc-web)
|
|
|
|
### Task 15: Install grpc-web tooling in dashboard
|
|
|
|
**Files:**
|
|
- Modify: `web-dashboard/package.json`
|
|
- Create: `web-dashboard/buf.gen.yaml` (protobuf codegen config)
|
|
|
|
**Step 1: Install connect-web and protobuf dependencies**
|
|
|
|
```bash
|
|
cd web-dashboard
|
|
npm install @connectrpc/connect @connectrpc/connect-web @bufbuild/protobuf
|
|
npm install -D @bufbuild/buf @connectrpc/protoc-gen-connect-es @bufbuild/protoc-gen-es
|
|
```
|
|
|
|
**Step 2: Create buf.gen.yaml**
|
|
|
|
```yaml
|
|
version: v2
|
|
plugins:
|
|
- local: protoc-gen-es
|
|
out: src/gen
|
|
opt: target=ts
|
|
- local: protoc-gen-connect-es
|
|
out: src/gen
|
|
opt: target=ts
|
|
inputs:
|
|
- directory: ../proto
|
|
```
|
|
|
|
**Step 3: Add codegen script to package.json**
|
|
|
|
```json
|
|
{
|
|
"scripts": {
|
|
"generate": "buf generate",
|
|
"dev": "vite",
|
|
"build": "tsc && vite build"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 4: Generate TypeScript proto clients**
|
|
|
|
```bash
|
|
cd web-dashboard && npx buf generate
|
|
```
|
|
|
|
This generates typed clients in `web-dashboard/src/gen/` for all proto services.
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/package.json web-dashboard/package-lock.json web-dashboard/buf.gen.yaml web-dashboard/src/gen/
|
|
git commit -m "feat(dashboard): add @connectrpc/connect-web for grpc-web communication"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 16: Create grpc-web transport layer
|
|
|
|
**Files:**
|
|
- Create: `web-dashboard/src/lib/grpc.ts` (replaces `api.ts` and `websocket.ts`)
|
|
|
|
**Step 1: Write the gRPC transport**
|
|
|
|
Create `web-dashboard/src/lib/grpc.ts`:
|
|
|
|
```typescript
|
|
import { createGrpcWebTransport } from "@connectrpc/connect-web";
|
|
import { createClient } from "@connectrpc/connect";
|
|
import { getToken } from "./auth";
|
|
|
|
// Import generated service definitions (from buf generate)
|
|
import { TradingService } from "../gen/trading_connect";
|
|
import { BacktestingService } from "../gen/foxhunt/tli/tli_trading_connect";
|
|
import { MLTrainingService } from "../gen/ml_training_connect";
|
|
import { RiskService } from "../gen/risk_connect";
|
|
import { ConfigService } from "../gen/config_connect";
|
|
import { MonitoringService } from "../gen/monitoring_connect";
|
|
import { MLService } from "../gen/ml_connect";
|
|
|
|
const baseUrl = import.meta.env.VITE_API_URL ?? "http://localhost:50051";
|
|
|
|
const transport = createGrpcWebTransport({
|
|
baseUrl,
|
|
interceptors: [
|
|
(next) => async (req) => {
|
|
const token = getToken();
|
|
if (token) {
|
|
req.header.set("authorization", `Bearer ${token}`);
|
|
}
|
|
return next(req);
|
|
},
|
|
],
|
|
});
|
|
|
|
// Typed service clients — use these throughout the dashboard
|
|
export const tradingClient = createClient(TradingService, transport);
|
|
export const backtestingClient = createClient(BacktestingService, transport);
|
|
export const mlTrainingClient = createClient(MLTrainingService, transport);
|
|
export const riskClient = createClient(RiskService, transport);
|
|
export const configClient = createClient(ConfigService, transport);
|
|
export const monitoringClient = createClient(MonitoringService, transport);
|
|
export const mlClient = createClient(MLService, transport);
|
|
```
|
|
|
|
Note: The exact import paths depend on the generated code from `buf generate`. Adjust after running codegen.
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/src/lib/grpc.ts
|
|
git commit -m "feat(dashboard): add grpc-web transport with JWT interceptor"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 17: Replace useApi hook with grpc-web calls
|
|
|
|
**Files:**
|
|
- Modify: `web-dashboard/src/hooks/useApi.ts` — rewrite to use grpc-web clients
|
|
|
|
**Step 1: Rewrite useApi.ts**
|
|
|
|
Replace the REST-based `useApiQuery` / `useApiPost` hooks with gRPC-based equivalents. Keep the React Query integration but change the fetch functions:
|
|
|
|
```typescript
|
|
import { useQuery, useMutation, type UseQueryOptions } from "@tanstack/react-query";
|
|
import { tradingClient, riskClient, mlClient, mlTrainingClient, configClient, monitoringClient } from "../lib/grpc";
|
|
|
|
// Example: replace GET /trading/positions with gRPC GetPositions
|
|
export function usePositions() {
|
|
return useQuery({
|
|
queryKey: ["positions"],
|
|
queryFn: () => tradingClient.getPositions({}),
|
|
refetchInterval: 5000,
|
|
});
|
|
}
|
|
|
|
export function useOrders() {
|
|
return useQuery({
|
|
queryKey: ["orders"],
|
|
queryFn: () => tradingClient.getOrderStatus({}),
|
|
refetchInterval: 5000,
|
|
});
|
|
}
|
|
|
|
export function useRiskMetrics() {
|
|
return useQuery({
|
|
queryKey: ["risk-metrics"],
|
|
queryFn: () => riskClient.getRiskMetrics({}),
|
|
refetchInterval: 5000,
|
|
});
|
|
}
|
|
|
|
export function useMlPredictions(symbol?: string) {
|
|
return useQuery({
|
|
queryKey: ["ml-predictions", symbol],
|
|
queryFn: () => mlClient.getPrediction({ symbol: symbol ?? "" }),
|
|
refetchInterval: 5000,
|
|
});
|
|
}
|
|
|
|
export function useTrainingJobs() {
|
|
return useQuery({
|
|
queryKey: ["training-jobs"],
|
|
queryFn: () => mlTrainingClient.listTrainingJobs({}),
|
|
refetchInterval: 5000,
|
|
});
|
|
}
|
|
|
|
// Mutation: submit order
|
|
export function useSubmitOrder() {
|
|
return useMutation({
|
|
mutationFn: (order: { symbol: string; side: string; quantity: number; price?: number }) =>
|
|
tradingClient.submitOrder(order),
|
|
});
|
|
}
|
|
```
|
|
|
|
Note: Exact method names depend on the proto service definitions. Adjust to match generated types.
|
|
|
|
**Step 2: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/src/hooks/useApi.ts
|
|
git commit -m "feat(dashboard): replace REST useApi hooks with grpc-web clients"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 18: Replace WebSocket with server-streaming RPCs
|
|
|
|
**Files:**
|
|
- Delete: `web-dashboard/src/lib/websocket.ts`
|
|
- Rewrite: `web-dashboard/src/hooks/useWebSocket.ts` → `web-dashboard/src/hooks/useGrpcStream.ts`
|
|
|
|
**Step 1: Create useGrpcStream hook**
|
|
|
|
Create `web-dashboard/src/hooks/useGrpcStream.ts`:
|
|
|
|
```typescript
|
|
import { useEffect, useRef, useState, useCallback } from "react";
|
|
import { tradingClient, riskClient, monitoringClient, mlClient } from "../lib/grpc";
|
|
|
|
// Generic hook for server-streaming RPCs
|
|
function useServerStream<T>(
|
|
streamFn: () => AsyncIterable<T>,
|
|
deps: unknown[] = []
|
|
) {
|
|
const [lastMessage, setLastMessage] = useState<T | null>(null);
|
|
const [connected, setConnected] = useState(false);
|
|
const abortRef = useRef<AbortController | null>(null);
|
|
|
|
useEffect(() => {
|
|
const controller = new AbortController();
|
|
abortRef.current = controller;
|
|
setConnected(true);
|
|
|
|
(async () => {
|
|
try {
|
|
for await (const msg of streamFn()) {
|
|
if (controller.signal.aborted) break;
|
|
setLastMessage(msg);
|
|
}
|
|
} catch {
|
|
// Stream ended or error
|
|
} finally {
|
|
setConnected(false);
|
|
}
|
|
})();
|
|
|
|
return () => {
|
|
controller.abort();
|
|
abortRef.current = null;
|
|
};
|
|
}, deps);
|
|
|
|
return { lastMessage, connected };
|
|
}
|
|
|
|
// Typed stream hooks replacing WebSocket topics
|
|
export function useMarketDataStream() {
|
|
return useServerStream(() => tradingClient.subscribeMarketData({}));
|
|
}
|
|
|
|
export function useOrderUpdatesStream() {
|
|
return useServerStream(() => tradingClient.subscribeOrderUpdates({}));
|
|
}
|
|
|
|
export function useRiskAlertsStream() {
|
|
return useServerStream(() => riskClient.streamRiskAlerts({}));
|
|
}
|
|
|
|
export function useMetricsStream() {
|
|
return useServerStream(() => monitoringClient.streamMetrics({}));
|
|
}
|
|
|
|
export function useMlPredictionStream() {
|
|
return useServerStream(() => mlClient.streamPredictions({}));
|
|
}
|
|
```
|
|
|
|
Note: Exact method names depend on proto definitions. `@connectrpc/connect-web` supports server-streaming via SSE for grpc-web.
|
|
|
|
**Step 2: Delete old websocket.ts**
|
|
|
|
```bash
|
|
rm web-dashboard/src/lib/websocket.ts
|
|
```
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/src/hooks/ web-dashboard/src/lib/
|
|
git commit -m "feat(dashboard): replace WebSocket with grpc-web server-streaming hooks"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 19: Update dashboard pages to use new hooks
|
|
|
|
**Files:**
|
|
- Modify: `web-dashboard/src/pages/TradingDashboard.tsx`
|
|
- Modify: `web-dashboard/src/pages/RiskDashboard.tsx`
|
|
- Modify: `web-dashboard/src/pages/MLDashboard.tsx`
|
|
- Modify: `web-dashboard/src/pages/PerformanceDashboard.tsx`
|
|
- Modify: `web-dashboard/src/pages/ConfigDashboard.tsx`
|
|
- Modify: `web-dashboard/src/pages/BacktestingDashboard.tsx`
|
|
- Modify: `web-dashboard/src/components/trading/OrderForm.tsx`
|
|
- Modify: `web-dashboard/src/components/common/StatusBar.tsx`
|
|
- Modify: `web-dashboard/src/lib/types.ts`
|
|
|
|
**Step 1: Update TradingDashboard**
|
|
|
|
Replace:
|
|
```typescript
|
|
import { useApiQuery, useApiPost } from "../hooks/useApi";
|
|
import { useWebSocketTopic } from "../hooks/useWebSocket";
|
|
```
|
|
with:
|
|
```typescript
|
|
import { usePositions, useOrders, useSubmitOrder } from "../hooks/useApi";
|
|
import { useMarketDataStream, useOrderUpdatesStream } from "../hooks/useGrpcStream";
|
|
```
|
|
|
|
Update component to use new typed hooks instead of generic REST fetch + WS topics.
|
|
|
|
**Step 2: Update RiskDashboard**
|
|
|
|
Replace REST `useApiQuery("/risk/metrics")` with `useRiskMetrics()`.
|
|
Replace WS `useWebSocketTopic("risk_alert")` with `useRiskAlertsStream()`.
|
|
|
|
**Step 3: Update MLDashboard**
|
|
|
|
Replace REST queries with `useMlPredictions()`, `useTrainingJobs()`.
|
|
Replace WS topic with `useMlPredictionStream()`.
|
|
|
|
**Step 4: Update PerformanceDashboard**
|
|
|
|
Replace REST queries with gRPC calls via `monitoringClient.getMetrics()`.
|
|
|
|
**Step 5: Update ConfigDashboard**
|
|
|
|
Replace REST `GET /config` and `PUT /config` with `configClient.getConfiguration()` and `configClient.updateConfiguration()`.
|
|
|
|
**Step 6: Update StatusBar**
|
|
|
|
Replace WebSocket connection status with gRPC stream connection status.
|
|
|
|
**Step 7: Update types.ts**
|
|
|
|
Remove WebSocket message types (now typed by generated proto types). Keep any UI-only types.
|
|
|
|
**Step 8: Delete old api.ts**
|
|
|
|
```bash
|
|
rm web-dashboard/src/lib/api.ts
|
|
```
|
|
|
|
**Step 9: Verify dashboard builds**
|
|
|
|
```bash
|
|
cd web-dashboard && npm run build
|
|
```
|
|
|
|
**Step 10: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/
|
|
git commit -m "feat(dashboard): migrate all pages from REST+WS to grpc-web"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 20: Update dashboard environment config
|
|
|
|
**Files:**
|
|
- Modify: `web-dashboard/.env.example`
|
|
- Modify: `web-dashboard/vite.config.ts` (remove REST proxy if present)
|
|
|
|
**Step 1: Update .env.example**
|
|
|
|
```bash
|
|
# OLD
|
|
# VITE_API_URL=https://gateway.example.com/api
|
|
# VITE_WS_URL=wss://gateway.example.com/api/ws
|
|
|
|
# NEW
|
|
# VITE_API_URL=https://api.fxhnt.ai (gRPC-web endpoint, port 50051)
|
|
```
|
|
|
|
**Step 2: Remove Vite proxy config if present**
|
|
|
|
If `vite.config.ts` has a proxy for `/api` → web-gateway, remove it and replace with proxy to gRPC port 50051.
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add web-dashboard/
|
|
git commit -m "chore(dashboard): update env config for grpc-web endpoint"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 4: Final Cleanup & Validation
|
|
|
|
### Task 21: Full workspace build and test
|
|
|
|
**Step 1: Full workspace check**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo check --workspace
|
|
```
|
|
|
|
Fix any remaining references to `api_gateway` or `web-gateway`.
|
|
|
|
**Step 2: Run clippy**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo clippy --workspace -- -D warnings
|
|
```
|
|
|
|
**Step 3: Run all lib tests**
|
|
|
|
```bash
|
|
SQLX_OFFLINE=true cargo test -p api --lib
|
|
SQLX_OFFLINE=true cargo test -p fxt --lib
|
|
SQLX_OFFLINE=true cargo test -p trading-service --lib
|
|
```
|
|
|
|
**Step 4: Build dashboard**
|
|
|
|
```bash
|
|
cd web-dashboard && npm run build
|
|
```
|
|
|
|
**Step 5: Commit any fixes**
|
|
|
|
```bash
|
|
git add -A
|
|
git commit -m "fix: resolve remaining api_gateway/web-gateway references"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 22: Clean up MEMORY.md and verify
|
|
|
|
**Step 1: Update MEMORY.md**
|
|
|
|
Update references to api_gateway and web-gateway:
|
|
- Architecture section: mention unified `services/api/`
|
|
- Web Dashboard section: note grpc-web migration
|
|
- Remove web-gateway test count
|
|
- Update api test count
|
|
|
|
**Step 2: Final grep for any remaining old names**
|
|
|
|
```bash
|
|
grep -r "api_gateway\|api-gateway\|web-gateway\|web_gateway" --include="*.rs" --include="*.toml" --include="*.yaml" --include="*.yml" --include="*.sh" --include="*.ts" --include="*.tsx" --include="*.json" | grep -v target/ | grep -v node_modules/ | grep -v ".git/"
|
|
```
|
|
|
|
Fix any remaining references.
|
|
|
|
**Step 3: Commit**
|
|
|
|
```bash
|
|
git add -A
|
|
git commit -m "chore: final cleanup — verify no stale api_gateway/web-gateway references"
|
|
```
|