Files
foxhunt/docs/plans/2026-03-04-gateway-unification-implementation.md
jgrusewski afa186b4f8 docs: add gateway unification implementation plan (22 tasks)
Phase 1: Create services/api/ from api_gateway, add tonic-web, update refs
Phase 2: Update K8s, CI, Docker, Prometheus, scripts, FXT CLI
Phase 3: Migrate dashboard from REST+WS to grpc-web (@connectrpc)
Phase 4: Final validation and cleanup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 23:18:10 +01:00

1145 lines
30 KiB
Markdown

# Gateway Unification Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Unify `services/api_gateway/` and `crates/web-gateway/` into a single `services/api/` gRPC service with tonic-web for browser access, then migrate the React dashboard from REST+WebSocket to grpc-web.
**Architecture:** Move all api_gateway code into `services/api/`, add `tonic-web` + CORS for grpc-web browser support, delete both old gateways and the web-gateway entirely, update ~30 reference files across workspace. Dashboard migrated from REST+WS to `@connectrpc/connect-web` with server-streaming replacing WebSocket topics.
**Tech Stack:** Rust (tonic 0.14, tonic-web), React 19, TypeScript, @connectrpc/connect-web, @bufbuild/protobuf
---
## Phase 1: Create services/api/ (Backend)
### Task 1: Scaffold services/api/ directory from api_gateway
**Files:**
- Create: `services/api/Cargo.toml`
- Create: `services/api/build.rs`
- Create: `services/api/src/` (copy all from `services/api_gateway/src/`)
- Create: `services/api/tests/` (copy all from `services/api_gateway/tests/`)
- Create: `services/api/benches/` (copy all from `services/api_gateway/benches/`)
**Step 1: Copy api_gateway into services/api/**
```bash
cp -r services/api_gateway services/api
```
**Step 2: Update Cargo.toml for the new crate**
In `services/api/Cargo.toml`, change:
```toml
# OLD
[package]
name = "api-gateway"
description = "API Gateway Service with 6-layer authentication and request routing"
[[bin]]
name = "api-gateway"
path = "src/main.rs"
```
to:
```toml
# NEW
[package]
name = "api"
description = "Foxhunt API Service — unified gRPC gateway with tonic-web for browser access"
[[bin]]
name = "api"
path = "src/main.rs"
```
**Step 3: Add tonic-web and tower-http CORS dependencies**
Add to `services/api/Cargo.toml` `[dependencies]`:
```toml
tonic-web = "0.13"
tower-http = { workspace = true, features = ["cors"] }
```
Remove the `axum` dependency (was only for ML REST handlers on port 8080).
**Step 4: Update build.rs proto paths**
In `services/api/build.rs`, proto paths change from `../../proto/` to `../../proto/` — this stays the same since `services/api/` is at the same depth as `services/api_gateway/`. No change needed.
**Step 5: Verify proto paths are correct**
```bash
ls services/api/../../proto/fxt_trading.proto # Should resolve to proto/fxt_trading.proto
```
**Step 6: Commit**
```bash
git add services/api/
git commit -m "feat: scaffold services/api/ from api_gateway (copy)"
```
---
### Task 2: Remove REST handlers and add tonic-web
**Files:**
- Delete: `services/api/src/handlers/` (ML REST endpoints + auth middleware for REST)
- Modify: `services/api/src/lib.rs` — remove handler re-exports, remove `handlers` module
- Modify: `services/api/src/main.rs` — remove Axum REST server, add tonic-web + CORS layer
- Modify: `services/api/Cargo.toml` — remove `axum` dep
**Step 1: Delete the handlers directory**
```bash
rm -rf services/api/src/handlers
```
**Step 2: Update lib.rs — remove handler module and re-exports**
Remove these lines from `services/api/src/lib.rs`:
```rust
pub mod handlers;
```
and:
```rust
// Re-export REST API handlers
pub use handlers::{
jwt_auth_middleware, ml_router, AuthMiddlewareState, BatchPredictRequest, BatchPredictResponse,
HotSwapRequest, HotSwapResponse, MlHandlerState, ModelStatusResponse, PredictRequest,
PredictResponse,
};
```
**Step 3: Update main.rs — replace Axum REST server with tonic-web**
Find the section that starts the Axum REST server on port 8080 and remove it entirely. Then wrap the gRPC server with tonic-web:
Add imports:
```rust
use tonic_web::GrpcWebLayer;
use tower_http::cors::{AllowHeaders, AllowMethods, CorsLayer};
use http::Method;
```
Modify the `Server::builder()` call to add tonic-web and CORS:
```rust
let cors_origins = std::env::var("CORS_ORIGINS")
.unwrap_or_else(|_| "http://localhost:5173".to_string());
let origins: Vec<_> = cors_origins
.split(',')
.filter_map(|s| s.trim().parse().ok())
.collect();
let cors = CorsLayer::new()
.allow_origin(origins)
.allow_headers(AllowHeaders::any())
.allow_methods([Method::POST, Method::OPTIONS])
.expose_headers([
"grpc-status".parse().unwrap_or_else(|_| http::HeaderName::from_static("grpc-status")),
"grpc-message".parse().unwrap_or_else(|_| http::HeaderName::from_static("grpc-message")),
]);
Server::builder()
.accept_http1(true) // Required for grpc-web (HTTP/1.1)
.layer(cors)
.layer(GrpcWebLayer::new())
// ... existing .add_service() calls unchanged ...
.serve(addr)
.await?;
```
Also remove the `axum` import and REST server startup code.
**Step 4: Update main.rs observability service name**
Change `init_observability("api_gateway", ...)` to `init_observability("api", ...)`.
**Step 5: Update main.rs clap command name**
Change `#[command(name = "api_gateway", ...)]` to `#[command(name = "api", ...)]`.
**Step 6: Remove axum dep from Cargo.toml**
Remove:
```toml
axum = "0.7"
```
Add (if not already present):
```toml
http = "1.1" # Already present
```
**Step 7: Verify it compiles**
```bash
SQLX_OFFLINE=true cargo check -p api
```
**Step 8: Commit**
```bash
git add services/api/
git commit -m "feat(api): remove REST handlers, add tonic-web + CORS for grpc-web browser access"
```
---
### Task 3: Update lib.rs internal naming
**Files:**
- Modify: `services/api/src/lib.rs`
**Step 1: Update crate-level doc comment**
Change:
```rust
//! API Gateway for Foxhunt HFT Trading System
```
to:
```rust
//! Foxhunt API Service — unified gRPC gateway with tonic-web for browser access
```
**Step 2: Verify all imports still work**
The lib crate name changes from `api_gateway` to `api`. Search `services/api/src/` for any `use api_gateway::` and replace with `use api::`:
```bash
grep -r "api_gateway" services/api/src/ --include="*.rs"
```
Replace all occurrences of `use api_gateway::` with `use api::` in the src/ directory.
**Step 3: Update tests similarly**
```bash
grep -r "api_gateway" services/api/tests/ --include="*.rs"
```
Replace `use api_gateway::` with `use api::` in all test files.
**Step 4: Verify compilation**
```bash
SQLX_OFFLINE=true cargo check -p api
```
**Step 5: Run tests**
```bash
SQLX_OFFLINE=true cargo test -p api --lib
```
**Step 6: Commit**
```bash
git add services/api/
git commit -m "refactor(api): rename crate references api_gateway → api"
```
---
### Task 4: Update workspace Cargo.toml
**Files:**
- Modify: `Cargo.toml` (workspace root)
**Step 1: Replace old members with new**
Find the `members` array and:
- Remove `"crates/web-gateway"`
- Remove `"services/api_gateway"`
- Add `"services/api"`
- Change `"testing/api-gateway-load"` to `"testing/api-load"`
**Step 2: Update workspace dependency if exists**
Search for `api-gateway` in the `[workspace.dependencies]` section. If present, rename to `api`.
**Step 3: Verify workspace resolves**
```bash
SQLX_OFFLINE=true cargo check -p api
```
**Step 4: Commit**
```bash
git add Cargo.toml
git commit -m "chore: update workspace members — api_gateway+web-gateway → api"
```
---
### Task 5: Update trading_service dependency
**Files:**
- Modify: `services/trading_service/Cargo.toml:106`
**Step 1: Update path dependency**
Change:
```toml
api-gateway = { path = "../api_gateway" } # For auth tests - no cyclic dependency
```
to:
```toml
api = { path = "../api" } # For auth tests - no cyclic dependency
```
**Step 2: Update any import references in trading_service**
```bash
grep -r "api_gateway\|api-gateway" services/trading_service/ --include="*.rs"
```
Replace `use api_gateway::` with `use api::` in any files found.
**Step 3: Verify compilation**
```bash
SQLX_OFFLINE=true cargo check -p trading-service
```
**Step 4: Commit**
```bash
git add services/trading_service/
git commit -m "refactor(trading-service): update dep api-gateway → api"
```
---
### Task 6: Rename testing/api-gateway-load → testing/api-load
**Files:**
- Rename: `testing/api-gateway-load/``testing/api-load/`
- Modify: `testing/api-load/Cargo.toml`
**Step 1: Move directory**
```bash
mv testing/api-gateway-load testing/api-load
```
**Step 2: Update Cargo.toml**
In `testing/api-load/Cargo.toml`, change:
```toml
name = "api_gateway_load_tests"
```
to:
```toml
name = "api_load_tests"
```
**Step 3: Verify compilation**
```bash
SQLX_OFFLINE=true cargo check -p api_load_tests
```
**Step 4: Commit**
```bash
git add testing/api-load/ testing/api-gateway-load/
git commit -m "refactor: rename testing/api-gateway-load → testing/api-load"
```
---
### Task 7: Update test crates referencing api_gateway
**Files:**
- Modify: `testing/service-integration/tests/common/auth_helpers.rs:235``get_api_gateway_addr()``get_api_addr()`
- Modify: `testing/service-integration/tests/backtesting_service_e2e.rs` — update imports
- Modify: `testing/service-integration/tests/service_health_resilience_e2e.rs` — update imports
- Modify: `testing/integration/fixtures/mod.rs` — update comment
- Modify: `testing/integration/smoke_tests/service_health.rs` — rename fn
- Modify: `testing/integration/smoke_tests/mod.rs:71` — rename fn
- Modify: `testing/integration/standalone/watch_tuning_progress_updated.rs` — rename param
**Step 1: Update auth_helpers.rs**
Rename `get_api_gateway_addr()``get_api_addr()` and update the env var it reads (likely `API_GATEWAY_URL` → keep same env var for backwards compat, or rename to `API_URL`).
**Step 2: Update all call sites**
Replace `get_api_gateway_addr` with `get_api_addr` across all test files. Replace `api_gateway_url` with `api_url` across smoke tests.
**Step 3: Run test crate compilation**
```bash
SQLX_OFFLINE=true cargo check -p service-integration-tests
SQLX_OFFLINE=true cargo check -p integration-tests
```
**Step 4: Commit**
```bash
git add testing/
git commit -m "refactor(tests): rename api_gateway references → api"
```
---
### Task 8: Delete old gateways
**Files:**
- Delete: `services/api_gateway/` (entire directory)
- Delete: `crates/web-gateway/` (entire directory)
**Step 1: Verify services/api/ compiles and tests pass**
```bash
SQLX_OFFLINE=true cargo check --workspace
SQLX_OFFLINE=true cargo test -p api --lib
```
**Step 2: Delete old directories**
```bash
rm -rf services/api_gateway
rm -rf crates/web-gateway
```
**Step 3: Verify workspace still builds**
```bash
SQLX_OFFLINE=true cargo check --workspace
```
**Step 4: Commit**
```bash
git add -A
git commit -m "chore: delete services/api_gateway/ and crates/web-gateway/ (replaced by services/api/)"
```
---
## Phase 2: Update Infrastructure & Config
### Task 9: Update K8s manifests
**Files:**
- Modify: `infra/k8s/services/api-gateway.yaml` → rename to `infra/k8s/services/api.yaml`
- Delete: `infra/k8s/services/web-gateway.yaml`
- Modify: `infra/k8s/network-policies/api-gateway.yaml` → rename to `infra/k8s/network-policies/api.yaml`
- Delete: `infra/k8s/network-policies/web-gateway.yaml`
**Step 1: Rename api-gateway.yaml → api.yaml**
```bash
mv infra/k8s/services/api-gateway.yaml infra/k8s/services/api.yaml
mv infra/k8s/network-policies/api-gateway.yaml infra/k8s/network-policies/api.yaml
```
**Step 2: Update api.yaml deployment**
In `infra/k8s/services/api.yaml`, search-replace:
- `app: api-gateway``app: api`
- `name: api-gateway``name: api`
- Binary name in initContainer: `api-gateway``api`
- Service name references
Remove the web-gateway environment variable (`WEB_GATEWAY_URL` or similar) if present.
Add `CORS_ORIGINS` env var:
```yaml
- name: CORS_ORIGINS
value: "https://dashboard.fxhnt.ai,http://localhost:5173"
```
**Step 3: Update api.yaml network policy**
In `infra/k8s/network-policies/api.yaml`:
- Update `app: api-gateway``app: api` in podSelector
- Remove web-gateway egress rule (port 3000)
- Keep all other egress rules (backend services, DB, Redis)
**Step 4: Delete web-gateway manifests**
```bash
rm infra/k8s/services/web-gateway.yaml
rm infra/k8s/network-policies/web-gateway.yaml
```
**Step 5: Update any other K8s files referencing api-gateway or web-gateway**
```bash
grep -r "api-gateway\|web-gateway" infra/k8s/ --include="*.yaml"
```
Update references in other service network policies that allow traffic to/from api-gateway or web-gateway.
**Step 6: Commit**
```bash
git add infra/k8s/
git commit -m "infra(k8s): rename api-gateway → api, delete web-gateway manifests"
```
---
### Task 10: Update CI pipeline (.gitlab-ci.yml)
**Files:**
- Modify: `.gitlab-ci.yml`
**Step 1: Rename compile job**
Change `compile-api-gateway` to `compile-api`:
- `SERVICE_PACKAGE: api-gateway``SERVICE_PACKAGE: api`
- Rules path: `services/api_gateway/**``services/api/**`
- Anchor: `&rules-api-gateway``&rules-api`
**Step 2: Delete web-gateway compile job**
Remove the entire `compile-web-gateway` job block (lines ~575-600).
**Step 3: Rename deploy job**
Change `deploy-api-gateway` to `deploy-api`:
- `needs: [{job: compile-api-gateway, ...}]``needs: [{job: compile-api, ...}]`
- `SERVICE_PACKAGE: api-gateway``SERVICE_PACKAGE: api`
- `rules: *rules-api-gateway``rules: *rules-api`
**Step 4: Delete web-gateway deploy job**
Remove the entire `deploy-web-gateway` job block (lines ~1689-1694).
**Step 5: Update JWT_SECRET comment**
Line 40: `# Stable JWT secret for api_gateway tests``# Stable JWT secret for api tests`
**Step 6: Commit**
```bash
git add .gitlab-ci.yml
git commit -m "ci: rename api-gateway → api, remove web-gateway jobs"
```
---
### Task 11: Update Docker Compose
**Files:**
- Modify: `docker-compose.yml`
- Modify: `docker-compose.override.yml`
**Step 1: Rename api_gateway service in docker-compose.yml**
Change service name `api_gateway:``api:` and update:
- `container_name: foxhunt-api-gateway``container_name: foxhunt-api`
- `dockerfile: services/api_gateway/Dockerfile``dockerfile: services/api/Dockerfile`
- `JWT_ISSUER=foxhunt-api-gateway``JWT_ISSUER=foxhunt-api`
- Add `CORS_ORIGINS=http://localhost:5173`
Also update any other services referencing `api_gateway` hostname to `api`.
**Step 2: Remove web-gateway service if present**
Search for `web-gateway` or `web_gateway` service block and remove it.
**Step 3: Update docker-compose.override.yml**
Line 4: `api_gateway:``api:`
**Step 4: Update JWT_ISSUER in other services**
Lines 202, 258, 316 reference `JWT_ISSUER=foxhunt-api-gateway` — change to `foxhunt-api`.
**Step 5: Commit**
```bash
git add docker-compose.yml docker-compose.override.yml
git commit -m "chore(docker): rename api_gateway → api, remove web-gateway"
```
---
### Task 12: Update Prometheus config
**Files:**
- Modify: `config/prometheus/prometheus.yml`
- Modify: `config/prometheus/rules/production-alerts.yml`
- Modify: `config/prometheus/rules/service-health-alerts.yml`
**Step 1: Update scrape target**
In `prometheus.yml`:
```yaml
# OLD
- job_name: 'api_gateway'
static_configs:
- targets: ['api_gateway:9091']
# NEW
- job_name: 'api'
static_configs:
- targets: ['api:9091']
```
**Step 2: Update alert rules**
In `production-alerts.yml`, replace all `api_gateway` with `api` in:
- `job="api_gateway"``job="api"`
- `service: api_gateway``service: api`
- String literals in annotations
In `service-health-alerts.yml`:
- `job="api_gateway"``job="api"`
- `service: api_gateway``service: api`
**Step 3: Commit**
```bash
git add config/prometheus/
git commit -m "config(prometheus): rename api_gateway → api in scrape targets and alert rules"
```
---
### Task 13: Update scripts
**Files:**
- Modify: `infra/scripts/smoke-test.sh` — rename web-gateway checks to api
- Modify: `infra/scripts/build-and-push.sh` — rename api-gateway, remove web-gateway
- Modify: `scripts/generate_dev_certs.sh` — update DNS alt name
- Modify: `scripts/enforce_coverage.sh` — update path
**Step 1: Update smoke-test.sh**
Replace web-gateway pod/service checks with api service checks. The section `"Services (gRPC + web-gateway)"` becomes just `"Services (gRPC)"`. Remove web-gateway health check (HTTP /health on port 3000) — api uses gRPC health check.
**Step 2: Update build-and-push.sh**
Replace `api-gateway` in the services list. Remove web-gateway build section and `--skip-webgw` flag.
**Step 3: Update generate_dev_certs.sh**
Line 75: `DNS.5 = api-gateway.foxhunt.local``DNS.5 = api.foxhunt.local`
**Step 4: Update enforce_coverage.sh**
Line 32: `"services/api_gateway"``"services/api"`
**Step 5: Commit**
```bash
git add infra/scripts/ scripts/
git commit -m "chore(scripts): rename api-gateway → api, remove web-gateway references"
```
---
### Task 14: Update FXT CLI references
**Files:**
- Modify: `bin/fxt/src/config.rs` — rename `api_gateway_url` field to `api_url`
- Modify: `bin/fxt/src/commands/config.rs` — update field reference
- Modify: `bin/fxt/src/auth/login.rs` — rename `is_api_gateway_available()`
- Modify: `bin/fxt/src/auth/jwt_generator.rs` — update any references
- Modify: `bin/fxt/src/tui/data_fetcher.rs` — update test data `"api-gateway"``"api"`
- Modify: `bin/fxt/src/tui/event_loop.rs` — update service name
- Modify: `bin/fxt/src/tui/cockpits/pods.rs` — update doc comment
**Step 1: Rename config field**
In `bin/fxt/src/config.rs`:
- `pub api_gateway_url: String``pub api_url: String`
- `fn default_api_gateway_url()``fn default_api_url()`
- `#[serde(default = "default_api_gateway_url")]``#[serde(default = "default_api_url")]`
- Update all tests referencing this field
- Keep the serde alias for backwards compat: `#[serde(alias = "api_gateway_url")]`
**Step 2: Update CLI and auth references**
In `bin/fxt/src/auth/login.rs`:
- `is_api_gateway_available()``is_api_available()`
In `bin/fxt/src/commands/config.rs`:
- `config.api_gateway_url``config.api_url`
**Step 3: Update TUI test data**
In `bin/fxt/src/tui/data_fetcher.rs`:
- `"api-gateway"``"api"` in test helper and assertions
In `bin/fxt/src/tui/event_loop.rs`:
- `"api-gateway"``"api"` in service names
In `bin/fxt/src/tui/cockpits/pods.rs`:
- Update doc comment example: `api-gateway-7686d84bdd-zl4mn``api-7686d84bdd-zl4mn`
**Step 4: Verify compilation and tests**
```bash
SQLX_OFFLINE=true cargo check -p fxt
SQLX_OFFLINE=true cargo test -p fxt --lib
```
**Step 5: Commit**
```bash
git add bin/fxt/
git commit -m "refactor(fxt): rename api_gateway → api in config, auth, and TUI"
```
---
## Phase 3: Dashboard Migration (REST+WS → grpc-web)
### Task 15: Install grpc-web tooling in dashboard
**Files:**
- Modify: `web-dashboard/package.json`
- Create: `web-dashboard/buf.gen.yaml` (protobuf codegen config)
**Step 1: Install connect-web and protobuf dependencies**
```bash
cd web-dashboard
npm install @connectrpc/connect @connectrpc/connect-web @bufbuild/protobuf
npm install -D @bufbuild/buf @connectrpc/protoc-gen-connect-es @bufbuild/protoc-gen-es
```
**Step 2: Create buf.gen.yaml**
```yaml
version: v2
plugins:
- local: protoc-gen-es
out: src/gen
opt: target=ts
- local: protoc-gen-connect-es
out: src/gen
opt: target=ts
inputs:
- directory: ../proto
```
**Step 3: Add codegen script to package.json**
```json
{
"scripts": {
"generate": "buf generate",
"dev": "vite",
"build": "tsc && vite build"
}
}
```
**Step 4: Generate TypeScript proto clients**
```bash
cd web-dashboard && npx buf generate
```
This generates typed clients in `web-dashboard/src/gen/` for all proto services.
**Step 5: Commit**
```bash
git add web-dashboard/package.json web-dashboard/package-lock.json web-dashboard/buf.gen.yaml web-dashboard/src/gen/
git commit -m "feat(dashboard): add @connectrpc/connect-web for grpc-web communication"
```
---
### Task 16: Create grpc-web transport layer
**Files:**
- Create: `web-dashboard/src/lib/grpc.ts` (replaces `api.ts` and `websocket.ts`)
**Step 1: Write the gRPC transport**
Create `web-dashboard/src/lib/grpc.ts`:
```typescript
import { createGrpcWebTransport } from "@connectrpc/connect-web";
import { createClient } from "@connectrpc/connect";
import { getToken } from "./auth";
// Import generated service definitions (from buf generate)
import { TradingService } from "../gen/trading_connect";
import { BacktestingService } from "../gen/foxhunt/tli/tli_trading_connect";
import { MLTrainingService } from "../gen/ml_training_connect";
import { RiskService } from "../gen/risk_connect";
import { ConfigService } from "../gen/config_connect";
import { MonitoringService } from "../gen/monitoring_connect";
import { MLService } from "../gen/ml_connect";
const baseUrl = import.meta.env.VITE_API_URL ?? "http://localhost:50051";
const transport = createGrpcWebTransport({
baseUrl,
interceptors: [
(next) => async (req) => {
const token = getToken();
if (token) {
req.header.set("authorization", `Bearer ${token}`);
}
return next(req);
},
],
});
// Typed service clients — use these throughout the dashboard
export const tradingClient = createClient(TradingService, transport);
export const backtestingClient = createClient(BacktestingService, transport);
export const mlTrainingClient = createClient(MLTrainingService, transport);
export const riskClient = createClient(RiskService, transport);
export const configClient = createClient(ConfigService, transport);
export const monitoringClient = createClient(MonitoringService, transport);
export const mlClient = createClient(MLService, transport);
```
Note: The exact import paths depend on the generated code from `buf generate`. Adjust after running codegen.
**Step 2: Commit**
```bash
git add web-dashboard/src/lib/grpc.ts
git commit -m "feat(dashboard): add grpc-web transport with JWT interceptor"
```
---
### Task 17: Replace useApi hook with grpc-web calls
**Files:**
- Modify: `web-dashboard/src/hooks/useApi.ts` — rewrite to use grpc-web clients
**Step 1: Rewrite useApi.ts**
Replace the REST-based `useApiQuery` / `useApiPost` hooks with gRPC-based equivalents. Keep the React Query integration but change the fetch functions:
```typescript
import { useQuery, useMutation, type UseQueryOptions } from "@tanstack/react-query";
import { tradingClient, riskClient, mlClient, mlTrainingClient, configClient, monitoringClient } from "../lib/grpc";
// Example: replace GET /trading/positions with gRPC GetPositions
export function usePositions() {
return useQuery({
queryKey: ["positions"],
queryFn: () => tradingClient.getPositions({}),
refetchInterval: 5000,
});
}
export function useOrders() {
return useQuery({
queryKey: ["orders"],
queryFn: () => tradingClient.getOrderStatus({}),
refetchInterval: 5000,
});
}
export function useRiskMetrics() {
return useQuery({
queryKey: ["risk-metrics"],
queryFn: () => riskClient.getRiskMetrics({}),
refetchInterval: 5000,
});
}
export function useMlPredictions(symbol?: string) {
return useQuery({
queryKey: ["ml-predictions", symbol],
queryFn: () => mlClient.getPrediction({ symbol: symbol ?? "" }),
refetchInterval: 5000,
});
}
export function useTrainingJobs() {
return useQuery({
queryKey: ["training-jobs"],
queryFn: () => mlTrainingClient.listTrainingJobs({}),
refetchInterval: 5000,
});
}
// Mutation: submit order
export function useSubmitOrder() {
return useMutation({
mutationFn: (order: { symbol: string; side: string; quantity: number; price?: number }) =>
tradingClient.submitOrder(order),
});
}
```
Note: Exact method names depend on the proto service definitions. Adjust to match generated types.
**Step 2: Commit**
```bash
git add web-dashboard/src/hooks/useApi.ts
git commit -m "feat(dashboard): replace REST useApi hooks with grpc-web clients"
```
---
### Task 18: Replace WebSocket with server-streaming RPCs
**Files:**
- Delete: `web-dashboard/src/lib/websocket.ts`
- Rewrite: `web-dashboard/src/hooks/useWebSocket.ts``web-dashboard/src/hooks/useGrpcStream.ts`
**Step 1: Create useGrpcStream hook**
Create `web-dashboard/src/hooks/useGrpcStream.ts`:
```typescript
import { useEffect, useRef, useState, useCallback } from "react";
import { tradingClient, riskClient, monitoringClient, mlClient } from "../lib/grpc";
// Generic hook for server-streaming RPCs
function useServerStream<T>(
streamFn: () => AsyncIterable<T>,
deps: unknown[] = []
) {
const [lastMessage, setLastMessage] = useState<T | null>(null);
const [connected, setConnected] = useState(false);
const abortRef = useRef<AbortController | null>(null);
useEffect(() => {
const controller = new AbortController();
abortRef.current = controller;
setConnected(true);
(async () => {
try {
for await (const msg of streamFn()) {
if (controller.signal.aborted) break;
setLastMessage(msg);
}
} catch {
// Stream ended or error
} finally {
setConnected(false);
}
})();
return () => {
controller.abort();
abortRef.current = null;
};
}, deps);
return { lastMessage, connected };
}
// Typed stream hooks replacing WebSocket topics
export function useMarketDataStream() {
return useServerStream(() => tradingClient.subscribeMarketData({}));
}
export function useOrderUpdatesStream() {
return useServerStream(() => tradingClient.subscribeOrderUpdates({}));
}
export function useRiskAlertsStream() {
return useServerStream(() => riskClient.streamRiskAlerts({}));
}
export function useMetricsStream() {
return useServerStream(() => monitoringClient.streamMetrics({}));
}
export function useMlPredictionStream() {
return useServerStream(() => mlClient.streamPredictions({}));
}
```
Note: Exact method names depend on proto definitions. `@connectrpc/connect-web` supports server-streaming via SSE for grpc-web.
**Step 2: Delete old websocket.ts**
```bash
rm web-dashboard/src/lib/websocket.ts
```
**Step 3: Commit**
```bash
git add web-dashboard/src/hooks/ web-dashboard/src/lib/
git commit -m "feat(dashboard): replace WebSocket with grpc-web server-streaming hooks"
```
---
### Task 19: Update dashboard pages to use new hooks
**Files:**
- Modify: `web-dashboard/src/pages/TradingDashboard.tsx`
- Modify: `web-dashboard/src/pages/RiskDashboard.tsx`
- Modify: `web-dashboard/src/pages/MLDashboard.tsx`
- Modify: `web-dashboard/src/pages/PerformanceDashboard.tsx`
- Modify: `web-dashboard/src/pages/ConfigDashboard.tsx`
- Modify: `web-dashboard/src/pages/BacktestingDashboard.tsx`
- Modify: `web-dashboard/src/components/trading/OrderForm.tsx`
- Modify: `web-dashboard/src/components/common/StatusBar.tsx`
- Modify: `web-dashboard/src/lib/types.ts`
**Step 1: Update TradingDashboard**
Replace:
```typescript
import { useApiQuery, useApiPost } from "../hooks/useApi";
import { useWebSocketTopic } from "../hooks/useWebSocket";
```
with:
```typescript
import { usePositions, useOrders, useSubmitOrder } from "../hooks/useApi";
import { useMarketDataStream, useOrderUpdatesStream } from "../hooks/useGrpcStream";
```
Update component to use new typed hooks instead of generic REST fetch + WS topics.
**Step 2: Update RiskDashboard**
Replace REST `useApiQuery("/risk/metrics")` with `useRiskMetrics()`.
Replace WS `useWebSocketTopic("risk_alert")` with `useRiskAlertsStream()`.
**Step 3: Update MLDashboard**
Replace REST queries with `useMlPredictions()`, `useTrainingJobs()`.
Replace WS topic with `useMlPredictionStream()`.
**Step 4: Update PerformanceDashboard**
Replace REST queries with gRPC calls via `monitoringClient.getMetrics()`.
**Step 5: Update ConfigDashboard**
Replace REST `GET /config` and `PUT /config` with `configClient.getConfiguration()` and `configClient.updateConfiguration()`.
**Step 6: Update StatusBar**
Replace WebSocket connection status with gRPC stream connection status.
**Step 7: Update types.ts**
Remove WebSocket message types (now typed by generated proto types). Keep any UI-only types.
**Step 8: Delete old api.ts**
```bash
rm web-dashboard/src/lib/api.ts
```
**Step 9: Verify dashboard builds**
```bash
cd web-dashboard && npm run build
```
**Step 10: Commit**
```bash
git add web-dashboard/
git commit -m "feat(dashboard): migrate all pages from REST+WS to grpc-web"
```
---
### Task 20: Update dashboard environment config
**Files:**
- Modify: `web-dashboard/.env.example`
- Modify: `web-dashboard/vite.config.ts` (remove REST proxy if present)
**Step 1: Update .env.example**
```bash
# OLD
# VITE_API_URL=https://gateway.example.com/api
# VITE_WS_URL=wss://gateway.example.com/api/ws
# NEW
# VITE_API_URL=https://api.fxhnt.ai (gRPC-web endpoint, port 50051)
```
**Step 2: Remove Vite proxy config if present**
If `vite.config.ts` has a proxy for `/api` → web-gateway, remove it and replace with proxy to gRPC port 50051.
**Step 3: Commit**
```bash
git add web-dashboard/
git commit -m "chore(dashboard): update env config for grpc-web endpoint"
```
---
## Phase 4: Final Cleanup & Validation
### Task 21: Full workspace build and test
**Step 1: Full workspace check**
```bash
SQLX_OFFLINE=true cargo check --workspace
```
Fix any remaining references to `api_gateway` or `web-gateway`.
**Step 2: Run clippy**
```bash
SQLX_OFFLINE=true cargo clippy --workspace -- -D warnings
```
**Step 3: Run all lib tests**
```bash
SQLX_OFFLINE=true cargo test -p api --lib
SQLX_OFFLINE=true cargo test -p fxt --lib
SQLX_OFFLINE=true cargo test -p trading-service --lib
```
**Step 4: Build dashboard**
```bash
cd web-dashboard && npm run build
```
**Step 5: Commit any fixes**
```bash
git add -A
git commit -m "fix: resolve remaining api_gateway/web-gateway references"
```
---
### Task 22: Clean up MEMORY.md and verify
**Step 1: Update MEMORY.md**
Update references to api_gateway and web-gateway:
- Architecture section: mention unified `services/api/`
- Web Dashboard section: note grpc-web migration
- Remove web-gateway test count
- Update api test count
**Step 2: Final grep for any remaining old names**
```bash
grep -r "api_gateway\|api-gateway\|web-gateway\|web_gateway" --include="*.rs" --include="*.toml" --include="*.yaml" --include="*.yml" --include="*.sh" --include="*.ts" --include="*.tsx" --include="*.json" | grep -v target/ | grep -v node_modules/ | grep -v ".git/"
```
Fix any remaining references.
**Step 3: Commit**
```bash
git add -A
git commit -m "chore: final cleanup — verify no stale api_gateway/web-gateway references"
```