# Per-Service CI Compilation & Selective Deploy — Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Replace monolithic compile-services with 8 per-service compile jobs and selective deploy, plus rename all service binaries to kebab-case.

**Architecture:** Each service gets its own GitLab CI compile job with dependency-aware `changes:` filters. The deploy job only uploads and restarts services whose binaries were actually built. Service crate names change from snake_case to kebab-case so binary names match k8s deployment names directly.

**Tech Stack:** GitLab CI YAML, Cargo workspace, Kubernetes manifests, shell scripts

**Key insight:** Rust auto-maps hyphens to underscores for crate identifiers, so `use trading_service::...` still compiles after renaming the package to `trading-service`. No Rust source `use` statements need changing.

---

### Task 1: Rename api_gateway crate to api-gateway

**Files:**
- Modify: `services/api_gateway/Cargo.toml` (lines 2, 11)
- Modify: `infra/k8s/services/api-gateway.yaml` (lines 49, 50, 57, 88)

**Step 1: Update Cargo.toml package and bin names**

In `services/api_gateway/Cargo.toml`, change:
```toml
name = "api_gateway"
```
to:
```toml
name = "api-gateway"
```

And the `[[bin]]` section:
```toml
name = "api_gateway"
```
to:
```toml
name = "api-gateway"
```

**Step 2: Update K8s manifest S3 path and command**

In `infra/k8s/services/api-gateway.yaml`, update the initContainer rclone line:
```
":s3:foxhunt-binaries/services/api_gateway"  →  ":s3:foxhunt-binaries/services/api-gateway"
"/binaries/api_gateway"  →  "/binaries/api-gateway"
```
And the container command:
```yaml
command: ["/binaries/api_gateway"]  →  command: ["/binaries/api-gateway"]
```
And the chmod+echo lines in the initContainer script.

**Step 3: Update cross-crate dependency in trading_service**

In `services/trading_service/Cargo.toml`, the dev-dependency:
```toml
api_gateway = { path = "../api_gateway" }
```
becomes:
```toml
api-gateway = { path = "../api_gateway" }
```
(Path stays the same — only the package key changes. Rust still uses `use api_gateway::` because of hyphen-to-underscore mapping.)

**Step 4: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p api-gateway`
Expected: compiles successfully

**Step 5: Commit**

```bash
git add services/api_gateway/Cargo.toml infra/k8s/services/api-gateway.yaml services/trading_service/Cargo.toml
git commit -m "refactor: rename api_gateway crate to api-gateway (kebab-case)"
```

---

### Task 2: Rename trading_service crate to trading-service

**Files:**
- Modify: `services/trading_service/Cargo.toml` (lines 2, 11)
- Modify: `infra/k8s/services/trading-service.yaml` (initContainer + command)
- Modify: `testing/integration/Cargo.toml` (line 22: `trading_service` dep)

**Step 1: Update Cargo.toml package and bin names**

```toml
name = "trading_service"  →  name = "trading-service"
[[bin]] name = "trading_service"  →  [[bin]] name = "trading-service"
```
Note: keep the second `[[bin]] name = "latency_validator"` unchanged (it's a separate binary).

**Step 2: Update K8s manifest**

In `infra/k8s/services/trading-service.yaml`:
```
":s3:foxhunt-binaries/services/trading_service"  →  ":s3:foxhunt-binaries/services/trading-service"
"/binaries/trading_service"  →  "/binaries/trading-service"
command: ["/binaries/trading_service"]  →  command: ["/binaries/trading-service"]
```

**Step 3: Update testing crate dependency**

In `testing/integration/Cargo.toml`:
```toml
trading_service = { path = "../../services/trading_service" }
```
→
```toml
trading-service = { path = "../../services/trading_service" }
```

**Step 4: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p trading-service`

**Step 5: Commit**

```bash
git commit -m "refactor: rename trading_service crate to trading-service (kebab-case)"
```

---

### Task 3: Rename broker_gateway_service crate to broker-gateway

**Files:**
- Modify: `services/broker_gateway_service/Cargo.toml` (lines 2, 11)
- Modify: `infra/k8s/services/broker-gateway.yaml` (initContainer + command)

**Step 1: Update Cargo.toml**

```toml
name = "broker_gateway_service"  →  name = "broker-gateway"
[[bin]] name = "broker_gateway_service"  →  [[bin]] name = "broker-gateway"
```

**Step 2: Update K8s manifest**

```
":s3:foxhunt-binaries/services/broker_gateway_service"  →  ":s3:foxhunt-binaries/services/broker-gateway"
"/binaries/broker_gateway_service"  →  "/binaries/broker-gateway"
command: ["/binaries/broker_gateway_service"]  →  command: ["/binaries/broker-gateway"]
```

No cross-crate dependencies reference this crate.

**Step 3: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p broker-gateway`

**Step 4: Commit**

```bash
git commit -m "refactor: rename broker_gateway_service crate to broker-gateway (kebab-case)"
```

---

### Task 4: Rename ml_training_service crate to ml-training-service

**Files:**
- Modify: `services/ml_training_service/Cargo.toml` (lines 2, 107)
- Modify: `infra/k8s/services/ml-training-service.yaml` (initContainer + command)
- Modify: `testing/e2e/Cargo.toml` (line 68: `ml_training_service` dep)

**Step 1: Update Cargo.toml**

```toml
name = "ml_training_service"  →  name = "ml-training-service"
[[bin]] name = "ml_training_service"  →  [[bin]] name = "ml-training-service"
```

**Step 2: Update K8s manifest**

```
":s3:foxhunt-binaries/services/ml_training_service"  →  ":s3:foxhunt-binaries/services/ml-training-service"
"/binaries/ml_training_service"  →  "/binaries/ml-training-service"
command: ["/binaries/ml_training_service", "serve"]  →  command: ["/binaries/ml-training-service", "serve"]
```

**Step 3: Update testing crate dependency**

In `testing/e2e/Cargo.toml`:
```toml
ml_training_service = { path = "../../services/ml_training_service" }
```
→
```toml
ml-training-service = { path = "../../services/ml_training_service" }
```

**Step 4: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p ml-training-service`

**Step 5: Commit**

```bash
git commit -m "refactor: rename ml_training_service crate to ml-training-service (kebab-case)"
```

---

### Task 5: Rename backtesting_service crate to backtesting-service

**Files:**
- Modify: `services/backtesting_service/Cargo.toml` (lines 2, 11)
- Modify: `infra/k8s/services/backtesting-service.yaml` (initContainer + command)
- Modify: `testing/service-integration/Cargo.toml` (line 52: `backtesting_service` dep)

**Step 1: Update Cargo.toml**

```toml
name = "backtesting_service"  →  name = "backtesting-service"
[[bin]] name = "backtesting_service"  →  [[bin]] name = "backtesting-service"
```

**Step 2: Update K8s manifest**

```
":s3:foxhunt-binaries/services/backtesting_service"  →  ":s3:foxhunt-binaries/services/backtesting-service"
"/binaries/backtesting_service"  →  "/binaries/backtesting-service"
command: ["/binaries/backtesting_service"]  →  command: ["/binaries/backtesting-service"]
```

**Step 3: Update testing crate dependency**

In `testing/service-integration/Cargo.toml`:
```toml
backtesting_service = { path = "../../services/backtesting_service" }
```
→
```toml
backtesting-service = { path = "../../services/backtesting_service" }
```

**Step 4: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p backtesting-service`

**Step 5: Commit**

```bash
git commit -m "refactor: rename backtesting_service crate to backtesting-service (kebab-case)"
```

---

### Task 6: Rename trading_agent_service crate to trading-agent-service

**Files:**
- Modify: `services/trading_agent_service/Cargo.toml` (lines 2, 11)
- Modify: `infra/k8s/services/trading-agent-service.yaml` (initContainer + command)

No cross-crate dependencies reference this crate.

**Step 1: Update Cargo.toml**

```toml
name = "trading_agent_service"  →  name = "trading-agent-service"
[[bin]] name = "trading_agent_service"  →  [[bin]] name = "trading-agent-service"
```

**Step 2: Update K8s manifest**

```
":s3:foxhunt-binaries/services/trading_agent_service"  →  ":s3:foxhunt-binaries/services/trading-agent-service"
"/binaries/trading_agent_service"  →  "/binaries/trading-agent-service"
command: ["/binaries/trading_agent_service"]  →  command: ["/binaries/trading-agent-service"]
```

**Step 3: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p trading-agent-service`

**Step 4: Commit**

```bash
git commit -m "refactor: rename trading_agent_service crate to trading-agent-service (kebab-case)"
```

---

### Task 7: Rename data_acquisition_service crate to data-acquisition-service

**Files:**
- Modify: `services/data_acquisition_service/Cargo.toml` (lines 2, 71)
- Modify: `infra/k8s/services/data-acquisition-service.yaml` (initContainer + command)

No cross-crate dependencies reference this crate.

**Step 1: Update Cargo.toml**

```toml
name = "data_acquisition_service"  →  name = "data-acquisition-service"
[[bin]] name = "data_acquisition_service"  →  [[bin]] name = "data-acquisition-service"
```

**Step 2: Update K8s manifest**

```
":s3:foxhunt-binaries/services/data_acquisition_service"  →  ":s3:foxhunt-binaries/services/data-acquisition-service"
"/binaries/data_acquisition_service"  →  "/binaries/data-acquisition-service"
command: ["/binaries/data_acquisition_service"]  →  command: ["/binaries/data-acquisition-service"]
```

**Step 3: Verify compilation**

Run: `SQLX_OFFLINE=true cargo check -p data-acquisition-service`

**Step 4: Commit**

```bash
git commit -m "refactor: rename data_acquisition_service crate to data-acquisition-service (kebab-case)"
```

---

### Task 8: Update scripts for kebab-case binary names

**Files:**
- Modify: `scripts/start.sh` (lines ~85-118: pkill and cargo run references)
- Modify: `scripts/stop.sh` (lines ~18-43: pid files and pkill references)
- Modify: `scripts/validate_data_quality.sh` (lines 109, 113: `-p backtesting_service`)
- Modify: `scripts/setup-database.sh` (line 112: `--bin trading_service`)
- Modify: `infra/scripts/build-and-push.sh` (lines 26-31: STANDARD_SERVICES array)

**Step 1: Update start.sh**

Replace all `trading_service` → `trading-service`, `backtesting_service` → `backtesting-service`, `ml_training_service` → `ml-training-service` in cargo run/build/pkill commands.

**Step 2: Update stop.sh**

Same pattern: update pid file names and pkill patterns.

**Step 3: Update validate_data_quality.sh**

```bash
cargo build --release -p backtesting_service  →  cargo build --release -p backtesting-service
```

**Step 4: Update setup-database.sh**

```bash
cargo run --bin trading_service  →  cargo run --bin trading-service
```

**Step 5: Update build-and-push.sh**

```bash
STANDARD_SERVICES=(
  api-gateway
  trading-service
  trading-agent-service
  ml-training-service
  backtesting-service
  broker-gateway
)
```

**Step 6: Commit**

```bash
git commit -m "refactor: update scripts for kebab-case service binary names"
```

---

### Task 9: Replace compile-services with per-service compile jobs in .gitlab-ci.yml

**Files:**
- Modify: `.gitlab-ci.yml` (replace compile-services job ~lines 411-459 with base template + 8 per-service jobs)

**Step 1: Add .compile-service-base YAML anchor**

Insert after the `.rust-base-cpu` template (after line 360):

```yaml
# Base template for per-service compile jobs
.compile-service-base:
  extends: .rust-base-cpu
  stage: compile
  needs: []
  interruptible: true
  variables:
    CARGO_BUILD_JOBS: "4"
  script:
    - mkdir -p "$SCCACHE_DIR" && sccache --zero-stats || true
    - PROFILE=${DEV_RELEASE:+dev-release}; PROFILE=${PROFILE:-release}
    - |
      YEAR=$(date -d "$CI_COMMIT_TIMESTAMP" +%Y 2>/dev/null || date +%Y)
      MONTH=$(date -d "$CI_COMMIT_TIMESTAMP" +%m 2>/dev/null || date +%m)
      export FOXHUNT_BUILD_VERSION="${YEAR}.${MONTH}.${CI_PIPELINE_IID:-0}"
    - echo "Building ${SERVICE_PACKAGE} with --profile $PROFILE"
    - TARGET_DIR="target/$PROFILE"
    - cargo build --profile $PROFILE -p ${SERVICE_PACKAGE}
    - sccache --show-stats || true
    - mkdir -p build-out/services
    - cp $TARGET_DIR/${SERVICE_PACKAGE} build-out/services/
    - strip build-out/services/${SERVICE_PACKAGE}
    - ls -lh build-out/services/
  artifacts:
    paths:
      - build-out/services/
    expire_in: 1 day
```

**Step 2: Delete the old compile-services job (lines 411-459)**

**Step 3: Add 8 per-service compile jobs**

```yaml
compile-api-gateway:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: api-gateway
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/api_gateway/**
        - crates/common/**
        - crates/config/**
        - crates/trading_engine/**
        - bin/fxt/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-trading-service:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: trading-service
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/trading_service/**
        - services/api_gateway/**
        - crates/common/**
        - crates/config/**
        - crates/data/**
        - crates/database/**
        - crates/ml/**
        - crates/risk/**
        - crates/storage/**
        - crates/trading_engine/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-ml-training-service:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: ml-training-service
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/ml_training_service/**
        - crates/common/**
        - crates/config/**
        - crates/data/**
        - crates/ml/**
        - crates/risk/**
        - crates/storage/**
        - crates/trading_engine/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-backtesting-service:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: backtesting-service
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/backtesting_service/**
        - crates/common/**
        - crates/config/**
        - crates/data/**
        - crates/ml/**
        - crates/model_loader/**
        - crates/risk/**
        - crates/storage/**
        - crates/trading_engine/**
        - bin/fxt/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-trading-agent-service:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: trading-agent-service
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/trading_agent_service/**
        - crates/common/**
        - crates/config/**
        - crates/ml/**
        - crates/risk/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-broker-gateway:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: broker-gateway
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/broker_gateway_service/**
        - crates/common/**
        - crates/config/**
        - crates/trading_engine/**
        - vendor/ctrader-openapi/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-data-acquisition-service:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: data-acquisition-service
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - services/data_acquisition_service/**
        - crates/common/**
        - crates/config/**
        - crates/storage/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"

compile-web-gateway:
  extends: .compile-service-base
  variables:
    SERVICE_PACKAGE: web-gateway
  rules:
    - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
      changes:
        - crates/web-gateway/**
        - crates/common/**
        - Cargo.toml
        - Cargo.lock
    - if: $CI_PIPELINE_SOURCE == "web" || $CI_PIPELINE_SOURCE == "api"
```

**Step 4: Verify YAML syntax**

Run: `python3 -c "import yaml; yaml.safe_load(open('.gitlab-ci.yml'))"`

**Step 5: Commit**

```bash
git commit -m "ci: replace compile-services with 8 per-service compile jobs"
```

---

### Task 10: Update deploy job for selective restart

**Files:**
- Modify: `.gitlab-ci.yml` (deploy job ~lines 1462-1639)

**Step 1: Update `needs:` to reference per-service jobs**

Replace the single `compile-services` need with 8 optional per-service needs:

```yaml
needs:
  - job: compile-api-gateway
    optional: true
    artifacts: true
  - job: compile-trading-service
    optional: true
    artifacts: true
  - job: compile-ml-training-service
    optional: true
    artifacts: true
  - job: compile-backtesting-service
    optional: true
    artifacts: true
  - job: compile-trading-agent-service
    optional: true
    artifacts: true
  - job: compile-broker-gateway
    optional: true
    artifacts: true
  - job: compile-data-acquisition-service
    optional: true
    artifacts: true
  - job: compile-web-gateway
    optional: true
    artifacts: true
  - job: compile-training
    optional: true
    artifacts: true
  - job: build-web-dashboard
    optional: true
    artifacts: true
```

**Step 2: Replace hardcoded rollout restart loop with selective restart**

Replace the `for svc in trading-service api-gateway ...` loops with:

```bash
# Restart only services whose binary was uploaded
DEPLOYED=""
for bin in build-out/services/*; do
  [ -f "$bin" ] || continue
  name=$(basename "$bin")
  DEPLOYED="$DEPLOYED $name"
done

if [ -n "$DEPLOYED" ]; then
  for svc in $DEPLOYED; do
    kubectl -n foxhunt rollout restart deployment/$svc
    echo "Restarted $svc"
  done
  for svc in $DEPLOYED; do
    kubectl -n foxhunt rollout status deployment/$svc --timeout=300s
  done
  echo "Deployed:$DEPLOYED"
else
  echo "No service binaries changed — manifests-only deploy"
fi
```

**Step 3: Verify YAML syntax**

Run: `python3 -c "import yaml; yaml.safe_load(open('.gitlab-ci.yml'))"`

**Step 4: Commit**

```bash
git commit -m "ci: make deploy job selective — only restart services with new binaries"
```

---

### Task 11: Full workspace compilation check

**Step 1: Run full workspace check**

Run: `SQLX_OFFLINE=true cargo check --workspace 2>&1 | tail -20`
Expected: all 37+ crates compile successfully

**Step 2: Run full workspace tests (lib only, fast)**

Run: `SQLX_OFFLINE=true cargo test --workspace --lib 2>&1 | tail -5`
Expected: all tests pass

**Step 3: Verify no warnings from the renames**

Run: `SQLX_OFFLINE=true cargo check --workspace 2>&1 | grep -i "warning.*rename\|error" | head -10`
Expected: no rename-related warnings or errors

**Step 4: Commit any fixups if needed**

---

### Task 12: Update GPU overlay manifests if they reference old binary names

**Files:**
- Check: `infra/k8s/gpu-overlays/trading-service-gpu.yaml`
- Check: `infra/k8s/gpu-overlays/ml-training-service-gpu.yaml`

**Step 1: Check for old binary name references**

Search for `trading_service`, `ml_training_service` in GPU overlay YAMLs.
These are strategic merge patches — they may reference container names or commands.

**Step 2: Update any found references to kebab-case**

**Step 3: Commit if changes were needed**

```bash
git commit -m "infra: update GPU overlay manifests for kebab-case binary names"
```