From 2eb8cd83331ee96821901a93f8abc354bf9d62da Mon Sep 17 00:00:00 2001 From: jgrusewski Date: Sun, 21 Jun 2026 15:16:28 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20spec=20=E2=80=94=20migrate=20Terraform?= =?UTF-8?q?=20state=20GitLab=20->=20Scaleway=20Object=20Storage=20(Phase?= =?UTF-8?q?=202A)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 --- .../2026-06-21-tfstate-to-scaleway-design.md | 94 +++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-21-tfstate-to-scaleway-design.md diff --git a/docs/superpowers/specs/2026-06-21-tfstate-to-scaleway-design.md b/docs/superpowers/specs/2026-06-21-tfstate-to-scaleway-design.md new file mode 100644 index 000000000..28fc54e35 --- /dev/null +++ b/docs/superpowers/specs/2026-06-21-tfstate-to-scaleway-design.md @@ -0,0 +1,94 @@ +# Migrate Terraform State to Scaleway Object Storage — Design (Phase 2A) + +**Date:** 2026-06-21 +**Status:** Design (approved) +**Repo:** foxhunt (IaC current home; relocation to fxhnt is Phase 2D) +**Part of:** Phase 2 (GitLab → Gitea consolidation). Sub-projects: **2A state migration (this)**, 2B Gitea +stand-up + repo migration + Argo wiring, 2C GitLab decommission, 2D IaC relocation into fxhnt. + +## Motivation + +Terraform state for the whole production cluster currently lives in the **self-hosted GitLab** http +backend (project ID=1), whose object data sits in **in-cluster MinIO**. That is a chicken-and-egg +fragility: the state describing the cluster lives *inside* the cluster it manages. GitLab is also slated +for decommission (Phase 2C, replaced by lightweight Gitea). Moving state to **external Scaleway Object +Storage** decouples it from both GitLab and the cluster — a prerequisite for removing GitLab and a +durability win on its own (state survives cluster loss). + +## Decision (keystone) + +**Target backend: Scaleway Object Storage (native S3), bucket `foxhunt-tfstate`** (already exists, +fr-par, created ~3 months ago but never wired). Reuse it (no rename — avoids a needless create; the +`foxhunt-` prefix is cosmetic and harmless). Locking via **OpenTofu native S3 lockfile** (`use_lockfile`, +OpenTofu v1.11.5 present) — no DynamoDB. + +## Scope — what changes + +1. **`infra/live/root.hcl`** — replace the single `remote_state` block: `backend = "http"` (GitLab) → + `backend = "s3"` (Scaleway). New config: + ```hcl + remote_state { + backend = "s3" + generate = { path = "backend.tf", if_exists = "overwrite" } + config = { + bucket = "foxhunt-tfstate" + key = "${path_relative_to_include()}/terraform.tfstate" + region = "fr-par" + endpoints = { s3 = "https://s3.fr-par.scw.cloud" } + skip_credentials_validation = true + skip_region_validation = true + skip_requesting_account_id = true + skip_metadata_api_check = true + use_lockfile = true + } + } + ``` + Credentials: Scaleway access/secret keys exported as `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` + (the s3 backend reads AWS-style env vars) from the existing `scaleway-credentials` k8s secret. +2. **State migration, per module, sequential** (`kapsule`, `public-gateway`, `dns`, `block-storage`): + - Back up first: `terragrunt state pull > /tmp/tfstate-backup-.tfstate` (from GitLab, BEFORE + changing the backend for that module). + - `terragrunt init -migrate-state -force-copy` → copies GitLab state → Scaleway S3. + - `terragrunt plan` → **must report `No changes`** before proceeding to the next module. +3. **Lock smoke test** (once, on one module): confirm `use_lockfile` works against Scaleway — a lock + object is created during apply/plan-with-lock and released after (verify no orphan `.tflock` left). + +## Migration order & gates + +`kapsule` → `public-gateway` → `dns` → `block-storage`. After EACH: backup taken, `-migrate-state` +succeeded, `plan = No changes` confirmed. **STOP and revert if any module's post-migration plan shows +drift** (revert = restore `http` backend in root.hcl + `terragrunt init` to re-attach GitLab state; the +GitLab copy is never deleted in 2A). + +## Safety / reversibility + +- **Old GitLab state is NOT deleted in 2A** — it stays as a live fallback until GitLab is removed in 2C. + So the entire migration is reversible by flipping `root.hcl` back to the http backend. +- `/tmp` state backups taken before each module (belt-and-suspenders; never committed — contains + resource IDs, not secrets, but treat as sensitive). +- One module at a time with a `plan = No changes` gate prevents a bad backend config from cascading. +- `endpoints.s3` (not deprecated top-level `endpoint`) + the four `skip_*` flags are required for the + AWS s3 backend to talk to Scaleway's S3-compatible API without AWS-specific preflight calls. + +## Risks + +- **Scaleway S3 quirks:** some provider/backend versions need `skip_s3_checksum = true` for Scaleway. If + `-migrate-state` errors on checksum, add it. (Documented as a known fallback in the plan.) +- **Lock semantics:** Scaleway Object Storage must honor conditional-write for `use_lockfile`. The lock + smoke test catches this; fallback is to run with locking disabled (single maintainer) if unsupported. +- **Credential confusion:** the s3 backend reads `AWS_*` env vars, NOT `SCW_*`. Plan must export both + (SCW_* for the provider, AWS_* for the backend) — a common foot-gun. + +## Out of scope (later sub-projects) + +Gitea stand-up + repo migration + Argo wiring (2B); GitLab helm removal + GitLab state/bucket cleanup +(2C); `git mv` of IaC from foxhunt → fxhnt + path updates (2D). Files stay in foxhunt for 2A; only the +backend changes. + +## Acceptance criteria + +- `infra/live/root.hcl` uses the Scaleway s3 backend; committed. +- All 4 modules migrated: each shows `terragrunt plan = No changes` against the new backend. +- State objects present in `foxhunt-tfstate` at `/terraform.tfstate` (verified via `scw`/`mc`). +- Lock smoke test passes (lock acquired + released, no orphan lockfile). +- GitLab http state still intact (untouched fallback) — not yet deleted.