deploy-t4-cs3-over-ceph

name

deploy-t4-cs3-over-ceph

description

Deploy (or clean-reinstall) Together T4 (gns control-plane + s3-proxy data-plane) and CS3 (s3-cache-proxy) on the transporter k3s cluster, backed by a Rook-Ceph RGW running on a separate cluster. Covers the two-cluster topology, the two data paths, helm-driven install with hand-created secrets (TLS from certs1, postgres, ECR pull secrets in us-west-2), wiping+reseeding gns postgres, and pointing everything at the live Ceph RGW. Use for "deploy t4 and cs3 over ceph", "reinstall gns/s3-proxy/cs3", "redeploy T4 on the transporter".

Deploy T4 + CS3 over Ceph

Topology (two separate clusters)

• Compute/T4 — <USER>@<TRANSPORTER_IP> (<TRANSPORTER_HOST>, +.74 worker), k3s. Namespaces: gns, s3-proxy, cs3, t4. KUBECONFIG=~/k3s.yaml (API also reachable from a laptop at https://<TRANSPORTER_IP>:6443). docker is podman; passwordless sudo. helm installed at /usr/local/bin/helm (v3).
• Storage/Ceph — <CEPH_USER>@<CEPH_IP> (<CEPH_HOST>), Rook-Ceph in ns rook-ceph (use sudo for kubectl). Object store my-store; RGW reachable cross-cluster via NodePort http://<CEPH_RGW_NODEPORT> (in-cluster name rook-ceph-rgw-my-store.rook-ceph.svc:80 does NOT resolve from the transporter).

Data paths

• client → s3-proxy (UMS auth) → GNS (bucket→site→store routing) → Ceph RGW. s3-proxy gets the store URL from GNS and the S3 creds from its s3-proxy-backends secret keyed by store name ceph.