Skip to content

Deploy

How a code change reaches dev and prod. The pipeline is fully GitOps: nothing is kubectl apply'd by hand in steady state.

High-level flow

git push
→
Cloud Build
push image
Artifact Registry
↓ Image Updater polls for new tags
Image Updater
commit manifest
Argo CD
reconcile
GKE rolling swap
Five actors, two halves. The top half (push → build → registry) is the producer side: a single git push triggers a build and lands the image. The bottom half (poll → manifest commit → sync → roll) is the GitOps consumer side: Image Updater notices the new tag, writes it into the manifest, and Argo CD swings the cluster onto it.

Full release process

Backend/web and the mobile app version independently. A merge auto-deploys to dev; prod is gated by a version tag; the mobile app ships most changes over-the-air.

merge to main (app repo)
↓ auto
dev (automatic)
Cloud Build (no approval)
↓ image :sha
Image Updater → Argo CD
↓
GKE tomoda-dev
prod (tag = gate)
tag v*
↓ auto
Cloud Build → Image Updater → Argo
↓
GKE tomoda-prod
mobile app
JS-only merge → OTA dev
↓ promote
OTA prod (Xavia)
↓ native change only
local build → TestFlight / Play
Three lanes. Backend/web: merge → dev auto, one v* tag → prod (together). Mobile: JS ships over-the-air (dev channel on merge, prod on promote); native binaries are built locally only when native code changes.

Dev pipeline (backend / web)

  1. Merge to main in the tomoda app repo.
  2. Cloud Build trigger fires and builds automatically — no approval gate. It runs tests and pushes the image to Artifact Registry tomoda-dev-repo tagged with the commit SHA.
  3. Argo CD Image Updater polls Artifact Registry; on a new SHA it edits the Kustomize image override in this devops repo and pushes a commit.
  4. Argo CD syncs the tomoda application and rolls the backend/web Deployment in tomoda-dev. The new pod passes readiness before the old one terminates.

Prod pipeline (backend / web)

Prod is gated by a version tag, and backend + web ship together on it:

  1. Cut a release tag vX.Y.Z — it ships backend + web together: git tag v1.2.3 && git push --tags.
  2. Cloud Build prod trigger fires automatically on the matching tag — the tag is the gate.
  3. Image is pushed to tomoda-prod-repo tagged with the version; Image Updater commits to the prod overlay; Argo CD syncs the tomoda-prod application.

Mobile app (native + OTA)

Native binaries are built locally and rarely (only on a native/SDK change); most changes ship over-the-air through the self-hosted Xavia OTA server. A JS-only merge publishes to the dev channel (internal testers); a vetted bundle is promoted to prod. An update reaches only binaries with a matching runtimeVersion, so a native change means a new local binary. See the app repo's docs/frontend/native/release.md.

Health checks and disruption

Every backend pod has readinessProbe and livenessProbe configured against /health. Argo CD waits for the new pod's readiness probe to pass before the rolling update proceeds.

Prod enforces a PodDisruptionBudget with minAvailable: 1. During a rolling update:

  • Argo CD spins up the new replica
  • Waits for readiness
  • Terminates the old replica

Since prod currently runs a single replica (see Scaling to grow it), the PDB means the old pod stays up until the new one is healthy — preventing a brief 503 window mid-deploy.

PDB with single replica

A minAvailable: 1 PDB with replicas: 1 blocks voluntary disruptions (node drains, autoscaler evictions). Before any planned node maintenance, scale the backend to 2 replicas first.

What you commit, what gets generated

Repo What you commit What the pipeline writes
tomoda (app) Code changes Cloud Build builds image
devops (this repo) Manifest changes, infra Image Updater writes Kustomize image SHA

If you change a manifest directly in devops/k8s/apps/tomoda/overlays/dev/, Argo CD syncs it on the next poll (default: 3 min). Force a sync with argocd app sync tomoda if you need it immediately.

Verifying a deploy

# Watch the rollout (both pools share the same image; check whichever rolled)
kubectl rollout status deployment/tomoda-api -n prod
kubectl rollout status deployment/tomoda-async -n prod

# Confirm the running image SHA
kubectl get deploy tomoda-api -n prod -o jsonpath='{.spec.template.spec.containers[0].image}'
kubectl get deploy tomoda-async -n prod -o jsonpath='{.spec.template.spec.containers[0].image}'

# Hit the health endpoint (served by tomoda-api via the ingress)
curl https://api.tomoda.life/health

If anything looks wrong, see Rollback.