Skip to content

Xavia OTA

Self-hosted Xavia OTA server: an expo-updates manifest host that ships JavaScript bundle updates to the mobile app over-the-air. Most merges are JS-only and reach installed apps through this server without a store round-trip.

Two instances

Xavia has no channel concept: an instance serves the latest bundle per runtimeVersion. To separate internal-tester updates from production, dev and prod run as two independent instances (separate buckets, Postgres and upload keys) rather than one instance split by channel.

Instance Host Published
xavia-ota-dev ota-dev.tomoda.life every app-repo merge (internal testers)
xavia-ota-prod ota.tomoda.life each release, once its deploy(prod) PR is merged (Deploy)

The app bakes which host it targets at build time and only accepts an update whose runtimeVersion matches (set in the app's app.config.js); that value bumps only on a native change, which requires a fresh binary. See the app repo's docs/frontend/native/release.md.

Shape

Each instance is a self-contained manifest set under k8s/envs/platform/xavia-ota-{dev,prod}/: its own application.yaml (Argo app, namespace platform) and manifests.yaml, discovered by the platform recurse.

Piece Where
Deployment + Service (port 3000) k8s/envs/platform/xavia-ota-{dev,prod}/manifests.yaml
Image ghcr.io/tomoda-labs/xavia-ota, built from upstream source in k8s/envs/platform/xavia-ota-image/
Dedicated Postgres (metadata) same file: single-instance postgres:16-alpine + 5 Gi PVC per instance
Bundle storage R2 buckets tomoda-ota-dev / tomoda-ota-prod (infrastructure/cloudflare/r2.tf), via the S3 API with keys from tomoda-r2-cluster-credentials
Secrets ExternalSecret per instance projects PRIVATE_KEY_BASE_64, ADMIN_PASSWORD, UPLOAD_KEY, POSTGRES_PASSWORD from GCP Secret Manager
Ingress ota-dev.tomoda.life / ota.tomoda.life: dashboard and admin APIs behind oauth2-proxy, device and publish paths public (see Access)
DB backups CronJob xavia-ota-db-backup-{dev,prod} (see Backups)

Access

Xavia's admin password is checked only in the browser: /api/login, /api/releases, /api/rollback and /api/tracking/* do no server-side auth. Each host is therefore split into two Ingresses:

Ingress Paths Gate
xavia-ota-{dev,prod}-public /api/manifest, /api/assets, /api/upload (Exact) none; bundles are code-signed, upload checks the upload key
xavia-ota-{dev,prod} everything else (dashboard, admin APIs) oauth2-proxy operator allow-list, then the Xavia admin password

Traefik ranks routers by rule length, so the Exact public paths win over the gated / prefix. policy/admin_ingress_auth.rego allows only those three Exact paths ungated on the OTA hosts; any other ungated path fails CI.

Backups

Each instance's Postgres is dumped daily (03:30 UTC, pg_dump -Fc) to s3://tomoda-db-backups/ota/{dev,prod}/ and pruned to 30 days, using the S3_* keys already in the instance's ExternalSecret. A failed run raises KubeJobFailed; no success in 26 h raises OTABackupStale. Schedule, retention and the restore procedure live in Disaster Recovery.

Secrets: shared vs per-env

Both instances share the code-signing key and the admin password; the upload key and DB password are per-env.

Secret Scope Why
tomoda-ota-signing-key shared One signing key → one public cert works in the app, whichever host it targets
tomoda-ota-admin-password shared Single dashboard login
tomoda-ota-{dev,prod}-upload-key per-env Dev-CI and prod-CI publish paths stay isolated
tomoda-ota-{dev,prod}-db-password per-env Each instance has its own Postgres

First-run setup

The secret containers are declared in infrastructure/gcp/secrets.tf; values are set by hand with gcloud secrets versions add:

  • Generate an RSA code-signing keypair with a long-lived self-signed cert. Store the private key in tomoda-ota-signing-key and copy the cert into the app repo at frontend/certs/ota-public.pem (the app validates OTA manifests against it). One cert covers both instances.
  • Generate the admin password, per-env upload keys and per-env DB passwords. Read an upload key when publishing (gcloud secrets versions access latest --secret=tomoda-ota-dev-upload-key). The DB passwords are internal-only.

Xavia uses BLOB_STORAGE_TYPE=s3 with S3_ENDPOINT (R2), S3_REGION=auto and S3_BUCKET_NAME per instance; the keys arrive in the instance's ExternalSecret. Publishing requires the instance's upload key as the uploadKey form field.

Image

Docker Hub's xaviaio/xavia-ota stops at 2.0.3, which has no S3 backend, so this repo builds upstream 2.0.4 from source as ghcr.io/tomoda-labs/xavia-ota (k8s/envs/platform/xavia-ota-image/, built by images.yml). The build applies one local patch to upstream 2.0.4: S3Storage.copyFile prefixes CopySource with the instance bucket, so dashboard rollback works on R2 and can't read other buckets. The S3 client needs no bucket-level permissions: it only gets, heads, lists, puts and copies objects.

To upgrade, follow k8s/envs/platform/xavia-ota-image/README.md; diff upstream's containers/database/schema/ between tags and carry any change into the xavia-ota-schema-{dev,prod} ConfigMaps, which the apply-schema init container applies on every rollout.