Architecture¶
This page is the production architecture in one place. Everything else in the docs is a deeper dive into one of the boxes below.
The big picture¶
Tomoda runs on a single k3s cluster on one OVH bare-metal server in Beauharnois, Quebec. API and app traffic resolves through Cloudflare to Traefik on the server's public IP; media is served by Cloudflare Workers from R2. Data services (Postgres, Redis, Photon, NATS) all live inside the cluster on local NVMe. Argo CD reconciles the entire desired state from this git repository.
tomoda-labs/devops.Component-by-component¶
Edge and DNS¶
Cloudflare hosts the tomoda.life zone. The app hostnames are proxied in both envs (WAF, DDoS, edge TLS) and prod is origin-locked with authenticated origin pulls; admin UIs are DNS-only. external-dns in the cluster writes the app records from Ingress resources (k8s/envs/platform/external-dns/values.yaml). See Cloudflare.
Media CDN and object storage¶
User media lives in private R2 buckets (tomoda-assets-{dev,prod}). A Worker per env (assets_worker.tf, source assets-worker/) serves assets[-dev].tomoda.life, verifies the backend's HMAC-signed URLs and caches at the edge. Clients upload directly to R2 with presigned PUTs. Backups, logs, traces, OTA bundles, the Photon index and seed sets are also in R2 (infrastructure/cloudflare/r2.tf).
Ingress¶
The Traefik LoadBalancer Service is bound to the node's public IP by k3s ServiceLB (k8s/envs/platform/traefik/values.yaml). TLS terminates at Traefik with a letsencrypt-prod cert-manager ClusterIssuer, and Traefik routes by host header (k8s/apps/tomoda/base/ingress.yaml and the prod overlay). A Cloudflare Tunnel (tunnel.tf) carries only the loki-push endpoint for the synthetic Worker.
Application Pods¶
The backend and frontend Deployments live in k8s/apps/tomoda/base/. The dev overlay deploys into tomoda-dev; the prod overlay deploys into tomoda-prod and adds a PodDisruptionBudget and HPAs. Images come from GHCR (ghcr.io/tomoda-labs/*), pulled with node-level credentials.
Data plane (in-cluster)¶
All stateful services run in the cluster on local-path volumes (RAID1 NVMe):
- Postgres is a CloudNativePG
Clusteron a custom image (PostGIS + PGroonga + pgvector). Seek8s/envs/prod/postgres/manifests/cluster.yaml. WAL archiving and daily base backups go to R2 through the Barman Cloud plugin. - Redis is the Bitnami chart in standalone mode. See
k8s/envs/prod/redis/values.yaml. - Photon serves geocoding from the planet index it downloads from
photon-index.tomoda.life(R2) on first start and on index updates. - NATS carries WebSocket fan-out between API replicas.
Secrets¶
Application secrets live in GCP Secret Manager. External Secrets Operator authenticates with a scoped service-account key and materializes them as native Secret objects. See Secrets management.
GitOps control plane¶
Argo CD is installed once by scripts/ovh/bootstrap-argocd.sh and then manages its own release (Helm chart 10.12.1, self-management) and exposed at argo-app.tomoda.life, with Dex + Google SSO restricted to the tomoda.life domain. k8s/envs/bootstrap.yaml seeds the app-of-apps that pulls in every other workload.
Why everything in-cluster
Postgres, Redis, and Photon could all be managed services. We chose in-cluster operators for cost, portability, and a single source of truth. The trade-offs are spelled out on the Decisions page.
Request paths in one sentence¶
- Media: User → Cloudflare → assets Worker → R2.
- API call: User → Cloudflare → Traefik (node IP) → backend Pod → CNPG / Redis / Photon (all in-cluster).
- Geocode: backend Pod → in-cluster Photon Service, which loaded its index from R2.