Skip to content

Architecture

This page is the production architecture in one place. Everything else in the docs is a deeper dive into one of the boxes below.

The big picture

Tomoda runs on a single k3s cluster on one OVH bare-metal server in Beauharnois, Quebec. API and app traffic resolves through Cloudflare to Traefik on the server's public IP; media is served by Cloudflare Workers from R2. Data services (Postgres, Redis, Photon, NATS) all live inside the cluster on local NVMe. Argo CD reconciles the entire desired state from this git repository.

User
→
Cloudflare
↙   ↘
assets.tomoda.life
assets Worker → R2 Cloudflare edge
api / app / www.tomoda.life
Traefik → k3s pods OVH Beauharnois
Two request paths from one Cloudflare zone. Everything stateful lives in-cluster and backs up to R2. Argo CD reconciles desired state from tomoda-labs/devops.

Component-by-component

Edge and DNS

Cloudflare hosts the tomoda.life zone. The app hostnames are proxied in both envs (WAF, DDoS, edge TLS) and prod is origin-locked with authenticated origin pulls; admin UIs are DNS-only. external-dns in the cluster writes the app records from Ingress resources (k8s/envs/platform/external-dns/values.yaml). See Cloudflare.

Media CDN and object storage

User media lives in private R2 buckets (tomoda-assets-{dev,prod}). A Worker per env (assets_worker.tf, source assets-worker/) serves assets[-dev].tomoda.life, verifies the backend's HMAC-signed URLs and caches at the edge. Clients upload directly to R2 with presigned PUTs. Backups, logs, traces, OTA bundles, the Photon index and seed sets are also in R2 (infrastructure/cloudflare/r2.tf).

Ingress

The Traefik LoadBalancer Service is bound to the node's public IP by k3s ServiceLB (k8s/envs/platform/traefik/values.yaml). TLS terminates at Traefik with a letsencrypt-prod cert-manager ClusterIssuer, and Traefik routes by host header (k8s/apps/tomoda/base/ingress.yaml and the prod overlay). A Cloudflare Tunnel (tunnel.tf) carries only the loki-push endpoint for the synthetic Worker.

Application Pods

The backend and frontend Deployments live in k8s/apps/tomoda/base/. The dev overlay deploys into tomoda-dev; the prod overlay deploys into tomoda-prod and adds a PodDisruptionBudget and HPAs. Images come from GHCR (ghcr.io/tomoda-labs/*), pulled with node-level credentials.

Data plane (in-cluster)

All stateful services run in the cluster on local-path volumes (RAID1 NVMe):

  • Postgres is a CloudNativePG Cluster on a custom image (PostGIS + PGroonga + pgvector). See k8s/envs/prod/postgres/manifests/cluster.yaml. WAL archiving and daily base backups go to R2 through the Barman Cloud plugin.
  • Redis is the Bitnami chart in standalone mode. See k8s/envs/prod/redis/values.yaml.
  • Photon serves geocoding from the planet index it downloads from photon-index.tomoda.life (R2) on first start and on index updates.
  • NATS carries WebSocket fan-out between API replicas.

Secrets

Application secrets live in GCP Secret Manager. External Secrets Operator authenticates with a scoped service-account key and materializes them as native Secret objects. See Secrets management.

GitOps control plane

Argo CD is installed once by scripts/ovh/bootstrap-argocd.sh and then manages its own release (Helm chart 10.12.1, self-management) and exposed at argo-app.tomoda.life, with Dex + Google SSO restricted to the tomoda.life domain. k8s/envs/bootstrap.yaml seeds the app-of-apps that pulls in every other workload.

Why everything in-cluster

Postgres, Redis, and Photon could all be managed services. We chose in-cluster operators for cost, portability, and a single source of truth. The trade-offs are spelled out on the Decisions page.

Request paths in one sentence

  • Media: User → Cloudflare → assets Worker → R2.
  • API call: User → Cloudflare → Traefik (node IP) → backend Pod → CNPG / Redis / Photon (all in-cluster).
  • Geocode: backend Pod → in-cluster Photon Service, which loaded its index from R2.