Skip to content

Photon

Photon is the self-hosted geocoder that backs Tomoda's place-search autocomplete. It runs as a single in-cluster Deployment in the platform namespace, serves the backend at http://photon.platform.svc.cluster.local:2322, and pulls its index from the R2 bucket tomoda-photon-index, built off-cluster by the Photon indexing pipeline.

The Argo CD Application at k8s/envs/platform/photon/application.yaml points at k8s/envs/platform/photon/manifests.yaml and syncs into namespace: platform.

Image and replication

image: rtuszik/photon-docker:2.2.0

The tag is pinned: Photon's index file format is version-sensitive, so a mismatch between the running pod and the bucket's index fails to load. When bumping, also bump PHOTON_RUNTIME_IMAGE in k8s/envs/platform/photon-indexer/Dockerfile and the tag in the app repo's docker-compose.dev.yml. rtuszik/photon-docker:2.2.0 ships Photon 1.1.0.

Deployment is single-replica with strategy.type: Recreate: the new pod comes up only after the old one is gone, since both mount the same ReadWriteOnce PVC. Multi-replica scale-out needs a per-pod index copy or RWX storage; see Scaling.

There are no node selectors; on one server it runs alongside everything else. The index is built off-cluster (scripts/photon-index-local.sh on a rented build machine), never in the cluster.

Storage

A single PersistentVolumeClaim named photon-data-pvc requests 250 Gi on the local-path StorageClass (a directory on the server's RAID1 NVMe; the size isn't enforced):

accessModes: [ReadWriteOnce]
storageClassName: local-path
resources:
  requests:
    storage: 250Gi

Sized for the index (~145 GB) plus the downloaded tarball (~83 GB) during an update. The server can't hold two extracted planet indexes, hence UPDATE_STRATEGY=SEQUENTIAL below.

Resources and probes

requests: { memory: 8Gi, cpu: 250m }
limits:   { memory: 12Gi }

JAVA_OPTS=-Xmx6g caps the heap inside the 12Gi memory limit; the rest is page cache for the index. No CPU limit: index extraction (bzip2) is bursty and CPU-heavy, and a limit would throttle it. The cold-start path (download, bzip2 extract, load from disk, warm Lucene) is slow, so probes are tuned long:

Probe Path Initial delay Period Notes
Startup /status 900s 30s 480 failures allowed (about 4.25h total) to cover a first-time index download + extract
Readiness /status 120s 30s Pod stays out of the Service until Photon answers
Liveness /status 300s 60s failureThreshold: 10 to tolerate transient stalls

Scaling

Photon is a read-only geocoder, memory- and CPU-bound per query, so it scales vertically first: raise the CPU/memory requests and the -Xmx heap. A single replica serves a large query volume.

Horizontal scale-out is blocked by the ReadWriteOnce PVC: only one pod mounts the index at a time, so raising replicas leaves the second pod Pending. The index is read-only once loaded, so the horizontal path is:

  1. Add servers and run one Photon per server with its own local-path copy (a StatefulSet with pod anti-affinity). R2 egress is free, so each copy downloading the index costs nothing but time.
  2. Put the replicas behind the existing ClusterIP Service, with a CPU HPA if needed.

Each copy needs ~230 GB of free disk on its server.

Index updates

Photon auto-updates from the R2 bucket's public domain:

env:
  - name: UPDATE_STRATEGY
    value: "SEQUENTIAL"   # disk can't hold two planet indexes
  - name: UPDATE_INTERVAL
    value: "24h"          # poll the MD5 once a day
  - name: FILE_URL
    value: "https://photon-index.tomoda.life/planet/photon-db-planet-multilang-latest.tar.bz2"
  - name: MD5_URL
    value: "https://photon-index.tomoda.life/planet/photon-db-planet-multilang-latest.tar.bz2.md5"

R2 egress is free, so re-downloads cost nothing. Photon polls the .md5 every 24h; on a change it stops serving, replaces the index and comes back (an hour or two for the planet). Uploading a new tarball is the only path that rolls Photon forward, so upload in a quiet window.

A pod restart does not re-download: the extracted index persists on photon-data-pvc, and Photon skips the download when the on-disk MD5 matches the remote. A full re-download happens only when the PVC has no complete index (first boot, or an interrupted extraction) or the remote index changes.

Multilingual index

The served index is the 28-language multilingual build: en, ja, ko, zh, zh-Hans, zh-Hant, ar, he, hi, vi, th, id, tr, es, fr, de, it, pt, nl, pl, ru, sv, no, da, fi, el, cs, uk. This mirrors services.SupportedLocalizationLanguages in the backend.

To rebuild and ship a new index, see the Photon indexing runbook.

Service

kind: Service
metadata:
  name: photon
  namespace: platform
spec:
  type: ClusterIP
  ports:
    - port: 2322
      targetPort: 2322

Backend pods reach Photon at http://photon.platform.svc.cluster.local:2322, wired into the backend as PHOTON_URL (k8s/apps/tomoda/base/backend-api-deployment.yaml, k8s/apps/tomoda/base/backend-async-deployment.yaml) and the target of the blackbox readiness probe (k8s/envs/platform/manifests/photon-probe.yaml). No Ingress; no public Photon endpoint.

Operations

  • New index: see Building the Photon multilang index. Upload to R2, update the *-latest aliases; the running pod picks it up within 24h.
  • Force an update: restart the pod; on boot Photon re-checks the MD5 and downloads if the local copy is stale.
  • Storage pressure: the PVC shares the server's NVMe; check df -h on the host. See the disk budget.
  • Version bump: coordinate the runtime image and the indexer image in the same PR; cross-version index loads will fail readiness.