Skip to content

Scripts

The scripts/ directory holds four operator scripts. None of them are wired into automation — each is run manually by a human operator when the scenario applies.

Script Purpose When to run Prerequisites
scripts/disaster-recovery.sh Restore Postgres from Barman backups (PITR / latest / full rebuild) DR scenarios: data corruption, lost cluster, accidental drop gcloud auth login, kubectl access to the cluster, terraform CLI for full-rebuild mode
scripts/photon-index-local.sh Build a multilingual Photon index from OSM on a large VM and upload to GCS Monthly Photon refresh, or whenever you bump the language set; while the in-cluster CronJob is suspended Terraform applied (provides bucket + SA), Docker + Compose v2, gcloud auth application-default login, the photon-indexer:local image built locally; a 32-vCPU/256-GB-RAM VM with NVMe scratch for the planet build
scripts/setup-gcp-secrets.sh Idempotently create/update tomoda secrets in GCP Secret Manager First-time project setup, rotating credentials, adding new third-party API keys gcloud auth login, gcloud config set project development-485000
scripts/update-photon.sh Trigger a Photon OSM-data update — Docker Compose locally, or the K8s CronJob remotely Refreshing the older single-language Photon data dir; less relevant since the multilingual rollout Running Photon container (local mode), or kubectl access (--k8s mode)

disaster-recovery.sh

Three modes:

  • --mode latest — restore the database from the most recent base backup + WAL into a sibling cluster (<name>-recovered) so you can validate before cutting over.
  • --mode pitr --target-time <ISO8601> — same flow, but recover to a specific timestamp. Requires the WAL archive in GCS to cover that window.
  • --mode full-rebuild — destroy-and-recreate path. Runs terraform apply (recreating the cluster if missing), waits for Argo to sync the CNPG operator and the Cluster CR, then leaves you with a fresh cluster ready to be data-restored via the latest mode.

The script never cuts traffic over automatically — after the recovery cluster is healthy, you patch the ExternalName service to point at it and scale the backend back up by hand. See the printed "Next Steps" block at the end of the run.

photon-index-local.sh

Two-stage pipeline orchestrated by k8s/apps/photon-indexer/docker-compose.build.yml:

  1. mediagis/nominatim imports an OSM extract into a Postgres database. This is the slow stage (hours for regional extracts, days for planet).
  2. photon-indexer:local runs photon -nominatim-import against the warm Nominatim DB to produce the Lucene index.

The script then tarballs the result and uploads it to the <project>-photon-index GCS bucket. Regions are canada, asia, or planet. Defaults assume an n2-highmem-32 VM with 4×375 GB NVMe in RAID-0; tune JAVA_HEAP, POSTGRES_SHARED_BUFFERS, POSTGRES_MAINTENANCE_WORK_MEM, and OSM2PGSQL_CACHE downward for laptop builds.

Suspended CronJob

The in-cluster CronJob in k8s/apps/photon-indexer/cronjob.yaml is suspend: true and will stay that way until we provision a warm in-cluster Nominatim. Until then, this script is the official way to rebuild the index.

setup-gcp-secrets.sh

Smart upsert against GCP Secret Manager. Behavior matrix:

  • Env var set and matches the current latest version → no-op.
  • Env var set and differs → adds a new version, disables the previous one.
  • Env var unset and the secret has no value → auto-generates for JWT, encryption key, DB password, Redis password; skips manual secrets (Stripe, Google, LINE, Apple, Klipy, email).
  • Env var unset and the secret already has a value → preserved.
  • --rotate → forces regeneration of the auto-generated secrets only.

This is the canonical entry point for managing the secrets that ESO then materializes into the cluster.

update-photon.sh

Older Photon data-update script. The Docker Compose path is still functional for local dev; the K8s path manually triggers a CronJob named photon-data-update (which is unrelated to the multilingual photon-indexer CronJob).

Note

The script's wget call points at download1.graphhopper.com for per-country Photon archives. The multilingual rollout (scripts/photon-index-local.sh) supersedes this for production; update-photon.sh is kept for the local docker-compose dev environment.