Loading the location seed set¶
How to publish a generated location seed set to R2 and load it into a target Postgres on demand. The seed set is the location backbone: countries, the admin-area hierarchy, the city gazetteer, curated collections (UNESCO, Michelin, 50 Best) and the notable-places and natural-places sets. Loading it populates locations + related tables so discovery and search have data to resolve against.
The pipeline has four stages, three of which live here in devops:
app catalogseeder
seed-publish.sh
r2:tomoda-location-seed/seed/<version>
on-demand, per env
tomoda-{dev,prod}
Prerequisites¶
- Seed bucket exists.
tomoda-location-seedininfrastructure/cloudflare/r2.tf. The load Job reads it withtomoda-r2-cluster-credentials, projected as ther2-seed-credentialsSecret (an rclone remote) by the ExternalSecret that ships with the Job. - Publisher access.
seed-publish.shneeds an rclone remote namedr2with write access to the bucket. - Async workers running. The post-load passes run on the app's async worker pods, not in the Job. Check
kubectl -n tomoda-<env> get deploy tomoda-asyncbefore loading. - A generated seed set. Produced by the app's
catalogseeder(app repo). The output directory holds NDJSON collections plus a checksummedmanifest.json. - The
tomoda-backendimage. The load Job runs the loader binary shipped inside it at/app/catalogseeder(validatethenload). See Image below.
1. Generate (app repo)¶
Run the app's catalogseeder to produce the seed set into a local directory. See the app repo's catalog-seeding docs for source selection and tuning. The output must contain manifest.json. The seed set is organized as:
seed/<version>/global/... planet-wide backbone (countries, cities, curated)
seed/<version>/local/... region-scoped supplements
seed/<version>/manifest.json
<version> is an operator-chosen tag, normally the generation date. It names an immutable prefix in the bucket. The current version is 2026-11-09.
Bucket layout¶
| Prefix | Holds | Loaded by the Job |
|---|---|---|
seed/<version>/ |
A place set: manifest.json (format version 2), NDJSON collections, place-photos/, curated-photos.ndjson |
Yes, with the seed version knob |
geo-covers/<set>/ |
A cover set: geo-covers.ndjson plus place-photos/geo/<kind>/<key>.jpg |
Yes, with the geo-cover set knob |
simulation/<version>/ |
The small bundle behind local SIM=1 runs |
No |
seed-archive/ |
Retired content kept for reference, such as the Overture density layer moved out of seed/* and simulation/* |
Never |
Versions and sets are immutable: a change is a new directory. The archive and cover locks are in Cloudflare.
2. Publish¶
scripts/seed-publish.sh validates the local directory and uploads it under seed/<version>/:
./scripts/seed-publish.sh --version 2026-11-09 --dir ./seed-out
It refuses to publish when:
manifest.jsonis missing from the directory.catalogseeder validatefails (only when the binary is on PATH; skipped with a warning otherwise).seed/<version>/already exists, unless you pass--force. Prefer a fresh--versionover--forceso published versions stay immutable.
The bucket defaults to tomoda-location-seed; override with --bucket. The script has no knowledge of collection names, so it works unchanged with manifest format version 2: the validate gate rejects older manifests, checks every file's sha256 and row count, and makes no network calls.
Republish an existing version¶
When the data is good but the manifest is outdated (no format version, retired entries), build a new version without regenerating. Data files are copied inside the bucket and only the manifest is rewritten, so per-file checksums are kept. The sequence is rclone copy --server-side-across-configs of the old version minus its manifest (and any retired file), then catalogseeder republish --from <old manifest> --out <new manifest>, then upload the new manifest. Rows whose primary provider is retired need --repair-from <local copy of the version> --repair-out <empty dir>; upload the files written to --repair-out plus the manifest over the copied version. Run validate on the pulled result before the first load.
The exact commands and the repair rules are in the app's Catalog seeding page.
3. Load (on demand, dev first)¶
The load is a Kubernetes Job, not a CronJob. It is never scheduled and never auto-applied: the manifests under k8s/apps/tomoda/location-seed-load/ are a template that is not referenced by any Argo CD Application, so Argo leaves them alone. An operator renders and applies a run by hand.
The Job's initContainer (rclone) pulls r2:tomoda-location-seed/seed/<version> into an emptyDir; the main container runs /app/catalogseeder validate --from /seed then /app/catalogseeder load --from /seed --city-highlights <N> against the env's Postgres, with DB creds from that env's backend-secrets-<env> + backend-config-<env>.
Three knobs per run:
| Knob | Placeholder | Meaning |
|---|---|---|
| Seed version | REPLACE_ME_SEED_VERSION |
seed/<version> to load. Leave empty for a cover-only run |
| City highlights | REPLACE_ME_CITY_HIGHLIGHTS |
Top-N attractions eager-promoted per city; 0 skips the pass |
| Geo-cover set | REPLACE_ME_GEO_COVER_SET |
Empty for a seed load. A set name makes the run cover-only and ignores the other two |
City highlights guidance:
- Use
0for a first load you want to verify, and a small N (the examples use25) once the load looks right. - The pass runs after city linking and is idempotent, so a re-run promotes nothing twice.
- Each promoted place queues enrichment, so a large N means a long tail of enrichment work on the
lowqueue.
Render, set the knobs, apply¶
Build the dev overlay, substitute the version + N, and apply with a unique name (timestamp keeps successive runs distinct):
TS=$(date -u +%Y%m%d-%H%M%S)
SEED_VERSION=2026-11-09
CITY_HIGHLIGHTS=25
kubectl kustomize k8s/apps/tomoda/location-seed-load/overlays/dev \
| sed \
-e "s/REPLACE_ME_SEED_VERSION/${SEED_VERSION}/" \
-e "s/REPLACE_ME_CITY_HIGHLIGHTS/${CITY_HIGHLIGHTS}/" \
-e "s/REPLACE_ME_GEO_COVER_SET//" \
-e "s/name: location-seed-load$/name: location-seed-load-${TS}/" \
| kubectl apply -f -
This is the one sanctioned direct kubectl apply
The GitOps rule forbids kubectl apply for Argo-managed resources. This Job is deliberately outside Argo, so applying it by hand is the intended trigger, not a bypass. The manifests carry no schedule and no Argo Application.
Watch it:
kubectl -n tomoda-dev get job -l app=location-seed-load
kubectl -n tomoda-dev logs -f job/location-seed-load-${TS}
Geo-cover-only load¶
To load a cover set without a seed version, set the geo-cover set and leave the other two empty. The initContainer pulls geo-covers/<set> into /seed and the Job runs /app/catalogseeder load --geo-covers-only --from /seed:
TS=$(date -u +%Y%m%d-%H%M%S)
GEO_COVER_SET=<set>
kubectl kustomize k8s/apps/tomoda/location-seed-load/overlays/dev \
| sed \
-e "s/REPLACE_ME_SEED_VERSION//" \
-e "s/REPLACE_ME_CITY_HIGHLIGHTS//" \
-e "s/REPLACE_ME_GEO_COVER_SET/${GEO_COVER_SET}/" \
-e "s/name: location-seed-load$/name: location-seed-load-${TS}/" \
| kubectl apply -f -
The Job needs the object-storage credentials already mounted for the seed load, and a second run of the same set reports a skip. Pruning covers a set no longer names is a local run, described in the app's catalog seeding docs.
Post-load passes¶
The Job does not run the post-load passes (city linking, highlights, embedding, enrichment). It enqueues the first step on the app's low queue and exits, and each step enqueues the next. The async worker pods (tomoda-async) must be running with Redis reachable, or the chain stalls after the Job reports success. Watch the worker logs and the low queue depth rather than the Job.
The Job cleans itself up 48h after finishing (ttlSecondsAfterFinished). backoffLimit: 0 means a failed load is inspected and re-run by hand rather than retried blindly.
4. Load prod¶
Repeat step 3 against the prod overlay once the dev load looks right. Prod targets tomoda_prod via postgres-prod and pins an immutable image tag rather than :latest:
TS=$(date -u +%Y%m%d-%H%M%S)
kubectl kustomize k8s/apps/tomoda/location-seed-load/overlays/prod \
| sed \
-e "s/REPLACE_ME_SEED_VERSION/${SEED_VERSION}/" \
-e "s/REPLACE_ME_CITY_HIGHLIGHTS/${CITY_HIGHLIGHTS}/" \
-e "s/REPLACE_ME_GEO_COVER_SET//" \
-e "s/name: location-seed-load$/name: location-seed-load-${TS}/" \
| kubectl apply -f -
Verify after a load¶
- Job succeeded:
kubectl -n tomoda-<env> get job -l app=location-seed-loadshowsCOMPLETIONS 1/1. - Logs: the loader prints per-collection
inserted/skipped/upgradedcounts and, when--city-highlights > 0, apromoted=line. - Row counts (read-only): connect via pgAdmin or
psqland checkSELECT count(*) FROM locations;andSELECT count(*) FROM countries;moved as expected. The load is checksum-gated per collection (seed_load_state), so re-running the same version is a near no-op.
Image¶
The loader ships inside the tomoda-backend image as a second binary at /app/catalogseeder (no dedicated image; backend/data is go:embed'd, so no data mount). Each overlay overrides the base placeholder IMAGE_PLACEHOLDER_CATALOGSEEDER to ghcr.io/tomoda-labs/tomoda-backend (:dev in dev; in prod the release's vX.Y.Z, which the deploy(prod) PR bumps along with the app), and the Job command invokes /app/catalogseeder validate --from /seed then /app/catalogseeder load ….
Deletion protection¶
Every R2 bucket (seed sets, backups, observability, media, the Photon index) carries lifecycle { prevent_destroy = true } in infrastructure/cloudflare/r2.tf, and R2 refuses to delete a non-empty bucket. A terraform destroy or a resource removal that would delete one fails the plan; removing one is a deliberate two-step change.