Skip to content

Locations

Purpose

The location stack converts a user's tap-on-the-map (or text query) into a persisted, enriched Location row. Every caller that needs a place — moments, events, voyages, waypoints, canonical-list ingestion — goes through this service. No duplicate location logic anywhere else in the system.

Every persisted Location is backed by a real Photon or Google Places match. When neither provider knows a place, no row is created and the caller is expected to keep the coords on its own row (moments and events both support this via their embedded Spatial). There is no manual placeholder row.

Three layers compose the service:

  1. LocationService — the central manager for all place work. Owns the Photon-first provider policy, the dedup ladder, the Redis cache, the create path, and the sole calling surface for the enrichment transport (EnrichCapture).
  2. ProviderAdapter — common interface around external providers. Two implementations: photonAdapter (wraps photon.PhotonService) and googleAdapter (wraps places.PlacesService). Handlers never see the underlying services.
  3. Platform pieces — platform/photon (self-hosted Photon/OSM geocoder) and platform/places (Google Places, reached either through the cheap Serper transport or a direct billed call). The location service is the only consumer of both.

Code map — where things live

The domain is a single Go package (backend/internal/services/location/) plus per-aggregate sqlc packages. Methods on locationService must stay in that package; standalone concerns are split into their own files or subpackages.

Concern Files
Service orchestration location_service.go (the LocationService interface + locationService), location_search.go (the searchCore pipeline), location_area.go, location_address.go, location_dedup.go, location_embedding.go
Ranking ranking/ subpackage — composite scoring, offline eval + harvest, the durable label sink (RankingStore), and the shared name-match normalizer. location_ranking_log.go (observability) and location_ranking_weights.go (config hot-refresh) stay in the domain package because they are locationService methods
Write / resolve / enrich location_resolver.go, location_enricher.go, location_ingest.go, provider_adapter.go, place_signals.go
Localization localization.go (fallback chain + trimming)
Persistence location_store.go (locations + MergeInto), region_store.go (RegionStore → regiondb), country_store.go (CountryStore → countrydb), mappers.go
HTTP / wiring handler.go, routes.go, wire.go

Reference tables. regions (admin½/3) is read at runtime through region_store.go. countries is read through country_store.go (list + get). Both tables are populated by the catalog seeder (cmd/catalogseeder), not written at runtime.

Provider policy

Photon is primary. Google is only consulted in three explicit cases:

  1. Caller hint — ResolveRequest.Provider == "google_places" (the user picked a Google-sourced place from some other surface, or the caller already has a Google Place ID).
  2. Rich-detail enrichment after a Photon hit — once a row is persisted from a Photon match AND its tag set intersects the rich-detail buckets (the buckets flagged rich_detail: true in backend/data/place_taxonomy.json: food, drinks, coffee, tea, nightlife, entertainment, music, culture, fitness, wellness, shopping, accommodation, health, travel), the background enrichment worker calls Google NearbySearch and merges photos / hours into the row. The Photon (provider, provider_id) pair stays canonical; the Google Place ID lands in secondary_providers. Google's rating + rating count are intentionally NOT copied — Location.AverageRating / TotalRatings are the Tomoda-only user aggregate maintained by LocationService.RecordRating from the moment + check-in publish paths; mixing Google's signal in would silently desync over time.
  3. EscalateOnMiss resolves — ResolveRequest.Escalation lets a caller declare its resolve derives from an explicit place share (a Maps link, page structured data). On a Photon miss the service escalates to the Places API: the place is fetched cheaply through the Serper transport first (same Google Place plus rich signals, no billed call), and only if that yields nothing does it fall to a direct billed Google lookup. A shared Maps link names a place that definitionally exists, so the escalation almost always lands. The capture parser is the only EscalateOnMiss caller today; coords-only creates (events, moments) stay Photon-only and fall back to their own off-grid fields. The service owns this decision — callers never issue their own second Resolve.

A caller that has already harvested a place through the Places API (the capture pipeline via EnrichCapture) passes it in with EscalateExternal, which mints from the supplied POI on a Photon miss without any fresh provider call.

Read paths (Nearby, Search, Autocomplete) never call Google. Text search runs DB-first: PGroonga matches the multilingual search_text (see Search & ranking) and Photon fills in only when the local catalog returns fewer than the caller's limit.

The rich-detail bucket list is owned by the shared taxonomy service (Discovery → Taxonomy vocabulary) and read at runtime via TaxonomyService.RichDetailTags(). Adding rich_detail: true to a bucket in backend/data/place_taxonomy.json is the only thing needed to widen the enrichment escalation.

Every provider call increments location_provider_calls_total{provider,reason} (Prometheus). Watch in Grafana to verify the policy is being honoured in production.

Why this matters

Self-hosted Photon has zero marginal cost; Google Places is billed per request. The escalation policy keeps the Google bill bounded while preserving rich metadata where it actually matters (restaurants, hotels, etc.). Flagging a tag as rich_detail directly increases Google spend — surface it intentionally.

This aligns with the existing ADR Photon as Google Places fallback.

Place identity

The same place id concept wears different names at different boundaries. The canonical vocabulary is the (provider, provider_id) pair on models.Location:

Where Field / name Shape
models.Location (canonical) Provider + ProviderID photon + "{osm_type}:{osm_id}" (e.g. N:12345), or google_places + a Google Place ID (ChIJ…)
models.Location.SecondaryProviders jsonb array Cross-provider ids for the same row (e.g. the Google Place ID on a Photon-canonical row)
LocationCandidate (provider transport DTO) GooglePlaceID, OsmID + OsmType Wire-in names from the provider clients, mapped to ProviderID at create
Serper /maps (PlacePOI) PlaceID (+ CID, FID) PlaceID is a Google Place ID; CID/FID are Google Maps-internal ids kept on the link's ItemLinkSignals, not on Location
Photon response State Renamed to Spatial.Region at the create boundary (createFromPhoton)

Rules of thumb: everything inside LocationService speaks Provider/ProviderID; the other names exist only at serialization boundaries and are mapped exactly once, at create time. A Google id is never a column of its own — it is either the canonical ProviderID or an entry in secondary_providers.

Provenance — Serper-minted rows and the upgrade budget

Location.Provenance records how a row's data was sourced, independent of its provider identity:

The full value set is the LocationProvenance enum in backend/internal/models/location.go:

Provenance Meaning
provider_api Hydrated by the provider's own API (Photon reverse geo, Google Places details/nearby). EnrichmentAt stamped when complete.
serper_maps Minted by ResolveExternalPlace from a Serper /maps POI. Sparse (name/address/coords only) until the enricher fills it.
geonames_seed Batch-loaded from the GeoNames gazetteer (cities + administrative areas).
unesco_seed Minted from the UNESCO open-data portal (World Heritage). Curated, so it earns an embedding.
wikidata_seed Batch-loaded from Wikidata (the notable overlay, Michelin/50 Best backing). Curated, so it earns an embedding.
osm_seed Reserved for the batch OSM/Overpass ingester.
bulk_import The high-cardinality density backbone (millions of Overture POIs). The enricher never paid-enriches these; they get Photon gap-fill only. Stamped by the Overture density source; promoted to promoted on meaningful engagement.
promoted A formerly bulk_import backbone row a user meaningfully engaged with (see Demand-driven promotion). No longer bulk, so it earns paid enrichment + an embedding on demand.

isBulkImport(loc) (which gates paid enrichment off) checks bulk_import only; isCuratedProvenance(loc) (which qualifies a row for an embedding) checks unesco_seed, wikidata_seed, and promoted.

The one enricher (EnrichLocation) is two-tier and provider-agnostic. Tier 1, Photon gap-fill (forward-geocode missing coords, reverse-geocode localizations), runs for every row. Tier 2, paid commercial enrichment (Serper /maps, then the Google Places API fallback under the global daily budget enrich:upgrade:google in Redis), fires only when the row is rich-detail (tagsAreRichDetail) and not a bulk_import backbone row (isBulkImport) and not already commercially enriched (hasFreshGoogleFields). So a capture-minted serper_maps row is enriched by category, the density backbone is Photon-only, and localization gap-fill runs for all rows. The semantic embedding is gated separately (see Embedding gate): high-value + engaged rows earn a vector, the cold long tail stays lexical + spatial + category-searchable.

Local seeding is Serper-free by default

The Tier 2 paid step is gated by config.Location.BackgroundPaidEnrichment (env ENRICH_BACKGROUND_PAID, default true). task dev sets it false so seeding the catalog locally with a real Serper key does not drain the background sweep through thousands of paid calls. Tier 1 Photon gap-fill, embeddings, and region_id still run locally. Only the background worker respects the flag: the interactive resolve escalation (location_resolver.go) and the capture path (EnrichCapture) call the Serper enricher directly, so devs can still test them locally.

Service surface

LocationService

The interface lives in backend/internal/services/location/location_service.go. The high-value methods:

type LocationService interface {
    Nearby(ctx, lat, lng, radius float64, lang string) ([]LocationCandidate, error)
    // Search parses the query through the taxonomy vocabulary and runs the
    // resulting {category, tags, free-text} predicates against the DB plus a
    // Photon fallback. lang biases localized display only.
    Search(ctx, query string, lat, lng float64, lang string) ([]LocationCandidate, error)
    // SearchFiltered is the lower-level entry taking already-parsed filters.
    SearchFiltered(ctx, query string, bias *GeoBias, filters SearchFilters, lang string) ([]LocationCandidate, error)
    // kinds allowlists which LocationKind rows come back: "poi" (default when
    // empty), "settlement" (cities/towns/suburbs), "address", "residential".
    // Plan + event pickers pass ["poi","settlement"]; the map picker keeps cities.
    Autocomplete(ctx, query string, p GeoPoint, limit int, lang string, kinds []LocationKind) ([]LocationCandidate, error)

    // Resolve returns the canonical Location for a place: existing row, or a
    // new one created from the first provider (Photon-first, Google when
    // hinted) that knows it. Returns (nil, nil) when no provider does — the
    // caller keeps coords on its own row; the /resolve endpoint returns 404.
    Resolve(ctx, req ResolveRequest) (*models.Location, error)
    // ResolveExternalPlace find-or-creates from a fully-described place (a
    // Serper /maps POI) with no provider details round-trip.
    ResolveExternalPlace(ctx, p ExternalPlace) (*models.Location, error)
    // MatchKnown resolves free-form text to a Location already stored, by
    // substring name match — the free fast-path before any provider call.
    MatchKnown(ctx, text string) (*models.Location, bool)

    Get(ctx, id uuid.UUID) (*models.Location, error)
    GetMany(ctx, ids []uuid.UUID) (map[uuid.UUID]*models.Location, error)
    ReverseGeocodeCity(ctx, lat, lng float64, lang string) (string, error)
    ReportOutdated(ctx, locationID uuid.UUID) error
    Merge(ctx, source, target uuid.UUID) (*models.Location, error)
    RecordRating(ctx, locationID uuid.UUID, rating float64) error
}

type ResolveRequest struct {
    Name       string
    Lat, Lng   float64
    Provider   ProviderName // "" lets the service pick (Photon-first)
    ProviderID string
    Escalation EscalationPolicy // opt Google retry on a Photon miss (Maps shares)
    Hint       *LocationHint    // optional: category, address, country code
}

Search is the taxonomy-aware entry the search endpoints call; SearchFiltered takes a pre-composed SearchFilters (categories, tags, city, country) for callers that parsed the query themselves.

ProviderAdapter

type ProviderAdapter interface {
    Name() ProviderName
    SearchText(ctx, query string, bias *GeoBias, lang string) ([]LocationCandidate, error)
    DetailsByID(ctx, externalID string) (*LocationCandidate, error)
    ReverseGeo(ctx, lat, lng float64, lang string) ([]LocationCandidate, error)
}

NewPhotonAdapter(svc photon.PhotonService) and NewGoogleAdapter(svc places.PlacesService) wrap the platform-piece clients. The Photon adapter's DetailsByID returns (nil, nil) because Photon doesn't have a fetch-by-id endpoint — the caller falls through to the spatial path.

HTTP endpoints

Method Path Auth Description
GET /api/v1/locations/nearby JWT DB + Photon nearby (lat, lng, optional radius, lang)
GET /api/v1/locations/nearby/autocomplete JWT Typeahead over the unified search core (q, optional lat, lng, limit, lang, kinds). Omitting coords gives an unbiased/area-driven search; kinds=poi,settlement keeps settlements (default is POI-only)
GET /api/v1/locations/reverse JWT Reverse-geocode coords to a city via Photon (lat, lng, lang)
GET /api/v1/locations/countries JWT The country reference list, prominence-ordered, each name localized to lang
GET /api/v1/locations/countries/{code} JWT One country by ISO 3166-1 alpha-2 code, name localized to lang; 404 when unknown
POST /api/v1/locations/resolve JWT Resolve a LocationCandidate into a persisted Location; returns 404 when neither Photon nor Google knows the place. The one resolve endpoint (pickers call it with name + coords, callers with a full candidate pass provider ids for an exact match). Frontends call this before submitting a moment/event so the payload carries a concrete location_id
POST /api/v1/locations/{id}/report-outdated JWT User reports that a location's info is stale. Always returns 200 (the response intentionally hides whether a background refresh actually fires, suppressed by the 7-day EnrichmentAt guard). Logged as location_outdated_reported

The main text-search bar and place pickers call GET /api/v1/discovery/search/locations (see Discovery), which is a thin adapter over the same location.Search core (rich SearchLocation shape with friend-visitors + detail). Saving a found place to the plan Stash is a plan write, POST /items/save-location (see Plans), not a location endpoint.

See Discovery → Taxonomy vocabulary for the search query parser. The same lang parameter that picks Photon's translation language picks the taxonomy locale, so a search bar typed in Japanese resolves Japanese category and tag labels through the same call. The handler resolves lang from ?lang= → Accept-Language → "en" via api.Lang.

Search & ranking

Every place-search surface (the main search bar, the place/area picker, the typeahead, the event and plan pickers) funnels through one core, searchCore in location_search.go. Discovery, pickers, and the typeahead only shape and enrich the response, they never re-rank. The core is:

  • Taxonomy-aware — taxonomy.ParseQuery runs inside the core, splitting the query into (category, tags, free-text), so the typeahead is as category-aware as the main search. It is a multilingual dictionary matcher (28-language aliases); the semantic layer augments it, it is not replaced.
  • Area-aware (near vs a named area) — detectArea (location_area.go) spots a gazetteer area in the query, by connective ("cafes in Taipei") or by a trailing place name ("Starbucks Shibuya"), resolves it to a centroid, and recenters the search there, searching only the remaining place text within a radius scaled by the area's prominence. Otherwise the viewer's location biases it; with neither, the search is global. Area lookups are Redis-cached, and a token span is only treated as an area when a settlement strongly resolves for it, so "Shibuya 109" the building still searches as a whole.
  • Source-selectable — the source param (all · tomoda · raw · places) picks which providers to draw from. all (default) is our catalog plus the Photon fallback; tomoda is catalog-only (the Find-tab browse); raw is Photon only. places (Google) is never queried on the read path and is reserved for a future paid fallback. Codenames keep third-party product names off the wire.

Under that, the same two-stage split: PGroonga provides multilingual recall, a Go composite scorer provides the ordering.

  • Recall — SearchLexical matches the trigger-maintained search_text column (base name plus every localized name, lowercased and accent-folded) via a PGroonga index normalized with NormalizerNFKC150 (width/kana folding), so a query in any language, script, or width (東京, 서울, café, Ｓｔａｒｂｕｃｋｓ) hits. It uses &@~ with pgroonga_query_escape, so a multi-word query ANDs its tokens ("Starbucks Shibuya" finds the branch whose name has "starbucks" and whose city has "shibuya") and is injection-safe. Category and tag filters are pushed into the query.
  • Typo-tolerant recall (cost-gated) — when the exact pass returns fewer than the limit, SearchLexicalFuzzy runs a second PGroonga pass with pgroonga_condition(..., fuzzy_max_distance_ratio => 0.3), an edit-distance budget (a ratio of term length) that recalls misspellings ("stabucks", "shibya") across English, CJK, and romanized scripts. Results merge into the lexical pool by id, so an exact hit is never displaced. The common exact-name case fills the page from the first pass and never runs the fuzzy one.
  • Semantic recall (cost-gated) — when lexical comes up short (a conceptual query like "quiet place to work", a synonym, a typo), the query is embedded (bge-m3 via a TEI server, Redis-cached) and SearchSemantic pulls the nearest rows over the embedding halfvec, merged into the lexical pool. The common exact-name case fills the page from lexical and never pays for the embed. Vectors are computed on write by the equalizer (embedRow), gated by shouldEmbed (see Embedding gate), so the cold long tail stays lexical-only. Retrieval is two-stage over a binary quantization of the vector (see Two-stage retrieval). Disabled → lexical-only.

  • Ranking — rankCandidates (location_ranking.go) scores each candidate with a name-dominant blend of six components, each normalized to a clean 0..1 so a weight reads as a genuine share of the decision (weightName 0.55 means an exact name match is 55% of the top score). Weights are tuned offline against the golden set (see below), not hand-guessed:

    Component Weight Normalized 0..1 shape
    Name relevance 0.55 Graded match tier across the base name and every localized name: exact 1.0, then whole-string prefix, word prefix, substring, token overlap. Each tier interpolates within a band by how much of the name the query covers, so two prefix matches of different depth do not tie, and the bands never cross (the weakest exact still beats the strongest prefix).
    Proximity 0.12 Gaussian decay exp(-d² / 2σ²) to the effective center, σ = 2 km (relevance halves at 2 km). Dropped (1.0) for unbiased queries.
    Semantic 0.11 Vector cosine similarity, so a meaning-match with no name hit still surfaces.
    Source trust 0.10 1.0 for our catalog rows, 0.6 for Photon.
    Prominence 0.08 Gazetteer prominence (population, heritage), clamped 0..1.
    Popularity 0.04 log1p(saves) capped at a 100-save saturation, blended 0.6 with external rating 0.4 (rating counts only past 5 ratings).

A recall hit that matched only via non-name text (a city, a tag) floors its name score at 0.35 (lexicalFloor) so it is never zeroed out. So an exact-name match a few hundred metres away outranks a nearer place that merely contains the term, and the city "Ottawa" beats "Ottawa Street".

PGroonga supplies recall, not ranking: its relevance score reflects term frequency, so on its own it cannot tell the city "Tokyo" from a "…Tokyo…" coffee shop. The name-tier scorer is what makes the exact match win.

The Photon fallback carries a gentle, config-tunable location bias so nearby results nudge up without overwhelming an exact-name match far away. platform/photon passes the viewer's coords as Photon's lat/lon/zoom bias with a scale factor, both tunable per environment via PHOTON_ZOOM and PHOTON_BIAS_SCALE (config.Photon.Zoom / config.Photon.BiasScale, with defaults applied in config/config.go). This shapes Photon's own ordering; the Go composite scorer still re-ranks the merged pool, so the bias is a recall nudge, not the final say.

Embedding gate — which rows carry a vector

Embedding the whole catalog does not scale: a 100M+ row POI backbone would need 100M+ vectors and an ANN index over all of them, at real cost, for rows almost no conceptual query reaches. So the equalizer embeds a row only when it earns one. shouldEmbed (location_embedding.go) embeds a row when any of these hold:

Signal Rule
Prominent prominence >= embedProminenceFloor (0.35), the log-scaled prior where a place is notable enough (a few hundred sitelinks, a mid-size town) to be worth a conceptual match
Curated / prestige provenance is unesco_seed or wikidata_seed (hand-curated or notable catalogs)
Engagement-promoted provenance is promoted (a backbone row upgraded on engagement, see below)
Rich text the row carries a signals or metadata blob, giving the embedding meaning beyond its name
Engaged save_count, moment_count, total_visits, review_count, or total_ratings > 0

A cold bulk_import backbone row with low prominence and no engagement is skipped. Only the vector is gated — search_text (lexical) and localizations are maintained for every row by the write-time trigger and the equalizer, so the long tail stays fully findable lexically, spatially, and by category. It just does not answer a pure meaning query until it earns a vector.

Demand-driven promotion. A cold bulk_import backbone row stays cheap (base data + Photon, no paid enrichment, no vector) until a user meaningfully engages with it, then it earns full enrichment. LocationService.PromoteOnEngagement is the one coherent promote path: via LocationStore.PromoteFromBulk it clears the row's bulk_import provenance (sets it to promoted, so isBulkImport is false) and nulls enrichment_at in the same update, then enqueues enrichment with reason EnrichReasonEngaged. Nulling enrichment_at matters: a bulk row's create-time Photon pass stamps it, and EnrichLocation's 7-day min-age guard would otherwise skip the promotion outright. With it nil the guard passes, so the enricher does the rest on the promoted row: paid commercial enrichment fires (the row is rich-detail and no longer bulk) and embedRow fires (shouldEmbed is true for promoted). The hasFreshGoogleFields guard still keeps a redundant re-promote idempotent. Promotion spends paid enrichment and embeddings only on the active subset users touch, never the 100M backbone.

Trigger on meaningful engagement only (select-from-search, open-detail, save, visit, check-in), never a mere search-result impression, or one search would promote every backbone row it returned. The engagement write paths live in other domains, so PromoteOnEngagement is the location-side entry they call: the save path (maps.Service.SaveLocation), the visit path (presence check-in), and the search-selection path (location Handler.Resolve). The save site carries a flag comment for that wiring.

Per-city highlight promotion. Demand-driven promotion is reactive: a place only earns enrichment once a user touches it, so a mid-size or small city's marquee sights stay cold until someone happens to engage. LocationService.PromoteCityHighlights closes that gap at load time. It reuses the same promotion mechanic, applied in bulk to each city's top attractions: for every city (grouped by the city_id self-FK the seed loader's set-based nearest-city pass assigns, so it works even before region_id resolves) it takes the top-N attractions by prominence and promotes those not already promoted. TopCityHighlightCandidates (queries.sql) is the set-based scan, one window function partitioned by city_id served by the partial idx_locations_city_highlights index (so it is an index scan, not a per-city seq scan). The candidate set is any active attraction with a resolved city_id, independent of provenance, so a curated Wikidata/prestige seed promotes just like an Overture backbone row (a curated-only load, with no bulk_import rows, still lights up each city's marquee sights). PromoteToHighlight flips the chosen rows to promoted regardless of source. The allowlist is derived from the taxonomy, not hardcoded: every leaf under the attraction buckets (culture, outdoors, entertainment, music, religion) plus destination shopping/food leaves (Market, Food Hall, Mall, …), so the generic long tail (restaurants, cafes, bars, grocery/convenience/supermarket, services, transit, accommodation) is never promoted. The ranking window spans every provenance including already-promoted rows, so the top-N set is stable across re-runs (an already-eager row keeps its rank rather than freeing a slot for the next-deepest candidate), and PromoteToHighlight skips anything already promoted, so re-runs do not thrash the enrichment queue. It runs from the catalog seeder's load path: catalogseeder load --city-highlights N runs it after the dataset is loaded (the DevOps load pipeline invokes it once the dataset is in place).

Two-stage retrieval — binary quantization and cosine re-rank

The ANN index is over the binary quantization of the embedding, not the full halfvec: idx_locations_embedding_bit_hnsw is an HNSW over binary_quantize(embedding)::bit(1024) with bit_hamming_ops. One bit per dimension instead of sixteen makes the index ~16x smaller, so it stays in memory as the embedded set grows. The full-precision embedding halfvec column stays for the re-rank.

SearchSemantic runs in two stages:

  1. Recall — pull a wide candidate pool (semanticRerankK, default 200) by Hamming distance on the bit index (<~>), applying the category + proximity filters.
  2. Re-rank — order that pool by full cosine distance on the halfvec (<=>) and return the caller's limit.

The tiny bit index does the fan-out; the precise cosine sees only a couple hundred rows, recovering full-precision ordering cheaply. HNSW skips NULL embeddings, so the async-populated column and the gated long tail are both fine.

Next lever: DiskANN, only if the gated set outgrows HNSW

The embedding gate keeps the vector set to high-value + engaged rows, which HNSW handles well. If that set itself grows past what an in-memory HNSW serves comfortably, the next step is a disk-backed ANN (pgvectorscale / DiskANN). It is deliberately not adopted now: it is a new extension, and the gate plus binary quantization keep us on pgvector-native for the foreseeable catalog size.

Tuning the weights — golden set + offline optimizer

The weights are validated offline, not hand-guessed. A committed golden set (internal/services/location/testdata/ranking_golden.json) is a set of hand-authored, network-free cases: a query, the geo-bias flag, a candidate pool carrying each candidate's raw ranking-signal inputs, and a graded relevance label per candidate (0 irrelevant .. 3 perfect). It covers multilingual queries, typo-recall hits, proximity-vs-name tension, and semantic-only hits. Many cases are deliberately adversarial (a mega-popular loose match against an exact-name row, a much-nearer street against a distant city), so a naive weighting orders them wrong and the objective has a real gradient. The shipped weights score NDCG@5 ~0.97 / MRR ~0.94 on the set, not a degenerate 1.0.

The shipped weights are a provisional prior

They are tuned to this hand-authored bootstrap set, not yet validated on real traffic. Treat them as a reasonable starting point, and re-tune with cmd/rankertune once the selection logs (below) have grown the golden set from real queries.

  • Regression guard — a Go test (location_ranking_test.go) scores the live ranker's ordering against the labels with NDCG@5 and MRR and asserts both stay above a floor (baseline minus ~0.03), so a weight or scorer change that hurts ranking fails CI. A companion test perturbs each weight by ±0.1 and asserts NDCG measurably moves, proving the set is discriminating (not a flat plateau where any weighting ties).
  • Offline optimizer — cmd/rankertune reads the golden set and searches (name-dominant coordinate ascent with seeded random restarts, every secondary weight floored at 0.03 so no signal is zeroed out) for the weight vector that maximizes NDCG, printing before/after metrics and the tuned weights. Deterministic, no network, no database. Run it, and if it beats the shipped weights, update the consts in location_ranking.go (round to clean shares; the raw output is false precision on a small synthetic set):
cd backend && go run ./cmd/rankertune          # NDCG@5 over the default golden set
go run ./cmd/rankertune -k 10 -golden <path>   # different cutoff / label file
  • Guardrailed traffic retune — cmd/rankertune -harvest closes the loop from real selections. It reads the durable label sink (ranking_impression joined to ranking_selection over a window), turns each selection into a labeled case reusing the recorded feature sub-scores (no re-scoring), corrects for click position bias with inverse-propensity weighting (1/rank^eta, default eta = 1.0, so a pick at rank 4 counts 4x a rank-1 pick that may just be position), then runs the same coordinate-ascent optimizer on a deterministic train split (hash of ranking_query_id, no randomness). It gates the proposal behind three guardrails and only recommends applying when all pass: at least 200 harvested labels, a held-out NDCG lift over 0.01, and no regression past 0.005 on the hand golden set (a permanent regression floor). It prints the shipped-vs-proposed RANK_WEIGHT_* lines and an APPLY / HOLD verdict; it never writes config or a DB row. Adopting an approved vector stays human-gated, but no longer needs a redeploy: apply it to the active ranking_weights row and the ranker hot-refreshes it in (see Config hot-refresh below).
cd backend && go run ./cmd/rankertune -harvest            # last 30 days
go run ./cmd/rankertune -harvest -days 14 -eta 1.5        # shorter window, steeper bias correction

The harvest/IPW/guardrail logic is a set of pure functions over []models.RankingLabeledCase in location_ranking_harvest.go, unit-tested without Postgres.

Config hot-refresh — applying weights without a redeploy

The composite weights resolve active ranking_weights DB row > env config (RANK_WEIGHT_*) > shipped default. The ranking_weights table holds every applied vector with exactly one active at a time, enforced by a partial unique index (... ON ranking_weights (active) WHERE active).

  • Boot — NewLocationService seeds the env-config vector (so the wire graph builds without a live pool), then StartWeightRefresh does one immediate best-effort DB read to apply DB precedence before the first search. An empty table keeps the config weights byte-for-byte; a read failure keeps the current vector and logs. The read never blocks startup.
  • Hot-refresh — the API ranker re-reads the active row every RANK_WEIGHTS_REFRESH_INTERVAL (default 2m, 0 disables) and atomically swaps it into the live blend, held behind an atomic.Pointer[rankWeights] so concurrent searches always read a consistent set. A failed refresh keeps the current weights. The loop runs on the API pods only, cancelled on shutdown.
  • Apply — POST /admin/locations/ranking-weights (gated by the TomodaAdmin capability) takes the six weights plus an optional note, validates each is in [0,1] and not all-zero at the boundary, persists it as the new active row (deactivating the prior one in one transaction), and swaps it in immediately so the change takes effect without waiting for the next refresh tick. This is how an approved cmd/rankertune -harvest recommendation is adopted.

Ranking observability — closing the loop

Two structured logs turn real search traffic into training data for cmd/rankertune without any schema change (location_ranking_log.go). Harvest both from Loki/Grafana and join on ranking_query_id to build labelled cases.

Event (app_event) When Sampling Fields
location_search_ranked Every ranked search, in searchCore 5% head sample; ≤10 candidates per event ranking_query_id, query, biased, returned, and a candidates array of {rank, location_id, source, score, f_name, f_proximity, f_popularity, f_prominence, f_source, f_semantic} (the normalized feature vector the blend combined)
location_selected A user resolves a result, in Handler.Resolve Unsampled (each is a label) ranking_query_id, query, location_id, selected_rank

The selection event reads its q, rank, and ranking_query_id from optional query params the client echoes from the search that produced the candidate; all are best-effort, so a selection still logs (with an empty correlation id) when the client omits them.

Localized display

Results are localized to the viewer's language, which the search core resolves from ?lang= → Accept-Language → en. Two shapes:

  • The autocomplete/detail payload (LocationCandidate) ships the localizations map (trimmed to the language's fallback chain), and the client picks the field via localizedName / localizedField.
  • The main search payload (SearchLocation) omits the map to stay small, so its name / city are localized server-side via location.LocalizedField before serializing.

Because the enrichment equalizer denormalizes 28-language name + city + country onto each row, display reads from the row's own localizations; the city_id geo-FK is for relational grouping/filtering, not needed for localized display.

Write path — the ingest funnel

Every location write, seeded or runtime, flows through one funnel, LocationService.Ingest (location_ingest.go), so seeds and runtime-minted rows get identical treatment and the write paths cannot drift:

  1. Exact dedup — batched location_providers lookup on the candidates' source refs (FindLocationsBySources); a re-seed (same provider ids) reconciles rather than duplicates.
  2. Fuzzy dedup — for candidates flagged FuzzyDedup (runtime + OSM POIs), fold a new row into an existing same-named one within fuzzyDedupRadiusM (150 m, a flat window on the batch funnel). PGroonga name-similarity recalls the nearest same-name candidate; the Go-side sameNameForMerge gate is what stops two distinct nearby places that merely share a token from merging. The gazetteer bulk load leaves the flag off (unique ids; two distinct "Springfield"s must not merge). The interactive Resolve ladder uses a per-category dedup radius instead (see Per-category dedup radius); the batch funnel is the flat window.
  3. Reconcile or create — a hit runs models.FillMissing (fill empty fields, union localizations/tags, max prominence, add new source refs); a miss is bulk-created via store.CreateBatch (COPY, which fires the search_text trigger so bulk rows are searchable exactly like single inserts).

Localizations cover the languages the production Photon index carries (SupportedLocalizationLanguages, 28); gazetteer rows arrive already localized from GeoNames, POIs fill from Photon. Enrichment (rich detail, embeddings) runs async off the same rows, so the create step never blocks on it.

Seeding — how rows get into locations

Rows arrive two ways, both through Ingest: the runtime resolve path (user-driven, one place at a time, provider-first via LocationService.Resolve) and the batch catalog seeder (offline, bulk — countries, cities, curated lists, with provenance geonames_seed / unesco_seed / wikidata_seed). See Catalog seeding.

Entry point Trigger Notes
Frontend autocomplete submit (the dominant path) User picks a candidate in the moment/event publish flow → frontend holds the candidate locally → at submit time hits POST /locations/resolve with the candidate Pre-resolution gives the moment/event a concrete location_id to attach to. Frontend never resolves on the pick itself — that would persist rows for abandoned forms.
Moment create handler (POST /moments) Defensive fallback inside the create handler — fires when LocationID is nil AND coords are present (map-pin moment, scripted client) Off-grid moments (Photon doesn't know the place) keep LocationID=nil and carry LocationLabel + their embedded Spatial instead.
Event create handler (POST /events) Same defensive fallback shape as moments Off-grid events use LocationLabel + the embedded Spatial.Address for a user-supplied street address.
Future: voyage waypoint create Same pattern, not yet shipped Will use the same Resolve ladder; off-grid waypoints stay on their own coords.
Admin merge (POST /admin/locations/merge) Folds a duplicate Location into a survivor Doesn't create a row — repoints the loser's location_providers rows onto the survivor so either id resolves going forward.

Reads (/nearby, /autocomplete, /search, /reverse) never persist — they merge DB results with live Photon results for display only.

Resolve flow

There is one entry point: Resolve. It walks the lookup ladder and persists a row from the first provider that knows the place; if neither does, it returns (nil, nil) and no row is written.

flowchart TD
  Req["ResolveRequest"] --> P1{Provider + ProviderID set?}
  P1 -- yes --> Cache["Redis lookup<br/>location:provider:{provider}:{id}"]
  Cache -- hit --> Return["return Location"]
  Cache -- miss --> DBProv["DB lookup by<br/>(provider, provider_id)"]
  DBProv -- hit --> Return
  DBProv -- miss --> Spatial
  P1 -- no --> Spatial["Spatial + name fuzzy<br/>(50m radius)"]
  Spatial -- hit --> Return
  Spatial -- miss --> Policy{Provider policy}
  Policy -- "caller hint = google_places" --> Google["Google: DetailsByID or NearbySearch"]
  Policy -- otherwise --> Photon["Photon ReverseGeo"]
  Photon -- match --> CreatePhoton["Create row<br/>provider=photon"]
  Photon -- empty --> Nil["return (nil, nil)"]
  Google -- match --> CreateGoogle["Create row<br/>provider=google_places<br/>enrichment_at=now"]
  Google -- empty --> Nil
  CreatePhoton --> Enqueue["Enqueue location:enrich"]
  CreateGoogle --> Return
  Enqueue --> Return
  Nil --> Return

Write paths today:

  • Frontend autocomplete picks call POST /locations/resolve at moment/event submit time (not on the pick itself) to convert a candidate into a concrete location_id. A 404 means the caller should submit without location_id and rely on its off-grid label.
  • Moment/event create handlers keep a defensive Resolve call as a safety net for callers that didn't pre-resolve (map-pin moments with no autocomplete pick, scripted clients). If Photon doesn't know the place, the moment/event keeps its coords on its own row.

Google is consulted on the synchronous path in exactly one case: when the caller hints Provider = ProviderGoogle (they already know it's a Google place). The "Photon hit + rich-detail" case picks up Google rich detail via the background enrichment worker.

Allowlist gate — uncategorizable rows never persist

After Photon returns a match, the mapper (mapOsmCategory in backend/internal/platform/photon/photon_osm_mapping.go) runs the OSM tag through a curated table. The match must produce a non-empty Category from place_taxonomy.json for a row to be created. If the mapper returns blank — because the OSM feature is residential, settlement, utility infrastructure, or otherwise not a Tomoda POI — resolveViaProvider returns (nil, nil) and no row is written. The moment/event handler then falls through to its LocationLabel + own Spatial off-grid path.

Defensive denylist entries in the mapper explicitly block: building:residential / apartments / house / yes / dormitory / garage / industrial / commercial / warehouse, all landuse:residential / commercial / industrial / forest / cemetery, all place:* settlements, all boundary:* administrative, all waterway:* flow features, and micro-infrastructure (amenity:vending_machine, bench, toilets, recycling, waste_basket, etc.). Even if a future per-key fallback gets added, these stay blocked.

Extras-aware disambiguation

The mapper takes properties.extra.tags (Photon's secondary OSM tags) as a third argument and uses them to resolve ambiguous primary tags:

  • amenity:place_of_worship + religion=buddhist/hindu/shinto → Temple (the default falls through to Church)
  • amenity:place_of_worship + religion=muslim → Mosque; + religion=jewish → Synagogue
  • amenity:place_of_worship + building=cathedral → Cathedral
  • amenity:marketplace + produce=* → Farmers Market; + second_hand=yes → Flea Market; default → Market
  • leisure:fitness_centre + sport=yoga → Yoga Studio; + sport=climbing → Climbing Gym; default → Gym
  • amenity:bar + live_music=yes → Music Venue
  • amenity:theatre + theatre:type=comedy → Comedy Club
  • tourism:resort + resort=ski → Ski Resort

The same extras pass feeds extractFacetTagsFromExtras which adds facet tags directly onto the candidate's Tags (cuisine values → italian/japanese/…, diet:vegan=yes → vegan, outdoor_seating=yes → outdoor-seating, wifi=yes → wifi, wheelchair=yes/limited → wheelchair-accessible, etc.). This is free enrichment — Photon-rooted rows now carry rich facets without waiting for the Google enrichment worker.

Dedup story

The "same physical place exists as multiple rows" failure mode is what the service is built to prevent. Without dedup, the travel log fragments, the game engine misses stamps, and the discovery map gets duplicate pins.

Two layers catch most cases at write time. Both are binary by design — there is no confidence score. The two rails below are high-confidence by construction; anything they don't catch goes to the admin queue for human review rather than being auto-merged at a threshold we'd have to tune.

Layer Catches Where
Provider-ID match Same provider returned the same place (provider, provider_id) unique index + Redis cache
Spatial + name similarity Different providers, lat/lng jitter, typo'd or multilingual names FindByNameSimilarity(radius, name) PGroonga &@* recall over the NFKC-folded name, gated in Go by sameNameForMerge — radius is per-category (see below)

The similarity rail is two steps. PGroonga's &@* operator does a loose recall over idx_locations_name_pgroonga (the normalized name index, immutable_unaccent(lower(name)) with NormalizerNFKC150), returning the single best nearby candidate within the radius. Loose recall can pull in a distinct place that merely shares a token, so sameNameForMerge (location_dedup.go) re-checks the candidate in Go before any merge: an exact normalized match, a Damerau-Levenshtein ratio at or above 0.85 (typos, width/accent variants), or one name's token set fully contained in the other ("Starbucks" vs "Starbucks Coffee"). A "Coffee" vs "Cafe" word swap and two different brands sharing "Coffee" both fall below the gate and stay as separate rows for admin review. Tight radius + LIMIT 1 + this gate is what makes the rail high-confidence with no false merges.

Per-category dedup radius

The spatial dedup radius isn't a single flat number. A 50m window is too tight for a 400m park (entries on opposite gates look like different places) and too wide for a city-block-dense restaurant row (next-door cafés get falsely merged). Each category in backend/data/place_taxonomy.json carries an optional dedup_radius_m field; LocationService.dedupRadiusFor(req.Hint) reads it via TaxonomyService.DedupRadiusFor(category). Categories without the field fall back to a 50m flat default.

Category bucket Radius Why
Restaurants, cafés, bars, bakeries, nightclubs, fast food 30 m Dense urban; tight to avoid collapsing next-door spots
Most categories (no override) 50 m Default fallback
Stations, malls, hospitals 100–150 m Big footprints with multiple labeled entries
Stadiums, arenas, universities, parks, beaches, hiking trails, botanical gardens 150–200 m Large polygons; two entries on opposite gates are the same place
Airports, theme parks, zoos, lakes 200–300 m Very large footprints
Mountains, forests, national parks 500 m Fuzzy, range-shaped boundaries

The list is open-ended: add dedup_radius_m to any category that produces noticeable false-merges or false-non-merges in prod. No deploy step beyond shipping the JSON change — the taxonomy reloads at startup.

Search-time same-name collapse

The taxonomy radii above govern the write-time merge. The read-time autocomplete/nearby path has its own pass, dedupeNearbyByName (location_service.go), that folds duplicate same-name rows out of a single result set (DB + Photon candidates combined) before they reach the client. It picks the radius from the pair's kind, classified by candRank (POI = 2, settlement = 1, blank/address = 0; settlementCategories is City, Town, Neighborhood, Place):

Pair Radius Why
Same-name settlements/places (candRank ≤ 1 on both) 25 km A city has several OSM representations (relation + boundary node) 10 km+ apart; a real POI (rank 2) never merges this way
Both blank-category (street segments) 5 km One street returns many same-named OSM segments
Anything else (POIs) 120 m Tight, so next-door same-name venues stay distinct

Survivor selection prefers the more specific kind (POI > settlement > blank), and a DB canonical only breaks a same-kind tie, so a curated POI wins over a Photon duplicate but a gazetteer locality never suppresses a landmark of the same name.

Admin merge — for what auto-dedup misses

When auto-dedup misses (e.g. "Apple Store - Ginza" vs "Apple Ginza" — different name, both Photon nodes, past the Levenshtein threshold), an admin can fold the duplicate into the survivor via POST /admin/locations/merge — see the admin docs. MergeInto is admin-triggered post-hoc cleanup; it is never called from any automatic path. The two rails above are the only automatic dedup.

MergeInto mechanics

The merge handler runs in one transaction:

  1. Rewrite every cross-reference from source to target. Driven by a private locationFKRewrites manifest in backend/internal/services/location/location_store.go, one row per (table, column) pointing at locations(id).
  2. Cross-link providers — append source's (provider, provider_id) to target's secondary_providers (plus any of source's own secondaries), deduped. Target stays reachable by either id from now on.
  3. Hard-delete source via tx.Unscoped().Delete(). Nothing alive references it any more (step 1 rewrote everything); the forensic trail lives entirely in audit_logs.
  4. Invalidate Redis — provider-pair cache entries on the surviving row are cleared so callers stop hitting stale pointers.

There is no merged_into redirect column. Hard-delete + the audit log is the entire story; stale client-side caches that hold a source id get a clean 404 on next fetch.

Adding a new table with a location_id FK

Every Postgres FK pointing at locations(id) MUST appear in the locationFKRewrites manifest. The store exposes VerifyMergeCoverage(ctx), which queries information_schema and fails if any FK isn't registered. It runs at server startup after Migrate (called from backend/internal/migrate/manual.go), so a missing registration fails the boot rather than silently leaving dangling references after the next admin merge.

When you ship a new model with location_id:

  1. Add {"your_table", "location_id"} to locationFKRewrites in backend/internal/services/location/location_store.go. Keep alphabetical unless a unique constraint forces a specific rewrite order.
  2. If your table has a unique constraint involving location_id (e.g. UNIQUE (user_id, location_id)), the naïve UPDATE collides when a single user has rows at both source and target. Add a pre-rewrite step that folds the source-side row into the target-side row first.
  3. The startup verifier will flag the missing entry on local boot; the same guard prevents a half-wired merge from shipping to prod.

Background enrichment

Enrichment is event-driven, never periodic. Two triggers exist; both enqueue the same location:enrich Asynq task (on the low priority queue) with different reason values, and the worker LocationService.EnrichLocation branches on the reason.

Trigger 1: row creation (reason = new_location)

Fires once, in createFromPhoton immediately after a new Photon-rooted row is persisted. createFromGoogle does not enqueue — Google rows are fully enriched at create time inside the synchronous path via MapPlaceToLocation.

Current provider Worker action
photon AND tags ∩ rich-detail set ≠ ∅ Search-enrichment first (Serper today, swappable via SearchEnricher). On match: merge rating / hours / website / menu / phone / thumbnail into Location.Signals JSONB + direct columns, rehost thumbnail via AssetService.UploadRaw, append the Google placeId to secondary_providers. On miss / disabled / budget-exceeded: fall through to Google Places NearbySearch + MapPlaceToLocation (existing path). Photon stays as the canonical (provider, provider_id).
photon AND no rich-detail tags No-op — Photon already gave us what it has.
google_places No-op — already enriched at create.

The rich-detail bucket set comes from TaxonomyService.RichDetailTags() (any bucket flagged rich_detail: true in backend/data/place_taxonomy.json, today the 14 buckets food, drinks, coffee, tea, nightlife, entertainment, music, culture, fitness, wellness, shopping, accommodation, health, travel). See Discovery → Taxonomy vocabulary.

Serper enrichment gates (belt + suspenders — mis-anchoring is worse than not enriching):

  • Coord gate: the returned Serper POI must be within 200m of the Photon row's coords. Farther than that = different physical place = refuse the merge.
  • Confidence gate: PlaceConfidence ≥ 0.6 (deterministic-guard matches score 0 and are accepted based on the coord gate alone; LLM-selected matches must clear 0.6).

The write path is naturally bounded (Photon dedup collapses duplicate rows; the endpoint rate-limit caps request rate; enrichment fires once per new rich-detail row), so no explicit daily budget guard is needed — a runaway would have to defeat both dedup layers first.

Google-provided identifiers: whether the enrichment came via Serper or a direct Google Places call, the CID / placeId are Google-issued (Serper is a scraping proxy on Google Maps). Both paths write the same {Provider: "google_places", ProviderID: <placeId>} secondary-provider entry, so a future direct-Google lookup for the same place dedups naturally against the Photon row.

Create-time signal persistence (ResolveExternalPlace)

The item-enrichment pipeline (itemService.enrichLink) already has a full Serper SearchEnrichment in hand when it resolves a link to a Location. Rather than throw the data away and let the async worker re-fetch, the item path calls LocationService.ResolveExternalPlace with ExternalPlace.Signals pre-filled. The resolver then:

  1. Deduplicates by primary + secondary provider ID (findByProviderID).
  2. Deduplicates by proximity + name similarity (FindByNameSimilarity, gated by sameNameForMerge).
  3. Photon-first: reverse-geo at the supplied coords. If a match passes the 200m coord gate, the row is created Photon-canonical with the Google placeId as a secondary and the signals attached via attachSignals.
  4. Fallback: createFromGoogle when Photon misses — row stays Google-canonical, signals still attached.

Either way, Location.EnrichmentAt = now() is stamped at create-time and the row is not enqueued for async enrichment. The 7-day suppression guard then no-ops any stray enrichment task that lands on this row.

Trigger 2: user report (reason = user_reported_outdated)

Fires when a user hits "Report outdated info" on the location detail view in the frontend. The endpoint POST /api/v1/locations/{id}/report-outdated always returns 200 — from the user's POV they're filing a report, not commanding a refresh. The actual provider re-fetch happens in the worker:

Current provider Worker action
photon Re-pull Photon ReverseGeo at the row's coords and overwrite name / address / city / country / category / tags / localizations from the fresh node. Then, if rich-detail, re-run the Google enrichment to refresh hours / photos / ratings.
google_places Re-fetch Google Places.Details by the row's place_id and re-run MapPlaceToLocation (refreshes hours / photos / ratings / business status).

The row's (provider, provider_id) identity is preserved — refreshes never re-key a row, so existing event / moment / waypoint references stay valid. Photon misses leave the existing fields intact rather than blanking them.

Soft-close lifecycle (tombstone, never delete)

A report-outdated refresh can conclude that a place is gone. When it does, the row is soft-closed, not deleted: it drops out of every forward-facing search but stays in the DB so historical linkages (past check-ins, moments, events, saves, passport visits, all FK'd to location_id) keep resolving.

A location is in one of three states, tracked on the existing is_active + business_status columns plus a closed_at timestamp:

State is_active business_status closed_at In search?
Operational true OPERATIONAL null yes
Permanently closed false CLOSED_PERMANENTLY set no (kept for history)
Temporarily closed true CLOSED_TEMPORARILY null yes (badge only)

deleted_at is a separate concern (removed / bad rows) and is untouched by this lifecycle.

The refresh (refreshFromProviders → applyLifecycle in location_enricher.go) decides the transition:

  • Provider reports CLOSED_PERMANENTLY (Google businessStatus, or Serper's permanentlyClosed flag on a Photon-canonical row) → authoritative close: is_active=false, business_status=CLOSED_PERMANENTLY, stamp closed_at.
  • Provider reports CLOSED_TEMPORARILY → keep it searchable, persist the status as a badge only.
  • Place is NOT FOUND by the provider → not trusted on its own (renames, transient provider gaps, and obscure places all produce false negatives). Instead bump a not_found_count corroboration counter; the row only soft-closes once the count reaches notFoundCloseThreshold (2). Below the threshold it stays active.
  • Provider re-confirms operational on a later refresh → reversible: reactivate (is_active=true), clear closed_at, reset not_found_count, business_status=OPERATIONAL.

Search exclusion is enforced in queries.sql: the forward-facing recall queries (lexical, fuzzy, nearby/browse, semantic) all carry AND is_active, so a closed row never surfaces. GetLocationByID / ListLocationsByIDs deliberately do not filter on is_active — a closed place must still be fetchable by id so deep links and historical references render it with its permanently-closed label.

The 7-day guard

Both triggers share a single suppression check: if EnrichmentAt is within enrichmentMinAge = 7 days, the worker no-ops before touching either provider. This is what makes "no rate limit needed" safe — a user can mash the report button forever, but at most one refresh per location per 7 days actually fires. Each refresh writes EnrichmentAt = now() on completion.

Other safety nets:

  • Asynq TaskID dedupe — collapses concurrent enqueues for the same (location_id, reason) while a task is in flight.
  • MapPlaceToLocation is shared between the create path and both refresh paths, so a row's shape is identical regardless of which trigger last wrote to it.

What enrichment doesn't do (yet)

  • Canonical-list tags (UNESCO / Michelin / 50 Best) — EnrichReasonCanonicalPublish is defined for use when the canonical-list ingester lands; the handler does nothing with it today.
  • Photo URL hydration — Google Place photos return as opaque resource names. Resolving them to a CDN URL is left to the read path.
  • Periodic sweeps — intentionally removed. There is no daily cron and no batched re-enrichment. If a row's data goes stale and no user reports it, it stays as it is. (The not_found_count corroboration counter is per-row and only advances on a report-outdated refresh, not a periodic scan.)

Data model

locations:

  • name, address, full_address
  • category (varchar(64)) — one canonical leaf from backend/data/place_taxonomy.json (Cafe, Restaurant, Bar, Mountain, Hospital, Wat, Synagogue, …). A place IS one thing; multi-membership lives in tags.
  • tags (text[], GIN-indexed as idx_locations_tags) — open-ended bucket + facet tag set from backend/data/place_taxonomy.json. Picks up the category's default tags at create time (e.g. Cafe → ["food","coffee","tea"]); additional facets like vegan, outdoor-seating, or cuisine tags can be appended by users, admin tooling, or canonical-list ingestion. WHERE tags @> ARRAY['vegan'] and tags && ARRAY['food','coffee'] are both index-served.
  • business_status (OPERATIONAL / CLOSED_PERMANENTLY / CLOSED_TEMPORARILY), coordinates (PostGIS geometry, GiST-indexed via idx_locations_coordinates_geog)
  • is_active (bool, indexed) — search visibility gate; false for a soft-closed (permanently closed) row. closed_at (timestamp, nullable) stamps the authoritative close; not_found_count (int) corroborates provider misses before closing. See the soft-close lifecycle.
  • Source refs — every external identity for the row lives one-row-per-ref in the location_providers table ((provider, provider_id), composite unique), not a column on locations. A row carries the full set (photon {osm_type}:{osm_id}, google_places Place ID, wikidata QID, geonames geonameid, unesco id, …); there is no primary. models.Location.Sources is the in-memory view; findByProviderID resolves a place by any of its refs (the merge cross-link path folds a loser's refs onto the survivor).
  • provenance — the LocationProvenance enum above; gates paid enrichment (isBulkImport) and embedding (isCuratedProvenance).
  • enrichment_at (timestamp, nullable) — last time enrichment ran; the 7-day guard reads it.
  • search_text (text, PGroonga-indexed as idx_locations_search_text_pgroonga with NormalizerNFKC150) — the trigger-maintained lexical blob: base name + every localized name/city/country + address + admin area + neighborhood + tags, lowercased and accent-folded. Maintained by the locations_search_text_sync trigger on every write that touches a searchable field, so bulk COPY inserts stay searchable exactly like single inserts.
  • embedding (halfvec(1024), nullable) — the bge-m3 semantic vector, populated async by the equalizer only for rows shouldEmbed passes. The ANN index (idx_locations_embedding_bit_hnsw) is over its binary quantization; see Two-stage retrieval.
  • city_id (uuid self-FK, nullable) — the row's containing city (cities are locations); assigned in one set-based nearest-city pass at load. The per-city highlight promotion and localized-city read paths follow it.
  • region_id (uuid FK → regions, nullable) — the containing administrative area; resolved by enrichment.
  • localizations (JSONB, 28 languages), photos, opening_hours, place_types
  • phone_number, international_phone_number, website, google_maps_uri
  • average_rating, total_ratings, price_level, last_activity_at, deleted_at
  • Geographic denorm (on the embedded Spatial struct, shared with Events + Moments):
    • district — sub-city neighborhood / suburb / borough; best-effort (Photon's OSM tags vary by country; Google fills it from sublocality)
    • city — populated by Photon's city or Google locality component
    • region — state / province / prefecture; nullable for city-states (Singapore, Monaco, Vatican). Populated by Photon's state or Google administrative_area_level_1
    • country, country_code — ISO 3166-1 alpha-2
    • continent, derived at write time from country_code via geo.ContinentForCountryCode (backend/internal/geo/continents.go); one of Africa | Asia | Europe | North America | Oceania | South America | Antarctica
    • The Google path consults addressComponents when available (structured, language-stable) and falls back to FormattedAddress parsing otherwise.

The denorm columns above are the display fast-path. The relational area tree lives in a dedicated regions table (region_store.go, regiondb/): one row per administrative area, with admin_level (1 = ADM1 state/province, 2 = ADM2 county/district, 3 = ADM3 where a country models it), a self-referential parent_region_id, coordinates, population, 28-language localizations, and the GeoNames feature_code + admin1_code/admin2_code that reconstruct the parent linkage. The seed loader resolves parents in level order (ADM1 before ADM2 before ADM3) via (country_code, admin1_code, admin2_code), index-served by idx_regions_admin_codes. A location's region_id FK points into this tree for grouping and for gazetteer recentering, separate from the flat region string denorm used for display.

Caching

Cache Key TTL Purpose
Provider-id resolve location:provider:{provider}:{provider_id} 1 h Skip the DB lookup on hot paths (same place referenced by many users in quick succession)
Per-user Google rate limit places:ratelimit:{userID} 1 min Existing — caps a user at 10 Google calls/min

Admin merges don't actively invalidate today; the 1 h TTL is the upper bound on inconsistency. Explicit invalidation lands with the merge handler.

Dependencies

  • LocationStore (location_store.go), the domain's persistence port: CreateBatch, GetByID, Update, FindLocationsBySources, SearchLexical, SearchLexicalFuzzy, SearchSemantic, FindByNameSimilarity, PromoteFromBulk, TopCityHighlightCandidates, UpdateEmbedding, MergeInto, VerifyMergeCoverage
  • region_store.go (region_store.go, regiondb/), the ADM1/ADM2 administrative-area store the seed loader and enrichment resolve against
  • platform/cache Redis abstraction: GetCache / SetCache for the provider-id cache and the query-vector cache, RateLimit for the Google rate limit
  • platform/photon geocoder client and platform/places Places client (Serper transport + direct Google), the two provider platform pieces
  • platform/places SearchEnricher, the optional enrichment transport. The location service is its sole caller (via EnrichCapture and the async enrichment worker); capture pipelines delegate here rather than calling it directly
  • platform/embeddings, the TEI (bge-m3) client the equalizer embeds rows and search queries through
  • platform/taxonomy, the canonical place vocabulary, rich-detail set, and dedup radii
  • observability: NewCounterVec for location_provider_calls_total

Notable behavior

Photon-first, Google sparingly

Photon is queried for every read path with no rate limit. A direct billed Google call happens only when the caller explicitly hints Provider = ProviderGoogle or when a Places escalation exhausts the cheap Serper transport first. All other rich-detail enrichment runs in the background worker so the sync p95 stays bounded. The location_provider_calls_total{provider,reason} Prometheus counter exposes both sync and async escalations.

Localizations are universal across write paths

Every persisted location carries name translations, from every write path, via free self-hosted Photon (buildLocalizations, a parallel reverse-geocode across the supported languages — wall-clock cost is one Photon RTT): Photon-resolved rows at create, Google-escalated rows at create, Serper-minted sparse rows via the background enrichment worker, and batch-seeded rows from their source data (Photon as an optional gap-filler). Serper and Google are not localization sources — Photon is. On every read path the response is trimmed to the caller's BCP 47 fallback chain via FilterLocalizationsForLang (~90 % size reduction). The full model is Localization; timezone is not stored — it is derived from coordinates (Timezones).

Photon never returns errors

PhotonService.doRequest and ReverseGeocodeCity deliberately swallow all errors and return empty slices/strings. Photon outages must not break user flows — the caller falls through to the off-grid path (no location_id, coords on the moment/event row).

Off-grid moments and events

Moments and events that resolve to nothing in Photon (private parties, pop-ups, unmapped spots) keep their coords on their own embedded Spatial rather than spawning a placeholder Location row.

Both also carry a location_label for the human-readable name in that case. Events additionally use the embedded Spatial.Address for a user-supplied street address — events typically have a place a guest needs to physically find, and "Park Avenue Backyard Party" without an address isn't useful.

Moments don't need an address — the coords are the address. The label is just for display ("Joe's Cabin", "Hidden Beach"), captured when the moment-create flow surfaces a prompt after the nearby lookup returns nothing.

Roadmap

The subsystem is built and merged: the cost-tiered enrichment spine, the unified ingest funnel, the multilingual lexical + spatial + demand-driven-semantic search core, the offline-tuned six-weight ranker with runtime observability logging, the two-stage binary-quantized vector retrieval, the ancestry-based Overture taxonomy mapping, the soft-close lifecycle, and the catalog seeder (countries, regions, cities, UNESCO, Michelin, 50 Best, notable overlay, Overture backbone). The datasets are generated; the first full production load has not yet run. What is left is the tuning-and-validation work that only real data can drive, plus the graduation levers that only pay off at scale.

Near-term (before the first full production load)

Item State today Planned
Relevance auto-tuning loop Built: a durable label sink persists a sampled ranking_impression (its ranked candidates + feature sub-scores) and every ranking_selection, joined by ranking_query_id, best-effort off the hot path, pruned at 180 days (cron:ranking_label_prune); labels now survive past Loki retention. The six weights are config-driven (RANK_WEIGHT_*, defaulting to the shipped constants) so a retuned vector deploys without a recompile. cmd/rankertune -harvest turns the sink into inverse-propensity-weighted labeled cases (correcting for click position bias) and reruns the optimizer on a deterministic train split, gated on a held-out NDCG lift, a hand-golden-set regression floor, and a minimum label count; it prints an APPLY / HOLD recommendation with the RANK_WEIGHT_* lines and never applies. Runtime logs still emit for observability. Weights now resolve active ranking_weights DB row > env config > default and hot-refresh in on an interval, so an approved vector applies without a redeploy (via POST /admin/locations/ranking-weights); the retune stays human-gated by design. Remaining: graduating the blend to learning-to-rank is a separate long-term item below.
Embedding recall measurement Built: cmd/vecrecall measures two-stage recall@k vs exact cosine and sweeps rerank_k (synthetic self-test proves the harness); the tomoda_locations_embedded_total gauge (cron:location_embedded_count) is the graduation trigger. Ships with semanticRerankK = 200. Remaining: run cmd/vecrecall against the loaded+embedded dataset to pick the production rerank_k (the measurement needs real embeddings, so it happens after the load).
Cold-backbone localization On-demand Photon fill (gapFillLocalizations) is the accepted default: a high-prominence backbone row gets its localizations when first touched. Optional: a bounded Photon localization gap-fill over the high-prominence cold-backbone slice, only if on-demand localization proves weak after the load lands.

Long-term

Item Trigger Direction
Learning-to-rank Enough real labels accumulate through the auto-tuning loop. Graduate the ranker from the six-weight linear blend to a learning-to-rank model (e.g. LambdaMART). The linear model is the right call while labels are scarce.
Disk-backed ANN The shouldEmbed-bounded embedded set outgrows in-memory HNSW, or measured recall is poor. Adopt pgvectorscale / DiskANN + SBQ. Today's binary-HNSW two-stage retrieval is the right call: the gate keeps the vector set small enough to fit in RAM.
Staged full-scale load The near-term items are ironed out. A staged rollout (local subset → one region → global) validates fuzzy dedup, set-based city-linking, and per-city promotion performance plus real search relevance at ~60M rows before the DevOps cron owns the load. This is deliberately the last step.
Planet-scale Photon The runtime resolver + enricher need global coverage. A planet-scale self-hosted Photon deployment (a DevOps concern) so a region-limited index does not become a global coverage gap.
Overture auto-latest Overture ships a new release monthly. The Overture release is a constant (overtureRelease in cmd/catalogseeder/source/overture/overture.go) today; auto-tracking the latest release removes a manual bump per month.

Where to look