Long-term Scaling & Growth¶

This page is the growth runbook for the backend. It exists so a future engineer can grow the system to very large scale as a series of pre-designed, just-in-time, zero-downtime steps, each plugging into a seam that already exists, rather than a rewrite. It records the heavy infrastructure we deliberately did not build yet, the trigger that says "now build it", and the exact playbook for each.

Core principle¶

Scale by adding infrastructure behind clean seams, just-in-time, not by pre-building. The stateless tier autoscales on its own. The data tier scales by deliberate, pre-planned steps, each of which is config or deployment only because the seam is already in the code.

Two rules follow from this:

Every heavy item has a seam waiting for it. Read replicas, a dedicated WebSocket tier, connection pooling, and partitioning are already reachable through config flags and process modes. Building them early buys nothing but carry cost.
Build at the trigger, not before. Each item below names a metric threshold. Until it fires, the item stays unbuilt. This keeps feature velocity high and complexity low.

Current posture — what scales automatically today¶

Capability	Mechanism	Where
Stateless pod autoscale	HPA on CPU + memory, min 2 / max 6	devops `k8s/apps/tomoda/overlays/prod/hpa-api.yaml`, `hpa-async.yaml`
Process-mode split	one image, `SERVER_MODE` selects roles	`backend/cmd/server/main.go` (`resolveMode`)
Media offload	S3 + CloudFront signed URLs, served straight to clients	`backend/config/config.go` (`S3Config`), `backend/internal/platform/assets`
Cross-pod real-time	Redis pub/sub fan-out between hub pods	`backend/internal/platform/ws/session_hub.go`, `client_hub.go`
Cache / hot path	Redis via the cache bundle	`backend/internal/platform/cache/cache.go`
Bounded-cost reads	keyset (cursor) pagination everywhere, one codec	`backend/internal/platform/cursor/cursor.go`
No ORM overhead	sqlc + pgx, hand-tuned SQL	`backend/internal/services//db/`
Safe boot migrations	goose under a Postgres advisory lock	`backend/internal/database/database.go` (`Migrate`, `migrationLockKey`)
SQL isolation	every query lives in the store layer	`backend/internal/services/*/store.go`
Background work	Asynq worker + leader-safe scheduler	`backend/internal/async/`

The store-layer isolation is the load-bearing convention. Because no handler or service issues raw SQL (see the backend layering rules), replicas, caches, and shard routers all plug in at one layer instead of being threaded through business logic.

Seams already installed¶

These are the enablers. Each makes its future scaling step config or deployment only.

Seam	What it is	What it unlocks
Read/write pool split	`database.ReadPool` embeds a second pgx pool; `DB_READ_HOST` selects it, else it aliases the primary	Read replicas become a config flip. Read-heavy stores already take `database.ReadPool` (e.g. `NewDiscoveryStore` in `backend/internal/services/discovery/store.go:37`).
WS process-mode split	`ws-hub` / `api-hub` / `multi-hub` modes in `resolveMode`	A dedicated WebSocket tier is a new Deployment plus a route, no code change.
`DB_MAX_CONNS`	per-pool connection cap (`PoolMaxConns`, default 20)	Lets a pooler be sized as pod autoscaling multiplies pools.
`chat_messages` partitioning	table is `PARTITION BY RANGE (created_at)` with a default partition	Query pruning, per-partition vacuum, and later cold-storage archival (not deletion) without a schema rewrite.
Redis pub/sub fan-out	`RedisSessionChannelFmt` / `RedisClientChannelFmt`, `PSUBSCRIBE session:` / `client:`	Real-time already survives multiple hub pods; the next step is narrowing subscriptions, not adding fan-out.

RequestHTTP / WS

↓

handler → service → storeSQL isolated here

↓

primary poolPool — writes

read poolReadPool — DB_READ_HOST

Rediscache + pub/sub

The read pool aliases the primary until a replica exists. Every seam below plugs in at this layer, not in business logic.

The scaling roadmap¶

Step	Trigger (do-it-now signal)	Zero-downtime?	Effort	Seam it plugs into
Read replicas	primary read CPU sustained high; replica-lag budget healthy	Yes	Hours	`database.ReadPool` / `DB_READ_HOST`
Connection pooler (transaction mode)	connection count nears primary `max_connections`	Yes (rolling)	Hours	`DB_MAX_CONNS`
Dedicated WS tier	WS connection count high; API-pod memory rising from socket buffers	Yes (rolling)	~Days	`ws-hub` / `api-hub` modes
Table partitioning (notifications)	table size / autovacuum pressure	Yes	Days	`PARTITION BY RANGE` pattern (proven on `chat_messages`)
Fan-out feed / timeline	feed-read DB load dominates	Yes (interface swap)	Weeks	discovery/feed read path in the store/service
WS directed routing, then presence sharding	pod count x message rate; then Redis pub/sub saturation	Yes (rolling)	1-2 months	`PSUBSCRIBE` fan-out in the hubs
Sharding (Citus or app-level)	write throughput a primary + replicas can't hold	Careful	Weeks	store layer contains the change
Multi-region	global latency / data residency / DR needs	Careful	3-6 months	externalized state (Postgres, Redis, S3)
Streaming + search cluster	Asynq / PostGIS-SQL hit their ceilings	Yes (additive)	Weeks	async event path, discovery query path

Read replicas¶

Trigger. Primary read CPU sits high under steady load while write volume is fine, and the replica-lag budget (staleness the read paths tolerate) is acceptable.
Playbook. Provision a CNPG replica (raise instances in the devops postgres-prod Cluster, k8s/envs/prod/postgres/manifests/cluster.yaml, currently instances: 1). Point DB_READ_HOST at the replica service. Stores that took database.ReadPool offload their reads on the next rollout with no code change.
Zero-downtime? Yes. The read pool aliases the primary until the env var is set (ReadDSN / HasReadReplica in backend/config/config.go).
Seam. database.ReadPool, DB_READ_HOST.

Connection pooler (transaction pooling)¶

Trigger. Total backend connections approach the primary's max_connections (currently 100 in the prod Cluster) as pod autoscaling multiplies pgx pools.
Playbook. Put a transaction-mode pooler (PgBouncer) in front of CNPG. Size DB_MAX_CONNS per pod so pods x DB_MAX_CONNS fits the pooler's server-side pool, not the raw Postgres cap.
Pooler-safe by config. Transaction-mode pooling breaks the default per-connection prepared-statement cache. This is already handled: set DB_POOLER=true and pgx.go switches to the pooler-safe CacheDescribe exec mode (backend/internal/database/pgx.go). Off by default so direct connections keep the faster cache, so the cutover is config-only, not a coupled app release.
Zero-downtime? Yes, rolling. The pooler is a network hop in front of the DSN.
Seam. DB_MAX_CONNS (PoolMaxConns).

Dedicated WebSocket tier¶

Trigger. WS connection count climbs, or API-pod memory rises from per-socket send buffers competing with request handling.
Playbook. Add a ws-hub Deployment (SERVER_MODE=ws-hub), route /ws to it at ingress, and flip the API pods to api-hub so they stop running the hubs. Both modes already exist in resolveMode and are reserved for exactly this split. The full-stack multi-hub mode stays the default until then.
Zero-downtime? Yes. Roll out the ws-hub Deployment first, move the route, then roll the API pods to api-hub. Cross-pod fan-out over Redis keeps delivery correct across the mixed fleet during the roll.
Effort. ~Days (mostly ingress routing and HPA tuning).
Seam. ws-hub / api-hub process modes.

Table partitioning¶

chat_messages is already PARTITION BY RANGE (created_at) with a chat_messages_default catch-all partition (backend/db/migrations/00001_init.sql). The proven pattern is: add time-window child partitions on a schedule (done at boot by ensureChatMessagePartitions in internal/migrate), keep autovacuum bounded per partition, and at large scale detach cold partitions to cheaper storage.

Retention policy (why there is no age-based prune). Chat history is core social data users expect to keep, so messages are retained, not aged out. The policy already lives in code: friend and DM messages persist until the user deletes their account (internal/services/chat/message_purger.go), event-chat messages are purged per that event's retention setting, and account deletion purges a user's chats and messages for compliance (internal/services/user/user_service.go). Partitioning here buys query pruning and vacuum relief, and later cheap archival, not deletion. So the scale step is archival (detach an old partition, move it to cold storage or a cheaper tablespace), never a blanket drop.

Trigger. A high-churn table's size or autovacuum pressure grows faster than the primary can comfortably vacuum.
Playbook (notifications, the next candidate). Notifications is blocked by its dedup design: idx_notifications_unique_open and idx_notifications_unique_unseen are partial-unique indexes on (user_id, group_key). A partitioned table requires the partition key (created_at) in every unique index, but adding created_at to these indexes breaks dedup, because the same open notification would become "unique" across time windows and stop merging. The fix, when the trigger fires, is either app-level dedup (an upsert guarded in the store instead of a partial-unique index) or a partition strategy that does not fight the dedup key. Until then, notifications stays a plain table.
Zero-downtime? Yes, when done as add-child-partition + background backfill.
Seam. The PARTITION BY RANGE pattern already in the schema.

Fan-out feed / timeline service¶

Trigger. Feed and discovery reads dominate DB load, and the read path can no longer be served cheaply from the primary plus replicas.
Playbook. Introduce precomputed timelines behind the existing discovery/feed read path. Because that path is a store/service interface, this is an interface swap: the handler and service stay, the store implementation changes from a live query to a read of a materialized timeline. Use a hybrid model, fan-out-on-write for normal accounts (push new items into follower timelines) and pull at read time for high-fan-out accounts (avoid writing to millions of timelines).
Zero-downtime? Yes. Ship the new store behind a flag, dual-read to compare, then cut over.
Effort. Weeks.
Seam. discovery/feed read path (backend/internal/services/discovery/store.go and its service).

WS directed routing, then presence sharding¶

Today every hub pod runs PSUBSCRIBE session:* (and client:*), so every pod receives every message and drops the ones it does not hold. That is fine at low pod counts and simple to reason about. It amplifies with scale: total Redis pub/sub traffic grows with pods x message rate.

Step 1 — directed routing. Pods subscribe only to the specific room channels they currently hold, instead of the wildcard pattern. This kills the fan-out amplification. It rewrites a tested delivery path (subscribe/unsubscribe on room join/leave, correct handling of the last member leaving), so it is deliberately just-in-time, not now.
Trigger. pod count x message rate makes wildcard fan-out a measurable share of Redis or pod CPU.
Step 2 — presence sharding. Shard the presence registry and route delivery directly to the pod holding a connection (a routing layer keyed by connection, not broadcast).
Trigger. Redis pub/sub itself saturates even with directed subscriptions.
Zero-downtime? Yes, rolling, but step 1 changes correctness-critical delivery code, so it ships with heavy testing (miniredis + real-stack).
Effort. 1-2 months across both steps.
Seam. PSUBSCRIBE fan-out in session_hub.go / client_hub.go.

Sharding¶

Trigger. Write throughput exceeds what a single primary plus read replicas can hold. This is the last resort for the write path, after replicas, pooling, partitioning, and feed fan-out.
Playbook. Shard the hot tables by a documented key (chat by chat_id, and similar per-table keys chosen so cross-shard queries stay rare). Use Citus (distributed Postgres, keeps the SQL surface) or app-level routing. Either way the change is contained in the store layer, because that is the only place SQL and connection selection live.
Zero-downtime? Only with care: online reshard, dual-write, backfill, cut over.
Effort. Weeks, and the carry cost is permanent and high. Cross-shard joins, rebalancing, and shard-aware migrations become a standing tax.
Seam. store layer.

Multi-region¶

Trigger. Global latency targets, data-residency requirements, or disaster-recovery needs that a single region can't meet.
Playbook. State is already externalized (Postgres, Redis, S3), so the app tier can run as regional stacks. Add per-region stacks, decide data locality and residency (which data is regional vs global), and add latency- or residency-aware routing at the edge.
Zero-downtime? Introducable region by region, but data locality is a design decision that must be right up front.
Effort. 3-6 months. Roughly 2-3x the infra bill and very high permanent carry (replication, routing, per-region ops).
Seam. externalized state.

Streaming + search cluster¶

Trigger. Asynq (Redis-backed queue) can't hold the event/fan-out volume, or PostGIS-plus-SQL discovery queries hit their latency ceiling.
Playbook. Add a streaming backbone (Kafka or equivalent) for high-volume event fan-out alongside the async event path, and a dedicated search cluster (a search engine) behind the discovery query path. Both are additive: the async handlers and the discovery store are the seams they attach to.
Zero-downtime? Yes, additive and behind the existing interfaces.
Effort. Weeks per subsystem.
Seam. async event path (backend/internal/async/), discovery query path.

Cost reality¶

Order-of-magnitude only. Build is lost feature velocity; Run is infra spend; Carry is the standing complexity tax after it ships.

Item	Build (eng-months)	Run ($/mo, o.o.m.)	Carry
Read replicas	days	$	Low
Connection pooler	days	$	Low
Dedicated WS tier	~days	small	Low
Table partitioning (notifications)	days	negligible	Low
Fan-out feed	2-3	$0.5-2k	Medium
WS directed routing + presence sharding	1-2	small-moderate	Medium
Sharding	2-4	$1.5-5k+	HIGH, permanent
Multi-region	3-6	2-3x current bill	VERY HIGH
Streaming + search cluster	weeks each	$1-3k	Medium-High

Takeaway. The cost of building early is mostly lost feature velocity plus permanent complexity, not the cloud bill. Every heavy item above waits for its trigger. Building sharding or multi-region before the metric demands it buys nothing and taxes every future change.

What to watch (SLOs / alerts)¶

These signals fire before the triggers above, giving lead time to build the next step. They live in the devops monitoring stack (Prometheus / pod monitors; the CNPG enablePodMonitor is already on), not in the app.

Signal	Warns of	Feeds trigger
DB primary CPU	read or write saturation	read replicas, sharding
Replica lag	replica can't keep up	read-replica read-budget
Connection count vs `max_connections`	pool exhaustion	connection pooler
WS connections per pod	socket-buffer memory pressure	dedicated WS tier
Redis pub/sub throughput	fan-out amplification	WS directed routing, presence sharding
Table size / autovacuum time	vacuum pressure	table partitioning
p99 request latency	end-to-end degradation	feed fan-out, search cluster
Asynq queue depth / latency	worker backlog	streaming backbone

The WS-connection-count metric is not yet exported; the API HPA notes it as a future custom metric via prometheus-adapter (see the comment in devops hpa-api.yaml). That exporter is a prerequisite for auto-triggering the dedicated WS tier.