Tomoda WS Pool Split (Planned)¶
Planned topology, not deployed: split the WebSocket Hub out of tomoda-api
into its own pool (tomoda-ws). The backend already ships the api-hub and
ws-hub modes; what's missing is the k8s wiring, a path-based ingress rule
and a WS-pool HPA. Don't pre-build it.
Why split¶
Today HTTP API and WebSocket termination share the tomoda-api pool
(multi-hub). Its HPA scales on CPU, memory and WS connections per pod, so one
pool has to be sized for two load shapes.
Splitting tomoda-api into tomoda-api (HTTP only) and tomoda-ws
(WebSocket only) gives us:
- Independent scaling shape. WS pool scales on connection count (long-lived, sticky, memory-bound). API pool keeps scaling on CPU/RPS.
- Different pod sizing. WS pods can be sized for connection state (more memory, less CPU). API pods stay lean.
- Blast-radius reduction. A WS-related rollout (Hub change, WebSocket library upgrade) doesn't touch the API pool, and vice versa.
The cost is more moving parts: another Deployment, another Service, an ingress-level routing rule, and a custom WS-connection metric for the HPA.
When to do this¶
Trigger criteria, any of which is sufficient:
- WS-connection count regularly exceeds 10k during peak.
tomoda-apiHPA spends meaningful time pinned atmaxReplicaswhile the HTTP request-rate metrics show idle pods. That gap is WS-driven load not visible to CPU-based scaling.- WS-related changes cause API-side incidents (a Hub bug bricks the API pool too) more than once.
Until one of these fires, the multi-hub topology is correct.
Target topology¶
Both pools run the same image. The Hub on tomoda-ws keeps the
NATS backplane (WS_BACKPLANE=nats) that tomoda-api uses
today for cross-pod fanout.
tomoda-async (worker + scheduler) is untouched.
Manifest sketch¶
In k8s/apps/tomoda/base/:
# backend-ws-deployment.yaml: Deployment tomoda-ws (mode ws-hub)
# Service ws-service (clusterIP, port 8080, selector app: tomoda-ws)
# backend-api-deployment.yaml: change SERVER_MODE multi-hub → api-hub
# (api-hub serves /api/v1/* + /health, no /ws/* routes, no Hub)
Update kustomization.yaml to add the new file.
In network-policy.yaml, add:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: tomoda-ws-policy
spec:
podSelector:
matchLabels:
app: tomoda-ws
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: traefik-system
ports:
- protocol: TCP
port: 8080
In ingress.yaml, replace the single-path backend-service rule with two
paths, /ws/* → ws-service and default → backend-service:
- host: api.tomoda.life
http:
paths:
- path: /ws
pathType: Prefix
backend:
service:
name: ws-service
port: { number: 8080 }
- path: /
pathType: Prefix
backend:
service:
name: backend-service
port: { number: 8080 }
Traefik picks the longer prefix first, so /ws/chat/... lands on the
WS pool and everything else stays on the API pool.
HPA: custom WS connection metric¶
prometheus-adapter already exposes
tomoda_ws_connections_active per pod (the prod tomoda-api HPA scales on
it); ws-hub pods emit it too. Reference it from tomoda-ws-hpa:
spec:
metrics:
- type: Pods
pods:
metric:
name: tomoda_ws_connections_active
target:
type: AverageValue
averageValue: "500" # scale to keep <=500 conns/pod
Tune averageValue from steady-state telemetry.
Migration plan (when triggered)¶
Sequence so we don't drop connections:
- Land the new
tomoda-wsDeployment + Service in the cluster (replicas 0). Verify it builds and reconciles via Argo CD without any traffic. - Bring it to 2 replicas, confirm
/healthsucceeds on the new pods. - Add the ingress
/wspath pointing atws-service. New WS connections start landing on the WS pool; existing connections remain ontomoda-apipods until they reconnect (Hub is per-pod; old pods still own their sockets). - Flip
tomoda-apifrommulti-hubtoapi-hub. Existing sockets on api pods are torn down on the rollout; clients reconnect to the WS pool via the new ingress path. - Drop unused metrics / HPA from
tomoda-api; deploy custom-metric HPA fortomoda-ws.
Before step 1, verify the frontend reconnect path handles the rolling pod swap
(see docs/architecture/realtime.md in the tomoda repo).