Skip to content

WebSocket Hubs

Real-time delivery runs over two WebSocket hubs, both in backend/internal/platform/ws:

Hub Scope Channel Route Carries
SessionHub a multi-user session (chat room, future collab) session:{id} GET /ws/chats/:id chat messages, reactions, read receipts, join/leave
ClientHub one user across all their devices client:{userId} GET /ws/client notifications, presence, profile / friend-graph sync

Both are per-pod brokers that own local connections and fan out across pods over a pluggable backplane. The backplane is selected by WS_BACKPLANE (nats or redis); deployed environments and local dev default to NATS (the nats compose service, NATS_URL), with Redis pub/sub as the alternative. Subjects are dot-namespaced (session.<id>, client.<id>); the Redis backplane maps dots to colons. Source: session_hub.go, client_hub.go, nats_backplane.go, redis_backplane.go. The SessionHub upgrade path is chat.Handler.HandleWebSocket; the ClientHub upgrade path is chat.ClientHubHandler.Handle.

Mental model

SessionHub groups connections by session; a message goes to everyone in the session. ClientHub groups connections by user; a message goes to every device that one user has open. Everything else (registry loop, ping/pong, Redis fanout, backpressure) is shared shape.

SessionHub
sessions[id] → Session
↓ fan out to members
Client (per socket)
ClientHub
users[id] → userScope
↓ fan out to devices
ClientConn (per socket)
↓ Broadcast also publishes to Redis
Redis pub/sub
PSUBSCRIBE session:* and client:*
Each hub tags outbound envelopes with its podID and drops loopback deliveries, so a message crosses pods exactly once.

Process modes and the hubs

The single tomoda-backend binary chooses what to start from --mode (or SERVER_MODE). resolveMode maps each mode to runAPI / runWS / runAsync; runWS starts both hubs (go app.SessionHub.Run() + go app.ClientHub.Run()).

Mode API /api/v1 WebSocket /ws/* Async (worker + scheduler)
full ✓ ✓ ✓
multi-hub ✓ ✓
async ✓
api-hub ✓
ws-hub ✓

api-hub / ws-hub are reserved for a future API / WS deployment split. Local dev defaults to full. In production the API deployment runs multi-hub and the async deployment runs async.

Connection lifecycle

The /ws group is gated by middleware.JWTAuth(app.AuthService). Browsers can't set headers on the WebSocket handshake, so the client appends the JWT as a ?token= query param (redacted in access logs by router.go). The middleware accepts a query token only under /ws/; every other route requires the Authorization header. After the upgrade:

  • SessionHub binds the :id path param to the client's SessionID for the connection's lifetime, one socket to one session.
  • ClientHub binds the authenticated userID; a user with three devices holds three ClientConn in one userScope.

Each connection runs two goroutines: ReadPump (reads frames, resets the read deadline on pong) and WritePump (drains the Send channel, sends a ping every 54s).

Ping / pong

Tunables shared by both hubs (session_hub.go):

writeWait  = 10 * time.Second
pongWait   = 60 * time.Second
pingPeriod = (pongWait * 9) / 10  // 54s

The server sends a WebSocket ping every 54s; the peer must produce any read (pong or data) within 60s or the socket closes. SessionHub clients may also send an application-level { "type": "ping" } and get { "type": "pong" } back.

Broadcasting and cross-pod fanout

Broadcast is a buffered channel (capacity 256). For each message the hub:

  1. Local fanout — pushes to every matching Send channel on this pod, skipping the excluded connection (the sender) when set.
  2. Remote publish — PUBLISH to the scope's Redis channel with an envelope carrying the origin podID, the exclude hint, and the payload.

Every pod runs a PSUBSCRIBE loop (session:* or client:*). On each envelope it drops the message if opid == podID (already delivered locally, the dedup contract), otherwise fans out to local connections. Because the originating *Client / *ClientConn pointer doesn't exist on sibling pods, the sender-skip is keyed by UserID (session) or connection ID (client) on the receive path.

ClientHub.Emit(userID, payload) is the service-facing entrypoint: notification, presence, and profile services call it and let the hub handle local fanout plus cross-pod publish.

Backpressure

If a connection's Send channel is full, the hub closes it and evicts the connection rather than block the broadcast loop. The WritePump exits and the socket closes; the client reconnects. This is the only backpressure mechanism: it prefers dropping a slow client over stalling everyone.

Reconnect

The subscriber reconnects with a 2-second backoff on transient Redis errors. Messages published during the gap are lost (Redis pub/sub is at-most-once); chat is already persisted by the chat service, so clients recover history on reconnect.

Empty-scope cleanup

SessionHub deletes a Session once its last client leaves; ClientHub deletes a userScope once a user's last device disconnects. Without this, a process that handled millions of distinct sessions or users would leak per-scope memory forever.

Message types

The on-wire protocol is JSON. SessionHub deserialises incoming frames into models.WSMessage (backend/internal/models/chat_message.go):

type WSMessage struct {
    Type MessageType `json:"type"`
    Data any         `json:"data"`
}
Type Direction Purpose
send_message client→server Persist + broadcast a new chat message
ping / pong both Application-level keepalive
error server→client Server-side error
user_joined / user_left server→client Membership change
mark_read both Read-receipt advancement
reaction_update server→client Reaction added / removed
message_updated / message_deleted server→client Edit / delete propagation
messages_expired server→client Disappearing-message purge

SessionHub answers ping itself; everything else is forwarded to ChatHandler, which decides whether to persist + broadcast or drop. ClientHub is server-driven: its ReadPump only services keepalive, so inbound frames are logged and ignored. Matching TypeScript shapes live in frontend/services/chatService.ts.

Presence is a separate channel

The hubs carry socket-bound traffic. Presence ("is the user online?") and active location sharing run over short HTTP requests to /api/v1/presence/*, backed by Redis keys with TTLs, and do not touch either hub. The split is deliberate: presence updates are high-frequency, idempotent, and must survive reloads, none of which fit a connection-bound socket. See backend/services/presence.

Frontend connection

The chat client opens a socket when it enters a room and closes it on leave (frontend/services/chatService.ts):

const wsUrl = `${WS_URL}/chats/${chatId}`;
this.ws = new WebSocket(`${wsUrl}?token=${token}`);

WS_URL resolves from EXPO_PUBLIC_WS_URL and already includes the /ws prefix. Reconnect is attempted with backoff on abnormal closes.

Horizontal scaling

Both hubs support replicas > 1 via the Redis fanout above. Ingress session affinity is not required: a client can connect to any pod and still receive messages from senders on other pods.

Per-pod counts

SessionHub.GetOnlineCount and ClientHub.ConnectedCount report this pod only. For a cross-pod number, use the chat:online:* Redis keys the chat service maintains.

For the system-level view see Architecture → Real-time.