The Socket Hub
Live updates without the polling tax — thousands of connections, one disciplined hub.
The incident
The chat feature shipped with a one-second polling loop. It worked in the demo. At ten thousand users it was ten thousand requests per second asking 'anything new?' — and the answer was almost always no. The servers were busy, the batteries were dying, and nothing was actually happening.
Why the obvious approach fails
Polling inverts the economics of real-time: cost scales with users, not with events. Most polls return nothing, so you pay full price for empty answers. The obvious fix — open a WebSocket per user — creates its own trap when done naively: connection state ends up in server memory, which breaks the moment you run two servers, and presence tracked in the primary database turns 'who's online' into your hottest write path. Real-time at scale is a routing problem: connections are cheap only if the state behind them lives somewhere fast and shared, and messages travel only to the people who need them.
The standard
- Real-time updates flow over WebSockets through a dedicated hub; polling loops for live data are banned.
- Presence and connection state live in Redis, not in server memory and never in the primary database.
- Broadcasts are room-partitioned: users subscribe to specific channels, and messages fan out only to subscribers — never to the whole fleet.
- Heartbeats detect dropped connections; reconnection replays missed messages, so a tunnel ride doesn't mean lost updates.
- The hub scales horizontally behind a pub/sub backbone — two hub nodes or twenty, the same message reaches the same rooms.
How Cortex validates it
At plan time
any feature with live-update requirements declares its channel model and presence strategy; a polling design for live data fails plan review.
At validation
assert: live_updates.transport == websocketassert: presence_store == redis (not primary_db)assert: reconnect → missed_messages.replayedAssertion syntax is illustrative.
At review
the approver sees the reconnection drill: connections dropped mid-stream, clients recovering with full message history, and server load flat while events spike.