Real-time Data Streaming at Scale: Architecture Patterns

Real-time at scale is a fan-out problem. One event becomes ten thousand deliveries, so the architecture that matters is the one between your data source and the socket.

Push, do not poll

Polling multiplies load by client count regardless of change rate. Server-sent events cover one-way feeds with plain HTTP, WebSockets cover bidirectional traffic, and both beat polling by orders of magnitude.

1export async function GET() {
2  const stream = new ReadableStream({
3    start(controller) {
4      const send = (event: string) =>
5        controller.enqueue('data: ' + event + '\n\n');
6      bus.subscribe('scores', send);
7    },
8  });
9  return new Response(stream, {
10    headers: { 'Content-Type': 'text/event-stream' },
11  });
12}

Absorb bursts with queues

Spikes will exceed what your consumers handle. A message queue between ingestion and delivery smooths bursts, and per-channel batching turns ten thousand socket writes into far fewer.

Design for the burst, not the average. Averages never page you at 3am.

Pick the right transport

• •Server-sent events for one-way feeds over plain HTTP with automatic reconnects.
• •WebSockets for bidirectional traffic like chat, presence, and live collaboration.
• •Short polling only as a fallback, and never as the primary strategy.

Scale out with pub and sub

One server cannot hold ten thousand sockets forever. Publish events to a shared bus and let every edge node subscribe, so any node can serve any client and capacity grows by adding identical nodes.

1bus.subscribe('scores', (event) => {
2  for (const socket of room.sockets(event.matchId)) {
3    if (socket.bufferedAmount < MAX_BUFFER) socket.send(event.payload);
4  }
5});

Notice the backpressure check: a slow client must never block the room. Drop or coalesce for laggards, keep the fast path fast, and log who fell behind so capacity planning has data instead of guesses.