Lessons from Scaling WebSockets

WebSockets: Essential for Realtime Apps, But Scaling Requires Careful Planning

The increasing demand for real-time, synchronized applications has made WebSockets a crucial component in modern software development. At Compose, WebSockets are the foundation of our service, enabling our backend SDKs to deliver low-latency interactive applications using only backend code. However, scaling WebSockets presents significant challenges. Here are key lessons learned:

Graceful Deployments: Maintaining Connection Persistence

Seamless deployments are crucial; users should never experience interruptions. To ensure WebSocket connection persistence during deployments, we employ a robust reconnection strategy:

1. New servers are launched.
2. Once healthy, old servers begin returning 503 (Service Unavailable) responses to health checks.
3. After four consecutive 503s, the load balancer removes the unhealthy servers. (Health checks occur every 5 seconds, for a maximum 25-second process.)
4. Old servers send a custom WebSocket close message, instructing clients to delay reconnection by a random interval, preventing a reconnection surge. This message includes:
   • A user-friendly message during the brief disconnection (~10 seconds).
   • A random delay to avoid thundering herd issues. Clients also exponentially increase the backoff for deployment-related reconnections.
   • A 20-second delay to allow the load balancer to redirect traffic.

Old servers shut down completely after client disconnections. Managed services like Render or Railway require special attention to ensure graceful client connection transfer during deployments. These services often wait for all requests to complete before shutting down, which can extend the downtime for persistent WebSocket connections significantly.

Consistent Message Schema: Defining Clear Communication

Unlike HTTP's built-in routing conventions, WebSockets require a custom message schema. At Compose, we use a 2-byte type prefix for message categorization:

• Space-efficient (only 2 bytes), scaling to 65,536 types.
• Allows clients to easily parse the type prefix without affecting data.
• Simplifies API upgrades through versioned message types.

const MESSAGE_TYPE_TO_HEADER = {
  RENDER_UI: "aa",
  UPDATE_UI: "ab",
  SHOW_LOADING: "ac",
  RENDER_UI_V2: "ad",
  /* ... */
};

We also use delimiters for separating message fields, improving encoding/decoding speed and memory efficiency compared to JSON.

const DELIMITER = "|";

function createDelimitedMessage(type: string, args: any[]) {
  return [MESSAGE_TYPE_TO_HEADER[type], ...args].join(DELIMITER);
}

function parseDelimitedMessage(message: string) {
  const [type, ...args] = message.split(DELIMITER);
  return { type, args };
}

Using TypeScript allows us to share message schemas between frontend and backend, preventing inconsistencies.

Heartbeat Mechanism: Detecting Silent Disconnects

Unexpected connection drops without close events can lead to stale connections. A robust heartbeat mechanism is essential:

• The server sends ping messages every 30 seconds, expecting pong responses.
• Clients disconnect and reconnect if a ping isn't received within 45 seconds.
• The server closes connections that miss pong responses within 45 seconds.

This bidirectional heartbeat monitoring detects and handles situations where the client's network appears functional, but the server doesn't receive responses.

HTTP Fallback: Handling Network Restrictions

WebSockets can be blocked on restrictive networks. Compose uses Server-Sent Events (SSE) as a fallback for receiving updates, and HTTP requests for client-to-server communication.

SSE's HTTP-based nature makes it less susceptible to blocking, offering a reliable alternative with relatively low latency.

Further Considerations

Scaling WebSockets involves additional complexities:

• Lack of standard tooling: Features like rate limiting and data validation often require custom implementation.
• Inability to cache responses: Unlike HTTP, WebSockets lack standard caching mechanisms.
• Per-message authentication: Ensuring each message's validity before processing is crucial for security.

Despite these challenges, WebSockets remain the optimal solution for building fast, real-time, and collaborative applications. At Compose, WebSockets power our entire platform, enabling developers to create full web applications from backend logic using our SDKs. Learn more in our documentation.

The above is the detailed content of Lessons from Scaling WebSockets. For more information, please follow other related articles on the PHP Chinese website!