Choosing Between SurrealDB Live Queries, Polling, and Webhooks for Real-Time Sync

Decision: How to receive SurrealDB changes in real time

When an application needs to react to data modifications immediately, you must choose a synchronization strategy based on your latency requirements and infrastructure constraints. SurrealDB supports three primary patterns:

• Live Queries: A server-side subscription that pushes document changes to clients via WebSocket or HTTP long-poll.
• Periodic Polling: The client repeatedly executes a standard query on a fixed timer.
• Webhook Callbacks: SurrealDB's changefeed triggers an HTTP POST request to a specified external endpoint when a transaction commits.

The following table compares these options across key operational dimensions.

Approach	Typical Latency	Server Overhead	Client Complexity	Ordering
Live Queries	Milliseconds	High (Persistent connections)	Low (Event-driven)	Preserved per connection
Polling	Interval + RTT	Low (Stateless HTTP)	Medium (Timer/Diffing)	None (Independent reads)
Webhooks	Variable (<1s)	Low (Push-on-commit)	Medium (Endpoint hosting)	Not guaranteed

Trade-off Analysis

• Use Live Queries for highly interactive UIs (e.g., collaborative dashboards or chat apps) where sub-second updates are critical and the client can maintain a stable WebSocket.
• Use Polling for low-frequency updates, background tasks, or environments where strict firewall rules block persistent WebSocket connections.
• Use Webhooks for decoupled microservices or serverless architectures where the data change must trigger a side effect in a different system (e.g., sending an email after a record is created).

Implementing a Live Query via WebSocket

To implement a Live Query, the client establishes a connection to the SurrealDB RPC endpoint and sends a subscription request. The server then pushes any record that matches the query criteria as it changes.

Prerequisites

• SurrealDB v1.x instance running with the WebSocket endpoint enabled (default: ws://localhost:8000/rpc).
• A user account with SELECT permissions on the target table.

Implementation Example (Node.js)

const WebSocket = require('ws');

// Connection details
const WS_URL = 'ws://localhost:8000/rpc';
const LIVE_QUERY = 'SELECT * FROM user WHERE active = true';

function startLiveQuery() {
  const ws = new WebSocket(WS_URL);

  ws.on('open', () => {
    console.log('Connected to SurrealDB');
    // Subscribe to changes
    const subMsg = {
      method: 'live',
      params: { query: LIVE_QUERY }
    };
    ws.send(JSON.stringify(subMsg));
  });

  ws.on('message', (data) => {
    const msg = JSON.parse(data.toString());
    // Results arrive as a 'result' array containing the changed documents
    if (msg.result) {
      console.log('Data change detected:', msg.result);
    }
  });

  ws.on('error', (err) => console.error('WS Error:', err));
  return ws;
}

const session = startLiveQuery();

// To stop the subscription and free server resources:
// session.close();

Verification and Validation

To verify the implementation, follow these steps in order:

1. Start SurrealDB: surreal start --log trace --bind localhost:8000.
2. Execute the Node.js script above.
3. In a separate terminal, use SurrealQL to insert a matching record:
   CREATE user SET name = 'Test User', active = true;
4. Check Result: The Node.js console should display the 'Data change detected' message immediately.
5. Check Server Logs: Verify that the logs indicate a new live query subscription was registered.
6. Latency Check: In a local environment, the time between the CREATE command and the client log should be under 100ms.

Operational Limitations

• Connection Scaling: Each Live Query consumes a file descriptor and memory. For thousands of concurrent users, you must implement a load balancer or utilize SurrealDB's enterprise clustering to distribute the connection load.
• Idempotency: During network partitions, you may receive duplicate or slightly out-of-order events. Ensure your client-side logic is idempotent (e.g., using a version field or upsert logic) to avoid data corruption.
• Orphaned Subscriptions: If a client crashes without sending an unlive request or closing the socket, the server maintains the subscription until a timeout occurs. Always implement a clean shutdown sequence.