Using SurrealDB Live Queries for Real‑Time Updates

Problem: avoiding constant polling for data changes

Applications that need to show database changes instantly—such as collaborative editors, live dashboards, or multiplayer game states—often resort to polling the server every few seconds. This approach wastes bandwidth and introduces latency between a change occurring and the UI reflecting it.

Thesis: SurrealDB Live Queries provide a built‑in mechanism to push only the changed rows to subscribed clients, respecting permissions and automatically attempting to recover from brief network interruptions.

How Live Queries work

A Live Query is created with SurrealQL:

START LIVE QUERY name ON table WHERE condition;

The server maintains a subscription linked to the client’s connection (WebSocket or HTTP long‑poll). When a row matching the WHERE clause is inserted, updated, or deleted, the server sends a message describing the change. The exact payload format is documented as a JSON object containing the record ID, the operation type, and the relevant field values, but you should consult the version‑specific documentation for the precise structure.

Worked example: setting up a Live Query locally

1. Start a disposable SurrealDB instance in memory (run in a terminal):

   surreal start --log trace --user root --pass root memory:

   This binds the HTTP/RPC endpoint to localhost:8000.

2. Create a table and insert a few rows via the SurrealDB CLI (or any HTTP client):

   surreal sql -w ws://localhost:8000/rpc -u root -p root
   DEFINE TABLE person SCHEMAFULL;
   DEFINE FIELD name ON person TYPE string;
   CREATE person SET name = 'Alice';
   CREATE person SET name = 'Bob';
   

3. Open a second terminal and connect a WebSocket client to the same endpoint. The SurrealDB CLI can act as a WS client:

   surreal sql -w ws://localhost:8000/rpc -u root -p root
   

   Once the prompt appears, start the Live Query:

   START LIVE QUERY person_changes ON person WHERE true;

   The CLI should output a confirmation that the live query has started. Subsequent changes generate JSON messages; for example, inserting a new row may produce a message similar to:

   { "id": "person:ja8v9k2l3m4n5o6p", "diff": { "op": "INSERT", "data": { "name": "Charlie" } } }
   

   Try inserting another row in the first terminal:

   CREATE person SET name = 'Dana';

   Observe the second terminal receive a diff message reflecting the new row.

4. Verify permission enforcement: create a role that lacks SELECT on person and attempt the Live Query with a token for that role. The server should reject the query with an authorization error instead of establishing the subscription.

   DEFINE ROLE reader;
   DEFINE PERMISSIONS FOR SELECT ON person WHERE false;
   DEFINE ACCESS reader ON ROLE reader;
   # authenticate as a user with only the reader role, then run the START LIVE QUERY command
   

   If the query is accepted, double‑check that the role actually has SELECT rights.

5. Test resilience to network interruption: in the WS client terminal, suspend the network (e.g., sudo ifconfig lo down on Linux or disconnect Wi‑Fi), insert a couple more rows on the server, then restore the connection. The client should receive a series of diff messages that replay the missed changes in order, assuming the server’s change feed retains them.

Trade‑offs and limitations

• Message volume: Each individual write triggers a separate notification. Under high‑frequency workloads (thousands of writes per second) this can increase network traffic and client memory usage. Consider batching changes on the server side or throttling at the client if your use case cannot tolerate the raw stream.

• Delivery guarantees: Live Queries aim for at‑least‑once delivery and automatically resend missed changes after a reconnection, but they do not guarantee exactly‑once delivery under all failure scenarios (e.g., if the server crashes before persisting a change). Design your client to be idempotent or to use a checksum/version field to detect and discard duplicates.

• Transport choice: WebSocket provides low‑latency push, while HTTP long‑polling is a fallback for environments where WS is blocked. The fallback introduces higher latency and additional HTTP overhead.

Actionable closing

If your application needs real‑time updates and can tolerate the per‑write message overhead, start by testing Live Queries locally with the steps above. Verify that permission rules correctly filter notifications and that your client handles reconnection gracefully. Once satisfied, integrate the WS endpoint into your frontend or service, and monitor message rates in staging to decide whether batching or throttling is necessary.