Pagination state management with Function node and Context in Node-RED

Answer to the Core Question

Use flow‑level context to store a pagination cursor that is shared by multiple Function nodes in the same flow. Flow context is the only scope that guarantees visibility across nodes that belong to the same flow instance.

Concurrency Risk with Flow Context

Storing a single cursor value in flow context does not cause unbounded memory growth, but it can lead to race conditions when several messages update the same key concurrently. The last write wins, so intermediate updates may be lost. To avoid this, store per‑message state under a unique key (e.g., context.set(msg.id + ‘_cursor’, value, ‘flow’)) or use a queueing strategy.

Memory vs. File‑Based Flow Context

Both stores expose the same API (context.get/set), but they differ in persistence and performance:

• Memory‑based – volatile; cleared on Node‑RED restart. Startup is fast because no disk I/O is required.
• File‑based – persists across restarts; the context JSON is written to disk on each write. Startup latency increases proportionally to the size of the context.

Switching the store type does not change how your code accesses the cursor; you simply change the contextStorage configuration in settings.js. The cursor value will survive a restart only when file‑based storage is enabled.

Practical Steps for This Scenario

1. Configure context storage in settings.js:

   contextStorage: {
       flow: {
           module: "node-red-contrib-context-storage",
           name: "file"
       }
   }

2. Set the cursor in a Function node:

   var cursor = msg.payload.cursor;
   context.set("paginationCursor", cursor, "flow");
   return msg;

3. Read the cursor in downstream nodes:

   var cursor = context.get("paginationCursor", "flow");
   msg.payload.cursor = cursor;
   return msg;

4. When handling multiple concurrent requests, prepend a unique key:

   var key = msg.id + "_cursor";
   context.set(key, cursor, "flow");

Diagnostic Detail Needed

To fine‑tune the strategy, please confirm whether the flow processes messages that share the same msg.id or whether each message has a distinct identifier. This determines whether a single shared cursor or per‑message keys are required.