Architecting a Secure, Resilient Node‑RED MQTT Ingest Flow for Time‑Series Storage

Problem Statement

When a fleet of IoT sensors publishes data to an MQTT broker, a common engineering challenge is to reliably ingest that data into a time‑series database (TSDB) using Node‑RED. The ingest pipeline must honour message delivery guarantees, protect credentials, and survive restarts without data loss. This article walks through the functional requirements, the smallest design that satisfies them, how to delineate trust boundaries, what operational checks to run, the failure modes to watch for, and when you should consider redesigning the flow.

Functional Requirements

1. Subscribe to one or more sensor topics.
2. Persist each payload to a TSDB (e.g., InfluxDB, TimescaleDB).
3. Guarantee at least once delivery (QoS 1) for critical sensors.
4. Encrypt traffic and store broker credentials securely.
5. Maintain state (e.g., counters, last‑seen timestamps) across Node‑RED restarts.
6. Expose a simple, reusable subflow that can be dropped into other projects.

Minimal Design

The core of the design is a subflow that bundles an mqtt in node, optional function nodes for payload transformation, a time‑series write node (custom or third‑party), and a debug node for troubleshooting. The subflow is parameterised with:

• brokerUrl
• topicPattern
• qos
• tlsOptions
• dbWriteConfig

Node‑RED’s context storage is configured to use the file storage option, which writes a JSON file to disk on each context change. This guarantees persistence across restarts. If your deployment requires high availability, switch to redis or mongodb storage.

Trust and Data Boundaries

Component	Trust Boundary	Security Controls
MQTT Broker	External	TLS 1.2+, username/password, client‑cert auth if needed
Node‑RED Runtime	Internal	Admin password, credential encryption, sandboxed function node execution
TSDB	Internal/External	TLS, role‑based access, network segmentation

All sensitive data (broker passwords, TLS certificates, DB credentials) are stored in credentials.json, which is encrypted automatically when Node‑RED starts with an admin password. Without that password, the file is plain text and should never be committed to source control.

Operational Checks

1. Broker Connectivity – Verify that the mqtt in node shows a green status and that a debug node receives messages. Run nc -vz broker.example.com 8883 from the Node‑RED host to confirm network reachability.
2. TLS Validation – Inspect the broker’s certificate chain with openssl s_client -connect broker.example.com:8883 -showcerts and confirm it matches the CA in the Node‑RED TLS options.
3. Context Persistence – After a flow change that writes to context (e.g., incrementing a counter), restart Node‑RED and check that the counter value is restored by querying the context API: curl -X GET http://localhost:1880/contexts/flow/counter.
4. Message Delivery – Publish test messages with QoS 0, 1, 2 from a separate client and confirm that Node‑RED receives them as configured. Use mosquitto_pub -h broker.example.com -t sensors/temperature -m "{\"temp\":22}" -q 1.
5. TSDB Write Confirmation – After a message is processed, query the TSDB to ensure the record exists: e.g., curl -G http://localhost:8086/query?pretty=true --data-urlencode "q=SELECT * FROM temperature ORDER BY time DESC LIMIT 1".

Failure Modes

• Broker Unreachable – The flow will sit idle; set a mqtt in node timeout or implement a watchdog that restarts Node‑RED if connectivity is lost for >30 s.
• Message Loss (QoS 0) – If your sensor cannot tolerate loss, avoid QoS 0 or add a local queueing mechanism in the function node.
• Function Node Blocking – Long‑running JS blocks the event loop. Keep transformations lightweight or off‑load heavy work to a child process.
• Context File Corruption – Corrupted flow.json or context.json can prevent Node‑RED from starting. Keep regular backups and consider a file‑system with journaling.
• Credential Exposure – Running Node‑RED without an admin password writes credentials in plain text. Enforce the password in CI/CD pipelines.

Redesign Triggers

1. Throughput > 10 kmsg/s – The single‑threaded Node‑RED loop will become a bottleneck; move processing to a micro‑service or use Node‑RED’s worker nodes.
2. Data Volume > 1 TB/month – The current TSDB write node may not scale; consider batching writes or using a dedicated ingestion service.
3. Security Breach – If TLS certificates are compromised, rotate them and update the Node‑RED TLS options immediately.
4. Schema Evolution – Adding new fields to the payload that the TSDB does not recognise may require a new subflow version.

Concrete Example Flow

Below is a minimal flow JSON that implements the design. Replace the placeholder values with your environment details.

{
  "nodes": [
    {
      "id": "mqttIn",
      "type": "mqtt in",
      "name": "Sensor Ingest",
      "topic": "sensors/+/temperature",
      "qos": 1,
      "broker": "mqttBroker",
      "x": 160,
      "y": 80,
      "wires": [["transform"]]
    },
    {
      "id": "transform",
      "type": "function",
      "name": "Parse & Enrich",
      "func": "// Parse JSON payload\nmsg.payload = JSON.parse(msg.payload);\n// Add timestamp if missing\nif (!msg.payload.time) { msg.payload.time = new Date().toISOString(); }\nreturn msg;",
      "x": 360,
      "y": 80,
      "wires": [["writeTSDB"]]
    },
    {
      "id": "writeTSDB",
      "type": "influxdb out",
      "name": "InfluxDB Write",
      "influxdb": "influxdb1",
      "measurement": "temperature",
      "x": 560,
      "y": 80,
      "wires": []
    },
    {
      "id": "debug",
      "type": "debug",
      "name": "Debug",
      "active": true,
      "tosidebar": true,
      "console": false,
      "complete": "false",
      "x": 360,
      "y": 140,
      "wires": []
    }
  ],
  "connections": [],
  "subflow": {
    "id": "mqttIngestSubflow",
    "name": "MQTT Ingest Subflow",
    "in": [{"type":"mqtt in"}],
    "out": [{"type":"influxdb out"}]
  }
}

Save this as mqtt-ingest.json and import it into Node‑RED. The mqtt in node references a broker configuration defined in settings.js:

module.exports = {
  mqtt: {
    servers: [
      {
        url: 'mqtts://broker.example.com:8883',
        options: {
          username: 'sensor_user',
          password: 'p@ssw0rd',
          rejectUnauthorized: false // replace with true in production
        }
      }
    ]
  }
}

Summary

By encapsulating the MQTT input, optional transformation, and TSDB write into a parameterised subflow, you achieve a clean, reusable design. Enforcing TLS, encrypting credentials, and persisting context ensures security and resilience. Regular operational checks guard against connectivity, delivery, and persistence issues. When throughput, volume, or security demands exceed the current limits, the design flags clear points for redesign.