Configuring the Node-Red MQTT Publish and Subscribe Nodes

node-red-mqtt-publish-subscribe

Node-Red provides both an MQTT subscribe (input) and publish (output) node.

The configuration for these nodes are almost Identical as the main part of the configuration concerns the actual client connection.

Because of this it is useful to think of the publish and subscribe nodes as consisting of two components as shown in the schematic below:

node-red-mqtt-subscribe-publish-schematic

Before we look a the actual configuration we will look at MQTT client connections in general.



How MQTT works

If you are unfamiliar with the MQTT protocol I would recommend that you take a read of how MQTT works before Continuing

Connecting to an MQTT Broker or Server.

To connect to an MQTT broker or server you need to know the Broker IP address or name, and the port that it is using (default 1883).

In addition a client needs to provide a client name to identify itself. Each client connection requires a unique client name.

A single client connection can be used to both publish and subscribe.

An MQTT broker can enforce encryption and username/password authentication in which case the connecting client must comply.

There are various flags and parameters that a client can set. These are:

  • Clean Session Flag
  • Time to Live
  • Last Will and Testament

Clean Session Flag

MQTT clients will usually by default establish a clean session with a broker.

A clean session is one in which the broker isn’t expected to remember anything about the client when it disconnects.

With a non clean session the broker will remember client subscriptions and may hold undelivered messages for the client.

Time to Live

This is a mechanism used to determine if a connection is still present. The default interval is 60 seconds.

Last Will and Testament

The idea of the last will message is to notify a subscriber that the publisher is unavailable due to network outage.

The last will message is set by the client on a topic and is stored on the broker.

It Is sent to all subscribers if the publishing client disconnects abnormally.

If the publisher disconnects normally the last Will Message is not sent.

By default the node connects automatically when you start the flow, however node-red now supports dynamic connections that allow you to control the connection.

Publishing MQTT Messages- The Publish Node

When a client publishes a message to a broker it needs to send:

  • The message topic
  • The message QOS
  • Whether the message should be retained.- Retain Flag

The retain Flag is normally set to False which means that the broker doesn’t keep the message.

If you set the retain flag to True then the last message received by the broker with the retained flag set will be kept.

The QOS of the published message has no effect on the retained message.

The retain flag setting is the only setting that is unique to the publish node.

However the publish topic and QOS settings in the publish node are applicable only to the publish node.

To publish a Message to an MQTT broker you use the MQTT publish node.

To use a publish node drag the node from the node palette on the left into the main workspace.

mqtt-publish

Double click on the node to configure it. The main configuration screen is shown below:

node-red-mqtt-publish-config-1.

The main setting is the broker or server setting.

You can select a previously configured server which you can edit by clicking on the edit button, or add a new broker by selecting the add new mqtt server option from the server list.

mqtt-node-add-broker

The server name that appears in the server list is a combination of the client ID and Broker name or IP address.

Note: The MQTT specification originally used the term broker but it has now been changed to server.

You can also configure the topic address,the QOS for the message and the retain flag.

The topic address,the QOS and the retain flag can also be provided by the preceding node as part of the message object using:

  • msg.topic =mytopic
  • msg.qos =0,1 or 2
  • msg.retain =true or false

set-mqtt-publish-settings

Publish Message Format

The message to be published can be either a JavaScript Object string or a binary buffer.

If you pass in a JavaScript object it is automatically converted to a JSON encoded string before being published.

If you pass in numeric data it is converted to a string and if you pass in Boolean data it is also converted into a string before being published.

Editing the Server Settings

If you click on the edit icon to the right of the server you can edit the server settings.

The server settings has 4 tabs.

  • connection tab
  • Security
  • Birth Message
  • Will Message

Connection Tab

broker-settings-node-mqtt

The port defaults to 1883 which is the standard MQTT port.

You can use either the IP address or FQDN for the server address.

The client name needs to be unique on the broker.

You can configure the client to use SSL for the connection by enabling the TLS box- see later

The default keep alive is 60 secs and clean session is False. It is normal to enable clean sessions.

Security Tab

This lets you configure  a user name and password for the connection.

security-tab-mqtt-connection

Whether you need a username/password for the connection is determined by the MQTT broker.

Birth Message

The birth message is published by the client when the MQTT node starts.

node-birth-message

A birth message is not part of the MQTT specification but it is part of the Sparkplug specification.

Will Message

The Will message is part f the MQTT specification and is stored on the broker, and sent in the advent of a failed client connection.

will-message-node-mqtt

The message payload will either be a message to indicate the connection has failed or a status indicator.

The retained message will usually be set so that new clients know the status.

Using SSL for Encryption

The client supports SSL encryption between the client and server as well as using certificate based authentication.

To configure a secure connection you will need enable TLS

tls-node-red-mqtt

and add the CA certificate to node-red by uploading it. In the screen shot below I used the same self generated certificate as I used on Mosquitto broker.

upload-certificate-node-red

If the CA file is on your local computer you can use it by enabling the use key and certificates from local files option., and entering the path and file name:

ssl-node-red-mqtt-local-files

Alternatively you can upload the files using the upload buttons.

Note: the verify server certificate checks the the name on the certificate is correct and should be enabled on live systems. Leaving it un-checked is like using the -insecure option of the mosquitto_pub /_sub tool. Entering a name in the box doesn’t seem to make any difference.

If you are using Client certificates then you will also need to upload or provide the pathname to the client key and certificate as shown below:

client-certificates-node-red

SSL Public Certificates

If you are using a public broker like cloudmqtt then you will need to use a public CA.

You will need to get a copy of ca-certificates.crt or ca-certificates.pem file.

This file contains a list of trusted CA certificates and is available from Mozilla.

If you are using Linux it is usually in /etc/SSL/certs, but you can download it from the curl site here.

You will then need to upload it, just like with the self generated certificate as shown in the screen shot below.:

cloudmqtt-ca

Configuring The Subscribe Node

When a client subscribes to a broker it needs to provide a topic and QOS.

The subscribe node has no unique settings.

However again the the topic and QOS settings apply only to the subscribe node and they need to be configured in the node as it doesn’t accept an input from other nodes.

node-red-mqtt-subscribe-iconTo use a subscribe node drag the node from the node palette on the left into the main workspace.

Double click on the node to configure it. The main configuration screen is shown below, and is almost identical to the publish node.

Node-Red-MQTT-Subscribe-Node-Configuration

Note: You don’t currently appear to be able to subscribe to multiple topics using the same node.

Receive Payload

Prior to version .20 the received payload was either a string or buffer object (if detected by the node)

In version .20 you can choose the expected payload type using a new drop down box as shown below.

mqtt-subscribe-node-message-type
If you select JSON the payload is converted into a JavaScript object, and so you don’t need the JSON node on the output.

Editing A subscribe or Publish Node Using the Same Client connection Name

If 2 or more nodes anywhere in the Workspace use the same connection name e.g.

A publish node uses the connection name

node-client@192.168.1.157:1883 and another node or nodes (publish or subscribe) use that connection name.

Then changes to the connection properties e.g port number,clean sessions etc will affect all of the nodes using that connection name.

Video

I’ve created a video on how to Publish and Subscribe to an MQTT broker Using Node Red



Node-Red V2

There have been quite a few changes in the MQTT nodes starting in version 2.

The first most important change was the addition to MQTT v5 features to the MQTT Nodes.

MQTT v5

MQTTv5 introduced many new features. See-   MQTT v5 Features for details

MQTT clients need to indicate to the broker what MQTT version they are using. Node-red currently defaults to 3.1.

It is part of the connection settings

mqtt-v5-node-red

If you configure the node for MQTTv5 the publish and subscribe node will contain extras settings other than those shown above.

To understand these extra settings you should acquaint yourself with the new features.

Dynamic Connections

Node-red version 2.1 has introduced new dynamic node features allowing connections,subscriptions and broker settings to be controlled dynamically.

Using these features you have much better control of the MQTT connections. see MQTT Dynamic Connections

MQTT Broker

If you are using mosquitto for the broker you should be aware of the security changes in mosquitto v2.

By default it requires authentication and doesn’t listen on a network address.

The following simple configuration file will make mosquitto start like previous versions:

listener 1883
allow_anonymous true

 



Related Tutorials

Click to rate this post!
[Total: 6 Average: 3.5]

42 comments

  1. Hi Steve,
    Ignore previous comment, I was looking in the wrong tab (under security) and assumed that the client id is auto generated as with most desktop mqtt clients; Found that you can change client id under connection tab. All good now, Thanks, Ken

  2. Hi Steve,
    Great article, I am trying to connect node red to thingspeak via mqtt (I know there is a thingspeak node, but I want to try mqtt as it allows for multiple feeds, etc).
    It would appears that thingspeak, apart from user name and password also requires “client id” to be set (same as their user name). NR mqtt node does not allow setting of client id, so I cant connect. Is there a work around or have I missed something?
    Thanks,
    Ken

  3. Steve , I have a problem that’s doing my head in. I am a novice with node red and any help will be appreciated. I have a numeric node i use to set a number i use on on a mega and it is sent via mqqt. (Using PubSubClient on mega) It appears an old number is being sent all the time from the server. Delete the node in the flow, deploy and it still keeps coming. Stop the subscription on the mega it stops, stop the server (on the pi) it stops. The qos is set to 1, played with retain and done everything i can think of….. (I have now printed all your stuff out on MQTT to read….) Other numeric nodes appear to work ok only when the offending node is not subscribed on the mega. Many thanks in anticipation, Chris

    1. Chris
      Sorry you lost me maybe I just need more coffee.
      Could you list it out . What is a mega?
      Perhaps send me the flow it might help. You can paste it in a comment I will just delete the comment when I’ve copied it.
      rgds
      steve

    2. Hello steve, looking over what i sent a bit of the arduino mega code wasnt copied over properly… (anyway the arduino code works ok)

      if( (j 30) ){ // callback() bombed out so just make it the average setpoint temp (divide by 10 because we multily x10 below)
      j = avsetpoint / 10;
      }

  4. Hi Steve
    I tried to get a mqtt publisher and subscriber connected in node-red 3.0. unfortunately I did not succeed.
    I am someone who rebuilds from something that works so that it is suitable for me.
    It seems that apart from you, almost no one has dealt with the subject of mqtt yet.
    in short: can you publish a working nodered flow. Just simple in and out via a well-established broker. future related would be v5 nice. but maybe a step too far for me (yet).

    1. Hi
      here is a simple flow
      //

      [{“id”:”3d40d417928e9121″,”type”:”tab”,”label”:”Flow 1″,”disabled”:false,”info”:””,”env”:[]},{“id”:”d71903614cb6fd5e”,”type”:”mqtt in”,”z”:”3d40d417928e9121″,”name”:””,”topic”:”test”,”qos”:”0″,”datatype”:”auto”,”broker”:”8a6b74c0.3c461″,”nl”:false,”rap”:true,”rh”:0,”inputs”:0,”x”:290,”y”:100,”wires”:[[“3b038eca35a2f8e2”]]},{“id”:”9cdeb05bbe16dfd5″,”type”:”mqtt out”,”z”:”3d40d417928e9121″,”name”:””,”topic”:”test”,”qos”:”0″,”retain”:”false”,”respTopic”:””,”contentType”:””,”userProps”:””,”correl”:””,”expiry”:””,”broker”:”8a6b74c0.3c461″,”x”:310,”y”:260,”wires”:[]},{“id”:”3b038eca35a2f8e2″,”type”:”debug”,”z”:”3d40d417928e9121″,”name”:”received”,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”true”,”targetType”:”full”,”statusVal”:””,”statusType”:”auto”,”x”:480,”y”:100,”wires”:[]},{“id”:”3c6b5842b2e48a0c”,”type”:”inject”,”z”:”3d40d417928e9121″,”name”:””,”props”:[{“p”:”payload”},{“p”:”topic”,”vt”:”str”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:”test”,”payloadType”:”date”,”x”:140,”y”:200,”wires”:[[“9cdeb05bbe16dfd5”]]},{“id”:”8a6b74c0.3c461″,”type”:”mqtt-broker”,”name”:”localhost:1883″,”broker”:”192.168.1.41″,”port”:”1883″,”clientid”:””,”autoConnect”:true,”usetls”:false,”protocolVersion”:”5″,”keepalive”:”60″,”cleansession”:true,”birthTopic”:””,”birthQos”:”0″,”birthRetain”:”false”,”birthPayload”:””,”birthMsg”:{},”closeTopic”:””,”closePayload”:””,”closeMsg”:{},”willTopic”:””,”willQos”:”0″,”willRetain”:”false”,”willPayload”:””,”willMsg”:{},”sessionExpiry”:””}]

      //

  5. hi,
    I’ve been watching your videos, I have started working with node red projects recently, I need to use mqtt to get data from a sensor, but we dont want to publish in the same sensor but in a website with a different address, I guess I have to use the ‘http request’ node but haven’t found any similar option in your videos. Does that mean that I need to use the ‘mqtt out’ if I use the ‘mqtt in’ , meaning, as a must?
    I’d appreciate your reply, thank you
    Steph

  6. All very clear, but in my opinion a fundamental part is missing (or I didn’t understand it). The “topic” to be sent, how do I get it? I mean the exact string of my device, with its payload attributes. Thanks in advance

  7. Hey Steve,
    thank you for this tutorial!
    I want to send a Byte array via MQTT. But somehow node-red is always packing it as JSON, which results in a way larger payload. Do you know how to manipulate the message, which will be send from MQTT?

    Regards,
    Toni

  8. Hi Steve,
    Need some guidance on how to catch success or error from MQTT publish node.
    I have a stream of incoming JSON files the contents of which need to be sent by MQTT.
    If the publish is a success, I need to delete the file and if the publish is a failure, I need to keep the file for later resend.

    Is this possible to achieve using NodeRed MQTT pulisher??
    Thanks
    Ajay

  9. Hello Steve,

    I am trying to get data from my MQTT server and save it to MySQL using NODE-RED. However I am struggling to pull payload_raw value from one module (I can see it on MQTT) below what I get on MQTT every10min:

    rg186/devices/4b14020b/up {“app_id”:”rg186″,”dev_id”:”4b14020b”,”hardware_serial”:”70B3D5B020038B08″,”port”:24,”counter”:8,”confirmed”:true,”payload_raw”:”/4hj+zz/5AEA9qBGAgAAAAAXAABIQxiPwqlBEwB9Ykc=”,”payload_fields”:{“battery”:65416,”light”:25595,”temperature”:156.15},”metadata”:{“time”:”2020-01-20T12:13:48.852152095Z”,”frequency”:867.3,”modulation”:”LORA”,”data_rate”:”SF7BW125″,”airtime”:92416000,”coding_rate”:”4/5″,”gateways”:[{“gtw_id”:”eui-c0ee40ffff296d5b”,”timestamp”:1505321075,”time”:””,”channel”:4,”rssi”:-29,”snr”:10.2,”rf_chain”:0,”latitude”:51.320107,”longitude”:-0.55894303,”location_source”:”registry”}],”latitude”:51.320133,”longitude”:-0.5588848,”location_source”:”registry”}}

    Any advises how I should configure mqtt in and function to pull payload_raw and payload_fields?

    Thank you in advance!

    1. Hi
      assume I assign your data to s
      s=your data
      the payload_raw is
      var payload_raw=s.payload_raw
      var t= s.metadata.time
      var latitude=s.metadata.latitude
      gateways
      is
      s.metadata.gateways

      Hope that helps
      rgds
      steve

      1. Hi Steve,

        Thank you for your answer, however my problem is to setup right topic in “mqtt in”? I have now “rg186/devices/4b14020b/#” I do not really know how to access the payload, to get readings from ttn I use this command: mosquitto_sub -h eu.thethings.network -t “+/devices/+/up” -u “rg186” -P “ttn-account-***…” -v .

        When there is new reading in MQTT will “mqttin” and function automatically save the data to database?

        flow 1:
        mqttin -> function -> MySQL

        function:
        var Data = msg.payload;
        s = Data;
        var payload_raw = s.payload_raw;
        var t = s.metadata.time;

        msg.topic = “INSERT INTO LoraTest (Bettery) VALUES (‘”+payload_raw+”‘)”;
        msg.payload = [payload_raw];
        return msg;

  10. Hi, i need help , how i can send a data from arduino mkr gsm to node red (not local) by using mqtt and mosquitto , thnks

      1. no i can’t , i can send data from arduino mkr to node red(localhost) but to other server i can’t send

        1. Hi
          If it works locally then there is something wrong either with the remote broker (i.e. may need username/password etc) of firewall setting.
          Check on an open broker like test.mosquitto.org and let me know
          Rgds
          Steve

          1. hi , locally it works with COM but in this situation i don’t use serial COMi in input i use mqtt nodes and broker is not localhost , but no result

  11. Steve,
    Appreciate your work. Also saw your separate post on multiple mqtt brokers on same host. I have 1 raspberry on 2 separate routers/LANs (wlan0 and etho). One is for IOT (no gateway) and the other is for home network (gateway). I am trying to configure 1 mqtt broker in node-red for the IOT, and the another mqtt broker for the home network. Would like to pass messages from IOT broker to home network broker so that Home Assistant on another Pi can can get the messages. IOT broker works (local host port 1883), but adding a second broker with the Pi’s other network address (192.168.1.62 on port 1884) is not working.
    Thoughts?? Thanks

    1. Bob
      It is probably the network bindings. Here is a copy of what I think is the relevant part of the mosq conf file

      General Options

      bind_address address

      Listen for incoming network connections on the specified IP address/hostname only. This is useful to restrict access to certain network interfaces. To restrict access to mosquitto to the local host only, use “bind_address localhost”. This only applies to the default listener. Use the listener variable to control other listeners.

      Not reloaded on reload signal.
      bind_interface device

      Listen for incoming network connections only on the specified interface. This is similar to the bind_address option but is useful when an interface has multiple addresses or the address may change.

      It is valid to use this option together with bind_address for the default listener, or the bind address/host part of the listener definition. Care should be taken to ensure that the address being bound to is on the interface being bound to. If you set the bind_interface to be eth0, and bind_address to 127.0.0.1, then the broker will start correctly but you will be unable to connect.

      This option is currently only available on Linux, and requires elevated privileges.

  12. Thank you – it helped me a lot.
    I have a few mosuitto mqtt brokers running on raspberry pi at home for home automation. Used node-red for programming und dashboards.
    I’m on the way to set up in my camper a local hotspot with wifi, raspberry pi zero wifi with node-red and mosquitto mqtt broker. Connected ESP-8266 Wifi to measure battery power und temperature. More to come. It works within my camper.
    Now I want to connect it to my home broker over the hotspots G4. I have also cloudmqtt, but also with an Domain set to my Home-IP. Its running, but now the challenge is to secure the communication with certificates and tsl/ssl. This is not my best known area.
    Bruno

  13. Hi I want provoid ssl certificate path from the function node to http node how to do that ? like we can set url like msg.url=”http://google.com” in function and send “msg” to http node

  14. What do you do is a topic publishes the same value all the time. What’s the easiest way in node-red of subscribing to that topic, but only doing something when that topic payload changes.

Leave a Reply

Your email address will not be published. Required fields are marked *