Node-Red provides both an MQTT subscribe (input) and publish (output) node.
The configuration for these nodes are almost Identical as the main part of the configuration concerns the actual client connection.
Because of this it is useful to think of the publish and subscribe nodes as consisting of two components as shown in the schematic below:
Before we look a the actual configuration we will look at MQTT client connections in general.
How MQTT works
If you are unfamiliar with the MQTT protocol I would recommend that you take a read of how MQTT works before Continuing
Connecting to an MQTT Broker or Server.
To connect to an MQTT broker or server you need to know the Broker IP address or name, and the port that it is using (default 1883).
In addition a client needs to provide a client name to identify itself. Each client connection requires a unique client name.
A single client connection can be used to both publish and subscribe.
An MQTT broker can enforce encryption and username/password authentication in which case the connecting client must comply.
There are various flags and parameters that a client can set. These are:
- Clean Session Flag
- Time to Live
- Last Will and Testament
Clean Session Flag
MQTT clients will usually by default establish a clean session with a broker.
A clean session is one in which the broker isn’t expected to remember anything about the client when it disconnects.
With a non clean session the broker will remember client subscriptions and may hold undelivered messages for the client.
Time to Live
This is a mechanism used to determine if a connection is still present. The default interval is 60 seconds.
Last Will and Testament
The idea of the last will message is to notify a subscriber that the publisher is unavailable due to network outage.
The last will message is set by the client on a topic and is stored on the broker.
It Is sent to all subscribers if the publishing client disconnects abnormally.
If the publisher disconnects normally the last Will Message is not sent.
By default the node connects automatically when you start the flow, however node-red now supports dynamic connections that allow you to control the connection.
Publishing MQTT Messages- The Publish Node
When a client publishes a message to a broker it needs to send:
- The message topic
- The message QOS
- Whether the message should be retained.- Retain Flag
The retain Flag is normally set to False which means that the broker doesn’t keep the message.
If you set the retain flag to True then the last message received by the broker with the retained flag set will be kept.
The QOS of the published message has no effect on the retained message.
The retain flag setting is the only setting that is unique to the publish node.
To publish a Message to an MQTT broker you use the MQTT publish node.
To use a publish node drag the node from the node palette on the left into the main workspace.
Double click on the node to configure it. The main configuration screen is shown below:
.
The main setting is the broker or server setting.
You can select a previously configured server which you can edit by clicking on the edit button, or add a new broker by selecting the add new mqtt server option from the server list.
The server name that appears in the server list is a combination of the client ID and Broker name or IP address.
Note: The MQTT specification originally used the term broker but it has now been changed to server.
You can also configure the topic address,the QOS for the message and the retain flag.
The topic address,the QOS and the retain flag can also be provided by the preceding node as part of the message object using:
- msg.topic =mytopic
- msg.qos =0,1 or 2
- msg.retain =true or false
Publish Message Format
The message to be published can be either a JavaScript Object string or a binary buffer.
If you pass in a JavaScript object it is automatically converted to a JSON encoded string before being published.
If you pass in numeric data it is converted to a string and if you pass in Boolean data it is also converted into a string before being published.
Editing the Server Settings
If you click on the edit icon to the right of the server you can edit the server settings.
The server settings has 4 tabs.
- connection tab
- Security
- Birth Message
- Will Message
Connection Tab
The port defaults to 1883 which is the standard MQTT port.
You can use either the IP address or FQDN for the server address.
The client name needs to be unique on the broker.
You can configure the client to use SSL for the connection by enabling the TLS box- see later
The default keep alive is 60 secs and clean session is False. It is normal to enable clean sessions.
Security Tab
This lets you configure a user name and password for the connection.
Whether you need a username/password for the connection is determined by the MQTT broker.
Birth Message
The birth message is published by the client when the MQTT node starts.
A birth message is not part of the MQTT specification but it is part of the Sparkplug specification.
Will Message
The Will message is part f the MQTT specification and is stored on the broker, and sent in the advent of a failed client connection.
The message payload will either be a message to indicate the connection has failed or a status indicator.
The retained message will usually be set so that new clients know the status.
Using SSL for Encryption
The client supports SSL encryption between the client and server as well as using certificate based authentication.
To configure a secure connection you will need enable TLS
and add the CA certificate to node-red by uploading it. In the screen shot below I used the same self generated certificate as I used on Mosquitto broker.
If the CA file is on your local computer you can use it by enabling the use key and certificates from local files option., and entering the path and file name:
Alternatively you can upload the files using the upload buttons.
Note: the verify server certificate checks the the name on the certificate is correct and should be enabled on live systems. Leaving it un-checked is like using the -insecure option of the mosquitto_pub /_sub tool. Entering a name in the box doesn’t seem to make any difference.
If you are using Client certificates then you will also need to upload or provide the pathname to the client key and certificate as shown below:
SSL Public Certificates
If you are using a public broker like cloudmqtt then you will need to use a public CA.
You will need to get a copy of ca-certificates.crt or ca-certificates.pem file.
This file contains a list of trusted CA certificates and is available from Mozilla.
If you are using Linux it is usually in /etc/SSL/certs, but you can download it from the curl site here.
You will then need to upload it, just like with the self generated certificate as shown in the screen shot below.:
Configuring The Subscribe Node
When a client subscribes to a broker it needs to provide a topic and QOS.
The subscribe node has no unique settings.
To use a subscribe node drag the node from the node palette on the left into the main workspace.
Double click on the node to configure it. The main configuration screen is shown below, and is almost identical to the publish node.
Note: You don’t currently appear to be able to subscribe to multiple topics using the same node.
Receive Payload
Prior to version .20 the received payload was either a string or buffer object (if detected by the node)
In version .20 you can choose the expected payload type using a new drop down box as shown below.
If you select JSON the payload is converted into a JavaScript object, and so you don’t need the JSON node on the output.
Editing A subscribe or Publish Node Using the Same Client connection Name
If 2 or more nodes anywhere in the Workspace use the same connection name e.g.
A publish node uses the connection name
node-client@192.168.1.157:1883 and another node or nodes (publish or subscribe) use that connection name.
Then changes to the connection properties e.g port number,clean sessions etc will affect all of the nodes using that connection name.
Video
I’ve created a video on how to Publish and Subscribe to an MQTT broker Using Node Red
Node-Red V2
There have been quite a few changes in the MQTT nodes starting in version 2.
The first most important change was the addition to MQTT v5 features to the MQTT Nodes.
MQTT v5
MQTTv5 introduced many new features. See- MQTT v5 Features for details
MQTT clients need to indicate to the broker what MQTT version they are using. Node-red currently defaults to 3.1.
It is part of the connection settings
If you configure the node for MQTTv5 the publish and subscribe node will contain extras settings other than those shown above.
To understand these extra settings you should acquaint yourself with the new features.
Dynamic Connections
Node-red version 2.1 has introduced new dynamic node features allowing connections,subscriptions and broker settings to be controlled dynamically.
Using these features you have much better control of the MQTT connections. see MQTT Dynamic Connections
MQTT Broker
If you are using mosquitto for the broker you should be aware of the security changes in mosquitto v2.
By default it requires authentication and doesn’t listen on a network address.
The following simple configuration file will make mosquitto start like previous versions:
listener 1883 allow_anonymous true
Related Tutorials
- MQTT Dynamic Connections
- How to Install and Run The Aedes Broker on Node-Red
- Working with JSON Data And JavaScript Objects in Node-Red
- MQTT Data Logger -Node-Red
- Node-Red HTTP Request Node for Beginners
- Node-Red Configuration Nodes Overview
- Encrypting and decrypting MQTT payloads
Hi Steve,
Ignore previous comment, I was looking in the wrong tab (under security) and assumed that the client id is auto generated as with most desktop mqtt clients; Found that you can change client id under connection tab. All good now, Thanks, Ken
No problem.It happens to me all the time.
Rgds
Steve
Hi Steve,
Great article, I am trying to connect node red to thingspeak via mqtt (I know there is a thingspeak node, but I want to try mqtt as it allows for multiple feeds, etc).
It would appears that thingspeak, apart from user name and password also requires “client id” to be set (same as their user name). NR mqtt node does not allow setting of client id, so I cant connect. Is there a work around or have I missed something?
Thanks,
Ken
I use node-red.
stevesnoderedguide.com
Rgds
Steve
Steve , I have a problem that’s doing my head in. I am a novice with node red and any help will be appreciated. I have a numeric node i use to set a number i use on on a mega and it is sent via mqqt. (Using PubSubClient on mega) It appears an old number is being sent all the time from the server. Delete the node in the flow, deploy and it still keeps coming. Stop the subscription on the mega it stops, stop the server (on the pi) it stops. The qos is set to 1, played with retain and done everything i can think of….. (I have now printed all your stuff out on MQTT to read….) Other numeric nodes appear to work ok only when the offending node is not subscribed on the mega. Many thanks in anticipation, Chris
Chris
Sorry you lost me maybe I just need more coffee.
Could you list it out . What is a mega?
Perhaps send me the flow it might help. You can paste it in a comment I will just delete the comment when I’ve copied it.
rgds
steve
Hello steve, looking over what i sent a bit of the arduino mega code wasnt copied over properly… (anyway the arduino code works ok)
if( (j 30) ){ // callback() bombed out so just make it the average setpoint temp (divide by 10 because we multily x10 below)
j = avsetpoint / 10;
}
Chris
Sent you and email
Rgds
Steve
Hi Steve
I tried to get a mqtt publisher and subscriber connected in node-red 3.0. unfortunately I did not succeed.
I am someone who rebuilds from something that works so that it is suitable for me.
It seems that apart from you, almost no one has dealt with the subject of mqtt yet.
in short: can you publish a working nodered flow. Just simple in and out via a well-established broker. future related would be v5 nice. but maybe a step too far for me (yet).
Hi
here is a simple flow
//
[{“id”:”3d40d417928e9121″,”type”:”tab”,”label”:”Flow 1″,”disabled”:false,”info”:””,”env”:[]},{“id”:”d71903614cb6fd5e”,”type”:”mqtt in”,”z”:”3d40d417928e9121″,”name”:””,”topic”:”test”,”qos”:”0″,”datatype”:”auto”,”broker”:”8a6b74c0.3c461″,”nl”:false,”rap”:true,”rh”:0,”inputs”:0,”x”:290,”y”:100,”wires”:[[“3b038eca35a2f8e2”]]},{“id”:”9cdeb05bbe16dfd5″,”type”:”mqtt out”,”z”:”3d40d417928e9121″,”name”:””,”topic”:”test”,”qos”:”0″,”retain”:”false”,”respTopic”:””,”contentType”:””,”userProps”:””,”correl”:””,”expiry”:””,”broker”:”8a6b74c0.3c461″,”x”:310,”y”:260,”wires”:[]},{“id”:”3b038eca35a2f8e2″,”type”:”debug”,”z”:”3d40d417928e9121″,”name”:”received”,”active”:true,”tosidebar”:true,”console”:false,”tostatus”:false,”complete”:”true”,”targetType”:”full”,”statusVal”:””,”statusType”:”auto”,”x”:480,”y”:100,”wires”:[]},{“id”:”3c6b5842b2e48a0c”,”type”:”inject”,”z”:”3d40d417928e9121″,”name”:””,”props”:[{“p”:”payload”},{“p”:”topic”,”vt”:”str”}],”repeat”:””,”crontab”:””,”once”:false,”onceDelay”:0.1,”topic”:”test”,”payloadType”:”date”,”x”:140,”y”:200,”wires”:[[“9cdeb05bbe16dfd5”]]},{“id”:”8a6b74c0.3c461″,”type”:”mqtt-broker”,”name”:”localhost:1883″,”broker”:”192.168.1.41″,”port”:”1883″,”clientid”:””,”autoConnect”:true,”usetls”:false,”protocolVersion”:”5″,”keepalive”:”60″,”cleansession”:true,”birthTopic”:””,”birthQos”:”0″,”birthRetain”:”false”,”birthPayload”:””,”birthMsg”:{},”closeTopic”:””,”closePayload”:””,”closeMsg”:{},”willTopic”:””,”willQos”:”0″,”willRetain”:”false”,”willPayload”:””,”willMsg”:{},”sessionExpiry”:””}]
//
hi,
I’ve been watching your videos, I have started working with node red projects recently, I need to use mqtt to get data from a sensor, but we dont want to publish in the same sensor but in a website with a different address, I guess I have to use the ‘http request’ node but haven’t found any similar option in your videos. Does that mean that I need to use the ‘mqtt out’ if I use the ‘mqtt in’ , meaning, as a must?
I’d appreciate your reply, thank you
Steph
You just need the http request node. Is the data on a web page or will it be JSON?Is there an API?
https://stevesnoderedguide.com/node-red-http-request-node-beginners
rgds
Steve
All very clear, but in my opinion a fundamental part is missing (or I didn’t understand it). The “topic” to be sent, how do I get it? I mean the exact string of my device, with its payload attributes. Thanks in advance
The topic is any string that you want. There is no formal naming structure and so it is anything that you want it to be provided it follows some basic rules see
http://www.steves-internet-guide.com/understanding-mqtt-topics/
Hey Steve,
thank you for this tutorial!
I want to send a Byte array via MQTT. But somehow node-red is always packing it as JSON, which results in a way larger payload. Do you know how to manipulate the message, which will be send from MQTT?
Regards,
Toni
Modbus uses bytes and I did a tutorial on it and several videos which should help
http://www.steves-internet-guide.com/node-red-modbus/
I also did one on buffers which is probably the best place to start
http://www.steves-internet-guide.com/understanding-buffers-node-red/
rgds
steve
Do you have a example for the ssl Security in node-red ?
Hi
It is covered for the client here
http://www.steves-internet-guide.com/configuring-the-mqtt-publish-node/
The sever setup is very similar.
Rgds
Steve
Hi Steve,
Need some guidance on how to catch success or error from MQTT publish node.
I have a stream of incoming JSON files the contents of which need to be sent by MQTT.
If the publish is a success, I need to delete the file and if the publish is a failure, I need to keep the file for later resend.
Is this possible to achieve using NodeRed MQTT pulisher??
Thanks
Ajay
You need an acknowledgement from the receiver to be sure. Are you in control of the receiver?
Hello Steve,
I am trying to get data from my MQTT server and save it to MySQL using NODE-RED. However I am struggling to pull payload_raw value from one module (I can see it on MQTT) below what I get on MQTT every10min:
rg186/devices/4b14020b/up {“app_id”:”rg186″,”dev_id”:”4b14020b”,”hardware_serial”:”70B3D5B020038B08″,”port”:24,”counter”:8,”confirmed”:true,”payload_raw”:”/4hj+zz/5AEA9qBGAgAAAAAXAABIQxiPwqlBEwB9Ykc=”,”payload_fields”:{“battery”:65416,”light”:25595,”temperature”:156.15},”metadata”:{“time”:”2020-01-20T12:13:48.852152095Z”,”frequency”:867.3,”modulation”:”LORA”,”data_rate”:”SF7BW125″,”airtime”:92416000,”coding_rate”:”4/5″,”gateways”:[{“gtw_id”:”eui-c0ee40ffff296d5b”,”timestamp”:1505321075,”time”:””,”channel”:4,”rssi”:-29,”snr”:10.2,”rf_chain”:0,”latitude”:51.320107,”longitude”:-0.55894303,”location_source”:”registry”}],”latitude”:51.320133,”longitude”:-0.5588848,”location_source”:”registry”}}
Any advises how I should configure mqtt in and function to pull payload_raw and payload_fields?
Thank you in advance!
Hi
assume I assign your data to s
s=your data
the payload_raw is
var payload_raw=s.payload_raw
var t= s.metadata.time
var latitude=s.metadata.latitude
gateways
is
s.metadata.gateways
Hope that helps
rgds
steve
Hi Steve,
Thank you for your answer, however my problem is to setup right topic in “mqtt in”? I have now “rg186/devices/4b14020b/#” I do not really know how to access the payload, to get readings from ttn I use this command: mosquitto_sub -h eu.thethings.network -t “+/devices/+/up” -u “rg186” -P “ttn-account-***…” -v .
When there is new reading in MQTT will “mqttin” and function automatically save the data to database?
flow 1:
mqttin -> function -> MySQL
function:
var Data = msg.payload;
s = Data;
var payload_raw = s.payload_raw;
var t = s.metadata.time;
msg.topic = “INSERT INTO LoraTest (Bettery) VALUES (‘”+payload_raw+”‘)”;
msg.payload = [payload_raw];
return msg;
Hi
You need to contact me via the ask steve page and send me the flow you are using.
Rgds
Steve
Hi, i need help , how i can send a data from arduino mkr gsm to node red (not local) by using mqtt and mosquitto , thnks
Hi
Are you currently able to send and receive messages using arduino and mqtt?
no i can’t , i can send data from arduino mkr to node red(localhost) but to other server i can’t send
Hi
If it works locally then there is something wrong either with the remote broker (i.e. may need username/password etc) of firewall setting.
Check on an open broker like test.mosquitto.org and let me know
Rgds
Steve
hi , locally it works with COM but in this situation i don’t use serial COMi in input i use mqtt nodes and broker is not localhost , but no result
Hi
Can you use the ask steve page and send me the flow
Rgds
Steve
Steve,
Appreciate your work. Also saw your separate post on multiple mqtt brokers on same host. I have 1 raspberry on 2 separate routers/LANs (wlan0 and etho). One is for IOT (no gateway) and the other is for home network (gateway). I am trying to configure 1 mqtt broker in node-red for the IOT, and the another mqtt broker for the home network. Would like to pass messages from IOT broker to home network broker so that Home Assistant on another Pi can can get the messages. IOT broker works (local host port 1883), but adding a second broker with the Pi’s other network address (192.168.1.62 on port 1884) is not working.
Thoughts?? Thanks
Bob
It is probably the network bindings. Here is a copy of what I think is the relevant part of the mosq conf file
General Options
bind_address address
Listen for incoming network connections on the specified IP address/hostname only. This is useful to restrict access to certain network interfaces. To restrict access to mosquitto to the local host only, use “bind_address localhost”. This only applies to the default listener. Use the listener variable to control other listeners.
Not reloaded on reload signal.
bind_interface device
Listen for incoming network connections only on the specified interface. This is similar to the bind_address option but is useful when an interface has multiple addresses or the address may change.
It is valid to use this option together with bind_address for the default listener, or the bind address/host part of the listener definition. Care should be taken to ensure that the address being bound to is on the interface being bound to. If you set the bind_interface to be eth0, and bind_address to 127.0.0.1, then the broker will start correctly but you will be unable to connect.
This option is currently only available on Linux, and requires elevated privileges.
Thank you – it helped me a lot.
I have a few mosuitto mqtt brokers running on raspberry pi at home for home automation. Used node-red for programming und dashboards.
I’m on the way to set up in my camper a local hotspot with wifi, raspberry pi zero wifi with node-red and mosquitto mqtt broker. Connected ESP-8266 Wifi to measure battery power und temperature. More to come. It works within my camper.
Now I want to connect it to my home broker over the hotspots G4. I have also cloudmqtt, but also with an Domain set to my Home-IP. Its running, but now the challenge is to secure the communication with certificates and tsl/ssl. This is not my best known area.
Bruno
Hi I want provoid ssl certificate path from the function node to http node how to do that ? like we can set url like msg.url=”http://google.com” in function and send “msg” to http node
Confused as the tutorial is on the mqtt node not http
rgds
steve
Hi, I’m working on a project which using MQTT-Node RED. I need to get data from a website: “https://intelliweb.mastervolt.com”. I followed this instruction but my flow isn’t worked yet: https://www.symcon.de/forum/attachment.php?attachmentid=34745&d=1467291401 . I think I have some problem with TLS Configuration because I don’t know how to configure it.
Can you use the ask-steve page and send me the flow you are using
rgds
steve
What do you do is a topic publishes the same value all the time. What’s the easiest way in node-red of subscribing to that topic, but only doing something when that topic payload changes.
I do this with the topic logger which you can download form the site.
The trick is to store the last payload in a context,flow or global variable so that you can compare the last value to the current value.
If there is no change you just ignore
http://www.steves-internet-guide.com/node-red-variables/
http://www.steves-internet-guide.com/node-red-mqtt-topic-logger/
Does that help?
Rgds Steve
Good job!
However, I believe the port should be 8883 for TLS.
Yes correct 8883 is generally used for ssl and mqtt
Very detailed tutorial.