a lot actually.
Most of us when we get started with MQTT probably encounter the same problem. We read up on retained messages and QOS and think that it sounds like a good idea so we set up all of our topics with the retained flag. Then we can’t figure out why the garage door opener keeps opening every time the NodeMCU attached to it reboots.
Then we go to the other extreme and turn the retained flag off on all the topics. But that means that we have to go through all sorts of hoops with OH System started Rules to trigger the devices to report their current states, or set the reporting on the device to be every so often even if the state didn’t change.
But when properly deployed, the retained flag is actually super powerful and can save us a whole lot of work.
A Retained flag primer
What is the retained flag and how does it work?
The retained flag is a flag that can be set on an MQTT message that tells the broker to hold on to that message. Any new client that subscribes to that topic will receive the most recent retained message published to that topic.
The message publisher sets the retained flag, not the subscriber.
The retained flag can be set on a message by message basis. However, the original MQTT 1.x binding only allowed one to set it on a broker connection by broker connection basis. The MQTT 2.x binding allows one to set it on a Channel by Channel basis. But remember the bullet point above, setting the retained flag in OH will only effect messages published by openHAB.
Only the most recent message on any given topic with the retained flag get’s saved.
If subscribing to a wild card topic, the client will receive all the retained messages on all the topics that match upon subscription.
When should I use a retained message?
Sensors and status topics are excellent candidates for retained messages. Consider the following example. Let’s say we have some devices that report the open/closed status of a door. We all know that systems are not 100% reliable so let’s assume that openHAB was rebooted. While openHAB is coming back online, the sensor reported a new state for the door.
Without using the retained flag, when openHAB comes back online, the Item will remain in the NULL state until the sensor reports it’s new state. If the sensor only reports changes then we have to wait for the door to open or close before we get a valid state. If the sensor wastes energy and cycles by reporting periodically whether there is a change or not, then we need to wait for the polling period. If the sensor supports requesting a report, one needs to code a System started Rule in openHAB to publish a message that causes the sensor to report it’s current state. Again, if this is a battery powered device it means that the device needs to be constantly connected so it can receive the message wasting power.
But, if the sensor uses the retained flag then openHAB will receive the most recently reported state immediately upon connecting to the broker and subscribing to the topic. All you have to do is set that one little flag and you don’t have to worry about any of the complications in the previous paragraph.
To put this more generically, I’ll quote from the HiveMQ page on the subject:
A retained message makes sense when you want newly-connected subscribers to receive messages immediately (without waiting until a publishing client sends the next message).
So any status or sensor reading that has a meaning beyond the mere instant when it was reported could benefit from using the retained flag.
When should you not use a retained message?
When the topic is a command topic. You do not want to retain commands to devices because every time the device restarts or loses it’s connection to the broker, it will receive the command again when it reestablishes the connection to the broker. You don’t want your garage door to open every time you reboot your WiFi router.
But this isn’t a hard and fast rule. For example, maybe one has a device that controls a light and the commands received indicate the state we want the light to be in. And let’s assume that the light doesn’t remember it’s previous state when it restarts. In that case, a retained message for the command makes sense and is useful as this will cause the light to receive the most recent command again when it comes online and enter its previous state.
Help, I sent a retained message and want to get rid of it!
This is actually pretty easy to manage, though a little counter intuitive. You need to send an empty message to the topic with the retain flag set. This will clear out the retained message. You can do this with any of the many MQTT clients.
A specific use case
Is my device online? How do I tell? If you don’t use retained messages and the LWT you will have to implement some sort of heart beat on the device to always know when it is online.
But if you do the following, you will always know when the device is online without the need for a heart beat.
Configure the device to publish “ONLINE” when it successfully connects to the broker to the LWT topic. Be sure to set the retained flag to true for this message.
Configure the LWT to publish “OFFLINE” to the LWT topic with the retained flag set to true so the broker changes the retained message on the LWT topic to “OFFLINE” when the device disconnects.
Now when ever OH or any other client comes online and subscribes to the LWT topic, it will immediately know whether the device is online or not because it will immediately receive the “ONLINE” or “OFFLINE” message.
I can’t claim credit for this use case. I believe I first saw it in use by the Shelly 1, though I could be missremembering. I think the Homie standard works like this too.
The nice thing about this approach is you have pretty much guaranteed that you can know when the device is online or offline without the wasted energy of setting up a heart beat.