you need to address the root-cause then (OH2 service freezing up)… let’s continue the discussion over here
Yes! Thanks! Until then I just disabled retain, so might lights won’t change state suddenly 
1 Like
So it seems that was just a temporary problem…
So about retain… How does it properly work? Because after disabling Retain on the devices, it still retained them, because it was defined in the mqtt cfg to retain… so how does it work? I just need to enable retain in mqtt.cfg? Why the devices has a Retain ON/OFF?
retain can be set: globally on the Broker config or on the publisher message
read more here: https://www.hivemq.com/blog/mqtt-essentials-part-8-retained-messages
Thanks! So if I want to use it on all MQTT devices then I just need to define it in the config and every(?) message sent on MQTT is retained…
However what happens if the broker lose connection to a device and then after a little time reconnects (and the current state has changed since its disconnected) it will revert the state. There is no solution for ‘pulling’ state from a device? I mean retain is good, but maybe best for devices which are just listening to different topics and not publishing info (Like in the example, so it will get the temperature immediately and it can display the temp - for example). But for light switches, it should prioritize the device current state, because it is (might be) more relevant than the last state stored by MQTT or any other database…
I my own opinion, retain is a pain in the ***
I use restoreOnStartUp with MapDB for OH restart
If the broker need a reboot (Different pi…) then when I reboot it OH reconnects within 30s or so.
The devices states will be updated the next time they publish.
1 Like
Yes, in your mosquitto.cfg (if using Eclipse Mosquitto as the MQTT Broker)
It also depends on the endpoint device.
Some endpoint devices (mqtt clients) will publish their status when they will re-establish a broker connection (well… actually… most won’t do this).
I think not… I maybe wrong.
Yes I thought the same, but I didn’t want to write this, because I thought I’m missing something on this Retain.
This is what I want to use now. So if I use MapDB it will just ‘postUpdate’ the states? So it will not change state right?
So for example if the OH goes down and the last state stored in mapdb is OFF and during the downtime it becomes ON, after booting up OH it will just update the switch to OFF (however it is ON), but won’t send an OFF command, right?
I think this is the best way for MQTT switches, because your lights won’t change state unpredictically and the state will become 100% right after a state change.
Thanks for the clearing this for me!
1 Like
Correct
Now if you want to use retain for some devices you just need to define 2 brokers in the mqtt.cfg.
Both connect to the same broker but one with retain enabled.
For example:
broker.url=tcp://192.168.0.34:1883
brokerretain.url=tcp://192.168.0.34:1883
# Optional. True or false. Defines if the broker should retain the messages sent to
# it. Defaults to false.
broker.retain=false
brokerretain.retain=true
And in the item’s binding definition use brokerretain instead of broker
But I am still trying to think of a use for retain in an Home Automation environment.
one more additional clarification here:
the sequence is: OH2 starts up → Item state = NULL → restoreOnStartup → Item state = OFF → MQTT subscription from Item binding config → Item state = whatever the broker reports (if message was retained)
The next time the client (endpoint device) will publish its state, everybody will be “synchronized” in the chain (OH2<->Broker<->Device)
edit:
Note: This only applies for the outbound (publish) messages originating from OH2 to the Broker.
It sets the retain flag to true for all MQTT messages that OH2 (client) sends to this Broker alias.
It doesn’t affect the publishing of the remote client (the endpoint device, e.g. a SOnOff)
I also do NOT use retain anywhere… it’s a PITA like Vincent correctly wrote 
look how big of a PITA is retain:
I have an ESP controlling my garage gate
I had set retain on OH2 MQTT binding config (as Vincent wrote above) and my command was retained in the MQTT Broker.
The next time the ESP client lost connection (due to a power failure) it would reconnect and receive the retained command… resulting to opening the freaking gate
So… if someone wanted to open my gate… he could easily toggle the mains power supply outside my house (the power meter is not secured 
and has an on/off switch)
1 Like
Yes! Thats the main problem I found. I dont know one use-case where this can be useful. First I wanted to use for ‘syncing’ states but as it seems this is not for that (thougth the same after enabling but I give it a try).
I dont know what is the alternative use of MQTT (as far as I know it is mainly for home assistant systems), but there might be useful this behaviour.
Anyway thanks for both of you to helping me understand MQTT better…
1 Like
Now if you want to clear the retained topics in mosquitto there are 2 solutions:
- Clear the mosquitto.db:
sudo service mosquitto stopsudo rm /var/lib/mosquitto/mosquitto.db-
sudo service mosquitto start
This will clear ALL the retained messages
- Send an empty payload to the retained topic
This will clear only the topics selected
by the way: I think that I have been wrong here…
there is no parameter to set globally “retain=true” flag for MQTT messages in mosquitto.conf man page | Eclipse Mosquitto
that’s normal since this is controlled by the client (not the server)
That’s weird because I had a flag set to true there! And it seems that it worked, because after removing it won’t change on retainted messages…
That is correct.
One concept that is important to realize is that MQTT is more of a push than a pull. When your client subscribes to a topic with a retained message, the broker will push that retained message. When the client subscribes to a topic without a retained message then there is no message to pull.
In none of these cases is there anything pulling any data from the devices or the broker.
This is also not unique to MQTT. Many/most technologies OH integrates with works similarly.
My question for you, how are the topics set up? Are there separate topics for commanding the lights and the lights reporting state updates? If so, what you may be seeing is that when you use the retain messages on the command topic the lights will get that retained message again when the broker comes back online, causing them to turn on or off.
So to do this correctly you need to configure the devices that are publishing the state updates to use retained messages (the client sending the message controls whether the message is retained or not) and not mqtt.cfg.
Another thing to realize about retained messages is that they are retained forever, even if you publish non-retained messages after the retained message. You need to explicitly clear out retained messages by sending an empty message with retain turned on to clear out those old retained messages. Perhaps this is why it appeared that MapDB was sending commands with restoreOnStartup.
If you have control over how the device works, you can implement something where you publish a message when OH starts up to a topic that all your devices subscribes to. When they receive that message the devices publish their current state. As I said above, MQTT is strictly push, not pull.
I couldn’t agree more. I avoid it.
Correct. But depending on how your MQTT binding configs are on the Items, a postUpdate might also be published. Make sure the type filed is command and not state or *. Otherwise postUpdate will cause the update to be published as well.
But remember it is the sending client that controls the retain so this will apply to messages published by OH only. In this case, the messages that really need to be retained are those that are sent by the devices.
Ha! I had the exact same problem with retain on my garage door openers. It took me weeks to figure out what was going on. I couldn’t figure out why the garage doors would open every time the controller restarted. I though there was a pulse or something on the GPIO. But no, it was working as designed, it was that retained message.
I can think of a couple.
-
It would indeed work if your devices were using the retain bit on the messages they send for their state updates. Then when OH comes up it will receive the latest state. MQTT was designed with a spotty network connectivity in mind and using the retain bit like this helps address this.
-
Send one retained message to set/reset a device to a default state when it loses connectivity. As I said above, that one retained message will persist even though each subsequent message to the topic isn’t retained.
Remember, MQTT was created for use in the industrial oil industry with remote devices and sensors on oil pipelines and stuff like that. It is super light weight, easy to implement, super energy efficient, and very reliable, all things that are useful in the home automation context as well. MQTT is exceptionally powerful and there is a reason it is heavily used in home automation.
This would make a good tutorial post, maybe with how to send the empty message using mosquitto_pub. All of us at one point or another run into retained messages problems and clearing them is not intuitive.
1 Like
Thanks for your long detailed post!
A few answers:
No I don’t I use the same topic, just with command and state attribute, like this:
mqtt=">[mosquitto:shellies/shelly1-056C51/relay/0/command:command:*:MAP(shelly.map)],<[mosquitto:shellies/shelly1-056C51/relay/0:state:MAP(shelly.map)]"
All of my MQTT switches follows this method, only the transform is different…
What do you mean by this? I have one Switch which can send command and updates the state of that Switch Item as well.
I have set my AllLight group to mapdb persistence with everyChange, restoreOnStartup. Is that correct? It seems working but maybe I’m missing here also something.
Didn’t know that, I haven’t read the story of it, just a few things which was required to get to know better this protocoll. However I thought that this is not mainly created for this!
Thanks!
If you have an MQTT binding config like
>[mosquitto:some/topic:state:*:some transform]
or
>[mosquitto:some/topic:*:some transform]
then then both commands and updates will be published to some/topic. You want to use command so it only publishes the message when the Item receives a command.
1 Like
Done:
2 Likes