Dear all,
I run the embedded MQTT broker of openhab 2.4 (openhabian / Release = Raspbian GNU/Linux 10 (buster) / Platform = Raspberry Pi 3 Model B Rev 1.2).
Although I use mapDB for persistence (and RRD4J for charting) - the status of switches wasn’t restored at startup.
How can one solve this issue?
Thanks,
Selter
as it was answered in here
MQTT and persistence is not working as others, so you have to use retained messages instead, which works as it’s needed.
I disagree. Retained messages can lead to all sort of unwanted behaviours.
I use a workaround. Although not perfect, I restored all the MQTT items. They are all is a special group and I run the following rule after start up:
rule "Update MQTT2 Items"
when
System started //or
// Item TestSwitch received command ON
then
createTimer(now.plusSeconds(120), [ | // you may need to experiment to find the optimum wait time
MQTTv2.members.filter[ i | i.state == UNDEF ].forEach[ ii |
ii.postUpdate(ii.previousState(false,"mapdb").state.toString)
Thread::sleep(150)
]
])
end
Basically, the binding initialises every item as UNDEF, as it should, the problem is that the start up order is not fixed and the restoreOnStartup normally occurs before the MQTT binding starts so your items revert to UNDEF.
So, 2 mins in the startup, to give time for the MQTT binding to start, I filter all the items belonging to the group MQTTv2, if they have an UNDEF state then I restore them using the .previousState persistence method. The little sleep is to give time for the DB access and not skip any items.
Thanks for sharing - it looks promising.
I’m just wondering if there is a need to alter “mapdb” for my configuration? (I’m not familiar with previousState syntax …)
It depends, my default persistence is influxDB so I need to specify mabdb in the method call to use mapdb otherwise it would use influxdb, which is slower at these kind of things.
For the syntax:
Ok, my default is RDD4J …
well, various standards (homie for example) are using retain as default, there is nothing wrong with it, if you implementation is correct.
It’s not just about if you have selected retained messages (or Homie has done it for you).
The MQTT binding currently acts unusually for bindings, and updates/the channel/Item at startup. You might get a retained value, you might get UNDEF if its not retained or the broker is sick or ill-informed.
Either way, it tramples over any restore-on-startup you may have configured, no choices.
I hope this will be addressed as soon as possible and that binding will act as others.
@vzorglub workaround looks ok, but it’s quite tidious to put all mqtt items to separate group just because of it… my retained msg’s are working ok after system start so i’ll stick with them for now
That turns out to be a whole philosophical question.
If the broker is set to retain (and that broker mechanism is configured and working properly), all is well.
If the target device chooses not to retain, the binding views the “no info” status in the broker as more up to date than anything openHAB has chosen to restore, and overwrites it with UNDEF.
It’s an unresolved conflict of views.
https://github.com/openhab/openhab2-addons/issues/5879
I think, if i remember correctly my logs during startup, mqtt items gets restored after restart from persistence, but then after binding gets online it’s switched to UNDEF (if there is no retained msg).
I don’t see much room for philosophical discussion as it seems to be just bug … "i have data, hey i’m online and dont see any data, lets reset everything. … "
should be
“i have data, hey i dont have any so i’ll use what i have till data gets updated to mqtt”
I agree with you, but let me present the opposing view:
Consider some other binding, that polls a device. It’s broken, the binding knows its broken, there is no sensible data, it should update channel/Item to UNDEF. This is a Good Thing.
The parallel for MQTT - at startup, we interrogate / refresh all topics, because they might be retained, we cannot know but the broker knows.
If retained, we update channel/Item with the broker supplied value, hurrah.
If not retained, the broker responds “no data” effectively. This gets sent to channel/Item as UNDEF, no sensible data. It’s a kind of start-up poll, but of the broker not the device.
Later on of course, we might get a message on topic and update to a proper value.
At the same time, other bindings adopt a different philosophy. zwave devices for example may “sleep” all day. If you poll such a device, the zwave binding does nothing immediately, just queues the request. No UNDEF.
Maybe later we’ll get good data, and then update channel/Item.
Or even a definite failure, when we will update to UNDEF.
But there is no comparable retain mechanism in zwave.
yeah make sense, but still … UNDEF is way worse state on data than persisted old ones.
I mean, I do persist data for a reason, broker should not overwrite them IF there aren’t new valid ones (and UNDEF is not a valid data) or retained .
If broker is broken eg. needs to update stuff to UNDEF, wouldnt be better to signal it on broker level than updating all data to UNDEF which may and will cause a lot of issues in rules and other stuff.
So basically second part is working now as expected, first one just should ignore empty ones and use retained till a valid update.
See Retained huh, what is it good for? Absolutely. The only time it leads to unwanted behaviors is when you are using a retained message for something that is momentary (e.g. a message from a button push) or the message itself is ephemeral (e.g. a temp reading that is only really valid for five minutes at which point it will be replaced with a new value). In all other cases where the message represents a state, the retained flag should be set to true. This is how MQTT was designed to be used and it is how we should be using it in OH.
See https://github.com/openhab/openhab2-addons/issues/5879#issuecomment-524520334 for the research I did into the retained flag.
Properly using the retained flag would indeed replace restoreOnStartup for these Items and furthermore will allow devices to also be restored to their last commanded state when they restore their connection with the broker. This is as the MQTT protocol was designed and how it is expected to be used.
The behavior that the binding does with restore on startup has been deemed to be in error and there is an issue opened to make the binding not replace restored Items with UNDEF when it first comes up.
This is actually going to change as a result of the issue I linked to above. The binding will not set the Items to UNDEF any longer once that issue is addressed. But as you will see when you read that thread, I agree that the way the binding behaves now is correct but the proper solution is to rely on the retained flag instead of restoreOnStartup.
This actually is not working (i.e. the flag doesn’t actually do anything on the broker thing) and J-N-K is going to remove the flag. See https://github.com/openhab/openhab2-addons/issues/6100. You can define the retained flag on a Generic Thing Channel by Channel basis and you have the option to set a retained flag with the publishMQTT Actions. And J-N-K will add the option to set the flag for the LWT message. But the setting on the broker Thing will go away and it isn’t even working right now.
To elaborate on that a bit, the whole purpose of the retained flag is to allow subscribers to get the latest state from the publisher even if the subscriber didn’t happen to be online and connected when that state was published. So if the MQTT is set up properly, any Item that you would use restoreOnStartup to restore it’s state restored would more properly have the publisher of that message publish it as a retained message in the first place.
In cases where the message represents something ephemeral like a button press or a temperature reading that is only valid for a certain period of time, leaving the Items as UNDEF shouldn’t matter because they probably shouldn’t have been restoreOnStartup in the first place since the state is ephemeral, and if the publisher is working, it will get updated soon enough anyway.
But this is indeed behavior that is different from how most other bindings work.
In that case though the channel represents a state, which in MQTT would properly be published as a retained message. When the device has an ephemeral event like a motion detection, the device immediately publishes that value. So it could be proper for the binding to set that Channel to UNDEF. But the zwave binding doesn’t make a distinction between ephemeral events and states. For MQTT, that distinction is a little bit more apparent though, hence the conflict.
I lost the argument on the PR and really, if you are properly using MQTT retained messages it doesn’t matter. The binding will work properly either way, assuming that restoreOnStartup continues to occur before the binding starts up. We will have a problem though if restoreOnStartup values start to overwrite retained messages retrieved by the binding.
It depends. Like I said, if MQTT retained messages are used as designed, then Items that you would want to use restoreOnStartup on don’t need that because the latest state is stored on the broker. This is superior to restoreOnStartup because the publisher might publish a new message while OH was offline. With restoreOnStartup you would completely miss that state. With retained you would get the latest state from the device on re connection to the broker.
If the message is ephemeral, than using restoreOnStartup shouldn’t be used on those Items in the first place. UNDEF or NULL is the proper state because you don’t know how old that data restored actually is. It may no longer be valid or useful. Or it may represent an action that has already been acted upon.
Your Rules should be handling NULL and UNDEF in the first place. You should never assume in a Rule that an Item has a usable state.
Beyond that, there are many use cases where one needs to know whether the states being used are the most recent and up to date. Who wants to control their HVAC based on a thermometer that stopped reporting hours ago?
Even after the update, one should use retained for any MQTT message that represents a persistent state.
Looks like I missed quite a bit in my absence
Hi, you made my day 
and I can not thank you enough
I was looking for easy solution for my case where I use sonoff rf bridge with Tasmota to get data from several 433MHz contacts and motion sensors. Issue I have is that there is 1 MQTT topic and selection of item/sensor is done by payload. Retaining is not working in this case.