MQTT Qos and Retain Issues With Home Device

1

Hi All

I recently deployed a homie sensor script to publish some values for my automation on my raspberry pi OH Hub. I have noted the following, when I add a thing for my homie service, everything works fine until i reboot. Then MQTT bridge on my OH starts misbehaving. The homie sensor values published via MQTT starts flipping between an old value and newly published value. The home thing and other MQTT things show offline. The items which which donot use QOS and retain are perfectly fine. But the homie items keep alternating values. The solution is to restart the mqtt service and then restart homie. Then all is good.
I have not been able to fully grasp why MQTT Bridge on Openhab starts misbehaving. Any ideas? I deleted it and recreated it with same ID but issue persists.

2

Is this Item configured with restoreOnStartup (the default)?

What published the new value and when in relation to the restart?

3

no. they are not configured to be restored. the Homie items are still very new in an otherwise stable environment.
My assumption is that since Homie items are published with QOS=1 and retain=true, that somehow contributes to retention on the part of MQTT broker. I am not sure if the retention is valid post reboots/restarts.

4

Just to be clear, you deliberately configured these Items so they are not restored on startup. The default behavior is to always restoreOnStartup all Items for most persistence add-ons.

Here’s a rundown on how retention works.

When a publisher posts a message with the retained flag the MQTT broker will store that message. If there was already retained message on that topic, that existing message gets replaced.

When a client connects to the broker and subscribes to a topic, if there is a retained message on that topic, that message is delivered to the client.

It doesn’t matter if the client has already received that message on a previous connection. Every time the client reconnects it will get that retained message. QOS does not change this behavior.

A non-retained message does not replace a retained message. Even if there are 100 non-retained messages sent after a retained message, that retained message remains and will be delivered.

Here’s a rundown on how QOS works.

QOS 0: fire and forget; the broker sends the message and assumes it got there
QOS 1: broker sends the message and waits a certain amount of time for the subscriber to acknowledge receipt of the message, sending it again if no ack is received in that time
QOS 2: sets up a series of handshake messages between the subscriber and the broker to ensure delivery of the message only once

Notice how this exchange is only between the broker and the subscriber.

So if you have a retained message with QOS 2, when the subscriber connects to the broker it will receive that retained message only once through the QOS message exchange protocol. But the broker doesn’t keep track that it’s sent that message to that client before. On every connect for every client, every time, it will send the retained message.

I believe that Homie will only publish sensors and status with retained. It doesn’t make sense to publish commands to actuators as retained.

I don’t think that the retained, by itself, is the root cause of this problem. But based on what you described restoreOnStartup + retained messages + timing might play into it. restoreOnStartup only happens once during startup and the retained message is only sent once when OH connects so that wouldn’t explain the loop.

Do you have any rules that trigger on these Items?

5

Thank you for the insights. I misunderstood the restoreonstartup, then it is definitely restoredonstartup as that is the default on my persistence.

Now given the explanation for MQTT QOS behavior, agreed. i dont see a problem and i dont see a problem with retention. What seems to be an issue is that the broker on OH is failing during startup and keeps trying to start. This probably causes the MQTT to keep sending the last message submitted. However the strange portion for me is that the last message provided is more than 3 days old. that value is being retrieved from somewhere. I could stop these from values from being restored on startup. One thing which is certain which i can highlight is that when i reboot or restart OH, i can see that Local broker is failed because half of my tasmota MQTT devices will be showing offline. I have to disable the local broker then enable it in OH. Only then will all MQTT devices come online. Then lastly i start my homie device.

6

That definitely could explain this behavior.

What do you see in the logs? You might need to put OH into debug logging.

Are you using a different client name for every client that connects to OH? Every client (e.g. every MQTT Broker Thing in OH, every external tool like MQTTExplorer, etc.) must have a unique client ID. If you reuse it, you’ll get into a loop as one client connects and kicks the already connected client with the same ID off the broker which then causes that client to try to reconnect which kicks your client off the broker and so on. Every time you connect you’ll get the retained messages again.

7

Thank you. I will enable debug and see what shows up. All my clients have unique ID.

8

Well after enabling debug for the mqtthomie, I did note that the homie thing is starting and stopping without raising an error.i will post the entries. Seems to be an issue on mqtt on OH, I enabled mqtt generic debug, but I have not yet picked up anything to go on. The behaviour is showing that MQTT broker is having issues. Today after multiple restarts I found the broker being the main trouble. Not sure if there is a way reset the mqtt binding without loosing the current devices which are a mix of manually created things and automatically detected things.

9

If you delete the Broker Thing and recreate it with the same UID everything will work just fine.

If you use a different UID, worst case senario is you’ll have to change the config for all the MQTT Things to use the new UID for their bridge and rediscover your Homie Things (I suspect you can’t change the broker through it’s configuration).

10

i already deleted and recreated broker with same id yesterday. That didnt resolve it.

here is the portion for the Homie debug log

image
image1441×342 9.65 KB

11

Created new broker and assigned the broker to the things which brought the issues. Behaviour is the same most mqtt things on both brokers will not start automatically. Uninstalled mqtt and reinstalled without change in behaviour. I guess my next stop probably to ignore the autodiscovery and use raw mqtt directly as I can see the correct topics and values on mqtt bus.

12

And file an issue.

13

Thanks Rich, Too late now, it was driving me nuts. All looks good now. Took the plunge and started from scratch with a new install. Basically moved from OH3 to OH4 new everything. It will be couple of weeks before all automation is working.