Reliability of binding-mqtt

Hi Guys,

I’m struggling arround with the MQTT Binding since two weeks and cannot get this working reliable. I’m using the most recent version of openHAB on Ubuntu 18 (VM) from the unstable repo.

Unfortunately the binding seems to break after an uncertain time. The behavior is different. The last days the MQTT connection to the broker was just stopped. I only got a message in the openhab.log, that the broker connection was stopped:

[INFO ] [penhab.io.transport.mqtt.MqttService] - Stopping broker connection 'mosquitto'

Yesterday I’ve

  • uninstalled the binding completely (again)
  • deleted the configuration via karaf console (config:delete org.openhab.mqtt)
  • stopped openhab2 service completely
  • issued openhab-cli clean-cache
  • started openhab2 service
  • re-installed the mqtt binding

and after that everything was working again. Today there is another issue, for which I’m currently completely blind. There are no log messages or anything else, but openhab doesn’t receive any MQTT messages (or just ignores them) and each command sent out is not received by the broker.

Now, after around 3 hours of MQTT not working in openhab, everything seems to work again. But for how long? I wasn’t able to get the MQTT broker connection working for more than 48 hours. Half of my rules are not working correctly, because they are time-based and MQTT didn’t work at the time the rules was executed.

I’m in despair. Even with enabled debug log through karaf there are no more logs, describing the issue.
Does any encounter the same issues with MQTT?

For now I only need MQTT for my Sonoff Switches. Currently I’m thinking of using the HTTP API of the Tasmota webserver in combination with the HTTP binding. That should be more stable and reliable.

I’m using mosquitto as broker and mqtt-spy for checking the reliability of mosquitto. For now I just had a single issue with mosquitto since it’s running. Each update is received from all Sonoff Devices. openhab is the only MQTT client, which doesn’t work well with mosquitto.

I appreciate every hint on that problem.

Thanks in advance

Cheers,
Sven

Had the same issues. MQTT 1.x stopped working today so I took some time and migrated my config to the brand new MQTT 2.4 binding. Can’t say how stable this will be as it is running for a few hours only but what I can say is: it seems to be faster. My lights turn on and off quicker (I use zigbee2mqtt for my zigbee lights).

I saw some strange behavior of the mqtt1 binding also during some recent snapshot build (I think it was 1455)
I didn’t spend time to debug it. For me, it was reconnecting continuously for no apparent reason (broker was up)
Since 1458 I haven’t experienced the same (so far). Upgrading now to 1460.

I’ve tried this as well, but I have two problems with this:

  1. The broker connection has issues. I saw the log message, that I should provide a valid hostname/IP. But everything was configured properly and thing state was online. I read somewhere, that someone else had the same issue and the solution was to still use the MQTT binding 1.x, because 2.4 binding is still in development and unstable.
  2. For Sonoff devices I need to consider 4 topics to always be aware of the current item state. The binding does only provide the possibility to configure one topic for state detection (or I didn’t find the correct syntax to configure several topics properly).

So it seems, that using the MQTT 2.4 binding is neither recommended or working for me.

But thanks for your suggestions.

Cheers,
Sven

I don’t know where to find the recent build number of the binding itself, but I’ve expected, that these bindings would be updated by the packetmanager or on the fly over the internet. Is there any other way to update a specific binding to a recent snapshot from Github, without putting it into the custom add-on folder?

It will “replace” (as in: make it obsolete) the mqttv1 binding within the next couple days,
which stops all support for the old binding.

This is an unusual configuration and I really blame the developer for an absolute misunderstanding of how mqtt topics should be assembled. But you can still make this work with Mqttv2. You need 4 channels, all linked to 1 item. The item will always be up-to-date this way.

These are amazing news. Thanks for sharing them.

Yeah, you’re right. But I think that this is not only up to the developer, but for my configuration as well. I’m still a newbie to openhab2 and advancing my knowledge from day to day.
I’m using several topics to get state updates. This is especially related to HABPanel, which do only show updated or persistent states, when HABPanel is started. For some reason my persistence may not work correctly. Therefore I’m using status updates from my Sonoff devices, to gather the current state of my switches, instead of changing the state of the switch manually after opening HABPanel, to get the current stain. For now my item configuration is like

Switch SonoffPs01_Switch "Sonoff Power Switch 01 Switch" (SonoffPowerSwitches_Basic) { mqtt="
  <[mosquitto:stat/sonoff-ps01/POWER:state:default],   
  <[mosquitto:stat/sonoff-ps01/RESULT:state:JSONPATH($.POWER)],
  <[mosquitto:tasmota/sonoff-ps01/STATE:state:JSONPATH($.POWER)],
  <[mosquitto:tele/sonoff-ps01/STATE:state:JSONPATH($.POWER)],
  >[mosquitto:tasmota/sonoff-ps01/cmnd/POWER:command:*:default]
" }

I’m pretty sure, that there may be a better way to get the current state in HABPanel without waiting for the updates from the Sonoff device. For now I don’t use retain in my MQTT configuration. This may be the solution for this, or am I’m wrong?

Cheers,
Sven

This would be a solution. You must set the retain on the sonoff itself, not in OH. It is the device that sends the message that controls whether it gets retained.

Make sure you are not reusing client IDs across multiple connection/devices to the broker. Each connection from each device needs to use a unique client ID or else it will kill off the old connection with that ID ending up and connecting/disconnecting loop as the two clients fight for access to the broker.

This is how I would expect it to work. In MQTT 1.x, for every <[blah blah blah] in the binding config you would need to configure a separate channel.

The OH core and all the bindings are upgraded as a unit. Typically, if you want an update to a binding you should upgrade the whole OH. In some circumstances you can upgrade an individual binding and run it on an older OH core but this requires downloading the binding jar file separately and dropping it into the addons folder.

You can find the versions of your bindings in the karaf console.

openhab> bundle:list | grep -i mqtt
247 x Active   x  80 x 1.13.0.201811220742    x openHAB MQTT Binding
260 x Active   x  80 x 1.13.0.201811220742    x openHAB MQTT Transport Bundle

The fourth column is the binding version.

Many thanks for all of the hints.

I’m gonna give that a try and will report back. Possibly my issues when trying the mqtt2 binding were related to the fact, that almost the whole time I had the mqtt1 binding installed and active as well (although they were using different client IDs).

The MQTT support uses the eclipse paho library as a base. Recently that was updated to the newest version for extended features.

It could be, that mqttv1 now eats up threads over time, because the disconnect API behaves slightly different now. If I find time, I’ll have a look at the that old code.

My first attempt is not successful. I was able to get the mqtt2 binding working correctly and I can control the switch via this binding now. But after the 4 channels were created I got 4 switches in PaperUI Control as well, which do not change, if I’m switching the on the device itself. I’ve made something wrong…

Paper UI will show you 4 switches that is correct. Paper UI is a maintenance solution and not perfect in regard to visually displaying controls.

But if you operate one switch all others should be in sync. AutoUpdate behaviour might need to be set to follow if you have manually set it to veto.

All switches have their correct state topic set? Have you tried to turn on the “is command” configuration option? I’m not sure if that is required in this configuration.

just in sense of transparency: I’m using OH 2.3 with MQTT 1.12 binding and no issues at all. BUT I’ve scheduled a restart of OH service each night 3am in order to avoid issue e.g. when I unplug and reconnect my logitech harmony devices. That means, MQTT binding will never run more than 24hours. Within these 24hours everything works like a charm.

That’s ok for me. Just wanted to make sure, that I’m using the correct channel type. Later on I’m using HABPanel for controlling everything.

OK. I’ve found the issue. The state topic was wrong. Now it works correctly! Many thanks!!

Is it possible to configure the whole thing with a configuration file? Currently I’m using 7 Sonoff switches and configure all of them via PaperUI is annoying. Based on the example for configuring things in configuration files I cannot figure out how to configure it, because the mqtt2 binding does still not have a valid documentation, that I’m aware of.

Thanks for the idea, but I’m not really thrilled by the idea of using scheduled restarts to workaround such issues, if there are other ways to achieve a reliable solution.

I’m currently in the process of migrating FHEM to openHAB. openHAB has a much cleaner configuration logic implemented than FHEM. I wasn’t able to configure advanced things in FHEM on my own after 2 years, without the use of community hints. The configuration in openHAB is much more straight-forward. However, for now FHEM was more reliable and less fragile when it comes to things like caching. But to be honest, I’ve never used MQTT with FHEM. That was the point where I’ve decided to give openHAB a try.

got it, but please keep in mind that the shared information about the scheduled restart (of the service! not the whole machine) actually wasn’t a suggestion :slight_smile:

I just wanted to share that within 24hours everything is fine with the MQTT service 1.12. and I didn’t implement the restart because of MQTT, but because of my logitech binding as soon as I noticed that a physical power outage of my logitech harmony hub generated problems in openhab. maybe that’s not the case anymore with the latest version. anyway, as it doesn’t harm I let it restart every night.

In an earlier response you said that we would ongoingly be able to run both the v1 and v2.4 MQTT bindings concurrently. That still OK ?

I ask because there seem to be several features in the v1 binding that are awkward or even not going to be possible in v2.4. Yes, indeed they might not have been elegant or correct but they worked. Backwards compatibility is important. There also seems to a rather antagonistic approach to solving these issues and a very dismissive view of some implementations that should be encouraged to change if they want to work with OH.

V1 works for me … my whole system revolves around MQTT and I was initially excited by v2.4 but now I’m apprehensive and indeed it’s tainted my OH comfortability a little. OH is not my only or even main controller but I sense discordances.

I believe that this is still valid (David will re-confirm for sure)

MQTTv1 will continue to use the “old” ConfigAdmin PID: org.openhab.mqtt and it’s own bundle names (org.openhab.binding.mqtt & org.openhab.io.transport.mqtt)
For MQTTv2 binding: PID: org.eclipse.mqtt and the bundles are: org.eclipse.smarthome.binding.mqtt & org.eclipse.smarthome.io.transport.mqtt & org.eclipse.paho.client.mqttv3)

You can test this theory in a demo system by installing both addons and putting some basic configs.
I used M8 and they work fine in parallel.

Because oh 1.x and 2.x architectures are so drastically different, backwards compatibility is simply not possible. See the exec 1.x and 2.x bindings and how fundamentally different they both work. This will be a similar case.

I can confirm that you can run MQTT 1.x and 2x at the same time. And so far the only thing that I’m aware of that isn’t going to be supported is translations on outgoing publishes, which you can implement in a Rule. And even this will come back eventually

But if you are expecting to be able to just copy your existing mqtt configs and go you will be disappointed. 1.x and 2.x bindings are fundamentally different.

You are late to the game to tell your requirements, at least for oh 2.4. What are you still missing from the binding?

With mqtt v1 not supported I do not mean that it will not run. But as usual in software development the underlying libraries will be updated over time and if mqtt1 at some point in the future behaves strange or limited, there will be no one that maintains the old implementation.