[CLOSED] MQTT binding version 2.4 (Pre-release !)

I think you don’t need to escape them with the new binding. Could you try?

Hi David.

I have tried that a few times before and didn’t get any data. Since I had the same issue as @opus with the No MQtt client, I have tried to separate the broker into a different file and this in addition to removing the escaping \ solved my issues.

Its working :wink:

No not really. The problem is: Each time you edit the thing file that contains the bridge config, openHAB is shutting the broker Thing down and starting it up again. And I messed up. All mqtt Things will not get the new established connection but an old one, that’s why they tell you that there is “No MQTT client”.

This should not happen as long as you don’t edit files.

1 Like

It is when every service you have, python broadlink-mqtt, a python/c# camera service, python raspberry pi based DHT sensors, Arduino senhsors already deployed, a python alarm bridge - is using mqtt, and not using homie to start off with.

This is a resolved issue, either in OH 2.4 release or OH 2.5 snapshot, not sure rn.
[/quote]

So a restart should (have) solved the problem.
However initially it seemed not:

21:11:21.171 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:broker:opusMQTT’ changed from UNINITIALIZED to INITIALIZING
21:11:21.409 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:broker:opusMQTT’ changed from INITIALIZING to OFFLINE
21:11:21.456 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:topic:opusMQTT:sonoff_1’ changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
21:11:21.533 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:topic:opusMQTT:sonoff_1’ changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
21:11:21.600 [INFO ] [o.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to ‘192.168.178.20’ with clientid paho51263220137752 and file store ‘/var/lib/openhab2/mqtt/192.168.178.20’
21:11:21.735 [INFO ] [ransport.mqtt.internal.ClientCallback] - MQTT connection to ‘192.168.178.20’ was lost
org.eclipse.paho.client.mqttv3.MqttException: MqttException
at org.eclipse.paho.client.mqttv3.internal.CommsCallback.run(CommsCallback.java:220) [211:org.eclipse.paho.client.mqttv3:1.2.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap.forEach(HashMap.java:1292) ~[?:?]
at org.eclipse.smarthome.io.transport.mqtt.MqttBrokerConnection$ConnectionCallback.onSuccess(MqttBrokerConnection.java:134) ~[?:?]
at org.eclipse.paho.client.mqttv3.internal.ConnectActionListener.onSuccess(ConnectActionListener.java:99) ~[?:?]
at org.eclipse.paho.client.mqttv3.internal.CommsCallback.fireActionEvent(CommsCallback.java:321) ~[?:?]
at org.eclipse.paho.client.mqttv3.internal.CommsCallback.handleActionComplete(CommsCallback.java:260) ~[?:?]
at org.eclipse.paho.client.mqttv3.internal.CommsCallback.run(CommsCallback.java:190) ~[?:?]
… 7 more
21:11:21.900 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:broker:opusMQTT’ changed from OFFLINE to OFFLINE (COMMUNICATION_ERROR): MqttException

but after a minute:

21:12:21.915 [INFO ] [t.reconnect.PeriodicReconnectStrategy] - Try to restore connection to ‘192.168.178.20’. Next attempt in 10000ms
21:12:21.957 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:broker:opusMQTT’ changed from OFFLINE (COMMUNICATION_ERROR): MqttException to OFFLINE
21:12:21.997 [INFO ] [o.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to ‘192.168.178.20’ with clientid paho51263220137752 and file store ‘/var/lib/openhab2/mqtt/192.168.178.20’
21:12:22.046 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:broker:opusMQTT’ changed from OFFLINE to ONLINE
21:12:22.076 [INFO ] [smarthome.event.ThingUpdatedEvent ] - Thing ‘mqtt:broker:opusMQTT’ has been updated.
21:12:22.091 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - ‘mqtt:topic:opusMQTT:sonoff_1’ changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE

The remaining problem now is weird:
Broker and generic Thing are online, the device is connected to the broker as well. Switching the device via its webpage works (showing all log-entries as expected), however switching the linked item on the sitemap does not switch the device (ALTHOUGH the log-entries are the same as above).
Checking the commands with mqttSpy, it is obvious that the command given via the sitemap aren’t coming through.

Edit:
Could the last be caused by the “Is Command” setting from the Thing-page on PaperUI? Since I created the thing in a file, that setting can’t be used.

Solved the problem by changing the the thing-file to:

Bridge mqtt:broker:opusMQTT [ host=“myIP”, secure=false, username=“myUser”, password=“myPassword” , clientID=“opusMQTTClient” ]
{
Thing topic sonoff_1 “Test” {
Channels:
Type switch : Sonoff_1 [ stateTopic=“tele/sonoff_1/STATE” , transformationPattern=“JSONPATH:$.POWER” , commandTopic=“cmnd/sonoff_1/POWER”, on=“ON”, off=“OFF” ]
}
}

Note the “cmnd” in the commandTopic!!

This is a real bug, not yet reported. Will look into it.

I only see that after a restart, changing the file afterwards is working for now without issues!

You are kidding me, aren’t you? I have to switch from a properly working solution with zero-effort (just have the mqtt-eventbus.cfg set up properly with the 2 lines) to creating a new thing for everything AND to reference them in the .items files with mqtt=… stuff?

In that case I think I will stick with the version 1 as long as it works with OpenHAB and when the time comes that it does not, I’ll see whether it’s worth the effort to migrate or switch to some other HA project as this what has been described would be a huge effort in my case and certainly not done in just on a weekend (which I don’t have for just migrating a binding version).

Sorry guys, but I don’t understand why a well working solution has been abandoned instead of carried over (I even haven’t had a demand to improve anything as it just works).

1 Like

If there was a way of pulling things from the jsondb or running items and making the things, I believe a lot of this migration shock would be alleviated. Thankfully, personally I can spin up a new instance to look at creating the thing files…and honestly proposing a move to HA you would be up for at least as much effort (if not more…trust me…I have both running, converting all your rules is a significant exercise and so is grouping, getting the ui to work).

There are no plans to cease the support.

If you are a very lazy person, you would create exactly one Broker Bridge Thing and one Generic MQTT Thing (2 lines). And then you add a channel for each item you have at the moment (x lines for x items).

No interruption so far with your working setup.

Next you would convert your x item configuration lines one by one whenever you have time and link them to the created channel. (Switch abc [channel="channel-id"]).

I’m really sorry to sound a little offending at this particular time, but openHAB 2 uses the Things/Channels/Items concept. So all your items are linked to channels, no configuration on items like in the 1.x days. If you still doing this, you are not really using OH 2 and you have exactly no right to complain, because you are technically doing it wrong you are doing it in a very old fashioned way.

No, that isn’t the best way to replace the event bus.

I’m looking into it and will post a tutorial when I get it to work but my current idea for an approach is:

Outgoing:

  1. Create a Group
  2. Add all the Items you want to publish to the event bus to this Group
  3. Create the following two Rules (or only one depending on how you have the event bus configured):
    a. triggers on Member of MyGroup received command
    b. triggers on Member of MyGroup received update
  4. The body of the Rule is pretty simple. Call the new MQTT Action along the lines of
    a. actions.publishMQTT("openhab-1/out/"+triggeringItem.name+"/command", receivedCommand.toString)
    b. actions.publishMQTT("openhab-1/out/"+triggeringItem.name+"/state", triggeringItem.state.toString)

That’s it for outgoing.

I haven’t thought much about the incoming side of things but I think it will require creating a separate Channel for those topics on the receiving OH and linking those channels to the Items that represent the remote Items. Perhaps two Channels as I don’t know if Channels are one way or not.

Like I said, I haven’t fully thought this through but as far as I can see we are looking at the addition on one new Group, two three line Rules, and creating Thing Channels for all the topics you care about in a remote OH instance. It’s not zero extra work but it is far less work than you make it out to be.

If there were a way to learn the topic that an event comes in on it would be even simpler. And I believe this IS possible in the NGRE and MQTT Rule triggers, but I haven’t got that far yet. But if I’m correct then you just need one Channel that subscribes to “openhab-2/out/+/command” and inside the Rule you can figure out which Item the new state needs to be sent to.

Yes everyone, the cheese has moved. It is going to be some work. And the roll-out of this new binding could have been managed better (hopefully lessons learned are being taken, especially when the HTTP 2.x version binding starts to roll out). But David is doing his best to give us the same capabilities we have with the MQTT 1.x version binding while remaining compliant with the architecture and constraints of OH 2.x.

And assuming I can make it work like I think I can in the NGRE, I can publish a library on github (maybe someday the IoT Marketplace) so you don’t have to implement anything yourself. Just import the Rules and you are good to go.

OH 2 works very differently from OH 1 which is why bindings have to be rewritten for OH 2, not just simply modified and ported. Many things that are possible and make sense in OH 1 are not possible in OH 2, or have to be done in radically different ways.

The event bus hasn’t been just willy nilly abandoned. Access to the event bus in the ways necessary to make it work is not allowed in OH 2.x, or at a minimum not encouraged.

For everything there is a price to pay. This is part of the price we have to pay to get automatic discovery of devices and many of the other and growing benefits that we get through the use of Things.

I don’t think this would be all that hard. One would need to write a parser that can extract the mqtt= from Items and create the appropriate Channels or Things. It’s just a text to text conversion really with maybe a REST call if the script is to store the Things/Channels in JSONDB instead of .things files.

Someone has to step up to do it though. It won’t be me. But it won’t be hard I think.

2 Likes

Hi David
Text examples of new MQTT is appreciated!

mqtt.things
Bridge mqtt:broker:My [host=“1.1.1.1”, …] {
Thing topic sensors {
Channels:

}
}

Hi there,

If you work with e.g. Node-Red as an external rule engine then there is definitely a need for an Event Bus.

I would need a State Publish topic like this ->

statePublishTopic=/hq/state/${item}

where I can get the state of my things and a CommandPublishTopic like

commandPublishTopic=/hq/command/${item}

to use Node-Red to modify the state of my things. How do I accomplish this with the new MQTT binding?

Regards
Stephan

1 Like

Node-Red can use the http REST API of openHAB, which is also preferable. Both MQTT and the REST API are utf-8 string based TCP connections, so there is no speed or efficiency difference.

You need rules like this:

rule "Publish all item changes"
when
  Item changed
then
  val actions = getActions("mqtt","mqtt:broker:myUnsecureBroker")
  actions.publishMQTT("hq/state/"+item.name,item.value)    
end

(This is a bit of a fantasy rule, I’m too busy to do the full work for you here)

Cheers,
David

1 Like

First of all, thank you for your example, I’ll give it a try.

I’ve got a quite complex rule set implemented in Node-Red and would have to change all input objects.
Why is the use of the HTTP REST API preferable over MQTT?

Regards
Stephan

The REST API is part of openHAB core and guaranteed to be available. MQTT is an extension.

I’m migrating to the new MQTT binding

I have some problems with a switch Item, can someone please help me out converting this to the new binding?

My problem is the MAP transformation for outgoing

Switch HP_RunSet		"Power"			            		                    	
{ mqtt=">[mosquitto:heatpump/set:command:*:MAP(ACPower.map)],<[mosquitto:heatpump:state:JSONPATH($.power)]" }

ACPower.map

ON={"power":"ON"}
OFF={"power":"OFF"}

I have tried this configuration, but it gives me an invalid Json error at the Broker

Thing topic Mitsubishi {
        Channels:
           
            Type switch : HP_RunSet "Power" [stateTopic="heatpump" , transformationPattern="JSONPATH:$.power", commandTopic="heatpump/set", profile="transform:MAP", function="ACPower.map" ]

My understanding is there is a proper openHAB Node in Node-Red and no need for MQTT at all.

See my post above.

You may have to set each Channel up individually, at least for the short term.

I believe the MQTT 2.4 binding does not support transforms for outgoing MQTT. The work around for the time being (besides sticking with MQTT 1 of course) is to write a Rule and use the MQTT Action to publish the transformed message. Something like

rule "HP Run Set"
when
    Item HP_RunSet receivedCommand
then
    val message = transform("MAP", "ACPower.map", receivedCommand.toString)
    val actions = getActions("mqtt", "mqtt:broker:myUnsecureBroker")
    actions.publishMQTT("heatpump/set", message)
end
1 Like

I think this is where we differ. Your understanding of “text files” seems to be “it’s json and not a binary blob”, and my understanding of “configuration via text files” is “I want to program my setup and recreate it in a similiar state”

Basically, I hate pointing and clicking and not understanding what the software underneath does. And if I have to reconfigure, say, all my Tasmota items, pointing and clicking and clicking and clicking ad nauseam. Loading my items or things file into my preferred editor, doing the search / replace, and then just rebuilding the docker image is what I consider “ease of use”

But on the latter point, I guess @David_Graeff and I would disagree a little bit :slight_smile: