Move from MQTT 1.14 to 2.5.12

I still use openHAB 2.5.12-1 on an Ubuntu 18.04 virtual machine and on 8 Raspberry Pi`s with openHABian. Of course I have a look at this. This first thing I was not sure is if the Remote openHAB binding could be used as an alternative to MQTT. We are using MQTT because of different robots and two Microsoft Hololens VR glasses. Also other external self written software are using MQTT. As far as I understand the remote openHAB binding would work to use multiple openHAB instances but it does not use MQTT. With the old MQTT 1.14 binding we have also both. Multiple openHAB instances work together and different devices and software can communicate with openHAB.

Sadly the old binding does not installed again. I dont know why because I have disabled that new addons should be installed. Maybe its a problem with sudo apt-get update or other typical linux things because I thought the if the openHAB addons will not change, it should reinstall also the legacy bindings from the cache because it should not look on the remote for new updates. Since this doesn’t work that way and somehow causes problems, I was forced, willy-nilly, to update the MQTT. Unfortunately, it was a bit unsuccessful.

I hope you can help me to rebuild the same topology as before. Thanks in advance.

The old MQTT 1.X configuration

On the VM I have this mqtt.cfg:


#
# Define your MQTT broker connections here for use in the MQTT Binding or MQTT
# Persistence bundles. Replace <broker> with an ID you choose.
#

# URL to the MQTT broker, e.g. tcp://localhost:1883 or ssl://localhost:8883
MQTTBroker.url=tcp://localhost:1883

# Optional. Client id (max 23 chars) to use when connecting to the broker.
# If not provided a random default is generated.
#<broker>.clientId=<clientId>

# Optional. True or false. If set to true, allows the use of clientId values
# up to 65535 characters long. Defaults to false.
# NOTE: clientId values longer than 23 characters may not be supported by all
# MQTT servers. Check the server documentation.
#<broker>.allowLongerClientIds=false

# Optional. User id to authenticate with the broker.
#MQTTBroker.user=openhabvm

# Optional. Password to authenticate with the broker.
#MQTTBroker.pwd=Ghj%57fD

# Optional. Set the quality of service level for sending messages to this broker.
# Possible values are 0 (Deliver at most once),1 (Deliver at least once) or 2
# (Deliver exactly once). Defaults to 0.
MQTTBroker.qos=0

# Optional. True or false. Defines if the broker should retain the messages sent to
# it. Defaults to false.
#<broker>.retain=<retain>

# Optional. True or false. Defines if messages are published asynchronously or
# synchronously. Defaults to true.
MQTTBroker.async=false

# Optional. Defines the last will and testament that is sent when this client goes offline
# Format: topic:message:qos:retained <br/>
#<broker>.lwt=<last will definition>

And this is the eventbus:

# Name of the broker as it is defined in the openhab.cfg. If this property is not available, no event bus MQTT binding will be created.
broker=MQTTBroker

# When available, all status updates which occur on the openHAB event bus are published to the provided topic. The message content will
# be the status. The variable ${item} will be replaced during publishing with the item name for which the state was received.
statePublishTopic=/messages/states/${item}

# When available, all commands which occur on the openHAB event bus are published to the provided topic. The message content will be the
# command. The variable ${item} will be replaced during publishing with the item name for which the command was received.
#commandPublishTopic=

# When available, all status updates received on this topic will be posted to the openHAB event bus. The message content is assumed to be
# a string representation of the status. The topic should include the variable ${item} to indicate which part of the topic contains the
# item name which can be used for posting the received value to the event bus.
#stateSubscribeTopic=

# When available, all commands received on this topic will be posted to the openHAB event bus. The message content is assumed to be a
# string representation of the command. The topic should include the variable ${item} to indicate which part of the topic contains the
# item name which can be used for posting the received value to the event bus.
commandSubscribeTopic=/messages/commands/${item}

On the Raspberry Pis I have this mqtt.cfg`:

#
# Define your MQTT broker connections here for use in the MQTT Binding or MQTT
# Persistence bundles. Replace <broker> with an ID you choose.
#

# URL to the MQTT broker, e.g. tcp://localhost:1883 or ssl://localhost:8883
MQTTBroker.url=tcp://192.168.0.5:1883

# Optional. Client id (max 23 chars) to use when connecting to the broker.
# If not provided a default one is generated.
#<broker>.clientId=<clientId>

# Optional. User id to authenticate with the broker.
#MQTTBroker.user=openhabvm

# Optional. Password to authenticate with the broker.
#MQTTBroker.pwd=Ghj%57fD

# Optional. Set the quality of service level for sending messages to this broker.
# Possible values are 0 (Deliver at most once),1 (Deliver at least once) or 2
# (Deliver exactly once). Defaults to 0.
#<broker>.qos=<qos>

# Optional. True or false. Defines if the broker should retain the messages sent to
# it. Defaults to false.
#MQTTBroker.retain=false

# Optional. True or false. Defines if messages are published asynchronously or
# synchronously. Defaults to true.
MQTTBroker.async=false

# Optional. Defines the last will and testament that is sent when this client goes offline
# Format: topic:message:qos:retained <br/>
#<broker>.lwt=<last will definition>

The mqtt-eventbus.cfg looks like following:

# Name of the broker as it is defined in the openhab.cfg. If this property is not available, no event bus MQTT binding will be created.
broker=MQTTBroker

# When available, all status updates which occur on the openHAB event bus are published to the provided topic. The message content will
# be the status. The variable ${item} will be replaced during publishing with the item name for which the state was received.
#statePublishTopic=

# When available, all commands which occur on the openHAB event bus are published to the provided topic. The message content will be the
# command. The variable ${item} will be replaced during publishing with the item name for which the command was received.
commandPublishTopic=/messages/commands/${item}

# When available, all status updates received on this topic will be posted to the openHAB event bus. The message content is assumed to be
# a string representation of the status. The topic should include the variable ${item} to indicate which part of the topic contains the
# item name which can be used for posting the received value to the event bus.
stateSubscribeTopic=/messages/states/${item}

# When available, all commands received on this topic will be posted to the openHAB event bus. The message content is assumed to be a
# string representation of the command. The topic should include the variable ${item} to indicate which part of the topic contains the
# item name which can be used for posting the received value to the event bus.
#commandSubscribeTopic=

So in short this means that the MQTT-Broker runs on the VM and all devices use the same broker. And of course:

|Instance|Pub/Sub|Topic|

|VM|statePublishTopic|/messages/states/${item}|
|VM|commandSubscribeTopic|/messages/commands/${item}|
|Raspberry X|stateSubscribeTopic|/messages/states/${item}|
|Raspberry X|commandPublishTopic|/messages/commands/${item}|

So the subscribe and publish topics for command or for state are the same. No in or out to distinguish. The VM does update or publish the state and the Raspberry Pi`s will subscribe it. Then a command will be executed and published and the VM will subscribe it.

MQTT 2.5.12 Setup and Configuration

On the VM I have installed mosquitto and the mosquitto clients with following command:

sudo apt install mosquitto mosquitto-clients

On the Raspberry Pi`s I only installed the mosquitto clients:

sudo apt install mosquitto-clients

I will skip the installation of the binding.

On the VM I have created a thing with this configuration:

And on the Raspberry Pi`s as example a thing with this configuration:

What works is that it is online:

mqtt3

You can check it if you will disable or uninstall the mqtt-clients. So it is possible that you only have to use one MQTT-Broker (not the MQTT Broker things) and multiple mqtt-clients. But on each device you should have to create a thing to connect to the MQTT-Broker, as you can see above, with the external address to this one.

So if this would work, what`s my problem? My problem is that I have simply said not topics. With this configuration, one could say okay, then maybe it won’t work to connect to an external MQTT broker after all. That will probably not be the point, because the VM, where the MQTT broker runs locally, does not publish anything. Otherwise, one could have narrowed down the error and said that you would have to install an MQTT broker locally on each device and that the clients are not sufficient.

But this is actually not the normal way to build MQTT. One broker, several clients. Otherwise, you would have to create a bridge between the different brokers. Or you can create as example a mosquitto cluster.

Of course I cloned and “installed” richs [openhab-rules-tools](https://github.com/rkoshak/openhab-rules-tools) and copied it to the $OH_CONF`-folder:

cp -r rules_utils/automation $OH_CONF
cp -r mqtt_eb/automation $OH_CONF

Also I used sudo chown -R openhab:openhab automation so that the openhab user could run it:

ls -la
total 84
drwxrwxrwx  18 openhab openhab  4096 Jun 11 13:16 .
drwxr-xr-x 147 root    root    12288 Jun  8 09:35 ..
drwxrwxrwx   4 openhab openhab  4096 Jun 11 13:16 automation
drwxr-xr-x   8 root    root     4096 Dez 11  2020 .git
drwxrwxrwx   6 openhab openhab  4096 Jun 14 09:32 html
drwxrwxrwx   3 openhab openhab  4096 Apr 23  2020 icons
drwxrwxrwx   2 openhab openhab  4096 Jun 14 08:53 items
drwxrwxrwx   2 openhab openhab  4096 Apr 23  2020 misc
drwxrwxrwx   2 openhab openhab  4096 Apr 28 11:24 persistence
-rwxrwxrwx   1 root    root     1395 Dez 11  2020 Readme.md
drwxrwxrwx   2 openhab openhab  4096 Jun 14 08:53 rules
drwxrwxrwx   2 openhab openhab  4096 Jun  1 13:28 scripts
drwxrwxrwx   2 openhab openhab  4096 Jun  2 10:40 services
drwxrwxrwx   2 openhab openhab  4096 Jun 14 08:53 sitemaps
drwxrwxrwx   2 openhab openhab  4096 Apr 28 11:24 sounds
drwxrwxrwx   2 openhab openhab  4096 Jun 14 08:53 things
drwxrwxrwx   2 openhab openhab  4096 Mai 28 14:11 transform
drwxrwxrwx   2 openhab openhab  4096 Dez  8  2020 .vs
drwxrwxrwx   2 openhab openhab  4096 Dez  7  2020 .vscode

Next, we have look in this configuration:

cat $OH_CONF/automation/lib/python/configuration.py
# The Channel ID of the MQTT Event Bus subscription channel
mqtt_eb_in_chan = "mqtt:broker:broker:eventbus"

# The name to use for this instance of openHAB, forms the root of the MQTT topic
# hierarchy.
mqtt_eb_name = "remote-openhab"

# Thing ID for the MQTT broker Thing.
mqtt_eb_broker = "mqtt:broker:mosquitto"

# Optional flag, when True all Item commands and updates are published.
# Defaults to True.
mqtt_eb_puball = True

# If the mqtt_eb_puball variable is left out or set to False,
# the next step is to add the following tags to only those Items
# that should publish to the event bus.

# eb_update Items tagged with this will have all their updates published to the event bus.
# eb_command Items tagged with this will have all ther command published to the event bus.

# Switch Foo [eb_command] // publishes only commands to the event bus
# String Bar [eb_update] // publishes only updates to the event bus
# Number Baz [eb_command,eb_update] // publishes both command and updates to the event bus

Okay, maybe mqtt_eb_in_chan is wrong.

Unfortunately, I didn’t know what to do with the items. There was the following hint in the tutorial:

Switch Foo [eb_command] // publishes only commands to the event bus
String Bar [eb_update] // publishes only updates to the event bus
Number Baz [eb_command,eb_update] // publishes both command and updates to the event bu

So of course I thought I could use [eb_update] and [eb_command] separately to rebuild my old topology. But if I use one of this inside an .items file the items (better the complete group of an item) will not shown in the sitemap. So there are any errors.

Okay, I thought I can go on and later have a check what causes the error. Then I can read in the tutorial that you have to do something like this:

Group:String PubItems_CMD
Group:String PubItems_UPD

So I have done following:

Group:String PubItems_CMD
Group:String PubItems_UPD

Group gSmartHome "Smart-Home (Fakultaet IN)" <house> (PubItems_UPD)

I hoped this would be enough. Sadly not.

Then I have created this rule for publishing:

val eb_br = "mqtt:broker:mosquitto" // Thing ID of the broker connection
val eb_name = "OpenHABVM"              // <client ID>

rule "Publish commands to the event bus"
when
    Member of PubItems_CMD received command // eb_cmd_gr
then
    val mqttActions = getActions("mqtt",eb_br)
    mqttActions.publishMQTT(eb_name+"/out/"+triggeringItem.name+"/command",receivedCommand.toString)
end

rule "Publish updates to the event bus"
when
    Member of PubItems_UPD received update // eb_upd_gr
then
    val mqttActions = getActions("mqtt",eb_br)
    mqttActions.publishMQTT(eb_name+"/out/"+triggeringItem.name+"/state",triggeringItem.state.toString)
end

Okay at least I know that I have to remove /out/ and so on that I can use the same topic like in the old binding. At this point, however, we can dispense with subscribing for the time being and leave it out. Using the tool MQTT.fx, I have noticed that not a single topic is published. Subscribing could only work at the earliest when I would also publish something. I mean, I could be wrong, but with MQTT.fx I can scan all topics of the broker. This means that even if I have not yet adjusted the address of the Topics in the rule, I should definitely have had some Topics displayed. Unfortunately, this was not the case.

I hope I haven’t forgotten any important information. It should be detailed. Many thanks in advance.

The remoteopenhab binding maps Items to OH: it does not matter what the Items link to. MQTT 1 would work fine.

The remote repository has changed.

Yes. One real broker, many openHABs each with their own Broker Bridge Thing representing their individual connection the the one real broker.

Note that the broker needs to be able to tell its clients apart - you must have a unique client ID at each client.

I have now copied the MQTT 1.14 binding into the directory /usr/share/openhab2/addons to bridge the gap.

ls
openhab-addons-2.5.12.kar                    org.openhab.binding.playstation-2.5.9.jar                           org.openhab.io.transport.mqtt-1.14.0.jar
openhab-addons-legacy-2.5.12-SNAPSHOT.kar    org.openhab.binding.synologysurveillancestation-3.1.0-SNAPSHOT.jar  org.openhab.persistence.mqtt-1.14.0.jar
org.openhab.binding.benqprojector-1.9.0.jar  org.openhab.io.gcal-1.14.0.jar                                      README

I think it would work if I only would use org.openhab.io.transport.mqtt-1.14.0.jar I am not sure what is in the org.openhab.persistence.mqtt-1.14.0.jar executable or if both is needed. For now, of course, it works again. But I should change over anyway.

If the devices publishing and consuming the messages are all openHAB you can use the Remote openHAB binding. It only works through the openHAB REST API so likely won’t work with anything else.

The old service where openHAB put the jar files that openHAB downloads from was shut down. So your OH instance is looking in the wrong place and can’t find the add-ons. I think an apt update will fix it but don’t know for sure. Search the forum, there were several threads and announcements.

When you uninstall a binding, you remove it from the cache. The cache is where bindings get installed to.

You can’t keep that topology using the Python library unless you change the topology to match what is described in the tutorial or change the Python code to use this topology.

There were a number of problems with the 1.x MQTT EvenBus implementation which made it very easy to run into infinite loops. It also made it very confusing to understand what message was coming from where and really only efficiently supported two OH instances, not more. So my tutorial makes changes to address all these problems. But as a result, your existing topology is not supported by the library.

From what I can tell, you either did not read or did not understand the MQTT 2.5 EventBus tutorial. And there is a lot of confusion here. It will be easier to help if you step back, figure out if you want to change your topology or you want to change the code to support your existing topology, and then address the setup in smaller chunks. First get publishing working. Then tackle subscribing.

The EventBus configuration is not required nor is it generally preferable for integrating with anything but another openHAB instance. Typically, for your other devices you would create separate Things and configure the subscriptions and publishers for those devices as Channels on that Thing. As I state at the beginning of that tutorial:

Of course there should be only one broker. But each client must have a unique client ID when it connects. If each client connects to a different broker they couldn’t send and receive messages from each other.

But each client is going to have to connect to the address of that one central broker.

Just because I wrote a Python library that you can use for this does not eliminate the need to read and understand that tutorial in its entirety.

There is a whole section of the tutorial that explains that you must configure a Channel on the MQTT Broker Thing and it shows you how to do that. You then need to update that file to use the Channel ID you created. See the “Subscribing to the event bus” section of the tutorial.

The Item names must match on both OH instances. One instance will publish to <mqtt_eb_name>/out/Foo/command for commands and <mqtt_eb_name>/out/Foo/state for state updates. The other will subscribe to <mqtt_eb_name>/out/+/command for the commands and <mqtt_eb_name>/out/+/state for the state updates. Note <mqtt_eb_name> is the same in both cases. When a message is received on the remote openHAB, it triggers a rule, extracts the Item name from the topic and issues an update or command to the Item. So the Item names must match on both instances.

The tags indicate whether you want that Item to only publish commands, only publish states, or publish both commands and states.

The tags only control what the publisher does. You still have to create the Items on the remote openHAB instance.

There are two implementations of the event bus presented in that tutorial. The Python version which does not use Groups and a Rules DSL version which necessarily does use Groups. When in doubt, read the readme on the library github for the Python version. If it’s not mentioned there it’s not used by the Python version.

My overall suggestions are as follows:

  1. Decide which language you plan on using, the Python or the Rules DSL.

  2. Go back and reread the tutorial. If using Python reread the README on github for it.

  3. Pay extra attention to the topology sections in both.

  4. When you move to OH 3, reconsider the use of the Remote openHAB binding for the OH to OH integration. Then use Generic MQTT Things for anything else that is not OH. It will be easier over all to get working and it is a bit easier to understand because you don’t have to mess with the complexities of topics and MQTT, just a Thing with Channels that link to Items.

  5. Only work on one way, publishing. Work with subscriptions once you get that working.

  6. Message are only sent when an update and/or command is received on an Item. Topics will not exists on the broker until a message is sent. So when using a tool like MQTT.fx, you need to make sure that Items are updating/being commanded.

  7. Add logging everywhere. Always check the logs. You’ve written some Rules DSL rules and created an Item that should publish updates. You’ve presented no indication that you’ve looked at events.log to see if the Item is in fact updating. You’ve not added any logging to the rules to verify whether or not the rule is running. There is a whole host of things that can go wrong between the Item receiving an update and you seeing a message in MQTT.fx. You have to narrow the problem down.

I know that my English is not the best. But this is my goal:

I just think we do it the other way round with commands and states. Local commands and remote states. Remote is my VM and local is one Raspberry Pi.

Edit: No, it is the same direction remote/VM subscribes commands and publishes states. Local/Raspberry Pi subscribes states and publishes commands.

I’ll have another look at DSL and Python. So I must have missed something and tried something twice, or what exactly was the point?

Edit: I tested now both separately and it does not work.

Then as shown in the diagram:

Setting Raspberry Pi X VM
Subscribes to RaspberryX/in/#/state RaspberryX/out/#/command
Publishes to RaspberryX/out/<Item Name>/command RaspberryX/in/<Item Name>/state
mqtt_eb_name RaspberryX VM
Item Tags eb_command, or member of PubItems_CMD eb_update or member of PubItems_UPD

That will set it up so commands generated in RaspberryX are published out and received by VM. Updates that occur on VM are published out and received by RaspberryX.

VM will “own” the state so it’s probably a good idea to set the Items on RaspberryX to turn off autoupdate so that the Item doesn’t change in response to a command until VM reports the changed state.

Note: VM never publishes commands. Only RaspberryX can publish the commands for those Items. RaspberryX never publishes state updates. Only VM can publish the state updates. This is required to avoid infinite loops.

There is an infinite number of ways that it could not work. What’s wrong? :man_shrugging: Maybe post the full config… after reading and applying what I’ve said above.

And again, I’ll strongly recommend that your time might be better spent upgrading to OH 3, at least on one of these OH instances, so you can use the Remote openHAB binding.

(Unfortunately, I am afraid of being misunderstood, because I am a very, very, very, very sweet and calm person. I only try to explain in detail what I need it for, what we use it for, what I have understood and how, etc. So yes, some things are obviously duplicated, but I can only provide as much information as I can. I don’t want to end up being seen as aggressive because I discuss a lot. I also don’t want to make anyone look stupid or as if they haven’t understood something, because realistically I’m the one who has a hard time understanding some things, both technically and because of my English. I’ll try my best and thank you for your patience and endurance.)

I also don’t quite understand what other configurations you might need. I had obviously done too much, because I had overlooked the sentence with Python and DSL and had only copy-pasted, but a certain point in time.

The point is that I can not use the Remote openHAB Binding because I have different robots, two hololens, one unity 3D model and one hololens which are using MQTT and communicate to different devices using openHAB. Not every device uses MQTT, so the solution was to use openHAB, maybe use Philips Hue, HomeMatic or some ZigBee or ZWave devices, integrate them in openaHAB. Well I use different openHAB instances okay. There I could use the Remote openHAB Binding. But the advantage of my current solution is that if I use as example a switch inside openHAB, I can easily switch on and off a device over MQTT because I use an MQTT eventbus to update the states with commands. On so many other devices like robots, magic mirror and so on, I do not have to install something for ZigBee, ZWave, HomeMatic etc. Or if I want to control multiple devices I don not have to use REST, MQTT, ZigBee etc. I only have to use MQTT which makes it as example much easier to write own code on the robot and call a function multiple times for different devices because only the Item name and the state changes but not the way I try to access this devices. This is then done for me by the openHAB system, where I have stored it once. I mean I can’t find anywhere that the remote openHAB plugin works with MQTT, but via REST. But I have countless projects based on MQTT and many devices, so I’m not going to convert every piece of code from MQTT to REST just because I suddenly need to access openHAB differently.

Because we have several projects currently running that use MQTT, it was also immensely important that I got the legacy binding working again, otherwise you can’t continue working on the current projects.

So if there was a solution to keep MQTT running, I’ll do that.

Or to put it another way: If I were to let the openHAB systems communicate with each other via REST, then I would still need the option of at least one system communicating via MQTT to the devices described above. But to do this, I would first have to get MQTT working on a system so that I can see and access the topics somewhere. At the moment it doesn’t work that anything is published. This will probably be the first important point. My robot may have to retrieve data first. And if these topics work, the return path that I subscribe from openHAB would only be the second step.

Setting to avoid infinite loop:

**VM:**
statePublishTopic=/messages/states/${item}
commandSubscribeTopic=/messages/commands/${item}

**Raspberry Pi X**:
stateSubscribeTopic=/messages/states/${item}
commandPublishTopic=/messages/commands/${item}

Setting for Infinite Loop:

statePublishTopic=/messages/states/${item}
stateSubscribeTopic=/messages/states/${item}
commandSubscribeTopic=/messages/commands/${item}
commandPublishTopic=/messages/commands/${item}

Alternative setting for infinite loop:

statePublishTopic=/messages/states/${item}
commandSubscribeTopic=/messages/commands/${item}
commandPublishTopic=/messages/commands/${item}

Alternative 2 for infinite loop:

statePublishTopic=/messages/states/${item}
stateSubscribeTopic=/messages/states/${item}
commandSubscribeTopic=/messages/commands/${item}

I’ll spare myself the other alternatives. On a device, the same topic may not be published and subscribed to on command and/or state at the same time. We agree on that. I have now simply assumed the old configuration. Of course, this connection also exists with a possible new MQTT configuration. I mean, then the state either never changes or changes constantly. It depends on what triggers the infinite loop. And either a Swicht then switches permanently via a command or via a state, etc. It doesn’t matter what is ultimately in the infinite loop. You give control over a state, a command and ultimately the item or device. You can press a button ten times, the system is constantly sending out something else.

Event bus is event bus. It does not matter whether I want to realise this solution via the old binding or the new one.

I have one broker and 8 clients. So 7 Raspberry Pi’s with clients and logically there is also 1 client on the VM. However, the broker is only on the VM.

Here my mosquitto configuration:

cat /etc/mosquitto/mosquitto.conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

allow_anonymous true

socket_domain ipv4

Mosquitto is running because I use currently the MQTT 1.14.0 binding. Anonymous requests are true. No username and no password is needed. The MQTT Broker is installed on 192.168.0.5 and uses the port 1883. As you can see in the first topic the VM runs on 192.168.0.5 and there I use localhost or 127.0.0.1 for IP/Hostname of the MQTT broker. Client ID was OpenHABVM. Last Will Message is OFFLINE and Last Will Topic was OpenHABVM/status.

On the Raspberry Pi Xs I use of coure 192.168.0.5 for IP of the MQTT broker, port 1883, as Client ID shlPiGeneral, shlPiBad, shlPi… The Last Will Message are also OFFLINE. Last Will Topic was also the Hostname/status. On all MQTT Broker things I use Quality of Servie 0, Reconnect Time 60000, Heartbeat 60 and so on. The default settings. Of course no username and password. All things are Online if the mosquitto broker is started. Good sign. If I uninstall the mosquitto-clients on one Raspberry Pi`x the thing goes offline. If I reinstall it, it goes online again.

On all devices the things are named MQTT Broker. What is different is the Thing ID: mqtt:broker:mosquitto on the VM, mqtt:broker:general on shlPiGeneral, mqtt:broker:bad on shlPiBad and so on. Well it`s not really important.

Let’s work on the example that a device should first be able to publish.

cat $OH_CONF/automation/lib/python/configuration.py
# The Channel ID of the MQTT Event Bus subscription channel
mqtt_eb_in_chan = "mqtt:broker:broker:eventbus"

# The name to use for this instance of openHAB, forms the root of the MQTT topic
# hierarchy.
mqtt_eb_name = "OpenHABVM"

# Thing ID for the MQTT broker Thing.
mqtt_eb_broker = "mqtt:broker:mosquitto"

# Optional flag, when True all Item commands and updates are published.
# Defaults to True.
mqtt_eb_puball = True

I mean I am not silly mqtt_eb_name is as example OpenHABVM and mqtt_eb_broker is mqtt:broker:mosquitto. If I want to publish everything from this VM, it is correct to reference the Thing and specify the name accordingly. I always orient myself on the host name. But if this configuration does not work why it should work with the correct replacements on the other devices?

If I want to publish everything from a device first, like here, mqtt_eb_puball = True is also correct, because I first have to try somewhere to get that to work. Ideally, I should see all commands and state topics in MQTT.fx from this VM when I scan there. But in fact I only see the topics from the MQTT 1.14.0 binding. The topics are also called differently, so it’s not that there would be a duplication otherwise.

So because it is set to true I ignore tagging the items.

I skip or removed

Group:String PubItems_CMD
Group:String PubItems_UPD

because I don`t want to use DSL. I want to use python. So of course I download it and executed:

cp -r rules_utils/automation $OH_CONF
cp -r mqtt_eb/automation $OH_CONF

That would not yet be necessary for the first step, that I publish something via MQTT at all. But yes, it has been so far, for logically the same reason.

From your github repository:

I use openHAB 2.5.12-1. And this Misc Add-on is installed:

Rule Engine (Experimental)
misc-ruleengine - 2.5.12

Does rules_utils and/or mqtt_eb need Jython? I thought only a few would need Jython. I have never read to install Jython for the MQTT 2.5 eventbus. If this was the error. It is clear that it`s not installed correctly.

I do not changed the rules_utils.py file. Also there a init.py file. And the mqtt_eb_pub.py and mqtt_eb_sub.py files are also not changed.

I do not know if it is needed but the /etc/default/openhab2 part for Jython:

EXTRA_JAVA_OPTS="-Xms1024m -Xmx1536m -XX:+HeapDumpOnOutOfMemoryError -Xbootclasspath/a:/etc/openhab2/automation/jython/jython-standalone-2.7.0.jar -Dpython.home=/etc/openhab2/automation/jython -Dpython.path=/etc/openhab2/automation/lib/python"

I think it was the problem with Jython. Sorry my fault:

Unbenannt

I moved it to /etc/openhab2/automation/jython instead to /etc/openhab2/automation/jython/jython-standalone-2.7.0.jar. So it could not be executed.

Post all your relevant rules as they exist on your system right now.
Post all your relevant Items as they exist right now.
Post all your relevant Things as they exist right now.
Post your logs around the time that you’ve attempted to use the event bus.

It’s not enough to just say “I did X and it doesn’t work.” We need to see “X” exactly as it exists, letter for letter. Anything short of that is going to result in us having to ask a whole lot of questions and get frustrated.

Don’t redact configurations. Don’t assume you know what is and is not relevant and only show us the relevant bits. When you withhold information you deprive us of the information we may need to help solve the problem.

And you’ve discussed a whole bunch of things that you’ve tried along the way. It’s really impossible for anyone on this thread to really understand what state your MQTT stuff is in. Did you remove the Rules DSL rules? You didn’t say so. Are you planning on only using the Rules DSL rules? You didn’t say that either. So I don’t even know at this point whether you are using the Python rules or Rules DSL rules.

Did you install Python and the Python Helper Libraries? You never said. I assumed yes but you never posted any evidence either way. Maybe the whole problem is that you’ve not set up Python but I can’t know that.

But I could quickly scan through the configs and the logs and answer all of those questions in a minute or two. We are now around 30-60 minutes into spending time on this thread from my side and I still don’t have the answers to these and other important questions I’d need to even begin to diagnose why “it doesn’t work.” I don’t even know in what way it doesn’t work. There are a dozen things that can be going wrong.

As I said, use Generic MQTT Things for those. The Event Bus configuration was never intended as a way to integrate with devices. Trying to use it that way is a misuse of the Event Bus and leads to a suboptimal solution.

You do not and should not use the Event Bus for that. You should use Generic MQTT Things for that. Each device should be represented by a separate MQTT Thing and each Thing will have one or more Channels that represent the individual sensors and actuators on that specific device. You have way more control over how the communication occurs, support for a device-by-device LWT for offline detection, and the ability to individually transform the messages each direction. The Event Bus supports none of that and it doesn’t need to because even the 1.x implementation was only ever intended as a way to integrate openHAB instances. It’s not a shortcut to integrate robots and other devices.

And with Generic MQTT Things you can even keep the same topics that you have now. Or you can rework the MQTT topics being used so they make sense from the device’s perspective. openHAB is supposed to bend to the requirements of the devices. You should not have to code your devices to meed the requirements of openHAB.

You seem to be thinking that the MQTT Event Bus is the only way to use MQTT in openHAB. In fact the Event Bus is a rarely used feature of the MQTT 1.x binding that cannot be supported in 2.x bindings because it uses features that bindings are not allowed to use any more. And it was never intended to be used for generic integration, as I said above.

So I’ll say again. For openHAB to openHAB you should use the Remote openHAB binding. For MQTT you should use Generic MQTT Things. You don’t have to change how your devices communicate. But to use the MQTT 2.x binding it will be less work in the short run and much more flexible to use the binding rather than the Event Bus rules.

It’s not clear but you’ve not mentioned in any of this whether you’ve created a Channel on the MQTT Broker Thing to subscribe to the event bus topics. See “Subscribing from the Event Bus: Setting up the Subscriptions” section of the MQTT Event Bus tutorial. That has you create a Channel on the MQTT Broker Thing.

Once created that Channel will have an ID. If you follow the tutorial it will probably be named mqtt:broker:mosquitto:eventbus. This is the ID that mqtt_eb_in_chan needs to be set to. This Channel generates events every time a new MQTT message comes in that triggers the rule that parses out the topic and the message and updates/commands the appropriate Item.

Because I don’t think you understand what each of these properties are for and how they are used.

Property What it means/how it’s used
mqtt_eb_chan Id for the Channel on the MQTT Broker Thing that subscribes to the incoming messages
mqtt_eb_name Used as the first part of the MQTT topics this instance will publish to.
mqtt_eb_puball When true, all Item commands and updates are published. When false only tagged Items are published.

Unless you are running openHAB on the device then these settings are completely irrelevant. You code MQTT on your device to publish what ever it needs to to what ever topics it needs to. These are only relevant settings for an openHAB running the Python version of the Event Bus rules. Therefore they only apply to that one instance of openHAB, nothing else.

And as I’ve stated, if the device is running openHAB, you are better off running the Remote openHAB binding instead. But if you still want to use the MQTT Event Bus on all your instances of openHAB then you will need to set up those parameters and the Thing Channel on each of them separately.

You’ve now switched the scenario. Originally you stated you were focusing on devices publishing and your VM instance subscribing. How you are trying to configure the VM to publish and ??? subscribing?

Yes. The only way to get support for Python in openHAB rules in OH 2 is through Jython. Furthermore, those libraries and rules for the event bus also require the Helper Libraries to be installed and working.

I don’t know how I could have been more clear and am open to suggestions. But that section you quote, at least to me, clearly states that the Jython rules and libraries:

  • only work on OH 2.5
  • require the Next-Gen Rules Engine add-on to be installed
  • require the Helper Libraries to be installed

That’s on the first page of my repo and therefore applies to all the Python/Jython code in that repo.

Note: seeing a few lines of openhab.log during startup would have immediately told me that was the problem.