MQTT 2.5 Event Bus

Tags: #<Tag:0x00007f61733f5968>

Without the Rule how do you define what Item’s updates and/or commands get published to the event bus? On the other side, how do you subscribe to the messages and update/command the corresponding local Items? That’s what the Rules do. You add the Items to the right Groups and the Rules publish based on the Group membership. For the subscription, the Rule determines what Item an update/command belongs to based on parsing the topic the message comes from.

To achieve this without Rules, you would need to create and configure Channels and Items to correspond with each Item you want to sync with the event bus. That is far more work over all than just adjusting an Item’s Group membership.

And it is possible to implement this without Group membership and just publish everything (at least in Scripted Automation, I don’t think this is possible in Rules DSL) but I think it’s better to give the users control over what gets published to the event bus.

The OP uses generic MQTT. I don’t understand the question. What errors are you seeing? What debug steps have you taken so far?

I’ve seen some report ser2net/socat is unreliable and unstable and others have reported that it is rock solid. I think YMMV. Either are an option.

I think the simple answer is via the channel linking and “FOLLOW” profile, instead of the “Default” profile. (See below)

I only have <10 data points to sync up, but here’s my configuration (Method Generic w/thing & items):

  • MQTT Broker Thing & Generic MQTT Thing on “Main” & “Remote” side.
  • Remote side: Generic MQTT Thing: Create channel(s) inside this thing, eg. “temp” & CmdTopic=property1/temp1. These channel(s) links to an actual existing Item (thats linked to a real temp sensor), but using the “FOLLOW” profile. This is important!
  • Main side: Generic MQTT Thing: Create channel(s) for each sensor or data point, eg. “temp” & StateTopic=property1/temp1. Create an item for each channel as normal, eg “temp”, maintaining the data type, number in this case, with the Default profile. Use this new item as normal.

My current topics:

  • property1/temp1 QOS:0
  • property1/humidity1 QOS:0
  • property1/door QOS:2
  • property1/doorbattery QOS:0
  • property1/motion1 QOS:2

This works & this alone (no rules, no scripting, nothing else except obviously a mosquitto server) sends updates from the remote sensors across the pipe to my main openhab instance. But I’m already guilty of using deprecated methods and “painting myself in a corner” it seems. So what’s the difference between what I did and the “Event Bus” method with rules? Is my way using the “Event bus”? Is your method just a different approach where yours gives you more granularity and control via scripting? Am I doing it wrong? I found this https://www.openhab.org/v2.3/addons/bindings/mqtt1/ (I see its for an older ver), but it does point out there are different methods such as “Item binding” & “Eventbus binding” - so maybe I’m doing it the deprecated way? (The MQTT Persistence Service was the easiest method I tried so far - too bad its going away )

Thanks again!

You have to individually set up the Channels and the Follow Profile for each and every Item you want to publish and you have to individually set up the Channels for each and every Item you want to subscribe and sync on the other side. That’s just standard MQTT configuration/integration like you would do with anything (e.g. Tasmota).

If you only have a few Items this isn’t that big of a deal. If you have hundreds or thousands of Items that approach is completely unmanageable. That’s what the Event Bus was created to address in MQTT v1 and that’s what the above was created to address. With a few lines of code, code you never even have to look at if you use Scripted Automation, you can get hundreds of Items publishing and subscribing.

I would say no. It’s not an event bus because you are essentially setting up the MQTT stuff individually on an Item by Item basis.

It’s not the degree of control. Both approaches have the same amount of control. But the event bus approach scales. Your approach does not. You may never need to scale, but for those that do, your approach is not suitable.

There really is no such thing as “wrong” if it works functionally. There are approaches that are harder or easier to understand, or take more or less work, or are easier or harder to maintain. But that doesn’t make them wrong, maybe just less good in certain circumstances.

No, if you are using the v2 binding you are not using the deprecated way. The code in the OP is the replacement for the “Eventbus binding” at that link because the v2 MQTT binding doesn’t support that natively for policy and technical reasons (i.e. the way the MQTT v1 event bus configuration works is not allowed in V2 bindings).

Easy doesn’t mean fit for purpose. But honestly, it was not much more easy to use than the OP above. You have to set up and configure the broker connection for both. You have to identify each Item that gets published to the event bus.

But with the MQTT Persistence it’s one way. You are on your own on the subscriber side and have to configure each and every Item you want to mirror with it’s work MQTT binding config.

With the OP above it’s two way. It handles both the publishing of updates and commands as well as what’s necessary to subscribe and update/command Items based on the published stuff. On the subscriber side you only need that 11(ish) lines of code (or import the MQTT Event Bus from the Helper Library) and create the Items (no links, no binding configs) and you are done.

MQTT Persistence only addresses half of the problem.

Crystal clear. Thank you Rich.

I’ll report back when I start digging into your approach & testing.

Trying to get this implemented, but I’m positive I’m missing something fundamental with the JSR223 initial setup (I never used this before. I’m not missing a binding am I?!) I’m on openhab ver openHAB 2.5.0-1 (Release Build) if that’s relevant. I found this in configuration.py.example: “Requirements MQTT 2.5 M1 or later Binding installed.” - I installed the MQTT binding inside the PaperUI, the binding that comes with it. Am I supposed to do something different? And I don’t see “M1” or whatever in my version, is that the issue?

Is a config file needed, like openHAB-conf\automation\lib\python\configuration.py ?

My Current error: (Rule is getting kicked off, but then I get the string split error - missing import? But not sure where I’d add that)

  • 2020-01-20 21:08:21.638 [INFO ] [ome.model.script.SubscribeMQTT.rules] - Subscribe update to the event bus
  • 2020-01-20 21:08:21.642 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule ‘Subscribe for commands and updates from the event bus’: ‘split’ is not a member of ‘org.eclipse.smarthome.core.thing.events.ChannelTriggeredEvent’; line 8, column 20, length 24
My Setup Steps for Reference: (Probably wrong!)
  • Install (all openhab instances):
    ** Prerequisites: Install MQTT Broker binding & Install “Rule Engine (Experimental)” binding
    ** Download scripts: from [https://community.openhab.org/t/mqtt-2-5-event-bus/7693], click [https://github.com/openhab-scripters/openhab-helper-libraries/pull/257 GitHub pull request link]->Commits tab->"<>" symbol to “Browse the repository at this point in history”->Clone or Download zip link
    ** Implement scripts:
    *** Create: /etc/openhab2/automation/jsr223/ folder (if not already exists)
    *** Copy \Community\MQTT Eventbus\automation\jsr223\python -> /etc/openhab2/automation/jsr223/
    ** Add Broker Thing: Set IP/Hostname, QOS:2, ClientId: [openhabX], openhabian_mqtt / password
  • Install - Publisher:
    ** Create Group item in /etc/openhab2/items/groups.items: Group:String PubItems
    ** Add Items to this group (All items in this group will get published to an MQTT topic): eg.
    Contact Door “Door [%s]” (PubItems) {channel=“zwave:device:415c59ca:node3:sensor_door”}
  • Install - Subscriber:
    ** MQTT Broker: Add “Publish Trigger” Channel: ID:openhab1, label:subscription to openhab2, MQTT Topic:openhab2/out/#, Seperator character:#
    ** Create all Items that will be synched…
  • Restart openhab service: sudo systemctl restart openhab2

Maybe. Did you follow the instructions at https://www.openhab.org/docs/configuration/jsr223.html#jsr223-scripting, in particular installing the Experimental Next Gen Rules Engine (available under the Misc tab in PaperUI. Once you’ve done that I recommend following the instructions at https://openhab-scripters.github.io/openhab-helper-libraries/index.html or even better at [beta testers wanted!] Jython addon w/ helper libraries (requires OH 2.5).

This is required to set up Scripted Automation so you can run Rules in Python. As you can see with the last link, the steps to install it are getting easier and soon it will just require installing the add-on through PaperUI and in OH 3 it will just come installed by default.

openHAB has versions. You have the release versions (e.g. 2.5.1), testing or milestone releases (e.g. 2.5 M6 or milestone 6), and snapshots (e.g. 3.0-SNAPSHOT). That sentence means you need to run a version of OH, or at least a version of the MQTT binding that as released after the 2.5 M1 release which happened in February of last year IIRC.

Absolutely if you are using https://github.com/openhab-scripters/openhab-helper-libraries/pull/257. It’s all in the docscrings (i.e. the comments between """ at the top of the file and in the Rule). In this case it says:

    Called when a member of the Group defined by eb_out_gr in configuration.py
    receives a command or an update. When the Rule triggers it publishes the
    state or command to the MQTT topic
    [eb_name]/out/[item name]/[state|command] where:
        - eb_name: defined in configurations, the name of this OH instance
        - item name: the name of the Item that triggerd the Rule
        - state: when the Rule was triggered by an update
        - command: when the Rule was triggered by a command.

When you first installed the Scripted Automation and the helper libraries per the instructions in the links above, one of the steps was to create this configuration.py file.

You missed the steps to install Python and the helper libraries.

I just updated to: 2.5.1-2 via the openhabian-config tool (01:Update, 02:Upgrade System, 40->41:Release version). I’m not sure if “2.5.1-2” is newer than 2.5 M1 release. Regardless, am I good now or do I need to further update based on the “Requirements MQTT 2.5 M1 or later Binding installed”? I’m also not clear if a snapshot is like alpha or beta (Id like to avoid alpha or beta at this point and I’d guess it is). I’ve been struggling the entire time I started using openhab finding documentation, or documentation that’s relevant given the quick evolution of the software (I find info for out-of-date versions that are no longer relevant all the time).

Since 2.5 has been released, any 2.5 release will be newer than any 2.5 Milestone. There is no reason to have a “testing” version for the already released version number.

OH 2.5.1-2 was released a few weeks ago. OH 2.5 M1 was released sometime in February of 2019. You are fine.

SNAPSHOT is literally everything that has been merged into the baseline in the past 24 hours. They are completely untested. And the 3.0 SNAPSHOTS are completely unusable right now anyway.

But pay attention to the repo you are using and the version number. Versions progress from

  • X.X SNAPSHOT: new version almost every day
  • X.X Milestone: about once a month when there are no known major breaking changes, a X.X SNAPSHOT will become a new Milestone.
  • X.X: about once every six months after a brief beta period, a SNAPSHOT will become the new point release.

You’re getting peoples’ hopes up for things that are not going to happen! The Jython bundle is never going to be available for installation through Paper UI in OH 2.5.x. As for OH 3.0, you’re making definitive statements about things that have not been finalized. There is a possibility that the new rule engine and Jython will be core features installed by default. It’s also possible that one or both will be optional. These things are currently being discussed and have not been finalized.

I FINALLY got the Jython hello world script to log. Since this took me a long time to figure out, I wrote up a guide: JSR223 Helper Libraries/ Jython Support Install Guide

Back to testing the event bus…

Why does everyone sound mean around here? Or as someone smart once told me, “Its a cultural thing” lol Oh well, I’ll take the help I can get. FWIW, I think optimism and hope is a good thing :slight_smile: Ok, back to the EB…

Ok, unfortunately I’m still having the same issue on the receiver/main openhab/subscriber. I’m still seeing my helloworld script running every 10 seconds so I think I may be good with the Helper Libraries install (these are the steps I did JSR223 Helper Libraries/ Jython Support Install Guide )

My Interpreted Install Steps:
  • Install (all openhab instances):
    • Prerequisites: Install MQTT Broker binding & JSR223-Python Scripting Install
    • Download scripts: From here, click GitHub pull request link->Commits tab->"<>" symbol to “Browse the repository at this point in history”->Clone or Download zip link
    • Implement scripts:
      • Copy /Community/MQTT Eventbus/automation/jsr223/python/community/mqtt_eventbus/. -> /etc/openhab2/automation/jsr223/python/community/mqtt_eventbus/.
      • Copy /Community/MQTT Eventbus/automation/lib/python/configuration.py.example -> /etc/openhab2/automation/lib/python/configuration.py (Rename existing to *.ORIG ?? Need it?)
      • Merge contents of /Community/MQTT Eventbus/automation/lib/python/configuration.py.example WITH /etc/openhab2/automation/lib/python/configuration.py & edit PUBLISHER or SUBSCRIBER sections as applicable
    • Add Broker Thing: Set IP/Hostname, QOS:2, ClientId: [openhabX], openhabian_mqtt / password
  • Install - Publisher:
    • Create Group item in /etc/openhab2/items/groups.items: Group:String PubItems
    • Add Items to this group (All items in this group will get published to an MQTT topic): eg.

Contact Door “Door [%s]” (PubItems) {channel=“zwave:device:415c59ca:node3:sensor_door”}

  • Install - Subscriber:
    • MQTT Broker: Add “Publish Trigger” Channel: ID:openhab1, label:subscription to openhab2, MQTT Topic:openhab2/out/#, Seperator character:#
    • Create all Items that will be synched ???
  • Restart openhab service: sudo systemctl restart openhab2
Logs around Errors:

2020-01-21 18:50:57.306 [INFO ] [ome.model.script.SubscribeMQTT.rules] - Subscribe update to the event bus
2020-01-21 18:50:57.310 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule ‘Subscribe for commands and updates from the event bus’: ‘split’ is not a member of ‘org.eclipse.smarthome.core.thing.events.ChannelTriggeredEvent’; line 8, column 20, length 24
2020-01-21 18:50:57.464 [vent.ChannelTriggeredEvent] - mqtt:broker:40b306d8:openhab1 triggered openhab2_EB/out/Multisensor_Humidity/state#32
2020-01-21 18:50:57.472 [INFO ] [ome.model.script.SubscribeMQTT.rules] - Subscribe update to the event bus
2020-01-21 18:50:57.477 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule ‘Subscribe for commands and updates from the event bus’: ‘split’ is not a member of ‘org.eclipse.smarthome.core.thing.events.ChannelTriggeredEvent’; line 8, column 20, length 24
2020-01-21 18:51:00.329 [DEBUG] [e.automation.internal.RuleEngineImpl] - The trigger ‘Time_cron_0_10_2b96c3003cb111ea867bdca6322133ca_2bf0068f3cb111eabc22dca6322133ca’ of rule ‘2d5e4294-5f63-4cd4-8346-a81fda7709f8’ is triggered.
2020-01-21 18:51:00.334 [.event.RuleStatusInfoEvent] - 2d5e4294-5f63-4cd4-8346-a81fda7709f8 updated: RUNNING
2020-01-21 18:51:00.356 [INFO ] [Jython Hello World (cron decorators)] - Hello World!
2020-01-21 18:51:00.358 [DEBUG] [e.automation.internal.RuleEngineImpl] - The rule ‘2d5e4294-5f63-4cd4-8346-a81fda7709f8’ is executed.
2020-01-21 18:51:00.360 [.event.RuleStatusInfoEvent] - 2d5e4294-5f63-4cd4-8346-a81fda7709f8 updated: IDLE

automation/lib/python/configuration.py config:

LOG_PREFIX = “jsr223.jython”
admin_email = “admin_email@some_domain.com”
openhabHost = “localhost”
openhabPort = “8080”# “8443”
eb_in_chan = “mqtt:broker:40b306d8:openhab1”

Please let me know if there’s anything else I can supply!

I will stop saying anything along these lines. It seemed clear that acceptance of this pr above was all that was missing to get to install it through the usual add-ons mechanism, since oh 2 5 add-on development is continuing.

It also seemed clear that if python was going to become the default and Rules DSL completely eliminated that Python would just come with OH 3.

I’m getting lies of mixed messages regarding where this is all going so I’ll just shut up.

If your see that then your Python is installed and working. Reply in the event bus thread and we will debug further there.

Man I’m losing it! But isnt this the event bus thread? (Apologies)

Wait now I’m losing it. Too many threads going on.

Give me a day and let me redo the event bus stuff and post it here with better instructions.

You know I’ll be your guinea pig…and good or bad…give feedback :slight_smile: And when were done it’ll support NOOBS that never even heard of an event bus or Jython! lol We can either do it here or PM. Thank you Rich!

I’ve rewritten the OP to include the Python example rather than just the link to the PR on the Helper Libraries. See if you can follow the new tutorial as written above to copy over the Rules and get them to run.

Works like a charm. :grinning: Thanks Rich for your patience!

(For new readers, my Rules DSL error ‘split’ is not a member of ‘org.eclipse.smarthome.core.thing.events.ChannelTriggeredEvent’ is now resolved via Richs OP.)

The “Subscribing” DSL code has a string detokenization bug if users configure their subscription channels in accordance with the screen shot (ie “#” as the “Separator Character”). The fix is:

var topic = receivedEvent.toString.split("#").get(0)
var state = receivedEvent.toString.split("#").get(1)
val itemName = topic.split("/").get(2)
val type = topic.split("/").get(3)

I’ve corrected the OP. It’s no secret that I don’t use Rules DSL any more so the error escaped detection before now. Thanks for the correction.