Remember this is an MQTT event bus. In order to use it, you also need broker etc. This is becoming more things you’re “forcing” on people.
There are other ways of constructing an event bus service, this is just one.
And in the end, it’s a niche service. There really aren’t that many people who want to use multi-instance OH installs.
Which is all to say, it’s fine - have a convenient utility or set of utilities, and a guide, and wizards and install aids, etc. But really don’t worry about making it into some standard core feature.
As someone who actually does run two instances federated using MQTT, I can say that there are not very many circumstances where I’d recommend running multiple instances. The complexity goes through the roof, the number of places you can go wrong greatly increases, and the number of possible errors that can occur greatly increases too. You need to have a rely completing need to justify the added complexity. In my case, I’m passively monitoring a remote instance which otherwise provides automation for my dad’s house.
Well, what transport mechanism would you like to use? You might write your own client/server service to work over bluetooth, or over a wired serial cable.
Or you might like ethernet or WiFi but not MQTT - that’s fine, write scripts to use REST API over HTTP.
There is complexity inherant in having more than one oh instance that will never be solvable. It will always be significantly more complex to run more than one oh than two. And this MQTT event bus is almost as simple as it gets.
I can move some of the complexity around but not eliminate it. No matter what, you have to deal with the fact that some items are remote and some are local and the network may fail, messages may get lost, and more than one oh is running.
There is also deminishing returns. This is a rare configuration and not a recommended recommendation. For me, I’m just not sure it’s worth the effort to spend a lot of time polishing this capability beyond what I’m already planning.
But anyone is free to disagree and volunteer to polish away.
Just like with kids and dogs, having 2 is more than just double the work. The only reason you don’t see this is because you lack the experience to know this.
Right now MQTT will have the most support and be the easiest to implement. Especially if you use scripted automation you just need to download the Helper library, configure the name of the Groups, then add your items to the groups. The MQTT configuration itself is pretty minimal.
When I get to it, you will just import the rule template and configure it using a form like you do with a binding.
Maybe the word complexity is was causing my confusion/misunderstanding/disagreement…
I already do run two different OH systems. They´re very far from simular, as one is a Rpi running the hasslefree image (2.5M2). The other system is a Odroid C2 running Ubuntu and openhab 2.5 release. Ofcouse it´s more plexed running these two systems, because they very far from simular. Thats my (stupid) unique situation. But if the Odroid had been an Rpi, they would probably be running the exact same setup. Then I no longer think this should be complexed after all, unless openhab itself is complexed. And if thats the case, openhab is complexed in one setup or in several setup. Or at least thats my believes.
Having more instance of a system running together can ofcouse have some issues. Like you mention, messages could be lost… I do however feel this would rather have something to do with the messaging system, than the actual LAN connection. A wired LAN doesnt fail, unless something else is wrong. A Wireless LAN however, that will probably fail more often.
I would have to deal with more items, agree… But as an experienced user, I dont see how this should be a concern. If more items is a problem, there are some fundemental understandings missing somewhere
Mind you - What my suggestion is all about, is making this “interfacing” between two/more OH instances as easy as possible. Ofcouse I wouldnt expect anyone not having fundenmental understandings of OH running more systems together like this… I believe this is a need which grows when things gets easyer/the user gets more experienced.
Ok you have two independent instances of OH running. Connecting those two together is only going to add additional complexity because you are talking two already complex systems and adding something new to them.
OH + OH < OH + OH + event bus
You don’t get simpler by adding.
But if you disagree, please by all means implement something simpler. Maybe I’m missing something you see clearly but all my years of experience with oh and software in general tells me that the event bus can do nothing but added to the complexity.
I got something similar working using 2 instances: a remote openhab using an MQTT broker with a generic MQTT thing with arbitrary items for each sensor data point, with my main openhab having a broker and a reflected generic MQTT thing and matching items to receive.
All worked good, but I wasnt able to use MQTT qos 1 or 2 because I understand you can’t have that with this setup; “MQTT persistent session”.
Then I did a different setup: I got the MQTT persistent service going, which is cool because you can easily setup ALL items to send to your configured broker, which my main openhab instances receives. (Soon I’ll post guides on how to do both of these methods.)
I’m pointing this out because it appears there are numerous methods to do what Rich is posting: just find what suites you.
I did have a question tho, if anyone knows: with the MQTT persistent service, if a remote openhab instances goes offline for, say 15 minutes, then reconnects to the network, it indicates the event (like a door opening) happens when it reconnects, NOT the time the real event occurred. I noticed the MQTT persistent service allows you to publish and send over just data, or a datetime+data, or whatever, in a comma separated list.
Does anyone know how to setup an mqtt receiver (subscriber) using the MQTT persistent service as a publisher, so you can utilize qos 1 or 2, and use the time stamp you send over, instead of the time it recieved the event?
The only warning is anything that relies on the MQTT v1 binding or MQTT Action or MQTT Persistence will not work in OH 3. Support for v1 add-ons will be dropped and these are all v1 add-ons. The above will be the only way, though it will come with OH. You won’t have much if anything to set up code wise yourself.
qos is managed by the broker connection configuration. I’m not sure what difficulty you had there. I also don’t know how support the MQTT v1 binding had for QOS. I remember you had to set it for the entire broker connection and couldn’t set it on a topic by topic or message by message basis. I don’t know anything about an “MQTT persistent session”.
But ultimately, if you are just setting this up now, I strongly recommend using the above, and you can control the QOS with the above, rather than getting started with the v1 add-ons. None of those are currently maintained and they will not work on OH 3 at all.
Mqtt qos 0 is fire and forget, which is fine for sending temperature data or the like. Qos 1&2 ensure the message was received. Without an “mqtt persistent session”, qos 1&2 simply don’t work.
My research lead me to believe the MQTT persistent service was the only way to get an mqtt persistent session, but I could be wrong. I got this persistence to work using this method, but not without this service (ie, I couldn’t get it to be “persistent” with the broker thing & generic mqtt thing on each side).
So, using the broker & generic thing on each side (remote openhab to send & main openhab to receive) - I got all the messages just fine. But testing qos 0: I unplugged the network in the remote openhab, forced some events that typically get recieved by the main openhab, then plugged the network back in 10min later - but the events were not received, as expected since it was qos0 (fire & forget).
Then I tested qos2 (setup everywhere: generic things, brokers): I unplugged the network, forced some events, plugged the network back in 10min later, the events were not received… Because qos1&2 don’t work without an “mqtt persistent session”.
Then I found the MQTT persistent service, and it had a setting to make, or that is it’s purpose, to allow an mqtt persistent session.
But my issue is I couldn’t find a way to receive & handle to event time with the event - tho I guess I’ll survive without it.
Btw all of this was very simple to setup, much easier than the above directions with the event bus: no rules, zero coding…
But then you tell me this binding is going away…ugh
I know what QOS is and I know that both the v1 and v2 bindings support QOS 0, 1, and 2. The “persistent connection” is managed by the MQTT client being used and has nothing to do with OH directly.
The MQTT Persistence Add-on uses the MQTT v1 binding for all of it’s MQTT communications. It can’t do anything that the binding can’t do.
Keep in mind that it’s the publisher that sets the QOS, not the subscriber.
QOS only matters when the subscriber is connected. It doesn’t guarantee that the message is delivered, it guarantees how many times the message gets delivered.
QOS 0: message is delivered 0 or more times
QOS 1: message is delivered 1 or more times
QOS 2 message is delivered exactly once.
But in all cases, the message is never delivered is the subscriber isn’t connected to the broker when the message is sent.
For that you need the retained flag. Like QOS, retained is set by the publisher of the message. When a message is retained, the subscriber will get the message when it reconnects to the broker. And that message will be delivered according to the QOS.
No, not all. You already have a persistent connection. But nothing can maintain a “persistent connection” across a network when you pull the plug.
The MQTT Persistence add-on is an openHAB Persistence add-on. You know, the add-ons that save Item states to databases. It was and only ever was a hacky way to publish everything that happens on an OH instance over MQTT. It became pointless once the MQTT v1 Eventbus capability was created. And that will become useless in OH 3. So the above will be the way to achieve this.
It’s worth noting that in the version of the 2.5 Event Bus that I’ve submitted to the Helper Libraries which will become a part of OH proper, Item states are published with the retained flag set by default. So even if one OH instance goes offline, when it comes back it will receive the latest message for Item states. Other nice features include the fact that they will report when they go offline using the LWT messages.
With the coding above you can easily add that if you want it or need it.
But in addition to v1 add-ons losing support, Rules DSL will be deprecated with Scripted Automation as the default. With Scripted Automation you don’t need to copy/paste/edit Rules. You can just install them, In this case it will just come with OH 3 and you just have to turn it on (which is the hope, lots of time and effort to go between now and then).
And if you don’t want to wait, the link in the first sentence is just such a reusable version. You just copy the files (soon it will just be there) and tell it what broker Thing to use and you are done. It will be just as simple and in some ways even more simple than the v1 Event Bus.
Have you, or anyone, gotten qos1 or 2 to work with a broker thing & mqtt generic thing on both sides? (And proved it worked) I haven’t, but wondered if Maybe I missed something. Yes, I set every “node” that participated in the communication to qos2 so it’s not that…
Btw I’m not trying to debate, just figure this all out
Thanks
Just throwing additional info out there for other viewers (not rich - I say that because you said “I know what QOS is” above, but I wasnt directing that to you, apologies). BTW MQTT retained messages wont work for my case - my use-case is knowing exactly when a door opens and closes, NOT what state its in (which one might find useful for a light on, or off or whatever).
I’ll have to re-read this whole thread tho because I was writing to this while on a plane
I do get what you mean about “if a network is unplugged” there is no technology in the world that will allow the communication to continue, that’s obviously true. But QOS2 should allow it to resume and not lose messages, if setup correctly…I’m not sure if my example and testing was clear but I’ll try again… I have a remote & a “main” openhab instance. The remote instance has an erratic connection. QOS2 with a persistent connection is required for that feature to work https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/ , but this reveals the setting I’m looking for may actually be called something else https://www.hivemq.com/blog/mqtt-essentials-part-7-persistent-session-queuing-messages/ clean-session-flag=false, to enable a persistent connection. (For those that don’t know, a persistent connection is required for QOS1/2 because if its NOT a persistent connection then you start “clean” with every reconnect - so there’s no way to check what messages you received and didn’t, if you started clean!)
So here’s what my tested revealed: a crappy network connection will safely deliver all messages with QOS2 and a persistent connection upon the network reconnecting! This is great news because it’s exactly what I require - imagine, then network disconnects, someone enters the house and triggers the door sensor, then the network reconnects - well, I’ll never get the door alert with a typical MQTT setup, but with QOS2 Im guaranteed to get the alert once the network resumes connection (this assumes everything is setup correctly and all publishers and subscribers have QOS2 enabled). – When I deliberately then set this up with QOS0 & tested, during the network outage I’d miss all alerts! (As expected, but definitely not the setup I’m seeking.) Hope that makes sense on why I’m focused on this very specific feature. It is awesome though btw.
So anyways, I’m going to reread this a few more times, and probably test out Richs implementation/method, and see if I can get it to work with QOS2 (across the line obviously-all nodes in the path) because I need this to work with a crappy network, as I mentioned. I’m also interested because Rich indicates Event bus is the way to go, and probably the only thing supported going forward. So yea, Id be foolish to continue down my existing path. (Thanks for the heads up ) I clearly missed that when I tested the other bindings (guess its not obvious to me what’s a 1.x binding, and what’s going to be deprecated).
Quick question: in a very simple 1 liner or example, how do you input an event time, instead of the system doing it for you?
Yes, I’ve used QOS 2. And I can see that QOS 2 is set in MQTT Explorer. And I can verify that the message is only delivered once. I’ve not set up Wireguard to see the message exchange that takes place in a QOS 2 but never felt the need to do so.
So make the very minor modifications to the code above or that is on GitHub to publish now.toString to <openhab name>/<Item>/state/dt. It is literally a one line of code addition. I’m not sure what you need to do with that DT on the other side but it will be an ISO 8601 String which can be easily parsed by just about anything that handles date times.
Than it may not be an option that is exposed. You should file an issue to add it, if adding it is possible.
Given that OH is event based and when it finally does receive the messages it’s going to act on those events right now, not go back to when it that event actually happened; you can’t event insert these events into the Persistence database with the right timestamps. What exactly do you need all of the missed messages for? What can you do with them that is meaningful in an OH context? I’m not arguing against the approach, just trying to understand as there might be alternative approaches to achieve the same thing. But typically, if OH is offline when a bunch of events occur, all it really cares about (or can ever know) is the most recent state the devices are in. Zwave doesn’t queue up all the on/off toggles of a switch that occurred when OH was offline. OH just gets “the switch is now OFF” when it starts up.
You will is the message was published with the reatined flag set, even if the message was set to use QOS0. QOS0 messages tend to only be lost if the connection between a client and the broker is bad.
You don’t. That’s why this whole focus on persistent connections over the retained flag feels like an XY Problem. Let’s say one OH instance goes offline. The online one sends a bunch of events over that five minutes. Five minutes later at T+5 the OH instance comes back online. With the configuration you seek, the OH instance that just came back online will receive all of those messages and process and handle them as if they all occurred at T+5. There is no way for OH to say “hey, that message is five minutes old, I need to go back and redo stuff as if I got it five minutes ago.” And because all the missed messages will be arriving all at once, you don’t even have a guarantee that all the messages will be processed by OH in order. So you could end up with the same situation you are trying to avoid, not knowing what state the door is in, because the events might be processed out of order.
So, unless you can explain some use case I don’t see or understand, even with the persistent connection you are still in the same boat of not knowing what state your devices are in. But if you use the retained flag, the OH instance will only receive the most recent message, but it will receive the most recent message. Consequently it will at least know what the current state of everything is in when it comes online. But it won’t know when because OH doesn’t have a way to tell it when something occurred in the past. Events occur now. So you would have to maintain a separate DateTime Item to store when the Item updated and publish that too over the event bus. This will let you see on your sitemap and your Rules when the event occurred, but as far as the rest of OH is concerned, the event occurred when the message was received.
