Tie multiple instances of openhab together using MQTT?

Tags: #<Tag:0x00007f616eee42c0>

UPDATE: If you hit this thread via title, I’ll save you the time of reading the entire thread. Go here: MQTT 2.5 Event Bus for the answer.

There is so much talk about this topic, as the subject says…yet no guide?!? It seems to be a very common request. I just discovered openhab less than a week ago and found it has so much potential, but so few guides out there. Like why did it take hours to figure out how to pick/buy a compatible z wave USB stick, then a few more hrs on how to set it up?!?. I will begin writing guides and publishing them here, but this is clearly the issue at this point - the community is totally there, but not in a proactive way, only to help… Only in repeat questions one after another… Ie, I have a raspberry pie 3+, I began following instructions, couldn’t figure out why openhabian 1.5 wouldn’t install…well, the setup guide doesn’t indicate it’s for raspberry 4 ONLY!!! Anyway… UPDATE: Pi will be longer boot from a USB stick-sd adapter in 2.5 (it will in 2.4) - You need to only use the built in Pi as card slot (I didn’t know, I’m a NOOB and forever will be :slight_smile: )

Anyone point me to a guide to link 3 openhab instances (3 different physical locations)? I know… MQTT… but any guide on steps? Please don’t post with partial help! Plenty of that already exists! I’m looking for a working guide on how to completely set this up - If it doesn’t exist, I’ll figure it out and create a guide - but then, any suggestions on where specifically I should post that guide?

Thank you so much!!! Btw I’m not complaining, rather voicing how “we” can make this better!! (I’m excited to begin contributing!!!)

FYI This statement is wrong… I just installed it on my pi 3… sorry don’t know how to sugar coat.

The problem with the MANY guides that are all out there already (most of the complete guides aren’t on this site, but on a variety of other blogs/sites) is that they are incredibly specific in an individual use case, using specific hardware/environment, within a defined period of time (soon to be out-dated dependencies / software versions).

This is a niche question you’re asking. There are some people who have been looking at z-wave bridging over wifi (an area of active development), and I could be wrong but I don’t think a whole lot of people have been putting out polished 3-instance circumstances.

In the end, like so many other problems/use cases, there are a lot of decisions that need to be made which would define the individual steps/ ‘parital help’. Ask specific questions (with enough information), and you will generally get helpful and specfic answers. Ask for vague overviews, and you’ll often get general direction suggestions/overviews. Unfortunately, you’re asking for a detailed solution without providing any real details. Are you within an isolated/secure network? Does it need to cross between networks? What sort of security are you considering? What architecture are you running the instances on? How are you wanting the instances tied - are you just wanting specific messages to be able to be communicated, or are you envisioning a much more involved lower-level communication to be going on?
Some of this may be possible, and others not.

If you’re brand-new to openhab, I’d encourage you to start small. There is definitely a bit of a learning curve, and unless you have a huge amount of spare time, it may be quite difficult to start too complicated.
There are plenty of complete tutorials on how to set up openhab, MQTT, and get an arduino/esp8266 device sending updates/temperatures/humidities/door events to openhab, and something like this could work quite well if you have any of these hanging around. The next step is sending a message back to the device (to light up an LED, open a door, etc). If you don’t want to mess around in arduino, simply flash a Tasmota binary to it, which is quite easy to get up and running with MQTT. Don’t want to use actual other hardware, pull up a couple terminal windows with a pubsub client in each, and send/view the messages manually.

Once you have this two-way communication, it should be fairly easy to take the next step to have each openhab instance subscribe to and publish to defined topics on an MQTT server.
For what I’m guessing you’re interested in, you need:

  1. a device (virtual / physical) to run each instance of openhab on
  2. a device (virtual / physical) to run an MQTT server on - ie mosquitto. This could be the same as one of the devices running openhab.
  3. configure an MQTT binding (clients) on each of the openhab instances, pointing to the MQTT server
  4. set up MQTT things (now a bit more of an abstract concept, compared to an arduino device which is much more understandably a ‘thing’) with defined channels that they will watch and post to.
  5. Set up rules to interpret/take action based on these events.

This is just an overview, and I’d encourage you to write up a detailed tutorial with whatever solution you end up at. Even if it is incredibly unique for your use-case, you may well find it helpful a year later when trying to remember what you had done.

2 Likes

openhabian-pi-raspbian-201908050414-gitca0976f-crc6a66b5a1.img.xz (v1.5) doesn’t install on raspberry pi 3 PLUS. Used etcher & image, really can’t screw it up. Tried a few times same issue.

openhabianpi-raspbian-201804031720-gitdba76f6-crc9e93c3eb.img.xz (v1.4.1) installs on pi 3+ no problem, same process.

Thank you for the very thorough responses already!

Some specifics to get started:
-all raspberry pi 3+s
-openhabian 4.1 on all instances
-z wave door & temperature sensors
-lutron light dinner devices at home (I already learned I need the caseta smart bridge PRO for lutron caseta & pico remotes to work)
-master will b on my home network, with another server and domain name for remote access, dynamicdns ties dynamic ip to fqdn, this works fine. That server will point port(s) to master openhabian - but not sure that is all relevant since openhab cloud connector could b find as well
-slave openhabian Pi’s will b at remote locations - they will connect to the internet via TMobile hotspots
-this is mostly a hobby setup so I don’t care much about security - only device I would care about is 1 schlage deadbolt on my front door
-I’m replacing 3 wink devices that work now but wink is likely going bankrupt and customization is next to zero - eg, if wink losses network for a second, a door opening never actually happens and I never get notification
-as far as what do I want “this” to do - home: lutron dimmers for lights both on demand & follow a schedule - remote: get notifications when front door opens with my zwave sensor as well as track this historically (does this historical tracking require a MySQL db? Point me to a guide? Also a guide for getting a chart to work with this data?) . Also get phone notifications if temperature gets to cold or hot. Pretty straight forward

You mean like MQTT 2.5 Event Bus

See How to get started (there is no step-by-step tutorial)

I can’t answer why you spent hours to get Zwave working. The binding doc states:

The binding uses a standard Z-Wave serial stick to communicate with the Z-Wave devices. There are many sticks available, and they all support the same interface so the binding does not distinguish between them.

If the Controller bears the Zwave logo it will work. If you have suggestions to make that more clear please suggest some edits. There is a link at the bottom of the file that will take you straight to the GitHub page where you can suggest edits to make it more clear.

Because it isn’t. It should run on anything from an RPi 2 B though an RPi 4. Only a minority of OH users running on RPis are using RPi 4s.

See the links above.

:+1:

This is indeed a very nitch configuration. Depending on the desired use case there may be several alternative approaches that would work better.

Errors? There might be a bug that needs to be fixed. Without out more details we can’t diagnose let alone fix it. It is supposed to work and it has worked in the very recent past.

But from what you’ve described you are going to have to expose something to the internet. You may not care about security but I guarantee the hack bots and script kiddies will love the fact that you don’t have any.

Except it’s not. What you describe is actually pretty complex. And even with all the details you’ve provided you’ve no where near provided enough information to actually “provide a guide.” That’s the problem here. There are a gazillion options, many of which do very similar or the same things but in ways that are more or less suitable to a given set of requirements.

Let’s take your “track this historically” with charts. Are you concerned about the size of the database? What sort of Item is the sensor represented as? What kind of chart? How much ability do you want to customize the appearance of the chart? The answer to any one of these questions will result in a completely different end-to-end guide. And that’s just for generating a chart.

Home automation is hard. It is a development exercise. And every one’s system is bespoke. I would write an end to end complete guide for how to set up my home automation system, and there is an audience of one person in the world who can/should follow it set by step. My requirements are different and therefore my system is different.

So what we have are what you call partial guides. Guides that tell you have to do one thing, and are very detailed about that one thing. But it’s up to you to piece together the guides to reach your end goal.

If you were to write end-to-end guides for all the possible combinations of openHAB configurations just counting addons, you would have 350! (350 factorial) guides. I’m guessing that would be more guides than there are written documentes currently in existence.

As for your specific desires, a good deal of it is outside the scope of openHAB as well. For example, what MQTT broker you use and how you secure the communication between the MQTT broker and your openHAB instances is largely outside the scope of openHAB. You have lots of options though:

  • OpenVPN between all your instances, self hosted MQTT broker (see OpenVPN for details)
  • Use a cloud based MQTT broker such as CloudMQTT (here’s a tutorial for that, though I don’t know what if anything different needs to be done for MQTT 2.5)
  • Expose a self hosted MQTT broker to the Internet (see your Broker’s vendor’s web site for details)

I do not recommend the last option.

Once you have the communication set up, than you can follow any one of the many postings to set up MQTT. Then you can use the link at the top of this post to set up the Event Bus.

NOTE: openHABian is just a bunch of scripts. It will autoupdate itself to the latest the first time you run openhabian-config.

Cloud Connector will not do anything for you for MQTT. It only proxies the openHAB REST API. All three instances of your OH need to be able to see and connect to the same MQTT broker.

It requires some sort of persistence. Which one you want to use depends on your requirements. Want to ensure the DB nevers grows beyond a certain size? rrd4j is a good choice, though it decimates the data as it ages. If you want lots of control over the charts MySql/MariaDB/InfluxDB with Grafana are popular choices. See InfluxDB+Grafana persistence and graphing.

Lots of options here too. myopenhab.org with the Cloud Connector addon can do this. Telegram, PushBullet, and several others are also supported.

Very good information, I appreciate that. I didn’t mean an end to end guide - I also agree that would be impossible. I ended up finding a list of like 5 recommended supported zwave devices, that’s what I was looking for. (I’m old school, I remember all the Linux & windows days where you spend days getting HW to work with a driver, or find out no driver exists. I never thought in a million years you just plug it in and it works!). I meant a few “for beginners” or common next steps guides. But the title of this post is what I was looking for a guide for, as it pertains to openhab, not every possible step covering every 3rd party connection to openhab. You sent some good links - I will certainly check them out.

Regarding openhabian 1.5 not working on my pie+, I’m too new to have captured a log. And very first time it not working, was a little depressing :slight_smile: but then 1.4 worked no problem. I posted the exact snapshot image I used above. Repeatability really is as simply as: pi3+, that snapshot burned via etcher, nothing else plugged into USB except USB thing drive and SanDisk sd card, power supply, and Ethernet plugged in. And HDMI to monitor. Please in freshly images as card into pi, pwr up, give it 5-10 min and it’s stuck on the console long list of [ok] s with an eye I don’t recall. Reproducibility really is that way - I reproduced it 2 or 3 times.

1 Like

Please post the errors so users can attempt to help. In the meantime glad you found a workaround.

I’m still too new & don’t know how to post install error log. But even if I did, I’d then have to blow away my semi working instance. It really is as simple as using etcher & the snap shot image I posted above in a pi 3+ for repeatability.

That’s what you did. Sure, that is simple enough. But what happens on that first boot is that a bunch of scripts run to download, install, and configure a bunch of software that converts a stock Raspbian Lite to a Raspbian Lite configured to run openHAB and various other related software. We can’t fix any problems with it if we don’t know where in the 25 script files and hundreds of lines of shell script code the error occurred.

Clearly it works for lots of people or else this wouldn’t be the first that we are hearing about a problem. Or you might be experiencing one of the few errors that are already known about. But without more info all we have is ¯\_(ツ)_/¯. We’ll never know what went wrong. We’ll never be able to fix it.

I understand that. But you are saying that people are in fact installing the exact snapshot I posted on Pi 3+ without issue? Since I couldn’t get it to work, yet I had no issue on 1.4, I figured that was expected (1.5 is for pi 4 and 1.4 is for pi 3 & 3+) - sounds like that isn’t correct tho.

Btw I will be buying more Pis - not sure if I should stick with 3+ or get 4 - so I can retest this and help that way if you want.
Will need to know how to get logs tho.
Thanks!

NOTE: this readme is a work in progress I believe so there may be something incomplete or incorrect. But it tells you where the important files are located.

But honestly, plugging the RPi into a monitor and taking a picture of the last few lines out output showing where it got stuck can often times be enough.

I will certainly do this; I’d like to help the community. Based on your knowledge, should I go with the pi 3+ or 4? Thanks!

I don’t run OH on an RPi so I couldn’t say. I know a lot of people moved to RPi 4 to get more RAM. But people have been happily running on RPi 3s since they first came out.

I found a spare micro SD chip & retried the 1.5 version again, this time taking a picture when it ends installation. Hope this helps. I know this isn’t the right place but I really want to try and help. Especially since I believe I’m doing a very vanilla install with very typical hardware.

I have it sitting here if anyone wants me to do anything else with it, get logs, etc…

I figured it out.

Apparently there is a microSD card slot on the Raspberry Pi 3+ - I bought this as a kit about a year ago, put it in the box and it sat there. The kit came with a microSD card / USB adapter so I assumed that’s how this all works. That was the issue - openHABian 1.5 doesnt work with a USB to microSD card adapter, it only works with a microSD card plugged directly into the Raspberry Pi 3+ board. I found this issue but have it working now without the USB adapter: https://github.com/openhab/openhabian/issues/731

Kernel Panic is unfortunately one of those things that is usually outside of anyone’s control. Either there is something physically wrong with the hardware or something corrupt with the kernel or device drivers. There is nothing in openHABian that does anything to either the kernel nor the device drivers as part of the setup so I’m going to guess what is happening is there was a bug introduced in Raspbian between the two versions of openHABian.

If you download and try to flash and boot to the latest Raspbian Buster image do you get the kernel panic? If not I’m not sure it really tells us much though because they could have fixed it. But if they did fix it, than openHABian should probably consider rebuilding the SD image based on the latest Raspbian.

For you, the work around would be to either:

  • install the previous SD card image and then run openhabian-config and perform an update
  • install Raspbian Lite and follow the manual instructions for installing openHABian on top of that.

Really appreciate the response as always. I posted I just figured it out & fixed it. Consider me tho a guinea pig and literally a first time user - I wasn’t aware we need to only use the microsd card slot on the pi board & NOT the USB adapter - so that should be in the documentation to make this easier or more obvious for the next users - how do we get that added?

Thanks!

There is a link at the bottom of every page that takes you straight to where you can edit the page. But we have to be careful about where we draw the line. We can’t be responsible for documentingv everything there is to know about Linux, Raspberry Pi’s and external software and technology. We’d spend all our time reproducing documentation that exists elsewhere without ever getting around to documenting openHAB itself