Help with architecture of new system

Absolutely you can both send and receive. Mosquitto seems to be the broker of choice and Paho seems to be the library of choice. I can say that setting up Mosquitto on my system took less than an hour and sending and receiving messages too about 24 lines of Python code from some custom scripts I wrote.

From the little I know about ZMQ I’d say it is very similar. I too do professional development and I come from a mainly JMS and AQMP messaging protocol background and can say MQTT is way lighter weight and requires way less configuration even when enabling encryption and authentication.

One really nice feature from a system monitoring perspective is the Last Will and Testament. If an MQTT registers a LWT message and topic, when the server detects the client is no longer connected it will post that message to the topic. You can then have openHAB listen on that topic and generate alerts or take restorative action when the connection is lost.

Because it is so light weight and requires so little configuration I think it is one of the preferred messaging bus technologies for home automation in general, not just openHAB.

Would you terribly mind giving me a link or some further description on setup of these simple nodes?

Very cool! I’m really stoked to pick a design and get started. I think I might try a couple of node types and then run with the one that works best. It seems like it’d be good to have some zwave in there, as some of the thermostats I’ve looked at are zwave.

Here is good starting point: https://github.com/Imroy/pubsubclient

There are tons of information about ESP8266 + arduino ide + mqtt on the net.

@Chuck_Claunch:

Just to add another viewpoint: MQTT seems to be a very popular choice for implementing new sensors. I have worked with MQTT-based sensors before, but when I started building temperature/humidity sensors, I decided against using MQTT.

My rationale is simple: I need another component (the broker). The broker can go wrong at any time, and for mosquitto (which seems to be the most popular choice) I saw some outages at work. On the other hand, with the readily available webserver components for the ESP8266, it is very easy to create a sensor that can be queried just via HTTP and discovered using MDNS.

HTTP is a component included in OpenHAB, it is well-understood and easy to use. So for me, it seemed like the better choice. But don’t get me wrong: I’m perfectly fine with other projects going different routes. In fact I think it is interesting to see how different ideas compete and evolve. The nice thing about OpenHAB is that it simplifies the integration of very different protocols, so even if you decide to abandon one technology and go for another one, you can keep your rules and datasets.

Maybe you want to look at New binding for home made hardware which is what I did to solve a pretty similar problem. The only drawback of this solution is that the XBee modules are about ~25 € each, but I got a very stable network which is self organized and independent of the entertainment (wifi) network.

Wow thanks again everyone for all the resources. I’ll have to well document what I go with and try and contribute back to the community.

@gonium I like that approach too, I wonder if there’s any difference in battery life between the two?

I really want to do everything in my power to avoid batteries altogether. I already have a PoE switch and was thinking a PoE solution might be awesome, but then it seems PoE parts are fairly expensive unless I design my own.

@juergen_richtsfeld Yeah I like the XBee too, I’ve used it in a project before and liked it. But yeah due to cost I think I’d easily go with the ESP8266 over it.

You mean HTTP vs. MQTT? I don’t think that there is any difference between the two in terms of power consumption. Both need a running wifi connection along with an IP stack - so basically, the processor is running most of the time.

I have my sensors connected to mains power all the time - I did a quick check and found that they are using 2W of power each. The main source of power loss seems to be the power supply itself. I had a bunch of old 3V3 wall-wart power supplies sitting on my desk and put them to use, no special consideration here.

But @mjptec mentioned in this thread that he wanted to investigate operating the sensors from a battery. The main power draw occurs when wifi is switched on, but in deepsleep mode, the ESP8266 draws only around 20 microamps. I therefore think it is a viable option to have the ESP8266 powered down most of the time and then wake up just to take a measurement and submit it to OpenHAB.

No matter what your technology, if you can push the data from your energy constrained sensors only when necessary as opposed to setting up the server to poll for the data it will be more energy efficient. I think this is one of the drivers behind having a gateway device to bridge between the ESP network and the IP network, one that has wall power so energy isn’t a problem. And once you have a bridge, it makes sense to configure that bridge to provide a nice and easy interface to the data, be that an HTTP REST API, MQTT, etc.

@rikoshak: I agree regarding the push vs. poll strategy. An energy-efficient sensor should sleep, then push its measurements somewhere, and then sleep again.

BUT: (here comes my not-so-secret plan): I don’t think you need any special bridge. I intend to try to use the OpenHAB REST API directly, making the raspberry pi running my OpenHAB instance effectively the bridge. It should be as simple as using Arduinos HTTPClient class, but I haven’t tried it yet. Another advantage of the stick-to-http-strategy :wink:

As the saying goes, “there is more than one way to skin a cat.” :slight_smile:

I am interested in your approach. I’ve settled on MQTT primarily because I already had it set up (I played around with OwnTracks for awhile) so I already had the broker set up and configured. It also seems really popular so I never changed. I also like the Last Will and Testament feature which really helps me detect when a client goes down (my Raspberry Pis are very flakey, though I’ve never have my Mosquitto server go down).

So my questions concerning your approach:

  1. How will you manage actuators (e.g. a relay)? I can’t see a way to use this approach without having the remote device either always on the network or constantly polling. Perhaps this approach mainly works for sensors only. (NOTE: I’m not implying that the MQTT approach has an answer for this either).

  2. What are you running your openHAB instance on and how are you getting it to talk to the ESP8266 network? To my knowledge there isn’t a binding for it yet; are you writing one or planing on using the Exec binding and scripts? Essentially this is the purpose of the bridge, to bridge the ESP8266 network and your LAN. Would you approach work on a general computer/synology or would it require openHAB be running on something that has GPIO pins?

I have five Arduinos with a bunch of sensors and RMF69HW transceivers all in parts in a drawer somewhere that I plan on building and deploying. I’m pretty much stuck with a gateway no matter what because I have openHAB running on an old laptop but I’m sure I can learn something from your approach.

Before I ever discovered OpenHAB, I was looking at this solution. Although it requires some SMD soldering, it’d end up around $6 per node and be hard wired ethernet. I think my pie in the sky solution would be to modify this solution to be powered via PoE (I already have a PoE switch) but that may be too much of a task to start with. I already have an arduino with ethernet shield laying around so I’ll probably begin with that as a test. Since I have the shield I suppose there’s no need for any gateway or anything, I can just send the data via IP to OpenHAB correct?

Yes, with the right code you can plop a web server on it or a HTTP client like @gonium 's approach, plop MQTT client on it like I’ve been talking about, or come up with some other IP based approach. There is a TCP binding if oyu prefer working at a lower level.

1 Like

You may also want to look at mysensors.org

They now support ESP8266

I also have been using mysensors for about a year, and the system works excellent, I use 12 or 13 of them around the house, some battery sensors others hard wired to a common power supply. They range from dimmer to temperature sensor. The range is over 40m in my case as they act as a repeater node if they need automagically. I will be honest in that the learning curve was 4 to 6 mths, and confidence time about 4 mths. My only gripe right now is not keeping track of software versions I use and what each node actually does! mysensors runs on a cut down MQTT from what I can see and I use a Arduino Ethernet gateway with 2nRF01 long range device. The MQTT service has never faulterd in the months it has been running, neither has my pi which is the Openhab server.
I like the Openhab platform as it also can talk to kodi, and I can send json requests, http requests, apply xml transforms to web pages trigger cctv alarms, its totally amazing what it can do. Needs a new interface though ;-(

I’d like to just second the MQTT approach. Installing a broker onto the Raspberry Pi is incredibly simple, and it’s easy to debug since you can subscribe to the messages on another PC to check your underlying MQTT network is fine, before proceeding to put those into OpenHAB.

I’m also trialling the NLRF+ wireless things for Arduino at the moment, and they’re an absolute ass to work with. They don’t like a clone Arduino’s 3v3 (original boards seem fine), they dont like long jumper cables, and theyre just difficult to work with. Eugh.

Part 2 of my guide for OpenHAB on Pi was published yesterday, which gives you a complete introduction to MQTT protocol and how to get it working on Pi/OpenHAB. I used Ethernet shield for Arduino but the wireless ones / ESP2866 will work fine too. http://www.makeuseof.com/tag/openhab-beginners-guide-part-2-zwave-mqtt-rules-charting/

3 Likes

As for your questions, @rlkoshak :

  1. I currently only have sensors - my actors are Homematic components. But if I had to implement an actor it would be always on - since most actuators switch power or depend on power to fulfill their function they would not depend on batteries. Another reason for the always-on approach is that the state of the actor might change locally (i.e. a local button to change the state of the builtin relay). So the actor should always represent the state of whatever is connected to it.

  2. Maybe there is a misunderstanding: the ESP8266s are just wifi devices. They sit in my internal network just as my RPi running OpenHAB is. There is no physical bridge necessary. If you look at the readme of my project you will find how I integrated them with OpenHAB: I simply use the HTTP binding and a JSON transform. No need to write a binding for that.

BUT: my devices are currently always-on. I haven’t thought this through, but if a device would sleep for some time, the only viable communication direction would be from the device to the OpenHAB instance. That should work for a sensor, but for an actor this seems inadequate. If I press my light button I want light now :wink: Another reason why actors should be always-on.

Another piece of excellent, well-written work.

Ah. Clearly my knowledge of all of these wireless chips is not that good. Most of what I’ve been looking at has been using lower power and simpler wireless protocols than WiFi so a gateway is required.

Z-Wave is certainly not a cheap choice, but effectively one is confronted with the problem of needing to use protocols which the products you intended to use communicate with. This is a good reason for multiple automation networks, as different communication needs can be handled appropriately to the budget, requirements, etc.

For example… One-Wire is a cheap and easy way to run a bunch of wired sensors around a resonably small area. It is a daisy-chained network with basic (cheap) cable, so you don’t get into the high cost of running many pairs of more expensive copper wire back to a central location. The devices are also cheap, and have been around for quite a long time. (there is a ready-to-use 1-Wire/Ethernet gateway which works with OpenHAB)

There is also 433MHz which can do A LOT on a budget, but if you want to avoid batteries then wiring would be best. Honestly, I tend to agree about avoiding batteries, but sadly there aren’t many products out there that you can easily externally power (and they’re mostly wireless).

You mentioned the hub, sensors, and gateway… for me, from an architectural perspective, it is important to also keep in mind the connectivity (network, protocols, transport) and network interface. The choice of sensors/actuators, network, interface, gateway, etc will all be dependent on your needs and how you architect your solution. MQTT is TCP/IP level, so once you have a network interface from whatever network to OpenHAB, you can put everything on MQTT (I do this with OpenHAB + RFXCom binding + MQTT binding). Or you can use MQTT directly from something like an Arduino.

Regarding your desire to use PoE… I too have a PoE switch… and would love EVERYTHING to run on PoE (for many reasons). However there are some reasons why this is not feasible for cheap sensors as the electronics required for appropriate power conditioning are apparently not cheap. There is a guy called Jonathan Oxer from Superhouse TV and Freetronics who has designed a custom Arduino which has PoE (and Ethernet obviously) built in for much less than adding a PoE shield to an Arduino. He has also wired his house almost entirely with Ethernet, PoE, and Arduinos to control everything.

1 Like