Local python service MQTT vs REST communication

restapi
python
mqtt
Tags: #<Tag:0x00007f51deb6f8a0> #<Tag:0x00007f51deb6f5d0> #<Tag:0x00007f51deb6f378>

(Daniel Dakhno) #1

I am developing a python script that runs locally on my openhab-server and handles some I2C communication to read out sensors, since there is no binding for those sensors yet.

  1. Is there any preferable way to accomplish communication with such sensors other than to run an external script?
    Would it make sense to write a binding that executes python scripts?
    I know about the “exec” binding, but i need such a script to run indefinitely.

  2. What is the faster and preferable way of my script to throw data at HAB?
    Should i publish my values over a local MQTT broker or just make calls to the REST api over localhost?
    The latter would save me quite some code, memory and hops, if i can call it that.


(David Graeff) #2

You should go with a generic binding like HTTP or MQTT, for sure.
Unfortunately both protocols have really good support only in recent versions.

  • You need OH 2.4 milestone at least for an easy to use MQTT binding.
  • There is a new http binding in the making, that will probably be available with 2.5-snapshot.

Cheers, David


(Daniel Dakhno) #3

Thanks for the quick answer, that is basically what i expected.

I have been using MQTT for quite a while,
but at some point it felt wrong to send message over MQTT
to the same machine knowing that the broker would add some overhead
and the message would be distributed over the whole network,
although it was never supposed to leave the machine.

Is there any way to communicate with HAB over something like a UNIX domain socket?
I am just trying to basically do some sort of IPC as fast as possible.

Regards,
Daniel


(Alpoy) #4

Hm, why would it be distributed over the whole network? if you run the MQTT server on the same machine as OH is running on, you would only use localhost (127.0.0.1) …


(David Graeff) #5

Your script can proactively interact with openHAB via the REST interface.
You can read the state of an item and also change it. In the following line, the item “YourItemName” is set to “100”:

curl -X POST --header "Content-Type: text/plain" --header "Accept: application/json" -d "100" "http://192.168.0.46/rest/items/YourItemName"

(Daniel Dakhno) #6

i was wrong, the message doesn’t get anywhere if none subscribes to the topic.

I still think some sort of IPC over domain sockets would be a lot faster.
Could one implement a binding to provide a unix domain socket?

I know how the REST api works, I am just fantasizing about using a domain socket instead of TCP over loopback.

Thanks for the explanation, anyways.


(Alpoy) #7

Out of curiousity, what’s the problem with delay from unix sockets vs. tcp sockets? We’re talking small milliseconds…how important are those sensors timings really? :slightly_smiling_face:


(Daniel Dakhno) #8

I really need to know when my humidity changed with a precision of 0.01 millisecond.

Just joking, technically there is really no noticeable difference.
I am quite new in Openhab and learned about unix domain sockets recently
and just want to see what is possible and try everything.

I don’t think OH should be used for anything really timing-critical anyway.

Thanks for your help anyway, i was really just curious :slight_smile:


(Vincent Regaud) #9

That exists already. It’s the exec binding

Run your python script as a service

MQTT is faster and more efficient
It only takes a few lines of code to get the python paho library up and running


(Vincent Regaud) #10

If you are running a Raspberry Pi, why dont you use node-red and the i2c node?

https://nodered.org/

and


(David Graeff) #11

Why does everybody thinks this? MQTT is TCP based. Every UDP based protocol is more efficient in terms of overhead and “speed”. Speed as in packet round trips. TCP requires at least 3 packets for the session establishment, UDP is stateless so the first packet contains the data.


(Alpoy) #12

I believe he compared MQTT vs HTTP/REST api.

In HA connection reliability and “confirmed” transfers are probably more prefered than speed (in relation to TCP vs UDP) but that’s another discussion …


(Rich Koshak) #13

I think because at the application layer HTTP is chattier than MQTT is and the proforma and formats of the messages are larger. I personally don’t really have an opinion as to which one is more or less efficient. Both are plenty efficient for home automation purposes. But I think that is where the perception comes in.

But, having said that, it was my understanding that HTTP worked over TCP too. So the distinction between TCP and UDP isn’t really relevant in this context (unless I’m wrong, my memory is a bit rusty on some of this stuff).

If there is a binding, use that. If not see if there is software someone has already written and use that. There may need to be minor tweaks to get it to publish to OH. Vincent recommended NodeRed. I wrote https://github.com/rkoshak/sensorReporter (but it doesn’t do I2C yet).

No, you need to make this script be a separate service and not something that OH kicks off.

If you are writing the code yourself and you already are using MQTT, MQTT is by far the fastest way to get it coded up. MQTT can provide some additional features like retained messages, the ability to receive messages too, etc. which might be useful in their own right.

If you are not already using MQTT it is probably a wash as to which would be fastest to code.

Either approach will run plenty fast enough for most purposes.

You can see examples of MQTT and REST calls to OH in Python in my sensorReporter code.


(David Graeff) #14

The OP could use coap instead of mqtt for example, that’s why I like to point out that mqtt is not fast/efficient, in comparison. Rest or mqtt doesn’t really matter, both are equally fast of course. Http is more chatty but the rest API directly talks to the core, while mqtt has slimmer messages but those need to be parsed, channels need to be looked up and the core only gets notified indirectly.


(Daniel Dakhno) #15

Thanks for all the replies, for now it seems that running a python script as an encapsuled service seems to be the best approach.

I had a look at Node-RED, and although that may work, to me it seems to add a bit of overhead compared to a single service.

Concerning coap i am really not sure if that would buy me anything since it does not seem to be natively supported, and i don’t really expect it to be significantly faster than anything else in this case.
Thanks for pointing that out anyway, it is always nice to get to know new things.

Having tested REST and MQTT by now, i think i’ll go with MQTT, simply because it seems to be the more fitting solution for a sensor, although it will continue to bug me that this will ad one or two hops to the chain.

Thanks to anyone who contributed to the discussion, even if the discussed benefits in this case are really insignificant.


(Rich Koshak) #16

One thing that will make that a little easier to accept is that the new (but not yet ready for prime time) 2.4 MQTT binding has a built in broker so you wouldn’t need to run a separate Mosquitto (or whatever your preferred broker is).


(Aitor Iturrioz) #17

Hi @dakhnod!

My suggestion will be to use python and Homie (an MQTT convention, more information here: https://homieiot.github.io/. Awesome work @David_Graeff ), as Homie support will be builtin openHAB 2.4 (we are few weeks away from the final release).

Here you have a Homie library for python that works well: https://github.com/jalmeroth/homie-python Sadly, the official version of the library doesn’t support Homie 3, but I have made a PR (https://github.com/jalmeroth/homie-python/pull/47) that adds support for the version that openHAB will support. You can download it from my fork: https://github.com/bodiroga/homie-python/tree/homie-v3.0.0, although the main idea is to update the primary repository, I don’t want to fork the project. Have a look at the examples folder, it’s very ease to use :wink:

IMHO, that’s the best interoperable solution: MQTT (well supported transport protocol) and Homie (openHAB independent -but well supported- convention). Your script will not be tied to any platform, but compatible with everything.

Good luck!

Aitor


(Daniel Dakhno) #18

So, the internal Broker seems to be available, although i am somehow unable to really find sufficient documentation.

Firstly, for some reason the misc-configuration for the broker is blank, any way to fix that?

Secondly, i am wondering whether that will really give me any improvement.
How tight is the broker integrated into OpenHAB?
Does the MQTT binding still have to connect to localhost like in the case of Mosquitto?
Or are events handled another way, really saving network traffic and time?


(David Graeff) #19

Nope. Internally there will still be a normal TCP connection established. Use Mosquitto if it is already installed.


(Lou Erickson) #20

Don’t think of the TCP or network connection as waste. Think of it as flexibility. In the future, if you ever need to move this service off this computer and on to another, MQTT will make that trivial. Right now it’s all on one host, but that can very easily change; using domain sockets makes that very hard. Using TCP makes it very easy.

Domain sockets are much less rarely used lately; the minor (milliseconds!) overhead of network traffic is offset by the flexibility, scalability, and interoperability that the network gives.