I made some BLE sensors and BLE gateways that publish the sensor data as a series of MQTT topics. So, a BLE environmental sensor publishes data as this:
Mosquitto_sub is fast enough to capture all of these topics.
So among other pieces of data, you see humidity (“HUM”) and Temperature (“TMP”) and battery voltage and RSSI. These topics get published pretty much in the order you see above, and it comes in very fast. The humidity topic follows the temperature topic, as fast as the BLE gateway is capable of publishing.
If I create items to subscribe to both the temperature topic and the humidity topic, OpenHAB misses many humidity events. In the event log below, there should the one hum for each temp. But because the temps are published first, it seems the HUM gets missed while openhab is processing the temp data. I only managed to catch one hum below for seven temp events.
[20:15:32] openhabian@openHABianPi:/etc/openhab2/services$ tail -F /var/log/openhab2/events.log
2017-11-08 20:10:40.564 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.91 to 68.93
2017-11-08 20:11:40.572 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.93 to 68.94
2017-11-08 20:12:40.554 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.94 to 68.93
2017-11-08 20:13:10.572 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.93 to 68.94
2017-11-08 20:13:40.562 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.94 to 68.98
2017-11-08 20:15:10.584 [ItemStateChangedEvent ] - itm_temp_2E changed from 68.98 to 69.00
2017-11-08 20:15:13.288 [ItemStateChangedEvent ] - itm_hum_2E changed from 59.9 to 59.8
2017-11-08 20:17:10.565 [ItemStateChangedEvent ] - itm_temp_2E changed from 69.00 to 69.04
I can put some delays in the way my gateway publishes the BLE data, but is there a better solution? I’ve tried changing the QOS in mqtt.cfg from default 0 to 1 and then 2, but that doesn’t seem to help. This kinda effects design patterns. I expect to be able to just publish to mosquitto broker and everything magically shows up on OpenHAB without having to worry about timing.
This effects displayed items. But also effects logged data, if I’m relying on the item persistence to log data into InfluxDB, I miss lots of data points.
Wanted to clarify: The topics get published very fast (in blocks of like a dozen), but then there are minutes long lulls where nothing happens. So I’m not continuously pummeling OpenHAB with MQTT data. Just a quick one-two hit.
Did you use the QOS-flag while publishing? would slow down the process, but would ensure delivery?
Quality of Service
MQTT defines three levels of Quality of Service (QoS). The QoS defines how hard the broker/client will try to ensure that a message is received. Messages may be sent at any QoS level, and clients may attempt to subscribe to topics at any QoS level. This means that the client chooses the maximum QoS it will receive. For example, if a message is published at QoS 2 and a client is subscribed with QoS 0, the message will be delivered to that client with QoS 0. If a second client is also subscribed to the same topic, but with QoS 2, then it will receive the same message but with QoS 2. For a second example, if a client is subscribed with QoS 2 and a message is published on QoS 0, the client will receive it on QoS 0.
Higher levels of QoS are more reliable, but involve higher latency and have higher bandwidth requirements.
0: The broker/client will deliver the message once, with no confirmation.
1: The broker/client will deliver the message at least once, with confirmation required.
2: The broker/client will deliver the message exactly once by using a four step handshake.
I only have QOS=0 for publishing (using Arduino), because that’s the only available QOS. But that should be OK, right? I know the mosquitto is seeing those messages from the publisher. I thought the MQTT broker isolates the publisher from the subscriber and QoS is only referring to the broker-publisher or broker-subscriber relationship, not publisher-subscriber. I thought that was the major advantage of using a broker in the first place?
I tried both QOS=1 and QOS=2 in the </etc/openhab2/services/mqtt.cfg>, but it didn’t make a difference.
That makes sure, the traffic between OH2 and the MQTT-Server has an QOS higher than 0. That’s just one part of it.
You should implement the QOS > 0 in your BLE Gateway. I don’t know, how your BLE gateway works, but if it’s based on e.g. mosquitto, you should use the -q argument:
One of my gateways is an ESP8266, and it can publish with QoS=2.
But that’s not the problem here, since I can see all of the topics being published live with mosquitto-sub. I know the publish is fine. The corresponding events are not being processed by OH2, when looking at the events log. At least not consistently.
So OH’s MQTT plugin either doesn’t see multiple topics published in quick succession, or OpenHAB itself isn’t processing them.
Hmm. That’s right, just out of curiosity, can you implement the QOS for your ESP8266?
You have the MQTT-Server on your OH2, right? Then perhaps to rule out your hardware running all that OH2-stuff and the MQTT-server being too weak to process all that information (and I doubt it, with an QoS of 2…) do you have the possibility to use an MQTT-Server outside your OH2 hardware?
edit:
Just for thinking: I read out a whole bunch of sensors (46) periodically every minute in one go and all items were published on a MQTT-Server on my RPI3 Installation without problems (with QoS of 0). I then moved to running the MQTT on my Synology for other reasons, and still with QoS of 0 everything works fine.
) do you have the possibility to use an MQTT-Server outside your OH2 hardware?