MQTT with openHAB 2.5: cannot get it working

Hi everyone!

In order to get sensor data from miflora compatible sensor devices I want to setup a MQTT server for my smarthome network, where Thomas Dietrich’s MQTT client enqueues the sensor data and my openHAB instance retrieves and provides it in items.

In order to keep the setup simple, I decided to let the MQTT server and client run on the same machine, i.e. the one that also runs openHAB. As MQTT server I’ve installed MQTT Broker Moquette vom den Misc tab in PaperUI.

To access the MQTT server I’ve additionally installed the MQTT binding from the Bindings tab in PaperUI. For parsing the JSON output I’ve installed some transformations via PaperUI aswell: Javascript, JSONPath and RegEx (JSON should be the default format stored in the broker).

I’ve created a miflora.things:

Bridge mqtt:systemBroker:embedded-mqtt-broker  "Moquette Embedded MQTT System Broker" @ "Sf" [ 
    host="localhost",
    port="1883",
    retain="false",
    qos="2",
    keep_alive_time_ms="60",
    reconnect_time_ms="10000",
    url="localhost:1883",
    clientid="Oh2Mqtt",
    username="openhabian",
    password="***"
] 
{
    Thing topic yucca "Yucca" @ "SrLr" {
        Channels:
            Type number : light         "Light Intensity"   [ stateTopic="miflora/Yucca", transformationPattern="JSONPATH:$.light" ]
            Type number : temperature   "Temperature"       [ stateTopic="miflora/Yucca", transformationPattern="JSONPATH:$.temperature" ]
            Type number : conductivity  "Soil Fertility"    [ stateTopic="miflora/Yucca", transformationPattern="JSONPATH:$.conductivity" ]
            Type number : moisture      "Soil Moisture"     [ stateTopic="miflora/Yucca", transformationPattern="JSONPATH:$.moisture" ]
            Type number : battery       "Battery Charge"    [ stateTopic="miflora/Yucca", transformationPattern="JSONPATH:$.battery" ]
    }
}

My items files looks like this:

Number  Yucca_Door_Battery "Yucca Batteriestand [%d %%]" <text>  {mqtt="<[broker:miflora/Yucca:state:JSONPATH($.battery)]"}
Number  Yucca_Door_Temperature " Yucca Lufttemperatur [%.1f °C]" <text>  {mqtt="<[broker:miflora/Yucca:state:JSONPATH($.temperature)]"}
Number  Yucca_Door_Moisture "Yucca Feuchtigkeit [%d %%]" <text>  {mqtt="<[broker:miflora/Yucca:state:JSONPATH($.moisture)]"}
Number  Yucca_Door_Conductivity "Yucca Fruchtbarketi [%d µS/cm]" <text>  {mqtt="<[broker:miflora/Yucca:state:JSONPATH($.conductivity)]"}
Number  Yucca_Door_Light "Yucca Lichteinfall [%d lux]" <text>  {mqtt="<[broker:miflora/Yucca:state:JSONPATH($.light)]"}

As for the /opt/miflora-mqtt-daemon/config.ini I’m using the following deviations from the default config:

# The period between two measurements in seconds (Default: 300)
period = 10

# The MQTT broker authentification credentials (Default: no authentication)
# Will also read from MQTT_USERNAME and MQTT_PASSWORD environment variables
username = openhabian
password = ***

[Sensors]
Yucca = C4:7C:3A:2F:92:19

First problem is that the miflora service seems to be started too early:

# systemctl status miflora.service
● miflora.service - Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon
   Loaded: loaded (/etc/systemd/system/miflora.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2020-07-05 04:00:50 CEST; 5h 44min ago
     Docs: https://github.com/ThomDietrich/miflora-mqtt-daemon
  Process: 1054 ExecStart=/usr/bin/python3 /opt/miflora-mqtt-daemon/miflora-mqtt-daemon.py (code=exited, status=1/FAILURE)
 Main PID: 1054 (code=exited, status=1/FAILURE)
   Status: "Jul 05 04:00:49 - MQTT connection error. Please check your settings in the configuration file "config.ini"."

Jul 05 04:00:49 openHABianPi systemd[1]: miflora.service: Unit entered failed state.
Jul 05 04:00:49 openHABianPi systemd[1]: miflora.service: Failed with result 'exit-code'.
Jul 05 04:00:50 openHABianPi systemd[1]: miflora.service: Service hold-off time over, scheduling restart.
Jul 05 04:00:50 openHABianPi systemd[1]: Stopped Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon.
Jul 05 04:00:50 openHABianPi systemd[1]: miflora.service: Start request repeated too quickly.
Jul 05 04:00:50 openHABianPi systemd[1]: Failed to start Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon.
Jul 05 04:00:50 openHABianPi systemd[1]: miflora.service: Unit entered failed state.
Jul 05 04:00:50 openHABianPi systemd[1]: miflora.service: Failed with result 'exit-code'.

Is there a way to have it retry connecting more often?

Retrying manually after openHAB is fully started up delivers:

# systemctl start miflora.service
# systemctl status miflora.service
● miflora.service - Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon
   Loaded: loaded (/etc/systemd/system/miflora.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-07-05 09:47:07 CEST; 1s ago
     Docs: https://github.com/ThomDietrich/miflora-mqtt-daemon
 Main PID: 11554 (python3)
   Status: "Jul 05 09:47:06 - MQTT connection established."
    Tasks: 3 (limit: 4915)
   CGroup: /system.slice/miflora.service
           ├─11554 /usr/bin/python3 /opt/miflora-mqtt-daemon/miflora-mqtt-daemon.py
           └─11562 /usr/local/lib/python3.5/dist-packages/bluepy/bluepy-helper 0

Jul 05 09:47:05 openHABianPi systemd[1]: Starting Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon...
Jul 05 09:47:07 openHABianPi systemd[1]: Started Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon.

I’ve also checked that events should arrive in the queue manually:

# sudo python3 miflora-mqtt-daemon.py

Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon
Source: https://github.com/ThomDietrich/miflora-mqtt-daemon

[2020-07-05 09:49:26] Connecting to MQTT broker ...
[2020-07-05 09:49:26] MQTT connection established

Adding sensor to device list and testing connection ...
Name:          "Yucca"
[2020-07-05 09:49:27] Initial connection to Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19) failed due to exception:

[2020-07-05 09:49:27] Announcing Mi Flora devices to MQTT broker for auto-discovery ...

[...]

[2020-07-05 09:49:49] Retrieving data from sensor "Yucca" ...
[2020-07-05 09:50:14] Retrying due to exception: Could not read data from Mi Flora sensor C4:7C:3A:2F:92:19
[2020-07-05 09:50:18] Result: {"light": 409, "temperature": 24.6, "moisture": 29, "conductivity": 226, "battery": 100}
[2020-07-05 09:50:18] Publishing to MQTT topic "miflora/Yucca"

Success rate of querying is not too high though, I should consider moving the Raspi closer to the devices.

Assuming that the MQTT connection to the broker/server should be fixable, let’s come to the next problem. Judging from the logs I think there’s a configuration issue in my miflora.things and some other problem related to a clientId and/or authentication, but I could not come up with a solution:

openhab.log:

2020-07-05 10:23:51.984 [INFO ] [roker.internal.EmbeddedBrokerService] - Broker persistence file: mqttembedded.bin
2020-07-05 10:23:56.414 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to 'localhost' with clientid embedded-mqtt-broker

events.log:

2020-07-05 10:24:53.506 [hingStatusInfoChangedEvent] - 'mqtt:systemBroker:embedded-mqtt-broker' changed from UNINITIALIZED to INITIALIZING
2020-07-05 10:24:53.532 [hingStatusInfoChangedEvent] - 'mqtt:systemBroker:embedded-mqtt-broker' changed from INITIALIZING to ONLINE
2020-07-05 10:25:08.964 [hingStatusInfoChangedEvent] - 'mqtt:topic:embedded-mqtt-broker:yucca' changed from UNINITIALIZED to INITIALIZING
2020-07-05 10:25:09.291 [hingStatusInfoChangedEvent] - 'mqtt:topic:embedded-mqtt-broker:yucca' changed from INITIALIZING to ONLINE
2020-07-05 10:30:07.257 [WARN ] [io.moquette.broker.MQTTConnection   ] - Broker doesn't permit MQTT empty client ID. Username: openhabian, channel: [id: 0x16af6309, L:/0:0:0:0:0:0:0:1:1883 - R:/0:0:0:0:0:0:0:1:48763]

I did some research here in the forums but could not find anything beyond “use another broker” regarding the empty client id. Since Moquetto was introduced in openHAB 2.5 I think that this shouldn’t be necessary – or is it known to malfunction and shouldn’t be included in this version?

Platform information:
Hardware: Raspberry Pi 3 Model B Plus Rev 1.3
OS: openHABian / Linux 4.19.66-v7+
Java Runtime Environment: OpenJDK Runtime Environment (Zulu8.40.0.178-CA-linux_aarch32hf) (build 1.8.0_222-b178)
openHAB version: 2.5.3-1

You have (probably) installed version 2 of the MQTT binding, but your items syntax is for a version 1 binding. You just need to update your syntax on the items - something like:

Number  Yucca_Door_Battery "Yucca Batteriestand [%d %%]" <text>  {channel="mqtt:topic:yucca:battery"}

Secondly, the in-built MQTT broker (Moquette) will not be supported for much longer, so it’s currently recommended that if you’re just starting out then install Mosquitto.

1 Like

Thank you for your valuable feedback! I think this took me one or two steps further towards a working system…

First of all, the miflora client service now does not exit at startup.
Secondly, I don’t see the warning regarding the empty client id.

Great! But, I don’t see any updates to my items.

My log files show

2020-07-05 11:28:49.504 [hingStatusInfoChangedEvent] - 'mqtt:topic:mqtt-broker:yucca' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
2020-07-05 11:28:56.237 [hingStatusInfoChangedEvent] - 'mqtt:broker:mqtt-broker' changed from UNINITIALIZED to INITIALIZING
2020-07-05 11:28:56.367 [hingStatusInfoChangedEvent] - 'mqtt:broker:mqtt-broker' changed from INITIALIZING to OFFLINE
2020-07-05 11:28:56.400 [hingStatusInfoChangedEvent] - 'mqtt:topic:mqtt-broker:yucca' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
2020-07-05 11:28:56.464 [hingStatusInfoChangedEvent] - 'mqtt:topic:mqtt-broker:yucca' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
2020-07-05 11:28:58.793 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to 'localhost' with clientid d388e3f0-514e-4635-979e-424af1229cdb
2020-07-05 11:28:59.378 [hingStatusInfoChangedEvent] - 'mqtt:broker:mqtt-broker' changed from OFFLINE to ONLINE
2020-07-05 11:28:59.389 [me.event.ThingUpdatedEvent] - Thing 'mqtt:broker:mqtt-broker' has been updated.
2020-07-05 11:28:59.446 [hingStatusInfoChangedEvent] - 'mqtt:topic:mqtt-broker:yucca' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE

I cannot spot anything problematic.

My item configuration looks like this:

Number  Yucca_Door_Battery "Yucca Batteriestand [%d %%]" <text>  {channel="mqtt:topic:mqtt-broker:yucca:battery"}
Number  Yucca_Door_Temperature " Yucca Lufttemperatur [%.1f °C]" <text>  {channel="mqtt:topic:mqtt-broker:yucca:temperature"}
Number  Yucca_Door_Moisture "Yucca Feuchtigkeit [%d %%]" <text>  {channel="mqtt:topic:mqtt-broker:yucca:moisture"}
Number  Yucca_Door_Conductivity "Yucca Fruchtbarketi [%d µS/cm]" <text> {channel="mqtt:topic:mqtt-broker:yucca:conductivity"}
Number  Yucca_Door_Light "Yucca Lichteinfall [%d lux]" <text>  {channel="mqtt:topic:mqtt-broker:yucca:light"}

I’ve copied the channel identifiers from the yucca thing configuration in PaperUI. There both, the Yucca thing and the Broker appear online.

I’ve checked that the miflora service periodically enqueues state information:

# systemctl status miflora.service
● miflora.service - Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon
   Loaded: loaded (/etc/systemd/system/miflora.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-07-05 11:31:18 CEST; 30min ago
     Docs: https://github.com/ThomDietrich/miflora-mqtt-daemon
 Main PID: 2136 (python3)
   Status: "Jul 05 12:02:09 - Status messages published."
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/miflora.service
           └─2136 /usr/bin/python3 /opt/miflora-mqtt-daemon/miflora-mqtt-daemon.py

Jul 05 11:31:17 openHABianPi systemd[1]: Starting Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon...
Jul 05 11:31:18 openHABianPi systemd[1]: Started Xiaomi Mi Flora Plant Sensor MQTT Client/Daemon.
Jul 05 11:32:14 openHABianPi python3[2136]: [2020-07-05 11:32:14] Retrying due to exception: Could not read data from Mi Flora sensor C4:7C:3A:2F:92:19
Jul 05 11:32:41 openHABianPi python3[2136]: [2020-07-05 11:32:41] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 50%
Jul 05 11:34:35 openHABianPi python3[2136]: [2020-07-05 11:34:35] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 60%
Jul 05 11:41:31 openHABianPi python3[2136]: [2020-07-05 11:41:31] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 84%
Jul 05 11:44:30 openHABianPi python3[2136]: [2020-07-05 11:44:30] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 83%
Jul 05 11:50:16 openHABianPi python3[2136]: [2020-07-05 11:50:16] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 85%
Jul 05 11:55:23 openHABianPi python3[2136]: [2020-07-05 11:55:23] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 85%
Jul 05 11:59:49 openHABianPi python3[2136]: [2020-07-05 11:59:49] Failed to retrieve data from Mi Flora sensor "Yucca" (C4:7C:3A:2F:92:19), success rate: 87%

Update:
I saw a configuration possibility in PaperUI at Configuration -> Services -> MQTT system broker connection. It’s from org.eclipse.smarthome.mqttbroker and comes with installing the MQTT binding. When I click “Manage” it leads to “Configure MQTT system broker connection”. I tried configuring another channel I could listen on, but that did not succeed either.

I found this to be a bit confusing: why are there two ways to connect a MQTT broker and which one should I use? Why does neither of them work?

Well, is there a way to debug this?

I think it should be somehow possible to isolate the problem, since there’s a chain of involved instances:

  1. miflora service which should enqueue events into Mosquitto
  2. Mosquitto topic which should be subcribed by some openHAB service
  3. openHAB addon which converts MQTT events to items
    (4. converters which map JSON)

Can I somehow narrow down where the problem is?

You need to first check that 1 is working, as that’s outside of openHABs control. As you have Mosquitto installed your also have mosquitto_sub - have a look online how to use it to watch your topics of interest to make sure that your miflora stuff is being sent correctly.

From memory, you could do:

mosquitto_sub -t '#'

from the command line or terminal on the same machine that is running Mosquitto to subscribe to all topics. Then wait!

1 Like

Mosquitto receives the events from miflora:

# mosquitto_sub -t '#' -u openhabian -P ***
{"yucca": {"refresh": 10, "location_clean": "", "mac": "C4:7C:3A:2F:92:19", "topic": "miflora/yucca", "name_pretty": "yucca", "firmware": "3.2.4", "location_pretty": ""}}
{"light": 478, "temperature": 25.3, "moisture": 28, "conductivity": 223, "battery": 100}
{"light": 297, "temperature": 25.2, "moisture": 28, "conductivity": 225, "battery": 100}
{"light": 329, "temperature": 25.3, "moisture": 28, "conductivity": 222, "battery": 100}
{"light": 220, "temperature": 25.2, "moisture": 28, "conductivity": 225, "battery": 100}

Can I debug the further processing somehow?

You can use the console to set logging at DUBUG or TRACE level for any binding.

1 Like

Is this the full output from your terminal? It doesn’t show what topics are sending these JSON strings…

This is the full output… I thought the first line indicated the topic it is listening on.

I’ve added a debug flag. Here’s the output again:

# mosquitto_sub -t '#' -u openhabian -P * -d
Client mosqsub/22007-openHABia sending CONNECT
Client mosqsub/22007-openHABia received CONNACK
Client mosqsub/22007-openHABia sending SUBSCRIBE (Mid: 1, Topic: #, QoS: 0)
Client mosqsub/22007-openHABia received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/22007-openHABia received PUBLISH (d0, q0, r1, m0, 'miflora/$announce', ... (170 bytes))
{"yucca": {"refresh": 10, "location_clean": "", "mac": "C4:7C:3A:2F:92:19", "topic": "miflora/yucca", "name_pretty": "yucca", "firmware": "3.2.4", "location_pretty": ""}}
Client mosqsub/22007-openHABia received PUBLISH (d0, q0, r0, m0, 'miflora/yucca', ... (88 bytes))
{"light": 249, "temperature": 25.1, "moisture": 28, "conductivity": 223, "battery": 100}
Client mosqsub/22007-openHABia received PUBLISH (d0, q0, r0, m0, 'miflora/yucca', ... (88 bytes))
{"light": 171, "temperature": 25.1, "moisture": 28, "conductivity": 223, "battery": 100}

@H102 Thanks, I’m trying that!

OK, this all looks fine.

What does your latest things file look like now?

Also, just to make sure: you haven’t tried to set anything up using PaperUI? Using configuration files and PaperUI to setup the same stuff never works well!

Really strange: I set the DEBUG log level for MQTT with log:set DEBUG org.openhab.binding.mqtt, then after a few seconds my items started receiving update events…

I’ve set the log level back to INFO again, and still, my items receive update events…

Maybe something was stuck, I don’t know. During my tests I did a couple of reboots, but none of them led to my items receiving updates…

Anyway, now everything looks good! I’ve done a reboot in the meantime as well, my items still get update events.

Thank you both so much for your help! I’m glad everything now works!

Glad you got it working. :+1:

Please click the square box on the post that provided the solution to mark the topic as solved.

Thanks