How to debug MQTT2 Binding issues?

Hi,

I am desperately trying to get zigbee2mqtt working with OpenHab2 but fail miserably.

My Setup:

  • Mosquitto and zigbee2mqtt running on same host as OH
  • I can subscribe to the zigbee/# wildcard topic with MQTT-Spy and see the messages created by my ZigBee sensors. The topics match those in my things configuration
  • JS and JsonPath transformations installed through PaperUI
  • MQTT2 binding installed through PaperUI

I followed the examples from this thread but I see absolutely no reaction of OH to the MQTT messages.

I increased log levels in the Karaf console using

openhab> log:set DEBUG org.eclipse.smarthome.binding.mqtt
openhab> log:set DEBUG org.eclipse.smarthome.io.transport.mqtt
openhab> log:set DEBUG org.eclipse.paho.client.mqttv3
openhab> log:set DEBUG org.openhab.binding.mqtt

but when a sensor is triggered or if I “artificially” post the exact same message through MQTT-Spy, there is nothing to be seen in the OH logs about this, except for these messages in openhab.log every minute, independent from what’s being sent through the broker:

2019-03-12 21:20:52.625 [DEBUG] [e.paho.client.mqttv3.TimerPingSender] - null: Check schedule at 1,552,425,652,625
2019-03-12 21:20:52.627 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: checkForActivity entered
2019-03-12 21:20:52.627 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping needed. keepAlive=60,000 lastOutboundActivity=1,552,425,592,618 lastInboundActivity=1,552,425,592,620
2019-03-12 21:20:52.628 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping message=PINGREQ
2019-03-12 21:20:52.629 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping token=org.eclipse.paho.client.mqttv3.MqttToken@28351f3a
2019-03-12 21:20:52.629 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: notifying queueLock holders
2019-03-12 21:20:52.630 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: new work or ping arrived
2019-03-12 21:20:52.631 [DEBUG] [o.client.mqttv3.internal.CommsSender] - paho3469211369349407: network send key=Ping msg=PINGREQ
2019-03-12 21:20:52.631 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: sent bytes count=2
2019-03-12 21:20:52.631 [DEBUG] [qttv3.internal.wire.MqttOutputStream] - null: Sent PINGREQ
2019-03-12 21:20:52.632 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received bytes count=1
2019-03-12 21:20:52.633 [DEBUG] [mqttv3.internal.wire.MqttInputStream] - null: Automatic Reconnect Successful: PINGRESP msgId 0
2019-03-12 21:20:52.633 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: key=Ping
2019-03-12 21:20:52.634 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: > key=Ping
2019-03-12 21:20:52.635 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: Schedule next ping at 60,000
2019-03-12 21:20:52.635 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping sent. pingOutstanding: 1
2019-03-12 21:20:52.636 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received key=0 message=PINGRESP msgId 0
2019-03-12 21:20:52.640 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.641 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.642 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: keyPing, msg=PINGRESP msgId 0, excep=null
2019-03-12 21:20:52.642 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: new workAvailable. key=Ping
2019-03-12 21:20:52.643 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping
2019-03-12 21:20:52.643 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: wait for new work or for space in the inflight window
2019-03-12 21:20:52.644 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping response received. pingOutstanding: 0
2019-03-12 21:20:52.645 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: callback and notify for key=Ping
2019-03-12 21:20:52.645 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received key=0 token=org.eclipse.paho.client.mqttv3.MqttToken@28351f3a message=PINGRESP msgId 0
2019-03-12 21:20:52.646 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.646 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: notify spaceAvailable
2019-03-12 21:20:52.646 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: wait for workAvailable
2019-03-12 21:20:52.647 [DEBUG] [client.mqttv3.internal.CommsReceiver] - paho3469211369349407: network read message

One topic - from a Xiaomi windows contact - that I see in MQTT Spy is for example zigbee/contact.OG.emil.window with a content of {"contact":true,"linkquality":23,"battery":97,"voltage":2995}.

And my /etc/openhab2/things/mqtt.things file that should link to that and other topics is as follows:

Bridge mqtt:broker:mqtt [ host="127.0.0.1", secure=false ]
{
        Thing topic zigbee {
        Channels:
                Type contact : marleneWindow "Fenster Marlene" [ stateTopic="zigbee/contact.OG.marlene.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : emilWindow "Fenster Emil" [ stateTopic="zigbee/contact.OG.emil.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : egEzTerrasseWindow "Terrasse Esszimmer" [ stateTopic="zigbee/contact.EG.EZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egWzTerrasseWindow "Terrasse Wohnzimmer" [ stateTopic="zigbee/contact.EG.WZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egToiletteWindow "Gaestetoilette" [ stateTopic="zigbee/contact.EG.toilette.window", transformationPattern="JS:doorsensor.js" ]
                Type switch  : ogGangBewegung "Bewegungsmelder OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JS:motionsensor.js" ]
                Type number  : ogGangHelligkeit "Helligkeit Flur OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JSONPATH:$.illuminance" ]
        }
}

and /etc/openhab2/transform/doorsensor.js:

(function(x){
    var result = "";
    var json = JSON.parse(x);
    if (json.contact)
    {
        result="CLOSED";
    }
    else
    {
        result="OPEN";
    }
    return result;
})(input)

To rule out the JavaScript file as source of error I changed from

Type contact : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JS:doorsensor.js" ]

to

Type string : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JSONPATH:$.contact" ]

but no change.

(i.e. string instead of contact and jsonpath instead of JS) but no change.

Now here’s /etc/openhab2/items/zigbee.items:

Contact marleneWindow      <contact> {channel="mqtt:topic:zigbee:marleneWindow"}
Contact emilWindow         <contact> {channel="mqtt:topic:zigbee:emilWindow"}
String  elternWindow       "Fenster Schlafzimmer [%s]" <contact> {channel="mqtt:topic:zigbee:elternWindow"}
Contact egEzTerrasseWindow <contact> {channel="mqtt:topic:zigbee:egEzTerrasseWindow"}
Contact egWzTerrasseWindow <contact> {channel="mqtt:topic:zigbee:egWzTerrasseWindow"}
Contact egToiletteWindow   <contact> {channel="mqtt:topic:zigbee:egToiletteWindow"}
Switch  ogGangBewegung     <motion>  {channel="mqtt:topic:zigbee:ogGangBewegung"}
Number  ogGangHelligkeit   <sun>     {channel="mqtt:topic:zigbee:ogGangHelligkeit"}

(that’s the version with JsonPath instead of JavaScript for the “elternWindow” contact)

The items are included in a sitemap but I also see no reaction there to anything happening on the MQTT broker.

The openhab MQTT client “paho3469211369349407” and the Zigbee2MQTT client “mqttjs_*” are connected to mosquitto:

grep -C1 127.0.0.1 /var/log/mosquitto/mosquitto.log | perl -pe 's/^(\d+)/localtime($1)/e'
[...]
Sun Mar 10 23:34:50 2019: Client paho3469187044616725 disconnected.
Sun Mar 10 23:34:50 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:34:50 2019: New client connected from 127.0.0.1 as paho3469210201847499 (c1, k60).
Sun Mar 10 23:34:51 2019: Client paho3469210201847499 disconnected.
Sun Mar 10 23:34:51 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:34:51 2019: New client connected from 127.0.0.1 as paho3469211369349407 (c1, k60).
Sun Mar 10 23:35:44 2019: Socket error on client mqttjs_eeeaf806, disconnecting.
Sun Mar 10 23:35:47 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:35:47 2019: New client connected from 127.0.0.1 as mqttjs_84c4ce72 (c1, k60).
Sun Mar 10 23:38:35 2019: Socket error on client mqttjs_84c4ce72, disconnecting.
Sun Mar 10 23:38:37 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:38:37 2019: New client connected from 127.0.0.1 as mqttjs_aab1dc92 (c1, k60).

OH seems to be happily connected to mosquito for two days now.

What else can I do to debug?

Regards
Alex

Configuration looks good. The MQTT Broker and Generic Thing are online in PaperUI?

Can you add a simple String channel/item for debugging and then use mosquitto_pub -t your_topic -m test_message?

Hi David,

thanks for the quick reply!
Both the Broker thing as well as the Generic MQTT thing show online in PaperUI.

I created a simple mqtt-to-sitmap-string-chain:
mqtt.things:

Bridge mqtt:broker:mqtt [ host="127.0.0.1", secure=false ]
{
        Thing topic zigbee {
        Channels:
                Type contact : marleneWindow "Fenster Marlene" [ stateTopic="zigbee/contact.OG.marlene.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : emilWindow "Fenster Emil" [ stateTopic="zigbee/contact.OG.emil.window", transformationPattern="JS:doorsensor.js" ]
                Type string  : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JSONPATH:$.contact" ]
// JS:doorsensor.js" ]
                Type contact : egEzTerrasseWindow "Terrasse Esszimmer" [ stateTopic="zigbee/contact.EG.EZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egWzTerrasseWindow "Terrasse Wohnzimmer" [ stateTopic="zigbee/contact.EG.WZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egToiletteWindow "Gaestetoilette" [ stateTopic="zigbee/contact.EG.toilette.window", transformationPattern="JS:doorsensor.js" ]
                Type switch  : ogGangBewegung "Bewegungsmelder OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JS:motionsensor.js" ]
                Type number  : ogGangHelligkeit "Helligkeit Flur OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JSONPATH:$.illuminance" ]
        }

        Thing topic mytopic {
        Channels:
                Type string : mytest "MQTT Test" [ stateTopic="testtopic/blabla" ]
        }

}

I.e. I added a Generic MQTT thing “mytopic” which also shows as online in PaperUI.

mqtttest.items:

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:mytopic:mytest" }

mqtt.sitemap:

sitemap mqtt label="mqtt" {
Text item=mqtttest
}

The site map only shows "MQTT Test - " in the browser. No change when publishing some text to testtopic/blabla (tried both mosquitto_pub locally on the OH/mosquitto server as well as from the desktop using MQTT-Spy).

“Reboot tut gut” - I’ll try a restart of the complete OpenHAB VM, you never know…

Regards
ALex

Rebooted the box, increased log level on mosquitto and see Openhab subscribe to all configured topics:

1552431502: New client connected from 127.0.0.1 as paho1623397930189 (c1, k60).
1552431502: Sending CONNACK to paho1623397930189 (0, 0)
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     homeassistant/# (QoS 0)
1552431502: paho1623397930189 0 homeassistant/#
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     homie/+/$homie (QoS 0)
1552431502: paho1623397930189 0 homie/+/$homie
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.marlene.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.marlene.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/motion.OG.floor (QoS 0)
1552431502: paho1623397930189 0 zigbee/motion.OG.floor
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.EZTerrasse.door (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.EZTerrasse.door
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.schlafzimmer.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.schlafzimmer.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.emil.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.emil.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/motion.OG.floor (QoS 0)
1552431502: paho1623397930189 0 zigbee/motion.OG.floor
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.WZTerrasse.door (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.WZTerrasse.door
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.toilette.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.toilette.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     testtopic/blabla (QoS 0)
1552431502: paho1623397930189 0 testtopic/blabla
1552431502: Sending SUBACK to paho1623397930189

Yet no reaction seen in the sitemap, in openhab.log, or in events.log for messages published to the relevant topics.

I’m running OpenHAB 2.4.0-1 from the official openhab debian repository and using 2.5.0-snapshot addons of the milight binding and the hueemulation (placed in /usr/share/openhab2/addons/):

root@vOpenHAB ~# apt show openhab2
Package: openhab2
Version: 2.4.0-1
Priority: optional
Section: java
Source: openhab2
Maintainer: https://community.openhab.org
Installed-Size: 83.5 MB
Depends: adduser
Suggests: zulu-8 | zulu-embedded-8 | oracle-java8-installer | openjdk-8-jdk-headless
Conflicts: openhab2-online, openhab2-offline
Homepage: www.openhab.org
Download-Size: 73.7 MB
APT-Manual-Installed: yes
APT-Sources: https://dl.bintray.com/openhab/apt-repo2 stable/main amd64 Packages
Description: openhab2
Linux installation package for openHAB 2.

N: There are 4 additional records. Please use the ‘-a’ switch to see them.
root@vOpenHAB ~# ll /usr/share/openhab2/addons/
total 164
drwxr-xr-x 2 openhab openhab 4096 Jan 28 23:05 ./
drwxr-xr-x 4 openhab openhab 4096 Jan 28 21:04 ../
-rw-r–r-- 1 openhab openhab 70 Dec 17 08:01 README
-rw-r–r-- 1 openhab openhab 77449 Jan 28 09:29 org.openhab.binding.milight-2.5.0-SNAPSHOT.jar
-rw-r–r-- 1 root root 76902 Jan 28 09:31 org.openhab.io.hueemulation-2.5.0-SNAPSHOT.jar

Not sure what other information I can provide.

Best regards
Alex

Try changing your items file to

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:broker:mytopic:blabla_whatever_your_needing_ to_do_like_power" }

No need to have mqttest, the items name, in the channel.

I don’t understand, the item name “mqtttest” is not in the channel. The names/strings I use map exactly to their use in the documentation.

Sorry, I messed up when replying, I think my eyes got crossed. Ignore the above about mqtttest in the channel.

What I meant to say was try adding the broker name in the channel after topic.

Example:

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:broker:mytopic:blabla_whatever_your_needing_ to_do_like_power" }

No need to have mqttest, the items name, in the channel.

Also note the 2.x binding might still have the issue where an OH restart is needed after making a file change. This was reported fixed in 2.5 so you may want to give OH a restart if you continue having a problem.

Hi all,

I do have the same issue, even with 2.5.0#1571. I do run a blank expert installation with only mqtt binding installed and try to get a text value updated to the sitemap.

From debug log I see that openhab gets the state updates

2019-04-16 13:25:57.908 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received bytes count=1

2019-04-16 13:25:57.909 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received bytes count=98

2019-04-16 13:25:57.909 [DEBUG] [mqttv3.internal.wire.MqttInputStream] - null: Automatic Reconnect Successful: PUBLISH qos:0 retained:false dup:false topic:"Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1" payload:[hex:4261636b206f6e207468652044616e636520466c utf8:"Back on the Dance Fl" length:39]

2019-04-16 13:25:57.909 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received key=0 message=PUBLISH qos:0 retained:false dup:false topic:"Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1" payload:[hex:4261636b206f6e207468652044616e636520466c utf8:"Back on the Dance Fl" length:39]

2019-04-16 13:25:57.909 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: new msg avail, notify workAvailable

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsReceiver] - jarvis2: network read message

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: call messageArrived key=0 topic=Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: notify spaceAvailable

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: wait for workAvailable

however the attached item is not updated.

Both thing and broker are online (whole config done via paper ui)


.. yet the item remains empty

Appreciate any ideas that could help me on this.

Might also be worth mentioning that adding a sonoff/tasmota device does work well without any trouble - at least for the power state.

Thanks
Tom

OK it seems that either the whitespace or the bracers are a problem for the binding. Removing those in the topics and it’s working like a charm.

You you please open a bug report about white-space and parentheses in mqtt topics?

Helps me keep track of this and to add a test case to the test suite.