How to debug MQTT2 Binding issues?

Hi,

I am desperately trying to get zigbee2mqtt working with OpenHab2 but fail miserably.

My Setup:

  • Mosquitto and zigbee2mqtt running on same host as OH
  • I can subscribe to the zigbee/# wildcard topic with MQTT-Spy and see the messages created by my ZigBee sensors. The topics match those in my things configuration
  • JS and JsonPath transformations installed through PaperUI
  • MQTT2 binding installed through PaperUI

I followed the examples from this thread but I see absolutely no reaction of OH to the MQTT messages.

I increased log levels in the Karaf console using

openhab> log:set DEBUG org.eclipse.smarthome.binding.mqtt
openhab> log:set DEBUG org.eclipse.smarthome.io.transport.mqtt
openhab> log:set DEBUG org.eclipse.paho.client.mqttv3
openhab> log:set DEBUG org.openhab.binding.mqtt

but when a sensor is triggered or if I “artificially” post the exact same message through MQTT-Spy, there is nothing to be seen in the OH logs about this, except for these messages in openhab.log every minute, independent from what’s being sent through the broker:

2019-03-12 21:20:52.625 [DEBUG] [e.paho.client.mqttv3.TimerPingSender] - null: Check schedule at 1,552,425,652,625
2019-03-12 21:20:52.627 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: checkForActivity entered
2019-03-12 21:20:52.627 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping needed. keepAlive=60,000 lastOutboundActivity=1,552,425,592,618 lastInboundActivity=1,552,425,592,620
2019-03-12 21:20:52.628 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping message=PINGREQ
2019-03-12 21:20:52.629 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping token=org.eclipse.paho.client.mqttv3.MqttToken@28351f3a
2019-03-12 21:20:52.629 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: notifying queueLock holders
2019-03-12 21:20:52.630 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: new work or ping arrived
2019-03-12 21:20:52.631 [DEBUG] [o.client.mqttv3.internal.CommsSender] - paho3469211369349407: network send key=Ping msg=PINGREQ
2019-03-12 21:20:52.631 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: sent bytes count=2
2019-03-12 21:20:52.631 [DEBUG] [qttv3.internal.wire.MqttOutputStream] - null: Sent PINGREQ
2019-03-12 21:20:52.632 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received bytes count=1
2019-03-12 21:20:52.633 [DEBUG] [mqttv3.internal.wire.MqttInputStream] - null: Automatic Reconnect Successful: PINGRESP msgId 0
2019-03-12 21:20:52.633 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: key=Ping
2019-03-12 21:20:52.634 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: > key=Ping
2019-03-12 21:20:52.635 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: Schedule next ping at 60,000
2019-03-12 21:20:52.635 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping sent. pingOutstanding: 1
2019-03-12 21:20:52.636 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received key=0 message=PINGRESP msgId 0
2019-03-12 21:20:52.640 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.641 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.642 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: keyPing, msg=PINGRESP msgId 0, excep=null
2019-03-12 21:20:52.642 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: new workAvailable. key=Ping
2019-03-12 21:20:52.643 [DEBUG] [ient.mqttv3.internal.CommsTokenStore] - paho3469211369349407: key=Ping
2019-03-12 21:20:52.643 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: wait for new work or for space in the inflight window
2019-03-12 21:20:52.644 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: ping response received. pingOutstanding: 0
2019-03-12 21:20:52.645 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: callback and notify for key=Ping
2019-03-12 21:20:52.645 [DEBUG] [o.client.mqttv3.internal.ClientState] - paho3469211369349407: received key=0 token=org.eclipse.paho.client.mqttv3.MqttToken@28351f3a message=PINGRESP msgId 0
2019-03-12 21:20:52.646 [DEBUG] [se.paho.client.mqttv3.internal.Token] - paho3469211369349407: >key=Ping response=PINGRESP msgId 0 excep=null
2019-03-12 21:20:52.646 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: notify spaceAvailable
2019-03-12 21:20:52.646 [DEBUG] [client.mqttv3.internal.CommsCallback] - paho3469211369349407: wait for workAvailable
2019-03-12 21:20:52.647 [DEBUG] [client.mqttv3.internal.CommsReceiver] - paho3469211369349407: network read message

One topic - from a Xiaomi windows contact - that I see in MQTT Spy is for example zigbee/contact.OG.emil.window with a content of {"contact":true,"linkquality":23,"battery":97,"voltage":2995}.

And my /etc/openhab2/things/mqtt.things file that should link to that and other topics is as follows:

Bridge mqtt:broker:mqtt [ host="127.0.0.1", secure=false ]
{
        Thing topic zigbee {
        Channels:
                Type contact : marleneWindow "Fenster Marlene" [ stateTopic="zigbee/contact.OG.marlene.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : emilWindow "Fenster Emil" [ stateTopic="zigbee/contact.OG.emil.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : egEzTerrasseWindow "Terrasse Esszimmer" [ stateTopic="zigbee/contact.EG.EZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egWzTerrasseWindow "Terrasse Wohnzimmer" [ stateTopic="zigbee/contact.EG.WZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egToiletteWindow "Gaestetoilette" [ stateTopic="zigbee/contact.EG.toilette.window", transformationPattern="JS:doorsensor.js" ]
                Type switch  : ogGangBewegung "Bewegungsmelder OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JS:motionsensor.js" ]
                Type number  : ogGangHelligkeit "Helligkeit Flur OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JSONPATH:$.illuminance" ]
        }
}

and /etc/openhab2/transform/doorsensor.js:

(function(x){
    var result = "";
    var json = JSON.parse(x);
    if (json.contact)
    {
        result="CLOSED";
    }
    else
    {
        result="OPEN";
    }
    return result;
})(input)

To rule out the JavaScript file as source of error I changed from

Type contact : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JS:doorsensor.js" ]

to

Type string : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JSONPATH:$.contact" ]

but no change.

(i.e. string instead of contact and jsonpath instead of JS) but no change.

Now here’s /etc/openhab2/items/zigbee.items:

Contact marleneWindow      <contact> {channel="mqtt:topic:zigbee:marleneWindow"}
Contact emilWindow         <contact> {channel="mqtt:topic:zigbee:emilWindow"}
String  elternWindow       "Fenster Schlafzimmer [%s]" <contact> {channel="mqtt:topic:zigbee:elternWindow"}
Contact egEzTerrasseWindow <contact> {channel="mqtt:topic:zigbee:egEzTerrasseWindow"}
Contact egWzTerrasseWindow <contact> {channel="mqtt:topic:zigbee:egWzTerrasseWindow"}
Contact egToiletteWindow   <contact> {channel="mqtt:topic:zigbee:egToiletteWindow"}
Switch  ogGangBewegung     <motion>  {channel="mqtt:topic:zigbee:ogGangBewegung"}
Number  ogGangHelligkeit   <sun>     {channel="mqtt:topic:zigbee:ogGangHelligkeit"}

(that’s the version with JsonPath instead of JavaScript for the “elternWindow” contact)

The items are included in a sitemap but I also see no reaction there to anything happening on the MQTT broker.

The openhab MQTT client “paho3469211369349407” and the Zigbee2MQTT client “mqttjs_*” are connected to mosquitto:

grep -C1 127.0.0.1 /var/log/mosquitto/mosquitto.log | perl -pe 's/^(\d+)/localtime($1)/e'
[...]
Sun Mar 10 23:34:50 2019: Client paho3469187044616725 disconnected.
Sun Mar 10 23:34:50 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:34:50 2019: New client connected from 127.0.0.1 as paho3469210201847499 (c1, k60).
Sun Mar 10 23:34:51 2019: Client paho3469210201847499 disconnected.
Sun Mar 10 23:34:51 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:34:51 2019: New client connected from 127.0.0.1 as paho3469211369349407 (c1, k60).
Sun Mar 10 23:35:44 2019: Socket error on client mqttjs_eeeaf806, disconnecting.
Sun Mar 10 23:35:47 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:35:47 2019: New client connected from 127.0.0.1 as mqttjs_84c4ce72 (c1, k60).
Sun Mar 10 23:38:35 2019: Socket error on client mqttjs_84c4ce72, disconnecting.
Sun Mar 10 23:38:37 2019: New connection from 127.0.0.1 on port 1883.
Sun Mar 10 23:38:37 2019: New client connected from 127.0.0.1 as mqttjs_aab1dc92 (c1, k60).

OH seems to be happily connected to mosquito for two days now.

What else can I do to debug?

Regards
Alex

Configuration looks good. The MQTT Broker and Generic Thing are online in PaperUI?

Can you add a simple String channel/item for debugging and then use mosquitto_pub -t your_topic -m test_message?

Hi David,

thanks for the quick reply!
Both the Broker thing as well as the Generic MQTT thing show online in PaperUI.

I created a simple mqtt-to-sitmap-string-chain:
mqtt.things:

Bridge mqtt:broker:mqtt [ host="127.0.0.1", secure=false ]
{
        Thing topic zigbee {
        Channels:
                Type contact : marleneWindow "Fenster Marlene" [ stateTopic="zigbee/contact.OG.marlene.window", transformationPattern="JS:doorsensor.js" ]
                Type contact : emilWindow "Fenster Emil" [ stateTopic="zigbee/contact.OG.emil.window", transformationPattern="JS:doorsensor.js" ]
                Type string  : elternWindow "Schlafzimmerfenster" [ stateTopic="zigbee/contact.OG.schlafzimmer.window", transformationPattern="JSONPATH:$.contact" ]
// JS:doorsensor.js" ]
                Type contact : egEzTerrasseWindow "Terrasse Esszimmer" [ stateTopic="zigbee/contact.EG.EZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egWzTerrasseWindow "Terrasse Wohnzimmer" [ stateTopic="zigbee/contact.EG.WZTerrasse.door", transformationPattern="JS:doorsensor.js" ]
                Type contact : egToiletteWindow "Gaestetoilette" [ stateTopic="zigbee/contact.EG.toilette.window", transformationPattern="JS:doorsensor.js" ]
                Type switch  : ogGangBewegung "Bewegungsmelder OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JS:motionsensor.js" ]
                Type number  : ogGangHelligkeit "Helligkeit Flur OG" [ stateTopic="zigbee/motion.OG.floor", transformationPattern="JSONPATH:$.illuminance" ]
        }

        Thing topic mytopic {
        Channels:
                Type string : mytest "MQTT Test" [ stateTopic="testtopic/blabla" ]
        }

}

I.e. I added a Generic MQTT thing “mytopic” which also shows as online in PaperUI.

mqtttest.items:

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:mytopic:mytest" }

mqtt.sitemap:

sitemap mqtt label="mqtt" {
Text item=mqtttest
}

The site map only shows "MQTT Test - " in the browser. No change when publishing some text to testtopic/blabla (tried both mosquitto_pub locally on the OH/mosquitto server as well as from the desktop using MQTT-Spy).

“Reboot tut gut” - I’ll try a restart of the complete OpenHAB VM, you never know…

Regards
ALex

Rebooted the box, increased log level on mosquitto and see Openhab subscribe to all configured topics:

1552431502: New client connected from 127.0.0.1 as paho1623397930189 (c1, k60).
1552431502: Sending CONNACK to paho1623397930189 (0, 0)
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     homeassistant/# (QoS 0)
1552431502: paho1623397930189 0 homeassistant/#
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     homie/+/$homie (QoS 0)
1552431502: paho1623397930189 0 homie/+/$homie
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.marlene.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.marlene.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/motion.OG.floor (QoS 0)
1552431502: paho1623397930189 0 zigbee/motion.OG.floor
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.EZTerrasse.door (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.EZTerrasse.door
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.schlafzimmer.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.schlafzimmer.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.OG.emil.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.OG.emil.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/motion.OG.floor (QoS 0)
1552431502: paho1623397930189 0 zigbee/motion.OG.floor
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.WZTerrasse.door (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.WZTerrasse.door
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     zigbee/contact.EG.toilette.window (QoS 0)
1552431502: paho1623397930189 0 zigbee/contact.EG.toilette.window
1552431502: Sending SUBACK to paho1623397930189
1552431502: Received SUBSCRIBE from paho1623397930189
1552431502:     testtopic/blabla (QoS 0)
1552431502: paho1623397930189 0 testtopic/blabla
1552431502: Sending SUBACK to paho1623397930189

Yet no reaction seen in the sitemap, in openhab.log, or in events.log for messages published to the relevant topics.

I’m running OpenHAB 2.4.0-1 from the official openhab debian repository and using 2.5.0-snapshot addons of the milight binding and the hueemulation (placed in /usr/share/openhab2/addons/):

root@vOpenHAB ~# apt show openhab2
Package: openhab2
Version: 2.4.0-1
Priority: optional
Section: java
Source: openhab2
Maintainer: https://community.openhab.org
Installed-Size: 83.5 MB
Depends: adduser
Suggests: zulu-8 | zulu-embedded-8 | oracle-java8-installer | openjdk-8-jdk-headless
Conflicts: openhab2-online, openhab2-offline
Homepage: www.openhab.org
Download-Size: 73.7 MB
APT-Manual-Installed: yes
APT-Sources: https://dl.bintray.com/openhab/apt-repo2 stable/main amd64 Packages
Description: openhab2
Linux installation package for openHAB 2.

N: There are 4 additional records. Please use the ‘-a’ switch to see them.
root@vOpenHAB ~# ll /usr/share/openhab2/addons/
total 164
drwxr-xr-x 2 openhab openhab 4096 Jan 28 23:05 ./
drwxr-xr-x 4 openhab openhab 4096 Jan 28 21:04 …/
-rw-r–r-- 1 openhab openhab 70 Dec 17 08:01 README
-rw-r–r-- 1 openhab openhab 77449 Jan 28 09:29 org.openhab.binding.milight-2.5.0-SNAPSHOT.jar
-rw-r–r-- 1 root root 76902 Jan 28 09:31 org.openhab.io.hueemulation-2.5.0-SNAPSHOT.jar

Not sure what other information I can provide.

Best regards
Alex

Try changing your items file to

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:broker:mytopic:blabla_whatever_your_needing_ to_do_like_power" }

No need to have mqttest, the items name, in the channel.

I don’t understand, the item name “mqtttest” is not in the channel. The names/strings I use map exactly to their use in the documentation.

Sorry, I messed up when replying, I think my eyes got crossed. Ignore the above about mqtttest in the channel.

What I meant to say was try adding the broker name in the channel after topic.

Example:

String mqtttest "MQTT Test [%s]" { channel="mqtt:topic:broker:mytopic:blabla_whatever_your_needing_ to_do_like_power" }

No need to have mqttest, the items name, in the channel.

Also note the 2.x binding might still have the issue where an OH restart is needed after making a file change. This was reported fixed in 2.5 so you may want to give OH a restart if you continue having a problem.

Hi all,

I do have the same issue, even with 2.5.0#1571. I do run a blank expert installation with only mqtt binding installed and try to get a text value updated to the sitemap.

From debug log I see that openhab gets the state updates

2019-04-16 13:25:57.908 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received bytes count=1

2019-04-16 13:25:57.909 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received bytes count=98

2019-04-16 13:25:57.909 [DEBUG] [mqttv3.internal.wire.MqttInputStream] - null: Automatic Reconnect Successful: PUBLISH qos:0 retained:false dup:false topic:"Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1" payload:[hex:4261636b206f6e207468652044616e636520466c utf8:"Back on the Dance Fl" length:39]

2019-04-16 13:25:57.909 [DEBUG] [o.client.mqttv3.internal.ClientState] - jarvis2: received key=0 message=PUBLISH qos:0 retained:false dup:false topic:"Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1" payload:[hex:4261636b206f6e207468652044616e636520466c utf8:"Back on the Dance Fl" length:39]

2019-04-16 13:25:57.909 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: new msg avail, notify workAvailable

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsReceiver] - jarvis2: network read message

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: call messageArrived key=0 topic=Hugo2C/Roon/Arbeitszimmer (HQ)/now_playing/one_line/line1

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: notify spaceAvailable

2019-04-16 13:25:57.910 [DEBUG] [client.mqttv3.internal.CommsCallback] - jarvis2: wait for workAvailable

however the attached item is not updated.

Both thing and broker are online (whole config done via paper ui)

image

… yet the item remains empty

Appreciate any ideas that could help me on this.

Might also be worth mentioning that adding a sonoff/tasmota device does work well without any trouble - at least for the power state.

Thanks
Tom

OK it seems that either the whitespace or the bracers are a problem for the binding. Removing those in the topics and it’s working like a charm.

You you please open a bug report about white-space and parentheses in mqtt topics?

Helps me keep track of this and to add a test case to the test suite.