Home Assistant auto discovery: Zigbee2mqtt restart makes things not responding

I have openHAB running on a dedicated Pi4 with an EnOcean and Zigbee stick directly connected to USB ports. As my installation is rather old and I have difficulties upgrading from 3.4.2 to current 4.x release, I want do to a fresh install on my Synology NAS and connect the USB sticks with a Pi Zero and ser2net.

My test environment now is the Pi Zero with an Sonoff ZBDongle-E and (for testing only) a Pi400 with the latest openHABian image running openHAB 4.2.0 Build #3945 and Zigbee2MQTT 1.35.3 commit 8b694432.

The dongle is detected and everything is working as expected. Pairing a simple OSRAM plug went smooth and the new thing was discovered by the Home Assistant auto discovery. I created an item for the switch channel and also the item works as expected. Switching directly in Z2M or openHAB always triggers the plug.

I am now working on handling connection losses. ser2net is restarting properly if I dis-/reconnect the dongle. Z2M also detects the lost connection and restarts so that the plug can be switched again in the Z2M dashboard.

openHAB with the Home Assistant auto discovery thing detects the offline/online state of Z2M and tries to reconfigure the thing afterwards “with slightly different config” and “making sure we re-create conflicting channels”. After that the connection of the channels is broken. There is no state update if the plug is switched on/off and triggering an item change in openHAB results in a warning message “channel … not supported”. The connection can be fixed by dis-/enabling the thing several times.

After digging deeper into this without any success, I created another “Generic MQTT thing” and a “switch” channel manually by adding all relevant topics. This thing has no problems at all and survives a restart of Z2M.

So the question I have is, do I really have to create all things and channels manually or is there anything I am missing with the Home Assistant auto discovery? I have something about 25 Zigbee devices here so a manual configuration would take some time.

I already played with nearly all Z2M settings I found in various posts here.

  • Legacy API
  • Legacy availability payload
  • MQTT output type
  • Home Assistant legacy triggers
  • Home Assistant legacy entity attributes
  • Retain (device setting)

I also monitored the broker with MQTTExplorer and found all configuration and corresponding state topics. Watching openHAB logs with frontail at least gave me some hints. Seems like there is a problem with subscription handling.

If I enable the thing and the item is functional afterwards, there are the following log entries.

==> /var/log/openhab/events.log <==
2024-02-28 11:43:27.257 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049' changed from UNINITIALIZED (DISABLED) to INITIALIZING
2024-02-28 11:43:27.277 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049' changed from INITIALIZING to UNKNOWN
2024-02-28 11:43:27.284 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049' changed from UNKNOWN to ONLINE
==> /var/log/openhab/openhab.log <==
2024-02-28 11:43:29.291 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/linkquality/
2024-02-28 11:43:29.291 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/last_seen/
2024-02-28 11:43:29.293 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/binary_sensor/0x7cb03eaa00ad2049/update_available/
2024-02-28 11:43:29.294 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/update/0x7cb03eaa00ad2049/update/
2024-02-28 11:43:29.294 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/switch/0x7cb03eaa00ad2049/switch/
2024-02-28 11:43:29.295 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/update_state/
2024-02-28 11:43:31.296 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:switch from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.297 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:sensor to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.298 [DEBUG] [al.handler.HomeAssistantThingHandler] - Received component homeassistant/binary_sensor/0x7cb03eaa00ad2049/update_available/ with slightly different config. Making sure we re-create conflicting channels...
2024-02-28 11:43:31.303 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:sensor from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.304 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:update to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.305 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:latestVersion to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.305 [DEBUG] [al.handler.HomeAssistantThingHandler] - Received component homeassistant/update/0x7cb03eaa00ad2049/update/ with slightly different config. Making sure we re-create conflicting channels...
2024-02-28 11:43:31.306 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:update from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.306 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:latestVersion from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.307 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:switch to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:43:31.307 [DEBUG] [al.handler.HomeAssistantThingHandler] - Received component homeassistant/switch/0x7cb03eaa00ad2049/switch/ with slightly different config. Making sure we re-create conflicting channels...
2024-02-28 11:44:13.648 [DEBUG] [qtt.generic.AbstractMQTTThingHandler] - Successfully published value OFF to topic zigbee2mqtt/0x7cb03eaa00ad2049/set
==> /var/log/openhab/events.log <==
2024-02-28 11:44:13.628 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'Steckdose_state' received command OFF
2024-02-28 11:44:13.631 [INFO ] [penhab.event.ItemStatePredictedEvent] - Item 'Steckdose_state' predicted to become OFF
2024-02-28 11:44:13.634 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Steckdose_state' changed from ON to OFF
2024-02-28 11:44:22.499 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'Steckdose_state' received command ON
==> /var/log/openhab/openhab.log <==
2024-02-28 11:44:22.559 [DEBUG] [qtt.generic.AbstractMQTTThingHandler] - Successfully published value ON to topic zigbee2mqtt/0x7cb03eaa00ad2049/set
==> /var/log/openhab/events.log <==
2024-02-28 11:44:22.545 [INFO ] [penhab.event.ItemStatePredictedEvent] - Item 'Steckdose_state' predicted to become ON
2024-02-28 11:44:22.547 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Steckdose_state' changed from OFF to ON
2024-02-28 11:44:22.648 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Steckdose_state' changed from ON to OFF
2024-02-28 11:44:22.674 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Steckdose_state' changed from OFF to ON

If I get a non functional state, there are these messages:

==> /var/log/openhab/openhab.log <==
2024-02-28 11:45:53.014 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/switch/0x7cb03eaa00ad2049/switch/
2024-02-28 11:45:53.015 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/linkquality/
2024-02-28 11:45:53.016 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/last_seen/
2024-02-28 11:45:53.020 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/binary_sensor/0x7cb03eaa00ad2049/update_available/
2024-02-28 11:45:53.024 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/sensor/0x7cb03eaa00ad2049/update_state/
2024-02-28 11:45:53.027 [TRACE] [ssistant.internal.DiscoverComponents] - Found HomeAssistant component homeassistant/update/0x7cb03eaa00ad2049/update/
==> /var/log/openhab/events.log <==
2024-02-28 11:45:52.807 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049' changed from OFFLINE to ONLINE
==> /var/log/openhab/openhab.log <==
2024-02-28 11:45:55.029 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:switch from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:45:55.031 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:sensor to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:45:55.033 [DEBUG] [al.handler.HomeAssistantThingHandler] - Received component homeassistant/binary_sensor/0x7cb03eaa00ad2049/update_available/ with slightly different config. Making sure we re-create conflicting channels...
2024-02-28 11:45:55.048 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Unsubscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:sensor from topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:45:55.049 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:update to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:45:55.051 [DEBUG] [ab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:homeassistant_zigbee2mqtt_5F0x7cb03eaa00ad2049:4b19daa721:zigbee2mqtt_5F0x7cb03eaa00ad2049:latestVersion to topic: zigbee2mqtt/0x7cb03eaa00ad2049
2024-02-28 11:45:55.052 [DEBUG] [al.handler.HomeAssistantThingHandler] - Received component homeassistant/update/0x7cb03eaa00ad2049/update/ with slightly different config. Making sure we re-create conflicting channels...

In the last log you can see, that the switch topic got unsubscribed und the subscription was not triggered although the HA component was discovered.
The configuration for this channel at topic “homeassistant/switch/0x7cb03eaa00ad2049/switch/config” looks good to me:

{
    "availability": [
        {
            "topic": "zigbee2mqtt/bridge/state"
        }
    ],
    "command_topic": "zigbee2mqtt/0x7cb03eaa00ad2049/set",
    "device": {
        "identifiers": [
            "zigbee2mqtt_0x7cb03eaa00ad2049"
        ],
        "manufacturer": "OSRAM",
        "model": "Smart+ plug (AB3257001NJ)",
        "name": "0x7cb03eaa00ad2049",
        "sw_version": "V1.05.09",
        "via_device": "zigbee2mqtt_bridge_0xe0798dfffebcdb63"
    },
    "name": null,
    "object_id": "0x7cb03eaa00ad2049",
    "origin": {
        "name": "Zigbee2MQTT",
        "sw": "1.35.3",
        "url": "https://www.zigbee2mqtt.io"
    },
    "payload_off": "OFF",
    "payload_on": "ON",
    "state_topic": "zigbee2mqtt/0x7cb03eaa00ad2049",
    "unique_id": "0x7cb03eaa00ad2049_switch_zigbee2mqtt",
    "value_template": "{{ value_json.state }}"
}

I am running out of ideas right now and would be very glad if someone with more knowledge can help me here.

@lutzbender76 did you find anything?

I’m currently experiencing the same issue with Hue Bulbs and changing there Settings in Z2M.
But are not getting any further than you where in your post.

I am quite busy right now and had no time for further research. I also can only post an issue but not provide a bug fix as i am not familiar with Java and the OpenHAB internal class structures.

So if I find some time for setting up my new installation, I will probably stick to the manual configuration and avoid using the auto discovery.

I don’t use the auto-discovery as the results are overall not that good.
I don’t think that manual creation is a big effort as it’s just a little bit of copy& paste that only takes some minutes even for a larger number of things. So my recommendation would be to stay manual.
Everything you need to know is well documented in the zigbee2mqtt-page of your device together with the “no more ugly transformations”-thread in the openhab community.

1 Like

@Larsen thank for the hint.
I also started this morning with moving to manual configuration with the attribute based mode.
I must say it looks way cleaner than the automated ids, but I have seen there is already an GitHub issue for these.
Maybe it would also make sense to open one for the auto discovery

@ccutrer is working on home assistant Mqtt discovery