Zigbee controller keeps initializing forever

[tonus (Ton)]

I use OH 4.0.1 with the Zigbee addon using Openhabian on Home Assistant Yellow hardware (no typing error, see other post for some background), so essentially this is a Raspberry Pi 4 with 2 GB and 32 GB eMMD disk.

The Zigbee module is the built-in Silicon Labs MGM210P Mighty Gecko Module

I assume this is on /dev/ttyAMA0 (ACM0 is my Z-wave stick for sure) due to:

$ dmesg | grep -i tty
[ 0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=0 snd_bcm2835.enable_headphones=0 snd_bcm2835.enable_hdmi=1 bcm2708_fb.fbwidth=0 bcm2708_fb.fbheight=0 bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 smsc95xx.macaddr=D8:3A:DD:3C:17:61 vc_mem.mem_base=0x3f000000 vc_mem.mem_size=0x3f600000 console=ttyS0,115200 console=tty1 root=PARTUUID=901fb067-02 rootfstype=ext4 fsck.repair=yes rootwait
[ 0.000902] printk: console [tty1] enabled
[ 1.810730] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 34, base_baud = 0) is a PL011 rev2
[ 3.315607] systemd[1]: Created slice system-getty.slice.
[77003.309580] cdc_acm 1-1.2:1.0: ttyACM0: USB ACM device

Furthermore, I used all defaults:

• Hardware CTS/RTS)
• Baud rate 115200 (I might have chosen that manually, not sure)
• Ember NCP Configuration : Boost
• Transmit Power : normal
• Child Aging Timeout : 1day
• Network size: medium (25), I have an sensor and one actuator, that will do

In the UI, I get

• Channel 11
• an PAN ID
• an Extened PAN ID
• a network security key
• a Link secuirty key
• Concentrator Type : Low Ram

So I assume this works to some degree. However, the Zigbee Ember Coordinator keeps on Initializing for ever. I’ve put trace on, and get a lot of Zigbee related messages. Here some:

2024-01-19 15:32:39.513 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.516 [DEBUG] [al.ZigBeeDefaultDiscoveryParticipant] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.internal.ZigBeeDefaultDiscoveryParticipant] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.518 [DEBUG] [.zigbee.handler.ZigBeeIsAliveTracker] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.handler.ZigBeeIsAliveTracker] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.521 [DEBUG] [zigbee.internal.ZigBeeHandlerFactory] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.internal.ZigBeeHandlerFactory] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.524 [DEBUG] [er.ZigBeeChannelConverterFactoryImpl] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.internal.converter.ZigBeeChannelConverterFactoryImpl] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.527 [DEBUG] [igBeeDefaultChannelConverterProvider] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.internal.converter.ZigBeeDefaultChannelConverterProvider] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.530 [DEBUG] [micWarningCommandDescriptionProvider] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.internal.converter.warningdevice.DynamicWarningCommandDescriptionProvider] : BundleComponentActivator : ComponentHolder created.
2024-01-19 15:32:39.535 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Dependency Manager created ZigBeeChannelConverterFactoryinterface=org.openhab.binding.zigbee.converter.ZigBeeChannelConverterFactory, filter=null, policy=static, cardinality=1..1, bind=setZigBeeChannelConverterFactory, unbind=unsetZigBeeChannelConverterFactory, updated=null, field=null, field-option=null, collection-type=null, parameter=null
2024-01-19 15:32:39.537 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Dependency Manager created ZigBeeCoordinatorHandlerinterface=org.openhab.binding.zigbee.handler.ZigBeeCoordinatorHandler, filter=null, policy=dynamic, cardinality=0..n, bind=addZigBeeCoordinatorHandler, unbind=removeZigBeeCoordinatorHandler, updated=null, field=null, field-option=null, collection-type=null, parameter=null
2024-01-19 15:32:39.538 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Dependency Manager created ZigBeeDiscoveryParticipantinterface=org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryParticipant, filter=null, policy=dynamic, cardinality=0..n, bind=addZigBeeDiscoveryParticipant, unbind=removeZigBeeDiscoveryParticipant, updated=null, field=null, field-option=null, collection-type=null, parameter=null
2024-01-19 15:32:39.538 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Dependency Manager created osgi.ds.satisfying.conditioninterface=org.osgi.service.condition.Condition, filter=(osgi.condition.id=true), policy=dynamic, cardinality=1..1, bind=null, unbind=null, updated=null, field=null, field-option=null, collection-type=null, parameter=null
2024-01-19 15:32:39.540 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Component created: DS=DS13, implementation=org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService, immediate=true, default-enabled=true, factory=null, configuration-policy=optional, activate=activate, deactivate=deactivate, modified=modified configuration-pid=[discovery.zigbee]
2024-01-19 15:32:39.541 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Component Services: scope=singleton, services=[org.openhab.core.config.discovery.DiscoveryService]
2024-01-19 15:32:39.542 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Component Properties: {osgi.ds.satisfying.condition.target=(osgi.condition.id=true)}
2024-01-19 15:32:39.542 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Querying state disabled
2024-01-19 15:32:39.543 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Querying state disabled
2024-01-19 15:32:39.544 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Component can not be activated since it is in state disabled
2024-01-19 15:32:39.545 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService] : Querying state disabled
2024-01-19 15:32:39.546 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService(370)] : Updating target filters
2024-01-19 15:32:39.547 [DEBUG] [bee.discovery.ZigBeeDiscoveryService] - bundle org.openhab.binding.zigbee:4.1.0 (301)[org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService(370)] : No change in target property for dependency ZigBeeChannelConverterFactory: currently registered: false

or, if i skip a lot of the less usefull left columns:

BundleComponentActivator : ComponentHolder created.
DiscoveryParticipant] : BundleComponentActivator : ComponentHolder created.
BundleComponentActivator : ComponentHolder created.
BundleComponentActivator : ComponentHolder created.
ConverterFactoryImpl] : BundleComponentActivator : ComponentHolder created.
ChannelConverterProvider] : BundleComponentActivator : ComponentHolder created.
DynamicWarningCommandDescriptionProvider] : BundleComponentActivator : ComponentHolder created.
Dependency Manager created ZigBeeChannelConverterFactoryinterface=org.openhab.binding.zigbee.converter.ZigBeeChannelConverterFactory, filter=null, policy=static, cardinality=1..1, bind=setZigBeeChannelConverterFactory, unbind=unsetZigBeeChannelConverterFactory, updated=null, field=null, field-option=null, collection-type=null, parameter=null
Dependency Manager created ZigBeeCoordinatorHandlerinterface=org.openhab.binding.zigbee.handler.ZigBeeCoordinatorHandler, filter=null, policy=dynamic, cardinality=0..n, bind=addZigBeeCoordinatorHandler, unbind=removeZigBeeCoordinatorHandler, updated=null, field=null, field-option=null, collection-type=null, parameter=null
Dependency Manager created ZigBeeDiscoveryParticipantinterface=org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryParticipant, filter=null, policy=dynamic, cardinality=0..n, bind=addZigBeeDiscoveryParticipant, unbind=removeZigBeeDiscoveryParticipant, updated=null, field=null, field-option=null, collection-type=null, parameter=null
Dependency Manager created osgi.ds.satisfying.conditioninterface=org.osgi.service.condition.Condition, filter=(osgi.condition.id=true), policy=dynamic, cardinality=1..1, bind=null, unbind=null, updated=null, field=null, field-option=null, collection-type=null, parameter=null
Component created: DS=DS13, implementation=org.openhab.binding.zigbee.discovery.ZigBeeDiscoveryService, immediate=true, default-enabled=true, factory=null, configuration-policy=optional, activate=activate, deactivate=deactivate, modified=modified configuration-pid=[discovery.zigbee]
Component Services: scope=singleton, services=[org.openhab.core.config.discovery.DiscoveryService]
Component Properties: {osgi.ds.satisfying.condition.target=(osgi.condition.id=true)}`
Querying state disabled
Querying state disabled
Component can not be activated since it is in state disabled
Querying state disabled
Updating target filters
No change in target property for dependency ZigBeeChannelConverterFactory: currently registered: false

So, where to go from here?
Would this ever work ?
I have quite some Zwave experience, but no Zigbee. I’d appreciate any help!

[rpwong (Russ)]

I’d suggest trying Zigbee2mqtt to see if you have better luck. It supports a wider range of adapters and devices. You can install it from openhabian-config.

Supported Adapters | Zigbee2MQTT

[chris (Chris Jackson)]

@rpwong

It supports a wider range of adapters and devices.

It’s interesting that you think this is an adaptor compatibility issue? Can you explain why?

The adaptor is using a Silabs interface which is generally well supported in the binding so it seems strange to me to suggest this is the issue.

From the log it’s hard to determine what’s going on - it looks like the binding is not even starting due to some services not loading or being disabled. To me it looks more fundamental than the adaptor not being supported.

[rpwong (Russ)]

Ah, sorry Chris. I thought that I’d read before that Zigbee2mqtt has more adapter support, but it’s admittedly been awhile since I looked at the documentation. I’m clearly mistaken.

I was really just thinking that it’s worth trying Zigbee2mqtt for comparison purposes (to see if the adapter can be reached), so I’ll delete that sentence from my earlier post.

[tonus (Ton)]

@chris

The adaptor is using a Silabs interface which is generally well supported in the binding so it seems strange to me to suggest this is the issue.

Thanks. That is important information to me.

@chris

From the log it’s hard to determine what’s going on - it looks like the binding is not even starting due to some services not loading or being disabled. To me it looks more fundamental than the adaptor not being supported.

Good point! That is something I must certainly look at.
Does anyone has any clues how to go on?
How to talk to the Silabs serial port low level, would that be possible? Using minicom or so ? Just to see whether there is any signal coming back?

I’d appreciate any help.

[tonus (Ton)]

Using my Zstick’s serial port, I can see something is going on:

$ cat /dev/ttyACM0
q?v q?v q?v q?v^C
$

However, on the port where I assume the Gecko interface should be, it is al silent, there is no activity at all. It returns immediately to the prompt:

$ sudo cat /dev/ttyAMA0
$

So @Chris is right, the issue is much more fundamental, it is deep down in the OS.

Could perhaps be a firmware issue…

[tonus (Ton)]

I got the Zigbee module working! And the drive LED now also works.

It appeared to be relatively simple. This is how I did it:

• I copied the file /boot/bcm2711-rpi-cm4-ha-yellow.dtb from the original yellow boot partition to the new OpenHab boot partition. This file can be see as a low level “driver”, necessary for telling the hardware specific device tree to the Linux kernel (my simple explanation). Of course, you have to save this file before you put Openhabian on your Yellow.
• Added ttyAMA2 console to the /boot/cmdline.txt. Not really sure whether this is a must, but it appeared to be so in the HA version of cmdline.txt :`

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 console=ttyAMA2,115200n8 root=PARTUUID=901fb067-04 rootfstype=ext4 fsck.repair=yes rootwait

(Note: do not fully copy this line as your PARTUUID is different for each installation)

• Use the original Yellow /boot/config.txt file as a template for the new one. Resulting in :

# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

# HAOS - don't change it!
disable_splash=1
#kernel=u-boot.bin
#arm_64bit=1

# Enable primary UART by default
enable_uart=1

# Free up full-fledged PL011 UART for J11
dtoverlay=miniuart-bt

# No HDMI on Yellow, but we can't set to 16 since we need the full firmware
# for codecs
gpu_mem=32

# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Uncomment this to enable the lirc-rpi module
#dtoverlay=lirc-rpi

# Uncomment this to enable GPIO support for RPI-RF-MOD/HM-MOD-RPI-PCB
#dtparam=i2c_arm=on
#dtoverlay=rpi-rf-mod

# Additional overlays and parameters are documented /boot/overlays/README

[all]
device_tree=bcm2711-rpi-cm4-ha-yellow.dtb

Note that I commented out the 64 bit line from HA, as openhabian is recommended to run in 32 bit mode.

• Then reboot.
• Added a Ember Coordinator type Zigbee controller in OpenHab.

In my case, the Zigbee controller came up at /dev/AMA1. This is seen in dmesg output as:

[ 3.058691] fe201800.serial: ttyAMA1 at MMIO 0xfe201800 (irq = 33, base_baud = 0) is a PL011 rev2

And in the openhab.log, once you have added it:

2024-01-23 17:50:31.577 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - Initializing ZigBee Ember serial bridge handler.
2024-01-23 17:50:31.579 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - ZigBee Ember Coordinator opening Port:‘/dev/ttyAMA1’ PAN:3365, EPAN:E11E48F166F20FF4, Channel:11
2024-01-23 17:50:31.580 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - Ember end device poll timeout set to (169 * 2^9) = 86528 seconds
2024-01-23 17:50:31.581 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Scheduling ZigBee start
2024-01-23 17:50:32.582 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - ZigBee network starting
2024-01-23 17:50:32.583 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Initialising ZigBee coordinator
17:50:32.585 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Connecting to serial port [/dev/ttyAMA1] at 115200 baud, flow control FLOWCONTROL_OUT_RTSCTS.
17:50:32.585 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - null: networkStateUpdated called with state=INITIALISING
17:50:32.610 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Serial port [/dev/ttyAMA1] is initialized.

Afterwards, I found a link that described more or less what I did, but giving more explanation about it in a HA context: Pi OS (instead of HA OS) on Yellow? - #16 by JJT - Installation - Home Assistant Community

My Zigbee devices were easily added, in the usual way, and run fine for a couple of days now. Would this be a long term stable solution? I don’t know.
Perhaps anyone can comment on that ?

[Larsen (Locke)]

@rpwong

Ah, sorry Chris. I thought that I’d read before that Zigbee2mqtt has more adapter support, but it’s admittedly been awhile since I looked at the documentation. I’m clearly mistaken.

I would say you were not so wrong:

• Zigbee2mqtt supports TI chipsets (CC2652…) and Conbee. Silicon Labs (ember) is eperimental: Supported Adapters | Zigbee2MQTT

• Zigbee-binding mainly supports the chipsets from Silicon Labs (ember) and XBee.

But I would recommend not making the choice of zigbee connection dependent on the zigbee coordinator you have lying around. They’re cheap anyway, so buy the one you need.

My recommendation would be:
If your zigbee network contains devices with loose zigbee implementations (the ones you buy cheap from ali…) then definitely go for zigbee2mqtt. Thousands of devices are supported cause a community (and Koen) is trying to make almost any device work. And you get better configurability and transparency of your network.

If your network only contains well supported devices (proper zigbee implementations) then you can avoid the additional configuration and go for the zigbee binding.