EU Zigbee dongle

chris · July 31, 2020, 8:41am

I’ve also seen this recently with my larger network tests but I’ve not had a chance to properly characterise this yet. I am surprised though to see these as the EFR32 should have “plenty” of spare memory and I don’t think the collective settings that the binding uses are that demanding on memory.

giuseppeiannello · July 31, 2020, 10:43am

By the way, can those devices be used as HIGH_RAM concentrators, or must be used as LOW_RAM?

chris · July 31, 2020, 10:49am

It should work fine as a high ram concentrator. In general this would be the best configuration (especially in larger networks) - I don’t use this as the default as many people are using older devices (eg the EM357) and I’m not sure how well it works with the 357 due to memory.

giuseppeiannello · July 31, 2020, 11:13am

what’s the rule for deciding for high or low ram? I’m going to update the docs soon and wanted to clarify those configuration parameters too.

Is it a parameter that can be changed without re-initializing the network?

chris · July 31, 2020, 11:20am

I’m not sure there is one as such. If possible, using the high-ram concentrator is better as it will reduce network traffic, but it requires more memory on the coordinator. If the coordinator has the resources to do this, then it should be used.

With the Ember NCPs, there are lots of configuration parameters - they “all” take certain memory, so it’s not possible to increase everything. ZigBee 3 pushes up the number of keys required to be stored, so if every device requires a link key, we use a lot of memory and we can’t then bump up the source route table… It’s always a trade…

This is kind of why I mentioned earlier (I think in this thread) that I wanted to look at doing the source routing on the host as this can relax the memory requirements on the coordinator for storing the source routes, and allow this memory to be used for other things that can’t be offloaded to the host - such as link key management.

Without checking, I suspect that it can’t - memory allocation is done prior to starting the stack for the reasons partly eluded to above… The stack needs to know all these settings, as there is another setting for the number of packet buffers used, and the stack will allocate all unused memory to this when the stack starts.

giuseppeiannello · July 31, 2020, 11:33am

Sorry, asked the wrong question there - obviously some parameters are set on stack startup, so disabling/re-enabling the coordinator is necessary. I wanted to ask if it’s necessary to wipe/re-pair all the devices when changing that value

chris · July 31, 2020, 11:49am

No. All that changes is the way that the coordinator requests routes. Once it has been configured as a concentrator, it will periodically send the MTORR (Many To One Route Requests) to discover routes.

There is I suspect some optimisation I can do in the driver, and I’ll look at this over the coming week or three.

giuseppeiannello · July 31, 2020, 12:00pm

Well, apart from testing/breaking/documenting, any other way I can help?

chris · July 31, 2020, 12:11pm

Probably not - but testing/breaking would be great (so long as I can fix ), and of course some documenting would really be good (and much appreciated ).

giuseppeiannello · July 31, 2020, 12:20pm

Regarding that - the network seems to work, even after the error during initialization. Does it fall-back to the default value? What’s the practical effect of the value not being correctly set?

chris · July 31, 2020, 12:22pm

I think it just ignores the last setting. If you are looking in the binding logs, you should see 3 sets of configuration logging - first, it reads the initial settings, then it updates, then it reads again, so you should be able to confirm the final setting.

Elelabs · August 2, 2020, 12:11am

@chris
@giuseppeiannello

Hello!

We will have a look into the issue with pairing more than 15 devices.

In the meantime I’m happy to announce that we have released a public utility to update ours (and others EZSP compatible adapters though not tested) as well as EZSP v8 firmware for our products.
Feel free to check out and leave your comments/feedback.

chris · August 2, 2020, 8:55am

Thanks @Elelabs. I assume you’re using the standard Xmodem bootloader? If so, it would be great if you could release the file zipped with the metadata so that it can be booted directly in openHAB?

giuseppeiannello · August 2, 2020, 10:09am

Great news, thanks!

Trying the updated firmware in 3…2…1…

giuseppeiannello · August 2, 2020, 10:25am

pi@openhab:~/elelabs-zigbee-ezsp-utility $ python3 Elelabs_EzspFwUtility.py probe -p /dev/ttyAMA0
2020/08/02 12:16:42 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:16:42 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:16:42 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:16:42 Elelabs_EzspFwUtility:   EZSP v6
pi@openhab:~/elelabs-zigbee-ezsp-utility $ python3 Elelabs_EzspFwUtility.py probe -p /dev/ttyUSB1
2020/08/02 12:16:47 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:16:47 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:16:47 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:16:47 Elelabs_EzspFwUtility:   EZSP v6
pi@openhab:~/elelabs-zigbee-ezsp-utility $ python3 Elelabs_EzspFwUtility.py ele_update -v v8 -p /dev/ttyUSB1
2020/08/02 12:18:34 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:18:34 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:18:34 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:18:34 Elelabs_EzspFwUtility:   EZSP v6
2020/08/02 12:18:35 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:18:35 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:18:35 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:18:35 Elelabs_EzspFwUtility:   EZSP v6
2020/08/02 12:18:35 Elelabs_EzspFwUtility:   Launch in bootloader mode
2020/08/02 12:18:41 Elelabs_EzspFwUtility:   EZSP adapter in bootloader mode detected:
2020/08/02 12:18:41 Elelabs_EzspFwUtility:   Gecko Bootloader v1.A.0
2020/08/02 12:18:42 Elelabs_EzspFwUtility:   Successfully restarted into X-MODEM mode! Starting upload of the new formware... DO NOT INTERRUPT(!)
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
....
2020/08/02 12:19:19 Elelabs_EzspFwUtility:   Firmware upload complete
2020/08/02 12:19:19 Elelabs_EzspFwUtility:   Rebooting NCP...
2020/08/02 12:19:26 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:19:26 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:19:26 Elelabs_EzspFwUtility:   Firmware: 6.7.0-149
2020/08/02 12:19:26 Elelabs_EzspFwUtility:   EZSP v8
pi@openhab:~/elelabs-zigbee-ezsp-utility $ python3 Elelabs_EzspFwUtility.py ele_update -v v8 -p /dev/ttyAMA0
2020/08/02 12:19:45 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:19:45 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:19:45 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:19:45 Elelabs_EzspFwUtility:   EZSP v6
2020/08/02 12:19:46 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:19:46 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:19:46 Elelabs_EzspFwUtility:   Firmware: 6.0.3-64
2020/08/02 12:19:46 Elelabs_EzspFwUtility:   EZSP v6
2020/08/02 12:19:46 Elelabs_EzspFwUtility:   Launch in bootloader mode
2020/08/02 12:19:52 Elelabs_EzspFwUtility:   EZSP adapter in bootloader mode detected:
2020/08/02 12:19:52 Elelabs_EzspFwUtility:   Gecko Bootloader v1.A.0
2020/08/02 12:19:53 Elelabs_EzspFwUtility:   Successfully restarted into X-MODEM mode! Starting upload of the new formware... DO NOT INTERRUPT(!)
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
.....
....
2020/08/02 12:20:27 Elelabs_EzspFwUtility:   Firmware upload complete
2020/08/02 12:20:27 Elelabs_EzspFwUtility:   Rebooting NCP...
2020/08/02 12:20:34 Elelabs_EzspFwUtility:   Elelabs adapter detected:
2020/08/02 12:20:34 Elelabs_EzspFwUtility:   Adapter: ELR023
2020/08/02 12:20:34 Elelabs_EzspFwUtility:   Firmware: 6.7.0-149
2020/08/02 12:20:34 Elelabs_EzspFwUtility:   EZSP v8
pi@openhab:~/elelabs-zigbee-ezsp-utility $

So far, so good - the only (minor) issue is that both the shield and the USB stick are detected as ELR023.
Will not re-initialize the network and try joining all the devices.
Thanks @Elelabs!

giuseppeiannello · August 2, 2020, 10:46am

So, I’ve disabled the existing Coordinator, and added two more: one for the shield, one for the stick.

Both of them don’t come up, with the following logs:

12:38:37.034 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'zigbee:coordinator_ember:bb5d7d8e' changed from UNINITIALIZED (DISABLED) to INITIALIZING
12:38:37.047 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'zigbee:coordinator_ember:bb5d7d8e' changed from INITIALIZING to UNKNOWN
12:38:37.049 [INFO ] [arthome.event.FirmwareStatusInfoEvent] - Firmware status of thing zigbee:coordinator_ember:bb5d7d8e changed to UNKNOWN.
12:38:38.047 [DEBUG] [ee.transaction.ZigBeeTransactionQueue] - Default: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=2, interTransactionDelay=50, maxRetries=2]
12:38:38.050 [DEBUG] [ee.transaction.ZigBeeTransactionQueue] - Broadcast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=3, interTransactionDelay=1200, maxRetries=0]
12:38:38.053 [DEBUG] [ee.transaction.ZigBeeTransactionQueue] - Multicast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=3, interTransactionDelay=1200, maxRetries=0]
12:38:38.056 [DEBUG] [rtsystems.zigbee.ZigBeeNetworkManager] - ZigBeeNetworkManager initialize: networkState=UNINITIALISED
12:38:38.059 [DEBUG] [rtsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to INITIALISING
12:38:38.061 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Initialize with protocol ASH2.
12:38:40.388 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - Ember: Link State change to true, initialised=false, networkStateUp=false
12:38:40.391 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - Ember: Link State change to true ignored.
12:38:40.419 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - RX EZSP: EzspGetLibraryStatusResponse [networkId=0, status=UNKNOWN]
12:38:48.101 [DEBUG] [tsystems.zigbee.dongle.ember.EmberNcp] - No response from ezspVersion command
12:38:48.104 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Version returned null. ASH/EZSP not initialised.
12:38:48.106 [DEBUG] [rtsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to OFFLINE
12:38:48.108 [WARN ] [ommon.WrappedScheduledExecutorService] - Scheduled runnable ended with an exception: 
java.lang.NullPointerException: null
	at org.openhab.binding.zigbee.handler.ZigBeeCoordinatorHandler.initialiseZigBee(ZigBeeCoordinatorHandler.java:425) ~[?:?]
	at org.openhab.binding.zigbee.handler.ZigBeeCoordinatorHandler.lambda$2(ZigBeeCoordinatorHandler.java:543) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_212]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
12:38:48.110 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'zigbee:coordinator_ember:bb5d7d8e' changed from UNKNOWN to OFFLINE (COMMUNICATION_ERROR)
12:38:49.109 [INFO ] [gbee.handler.ZigBeeCoordinatorHandler] - ZigBee dongle inactivity timer. Reinitializing ZigBee
12:38:49.112 [DEBUG] [rtsystems.zigbee.ZigBeeNetworkManager] - ZigBeeNetworkManager shutdown: networkState=OFFLINE
12:38:49.114 [DEBUG] [database.ZigBeeNetworkDatabaseManager] - Data store: Deferred Write Time set to 0ms
12:38:49.116 [DEBUG] [database.ZigBeeNetworkDatabaseManager] - Data store: Deferred Write Time set less than Max Deferred Write Time
12:38:49.118 [DEBUG] [database.ZigBeeNetworkDatabaseManager] - Data store: Max Deferred Write Time set to 0ms
12:38:49.121 [DEBUG] [rtsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to SHUTDOWN
12:38:49.123 [DEBUG] [database.ZigBeeNetworkDatabaseManager] - Data store: Shutdown
12:38:49.125 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Shutdown
12:38:49.127 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - Ember: Link State change to false, initialised=false, networkStateUp=false
12:38:49.129 [DEBUG] [.zigbee.dongle.ember.ZigBeeDongleEzsp] - Ember: Link State change to false ignored.
12:38:49.137 [DEBUG] [.transaction.ZigBeeTransactionManager] - Transaction Manager: Shutdown
12:38:49.139 [DEBUG] [ee.transaction.ZigBeeTransactionQueue] - Broadcast: Queue shutdown
12:38:49.141 [DEBUG] [ee.transaction.ZigBeeTransactionQueue] - Multicast: Queue shutdown
12:38:49.144 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'zigbee:coordinator_ember:bb5d7d8e' changed from OFFLINE (COMMUNICATION_ERROR) to UNKNOWN

Oddly enough, if I disable the two “new” Things and I enable the “old” one, the networks comes up just fine.

chris · August 2, 2020, 10:50am

I’d need to see a full debug log to see why it’s not coming up.

giuseppeiannello · August 2, 2020, 10:52am

Hm, it’s already at DEBUG - what’s missing?

EDIT:

openhab> log:Get
ROOT                                                               │ INFO
com.zsmartsystems.zigbee                                           │ DEBUG
com.zsmartsystems.zigbee.dongle.ember.internal.ash.AshFrameHandler │ INFO

Bruce_Osborne · August 2, 2020, 10:56am

How about a file with the whole debug log from the beginning of binding startup? Filtered debug logs are useless for troubleshooting.

chris · August 2, 2020, 11:01am

I can see there’s debug there, but I expected to see a lot more than this (ie I see no communication from the device). However, given your debug config, I now suspect there’s a basic communication problem. Please can you enable ASH debug as well.

My guess is that there is no communication with the device.