Zigbee binding

[chris (Chris Jackson)]

I suspect that you need to update to a newer version of OH as the TP has changed.

[5iver (Scott Rushworth)]

@chris, after deleting the coordinator and recreating it with my old settings, I’m having trouble getting it back online. It gets to here and then stops…

2018-09-12 03:31:40.915 [DEBUG] [com.zsmartsystems.zigbee.dongle.ember.EmberNcp] - EzspGetCurrentSecurityStateResponse [status=EMBER_NOT_JOINED, state=EmberCurrentSecurityState [bitmask=[], trustCenterLongAddress=0BCC000000030000]]
2018-09-12 03:31:40.923 [DEBUG] [com.zsmartsystems.zigbee.dongle.ember.EmberNcp] - Error during retrieval of security parameters: EzspGetCurrentSecurityStateResponse [status=EMBER_NOT_JOINED, state=EmberCurrentSecurityState [bitmask=[], trustCenterLongAddress=0BCC000000030000]]
2018-09-12 03:31:40.939 [DEBUG] [com.zsmartsystems.zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP dongle initialize done: Initialised true
2018-09-12 03:31:40.951 [DEBUG] [org.openhab.binding.zigbee.internal.ZigBeeNetworkStateSerializerImpl] - Loading ZigBee network state: Start.
2018-09-12 03:31:41.058 [DEBUG] [com.zsmartsystems.zigbee.ZigBeeNetworkManager] - 000D6F000AFF6FC8: Node 0 added to the network
2018-09-12 03:31:41.058 [DEBUG] [org.openhab.binding.zigbee.internal.ZigBeeNetworkStateSerializerImpl] - Loading ZigBee network state: Done.
2018-09-12 03:31:41.059 [DEBUG] [com.zsmartsystems.zigbee.internal.ZigBeeNetworkDiscoverer] - Network discovery task starting

I’ve updated the binding and restarted OH. I’m not sure what else to try. Any ideas?

[chris (Chris Jackson)]

@5iver

Any ideas?

Yes - I think it is related to this -:

Set the default link key

I don’t have a suggestion to solve it though until this is merged.

[Kergilo (Antonio Jose)]

@chris

In future can you provide times that these issues are occurring? If I look at this log, it runs for many hours, and I’m seeing messages with the contact state, but I can’t tell if these messages are happening when you are not seeing them in the UI. This means we can’t decide if this is a UI or system configuration issue, or a binding issue.

You are right, my fault. Sorry.

@chris

edit: Note that you can also display the logs using the log viewer on my website . This displays the low level packets that the binding is receiving and sending - it has some limitations, but in general it can show you what the binding receives, so if the UI isn’t updating, but you see the STATE UPDATE messages, then it means the binding is receiving the messages and passing them to OH.

I will. Thanks.

Do you know what that error means?

[chris (Chris Jackson)]

No, sorry, I don’t know what this error is. Hi I me it looks like a low level error - writing to disk, or possibly an issue with writing to some port, but it’s clearly a very low level error as the error is in C code.

[5iver (Scott Rushworth)]

@chris

I don’t have a suggestion to solve it though until this is merged.

I downgraded to a binding with 1.0.14 libraries, then upgraded again. Seems to have gotten me running again.

[chris (Chris Jackson)]

@5iver

I downgraded to a binding with 1.0.14 libraries, then upgraded again. Seems to have gotten me running again.

Nice workaround - thanks. The issue is only when the dongle is initialised - once it’s initialised, the keys are set again.

Hopefully I can get some of these PRs merged - unfortunately as the DT people are all on holiday, it’s a little difficult!

[chris (Chris Jackson)]

@puzzle-star what are the sensors you have that are reporting as the lumi sensor ht? I have a couple of different Xiaomi devices here, but they don’t seem to report as this. The latest ones I have report as lumi weather and have temp/humidity and pressure, and these devices correctly report their clusters…

I’ve created an update mostly based on your suggested change, but would like to test it if possible…

[puzzle-star (Pedro Garcia)]

@chris

@puzzle-star what are the sensors you have that are reporting as the lumi sensor ht? I have a couple of different Xiaomi devices here, but they don’t seem to report as this. The latest ones I have report as lumi weather and have temp/humidity and pressure, and these devices correctly report their clusters…

The lumi.weather ones (the square ones) are OK, as you say (I also have some of these). The lumi.sensor_ht I am speaking of are the round ones, and they do not report pressure:

The aqara switches (the battery powered ones) also need this type of hack

Pedro

[puzzle-star (Pedro Garcia)]

@chris

I’ve created an update mostly based on your suggested change, but would like to test it if possible…

I was also trying to keep it simple for non-tech users, but the support of “non-standard” devices in the end may either:

• Not being maintainable in the binding itself (how often and how many devices can be added?)
• If users need to go to Snapshot version, I think it would not be targetting at non-tech users either
• “Tech” users may need a way to override things without having to tweak with the binding

I would suggest to add (I can code this if needed, or indeed adapt my already working code), if possible:

• A configuration parameter on the binding (sort of text box) to define such overrides (so that not-too-techie users could just put the definitions found on the forums there, but in the end this is OH,…)
• A configuration parameter on the standard device thing to override endpoints (something like the zigbee_endpoints=“1:1026,1029”)
• A configuration parameter on the standard device thing to prevent discovery from the binding (something like the zigbee_discovery=“false”)

@chris

I’m not so keen on this. It means we go back to the OH1 style of defining devices and require users to configure everything manually. For starters, this goes against OH2 principals which allows everything to be defined through the UI, and secondly it means that people need to understand more about ZigBee, clusters etc and it’s much less “plug and play”.

This does not force to define the things à la OH1 style. These configurations parameters work the same in the “.things” files as in the UI config. I am just using the files for convenience

I just think that trying to keep a non-standard device inventory inside the binding may end up not fitting the “one-for-all” purpose, specially once the binding is stable and does not need so much “maintenance”

Pedro

[chris (Chris Jackson)]

@puzzle-star

• A configuration parameter on the binding (sort of text box) to define such overrides (so that not-too-techie users could just put the definitions found on the forums there, but in the end this is OH,…)

I’m not really keen on this. I much prefer to use the standard XML system - it ensures that people can use this without being technical. You refer to people downloading definitions off the forum - but please remember that most users of OH don’t use the forum, and many users of the binding are not using OH even. In my opinion, support should be added to the binding to support new devices.

No matter what, the XML is definitely required as this is a basic principal for this binding - it must be able to be used by people who are completely non-technical, with no configuration.

@puzzle-star

• A configuration parameter on the standard device thing to prevent discovery from the binding (something like the zigbee_discovery=“false” )

chris:

What is the purpose of this, and what is it meant to actually stop? The binding really needs to do discovery or it doesn’t add devices into the binding

@puzzle-star

I just think that trying to keep a non-standard device inventory inside the binding may end up not fitting the “one-for-all” purpose,

I don’t know what this means? If we add support for a device, then it’s clearly available for everyone and no further configuration is required.

I’ve created an initial PR to add this support to the XML. Somewhere I have one of these old Xiaomi temperature sensors, but I can’t find it to test with. If you like, please feel free to test it (it needs to discovery information added to the PR for testing .

Add ability to add endpoints and clusters to a device via the XML

[puzzle-star (Pedro Garcia)]

Hi @chris,

@chris

I much prefer to use the standard XML system - it ensures that people can use this without being technical.

But is there (or could be) a way for a user to place these xmls out of the jar, so that a device can be supported even if not part of the official distribution?

@chris

many users of the binding are not using OH even.

Now I am lost. Isn’t the binding specific for openhab? (not speaking of the zsmartsystems part, just about the org.openhab.binding.zigbee part)

@chris

puzzle-star:

• A configuration parameter on the standard device thing to prevent discovery from the binding (something like the zigbee_discovery=“false” )

chris:

What is the purpose of this, and what is it meant to actually stop? The binding really needs to do discovery or it doesn’t add devices into the binding

doNodeInitialization(): (being configStatic below the check on the config parameter)

         // Check if discovery is complete and we know all the services the node supports
-        if (!node.isDiscovered()) {
+        if (!configStatic && !node.isDiscovered()) {

Some devices (I agree, badly implemented ones, but they do exist anyway), do not respond too well (or at all) to discovery. The result of this is that, as the node.isDiscovered() check returns true only when the endpoints and node descriptor have been initialized, if for some reason this does not work (or does not work on a timely manner), such devices get into the system, but never go futher the isDiscovered check in the binding:

• Sleepy devices

• Devices not responding to some commands during discovery. Example: the discovery mecanism puts in a queue the discovery tasks. I have seen in the discovery task list the ROUTE discovery before the NODEDESCRIPTOR one (not sure it should be OK), so for END DEVICES it will stay forever retrying the ROUTE discovery… Maybe the issue is on the discovery process, and I should look at there instead

• Devices not responding at all (or hard to make to respond) to NodeDescriptor or EndPoints request (i.e. Aqara battery powered switches) due to being too “sleepy”

@chris

The binding really needs to do discovery or it doesn’t add devices into the binding

It does. I guarrantee (of course, with the hack above)… Again speaking of the org.openhab.binding.zigbee part, not of the zsmartystems part, and not speaking of disabling the discovery service or something like that: just the check above.

@chris

If we add support for a device, then it’s clearly available for everyone and no further configuration is required.

Probably I did not explain it correctly. What I mean is: yes, if support for a device is added, it will work. But how often will you be able to add new devices? With which frequency there is a new release ? And in three years, how often there will be a new release?

I mean, tying the release of newly supported devices that only need an xml to work, to the release cycle of the binding + OH, may not be sustainable over time, and probably some users would like to be able to add a new supported device by just dropping a xml somewhere, if not possible to configure it on the binding itself, without having to go to snapshot version.

@chris

I’ve created an initial PR to add this support to the XML. Somewhere I have one of these old Xiaomi temperature sensors, but I can’t find it to test with. If you like, please feel free to test it (it needs to discovery information added to the PR for testing

Will give it a try

Pedro

[chris (Chris Jackson)]

@puzzle-star

But is there (or could be) a way for a user to place these xmls out of the jar, so that a device can be supported even if not part of the official distribution?

This isn’t currently possible in ESH.

@puzzle-star

Now I am lost. Isn’t the binding specific for openhab? (not speaking of the zsmartsystems part, just about the org.openhab.binding.zigbee part)

There are many (most) users of OH that do not regularly access the forum, and there are other systems that use OH bindings.

@puzzle-star

But how often will you be able to add new devices?

As often as is needed. In ZWave, it’s normally twice per week, but if someone creates a PR to add a device, it can easily be added.

@puzzle-star

And in three years, how often there will be a new release?

The same - as often as people add the data to the binding. It’s controlled this way - if you’re getting at the fact that in 3 years no-one will be supporting this, then the binding will likely not work with new devices anyway. I simply don’t think this is an issue. If people aren’t working out the device information and publishing it somewhere, then it’s all irrelevant isn’t it?

@puzzle-star

I mean, tying the release of newly supported devices that only need an xml to work, to the release cycle of the binding + OH, may not be sustainable over time

I don’t understand. You’ve said that there will not be many such devices. I would like to think that in future, more devices will be more compliant. In any case, as I said above, for the past 3 years I’ve personally been doing updates to the ZWave binding a few times a week to add devices - it’s exactly the same.

This ensures the data and device support is available for everyone - not just those with the knowledge to work out how to add devices for themselves.

[puzzle-star (Pedro Garcia)]

@chris

This isn’t currently possible in ESH.

Then I understand why you do not like the idea

@chris

You’ve said that there will not be many such devices. I would like to think that in future, more devices will be more compliant. In any case, as I said above, for the past 3 years I’ve personally been doing updates to the ZWave binding a few times a week to add devices - it’s exactly the same.

This ensures the data and device support is available for everyone - not just those with the knowledge to work out how to add devices for themselves.

Fair enough

Will look anyway deeper into the discovery mechanism in the ZigBeeNode, as clicking the Xiami buttons for 3 minutes twice per second while looking at the logs till it goes through discovery is not very “user firendly…”

I think this could be optimized a bit, but will check if I can propose something “clean” and faster for discovery in general

[chris (Chris Jackson)]

@puzzle-star

I think this could be optimized a bit, but will check if I can propose something “clean” and faster for discovery in general

Thanks - I’m certainly open to ideas, and appreciate your input and thoughts .

[nibi79 (NB)]

I try to to start the binding with eclipse smarthome, windows 10 and CC2531 (COM4). When I set the port to ‘COM4’ in PaperUI I get the error message that the port doesn’t exist.
That’s the log when I launch openHAB_Runtime:

2018-09-14 18:12:21.886 [DEBUG] [b.z.h.ZigBeeCoordinatorHandler:435  ] - ZigBee network starting
2018-09-14 18:12:21.887 [DEBUG] [b.z.h.ZigBeeCoordinatorHandler:331  ] - Initialising ZigBee coordinator
2018-09-14 18:12:21.931 [DEBUG] [b.z.h.ZigBeeCoordinatorHandler:347  ] - Key initialise 5EF585BC0FC6AAB23371C521D6C5F136
2018-09-14 18:12:21.934 [DEBUG] [o.b.z.handler.ZigBeeSerialPort:122  ] - Connecting to serial port [COM4] at 115200 baud, flow control FLOWCONTROL_OUT_RTSCTS.
2018-09-14 18:12:21.961 [ERROR] [o.b.z.handler.ZigBeeSerialPort:151  ] - Serial Error: Port COM4 does not exist.
2018-09-14 18:12:21.962 [ERROR] [z.z.d.c.n.ZigBeeNetworkManager:305  ] - Failed to open the dongle.
2018-09-14 18:12:21.962 [DEBUG] [gBeeNetworkStateSerializerImpl:144  ] - Loading ZigBee network state: Start.
2018-09-14 18:12:21.969 [DEBUG] [gBeeNetworkStateSerializerImpl:182  ] - Loading ZigBee network state: Done.
2018-09-14 18:12:21.979 [INFO ] [ome.event.ThingStatusInfoEvent:53   ] - 'zigbee:coordinator_cc2531:2732c9b9' updated: OFFLINE: Failed to open communications port
2018-09-14 18:12:21.989 [INFO ] [nt.ThingStatusInfoChangedEvent:53   ] - 'zigbee:coordinator_cc2531:2732c9b9' changed from UNKNOWN to OFFLINE: Failed to open communications port

Any ideas what I’m doing wrong?

[tebore (JT)]

The 3rd Party Bluetooth binding found a way to support user provided XMLs outside of the binding.

3rd Party Bluetooth Binding. Beta testers needed

Hello community, I’m pleased to announce that a new binding for Bluetooth devices is available for betta testing. It was a long journey for me to build this binding - 1 year of hard work and 15k+ lines of code. Nevertheless this is just the beginning. Here are the main use cases this binding supports: Presence detection. Bluetooth devices can be monitored whether they are in range of a bluetooth adapter (or multiple bluetooth adapters). This feature makes it possible to detect if an object o…

[chris (Chris Jackson)]

@tebore

The 3rd Party Bluetooth binding found a way to support user provided XMLs outside of the binding.

3rd Party Bluetooth Binding. Beta testers needed 3rd Party

Can you provide a reference or link to how this works? Maybe it’s a different XML - ie not the ESH thing files?

[tebore (JT)]

sputnikdev/eclipse-smarthome-bluetooth-binding

It is a different XML.

Coles notes on how it works:
Create a directory and give the binding(OH) the ability to read it.
Create XML files with definition of how to interpret the raw readouts of the device.

Seems like a good compromise between 0 configuration and an option for those who are willing to do a little bit of legwork outside of standard support.

[chris (Chris Jackson)]

So this is a different system than ESH. We need to stick to standards and IMHO there is no benefit to this and it really makes a nightmare for support…

Let’s just do it properly so that things are supported for everyone.