2.5.x ZWave Thing Struggles

Add-ons Bindings
1

Apologies if this is a dupe or related to a dupe. I did quite a lot of Google and forums searches prior to posting here.

My historical setup for OpenHAB + zwave (mainly) has used configuration files since I deployed version 2.0. The exception has historically been Thing definition, since the ZWave add-on seemed to prefer auto-discovery. When I upgraded to 2.4 (snapshots, and then release) to take advantage of Chris’ ZWave Security I had my first full jsonDB reset, did some cleanup, and I have been running that version since. Just recently (with a forcing function of some new devices) I decided to upgrade to 2.5.x.

I decided to take this opportunity to get my Thing definitions (the last GUI hold-over) moved to config files so I could more easily move between deployments–and not have to re-enter all the devices names in the UI each time I decide that openHAB needs a reset.

The other bindings (astro, unifi, ntp, etc.) all behave as I expect, but I am really struggling with one facet of the zwave piece that I hope someone can clue me in on.

I define the Bridge for the controller, and it works fine. All the channels are listed, and the channels correctly map to the items that consume them. For completeness, this is that definition. Key changed to protect the innocent:

Bridge zwave:serial_zstick:controller "ZWave Plus USB Dongle" [ controller_softreset=false, security_networkkey="11 22 33 44 55 66 77 88 99 AA BB B7 32 BC 23 8E", security_inclusionmode=0, controller_sisnode=1, controller_sync=false, port="/dev/ttyACM0", controller_master=true, inclusion_mode=2, controller_wakeupperiod=3600, heal_time=2, controller_exclude=false, controller_inclusiontimeout=30, controller_hardreset=false ]

The parameters were datamined from the 2.4.x jsonDB, and seem to work fine.

My confusion:
If I follow the (admittedly dated) tutorials and configure a node as such:

Thing zwave:ge_ge14291_00_000:controller:node38 "Light_LF_Office_Ceiling" (zwave:serial_zstick:controller)   [node_id=38]

it will show up in HABmin (and PaperUI), the channels are listed, but the items do not function. Additionally, there is still an auto-discovered version of that switch (as zwave:device:controller:node38) in the inbox.

If I adjust the Item to attach to the auto-discovered device, the channels work. However, if I use the zwave:device:controller:node38 in the zwave.things file (for example), quite naturally the zwave binding has no idea what the device type is–leaving me unable to use the Thing.

The objective here is to have the configuration file have my names for the Things, and be able to (for example) just move to 2.5.4 or whatever without having to manually re-input the discovered zwave device Things.

Is there something I am missing? Is there a way to use Thing zwave:device:controller:node38 and pass the zwave device UID as an additional parameter?

I know the use of configuration files is a vaguely contentious topic, but given my historical success using them for literally everything else I was a bit frustrated that I could not seem to get the Things to behave.

Again, apologies if this is answered and my search skills failed me.

2

Thing files are prone to error. If I was going to do that I would try to use the REST API to capture the Thing definition of a GUI created Thing that functions and then replicate it in a file.

1 Like
3

Thing files are prone to error. If I was going to do that I would try to use the REST API to capture the Thing definition of a GUI created Thing that functions and then replicate it in a file.

Thank you for the suggestion. That is indeed what I did to generate the definitions. I wrote a quick Python script to handle the enumeration of the attributes and output the thing syntax.

The problem appears to be that the zwave binding does not allocate the “node” entries from the controller to my thing definitions that include the type_id–required for the binding to understand the command groups and whatnot of an individual zwave device.

So the auto discovered ones are “zwave:device:blah” rather than the necessary “zwave:<type_id>:blah”. If I create a thing with just the “device” literal, the zwave binding reports it as being an unknown device type with no available channels. So the auto-discover is doing some mapping that either I do not understand or is not exposed in the Things file.

The relevant setting is actually in the payload when you do the get–it is thingTypeUID. I just do not see how to apply that particular value in the Thing file syntax without breaking the linkage, as described in my first post.

Apologies if I am being unclear, and thanks again for the suggestion.

4

I can’t/won’t help with the .things file problems. But I want to understand this “manually re-input discovered Things” part. What you do mean? What do you have to manually input? They’re discovered. Assuming that you do not recreate the Zwave Controller Thing (or recreate it with the same Thing ID) all of the rediscovered Things will be rediscovered with the same IDs as before so there will be no need to relink them to Items.

Unless there is something I don’t understand, this seems like a whole lot of work to avoid a teeny tiny amount of work that very rarely needs to be done and which could be achieved in other easier to deal with ways.

The autodiscovered IDs are the correct ones. All zwave Thing IDs are zwave:device:<UID of the controller Thing>:node<number>. I’m pretty sure you can’t just change that expected pattern even if you are using .things files. That second field (device/serial_zstick) is what tells the binding whether it’s dealing with the bridge/controller Thing or something it communicates with through the controller.

The only part of the Thing ID I think you can make what ever you want is the part you’ve defined as “controller”.

Also, take note that there are some important things you cannot do with Zwave devices when the Things are defined in .things files including setting parameters on the device (e.g. turn off the little indicator light, setting the timeout on a motion sensor, etc.).

2 Likes
5

Thank you for the attention and the reply.

Unless there is something I don’t understand, this seems like a whole lot of work to avoid a teeny tiny amount of work that very rarely needs to be done and which could be achieved in other easier to deal with ways.

It is coming up enough that it annoys me, and that was the catalyst. In PaperUI, every time I add one of my zwave switches from autodiscovery (I have 40 or so) I have to manually enter (or paste) the correct device name. Otherwise it gets added as “Z-Wave Node 005: ZW4005, 12722 On/Off Relay Switch” rather than something more helpful to me visually. That device, for example, should be named “Light_UF_MBR_Ceiling_Switch”. If there is some other way to codify the name than a .things definition, I would welcome it.

Perhaps as I am tinkering and upgrading I am blowing away my configuration more often than most people, but I have yet to really have openHAB’s configuration survive an upgrade. So it is back to re-entry time.

The only part of the Thing ID I think you can make what ever you want is the part you’ve defined as “controller”.

In either event I am not making it want I want. In the context of zwave, the binding needs additional information about the device, the central repository for which is Chris’ database. In the things file, if I do what you are suggesting, e.g.:

Thing zwave:device:controller:node5 <rest of the stuff>

then the zwave binding is unable to figure out that it needs to use the database entries for a “ZW4005, 12722” switch, which is unique ID zw4005. As a consequence, it does not expose the channels for my Items to use.

In zwave land, the Thing definition that provides that information to the binding (through the bridge) is to define it as:

Thing zwave:zw4005:controller:node5 <rest of the stuff>

However, when I do that, nothing works, and the auto-discovered entries continue to be the ones that the binding “knows about”.

That is what I am trying to figure out.

If the answer is ultimately “what you are doing is stupid” then I will eventually waive off. But this very much should work fine the way I am doing it, and at a minimum I would like to understand why it doesn’t.

6

Well, if you are waiting six months to a year between upgrades I’m not surprised. I’ve never lost an OH config during an upgrade and have only had to rediscover my Zwave things once. But I upgrade about once a month so the changes are smaller. YMMV.

Then you must know more than me so I’ll leave you to it.

7

I have not lost the configuration either. Experience from people here would be to clear the cache. After the upgrade it can take between 1 and 3 restarts of openHAB to settle the system.

Last time I upgraded the system did not properly read my text Item files until I restarted.

8

As I indicated, I am aware that the use of the configuration files is a contentious topic in the forums. Nevertheless, it was my hope that someone with additional expertise or experience might have some insight as to what is going on. It may be that only Chris can answer, but I hate to ping him. :slight_smile:

My recent experience was that my 2.4 configuration did not work when I attempted to upgrade to 2.5.0 (as a milestone toward 2.5.4) at all. Perhaps there were interim 2.4 releases that I did not note in that progression. Some of the brokenness was very managable and documented, such as service movement from misc to ui, etc. For sure, my 2.4 installation was running sufficiently well that I was not particularly eager to break it. So yes, in this instance it is a wide upgrade interval, but I believe I am going between official release builds? I have not run any snapshots since the Z-Wave security stuff was folded into production.

And yes, I too am accustomed to needing a few restarts before things truly settle after a fresh install (even of the same version).

Using .items files (and rules, and maps, and sitemaps) has always worked extremely well for me, and I do wish to achieve the same consistency on the Things side. I certainly have seen plenty of ideological opposition to doing so here–though I cannot profess to understand the basis for it.

9

No. 2.5.0 was originally released last December. This is the 5th 2.5 release, all official.

10

No - if you want to use the thing files, you will need to determine the thing type and configure everything manually. The binding always uses the device UID initially and then changes it once it has determined the real type, but this can’t be done when you use the thing files. When the thing type changes, the thing UID remains the same. There’s a lot of ESH history behind this…

1 Like
11

No problem. Thanks for answering, and for your work on this. Does it match your expectations that despite having the entries correct (thing-type inclusive) duplicate items still show up in my inbox in HABmin with the auto-detected (type-less) name?

And, the Items do not work when linked to channels from my .thing definitions. Happy to be more verbose with what I see if it would help.

Also, thanks for the considerable work you put into the Z-Wave binding (and device database). It has been working well for me for years.

12

I beg your pardon for being unclear. I meant that going from 2.4.0 to 2.5.0 would be an adjacent release version upgrade, and thus least likely to have problems with configuration. Unlike, for example, going from 2.4.0 to 2.5.4 directly. Perhaps that is inaccurate.

I am aware that 2.5.0 is old, and 2.5.4 is current. I was doing a springboard thing.

13

I don’t use the thing files so I’m not really able to answer this. I have a suspicion that there was a change in the ESH/OH core that may have impacted this since a check on existing entries in the database was removed from the core a while back and maybe that’s part of the problem you’re seeing.

You should be able to set the type, and the channels should all show up.

Going back to this comment I made about the ESH history… In the past, the type type was defined as : <thing_type> : . This was ultimately changed so that the second part was not the thing_type and thing_type is set separately. Now… I’m not sure what was done for thing files - ie if the thing UID and thing_type_uid can be separated in this way when using these files.

Sorry - I appreciate I’m not being super helpful here but I’m really not familiar with how ESH is using the files as everything I do is using the binding to configure this.

1 Like
14

You’re actually being extremely helpful and generous with your time. Helping that does not yield an immediate solution is still helping.

I am giving up on this and reverting to how I did it before (with the exception of a .things entry for the controller), but I did mess with it further. I fiddled some more with one of the above examples (node38). Doing this, per your example from many moons ago does not work:

Thing zwave:ge_ge14291_00_000:controller:node38 "Light_LF_Office_Ceiling" (zwave:serial_zstick:controller)   [node_id=38]

With that entry in the .things file, there is still a discovered thing in the inbox at “zwave:device:controller:node38”. What is weird is that item (the experimentation target) ends up showing up grayed out with a disabled sign on it in HABmin. The other Things intermittently actually work fine. My supposition is that Things that are identified ahead of the discovery process completing seem to work and present their channels… but any changes to the .things file go sideways.

In the end is is clear that this is going to create more headaches for me than it solves. That I can get it to work intermittently is somehow worse than complete failure though.

Thanks again for the insight. I am sad the hooks for specifying everything in the .things file were deprecated, but I will get over it.

I have 2.5.4 up now and things seem pretty good. Just need to verify all my sensor rules.

15

One more element of closure on this. @Bruce_Osborne nudged me in a superior direction. I had been using the REST API for some time to interrogate openHAB. I had not fully considered that I could very easily achieve my desired end state by writing label updates via REST as well. Doing so is not hard, and better aligns with how Z-Wave devices should be managed.

I tossed together a trivial python script that will ingest my list of Things and their correct names, and use REST to update their names in openHAB, and it works fine against 2.5.4. Woohoo!

So my build-experimentation process can simply rely on HABmin’s “add all” button for discovered Z-Wave devices and a quick script run to update the names to comply with my preferences. No more data entry.

Thanks all for the discussion.

1 Like
16

Actually all the UIs use the REST API to communicate with OH. Whatever the UI can do you should be able to do through the API.

17

Just an oversight on my part that it did not occur to me to try. It is indeed glorious. Thanks again.

1 Like
18

I just want to add some clarification to the way OH releases are managed.

Release:
Normally, OH will have two releases a year, once every six months (plus or minus a few weeks). These releases would be like going from 2.3 to 2.4. There will be six months worth of development during between those releases and usually a dozen or so breaking changes (mostly in bindings). There is usually a two week or so feature freeze just prior to a 2.4 release to look for any serious bug but in my experience a release is just about as likely to have a major bug as a milestone or snapshot. I don’t know if the devs do any specific testing during this freeze or if they are relying on us.

Milestone:
Once a month, at a point where there is no known major bugs that would prevent it, a milestone release sometimes called a testing release is made. These will have a version along the lines of 2.4 M4.

Snapshots:
Every day, unless the build fails, there is a new snapshot that includes everything that has been merged into the baseline. This is the bleeding edge and has the least amount of testing overall. Yet it’s surprisingly sable over all.

Deviations:
For six months or so, IIRC, development during 2.4 was broken due to a move to a new build system and abandoning Eclipse Smart Home (the great merge). So there was more than a year between 2.4 and 2.5. Development was slowed during this time but I don’t think it was completely stopped.

We are also currently in a strange place for releases. OH core is frozen at 2.5. No new features and no new fixes (unless they are determined to be really big problems which seems unlikely at this point). All new development to the core is currently on OH 3 which will include a lot of breaking changes. However, we don’t want to stop binding development for all this time so once a month, instead of milestone releases like normal we have sub-point releases (e.g. 2.4.5). These are like milestone releases but the only changes are made to add-ons.

Given the above, a jump from 2.4 to 2.5.4 would have been nearly a year and a half jump with over a year’s worth of development. Under normal circumstances such a jump would be six months worth of development which is still quite a lot of changes. This is why on another thread my recommendation is to either upgrade frequently (once a month for the milestones/point releases) or never upgrade. When you wait too long the amount of effort and the risks of a whole bunch of stuff breaking all at once is too high. And if you choose to never upgrade, that means not upgrading anything else on that machine because you never know when an upgrade to some library or Mosquitto or something may break your OH instance.

I would guess that once OH 3 comes out (end of the year?) that we will return to the release/milestone/snapshot releases.

19

I’ve got all of my ZWave things/items defined via text files.

The trick for me was listing them within the Bridge defintion:

└> cat openhab/conf/things/zwave.things
Bridge zwave:serial_zstick:controller_2 "Z-Wave Controller" @ "Z-Wave" [ port="/dev/ttyACM0", controller_softreset="false", controller_master="true", heal_enable="true", heal_time=2, security_networkkey="#@#@ZWAVE:SECRUITY_NETWORK_KEY#@#@" ] {
	Thing hank_dws01_00_000 node3 "Z-Wave Node 003: DWS01 Door and Window Sensor (Laundry Room)" [ node_id = 3 ]
	Thing fibaro_fgwpb121_00_000 node4 "Z-Wave Node 004: FGWPB-121 Fibaro Wall Plug" [ node_id = 4 ]
 	Thing hank_dws01_00_000 node6 "Z-Wave Node 006: NAS-DS01Z Door/Window Sensor
 (Safe)" [ node_id = 6]
 	Thing zooz_zse40_00_000 node7 "Z-Wave Node 007: ZSE40 Zooz 4-in-one motion/temperature/humidity/luminance sensor (BreJ {Breland} bed motion)" [ node_id = 7]
	Thing zooz_zse40_00_000 node8 "Z-Wave Node 008: ZSE40 Zooz 4-in-one motion/temperature/humidity/luminance sensor (BreJ {TJ} bed motion)" [ node_id = 8 ]
	Thing hank_dws01_00_000 node9 "Z-Wave Node 009: NAS-DS01Z Door/Window Sensor
 (Front Door)" [ node_id = 9]

}

My items look like:

└> cat openhab/conf/items/entryway.items
Number FrontDoorBattery "Front Door Battery" (gBatteries) {channel="zwave:hank_dws01_00_000:controller_2:node9:battery-level"}
Contact FrontDoor "Front Door State" {channel="zwave:hank_dws01_00_000:controller_2:node9:sensor_door"}

I use @chris’s zwave database export to derive the type. This works pretty well, I just end up hiding the “inbox” items in paper UI.

My motivation for using the text files over the “database” is I don’t trust the device (or rather the storage) this hosted on. It’s a raspberry pi - but I end up putting openhab in a docker container and source control all my rules / items / configuration.

If the device dies, I should be able to somewhat trivally move this to another device. I also use CI/CD (GitLab + Balena) to roll out changes to it. I use placeholders for secrets (like the #@#@ZWAVE:SECRUITY_NETWORK_KEY#@#@ annotation) - and those get replaced at “deploy time”. The device (pi) has access to a secret store (AWS Secrets Manager in this case) to find out the values for secrets. It even allows me to open source the rules/configuration at some point without fear of leaking creds or network topology of my internal setup.

Hope that helps a smidge

20

Not arguing that you should change anything. I just want to mention that this is all possible with JSONDB defined stuff too. They are just JSON formatted text files.