Share Z-wave dongle over IP (USB over IP using ser2net / socat ) guide

[Rafal (Rafal Lukawiecki)]

@chris

The binding knows absolutely nothing about RFC2217.

As I said, all the magic happens in the serial abstraction layer - the binding is just passing the port etc through. As I understand it, that should be all that’s need - the idea of this is that bindings do not need to be modified in order to support different serial abstractions as the general serial abstraction does this.

I did some experimenting with socat and creating the port in different directories. The summary of that is that except for /dev locations I always get “Port does not exist” message unless I add a Java runtime option “-Dgnu.io.rxtx.SerialPorts=/somewhere/else/ttyUSB0” at which point the binding starts working. I cannot specify the URI-like location (rfc2217://xxxx) as that option, it seems. I wonder if that is the reason it gets rejected by the binding outright.

@chris, is there anyway that you could force gnu.io.rxtx to recognise the URI as a serial port? I know very little Java, I am afraid.

[chris (Chris Jackson)]

Sorry - as I think I said earlier I don’t know anything about the way RFC2217 is handled. rxtx is just the serial driver - probably there is another configuration for RFC2217.

I’d I’d suggest to do a search in the ESH repository for RFC2217 and have a read of the issues there.

For example -:

Add Serial-over-IP (rfc2217) support in ESH Serial API

I raised that earlier though, and you said you’d read those mails so maybe you’ve already seen this. I’m sorry I can’t really help here as it’s not part of the binding and I don’t know anything about it.

[Rafal (Rafal Lukawiecki)]

Thanks, @chris, but it did not shed any new light. I am giving up on the OH2 upgrade for the week, glad that OH 1.8 is still so stable. I will have another look next weekend, hopefully someone else has more insight in the meantime, especially as also fighting the issue with ZWave serial dev lockfiles causing Java abort traps…

[dastrix80 (Kris K)]

Hi Rafal i also tried with no luck. Gave up on it. 2.4 has been rock solid using MQTT to bridge to instances, no more Zwave issues and it just works so so well. 80 days rock solid.

[Rafal (Rafal Lukawiecki)]

For what it is worth, I have reported this as a bug related to RFC2217 on the Github openhab-core repo.

[Rafal (Rafal Lukawiecki)]

@chris

think my statement that it was removed was wrong - I think it was relted to the serial discovery. Remember, this thread is nearly 3 years old now so there’s a lot of different stuff in here

I have just found the references to the comments about removal of a functionality that I have misread to mean removal of support for RFC2217 in the ZWave binding: [SOLVED] [Zwave, Zigbee, ...] RFC2217 remote serial port HowTo? - #4 by chris

I stand corrected by @chris that the functionality is present in the current ZWave binding, albeit no one has reported it as working, while there are 3 of us, @dastrix80, @vossivossi (who got a bit further that I and could see something in the logs) and I, at least, reporting it as not functioning.

Just found also this comment by @curlyel “For the Zwave I have to stick with the socat stuff anyway due to the reverted implementation. Thanks Chris for pointing this out!”. Based on the comments from @chris I assume this must be factually incorrect as there was no reverted implementation of the RFC2217 support. I hope I am getting this right!

[chris (Chris Jackson)]

We can see in the code that the ESH serial port abstraction classes are used, and not the NRJavaSerial, or any other library (directly).

openhab/org.openhab.binding.zwave/blob/2.5.x/src/main/java/org/openhab/binding/zwave/handler/ZWaveSerialHandler.java#L34

So in theory, the ZWave binding should be able to use any serial abstraction that is implemented in the ESH (now OH) serial abstraction layer. I understood from the original PRs on ESH that this included RFC2217, but I have never personally tested this.

Add Serial-over-IP (rfc2217) support in ESH Serial API

I hope that clarifies this?

[Rafal (Rafal Lukawiecki)]

Thank you, again, @chris, for taking the time to patiently explain how things are. Would I be right in thinking, then, that when I specify “rfc2217://ip:port” as a port in your ZWave binding this is not handled using nrjavaserial. However, when I specify “/dev/something” it does get passed to nrjavaserial? I wonder what generates the “Port does not exist” error message string that I am seeing.

[chris (Chris Jackson)]

@Rafal

Would I be right in thinking, then, that when I specify “rfc2217://ip:port” as a port in your ZWave binding this is not handled using nrjavaserial. However, when I specify “/dev/something” it does get passed to nrjavaserial?

That would be my expectation. I would expect that the ESH abstraction does some sort of check on the “domain” of the port and decides where to handle this. However, again, I’ve never looked at this code so I can’t confirm this.

[Rafal (Rafal Lukawiecki)]

@chris, could you point me at the code file/folder that deals with this, and I will see if I can figure out anything? I am not familiar with the OH codebase, I am afraid, so any pointers will jump start me. Or is it just “org.eclipse.smarthome.io.transport.serial”?

[chris (Chris Jackson)]

@Rafal

Or is it just “org.eclipse.smarthome.io.transport.serial”?

Yes- if you look in the OH2.5 branch in the core then this should be the folder I think. As I’ve said, I’ve never personally looked at or for this code, so I’m not really able to help with any level of knowledge. Unfortunately I don’t really have the time to spend trying to work this out right at the moment.

[Rafal (Rafal Lukawiecki)]

I have tried recompiling openhab-core with the new nrjavaserial-5.0.0. I can see that rfc2217 URI in the ZWave binding port config is now behaving differently. It still does not work but it gives a different error. See this Github issue for more detail, please feel free to make any suggestions that could help move it forward.

[Rafal (Rafal Lukawiecki)]

@chris, in case you are not following the two threads on the nrjavaserial repo, I have a quick update. Indeed, the issue with the lack of RFC2217 support was in nrjavaserial. It was missing the logic that checks for the URI and it was never trying to instantiate rfc2217 serial port, ie. it was always treating it as a “real” port. This logic is now in place in the master branch, hopefully it will be in a release soon.

As for the issues @MrRusch and my system has in opening even regular ports under FreeBSD, this is still an open case. I am running out of time due to low Java skills (maven scares me), but I will try to help if I can.

[Rafal (Rafal Lukawiecki)]

My bad. I have just re-run the test, using the freshest version of nrjavaserial (5.1.1+) and I can confirm that as long as I do not use OpenHAB, ie. I am only using a simple test harness in Java to open the port it is now working without errors and remains so for up to 2 minutes. I have not tried for longer.

Once the new build is in Maven I will rebuild OH 2.5 to see if it fixes our issues. I would have tested it locally now, but I lack the skills to reconfigure the Maven pom.xml files to use my locally built nrjavaserial. I also lack the skills to do what OpenHAB needs and to use a new jar, it seems. Hopefully I will know more tomorrow or in the next few days.

[wborn (Wouter Born)]

I’ve made some changes to the ZWave binding that should help with resolving some of the rfc2217 issues. Maybe some of you can test them?

See: [SOLVED] [Zwave, Zigbee, ...] RFC2217 remote serial port HowTo?

[Rafal (Rafal Lukawiecki)]

I will test this weekend. Many thanks for your work, @wborn!

[mvbergen]

[SOLVED] [Zwave, Zigbee, ...] RFC2217 remote serial port HowTo?

It works remote z-wave stick “online” without using socat!! Configuration at the system with the z-wave controller connected (/etc/ser2net.conf): 3001:telnet:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT remctl and as serial port for the z-wave thing in openHAB I entered (my server is called “meterkast”): rfc2217://meterkast:3001 Major step for me and many other openHAB users I think who use multiple or remote z-wave controllers!!! I will test reliability over the next days, tha…

[Christian_Hofstatter (Christian Hofstätter)]

May I ask how the reliability is? Even in the event of a power failure or reboot of the server or client?
I would like to pass my Zwave Stick to OH on Proxmox and hope that this also works if OH changes host.
At the moment I’m using USBIP, but it doesn’t reconnect automatically when the USBIP server is gone.

[mvbergen]

Actually rock solid, when the openHAB server reboots, remote serial ports are reconnected again and when I reboot the satellite running ser2net the connection is obviously lost but immediately reconnects and the connected devices are online and reporting again as can be seen from below log. This is running on the latest stable (2.5.9).

2020-10-20 08:12:30.062 [hingStatusInfoChangedEvent] - 'zwave:serial_zstick:173f800265d' changed from ONLINE to OFFLINE (COMMUNICATION_ERROR): Controller is offline
2020-10-20 08:12:30.063 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node5' changed from ONLINE to OFFLINE (BRIDGE_OFFLINE): Controller is offline
2020-10-20 08:12:30.065 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node2' changed from ONLINE to OFFLINE (BRIDGE_OFFLINE): Controller is offline
2020-10-20 08:12:30.066 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node3' changed from ONLINE to OFFLINE (BRIDGE_OFFLINE): Controller is offline
2020-10-20 08:12:30.067 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node4' changed from ONLINE to OFFLINE (BRIDGE_OFFLINE): Controller is offline

==> /var/log/openhab2/openhab.log <==

2020-10-20 08:13:17.978 [INFO ] [ve.internal.protocol.ZWaveController] - Starting ZWave controller
2020-10-20 08:13:17.979 [INFO ] [ve.internal.protocol.ZWaveController] - ZWave timeout is set to 5000ms. Soft reset is false.
2020-10-20 08:13:17.983 [WARN ] [nal.protocol.ZWaveTransactionManager] - NODE 2: Not initialized (ie node unknown), ignoring message.
2020-10-20 08:13:17.988 [WARN ] [nal.protocol.ZWaveTransactionManager] - NODE 2: Not initialized (ie node unknown), ignoring message.
2020-10-20 08:13:17.992 [WARN ] [nal.protocol.ZWaveTransactionManager] - NODE 2: Not initialized (ie node unknown), ignoring message.
2020-10-20 08:13:17.997 [WARN ] [nal.protocol.ZWaveTransactionManager] - NODE 2: Not initialized (ie node unknown), ignoring message.

==> /var/log/openhab2/events.log <==

2020-10-20 08:13:21.302 [hingStatusInfoChangedEvent] - 'zwave:serial_zstick:173f800265d' changed from OFFLINE (COMMUNICATION_ERROR): Controller is offline to ONLINE
2020-10-20 08:13:21.307 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node3' changed from OFFLINE (BRIDGE_OFFLINE): Controller is offline to ONLINE
2020-10-20 08:13:21.309 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node2' changed from OFFLINE (BRIDGE_OFFLINE): Controller is offline to ONLINE
2020-10-20 08:13:21.315 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node5' changed from OFFLINE (BRIDGE_OFFLINE): Controller is offline to ONLINE
2020-10-20 08:13:21.315 [me.event.ThingUpdatedEvent] - Thing 'zwave:device:173f800265d:node5' has been updated.
2020-10-20 08:13:21.316 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node4' changed from OFFLINE (BRIDGE_OFFLINE): Controller is offline to ONLINE
2020-10-20 08:13:21.316 [me.event.ThingUpdatedEvent] - Thing 'zwave:serial_zstick:173f800265d' has been updated.
2020-10-20 08:13:21.331 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node3' changed from ONLINE to ONLINE: Node initialising: PING
2020-10-20 08:13:21.342 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node5' changed from ONLINE to ONLINE: Node initialising: PING
2020-10-20 08:13:21.351 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node2' changed from ONLINE to ONLINE: Node initialising: PING
2020-10-20 08:13:21.362 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node4' changed from ONLINE to ONLINE: Node initialising: PING
2020-10-20 08:13:21.392 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node3' changed from ONLINE: Node initialising: PING to ONLINE: Node initialising: REQUEST_NIF
2020-10-20 08:13:21.547 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node5' changed from ONLINE: Node initialising: PING to ONLINE: Node initialising: REQUEST_NIF
2020-10-20 08:13:21.937 [me.event.ThingUpdatedEvent] - Thing 'zwave:device:173f800265d:node5' has been updated.
2020-10-20 08:13:23.600 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node5' changed from ONLINE: Node initialising: REQUEST_NIF to ONLINE
2020-10-20 08:13:23.836 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node2' changed from ONLINE: Node initialising: PING to ONLINE: Node initialising: REQUEST_NIF
2020-10-20 08:13:24.140 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node3' changed from ONLINE: Node initialising: REQUEST_NIF to ONLINE
2020-10-20 08:13:24.438 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node4' changed from ONLINE: Node initialising: PING to ONLINE: Node initialising: REQUEST_NIF
2020-10-20 08:13:24.514 [vent.ItemStateChangedEvent] - PoolhousePM25 changed from 18 to 15
2020-10-20 08:13:25.800 [vent.ItemStateChangedEvent] - PoolhouseLoudness changed from 60 to 57
2020-10-20 08:13:26.399 [vent.ItemStateChangedEvent] - PoolhouseTemperature changed from 8.4 °C to 8.5 °C
2020-10-20 08:13:27.587 [vent.ItemStateChangedEvent] - PoolhouseVOC changed from 0.006 to 0.008
2020-10-20 08:13:28.887 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node4' changed from ONLINE: Node initialising: REQUEST_NIF to ONLINE
2020-10-20 08:13:35.406 [vent.ItemStateChangedEvent] - PoolhouseCO2 changed from 437 to 441
2020-10-20 08:13:35.417 [hingStatusInfoChangedEvent] - 'zwave:device:173f800265d:node2' changed from ONLINE: Node initialising: REQUEST_NIF to ONLINE

[dirandad]

Hi all, Thank you for this very usefull discussion.
I achieve to get my openhab docker hosted in a NAS communicating with a remote zwave controller (Razberry) using ser2net and socat.
But like some of you, the controller will stay offline when the connection between the docker and the razberry is lost. A restart of socat solve the problem. As there is no systemctl on my docker i cannot achieve to monitor and auto-restart socat. May be I haven’t found the final solution in this long discussion, but have someone get a stable solution in a docker environment to maintain socat connected ?