Zwave remote connection rfc2217 vs tcp (raw). running zwave-js-ui with zwave binding

billfor · November 22, 2022, 8:37pm

If anyone has run the zwave binding concurrently with zwave-js-ui against remote ser2net I would be interested to know the configuration. The preferred way to run remote OH zwave binding seems to use rfc2217 which requires a ser2net like this:

10002:telnet:0:/dev/zstick:115200 8DATABITS NONE 1STOPBIT max-connections=3 remctl

Unfortunately zwave-js-ui operates on raw socket so the configuration is like:

10001:raw:600:/dev/zstick:115200 8DATABITS NONE 1STOPBIT max-connections=3

If anyone has figured out the edge case to get both running reliably I am all ears. For the moment I switched the zwave binding back to using a local device created by socat so that I can use the raw setting for both. This is OK for reads but writes from zwave-js-ui seem to fail when I use the same listener. I’m also expecting OH to crash at some point (as I recall this was a problem when using socat to represent the remote device).

Oekel · February 12, 2024, 12:25pm

I am actual searching the net for the same question.

Initially I activated USBIP (too much traffic)
Then found ser2net/socat in many posts.

Then some person asking for “why socat” when OH supports rfc2217 since 2.5

So I found this thread:
success story for RFC 2217/Z-Wave (andematt 258/269)

But as you I want to switch do docker and to zwave-js-ui grapping full support fo my serial/usb.
Will sum up here, if I found out if/how rfc2217 can be used in the zwave-js-ui container

As far as I could not see support OOTB (in any documentation) I started:

apella12 · February 12, 2024, 4:29pm

Unsure of what you trying to be accomplished as zwave-js-ui and the OH ZW binding are redundant. If you just want to get Zwave data to another machine, run one (ZUI or OHZ) then use the Remote OH binding to bring over the relevant items to your other machine

Oekel · February 12, 2024, 4:53pm

All not maintained in case of Z-Wave aliance. And Driving another OH for a single binding is overhead and error pruned.

apella12 · February 12, 2024, 7:01pm

I don’t understand. Are you saying a device is “Unknown” in either OHZ or ZUI? Both apps have ways to add unknown devices.

For example I have a remote ZUI in docker on a Rpi3 that uses 300MB of the 1GB with 64 bit OS (no OH). It was only about 150 MB with 32 bit. Mqtt and OH is on the Main rpi4.

Oekel · February 16, 2024, 4:16pm

Ok lets start from the beginning to avoid misunderstandings:

Working with Docker and VLANs
Want to Use OpenHab --MQTT–> Zwave-JS-UI --IP(ser2net)–> MiniPC(pi3 or 4) “Z-Stick7” or “Zwave.me7”
DockerHost (Ubuntu with Portainer) 192.168.1.111 (up and running great)

OpenHab-container with ipvlan 192.168.5.201
MQTT-contaner with ipvlan 192.168.5.202
zwave-js-ui-container wht ipvlan 192.168.5.203
Pi3 as Test-Server with plugged in Z-Stick7 (/dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0)

On the Pi3 (4) I allso installed a TestEnv with portainer and zwave-js-ui-container.
quicktest with

device:
    - /dev/zwave : dev/ttyUSB0

or

device:
    - /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0

or

device:
    - /dev/zwave : /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0

Works great and I added one Node2 for Testing purposes.

Then installed ser2net with following config under /etc/ser2net.yaml

%YAML 1.1
---
# This is a ser2net configuration file

define: &banner \r\nser2net port \p device \d [\B] (Debian GNU/Linux)\r\n\r\n

connection: &zwavestick
    accepter: telnet(rfc2217),tcp,4000
    enable: on
    options:
      kickolduser: true
    connector: serialdev,
              /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0,
              115200n81,nobreak,local

# nc 127.0.0.1 4000
??????????????,

(no guess why I am not gettsing “succeeded!” here on localhost like in the next line (same pi, same sh)

# nc -v 192.168.1.102 4000
Connection to 192.168.1.102 4000 port [tcp/*] succeeded!
??????????????,

# sudo minicom -D /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0
Device /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0 is locked.

(guess it is locked by ser2net, what is ok so far)

# sudo lsof -i -P -n | grep LISTEN | grep ser2net
ser2net    4004  root    7u  IPv6 112288      0t0  TCP *:4000 (LISTEN)

# sudo service ser2net status
● ser2net.service - Serial port to network proxy
     Loaded: loaded (/lib/systemd/system/ser2net.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-02-16 16:15:05 CET; 54min ago
       Docs: man:ser2net(8)
   Main PID: 4004 (ser2net)
      Tasks: 1 (limit: 1595)
        CPU: 1.033s
     CGroup: /system.slice/ser2net.service
             └─4004 /usr/sbin/ser2net -n -c /etc/ser2net.yaml -P /run/ser2net.pid

Feb 16 16:15:05 zwave systemd[1]: Starting Serial port to network proxy...
Feb 16 16:15:05 zwave systemd[1]: Started Serial port to network proxy.

# sudo fuser -v /dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0
                     USER        PID ACCESS COMMAND
/dev/ttyUSB0:        root       4004 F.... ser2net

And now I am out if ideas how to get it into the client (zwave-js-ui)
as replacement for direct hardware access…

Next step to try will be “backconverting” from ip to serial with help of socat?? (I wanted to avoid this)

PS: I also wanted to avoid experimenting with usbip (raw data) regarding to much traffic.

Greating Oekel

apella12 · February 16, 2024, 7:10pm

Sorry, but I have no experience with your setup. Good luck.

pauli_anttila · February 17, 2024, 8:05am

Try acceptor without telnet filter as it probably modify the raw tcp stream. I’m using ser2net and zwavejs2mqtt without it.

accepter: tcp,4000

in zwavejs2mqtt side, the tcp://192.168.1.102:4000 should be the correct format.

Oekel · February 19, 2024, 11:49am

Ok thanks for you reply that gave me new courage to test again.

pauli_anttila:

Oekel:

accepter: telnet(rfc2217),tcp,4000

Try acceptor without telnet filter as it probably modify the raw tcp stream. I’m using ser2net and zwavejs2mqtt without it.
accepter: tcp,4000

May be right but after it is working now with “tcp” I can say it makes no different if the accepter has one or two values. So lets stay with your shortened version becourse:

unplug/replug USB/Netzwork also works OOTB
(No matter what I wrote in the ser2net config: rfc2217//:ip:port NEVER worked!)

As conclusion I can say it is a great tool to work within Docker (caurse I can expose my USB-Devices everywhere)

Will now also give a try routing one device over Internet/VPN and give featback afterward.

The last thing regarding docker-compose I do not check (but which is not so important becaurse we can configure device directly in OH or zwavejs2mqtt):

devices:
    - "tcp://192.168.1.102:4000:/dev/zwave"

is not correct syntax and will not be accepted (I guess because the “:” after tcp and before 400)
But who knows, maybee also for this someone has an idea.

CU
David

PS:
Let me give you some comandlines I used to find my problems

man ser2net //config files are in */etc/ser2net/ser2net.yml NOT* /etc/ser2net.yml

EDIT: wrong man pages!!! /etc/ser2net.yml is correct

# sudo lsof -i :4000
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
ser2net 5688 root    2u  IPv6 202404      0t0  TCP 192.168.1.102:4000->192.168.1.111:50348 (ESTABLISHED)
ser2net 5688 root    7u  IPv6 199312      0t0  TCP *:4000 (LISTEN)

$ sudo lsof -i | grep socat
socat     1218350            root    8u  IPv4 10581656      0t0  TCP ubuntudocker:36772->192.168.1.102:4000 (ESTABLISHED)

sudo nano /etc/default/socat.conf
# systemctl stop socat
pkill socat
pgrep socat

Oekel · March 3, 2024, 1:53pm

Ok, after days I got the solution (without socat).
For all of the german guys, please read here: Razberry7 testen (ohne OH) - openhabforum.de

For all others, just let me pronounce that it was a failute in too many linebreaks inside ser2net.yaml
Here a working configuration:

define: &banner \r\nser2net port \p device \d [\B] (Debian GNU/Linux)\r\n\r\n

connection: &aeotecstick7
  accepter: tcp,4000
  enable: on
  options:
    banner: *banner
    kickolduser: true
    telnet-brk-on-sync: true
  connector: serialdev,/dev/serial/by-id/usb-Silicon_Labs_CP2102N_USB_to_UART_Bridge_Controller_5828e4a40755ed1186a193131d62bc44-if00-port0,115200n81,local

connection: &razberry7
  accepter: tcp,4001
  enable: on
  options:
    banner: *banner
    kickolduser: true
    telnet-brk-on-sync: true
  connector: serialdev,/dev/ttyAMA0,115200n81,local

connection: &ender3pro
  accepter: tcp,5000
  enable: on
  options:
    banner: *banner
    kickolduser: true
    telnet-brk-on-sync: true
  connector: serialdev,/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0,115200n81,local

Should then look like that:

root@zwave:~# lsof -i
COMMAND    PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
...
ser2net   3443  root    7u  IPv6  27543      0t0  TCP *:4000 (LISTEN)
ser2net   3443  root    8u  IPv6  27544      0t0  TCP *:4001 (LISTEN)
ser2net   3443  root    9u  IPv6  27545      0t0  TCP *:5000 (LISTEN)
ser2net   3443  root   10u  IPv6  27546      0t0  TCP 192.168.1.14:4000->192.168.5.203:48282 (ESTABLISHED)
ser2net   3443  root   12u  IPv6  27547      0t0  TCP 192.168.1.14:4001->172.17.0.2:40624 (ESTABLISHED)
ser2net   3443  root   14u  IPv6  29343      0t0  TCP 192.168.1.14:5000->192.168.1.111:39644 (ESTABLISHED)