Share Z-wave dongle over IP (USB over IP using ser2net / socat ) guide

snoekieboe · October 7, 2017, 6:17pm

Hi,

Recently i’ve been working on virtualising all IT systems in my home. I’ve virtualised my NAS, Firewall, Router, Network Video Recorder and ofcourse my Openhab2 setup.
During this virtualisation effort i ran into a few issues with regards to passing through my Z-wave.me USB1 z-wave dongle to a virtual machine.

Community member Rikoshak pointed me to an alternative solution where i would leave my USB dongle connected to a physical machine (e.g. Raspberry Pi) and share the USB device over the network.
Not only would this solve my passthru issues in VMware esxi 6.5 but this would also allow me to position my Z-wave dongle in the most optimum location, away from my server that is located in the basement.

Since i’ve got this up and running, and information was hard to find, i would like to share my steps so you can benefit as well.

General architecture overview:

Untitled Diagram

To make this work we have some work to do:

install & configure “ser2net” in the RPI3 who is going to serve the USB z-wave dongle
install & configure “socat” on the virtual Openhab instance as client (Ubuntu 16.04 in my case)
Make sure all services on both machines auto-start and do so with the correct priveleges

The following section will show you all the steps you need to take on both the server and the client:

USB device server (RPI3 with Zwave.me USB dongle IP=172.16.11.5 replace with your IP)

I started with a fresh install of Debian Jessie (installed using noobs installer)
plug in your USB dongle and verify the name of the port (ttyACM0 in my case)

cd /dev
ls

autofs           gpiochip1  loop7               null   ram5     tty    tty21  tty35  tty49  tty62      vcs    vhci
block            gpiochip2  loop-control        ppp    ram6     tty0   tty22  tty36  tty5   tty63      vcs1   watchdog
btrfs-control    gpiomem    mapper              ptmx   ram7     tty1   tty23  tty37  tty50  tty7       vcs2   watchdog0
bus              hwrng      mem                 pts    ram8     tty10  tty24  tty38  tty51  tty8       vcs3   zero
cachefiles       initctl    memory_bandwidth    ram0   ram9     tty11  tty25  tty39  tty52  tty9       vcs4
char             input      mmcblk0             ram1   random   tty12  tty26  tty4   tty53  **`ttyACM0`**    vcs5
console          kmsg       mmcblk0p1           ram10  raw      tty13  tty27  tty40  tty54  ttyAMA0    vcs6
cpu_dma_latency  log        mmcblk0p2           ram11  rfkill   tty14  tty28  tty41  tty55  ttyprintk  vcsa
cuse             loop0      mmcblk0p5           ram12  serial   tty15  tty29  tty42  tty56  uhid       vcsa1
disk             loop1      mmcblk0p6           ram13  serial1  tty16  tty3   tty43  tty57  uinput     vcsa2
fb0              loop2      mmcblk0p7           ram14  shm      tty17  tty30  tty44  tty58  urandom    vcsa3
fd               loop3      mqueue              ram15  snd      tty18  tty31  tty45  tty59  vc-cma     vcsa4
full             loop4      net                 ram2   stderr   tty19  tty32  tty46  tty6   vchiq      vcsa5
fuse             loop5      network_latency     ram3   stdin    tty2   tty33  tty47  tty60  vcio       vcsa6
gpiochip0        loop6      network_throughput  ram4   stdout   tty20  tty34  tty48  tty61  vc-mem     vcsm

now we need to install ser2Net", the application that will share ttyACM0 over the network

sudo apt-get install ser2net

Once the installer is finished we can configure ser2net

sudo nano /etc/ser2net.conf

This opens the ser2net.conf file in the nano editor.
add the following line to the bottom of the file (modify according to your setup)

3333:raw:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT

this line tells ser2net to share device /dev/ttyACM0 on TCP port 333 at 115200 baud
press ctrl+x to save the modified file and exit the nano editor
next up is to restart ser2net to apply the modified condiguration

sudo service ser2net restart

Check to see of ser2net is sharing the device using netstat… look for an entry that contains 0.0.0.0:3333

netstat -antp

if the TCP entry is shown you have successfully finished the server side setup of this USB over IP adventure. Ser2net is already installed as a service and will automatically start after a reboot or cold start.

USB device client (OpenHAB running on a virtual linux machine IP=172.16.11.4 replace with you IP)

The Openhab machine will be the client connecting to the shared USB device on the RPI3. In order to do so we need to install ser2net’s counterpart called “socat”.

install socat on the openhab machine

sudo apt-get install socat

download and install an init.d script that allows socat to run as a service and auto-start at startup

git clone https://github.com/asaif/socat-init.git

sudo cp socat-init/socat /etc/init.d/
sudo chmod +x /etc/init.d/socat
sudo update-rc.d socat defaults
sudo cp socat-init/defaults/socat.conf /etc/default/

now we have all the necessary scripts installed it’s time to configure them:

sudo nano /etc/default/socat.conf

add the following line to “socat.conf”

it creates a new serial USB device known as ttyUSB0
it creates the device as user openhab and adds it to the dialout group
it tells socat to connect to the RPI3 USB server using it’s IP and port 333

OPTIONS="pty,link=/dev/ttyUSB0,raw,user=openhab,group=dialout,mode=777 tcp:172.16.11.5:3333"

ctrl+x to save and exit nano editor
we are no ready to (re)start socat and check if we have successfully connected to the z-wave dongle using the network

sudo service socat restart

check the logs to see if connection was successful

tail -f /var/log/socat.log

2017/10/07 19:01:37 socat[3657] I setting option "symbolic-link" to "/dev/ttyUSB0"
2017/10/07 19:01:37 socat[3657] I setting option "raw"
2017/10/07 19:01:37 socat[3657] I openpty({6}, {7}, {"/dev/pts/1"},,) -> 0
2017/10/07 19:01:37 socat[3657] N PTY is /dev/pts/1
2017/10/07 19:01:37 socat[3657] N opening connection to AF=2 172.16.11.5:3333
2017/10/07 19:01:37 socat[3657] I starting connect loop
2017/10/07 19:01:37 socat[3657] I socket(2, 1, 6) -> 8
2017/10/07 19:01:37 socat[3657] N successfully connected from local address AF=2 172.16.11.4:40282
2017/10/07 19:01:37 socat[3657] I resolved and opened all sock addresses
2017/10/07 19:01:37 socat[3657] N starting data transfer loop with FDs [6,6] and [8,8]

check /dev to see if your new ttyUSB0 device is listed

cd /dev
ls

install the Openhab Z-wave addon
configure the serial port for the z-wave controller thing (node1) for /dev/ttyUSB0
restart openhab and enjoy z-wave connected over IP

UPDATE 14-01-2020 ** MULTIPLE SOCAT TO MULTIPLE SER2NET INSTANCES

I’ve noticed some discussion on running multiple socat connections from the OpenHAB machine to different Ser2Net enabled pi’s in larger or complex buildings and installations.

Up until now i did not have time to fiddle with this, but yesterday i ran into the same challenge… a large premises that required multiple Z-wave nodes to cover it entirely.

Please allow to me to post a short write up on how i made it work. Might not be the nicest way of doing it… but hey… it works

INSTRUCTIONS:
Please follow the instructions above for the first/initial Z-wave node deployment
For each additional Z-wave node, follow the steps below but make sure you increase the number for each additional node.

Create a copy of the socat file in /etc/init.d and name it socat2:

cd /etc/init.d
sudo cp socat socat2

make the new socat2 file executable:

sudo chmod+x /etc/init.d/socat2

Modify the socat2 file so it starts to behave as an additional socat instance (nr. 2 in this case)

sudo nano /etc/init.d/socat2
modify the following lines: (changes in bold)
NAME=socat2
SOCAT_DEFAULTS= ‘-d -d -d -lf /var/log/socat2.log’
. /etc/default/socat2.conf
–pidfile /var/run/socat2.pid \
start-stop-daemon --oknodo --stop --quiet --pidfile /var/run/socat2.pid --exec $DAEMON
rm -f /var/run/socat2.pid
for PIDFILE in ls /var/run/socat2.pid 2> /dev/null; do
status_of_proc -p /var/run/socat2.pid /usr/bin/socat socat && exit 0 || exit $?

Register the new socat2 instance (might require “systemctl deamon-reload”)

sudo update-rc.d socat2 defaults

Now let’s create a config file for the new socat2 instance to work with

sudo cp /etc/defaults/socat.conf /etc/defaults/socat2.conf

modify the contents of this file according to your new/additional ser2net connection

sudo nano /etc/default/socat2.conf

e.g. OPTIONS=“pty,link=/dev/ttyUSB1,raw,user=openhab,group=dialout,mode=777 tcp:172.16.5.7:3333”

Make sure you increment the /dev/ttyUSBx number with every additional connection you make!

once configured make sure to save the file and restart all socat instances

ctrl-x to exit nano and save the modified file
sudo service socat restart (this restarts the original socat instance)
sudo service socat2 restart (this restart the new additional socat instance)

if all went well you should now have 2 socat instances running

check with sudo service socat(2) status
check for multiple USB devices present /dev (should now have ttyUSB0 and ttyUSB1)
go ahead and setup a new z-wave controller in Paper UI and point it to the new USB interface (ttyUSB1)

Cheers

Roel

Udo_Hartmann · October 7, 2017, 8:34pm

not /dev/ttyUSB0 ?

snoekieboe · October 7, 2017, 8:41pm

That depends on your specific USB device… the Aeotec dongle present as /dev/ttyUSB0 whilst the device in my case, Z-wave.me USB1, present itself as /dev/ttyACM0

Udo_Hartmann · October 7, 2017, 8:50pm

But you did forward it to /dev/ttyUSB0!?!

snoekieboe · October 7, 2017, 8:54pm

This is what i share on the server:
3333:raw:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT

This is what i connect on the client:
OPTIONS=“pty,link=/dev/ttyUSB0,raw,user=openhab,group=dialout,mode=777 tcp:172.16.11.5:3333”

So yes… i forward /dev/ttyACM0 to /dev/ttyUSB0

Udo_Hartmann · October 7, 2017, 10:04pm

And openHAB is configured to use /dev/ttyACM0?

snoekieboe · October 8, 2017, 6:02am

no openhab is configured to use /dev/ttyUSB0.

in my case Raspberry PI3 shares Zwave device (dev/ttyACM0) on 172.16.11.5:3333.
Openhab server connect to 172.16.11.5:3333 and creates a serial port on /dev/ttyUSB0
Openhab is connected to /dev/ttyUSB0

mashborn · October 8, 2017, 9:21am

Interesting idea.
I tried it with an old raspi2 with my zwave stick - and it seems to work!
Is it possible to conncet more than 1 client to the zwave-server?

Michael

snoekieboe · October 8, 2017, 4:41pm

Yes you can connect multiple clients to the same server but be aware of the problems you might run into with multi openhabs sending data to the same z-wave device… you could end up with unwanted behaviour.

You could use such a setup for redundancy purposes:
1 x USB server using ser2net
2 x openhab client using socat of which 1 is active and the other one is standby

Udo_Hartmann · October 8, 2017, 5:55pm

So please update your posting (5th line from bottom) you wrote /dev/ttyACM0…

snoekieboe · October 8, 2017, 6:04pm

Djeezz… you could have just said so in your first message

Udo_Hartmann · October 8, 2017, 6:30pm

Sorry, I thought it would be clear (I had quoted the line…)

mashborn · October 9, 2017, 10:43am

Hi Roel,
your solution works great for me with zwave.
Now I would like to put my enocean stick together with the zwave-stick in the Pi.
What is the correct syntax to put 2 OPTIONS in the socat.conf?
As I understand I could provide the zwave over 3333 and the enocean over 3334 - correct?

Michael

rlkoshak · October 9, 2017, 3:05pm

Excellent tutorial! Thanks for contributing it to the community.

snoekieboe · October 9, 2017, 5:58pm

Exactly… add a new line to ser2net.conf with an incremental port number and new USB_port you would like to share and you’re golden!

przemas75 · October 13, 2017, 4:12pm

I have Rpi in ma living room with Z-wave stick and forwarding with USBIP package to virtualmachine in my basement where OpenHAB is installed.
Works very well, just need to prepare some automation once i reboot rpi.
http://usbip.sourceforge.net/

usbip in recent linux distros comes with kernel and not as external package like in my raspbian on Rpi.

happenpappen · October 13, 2017, 6:17pm

Hi Michael,

you may use systemd (which is part of current Raspian) to start multiple instances of socat. Just create a file “/etc/systemd/system/socat@.service” with this content:

[Unit]
Description=Multipurpose relay (SOcket CAT)
After=network.target

[Service]
EnvironmentFile=/etc/default/socat-%i.conf
Type=simple
PIDFile=/var/run/socat-%i.pid
ExecStart=/usr/bin/socat -L/var/run/socat-%i.pid $SOCAT_DEFAULTS $SOCAT_CONNECTION
ExecStop=/bin/kill -SIGKILL $MAINPID

[Install]
WantedBy=multi-user.target

and reload systemd configuration with:

systemctl daemon-reload

The configuration of a port goes into a file called “/etc/default/socat-<name of Device>.conf” - for example “/etc/default/socat-ttyUSB37.conf”, if you want to create a device /dev/ttyUSB37. The contents of the file are like this:

SOCAT_DEFAULTS=-d -d -s -lf /var/log/socat.log

SOCAT_CONNECTION=pty,link=/dev/ttyUSB37,raw,user=openhab,group=dialout,mode=777 tcp:<ip-address>:<port>

Just replace <ip-address> and <port> with the ip address and port of the ser2net server. The command:

systemctl enable socat@ttyUSB37

will enable the service at system startup and:

systemctl start socat@ttyUSB37

will start it directly. If you want to add another socat daemon for device /dev/ttyUSB38 which is mapped to another ip address/port combination, just copy the file /etc/defaults/socat-ttyUSB37.conf to /etc/defaults/socat-ttyUSB38.conf and edit it to match the new configuration. After this enable the service at system startup with:

systemctl enable socat@ttyUSB38

kind regards,

Christoph

tinuva · October 15, 2017, 5:07pm

Just want to say, +1 for this guide.

Using it to forward the usb to serial device for my DSC alarm and works 100%. Now I can move the adapter to a Pi Zero and let it work over wifi instead of a cable hanging through the roof!

higgers · October 22, 2017, 8:20am

Does anyone have any experience with a Windows frontend for this? I’d like to be able to shutdown my OpenHAB Ubuntu VM and then connect to my USB Zwave stick on Windows in order to do a backup of the stick.

snoekieboe · October 22, 2017, 2:16pm

There are 2 windows alternatives available to my knowledge:

USBIP - http://usbip.sourceforge.net/
virtualhere - https://www.virtualhere.com/

I have personally tried virtualhere and it worked perfectly fine