Share Z-wave dongle over IP (USB over IP using ser2net / socat ) guide


(Roel) #1

Hi,

Recently i’ve been working on virtualising all IT systems in my home. I’ve virtualised my NAS, Firewall, Router, Network Video Recorder and ofcourse my Openhab2 setup.
During this virtualisation effort i ran into a few issues with regards to passing through my Z-wave.me USB1 z-wave dongle to a virtual machine.

Community member Rikoshak pointed me to an alternative solution where i would leave my USB dongle connected to a physical machine (e.g. Raspberry Pi) and share the USB device over the network.
Not only would this solve my passthru issues in VMware esxi 6.5 but this would also allow me to position my Z-wave dongle in the most optimum location, away from my server that is located in the basement.

Since i’ve got this up and running, and information was hard to find, i would like to share my steps so you can benefit as well.

General architecture overview:

Untitled Diagram

To make this work we have some work to do:

  • install & configure “ser2net” in the RPI3 who is going to serve the USB z-wave dongle
  • install & configure “socat” on the virtual Openhab instance as client (Ubuntu 16.04 in my case)
  • Make sure all services on both machines auto-start and do so with the correct priveleges

The following section will show you all the steps you need to take on both the server and the client:

USB device server (RPI3 with Zwave.me USB dongle IP=172.16.11.5 replace with your IP)

  • I started with a fresh install of Debian Jessie (installed using noobs installer)
  • plug in your USB dongle and verify the name of the port (ttyACM0 in my case)
cd /dev
ls

autofs           gpiochip1  loop7               null   ram5     tty    tty21  tty35  tty49  tty62      vcs    vhci
block            gpiochip2  loop-control        ppp    ram6     tty0   tty22  tty36  tty5   tty63      vcs1   watchdog
btrfs-control    gpiomem    mapper              ptmx   ram7     tty1   tty23  tty37  tty50  tty7       vcs2   watchdog0
bus              hwrng      mem                 pts    ram8     tty10  tty24  tty38  tty51  tty8       vcs3   zero
cachefiles       initctl    memory_bandwidth    ram0   ram9     tty11  tty25  tty39  tty52  tty9       vcs4
char             input      mmcblk0             ram1   random   tty12  tty26  tty4   tty53  **`ttyACM0`**    vcs5
console          kmsg       mmcblk0p1           ram10  raw      tty13  tty27  tty40  tty54  ttyAMA0    vcs6
cpu_dma_latency  log        mmcblk0p2           ram11  rfkill   tty14  tty28  tty41  tty55  ttyprintk  vcsa
cuse             loop0      mmcblk0p5           ram12  serial   tty15  tty29  tty42  tty56  uhid       vcsa1
disk             loop1      mmcblk0p6           ram13  serial1  tty16  tty3   tty43  tty57  uinput     vcsa2
fb0              loop2      mmcblk0p7           ram14  shm      tty17  tty30  tty44  tty58  urandom    vcsa3
fd               loop3      mqueue              ram15  snd      tty18  tty31  tty45  tty59  vc-cma     vcsa4
full             loop4      net                 ram2   stderr   tty19  tty32  tty46  tty6   vchiq      vcsa5
fuse             loop5      network_latency     ram3   stdin    tty2   tty33  tty47  tty60  vcio       vcsa6
gpiochip0        loop6      network_throughput  ram4   stdout   tty20  tty34  tty48  tty61  vc-mem     vcsm
  • now we need to install ser2Net", the application that will share ttyACM0 over the network
sudo apt-get install ser2net
  • Once the installer is finished we can configure ser2net
sudo nano /etc/ser2net.conf
  • This opens the ser2net.conf file in the nano editor.
  • add the following line to the bottom of the file (modify according to your setup)
3333:raw:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT
  • this line tells ser2net to share device /dev/ttyACM0 on TCP port 333 at 115200 baud

  • press ctrl+x to save the modified file and exit the nano editor

  • next up is to restart ser2net to apply the modified condiguration

sudo service ser2net restart
  • Check to see of ser2net is sharing the device using netstat… look for an entry that contains 0.0.0.0:3333
netstat -antp
  • if the TCP entry is shown you have successfully finished the server side setup of this USB over IP adventure. Ser2net is already installed as a service and will automatically start after a reboot or cold start.

USB device client (OpenHAB running on a virtual linux machine IP=172.16.11.4 replace with you IP)

The Openhab machine will be the client connecting to the shared USB device on the RPI3. In order to do so we need to install ser2net’s counterpart called “socat”.

  • install socat on the openhab machine
sudo apt-get install socat
  • download and install an init.d script that allows socat to run as a service and auto-start at startup
git clone https://github.com/asaif/socat-init.git

sudo cp socat-init/socat /etc/init.d/
sudo chmod +x /etc/init.d/socat
sudo update-rc.d socat defaults
sudo cp socat-init/defaults/socat.conf /etc/default/
  • now we have all the necessary scripts installed it’s time to configure them:
sudo nano /etc/default/socat.conf
  • add the following line to “socat.conf”
  • it creates a new serial USB device known as ttyUSB0
  • it creates the device as user openhab and adds it to the dialout group
  • it tells socat to connect to the RPI3 USB server using it’s IP and port 333
OPTIONS="pty,link=/dev/ttyUSB0,raw,user=openhab,group=dialout,mode=777 tcp:172.16.11.5:3333"
  • ctrl+x to save and exit nano editor

  • we are no ready to (re)start socat and check if we have successfully connected to the z-wave dongle using the network

sudo service socat restart

check the logs to see if connection was successful

tail -f /var/log/socat.log
2017/10/07 19:01:37 socat[3657] I setting option "symbolic-link" to "/dev/ttyUSB0"
2017/10/07 19:01:37 socat[3657] I setting option "raw"
2017/10/07 19:01:37 socat[3657] I openpty({6}, {7}, {"/dev/pts/1"},,) -> 0
2017/10/07 19:01:37 socat[3657] N PTY is /dev/pts/1
2017/10/07 19:01:37 socat[3657] N opening connection to AF=2 172.16.11.5:3333
2017/10/07 19:01:37 socat[3657] I starting connect loop
2017/10/07 19:01:37 socat[3657] I socket(2, 1, 6) -> 8
2017/10/07 19:01:37 socat[3657] N successfully connected from local address AF=2 172.16.11.4:40282
2017/10/07 19:01:37 socat[3657] I resolved and opened all sock addresses
2017/10/07 19:01:37 socat[3657] N starting data transfer loop with FDs [6,6] and [8,8]
  • check /dev to see if your new ttyUSB0 device is listed
cd /dev
ls
  • install the Openhab Z-wave addon
  • configure the serial port for the z-wave controller thing (node1) for /dev/ttyUSB0
  • restart openhab and enjoy z-wave connected over IP :slight_smile:

This is a rough and fast write up, i will try to polish it a bit over the next coming days.

br

Roel


2 ZWAVE Sticks?
Extend zwave to another building
Openhab2+ESXI 6.5+zwave.me usb dongle
openHAB or Vera?
Hardware recommendations
IP to Serial Control
Zwave Gateway Binding without USB
3rd Party Bluetooth Binding. Beta testers needed
openHAB project - considerations and questions of a newbe
I have two identical z-wave controllers. I can't make serial USB ports persistent via symlinks
[SOLVED] MQTT-Eventbus: only slave should send its items?
Controlling devices without gateways? - Raspberry Pi
Which should I use to run openHAB... Raspbery Pi2 or QNAP NAS?
Signal strength with Razberry + Zwave
New binding - Rego 6xx based heat pumps
Received update triggered twice
New Z-Way Binding
Port JSR223 bundle to openHAB 2
Best hardware platform
Ethernet Z-wave controler
Need remote Z-Wave Gateway, will my current HW do?
IP Z-Wave Bridge
Open Source Thing Gateway / Hub
RaZberry as client device
3rd Party Bluetooth Binding. Beta testers needed
Qnap TS431P an Aeon labs usb
QNAP Virtual Station for openhab?
Extending z-wave with secondary controller - Using Razberry with Aeotec z-stick
MQTT in layman’s terms, best practices, use cases?
(Udo Hartmann) #2

not /dev/ttyUSB0 ?


(Roel) #3

That depends on your specific USB device… the Aeotec dongle present as /dev/ttyUSB0 whilst the device in my case, Z-wave.me USB1, present itself as /dev/ttyACM0


(Udo Hartmann) #4

But you did forward it to /dev/ttyUSB0!?!


(Roel) #5

This is what i share on the server:
3333:raw:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT

This is what i connect on the client:
OPTIONS=“pty,link=/dev/ttyUSB0,raw,user=openhab,group=dialout,mode=777 tcp:172.16.11.5:3333”

So yes… i forward /dev/ttyACM0 to /dev/ttyUSB0


(Udo Hartmann) #6

And openHAB is configured to use /dev/ttyACM0?


(Roel) #7

no openhab is configured to use /dev/ttyUSB0.

in my case Raspberry PI3 shares Zwave device (dev/ttyACM0) on 172.16.11.5:3333.
Openhab server connect to 172.16.11.5:3333 and creates a serial port on /dev/ttyUSB0
Openhab is connected to /dev/ttyUSB0


(Michael Aschenborn) #8

Interesting idea.
I tried it with an old raspi2 with my zwave stick - and it seems to work!
Is it possible to conncet more than 1 client to the zwave-server?

Michael


(Roel) #9

Yes you can connect multiple clients to the same server but be aware of the problems you might run into with multi openhabs sending data to the same z-wave device… you could end up with unwanted behaviour.

You could use such a setup for redundancy purposes:
1 x USB server using ser2net
2 x openhab client using socat of which 1 is active and the other one is standby


(Udo Hartmann) #10

So please update your posting (5th line from bottom) :wink: you wrote /dev/ttyACM0…


(Roel) #11

Djeezz… you could have just said so in your first message :wink:


(Udo Hartmann) #12

Sorry, I thought it would be clear (I had quoted the line…)


(Michael Aschenborn) #13

Hi Roel,
your solution works great for me with zwave.
Now I would like to put my enocean stick together with the zwave-stick in the Pi.
What is the correct syntax to put 2 OPTIONS in the socat.conf?
As I understand I could provide the zwave over 3333 and the enocean over 3334 - correct?

Michael


(Rich Koshak) #14

Excellent tutorial! Thanks for contributing it to the community.


(Roel) #15

Exactly… add a new line to ser2net.conf with an incremental port number and new USB_port you would like to share and you’re golden!


(Przemas75) #16

I have Rpi in ma living room with Z-wave stick and forwarding with USBIP package to virtualmachine in my basement where OpenHAB is installed.
Works very well, just need to prepare some automation once i reboot rpi.
http://usbip.sourceforge.net/

usbip in recent linux distros comes with kernel and not as external package like in my raspbian on Rpi.


(happenpappen) #17

Hi Michael,

you may use systemd (which is part of current Raspian) to start multiple instances of socat. Just create a file “/etc/systemd/system/socat@.service” with this content:

[Unit]
Description=Multipurpose relay (SOcket CAT)
After=network.target

[Service]
EnvironmentFile=/etc/default/socat-%i.conf
Type=simple
PIDFile=/var/run/socat-%i.pid
ExecStart=/usr/bin/socat -L/var/run/socat-%i.pid $SOCAT_DEFAULTS $SOCAT_CONNECTION
ExecStop=/bin/kill -SIGKILL $MAINPID

[Install]
WantedBy=multi-user.target   

and reload systemd configuration with:

systemctl daemon-reload

The configuration of a port goes into a file called “/etc/default/socat-<name of Device>.conf” - for example “/etc/default/socat-ttyUSB37.conf”, if you want to create a device /dev/ttyUSB37. The contents of the file are like this:

SOCAT_DEFAULTS=-d -d -s -lf /var/log/socat.log

SOCAT_CONNECTION=pty,link=/dev/ttyUSB37,raw,user=openhab,group=dialout,mode=777 tcp:<ip-address>:<port>

Just replace <ip-address> and <port> with the ip address and port of the ser2net server. The command:

systemctl enable socat@ttyUSB37

will enable the service at system startup and:

systemctl start socat@ttyUSB37

will start it directly. If you want to add another socat daemon for device /dev/ttyUSB38 which is mapped to another ip address/port combination, just copy the file /etc/defaults/socat-ttyUSB37.conf to /etc/defaults/socat-ttyUSB38.conf and edit it to match the new configuration. After this enable the service at system startup with:

systemctl enable socat@ttyUSB38

kind regards,

Christoph


(Tinuva) #18

Just want to say, +1 for this guide.

Using it to forward the usb to serial device for my DSC alarm and works 100%. Now I can move the adapter to a Pi Zero and let it work over wifi instead of a cable hanging through the roof!


(Steve Higton) #19

Does anyone have any experience with a Windows frontend for this? I’d like to be able to shutdown my OpenHAB Ubuntu VM and then connect to my USB Zwave stick on Windows in order to do a backup of the stick.


(Roel) #20

There are 2 windows alternatives available to my knowledge:

I have personally tried virtualhere and it worked perfectly fine