Forwarding of serial and USB ports over the network to openHAB

bluetooth
serial
zwave
Tags: #<Tag:0x00007f0156c8f088> #<Tag:0x00007f0156c8ef20> #<Tag:0x00007f0156c8ec78>

(Thomas Bail) #1

As promised, here comes the tutorial on forwarding serial or USB ports to openHAB. This tutorial is the result of my efforts to use Vlad Kolotov’s Bluetooth binding (kudos to @vkolotov ) for my MiFlora plant sensors.

Problem description
Services such as Zwave or Bluetooth are limited in their transmission range for technical reasons. To improve the range, the corresponding adapters can be connected with extension cables. However, if the cable length is limited or the installation of cables is not possible, this does not lead to the desired success.

In order to solve this problem, there are two solutions, depending on the task, which are briefly presented below. For serial ports the combination of ser2net and socat and for USB ports usbip is a possible solution.

Before we start, here are some definitions. The server on which openHAB runs is always referred to as the client and the server to which the respective adapter is connected is referred to as the server.

I use a first generation Raspberry Pi with DietPi as OS for the server (which comes with the adapter). And openhab runs on a Raspberry Pi 3 with SSD and a manually installed openHabian and Raspbian Stretch (Kudos to @ThomDietrich)

Solution 1: USBIP for non-serial USB ports
In contrast to the ser2net / socat combination, this solution can be used for any USB port. Disadvantage: I got it to run on the server (the one with the adapter), but not on the openHAB site (the client) yet.

The guide is based on the blog entry “Linux, RPi and USB over IP” by Piotr Król and is a little shortened here.

Erste Schritt ist das Aufsetzen des Servers, an dem der USB Adapter angeschlossen ist. Hierzu muss zunächst usbip installiert werden. We omit the check whether the kernel is prepared for it, since this is usually the case in current kernels

sudo apt install usbip

Then the usbip daemon is started and the matching USB adapter is identified and shared.

sudo modprobe usbip-core
sudo modprobe usbip-host
sudo usbipd -D

Now we can display the USB adapters.

user@server:~ $ usbip list -l
 - busid 1-1.1 (0424:ec00)
   Standard Microsystems Corp. : SMSC9512/9514 Fast Ethernet Adapter (0424:ec00)

 - busid 1-1.3 (0a12:0001)
   Cambridge Silicon Radio, Ltd : Bluetooth Dongle (HCI mode) (0a12:0001)

The one we are looking for has the bus id 1-1.-3 and is a bluetooth adapter. With a bind it is bound and shared on the network.

sudo usbip --debug bind -b 1-1.3

And now we come to the client side, the openhab server. Here too, usbip must be installed. After that we can check if our enabled adapter is visible. IP address should be replaced with the IP address of your server.

sudo usbip list -r <IP address>

the result of the list command should be something like this.

Exportable USB devices
======================
 - 10.42.42.205
      1-1.3: Cambridge Silicon Radio, Ltd : Bluetooth Dongle (HCI mode) (0a12:0001)
           : /sys/devices/platform/soc/20980000.usb/usb1/1-1/1-1.3
           : Wireless / Radio Frequency / Bluetooth (e0/01/01)

The Bluetooth adapter is now shared on the network and can be integrated with the following commands. Attention: On a Raspberry I did not succeed in integrating the adapter.

sudo modprobe vhci-hcd
sudo usbip attach -r <IP address> -b 1-1.3

For the removal of the connection refer to the above mentioned blog article “Linux, RPi and USB over IP updated

Solution 2: ser2net and socat for serial USB ports
If you now have a USB adapter that looks like a serial interface, e.g. the Bluetooth adapter from BlueGiga or a Z-Wave adapter like the one from Aeotech, the solution can be the combination of ser2net and socat.

ser2net is installed on the server and shares the serial port on the network and socat is installed on the client and provides the interface there.

The guide is essentially a compilation of various posts and your own research. Therefore kudos to @snoekieboe, @snackboards, 2devnull

As with the usbip solution, we start with the server. My server is a Raspberry 1 with DietPi () and a BlueGiga Bluetooth adapter.

First we check if the adapter works. For this there is a Python script from BlueGiga which queries the adapter.

apt install python-serial
wget https://raw.github.com/jrowberg/bglib/master/Python/Examples/bled112_scanner.py
chmod +x ./bled112_scanner.py
./bled112_scanner.py

If the output somehow looks like this, then the adapter already works.

user@server:~ $ ./bled112_scanner.py -p /dev/ttyACM0
================================================================
BLED112 Scanner for Python v2013-04-07
================================================================
Serial port:    /dev/ttyACM0
Baud rate:      115200
Scan interval:  200 (250.00 ms)
Scan window:    200 (250.00 ms)
Scan type:      Passive
UUID filters:   None
MAC filter(s):  None
RSSI filter:    None
Display fields: - Time
                - RSSI
                - Packet type
                - Sender MAC
                - Address type
                - Bond status
                - Payload data
Friendly mode:  Disabled
----------------------------------------------------------------
Starting scan for BLE advertisements...
1529261718.724 -90 2 11880C3159C7 1 255 1EFF0600010920029D1D359B9D58E977F053975C0B48A5A2756C7CD8B470DB
1529261718.837 -87 2 11880C3159C7 1 255 1EFF0600010920029D1D359B9D58E977F053975C0B48A5A2756C7CD8B470DB
1529261719.129 -90 2 11880C3159C7 1 255 1EFF0600010920029D1D359B9D58E977F053975C0B48A5A2756C7CD8B470DB
1529261719.185 -87 0 DCD93522D048 1 255 0201060302AFFE06094E30305558
1529261719.244 -87 2 11880C3159C7 1 255 1EFF0600010920029D1D359B9D58E977F053975C0B48A5A2756C7CD8B470DB
1529261719.290 -62 0 FB45E00C3C13 1 255 0201060302AFFE06094E30343130
1529261719.554 -87 2 11880C3159C7 1 255 1EFF0600010920029D1D359B9D58E977F053975C0B48A5A2756C7CD8B470DB
1529261719.622 -85 0 C47C8D65B8A1 0 255 020106030295FE141695FE71209800BEA1B8658D7CC40D0910026208

No output or an error message? By default, /dev/ttyACM0 is used as the port. If this is not the correct one, it can be specified on the command line.

./bled112_scanner.py -p /dev/ttyACM0

The desired serial port is then made available on the network using the ser2net tool. To do this, ser2net must first be installed.

sudo apt install ser2net

Then edit the configuration file.

sudo nano /etc/ser2net.conf

and then add the linet as follows.

3001:raw:600:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT

All other lines are commented out. This line is then used to make our serial port available on the network on port 3001. If you like you can also try a higher speed than 115200 baud. After saving, restart the service.

service ser2net restart

With netstat you can see if ser2net on port 3001 listens for a connection.

netstat -antp

And now we come to the client side, the openhab server. The client is not set up quite as quickly as the server. First, the socat package must be installed.

sudo apt install socat

Now the configuration follows, so that we can easily integrate several shared ports. The serial port should be accessible under /dev/ttyNET0. First we create the service file

sudo nano /etc/systemd/system/socat@.service

The content should look like this.

[Unit]
Description=Multipurpose relay (SOcket CAT)
After=network.target
StartLimitIntervalSec=0
StartLimitBurst=3000

[Service]
Restart=always
RestartSec=3
EnvironmentFile=/etc/default/socat-%i.conf
Type=simple
PIDFile=/var/run/socat-%i.pid
ExecStart=/usr/bin/socat -L/var/run/socat-%i.pid $SOCAT_DEFAULTS $SOCAT_CONNECTION
ExecStop=/bin/kill -SIGKILL $MAINPID

[Install]
WantedBy=multi-user.target

After saving the configuration is reloaded

systemctl daemon-reload

Now comes the configuration file for the serial device /dev/ttyNET0

sudo nano /etc/default/socat-ttyNET0.conf

The content should look like this.

SOCAT_DEFAULTS=-d -d -s -lf /var/log/socat.log

SOCAT_CONNECTION=pty,link=/dev/ttyNET0,raw,user=openhab,group=dialout,mode=777 tcp:<ip-address>:<port>

You should replace IP address and port with the IP of the server and the corresponding port and enable and start it after saving for system startup.

systemctl enable socat@ttyNET0
systemctl start socat@ttyNET0

The same can now be repeated with wide configurations until all remote interfaces are connected. With the script of BlueGiga you can check on the client whether data arrive on the new interface now.

Now the new serial interface is available on the openHAB server, but two steps are still missing to use it in openHAB. In the first step, the serial port must be built into the openhab service options so that Java can access it.

sudo nano /etc/default/openhab2

Adjust the EXTRA_JAVA_OPTS accordingly

EXTRA_JAVA_OPTS="-Dgnu.io.rxtx.SerialPorts=/dev/ttyUSB0:/dev/ttyS0:/dev/ttyS2:/dev/ttyACM0:/dev/ttyAMA0:/dev/ttyNET0"

Then you have to log into the Karaf console (see also the openHAB Documentation) and install the serial transport. Simply enter the following.

feature:install openhab-transport-serial
logout

And finally, the openHAb should be restarted. Now there are some tricks with udev rules left, but I haven’t implemented them yet.

Questions, feedback, additions, corrections etc. are always welcome and I hope the tutorial will help.

Thomas


Share Z-wave dongle over IP (USB over IP using ser2net / socat ) guide
3rd Party Bluetooth Binding. Beta testers needed
3rd Party Bluetooth Binding. Beta testers needed
3rd Party Bluetooth Binding. Beta testers needed
New openHab2 EnOcean binding
Share Z-wave dongle over IP (USB over IP using ser2net / socat ) guide
3rd Party Bluetooth Binding. Beta testers needed
3rd Party Bluetooth Binding. Beta testers needed
(Roel) #2

What exactly is the benefit of this solution over the Ser2net/socat solution?
Are there any differences?

You are saying USBIP is for USB devices but that also works just fine with Ser2Net/socat

Thanks


(Thomas Bail) #3

The difference is that not all devices could accesses using a serial connection.

Example: The Bluegiga Blutooth Adapter is seriell, so both solutions may work, but a Logilink Bluetooth adapter ist not a serial adapter so the ser2net/socat solution will not work.

Another example ist that with the usbip solution you may link a USB Memory stick over the net or other usb devices like USb Soundcards etc.

If the ser2net/socat solution ist fine for you do not heasitate to use it.

Hope this helps


Cheap Bluetooth antenna with raspberry pi zero
Cheap Bluetooth antenna with raspberry pi zero
(Thomas Bail) #4

Corrected above some minor errors while rebuilding my own system :slight_smile:


(Michael Cumming) #5

Which DietPI OS should I install on a PI Zero?


(Stefan) #6

Did you got usbip working on openhabian?

I’m stuck on:

sudo usbip --debug attach -r 10.0.1.1 -b 3-2.4.1
usbip: debug: usbip.c:141:[run_command] running command: `attach'
libusbip: error: udev_device_new_from_subsystem_sysname failed
usbip: error: open vhci_driver
usbip: error: query

UPDATE: I found a solution. Default usbip on openHABian is way too old (2.0+4.9.82-1+deb9u3+rpi1).

When you install the latest version (2.0+4.18.6-1) it works perfect. You can find it here.


(Thomas Bail) #7

Really great!! i was looking for a solution a long time. It works perfect the binding immidiately found the adapter

Thanks


(Kris K) #8

Hi Team

can someone assist? as soon as ser2net is restarted, socat exits and then openhab fails on zwave

As this thread suggests, its not binding related and that a restart is the only way to fix. So what happens when you restart the host that serves the USB? You must also restart Oh2?! surely not


06:44:40.562 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.
06:44:42.571 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.
06:44:54.572 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.
06:44:56.579 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.
06:45:08.580 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.
06:45:10.490 [INFO ] [smarthome.event.ItemStateChangedEvent] - OpenHAB_Cpu_Load1                                                        changed from 0.1 to 0.3
06:45:10.497 [INFO ] [smarthome.event.ItemStateChangedEvent] - OpenHAB_Cpu_Load5                                                        changed from 0.2 to 0.3
06:45:10.502 [INFO ] [smarthome.event.ItemStateChangedEvent] - OpenHAB_Sensors_C                                                       puTemperature changed from 30.0 to 29.0
06:45:10.507 [INFO ] [smarthome.event.ItemStateChangedEvent] - OpenHAB_Cpu_Load                                                        changed from 2.6 to 2.4
06:45:10.589 [ERROR] [ding.zwave.handler.ZWaveSerialHandler] - Got I/O exception                                                        Input/output error in writeArray during sending. exiting thread.

kris@openhab2:/etc/systemd/system$ systemctl restart socat
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to restart 'socat.service'.
Authenticating as: kris
Password:
==== AUTHENTICATION COMPLETE ===
kris@openhab2:/etc/systemd/system$ tail -f /var/log/socat.log
2018/10/03 06:44:50 socat[20383] I openpty({6}, {7}, {"/dev/pts/1"},,) -> 0
2018/10/03 06:44:50 socat[20383] N PTY is /dev/pts/1
2018/10/03 06:44:50 socat[20383] N opening connection to AF=2 192.168.0.12:3333
2018/10/03 06:44:50 socat[20383] I starting connect loop
2018/10/03 06:44:50 socat[20383] I socket(2, 1, 6) -> 8
2018/10/03 06:44:50 socat[20383] N successfully connected from local address AF=2 192.168.0.3:36958
2018/10/03 06:44:50 socat[20383] I resolved and opened all sock addresses
2018/10/03 06:44:50 socat[20383] N starting data transfer loop with FDs [6,6] and [8,8]
2018/10/03 06:44:51 socat[20383] I transferred 63 bytes from 8 to 6
2018/10/03 06:44:51 socat[20383] I transferred 110 bytes from 6 to 8

The only way to fix it is to restart openhab2

Surely this isnt right.


(Thomas Bail) #9

Maybe the problem is that you do not start ser2net as root try it with

sudo service xxx restart

(Kris K) #10

Thanks Thomas, is there a way I can determine if its been started as root?


(Thomas Bail) #11

typicalls on systemstartup it should be started as root. Have you checked all your config files if the restart option is there?


(J Wee) #12

is the USB IP confirmed working with Aeotec z wave Gen 5 stick?
Would it also work with OH installed as as docker?

I am asking as ser2net and socat does not work with OH as a docker


(Thomas Bail) #13

usbip has several problems. I didn’t work on my raspi 3 with openhabian (Stretch), i tried it in a VM with debian (Buster) and i sometimes work.

Try it. If it is working fine, but then do not change anything.

Thats my experience


(J Wee) #14

I kept getting an error when I typed in this command
“sudo systemctl enable usbipd.service”
Does it have something to do with the usbip being old? how do I install a newer one?


(Ben AZ) #15

Hi, is there an easy way to forward 2 USB ports at the same time? I have a ZigBee and Zwave adapter and want to forward both to OpenHABian. The only solution I found was to have two insances of socat running which is quite a little awkward i.m.h.o.
Any ideas?