[SOLVED] [Zwave, Zigbee, ...] RFC2217 remote serial port HowTo?

vossivossi · December 29, 2020, 2:17pm

Yes, it does. I have defined the the Controller thing via a textual config file:

Bridge zwave:serial_zstick:ZWAB "ZWAB 2217 " [ port="rfc2217://192.168.176.77:7000", controller_softreset="false", controller_master="true", heal_enable="true"]

Bruce_Osborne · December 29, 2020, 2:40pm

Does it work through the UI too?

curlyel · December 29, 2020, 3:06pm

Can you share your experience with overall reliability of ZWave via rfc2217 -especially the re-connect behaviour to the remote controller in case of any disruption on network level?

Is it re-connecting/recovering from such failures automatically?

Rafal · December 29, 2020, 3:19pm

Assuming you are asking generically (not specifically OH3) I find the ZWave binding over rfc2217 in OH2 to be outstanding in its reliability. I have run it for 6 months with only a need to restart on 2 occasions when ZWave items went offline, other than after a messed up minor OH upgrade. It has exceeded my expectations. FYI, I run OH2 in a jail on FreeBSD/FreeNAS on a “proper” server, while the Aotec ZWave Gen5 stick is in a remotely located RPi.

Bruce_Osborne · December 29, 2020, 3:33pm

Was your stick defined through HABmin / Paper UI or text file? If UI I assume you needed to set some EXTRA_JAVA_OPTS ?

Rafal · December 29, 2020, 3:50pm

Indeed, my /etc/rc.conf has the recommended Java options (running FreeBSD 12.2):

cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
# Disabled for OH2
ipv6_activate_all_interfaces="NO"
hostname="openhab"
ifconfig_epair0b="SYNCDHCP"
sshd_enable="YES"

# More OH settings
openhab2_java_opts="-Djava.net.preferIPv4Stack=true"
openhab2_enable="YES"

mysql_enable="YES"

but other than that, after a fair amount of initial wrestling with earlier versions of rfc2217 and unrelated nrjavaserial issues the install is pretty standard using pkg install openhab2. I have a mixture of UI and text file definitions, but the ZWave stick was done using the Paper UI alone. There was no need to add the rfc2217 port definition to Java opts since a commit a few months ago that followed the initial birthing pains.

Overall, it is amazingly stable and really fast, and the rfc2217 has dealt well with an odd power failure of the UPS and there is no need to synchronise reboots etc.

Bruce_Osborne · December 29, 2020, 3:51pm

Thanks.
I really need to test this out. If I understand correctly, that physically untethers the stick from the OH server hardware.

Rafal · December 29, 2020, 3:55pm

Correct! That has many advantages, notably allowing a better placement of the ZWave antenna in the house while the OH2 server runs in its dedicated, IT-friendly part of the home. Also, it meant I could run OH on FreeBSD. As there is little support for the ZWave stick on FreeBSD, the stick is the only thing that I run on Linux in the house on its dedicated RPi, while everything else runs on FreeBSD. RFC2217 made that possible by also decoupling the OSes, so to speak. Might be of use to someone even on the same hardware, theoretically.

PS. I should clarify that I am also running RFXCOM on the same RPi, also presenting it over rfc2217 to the remote OH2 server. In other words, my RPi functions as an antenna for both.

Bruce_Osborne · December 29, 2020, 4:11pm

Right now mu OH is on a Hyper-V VM. The stick drivers are in the host server OS and the resulting serial port is mapped to the Linux VM. Being able to remove that mapping could permit rebooting the VM without needing to tear down the mapping first.

vossivossi · December 29, 2020, 4:29pm

Honestly I don’t know with OH3. I tested it some time ago with OH2.5 and there it did not work (as PaperUI only offered the physical COM-Ports and would not allow any alternative entry for the Port). As it worked fine with a textual .things-file I kept this solution.

Unfortunately I am not happy with that. After a network interruption I have to restart the Z-Wave binding. Otherwise it won’t recover by itself.
The revocery was better with the previous solution I had with ser2net on a RPI (serving as “antenna”) and hub4com on my Windows OH server.
But now I anyhow changed again the setup and also run OH3 on the “Antenna”-RPI and connect to the items via the new Remote-OH-Binding (and no longer user RFC2217). The advantage with this new setup is that my Z-Wave networks can run through when I update the main OH instance and are immediately available after upgrade (no discovery, no initialization of devices etc.)

Yes, absolutely. That was also exactly my reason to use it. Especially as I need 3 ZWave antennas within my installation.

curlyel · December 29, 2020, 4:29pm

Many thanks Rafal,
happy to hear that. Even though I started this thread, I still have my remote serial
adaptors on a socat/ser2net combination after of some initial issues with rfc2217.

Because of the promising posts here I currently reconsider moving to rfc2217 while migration to openHAB3 (of course )

That’s very similar to my setup and mainly for the same reasons
Just additional Zigbee and Bluegiga (unused currently) and as said yet - with socat instead of the built-in rfc2217. It has been very stable for years! Admittedly, I dealt with some scripts to solve the re-connect in case the tcp connection got lost.

So from the stability perspective I could stay with socat. But rfc2217 would help me getting rid of some aditional scripts and systemd services and therefore slim down the setup

Therefore:
May I ask You too about your re-connect experiences when using rfc2217 ?

curlyel · December 29, 2020, 4:39pm

Mmmh. That’s what I feared

Good point! Maybe I should consider this too…

Only drawback would be, that my “Antenna-Pi’s” no longer can run off a read-only filesystem…

Anyway: Thank’s for sharing your experiences!

Rafal · December 29, 2020, 4:49pm

Hmm, I guess I have not been “lucky” enough to experience enough disconnections over the 6 months to notice any issues. I blame my networking. On the other hand, clearly the rfc2217 experience @vossivossi had was much poorer than mine—I am sorry to hear. I suppose it all depends on other aspects of the set-up.

curlyel · December 29, 2020, 5:01pm

Fully agree. Same here … usually

But you know, bad things happen always at the worst time…
Means for my setup: “When I’m not available to fix it quickly”

So some sort of “auto-recovery-procedure” is needed…

cesarvog · January 5, 2021, 8:30pm

At least in the released version of OH3, yes, it does. I have just done that in the z-wave thing GUI.
For it to work, on the remote system where the z-wave interface is connected, just make sure you define everything correctly in the /etc/ser2net.conf file, as per post 31 above ([SOLVED] [Zwave, Zigbee, ...] RFC2217 remote serial port HowTo? - #31 by mvbergen)

tardismechanic · January 13, 2021, 11:35pm

Has anyone gotten this to work with Zigbee? I’m testing OH3 in a docker container in a VM on my server and while the ZWave binding works great connecting to rfc2217://192.168.0.5:3001, the Zigbee binding configured for rfc2217://192.168.0.5:3002 gives me this in the debug log. The box at 192.168.0.5 is a PC that currently has my OH2 installation (OH2 service stopped) and has ser2net running with this config and I have verified with netstat that both ports 3001 and 3002 are open and listening

3001:telnet:0:/dev/ttyUSB-ACM0:115200 8DATABITS NONE 1STOPBIT remctl
3002:telnet:0:/dev/ttyUSB0:57600 8DATABITS NONE 1STOPBIT remctl

Debug log:

2021-01-13 15:32:59.995 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - ZigBeeNetworkManager shutdown: networkState=OFFLINE
2021-01-13 15:32:59.996 [DEBUG] [atabase.ZigBeeNetworkDatabaseManager] - Data store: Deferred Write Time set to 0ms
2021-01-13 15:32:59.997 [DEBUG] [atabase.ZigBeeNetworkDatabaseManager] - Data store: Deferred Write Time set less than Max Deferred Write Time
2021-01-13 15:32:59.998 [DEBUG] [atabase.ZigBeeNetworkDatabaseManager] - Data store: Max Deferred Write Time set to 0ms
2021-01-13 15:32:59.999 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to SHUTDOWN
2021-01-13 15:33:00.000 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - null: networkStateUpdated called with state=SHUTDOWN
2021-01-13 15:33:00.000 [DEBUG] [atabase.ZigBeeNetworkDatabaseManager] - Data store: Shutdown
2021-01-13 15:33:00.000 [DEBUG] [zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Shutdown
2021-01-13 15:33:00.001 [DEBUG] [zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Shutdown frameHandler is null
2021-01-13 15:33:00.001 [DEBUG] [transaction.ZigBeeTransactionManager] - Transaction Manager: Shutdown
2021-01-13 15:33:00.004 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Broadcast: Queue shutdown
2021-01-13 15:33:00.004 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Multicast: Queue shutdown
2021-01-13 15:33:00.005 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - ZigBee network [zigbee:coordinator_ember:40e943dd3b] closed.
2021-01-13 15:33:00.005 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Initializing ZigBee network [zigbee:coordinator_ember:40e943dd3b].
2021-01-13 15:33:00.005 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Channel 11
2021-01-13 15:33:00.009 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - PANID XXXX
2021-01-13 15:33:00.012 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - EPANID XXXX
2021-01-13 15:33:00.013 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network Key XXXX
2021-01-13 15:33:00.013 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link Key XXXX
2021-01-13 15:33:00.013 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Config: zigbee_initialise found, initializeNetwork=false
2021-01-13 15:33:00.013 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network Key String XXXX
2021-01-13 15:33:00.015 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Network key final array XXXX
2021-01-13 15:33:00.015 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link Key String XXXX
2021-01-13 15:33:00.015 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Link key final array XXXX
2021-01-13 15:33:00.016 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - Initializing ZigBee Ember serial bridge handler.
2021-01-13 15:33:00.016 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - ZigBee Ember Coordinator opening Port:'rfc2217://192.168.0.5:3002' PAN:XXXX, XXXX, Channel:11
2021-01-13 15:33:00.038 [DEBUG] [ng.zigbee.ember.handler.EmberHandler] - Ember end device poll timeout set to (169 * 2^9) = 86528 seconds
2021-01-13 15:33:00.040 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Scheduling ZigBee start
2021-01-13 15:33:01.040 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - ZigBee network starting
2021-01-13 15:33:01.041 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - Initialising ZigBee coordinator
2021-01-13 15:33:01.042 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Default: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=1, interTransactionDelay=50, maxRetries=2]
2021-01-13 15:33:01.042 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Broadcast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=2, interTransactionDelay=4000, maxRetries=0]
2021-01-13 15:33:01.042 [DEBUG] [e.transaction.ZigBeeTransactionQueue] - Multicast: Set profile to ZigBeeTransactionProfile [maxOutstandingTransactions=3, interTransactionDelay=1200, maxRetries=0]
2021-01-13 15:33:01.043 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - ZigBeeNetworkManager initialize: networkState=UNINITIALISED
2021-01-13 15:33:01.043 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to INITIALISING
2021-01-13 15:33:01.050 [DEBUG] [zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Initialize with protocol ASH2.
2021-01-13 15:33:01.050 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - Connecting to serial port [rfc2217://192.168.0.5:3002] at 57600 baud, flow control FLOWCONTROL_OUT_XONOFF.
2021-01-13 15:33:01.051 [DEBUG] [nding.zigbee.serial.ZigBeeSerialPort] - No communication ports found, cannot connect to [rfc2217://192.168.0.5:3002]
2021-01-13 15:33:01.051 [ERROR] [zigbee.dongle.ember.ZigBeeDongleEzsp] - EZSP Dongle: Unable to open serial port
2021-01-13 15:33:01.051 [DEBUG] [tsystems.zigbee.ZigBeeNetworkManager] - Network state is updated to OFFLINE
2021-01-13 15:33:01.052 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - null: networkStateUpdated called with state=INITIALISING
2021-01-13 15:33:01.053 [DEBUG] [bee.handler.ZigBeeCoordinatorHandler] - null: networkStateUpdated called with state=OFFLINE

Here’s my Zigbee coordinator thing config as well. I’ve matched everything but the port to match what they were in my OH2 config.

UID: zigbee:coordinator_ember:40e943dd3b
label: Ember EM35x Coordinator
thingTypeUID: zigbee:coordinator_ember
configuration:
  zigbee_port: rfc2217://192.168.0.5:3002
  zigbee_initialise: false
  zigbee_channel: 11
  zigbee_concentrator: 0
  zigbee_trustcentremode: TC_JOIN_SECURE
  zigbee_extendedpanid: XXXXXX
  zigbee_flowcontrol: 2
  zigbee_baud: 57600
  zigbee_panid: XXXXXX
  zigbee_powermode: 1
  zigbee_txpower: 0
  zigbee_networksize: 25
  zigbee_linkkey: XXXXXXX
  zigbee_childtimeout: 86400
  zigbee_networkkey: XXXXXX
  zigbee_meshupdateperiod: 86400

Rafal · January 23, 2021, 5:36pm

I have just done a couple of restarts of my Pi, which runs the ZStick and RFXCOM antennas, to see if and how rfc2217 ZWave recovers. I am pleased to say that in the current version (OH2.5_11) both ZWave and RFXCOM have recovered well. RFXCOM has always been rock-solid, so this is more of a confidence vote for ZWave over rfc2217. Log does not show ZWave losing the connection, but it shows the restart. RFXCOM shows both:

2021-01-23 17:14:38.370 [ERROR] [internal.handler.RFXComBridgeHandler] - Error occurred: End of stream
2021-01-23 17:15:25.093 [INFO ] [nternal.connector.RFXComTcpConnector] - Connecting to RFXCOM at openhab-relay:2002 over TCP/IP
2021-01-23 17:15:31.036 [INFO ] [ve.internal.protocol.ZWaveController] - Starting ZWave controller
2021-01-23 17:15:31.036 [INFO ] [ve.internal.protocol.ZWaveController] - ZWave timeout is set to 5000ms. Soft reset is false.

Other than simulating the disconnections, I have not experienced any issues with RFC2217 for many months now.

FrankR · March 17, 2021, 8:21am

Can someone summarize the situation regarding OH3?

I am quite sure that restarting either the openhab machine as well as the ser2net machine causes a loss of connection between OH3 and a zwave stick. Am I correct?

My setup is so far:

OH 3.0.1 ReleaseBuild on a rpi4 running buster
Z-Wave Serial Controller configured to “rfc2217://remoteusb:3001” through the UI
ser2net on a rpi1 (hostname “remoteusb”) running buster with “3001:telnet:0:/dev/ttyACM0:115200 8DATABITS NONE 1STOPBIT remctl” in my /etc/ser2net.conf

Rebooting one of these machines leeds to a connection loss that I can only heal by clicking onto “Synchronize Network” in the OH3 UI.

Is there a workaround or did I mix some things up or miss a detail?

FrankR · March 18, 2021, 4:07pm

@curlyel Could you remove [SOLVED] from the title as there are obviously still todos regarding OH3?

curlyel · March 21, 2021, 4:03pm

@FrankR , you decided to post your question in a very old thread … certainly to reach out to others which have contributed/posted here and therefore - for sure - are interested in RFC2217-related stuff.

So you got the intended audience - perfectly fine with me

But you should have noticed that my initial questions had nothing to do with openHAB3 - and were answered/solved meanwhile.

So: It’s absolutely o.k. for me to proceed here with RFC2217 questions.

But if you feel the topics headline is not matching your issue, then it might be better opening a new thread instead and hoping for enough attention there to get it solved too.