Receiving binary values in the tcp binding

I am trying to receive status updates from my Solis PV inverter, which are sent as a string of hex data. I have pointed the inverter at my OH installation and set up a tcp item to receive the incoming data. The binding accepts the incoming data but but it breaks off at the first occurrence of 0x00. Unfortunately, all of the useful data come later in the string.

I have looked elsewhere on the community and found references to changing the characterset so I have added the following to my tcp.cfg file

charset=ISO-8859-1

The non-ascii incoming characters look different after setting that but the binding still breaks the string in the same place.

I can’t see a way to read the incoming tcp data as hex instead of a character string. Any thoughts on what I could try next?

OH 2.5.1-8 running on openhabian on Raspberry Pi 3

Can you use item string type and post what is in the log?

Hi, thanks for the reply.

This is the item definition:

String Solar_rx_data "Data recieved from the inverter [%s]" { tcp="<[192.168.1.193:*:default]" }

This is an example log entry (character set may be affected by the forum config)

2020-11-28 08:24:51.689 [vent.ItemStateChangedEvent] - Solar_rx_data changed from h)Q�vZ�$vZ�$�^AH4.01.51Y4.0.02W1.0.57(GL17-07-261-D)Q to hYQ�vZ�$vZ�$�^A^E000351015104869

The new (to) value (starting hYQ) is the message type I’m interested in. Using another program (a python app for decoding the message), I can see that it is 103 hex bytes, whereas the String item only has the first 31.
Looking at the hex data, I can see that the point where it breaks is a hex character 00.

The “default” entry at the end of the item definition should update the item with the data received by the binding with no transformation but I’m going to experiment with some other transformations (bin2js and javascript) to see if I can catch the whole message and control it so it arrives in a usable form.

Can you try

charset=UTF-16

OK. I tried that. Now I get every character coming back as 3F:

2020-11-28 11:24:28.813 [vent.ItemStateChangedEvent] - Solar_rx_data changed from ?????????????????? to ??????????????????????????\?

Plus my other tcp connected kit fails.

I tried adding a JS transformation with a function to return the length of the string and it returns 31 (instead of 103) so the string is getting truncated inside the binding before it gets as far as any transformation.

Well I guess the binding is not going to do much.

Maybe CURL will be the saviour

What do you see in the command line if you type

curl --location --request GET '192.168.1.193'

I get a 401 Authorization required response. I’m not surprised as the inverter pushes the messages from a random selection of ports rather than acting as a server.

I used to capture the data using a python script that updated the items using the REST API. I could go back to that method but I was hoping to set up something a bit more integrated that relied solely on OH2 configurations rather than additional scripts.

You could write a binding for it then it will make it easy for people that follow.

I’d love to but I’m not sure I’m up to it. Although I’ve written in a few languages, I’ve never touched Java (other than a quick copy/paste of someone else’s script to try to make the JS transformation work). As it stands, I wouldn’t even know how to compile something I wrote. Plus having watched some other binding developments, I can see there’s a stack of naming and other conventions conventions to learn and comply with before a binding can be accepted.

Can you share your script so others can benefit from your work ?

This is the script. Only the connection to OpenHAB is my work, the rest is taken from Graham Whiteside to whom most of the credit is due.

Different series of inverters place the results in different places in the string so a bit of fiddling around might be required.

#!/usr/bin/python

###################################################################################################
#
#  Copyright 2015 Graham Whiteside, Manchester, UK. Version 0.3 Oct 2015.
#
#  Script modified by Barny Daley to suit Solis 3G and post outputs to OpenHAB items using REST API
#
#  read-ginlong is free software: you can redistribute it and/or modify it under the terms of the
#  GNU General Public License as published by the Free Software Foundation, either version 3 of the
#  License, or (at your option) any later version.
#
#  read-ginlong is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
#  even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
#  General Public License for more details.
#
#  You can browse the GNU license here: <http://www.gnu.org/licenses/>.
#
###################################################################################################

###################################################################################################
#
#  Python program to read data sent from a single Ginlong/Solis 2G inverter equipped with a
#  Wi-Fi 'stick'.
#
#  Requires setting up the inverter stick to send data to the computer running the read-ginlong
#  script. Settings located in the advanced settings, then remote server. Add a new 'remote
#  server' with the ip address of your computer and port 9999. The inverter will send data every
#  five minutes.
#
#  Output file format (space separated):-
#	Date  Time   Watts_now   Day_kWh   Total_kWh
#
#  Output of webserver file, format (space separated) file overwritten each update:-
#	Date Time Watts_now Day_kWh Total_kWh DC_volts_1 DC_amps_1 DC_volts_2 DC_amps_2 AC_volts AC_amps AC_freq kwh_yesterday kwh_month kwh_last_month
#
#  The read-ginlong.py program is deliberately left simple without error reporting. It is intended
#  as a 'starting point' and proof of concept. It could easily be modified to provide more
#  information from the inverter. Furthermore the output log file can be further processed or
#  loaded into other software such as LibreOffice Calc.
#
###################################################################################################

import socket, binascii, time, requests
# import urllib2

# print "Import complete"

# change these values to suit your requirements:-
HOST = ''	        	 			# Hostname or ip address of interface, leave blank for all
PORT = 9999              				# listening on port 9999
logfile = 'ginlong.log'					# location of output log file
webfile = 'ginlong.status'				# location of web file


# inverter values found (so far) all big endian 16 bit unsigned:-
header = '685951b0' 					# hex stream header
data_size = 206                     			# hex stream size
inverter_temp = 31 					# offset 31 & 32 temperature (/10)
inverter_vdc1 = 33 					# offset 33 & 34 DC volts chain 1 (/10)
inverter_adc1 = 39 					# offset 39 & 40 DC amps chain 1 (/10)
inverter_aac = 45					# offset 45 & 46 AC output amps (/10)
inverter_vac = 51 					# offset 51 & 52 AC output volts (/10)
inverter_freq = 57 					# offset 57 & 58 AC frequency (/100)
inverter_now = 59 					# offset 59 & 60 currant generation Watts
inverter_yes = 67 					# offset 67 & 68 yesterday kwh (/100)
inverter_day = 69 					# offset 69 & 70 daily kWh (/100)
inverter_tot = 73 					# offset 73 & 74 total kWh 
inverter_mth = 87					# offset 87 & 88 total kWh for month
inverter_lmth = 91					# offset 91 & 92 total kWh for last month


sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)   # create socket on required port
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((HOST, PORT))

print "Socket bound"

while True:		# loop forever
    sock.listen(1)							# listen on port
    conn, addr = sock.accept()				# wait for inverter connection
    rawdata = conn.recv(1000)				# read incoming data
    hexdata = binascii.hexlify(rawdata)		# convert data to hex
    print(hexdata)

    if(hexdata[0:8] == header and len(hexdata) == data_size):		# check for valid data

																			# extract main values and convert to decimal
        watt_now = str(int(hexdata[inverter_now*2:inverter_now*2+4],16))    		# generating power in watts
        kwh_day = str(float(int(hexdata[inverter_day*2:inverter_day*2+4],16))/100)	# running total kwh for day
        kwh_total = str(float(int(hexdata[inverter_tot*2:inverter_tot*2+4],16))/2)	# running total kwh from installation

	temp = str(float(int(hexdata[inverter_temp*2:inverter_temp*2+4],16))/10)		# temperature																		# extract dc input values and convert to decimal
        dc_volts1= str(float(int(hexdata[inverter_vdc1*2:inverter_vdc1*2+4],16))/10)	# input dc volts from chain 1
        dc_amps1 = str(float(int(hexdata[inverter_adc1*2:inverter_adc1*2+4],16))/10)	# input dc amps from chain 1

																			# extract other ac values and convert to decimal
        ac_volts = str(float(int(hexdata[inverter_vac*2:inverter_vac*2+4],16))/10)		# output ac volts
        ac_amps = str(float(int(hexdata[inverter_aac*2:inverter_aac*2+4],16))/10)		# output ac amps
        ac_freq = str(float(int(hexdata[inverter_freq*2:inverter_freq*2+4],16))/100)	# output ac frequency hertz

																			# extract other historical values and convert to decimal
        kwh_yesterday = str(float(int(hexdata[inverter_yes*2:inverter_yes*2+4],16))/100)	# yesterday's kwh
        kwh_month = str(int(hexdata[inverter_mth*2:inverter_mth*2+4],16))					# running total kwh for month
        kwh_lastmonth = str(int(hexdata[inverter_lmth*2:inverter_lmth*2+4],16))				# running total kwh for last month


        timestamp = (time.strftime("%F %H:%M"))		# get date time

        log = open(logfile,'a')        # write data to logfile, main values only
        log.write(timestamp + ' ' + watt_now + ' ' + kwh_day + ' ' + kwh_total + '\n')
        log.close()

        web = open(webfile,'w')        # output all values, possibly for webpage
        web.write(timestamp + ' ' + watt_now + ' ' + kwh_day + ' ' + kwh_total + ' ' + dc_volts1 + ' ' + dc_amps1 + ' ' + ac_volts + ' ' + ac_amps + ' ' + ac_freq + ' ' + kwh_yesterday + ' ' + kwh_month + ' ' + kwh_lastmonth + ' ' + temp + '\n')
        web.close()

        urlbase = "http://openhabian:8080/rest/items/"

#	print (watt_now)
        url = urlbase + "Solar_watt_now" + "/state"
	requests.put(url, str(watt_now))

#	print (kwh_day)
        url = urlbase + "Solar_kwh_day" + "/state"
	requests.put(url, str(kwh_day))

        url = urlbase + "Solar_kwh_total" + "/state"
        requests.put(url, str(kwh_total))

        url = urlbase + "Solar_dc_volts" + "/state"
        requests.put(url, str(dc_volts1))

        url = urlbase + "Solar_dc_amps" + "/state"
        requests.put(url, str(dc_amps1))

        url = urlbase + "Solar_ac_volts" + "/state"
        requests.put(url, str(ac_volts))

        url = urlbase + "Solar_ac_amps" + "/state"
        requests.put(url, str(ac_amps))

        url = urlbase + "Solar_ac_freq" + "/state"
        requests.put(url, str(ac_freq))

#	print (temp)
	url = urlbase + "Solar_temp" + "/state"
	requests.put(url, str(temp))

conn.close()

These are the accompanying OpenHAB items:

Number Solar_watt_now "Current solar generation [%.0f W]" (Solar)
Number Solar_kwh_day "Solar generated energy today [%.1f kWh]" (Solar)
Number Solar_kwh_total "Solar generated energy [%.0f kWh]" (Solar)
Number Solar_dc_volts "Solar array DC [%.1f V]" (Solar)
Number Solar_dc_amps "Solar array DC [%.1f A]" (Solar)
Number Solar_ac_volts "Solar AC [%.1f V]" (Solar)
Number Solar_ac_amps "Solar AC [%.1f A]" (Solar)
Number Solar_ac_freq "Solar AC [%.1f Hz]" (Solar)
Number Solar_temp "Temperature at solar inverter  [%.1f °C]" (Solar)
1 Like

I’m getting the same issue. If the incoming string from the TCP connection contains 0X00 it get clipped short at the first 0X00 occurrence. Is this a bug? Did you find a way around this issue using a different binding?

The TCP/UDP binding v1 is now quite elderly, and unmaintained. There may well be a bug causing it to truncate at x0000, or rather some missing option feature to handle it.

The docs say

They send and receive data as ASCII strings.

which I think means you are goosed because x00 is ASCII nul character and will terminate a string.

Can’t even offer an OH3 version as it hasn’t been invented yet

Thanks for the reply.

The lack of TCP/UDP binding is why I am starting out with Open HAB 2. I have now also tried with a passthrough Javascript transform and that does the same which I guess backs up what you are saying that it is an inherent problem with the TCP/UDP binding.

I have postamble set to \r (0D) so I suppose this is a bug. It is also inconsistent as I can send 0x00 OK but cannot receive 0x00.

I’m actually trying to get it to talk to my Arcam AVR. The same protocol is used by recent Arcam, Lexicon, JBL Synth and Audiocontrol AVRs. It is possible to report a bug? Is the TCP/UDP binding abandoned? Or should a beginner like me look into how a binding is created?

Curious - how can you do that, you have to send an ASCII string? Or are you somehow concatenating a string with a nul buried in it?

The trouble with incoming processing is likely that the string length is not known and opts to terminate on x00.

Yes. All v1 bindings are out of support. I don’t even know if any further OH2 maintenance releases are expected now, but this is not even a maintenance issue really, it’s an enhancement request.

Your root problem is this old binding only supports ASCII mode, a replacement will need some extended binary mode option.

I had to set the TCP/UDP binding and the Java service set to an iso-8859-1 character set (ie. 1 byte).

I can then send binary/hex (Including 0x00) as escaped characters, eg:
Arcam.sendCommand(“\u0021\u0002\u0000\u0001\u00F0”)

My overall impression is working with binary/hex in OpenHAB seems to be difficult; I found this easier in Commandfusion JS.