Frugal home temperature measurements with OpenHAB

timo12357 · September 6, 2023, 8:10am

The concept
In fall 2023 there are very affordable Tuya zigbee temperature sensors on the market. A temperature and humidity sensor can be found on Aliexpress for about 5€ and a zigbee gateway for about 15€. This article describes one approach to connect a bunch of such cheap zigbee sensors and a zigbee gateway to OpenHAB 3 or 4.

The procedure is as follows

Install mosquitto
Install json transformations add-on
Install Javascript Scripting add-on
Install divideby10.js to transforms
Connect sensors to the zigbee gateway with the phone app
Discover local sensor addresses to enable local access to sensors
Read sensor values from sensors with a script locally using tuya-cli
Publish sensor values on local mqtt using Mosquitto
Import sensor values to OH using a mqtt generic thing to assigned individual channels and items

Install necessary add-ons
Install mosquitto like described here. Mosquitto is the MQTT broker that delivers the temperature measurements to you.

Settings => Other add-ons
Install Jsonpath transformations:

Jsonpath transformations are necessary to be able parse the measurement value from the mqtt output.

Settings => Other add-ons
Install Javascript Scripting to enable running the divideby10.js script

Further, to be able to show the measurement correctly, the value read from the sensor needs to be divided by ten. To do this, create a file with the name divideby10.js in /etc/openhab/transforms with the following content

( function(i) { return i / 10  } ) (input)

Remember to change ownership of the file to openhab:openhab with

sudo chown openhab:openhab divideby10.js

Discover your tuya device local keys
Without the sensor local keys you can not read the sensor data to OpenHAB. Discover your devices’ local keys, ids and addresses as described here: https://github.com/codetheweb/tuyapi/blob/master/docs/SETUP.md

Make a rule script to read and publish sensors on mqtt regularly.

In the example below:

First line reads the sensors
Second line formats the output of tuya-cli for mqtt
Third line publishes the result on your mqtt network

var yourSensorName=actions.Exec.executeCommandLine(time.Duration.ofSeconds(20), "tuya-cli", "get", "--ip", "your.zigbee.gateway.ip.number", "--id", "yourZigbeeGatewayId", "--key", "yourZigbeeGatewayLocalKey", "--cid", "yourSensorCID", "--protocol-version", "3.3");
yourSensorName=yourSensorName.replace(" '","\"").replace(": ",":").replace(" '","\"").replace("'","\"").replace("'","\"").replace(": ",":").replace(" '","\"").replace("': ","\":").replace(" }","}");
actions.Things.getActions("mqtt", "mqtt:broker:******").publishMQTT("tuya/yourSensorName", yourSensorName, false);

yourSensorName is a variable name for your sensor e.g. masterBedroomSensor
your.zigbee.gateway.ip.number is the IP of your zigbee gateway on your local network. Check the gateway mac adress from the Smart Life app, and look for that mac adress on your router DHCP client list. The corresponding local IP adress is what you need. e.g. 192.168.0.100
yourZigbeeGatewayId, yourZigbeeGatewayLocalKey and yourSensorCID are discovered in the previous section

Set the script to trigger e.g. every hour.

Create a thing where you collect all your measurements

Things => plus in blue ring => MQTT binding => Generic MQTT thing
Name the thing e.g. “yourMeasurementThing”

Create a channel to your sensor
Select Channel tab under your newly created Thing => Add channel
Name the Channel e.g. “yourMeasurementChannel”
Add your device into the MQTT State Topic as below. Replace “takka” with yourSensorName

Configure the transformations as below

Create an item for the sensor
yourMeasurementThing => channels => yourMeasurementChannel => Add link to item => Create New Item
Configure the new item as below

Repeate the create channel and create item phase for each of your sensors.

Conclusion
You should now have a temperature item that is automatically updated once every hour with proper units and formatting. You can add new sensors by adding channels and corresponding items under yourMeasurementThing. I have 8 sensors running currently all around the house.

I initially tried the Smarthome/J tuya binding to read the sensors, but it does unfortunately not in fall 2023 support devices behind a zigbee gateway and therefore I resorted to this solution. The procedure described above works, but is unncessary complicated. It would make sense to update the item value directly from the output of tuya-cli and skip the mqtt publisihing totally. I did not find out how to parse the measurement value directly from the tuya-cli output, so I resorted to this cludge with mqtt. This solution will be obsolete once the Smarthome/J Tuya binding starts supporting devices behind a zigbee gateway.

I’d be happy to receive ideas on how to simplify the method.

Pedro_Liberal · September 6, 2023, 8:29am

Simplification:
Instead of buying a zigbee gateway from tuya, you buy a sonoff zigbee dongle.
If you buy zigbee 3.0 certified sensors, then that’s all you need. openhab has zigbee, expose the usb dongle to openhab and the sensors will be found.
If the sensors are xiaomi or aqara, or otherwise not following the zigbee 3.0 standard, use something like zigbee2mqtt.
Turn on auto discovery in z2m and you’ll get auto discovery of sensors.

I think the above will trivialize a lot of your detailed steps because you don’t need to dance around the tuya gateway.

anon71759204 · September 6, 2023, 4:50pm

It does - as long as the Tuya device is supported by the Zigbee binding or by Zigbee2MQTT …

rlkoshak · September 6, 2023, 4:52pm

Your are missing steps to install the JS Scripting add-on which is also required to run the divide10 transform.

Because this is so simple, it could be defined inline on the Channel or a profile. For example, on the MQTT Channel it would look like: JS:js:| parseFloat(input) / 10. No need for a file.

To make configuring a script like this easier, I find it best to define each of these parameters as variables at the top of the script. It’s easier for the end user to see what needs to be changed and gives a uniform place and way to configure the script.

Going through MQTT after getting the data into OH seems excessive. It would be more coding for sure but less redirection over all if you parsed the output of the python script in the rule and updated Items manually.

Another approach would be to call the python script using the Exec binding and then use transformation profiles to extract the data needed from the output of the script for each individual Item. Then you wouldn’t need the rule at all.

Yet another approach is to make the python script run independently and publish the sensor readings to MQTT itself. Then all that’s needed on the OH side is the MQTT config. You could even do the divide by 10 in the python script.

Show an example of the raw output and I can help with that.

Pedro_Liberal · September 6, 2023, 5:34pm

That’s exactly what I said did i express myself incorrectly?

timo12357 · September 6, 2023, 6:11pm

This is what tuya-cli churns out:

{ '1': 203, '2': 446, '4': 100 }

The first value is temperature x 10 in celsius, second number is humidity % x 10 and third is battery status %.

rlkoshak · September 6, 2023, 6:35pm

OK, that’s easy.

var zigbeeGatewayIP =       '192.168.1.X';
var zigbeeGatewayId =       'id';
var zigbeeGatewayLocalKey = 'key';
var sensorCID =             'cid';
var temperatureItemName =   'MyTemperatureItem';
var humidityItemName =      'MyHumidityItem';
var batteryItemName =       'MyBatteryItem';

var sensorReadings=actions.Exec.executeCommandLine(time.Duration.ofSeconds(20), "tuya-cli", "get", "--ip", zigbeeGatewayIP, "--id", zigbeeGatewayId, "--key", zigbeeGatewayLocalKey, "--cid", sensorCID, "--protocol-version", "3.3");
var parsed = JSON.parse(sensorReadings);

var adjusted = parsed['1'] / 10; // may need to call parseFloat(parsed['1'])
items[temperatureItemName].postUpdate(adjusted + ' °C'); 
items[humidityItemName].postUpdate(parsed['2'] + ' %');
items[batteryItemName].postUpdate(parsed['4'] + ' %');

If you were to configure this using the exec binding:

install Exec the add-on
create a new Thing with the call to tuya-cli, fill in all the values inline on the command and set the polling period
copy the command exactly as written to the whitelist (see the Exec binding docs)
link the Output Channel to the Temperature Item using a transform profile on the link with JS:js:| JSON.parse(input)['1'] / 10 as the transform
link the Output Channel to the Humidity Item using a transform profile on the link with JSONPATH:$.2 as the transformation
link the Output Channel to the Battery Item using a transform profile on the link with JSONPATH:$.4 as the transformation

No rule nor MQTT required.

JimT · September 7, 2023, 10:23am

Does js allow single quotes in json?

rlkoshak · September 7, 2023, 2:04pm

I think so. I know it’s not the standard but I think I’ve parsed JSON with single quotes like that before. Maybe I was using JSON5.

If it doesn’t work it’s a simple find and replace on the input string. That should work here because the output from the script is pretty fixed an unlikely to include stray ' in a value somewhere. It wouldn’t work if this were pulling JSON down from some website though.

var parsed = JSON.parse(sensorReadings.replace("'", '"'));

Even better would be to fix the script to return proper JSON in the first place.

timo12357 · September 8, 2023, 5:43am

2023-09-08 08:42:30.421 [WARN ] [ofiles.JSonPathTransformationProfile] - Could not transform state '{ '1': 206, '2': 494, '4': 100 }' with function 'JS:js:| JSON.parse(input)['1'] / 10' and format '%s'

How?

rlkoshak · September 8, 2023, 2:23pm

There are two parts to step 4. First, link the Output Channel to the Temperature Item.

When you create that link, a link configuration page comes up. At the bottom of that form you will find the profile configuration. Choose “SCRIPT ECMAScript (ECMAScript 262 Edition 11)” there and use the above inline transform, without the JS: starting part (I forgot that part is understood and not needed with a profile) for the Thing to Item transformation direction.

Do not put the transform on the Thing configuration because you need to link this same Channel to two more Items with different transformations required.

timo12357 · September 8, 2023, 5:00pm

2023-09-08 20:00:04.121 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'js:| JSON.parse(input)['1'] / 10': Could not get script for UID 'js:| JSON.parse(input)['1'] / 10'.
2023-09-08 20:00:17.718 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'js:| JSON.parse(input)['1'] / 10': Could not get script for UID 'js:| JSON.parse(input)['1'] / 10'.
2023-09-08 20:00:30.827 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'js:| JSON.parse(input)['1'] / 10': Could not get script for UID 'js:| JSON.parse(input)['1'] / 10'.
2023-09-08 20:00:44.048 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'js:| JSON.parse(input)['1'] / 10': Could not get script for UID 'js:| JSON.parse(input)['1'] / 10'.

rlkoshak · September 8, 2023, 6:12pm

Based on my understanding of the docs that should have worked. Maybe it just needs to be | JSON.parse(input)['1'] / 10? I don’t have an easy way to test this.

timo12357 · September 8, 2023, 7:17pm

Unfortunately that also gives the same error: To me the error message hints at it expecting an UID not a scrip

rlkoshak · September 8, 2023, 8:15pm

It should accept both. The | is supposed to tell it it’s an inline transform instead of referring to a file or a managed transform.

Ultimately we can keep fighting this or create a transform. If you go to Settings → Transformations → + → JS and paste in the following then you’ll be able to just select that from the profile.

(function(data) {
  return JSON.parse(data)['1'] / 10;
})(input)

Though now that I type it out I’m reminded that we don’t know for certain that JSON.parse will work with the single quotes.

(function(data) {
  return JSON.parse(data.replace("'", '"')['1'] / 10;
})(input)

timo12357 · September 10, 2023, 8:54am

I would like to keep fighting. This is an interesting simplification to my spaghetti.

rlkoshak · September 10, 2023, 3:24pm

Unfortunately I don’t have a channel where I can apply a transform like this right now. It’ll take some time for me to rig something up to test. Based on the docs, one of the two above should have worked.

timo12357 · September 11, 2023, 5:45am

Result:

2023-09-11 08:43:15.077 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'config:js:parseTuyaSensorData': org.graalvm.polyglot.PolyglotException: SyntaxError: Invalid JSON: <json>:1:2 Expected , or } but found '
{ '1': 202, '2': 532, '4': 100 }
  ^

(function(data) {
  return JSON.parse(data.replace("'", '"')['1'] / 10;
})(input)

Result:

2023-09-11 08:44:43.724 [ERROR] [.module.script.profile.ScriptProfile] - Failed to process script 'config:js:parseTuyaSensorData': org.graalvm.polyglot.PolyglotException: SyntaxError: <eval>:2:52 Expected , but found ;
  return JSON.parse(data.replace("'", '"')['1'] / 10;
                                                    ^

JimT · September 11, 2023, 6:32am

You missed the closing parenthesis for JSON.parse()

(function(data) {
  return JSON.parse(data.replace("'", '"'))['1'] / 10;
})(input)

timo12357 · September 11, 2023, 7:02am

With this correction the rule runs without errors in the log. The output is unfortunately wrong

This input

{ '1': 201, '2': 532, '4': 100 }

Results in an output of