RTL_433 to MQTT

Hello,

today I’ve installed RTL_433 within my Openhab2.4 and it works fine. The System found a device I want to use within Openhab.

time      : 2019-08-25 12:10:10
model     : TS-FT002     Id        : 20
Depth     : 70           Temperature: 20.1 C       Transmit Interval: 180    Battery Flag?: 
8          MIC       : CHECKSUM

I’ve also installed the MQTT Binding in the PaperUI.

So what is now the next step to get the information to MQTT? I saw some hints in other posts but I can’t interprete the command for RTL_433.

  • Do you have an example how the command has to be in my case to send the information with JSON to MQTT?

  • Will the system send all information to MQTT or only for my device_ID: 20?

  • What kind of thing I need to use in Openhab to get the information from the value “depth” and temperature?

Thanks in advance…

I’ve modified a Python script found somewhere which parses the -json output from RTL433 and publishes it to my MQTT broker.

So I can easily set up some MQTT things:

Bridge mqtt:broker:My-MQTT "MQTT Broker" @ "RTL433" [
  host="192.168.x.x",
  secure=false,
  port=1883,
  qos=0,
  retain=false,
  clientid="Oh2Mqtt2Thing",
  keep_alive_time=30000,
  reconnect_time=60000
]
{
    Thing topic RTL_433 "433MHz Empfänger" @ "RTL433"  {
      Channels:
          Type number : temp "Temperatur" [ stateTopic="sensors/rtl_433/P25/C1/temperature_C" ]
          Type number : hum   "Luftfeuchtigkeit" [ stateTopic="sensors/rtl_433/P25/C1/humidity" ]
          Type switch : batt  "Battery schwach" [ stateTopic="sensors/rtl_433/P25/C1/battery", transformationPattern="MAP:battery.map"]
    }
}

… and the related items:

Number Temp1 "Temperatur"       { channel="mqtt:topic:My-MQTT:RTL_433:temp"}
Number Hum1 "Luftfeuchtigkeit"  { channel="mqtt:topic:My-MQTT:RTL_433:hum"}
Switch Bat1 "Batterie schwach"  { channel="mqtt:topic:My-MQTT:RTL_433:batt"}

Thats my current script:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import subprocess
import sys
import time
import paho.mqtt.client as mqtt
import os
import json

from config import *

rtl_433_cmd = "/usr/local/bin/rtl_433 -q -M protocol -R 12 -R 25 -R 79 -F json"

# Define MQTT event callbacks
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

def on_disconnect(client, userdata, rc):
    if rc != 0:
        print("Unexpected disconnection.")

def on_message(client, obj, msg):
    print(msg.topic + " " + str(msg.qos) + " " + str(msg.payload))

def on_publish(client, obj, mid):
    print("mid: " + str(mid))

def on_subscribe(client, obj, mid, granted_qos):
    print("Subscribed: " + str(mid) + " " + str(granted_qos))

def on_log(client, obj, level, string):
    print(string)

# Setup MQTT connection

mqttc = mqtt.Client()
# Assign event callbacks
#mqttc.on_message = on_message
mqttc.on_connect = on_connect
#mqttc.on_publish = on_publish
mqttc.on_subscribe = on_subscribe
mqttc.on_disconnect = on_disconnect

# Uncomment to enable debug messages
#mqttc.on_log = on_log

# Uncomment the next line if your MQTT server requires authentication
#mqttc.username_pw_set(MQTT_USER, password=MQTT_PASS)
mqttc.connect(MQTT_HOST, MQTT_PORT, 60)

mqttc.loop_start()

# Start RTL433 listener
rtl433_proc = subprocess.Popen(rtl_433_cmd.split(),stdout=subprocess.PIPE,stderr=subprocess.STDOUT,universal_newline
s=True)


while True:
    for line in iter(rtl433_proc.stdout.readline, '\n'):
        if "time" in line:
            mqttc.publish(MQTT_TOPIC, payload=line,qos=MQTT_QOS)
            protocol="P0"
            channel="C0"
            id="I0"
            data="D0"
            data_val=0
            data_cnt=0
            json_dict = json.loads(line)
            for item in json_dict:
                value = json_dict[item]
                if "protocol" in item:
                    protocol="P"+str(value)
                if "channel" in item:
                    channel="C"+str(value)
                if "rows" in item:
                    for row in json_dict['rows']:
                        if row['data'] == data_val:
                                data_val = row['data']
                                data_cnt += 1
                                data ="D"+str(data_val)
                        elif data_cnt < 5:
                                data_val = row['data']
                                data_cnt = 0
                    if data_cnt <= 5:
                        data="D0"

            subtopic =protocol+"/"+channel
            if protocol == "P0":
                subtopic +="/"+data

            if not(protocol == "P0" and data == "D0"):
                for item in json_dict:
                    value = json_dict[item]
                    if not ("brand" in item or "model" in item or "channel" in item or "protocol" in item or "rows"
in item or "codes" in item):
                        mqttc.publish(MQTT_TOPIC+"/"+subtopic+"/"+item, payload=value,qos=MQTT_QOS)

… and the included config.py:

# Config section
# Uncomment these lines if your MQTT server requires authentication
#MQTT_USER="mqtt-user"
#MQTT_PASS="mqtt-password"
MQTT_HOST="localhost"
MQTT_PORT=1883
MQTT_TOPIC="sensors/rtl_433"
MQTT_QOS=0
# End config section

EDIT: the original script was taken from (credits to mverleun :wink: ):

Wait…

I helped some weeks ago with improving the direct MQTT publish from RTL_433 - to be precise with the MQTT format string. See the link to the issue at the bottom of this post.

With this quite recent feature of rtl_433 you can specify some formatting for the mqtt publish, e.g.:

output mqtt://localhost:1883,retain=0,devices=sensors/x[/channel:0][/id]

This produces some topics in the broker like this:

sensors/x/0/147/id 147
sensors/x/0/147/temperature_C 33.200001
sensors/x/0/147/rain_mm 107.699997
sensors/x/0/147/battery_ok 1
sensors/x/0/147/mic CRC

… which can be consumed conviniently by a MQTT thing as shown in the former post in openHAB without the hassle with the conversion script :wink:

Seems, I should rework my own setup leveraging rtl_433’s new MQTT feature and get rid of the python script myself :wink:

See: https://github.com/merbanan/rtl_433/pull/1079#issuecomment-505065206

Hello curlyel,

thanks for your answers. But I can’t get it.

The documentation of rtl_433 told us to use a -F option for the output to MQTT with JSON. You are using it in your script but not in the last answer.

I’m looking for a simple primary command to send the rtl_433 information of my devices to openhab. Is there no simply command for it?

I found this hint here where is only one command with in.

rtl_433 -F json -U | mosquitto_pub -h mqtt.unixweb.de -t home/rtl_433 -l

or like this here

rtl_433 -F json -M utc | mosquitto_pub -t home/rtl_433 -l

But I can’t transfer it to my connection to openhab with the MQTT binding.

Because of this I’m a litte bit confused now. I need a step by step manual.

Thanks

You have two options to pass configuration arguments to rtl_433: via command line or via config file.
If you have a more complex setup with lots of options, it might be more handy with the config file.
That’s what I did in the second post. The confusion might come from that :wink:

That’s not required anymore: No need to pipe the json output to some external script or program to forward the content to the MQTT broker.

Since a couple of weeks (months?), rtl_433 can directly publish to your MQTT broker (commandline -F mqtt <further options> or in the config file output mqtt://...).

Check the readme for rtl_433 again:

		= Output format option =
  [-F kv|json|csv|mqtt|syslog|null] Produce decoded output in given format.
	Without this option the default is KV output. Use "-F null" to remove the default.
	Append output to file with :<filename> (e.g. -F csv:log.csv), defaults to stdout.
	Specify MQTT server with e.g. -F mqtt://localhost:1883
	Add MQTT options with e.g. -F "mqtt://host:1883,opt=arg"
	MQTT options are: user=foo, pass=bar, retain[=0|1], <format>[=topic]
	Supported MQTT formats: (default is all)
	  events: posts JSON event data
	  states: posts JSON state data
	  devices: posts device and sensor info in nested topics
	The topic string will expand keys like [/model]
	E.g. -F "mqtt://localhost:1883,user=USERNAME,pass=PASSWORD,retain=0,devices=rtl_433[/id]"
	Specify host/port for syslog with e.g. -F syslog:127.0.0.1:1514

thanks for your answer again.

Please check the following rows… does this could fit?

mqtt.thing
Bridge mqtt:broker:My-MQTT “MQTT Broker” @ “RTL433” [
host=“192.168.178.xx”,
secure=false,
port=1883,
qos=0,
retain=false,
clientid=“Oh2Mqtt2Thing”,
keep_alive_time=30000,
reconnect_time=60000
]
{
Thing topic RTL_433 “433MHz Empfänger” @ “RTL433” {
Channels:
Type number : depth “Depth” [ stateTopic=“sensors/20/Depth” ]
}
}

mqtt.items
Number Depth “Depth” { channel=“mqtt:topic:My-MQTT:RTL_433:depth”}

Primary Command for rtl_433:
rtl_433 -F “mqtt://192.168.178xx:1883,retain=0,devices=sensors[/id]”

I did the following things:

First mqtt.things

> Bridge mqtt:broker:brokerhome [ host="192.168.178.42", secure=false ]{
>     Thing topic sensoren {
>     Channels:
>        Type number : Depth "Depth" [ stateTopic="sensors/20/Depth" ]
>     }
> }

Second in my.itelms

> Number Depth "Depth" <socket> (gRTL) { channel="mqtt:topic:brokerhome:sensoren:depth"}

In PaperUI the broker is shown with: OFFLINE - COMMUNICATION_ERROR
And the Thing with: OFFLINE - BRIDGE_OFFLINE

In openhab.log the system is trying to reconnect.

2019-08-26 18:03:14.852 [INFO ] [.reconnect.PeriodicReconnectStrategy] - Try to restore connection to ‘192.168.178.42’. Next attempt in 10000ms
2019-08-26 18:03:14.900 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to ‘192.168.178.42’ with clientid paho108659430827728 and file store ‘/var/lib/openhab2/mqtt/192.168.178.42’

After this I send the following command.

rtl_433 -F “mqtt://192.168.178.42:1883,retain=0,devices=sensors[/id]”

But I get this error:

MQTT connect error: Connection refused

openhab and RTL_433 are running on the same raspberry pi.

What is my mistake? Is it a problem in openhab or at the side of RTL_433?

Do you have a MQTT broker installed and running? Mosquitto or the internal broker?
Seems that no process is listening to the mqtt-port 1883 :wink:

OK. The mosquitto service was down. I’ve started it and the Thing and Broker get ONLINE in Paper UI.

Then I send my command:

rtl_433 -F “mqtt://192.168.178.42:1883,retain=0,devices=sensors[/id]”

I get this Error:

Invalid output format “mqtt://192.168.178.42:1883,retain=0,devices=sensors[/id]”

How did you install rtl_433? From your distribution (apt-get …) or from the source code by following the build instructions:

Your command line looks o.k. - so no idea what’s wrong other than having an old version of rtl_433…

I said above, you can use a quite recent feature of rtl_433 for your need.
So, if you have not build from source this might the issue (since the version from the distro is certainly too old).

If this is not the case, you should raise a issue at the rtl_433 repository.

Well, before advising wrongly I’ve tried a new install myself to confirm it is still working with the current master branch.

I’ve recorded all steps I made to give you a reference. Sure, the following is likely identical to the thousend other rtl_433-install-howto’s out there…

Let’s go …

Prerequisites:

sudo apt-get install libtool libusb-1.0-0-dev librtlsdr-dev rtl-sdr build-essential autoconf cmake pkg-config
sudo apt-get install -y mosquitto mosquitto-clients

Step 1: clone the rtl_433 master branch:

root@raspberrypi(rw):/home/pi# git clone https://github.com/merbanan/rtl_433.git rtl_433_MQTT
Klone nach 'rtl_433_MQTT' ...
remote: Enumerating objects: 51, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 12578 (delta 20), reused 32 (delta 15), pack-reused 12527
Empfange Objekte: 100% (12578/12578), 4.16 MiB | 1.42 MiB/s, Fertig.
Löse Unterschiede auf: 100% (9701/9701), Fertig.

Step 2: Make it:

root@raspberrypi(rw):/home/pi# cd rtl_433_MQTT/

root@raspberrypi(rw):/home/pi/rtl_433_MQTT# mkdir build && cd build

root@raspberrypi(rw):/home/pi/rtl_433_MQTT/build# cmake ..
-- The C compiler identification is GNU 6.3.0
           < -- truncated -->
-- Build files have been written to: /home/pi/rtl_433_MQTT/build

root@raspberrypi(rw):/home/pi/rtl_433_MQTT/build# make
Scanning dependencies of target rtl_433
[  0%] Building C object src/CMakeFiles/rtl_433.dir/abuf.c.o
[  1%] Building C object src/CMakeFiles/rtl_433.dir/am_analyze.c.o
           <-- truncated -->
[ 99%] Building C object tests/CMakeFiles/test_fileformat.dir/__/src/fileformat.c.o
[100%] Linking C executable test_fileformat
[100%] Built target test_fileformat
root@raspberrypi(rw):/home/pi/rtl_433_MQTT/build#

Step 3: If succeeded - install it:

root@raspberrypi(rw):/home/pi/rtl_433_MQTT/build# make install
Install the project...
-- Install configuration: "Release"
<-- truncated -->
root@raspberrypi(rw):/home/pi/rtl_433_MQTT/build#

Step 4: Start it with your options (protocols, output, etc.):

root@raspberrypi(rw):/home/pi# rtl_433 -R12 -R25 -R79 -F mqtt://localhost:1883,retain=0,devices=sensors[/type][/model][/subtype][/channel:0][/id]
rtl_433 version 18.12-284-g7c6b415 branch master at 201908261453 inputs file rtl_tcp RTL-SDR
Trying conf file at "rtl_433.conf"...
Trying conf file at "/root/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...
Publishing MQTT data to localhost port 1883
Publishing device info to MQTT topic "sensors[/type][/model][/subtype][/channel:0][/id]".

        Consider using "-M newmodel" to transition to new model keys. This will become the default someday.
        A table of changes and discussion is at https://github.com/merbanan/rtl_433/pull/986.

Registered 3 out of 136 device decoding protocols [ 12 25 79 ]
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
[R82XX] PLL not locked!
Sample rate set to 250000 S/s.
Tuner gain set to Auto.
Tuned to 433.920MHz.
MQTT Connected...
MQTT Connection established.

Step 5: Verify the result:

pi@raspberrypi(rw):~$ mosquitto_sub -v -t sensors/#
sensors/GT-WT02/1/215/id 215
sensors/GT-WT02/1/215/channel 1
sensors/GT-WT02/1/215/battery OK
sensors/GT-WT02/1/215/button 0
sensors/GT-WT02/1/215/temperature_C 26.100000
sensors/GT-WT02/1/215/humidity 56.000000
sensors/GT-WT02/1/215/mic CHECKSUM
sensors/THGR122N/2/238/id 238
sensors/THGR122N/2/238/channel 2
sensors/THGR122N/2/238/battery LOW
sensors/THGR122N/2/238/temperature_C 25.000000
sensors/THGR122N/2/238/humidity 59
^C

So - I can confirm, the things suggested above are working with the current master branch:
rtl_433 version 18.12-284-g7c6b415 branch master

Sorry, if this doesn’t help. Can’t provide more :wink:

Hello and thanks for all the answers.
I’ll try this in the next days. You will get a reply after this. Thanks

Hi I’m back again.

I had this release first…

rtl_433 version 18.12-278-g8204e1c branch master at 201908241304 inputs file rtl_tcp RTL-SDR
Trying conf file at "rtl_433.conf"...
Trying conf file at "/home/openhabian/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...
Generic RF data receiver and decoder for ISM band devices using RTL-SDR and SoapySDR.

after this I made a update to this.

[18:07:15] openhabian@openHABianPi:~/rtl_433_MQTT/build$ rtl_433 -V             rtl_433 version 18.12-290-g39f0cfe branch master at 201908281457 inputs file rtl_tcp RTL-SDR
Trying conf file at "rtl_433.conf"...
Trying conf file at "/home/openhabian/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...

After this I made a test with: rtl_433 -G

This was looking ok.

time      : 2019-08-28 18:13:05
model     : TS-FT002     Id        : 20
Depth     : 68           Temperature: 24.2 C       Transmit Interval: 180
Battery Flag?: 8         MIC       : CHECKSUM

At the end I send my primary command again:
rtl_433 -F “mqtt://192.168.178.42:1883,retain=0,devices=sensors[/id]”

And it worked!!

MQTT Connected...
MQTT Connection established.

But I seems to be a very special error before because of the ". Its a difference to use " instead of ”. If you make a copy and paste the browser or some community pages will switch this character. If you import it to the command line again it doesn’t work.

Thanks for your help…

Last question: Is there a simple way to see what mqtt broker is receiving on openhab?

With my coding no value comes to the thing. Its NaN. I want to see the receiving rows on my channel. maybe there is a error in my definition of the thing or something else.

I’ve got it. The MQTT Exploration App is a wonderful tool. :slight_smile:

When I close the box with

MQTT Connected…
MQTT Connection established.

No more rows will published to the broker. But I want it to send permanently. What is the mistake?

How do you deal with the rain data, does someone have a rule that calculates the daily rainfall, for me the mm² only grows and never resets

(spät aber doch)
yes, I am calculating the raindata with rules:
5minutes, hourly and daily. the source-data (that comes from rtl) need to have some persistance and then
*) every 5 minutes read the .deltaSince(now.minusMinutes(5)) of the rtl_data and write it into a new item e.g. regenrate_5m
*) at minute 59 of every hour read the .deltaSince(now.minusMinutes(60)) of the rtl_data and write it into a new item
) at minute 59 of hour 23 every day read the .deltaSince(now.minusMinutes(6024)) of the rtl_data and write it into a new item

but be careful with the rtl_data. my sensor eg. sends 2 different rain-values that mixed up my calculations

Good evening everyone, I’ve been banging my head and looking around for a while … I can’t understand how I can select only my weather station (Fineoffset-WH65B) and ignore that of my neighbor (Fineoffset -WH24).

I will then send the output to MQTT via the -F option
Thanks

hi gabriele
if you include type and subtype in the mqtt-config you can be very selective on what you get

-f "mqtt:127.0.0.1:1883,user=YOURUSER,pass=YOURPASSWORD,retain=0,devices=sensors[/type:0][/model:0][/subtype:0][/channel:0][/id:255][/protocol:255],events=event_rtl/,states=states_rtl/"

(btw: i have the same thing with this sensor and I am sure, my nighbour does not have one)