Push sensor data to luftdaten.info aka sensor.community using Tasmota and openHAB

»Sensor.Community is a contributors driven global sensor network that creates Open Environmental Data.« That’s what they say about themselves on their website. As far as I know they started with measuring fine dust in Stuttgart and then grew by time.

They are offering instructions to build airrohr, an ESP8266 based sensor platform to connect environmental sensors (BME280, DHT22, …) and fine dust sensors (SDS011, PPD42NS). Sadly this firmware pushes directly to their server, not to openHAB. As I didn’t get their software to boot anyways, I did it the other way round: Use Tasmota-sensors firmware, push data to my MQTT server and let openHAB push it to sensor.community - this is how I have done it.

Prerequisites

Software

  • openHAB 2.5 with *.jar file from here put to your openHAB addons folder
  • MQTT Server running and openHAB configured to use it

Hardware:

Setup

Tasmota Hardware

Configure Tasmota as usual to connect to your Wifi and push data to your MQTT server

I/O configuration for Tasmota using BME280 and SDS011 sensor connected as in Airrohr instructions:

image
Template as JSON:

{"NAME":"Airrohr Air Quality","GPIO":[6,148,5,149,101,70,0,0,0,0,0,0,0],"FLAG":0,"BASE":18}

Go to Tasmota console, enter STATUS 4 and look for the value after FlashChipId - this is an hexadecimal value. Change that to decimal, and you have your Sensor ID to register at https://devices.sensor.community/register

openHAB

replace YOURBROKER with the thing name of your broker
replace TASMOTAID with ID of your tasmota as seen in console
replace YOURSENSORID with the decimal ID of your sensor

tasmota.things

Thing mqtt:topic:tasmota:tasmota_TASMOTAID "Air Quality Balcony" (mqtt:broker:YOURBROKER) {
    Channels:
        Type switch : Reachable [stateTopic="tele/tasmota_TASMOTAID/LWT", transformationPattern="MAP:tasmota-reachable.map"]
        Type string : RestartReason [stateTopic="tele/tasmota_TASMOTAID/INFO3", transformationPattern="JSONPATH:$.RestartReason"]
        // old one, have to query it
        Type string : Version2 [stateTopic="stat/tasmota_TASMOTAID/STATUS2", transformationPattern="JSONPATH:$.StatusFWR.Version"]
        // new one - comes for free at startup
        Type string : Version [stateTopic="tele/tasmota_TASMOTAID/INFO1", transformationPattern="JSONPATH:$.Version"]
        Type number : RSSI [stateTopic="tele/tasmota_TASMOTAID/STATE", transformationPattern="JSONPATH:$.Wifi.RSSI"]
        Type string : WifiDowntime [stateTopic="tele/tasmota_TASMOTAID/STATE", transformationPattern="JSONPATH:$.Wifi.Downtime"]
        Type number : LoadAvg [stateTopic="tele/tasmota_TASMOTAID/STATE", transformationPattern="JSONPATH:$.LoadAvg"]
        Type number : Uptime [stateTopic="tele/tasmota_TASMOTAID/STATE", transformationPattern="JSONPATH:$.UptimeSec"]
        Type string : Result [stateTopic="stat/tasmota_TASMOTAID/RESULT"]
        Type number : BME280_Temperature [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.BME280.Temperature"]
        Type number : BME280_HumidityRel [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.BME280.Humidity"]
        Type number : BME280_DewPoint [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.BME280.DewPoint"]
        Type number : BME280_Pressure [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.BME280.Pressure"]
        Type number : SDS011_PM25 [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.SDS0X1['PM2.5']"]
        Type number : SDS011_PM10 [stateTopic="tele/tasmota_TASMOTAID/SENSOR", transformationPattern="JSONPATH:$.SDS0X1['PM10']"]
}

AirQuality.items

Switch               Tasmota_TASMOTAID_Reachable     "Reachable"            { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:Reachable" }
String               Tasmota_TASMOTAID_RestartReason "Restart Reason [%s]"  { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:RestartReason" }
String               Tasmota_TASMOTAID_Version       "Tasmota Version [%s]" { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:Version", channel="mqtt:topic:tasmota:tasmota_TASMOTAID:Version2" }
Number:Dimensionless Tasmota_TASMOTAID_RSSI          "RSSI [%d %%]"         { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:RSSI" }
String               Tasmota_TASMOTAID_WifiDowntime  "Wifi Downtime [%s]"   { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:WifiDowntime" }
Number:Dimensionless Tasmota_TASMOTAID_LoadAvg       "Load [%d %%]"         { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:LoadAvg" }
Number:Time          Tasmota_TASMOTAID_Uptime        "Uptime [%.1f s]"      { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:Uptime" }
String               Tasmota_TASMOTAID_Result        "Result [%s]"          { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:Result" }

Number:Temperature   BME280_Temperature "Temperature [%.1f °C]"   <temperature> { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:BME280_Temperature" }
Number:Dimensionless BME280_HumidityRel "Humidity [%.1f %%]"      <humidity>    { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:BME280_HumidityRel" }
Number:Temperature   BME280_DewPoint    "DewPoint [%.1f °C]"      <rain>        { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:BME280_DewPoint" }
Number:Pressure      BME280_Pressure    "Air Pressure [%.1f hPa]" <pressure>    { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:BME280_Pressure" }
Number:Density       SDS011_PM25        "PM 2.5 µm [%.1f µg/m³]"  <smoke>       { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:SDS011_PM25" }
Number:Density       SDS011_PM10        "PM 10 µm [%.1f µg/m³]"   <smoke>       { channel="mqtt:topic:tasmota:tasmota_TASMOTAID:SDS011_PM10" }

AirQualityToSensorCommunity.rules

var Timer changeTimerBME = null
var Timer changeTimerSDS = null
val String sensorID = "esp8266-YOURSENSORID"

rule "Send BME280 data to sensor.community"
when
    Item BME280_Temperature changed or
    Item BME280_HumidityRel changed or
    Item BME280_Pressure    changed
then
    if (changeTimerBME === null) {
         changeTimerBME = createTimer(now.plusSeconds(1), [ |
            var String url = "https://api.sensor.community/v1/push-sensor-data/"
            val String contentType = "application/json"
            val String content = '{
   "sensordatavalues":[
     {"value_type":"temperature","value":"' + (BME280_Temperature.state as Number).floatValue + '"},
     {"value_type":"humidity","value":"' + (BME280_HumidityRel.state as Number).floatValue + '"},
     {"value_type":"pressure","value":"' + (BME280_Pressure.state as Number).floatValue * 100 + '"}' + // API expects pressure in Pa, not in hPa
'  ]
}'
            val headers = newHashMap("X-Pin" -> "11", "X-Sensor" -> sensorID)
            val timeout = 60000

            // var String result =
            sendHttpPostRequest(url, contentType, content, headers, timeout)
            //logInfo("SensorCommunity", "sent »" + content + "«, result »" + result + "«")

            changeTimerBME = null // clear for another time
         ] )
   }  // else timer already running and we ignore trigger 
end

rule "Send SDS011 data to sensor.community"
when
    Item SDS011_PM25 changed or
    Item SDS011_PM10 changed
then
    if (changeTimerSDS === null) {
         changeTimerSDS = createTimer(now.plusSeconds(1), [ |
            var String url = "https://api.sensor.community/v1/push-sensor-data/"
            val String contentType = "application/json"
            val String content = '{
   "sensordatavalues":[
     {"value_type":"P1","value":"' + (SDS011_PM10.state as Number).floatValue + '"},
     {"value_type":"P2","value":"' + (SDS011_PM25.state as Number).floatValue + '"}
  ]
}'
            val headers = newHashMap("X-Pin" -> "1", "X-Sensor" -> sensorID)
            val timeout = 60000

            //var String result =
            sendHttpPostRequest(url, contentType, content, headers, timeout)
            //logInfo("SensorCommunity", "sent »" + content + "«, result »" + result + "«")

            changeTimerSDS = null // clear for another time
         ] )
   }  // else timer already running and we ignore trigger 
end

(I wrote two rules to later on be able to change refresh time of the sensors individually)

To adapt this to other sensor types, you will have to change the “X-Pin” header to reflect the correct pin - you see this get this during sensor registration.

2 Likes

Hi,
can you please also publish your Tasmota template file?
Would great.
Thanks Joerg

Sure, I edited and added the JSON too!

Perfect!
Thanks.

I’d never heard of the sensor.community project before this post, so thanks for bringing it to my attention!

I’ve just hacked together some Jython code to extract the current temperature and humidity data from sensors in my local area. This may prove useful to some!

Items

Number nTempExternal "Current temperature "
Number nHumidExternal "Current humidity"

personal_functions.py

In automation/lib/python/personal/personal_functions.py I now have the following. The comma separated numbers in the url is the bounding box of the area from which I want sensor data. You can determine what numbers would suit you best using http://bboxfinder.com/. I take all the temperature and humidity values within this area, and return the median of each.

import requests, json
def get_current_outdoor_conditions():
    url = "http://data.sensor.community/airrohr/v1/filter/box=52.488634,13.402548,52.530321,13.515759"

    if(is_json(r.content)):
        jsonDict = json.loads(r.content)

        listTemperatures = []
        listHumidities = []

        for key in jsonDict:
            for key2 in key["sensordatavalues"]:
                if key2["value_type"] == "temperature":
                    if is_number(str(key2["value"])): listTemperatures.append(float(key2["value"]))
                if key2["value_type"] == "humidity":
                    if is_number(str(key2["value"])): listHumidities.append(float(key2["value"]))

        returnDict = {"temperature":round(median(listTemperatures),1), "humidity":round(median(listHumidities),1)}

        return returnDict
    
    else:    
        return False

def is_number(s):
    try:
        float(s)
        return True
    except ValueError:
        return False

def is_json(j):
    try:
        json_object = json.loads(j)
        return True
    except ValueError as e:
        return False

def median(lst):
    n = len(lst)
    s = sorted(lst)
    return (sum(s[n//2-1:n//2+1])/2.0, s[n//2])[n % 2] if n else None

weather_rules.py

In automation/jsr223/python/personal/weather_rules.py I now have the following. The function defined above is called every ten minutes, and the returned average temperature and humidity is sent to the nTempExternal and nHumidExternal Items respectively.

import personal.personal_functions
reload(personal.personal_functions)
from personal.personal_functions import get_current_outdoor_temperature

@rule("Get local sensor data", description="Get local sensor data from https://github.com/opendata-stuttgart/meta/wiki/EN-APIs")
@when("Time cron 0 0/10 * ? * * *")
def get_local_sensor_data(event):
        
    currentConditions = get_current_outdoor_conditions()
    if currentConditions != False:
        events.sendCommand("nTempExternal", str(currentConditions["temperature"]))
        events.sendCommand("nHumidExternal", str(currentConditions["humidity"]))

Sitemap

image

Text item=nTempExternal label="Outside [%.1f °C]" icon="temperature"
Text item=nHumidExternal label="Outside [%.1f %%]" icon="humidity"

Just two things that I see in your code:

  • sum([float("nan"), 1., 1.]) yields nan - so if one of that sensors reports nan (which I sometimes see) you will not get any result
  • rater use median (included in statistics) than average - otherwise some extreme (and maybe even wrong) values destroy your result.

I’m just a bit confused about the high humidity results in Berlin, but it happens in other cities too - it’s always the DHT22 sensors maxing out at 99.9 % humidity for hours - I only could one BME280 reporting such values, other ones report smaller values.

I’m not actually in Berlin - I faked the bbox values for this example!

statistics comes as standard with Python3. Jython is stuck with Python2, so I’d need to pip it, then add it to the Java path and yada yada. I actually adjusted my code to reject the first few and last few values in each list, then averaging from the rest, so hopefully outliers are accounted for! I’ll update my previous post when I’m back at my PC.

Thanks for the feedback - much appreciated!

EDIT: Updated my previous post:

  • I stole a median function from StackOverflow
  • I now also use json in addition to requests as the requests implementation of a JSON reader was stack overflowing.
  • I check the data to make sure that I receive a number (thanks for the hint @StefanMUC!)
  • I check the data to make sure it is valid JSON: every once in a while some garbage characters are added to the API response, which invalidates the JSON* and makes json.loads panic with a ValueError. I just catch it and return False if there’s an issue.
    • *This may be the cause of the stack overflow using requests to parse the JSON. Maybe I’ll check one day, but because the is_json function uses the json module I’ll probably leave it as is.