Importing Weather Forecast from api.met.no (aka yr.no) via MQTT 2.x Binding

weather
mqtt
Tags: #<Tag:0x00007f0156ae8518> #<Tag:0x00007f0156ae83b0>

(sirius324) #1

Hello,

my goal was to receive the current outside weather temperatures from our norwegian weather service (yr.no). This service provides also data for all other locations using the european weather service. I wasn’t too happy to register at weatherundergrood and yahoo data wasn’t that good?

Dataflow: API --> Script (bash) --> XMLtoJSON --> MQTT --> OpenHab
Goal: Get the weather data and have the possibility to extract more info of the forecast.
Status: Archieved! :smiley:

Why yr.no/met.no: quite updated weather reports, no registration - free as in beer.

technical gotchas :

  1. large JSON files (without JSONPATH filter) cause out of memory errors using the current binding
  2. Finding the JSONPath was easy with: http://jsonpathfinder.com/ and http://jsonpath.com (The first has a nice point and click UI so that you can “browse” to your jsonpath :heart_eyes:

If you need to get started with MQTT 2.x try one of my earlier posts: Trouble with MQTT Bindings 2.x - eventsTriggered - Item stale

So here is what I have done:

  1. Script to to talk to yr.no / met.no
#!/bin/bash
# Getting data to MQTT broker
# Location is Karmøy. Where else would you be?
data=$(curl 'https://api.met.no/weatherapi/locationforecast/1.9/?lat=59.2735&lon=5.1941&msl=30' -X GET >/tmp/yr.xml)
#todo move to temporary yr file and utilize parameter
dataJson=$(python /usr/local/bin/openhab/xmltojson.py /tmp/yr.xml)
dataJson="${dataJson:1:${#dataJson}-2}"
echo $dataJson  >/tmp/yr.json
#$.weatherdata.product.time[0].@from
#x.weatherdata.product.time[0].location.temperature.@value
mosquitto_pub -t /Yr -f /tmp/yr.json
#utilize this one to find path http://jsonpathfinder.com/
mv /tmp/yr.json /tmp/yr.json.old
mv /tmp/yr.xml /tmp/yr.xml.old

  1. Converting the XML to JSON Script
user@openhab:/usr/local/bin/openhab# cat xmltojson.py
### Extremely simple XML --> JSON Parser
import xmltodict
import pprint
import json
import sys
fileName=sys.argv[1]
with open(fileName) as fd:
    doc = xmltodict.parse(fd.read())
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(json.dumps(doc))

Here is how I have created the MQTT channels and linked them to Items:

Detailed Channel Config:

JSONPath:

JSONPATH:$.weatherdata.product.time[0].location.temperature.@value

Result:

image

Again, grafana to the rescue to create graphs:

I hope you find this useful! :slight_smile:

Continously running the script

openhabcron@openhab:~$ crontab -l
21 * * * * /usr/local/bin/openhab/yr_hourly_outside_temp.sh

(Angelos) #2

Nice stuff!

I like the JSONPath transformation in the MQTTv2 Channel. This example can be used in multiple scenarios.

(moved the thread over to “Tutorials & Examples”) I think that it belongs there :+1:


(David Graeff) #3

The data is xml. Why not use the XPATH transformation to extract the correct value, instead of first converting it to json and then using the json path? ^^


(sirius324) #4

David, totally agree. I tried, but my time cap was hit (multiple times) and I found it more useful to have a solution - than not having one :).

My Issue was that the available tools (xmllint) that I have tried didn’t want to play with me. Can you assist with the correct XPATH transformations and i’ll be happy to update my howto / create a new one!


(David Graeff) #5

JSONPath is actually modelled after XPATH. Should be very close to what you have right now. Use a tool like https://www.freeformatter.com/xpath-tester.html to test the xml with the XPATH

My next question would of course be, why not using the Http binding, instead of the MQTT binding :smiley:

Cheers, David


(sirius324) #6

Thanks mate. I had multiple reasons.

1)utilizing MQTT I could update multiple clients using the same data. HTTP Data will be a “dump only” approach. Having a “mqtt subscriber” in debug mode will give me the possibility to TSHOOT any issues :).
2) MQTT 2.x examples (working) are very few off. So by learning, documenting and sharing I hope to give something back to the community :).