Importing Weather Forecast from api.met.no (aka yr.no) via MQTT 2.x Binding

Hello,

my goal was to receive the current outside weather temperatures from our norwegian weather service (yr.no). This service provides also data for all other locations using the european weather service. I wasn’t too happy to register at weatherundergrood and yahoo data wasn’t that good?

Dataflow: API --> Script (bash) --> XMLtoJSON --> MQTT --> OpenHab
Goal: Get the weather data and have the possibility to extract more info of the forecast.
Status: Archieved! :smiley:

Why yr.no/met.no: quite updated weather reports, no registration - free as in beer.

technical gotchas :

  1. large JSON files (without JSONPATH filter) cause out of memory errors using the current binding
  2. Finding the JSONPath was easy with: http://jsonpathfinder.com/ and http://jsonpath.com (The first has a nice point and click UI so that you can “browse” to your jsonpath :heart_eyes:

If you need to get started with MQTT 2.x try one of my earlier posts: Trouble with MQTT Bindings 2.x - eventsTriggered - Item stale

So here is what I have done:

  1. Script to to talk to yr.no / met.no
#!/bin/bash
# Getting data to MQTT broker
# Location is Karmøy. Where else would you be?
data=$(curl 'https://api.met.no/weatherapi/locationforecast/1.9/?lat=59.2735&lon=5.1941&msl=30' -X GET >/tmp/yr.xml)
#todo move to temporary yr file and utilize parameter
dataJson=$(python /usr/local/bin/openhab/xmltojson.py /tmp/yr.xml)
dataJson="${dataJson:1:${#dataJson}-2}"
echo $dataJson  >/tmp/yr.json
#$.weatherdata.product.time[0].@from
#x.weatherdata.product.time[0].location.temperature.@value
mosquitto_pub -t /Yr -f /tmp/yr.json
#utilize this one to find path http://jsonpathfinder.com/
mv /tmp/yr.json /tmp/yr.json.old
mv /tmp/yr.xml /tmp/yr.xml.old

  1. Converting the XML to JSON Script
user@openhab:/usr/local/bin/openhab# cat xmltojson.py
### Extremely simple XML --> JSON Parser
import xmltodict
import pprint
import json
import sys
fileName=sys.argv[1]
with open(fileName) as fd:
    doc = xmltodict.parse(fd.read())
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(json.dumps(doc))

Here is how I have created the MQTT channels and linked them to Items:

Detailed Channel Config:

JSONPath:

JSONPATH:$.weatherdata.product.time[0].location.temperature.@value

Result:

image

Again, grafana to the rescue to create graphs:

I hope you find this useful! :slight_smile:

Continously running the script

openhabcron@openhab:~$ crontab -l
21 * * * * /usr/local/bin/openhab/yr_hourly_outside_temp.sh
4 Likes

Nice stuff!

I like the JSONPath transformation in the MQTTv2 Channel. This example can be used in multiple scenarios.

(moved the thread over to “Tutorials & Examples”) I think that it belongs there :+1:

1 Like

The data is xml. Why not use the XPATH transformation to extract the correct value, instead of first converting it to json and then using the json path? ^^

2 Likes

David, totally agree. I tried, but my time cap was hit (multiple times) and I found it more useful to have a solution - than not having one :).

My Issue was that the available tools (xmllint) that I have tried didn’t want to play with me. Can you assist with the correct XPATH transformations and i’ll be happy to update my howto / create a new one!

1 Like

JSONPath is actually modelled after XPATH. Should be very close to what you have right now. Use a tool like https://www.freeformatter.com/xpath-tester.html to test the xml with the XPATH

My next question would of course be, why not using the Http binding, instead of the MQTT binding :smiley:

Cheers, David

1 Like

Thanks mate. I had multiple reasons.

1)utilizing MQTT I could update multiple clients using the same data. HTTP Data will be a “dump only” approach. Having a “mqtt subscriber” in debug mode will give me the possibility to TSHOOT any issues :).
2) MQTT 2.x examples (working) are very few off. So by learning, documenting and sharing I hope to give something back to the community :).

1 Like

Hello,

Thanks for the idea ! I used the HTTP Binding for doing the same.

services/http.cfg

metNo.url=https://api.met.no/weatherapi/locationforecast/1.9/?lat=X&lon=Y&msl=Z
metNo.updateInterval=60000

X = latitude
Y = longitude
Z = altitude

items/temps.items

Number temperature_metNo "[%.1f]" {http="<[metNo:10000:XPATH(number(/weatherdata/product/time[1]/location/temperature/@value))]"}
Number humidity_metNo "[%.1f]" {http="<[metNo:10000:XPATH(number(/weatherdata/product/time[1]/location/humidity/@value))]"}

sitemaps/x.sitemap

Text item=temperature_metNo icon="temperature" label="Température extérieur [%.1f °C]"
Text item=humidity_metNo icon="humidity" label="humidité extérieur [%.1f rel%%]"

Best regards

1 Like

Great solution @Tycale

I’'ve been searching for a simple way to get the data used on Yr.no for my OpenHAB sitemap. Load up XPATH and the HTTP binding, use your sample above and all the data I need is available.

Just one little typo in the met.No.url above. There should be a “?” between “1.9/” and “lat”.

Thanks for sharing.

Thanks for the feedback, it is fixed :slight_smile:

For OH3, this is an example of a textual configuration for getting the snow depth at a particular location in Norway:

Thing http:url:yrmountain "HTTP thing for snow depth" [baseURL="https://frost.met.no/observations", refresh="3600", username="", username="exxxxxx8-9xx8-4xxb-8xxa-ba77xxxxxx8", password=""] {
    Channels:
        Type number : snowdepth "Snowdepth" [stateExtension="v0.jsonld?sources=SN51800&referencetime=latest&elements=surface_snow_thickness", mode="READONLY", 
                                          stateTransformation="JSONPATH:$.data[0].observations[0].value"]
        Type number : temperatur "Temperatur" [stateExtension="v0.jsonld?sources=SN51800&referencetime=latest&elements=air_temperature", mode="READONLY",
                                          stateTransformation="JSONPATH:$.data[0].observations[0].value"]
}

and associated items:

Number Snowdepth {channel="http:url:yrmountain:snodybde", expire="48h"}
Number Mountaintemperature {channel="http:url:yrmountain:temperatur", expire="48h"}

You need to obtain a client id via a user account at https://api.met.no, and obtain the location id via https://frost.met.no/sources/v0.jsonld

1 Like