Australian Bureau of Meteorology integration with OpenHAB

Tutorials & Examples
, ,
1

The weather suppliers through the binding give widely different values for Australian forecasts.

The definitive site for Australians is the BOM (Bureau of Meteorology). They provide a full API with heaps of FTP files available for download. http://www.bom.gov.au/catalogue/data-feeds.shtml

I have just started learning how to use Python and XPath to retrieve the data and then update OpenHAB. Just running from the IDE at the moment and it works fine. Next steps to call on a regular basis from a rule in OpenHAB, add more parameters and maybe add some error checking.

Any suggestions welcome, will potter along with this and update progress. :slight_smile:

Here is the code running on Python 3 for temperature forecast for tomorrow in Perth, WA:

import urllib.request
from lxml import etree

url = 'ftp://ftp.bom.gov.au/anon/gen/fwo/IDW12300.xml'
print('Requesting file...')
rawPage = urllib.request.urlopen(url)
print('File received...')

read = rawPage.read()
tree = etree.XML(read)

print('Parsing data...')
Weather_Temp_Max_1 = tree.xpath("//forecast/area[@aac='WA_PT053']/forecast-period[@index='1']/element[@type='air_temperature_maximum']/text()")[0]
Weather_Temp_Min_1 = tree.xpath("//forecast/area[@aac='WA_PT053']/forecast-period[@index='1']/element[@type='air_temperature_minimum']/text()")[0]

print('Updating OpenHAB...')
urlBase = 'http://192.168.1.110:8080/classicui/CMD?'
rawPage = urllib.request.urlopen(urlBase + 'Weather_Temp_Max_1=' + Weather_Temp_Max_1)
rawPage = urllib.request.urlopen(urlBase + 'Weather_Temp_Min_1=' + Weather_Temp_Min_1)

print('End')

I was amazed at how little code you need to achieve this. Standing on the shoulders of Giants.

5 Likes
2

Thanks Andrew. I’ve been thinking about doing something like this for a while as I’m currently using Wunderground through the weather addon but would prefer more accurate/authoritative data.

1 Like
3 
import urllib.request
from lxml import etree

urlBase = 'http://192.168.1.110:8080/classicui/CMD?'
url = 'ftp://ftp.bom.gov.au/anon/gen/fwo/IDW12300.xml'

def setFWOItem(BOMName,OpenHABName,day,nodeType):
    value = tree.xpath("//forecast/area[@aac='WA_PT053']/forecast-period[@index='" + day + "']/" + nodeType + "[@type='" + BOMName + "']/text()")[0]
    OpenHABName = OpenHABName + "_" + day
    print("    " + OpenHABName + "=" + value)
    value = urllib.parse.quote_plus(value, safe='', encoding=None, errors=None)
    rawPage = urllib.request.urlopen(urlBase + OpenHABName + '=' + value)

print('Requesting file...')
rawPage = urllib.request.urlopen(url)
print('File received...')
read = rawPage.read()
tree = etree.XML(read)

print('Updating OpenHAB...')
setFWOItem("air_temperature_maximum","BOM_Temp_Max","1","element")
setFWOItem("air_temperature_minimum","BOM_Temp_Min","1","element")
setFWOItem("precipitation_range","BOM_Precipitation_Range","1","element")
setFWOItem("forecast_icon_code","BOM_Forecast_Icon_Code","1","element")
setFWOItem("precis","BOM_Precis","1","text")
setFWOItem("probability_of_precipitation","BOM_Precipitation","1","text")

print('End')

Item definitions:

Number BOM_Temp_Max_1 "Forecast max tomorrow [%.2f °C]" <temperature> (Weather)
Number BOM_Temp_Min_1 "Forecast min tomorrow [%.2f °C]" <temperature> (Weather)
String BOM_Precipitation_Range_1 "Forecast Rain tomorrow [%s]" (Weather)
String BOM_Precipitation_1 "Precip probability tomorrow [%s]" (Weather)
Number BOM_Forecast_Icon_Code_1 "Forecast Icon Code tomorrow [%d]" (Weather)
String BOM_Precis_1 "Forecast Condition tomorrow [%s]" (Weather)
2 Likes
4

This looks great Andrew, I to am in Perth, and have been looking for a way to integrate my ospi retic to openhab.

Do you know if there is a OH group here in Perth?

Chris

5

So Andrew, If you dont mind letting me know where did you save your script in the configurations folder?

6

I haven’t finished the integration with OH yet, this Python script just runs manually at the moment as a proof of concept. Will post here when I have more.

Not aware of an OH group in Perth.

7

I have placed the Python script in:

/home/pi/Weather.py

and call it from the OH rules with:

rule "Get weather from BOM"
when
    Time cron "0 0 0/1 1/1 * ? *" // update every hour
then    
	executeCommandLine("python3@@/home/pi/Weather.py@@")
end

Works nicely.

You need to install lxml for Python:

sudo apt-get install python3-lxml
8

Looks great, where do we find the location codes?

9

Thanks :slight_smile:

Full list of files here:

http://www.bom.gov.au/catalogue/anon-ftp.shtml

You need to use the one that says “City Forecast”:

IDW12300 City Forecast - Perth (WA)
IDN10064 City Forecast - Sydney (NSW)
IDV10450 City Forecast - Melbourne (VIC)

etc.

10

Nice work. I’ve been meaning to do similar with the metservice.com page for NZ weather. I worked up an alexa script to have her tell me the weather, and forecast, in a nice spoken string in my flash briefing if you’re interested.

11

Thanks. Good timing I just received my echo dot yesterday! Would be interested in how you did it.

12

If you are looking to integrate OpenSprinkler, there is a binding in OH2:

http://docs.openhab.org/addons/bindings/opensprinkler/readme.html

Haven’t used it myself.

13

Ah - OH2, the change of my life. One day I will migrate, but if it aint
broken…Its a long story going to OH2, but I have DMX on ArtNet protocol
which I dont think is supported yet. The learning curve for OH2 seems quite
steep too.

Chris

14

An alternative to using Python and executeCommandLine is to use the HTTP binding with caching, and the Javascript transform to pick the bits out of the JSON payload you want. For example, this link:

http://www.bom.gov.au/fwo/IDV60901/IDV60901.95936.json

could be used to pull the air temperature every 15 minutes. Untested example:

services/http.cfg:

bom.url=http://www.bom.gov.au/fwo/IDV60901/IDV60901.95936.json
bom.updateInterval=900000

items/bom.items:

Number AirTemp "Air Temp [%.1f °C]" { http="<[bom:900000:JS(airtemp.js)]" }

transform/airtemp.js:

result = JSON.parse(input).observations.data[0].air_temp;
1 Like
15

My code for the Alexa flash briefing skill is below. I had some help from one of the other guys on Geekzone on the alexa specific parts.

To make it work you need to create a custom flash briefing skill and host your code somewhere so isn’t plug and play i’m afraid. That being said, it shouldn’t be too tricky to modify the code to accept the BOM web link and locations.

 <?php

/*
    Usage:
        Set the feed url to http://alexa.vagabond.net.nz/metservice_weather.php?region=xxxxxx
        Where region is as defined on the metservice website
*/

    header('Content-Type: application/json');

    $data = array();

    //did they specify a location?
    $forecast_region = "lyall-bay";  //if not, default to Lyall Bay in honour of Disrespective :-)

    if(isset($_GET["region"]))
    {
        //is it valid? First, let's sanitise the variable
        $matches = array();
        if(preg_match("/([a-zA-Z\-]+)/", $_GET["region"], $matches))
        {
            $forecast_region = strtolower($matches[1]);
        }
    }

    //format the update time as UTC like amazon expects
    $current_time = date("U");
    $utc_time = $current_time - date("Z");
    $update_time = date("Y-m-d", $utc_time) . "T" . date("H:i:s", $utc_time) . ".0Z";

    $data["uid"] = "urn:uuid:1335c695-cfb8-4ebb-abbd-80da344efa6b";
    $data["updateDate"] = $update_time;
    $data["titleText"] = "Metservice Weather for " . $forecast_region;
    $data["mainText"] = get_forecast($forecast_region);
    $data["redirectionUrl"] = "http://www.metservice.com";

    echo json_encode($data);

    function get_forecast($region)
    {
        $directions = array('N' => 'northerly', 'S' => 'southerly', 'E' => 'easterly', 'W' => 'westerly', 'NE' => 'north-easterly', 'NW' => 'north-westerly', 'SE' => 'south-easterly', 'SW' => 'south-westerly');
        $cache_url = "http://newscache.vagabond.net.nz/metservice_cache.php?region=";

        //URL of targeted sites
        $forecastUrl = 'http://www.metservice.com/publicData/localForecast' . $region;

        if(!UR_exists($forecastUrl))
        {
            $result = "I was unable to find a forecast for " . $region;
        } else {
            //load from the cache (if not cached, we'll get it and cache it in the future)
            $rawJsonData = file_get_contents($cache_url . $region);
            $forecast = json_decode($rawJsonData);
            if(isset($forecast->{'error'}))
            {
                return $forecast->{'error'};
            }

            $forecastData = $forecast->{'forecastFile'}->{'days'}[0]->{'forecast'};
            $maxData = $forecast->{'forecastFile'}->{'days'}[0]->{'max'};
            $minData = $forecast->{'forecastFile'}->{'days'}[0]->{'min'};

            $tempNow = $forecast->{'obsFile'}->{'threeHour'}->{'temp'};
            $windDir = isset($directions[$forecast->{'obsFile'}->{'threeHour'}->{'windDirection'}])? $directions[$forecast->{'obsFile'}->{'threeHour'}->{'windDirection'}] : $forecast->{'obsFile'}->{'threeHour'}->{'windDirection'};
            $windNow = $forecast->{'obsFile'}->{'threeHour'}->{'windSpeed'};
            $tempFeels = $forecast->{'obsFile'}->{'threeHour'}->{'windChill'};

            $result = ('The weather today will be ' .$forecastData. ' with a high of ' .$maxData. ' degrees and low of ' .$minData. ' degrees. Right now it is ' .$tempNow. ' with ' .$windDir. ' winds of ' .$windNow. ' kilometers per hour which makes it feel like ' .$tempFeels. ' degrees');

        }

        return $result;
    }

    function UR_exists($url){
        //function copied from Patrick at StackOverflow
        //http://stackoverflow.com/questions/7684771/how-check-if-file-exists-from-the-url
       $headers=get_headers($url);
       return stripos($headers[0],"200 OK")? true:false;
    }
?>
16

@chrishiscox Yes, if your gear isn’t supported, that makes it hard to move over. I made the change over once it went general release, very smooth and enjoyable process. I found the learning curve not steep with all the helpful tutorials.

17

@watou That’s a really neat way of doing it, but unfortunately, it doesn’t appear as though the forecast files are available in json, only the observations.

18

Very nice Paul. Will have to find some time to experiment with that one. :slight_smile:

19

@davoque I have used your XMLparse python method above, in combination with @watou json snippet
and now have a 3 day forcast, and current weather conditions.

What I cant seem to find in any of the dats is TODAY’s forcast. Only tomorrow and the next 6 days.

Can anyone help?

20

Excellent work Andrew, glad it was of use and @watou 's addition is great too. @Andrew_Pawelski

Today’s forecast is day 0 in my def statement:

def setFWOItem(BOMName,OpenHABName,day,nodeType):

Call it with:

setFWOItem("air_temperature_maximum","BOM_Temp_Max","0","element")

This retrieves the nodes with:

<forecast-period index="0"

The forecasts for today are there early in the day but are dropped out of the XML as the day progresses, that’s why I trap any errors in the xpath call and ignore them. But, it’s OK as your OH items will remember the last value that you successfully retrieved from the XML when it was last there.

My code is now:

import urllib.request
from lxml import etree

urlBase = 'http://192.168.1.110:8080/classicui/CMD?'
url = 'ftp://ftp.bom.gov.au/anon/gen/fwo/IDW14199.xml'

def setFWOItem(BOMName,OpenHABName,day,nodeType):
    try:
        value = tree.xpath("//forecast/area[@aac='WA_PT058']/forecast-period[@index='" + day + "']/" + nodeType + "[@type='" + BOMName + "']/text()")[0]
        OpenHABName = OpenHABName + "_" + day
        print("    " + OpenHABName + "=" + value)
        value = urllib.parse.quote_plus(value, safe='', encoding=None, errors=None)
        rawPage = urllib.request.urlopen(urlBase + OpenHABName + '=' + value)
    except:
        print("    Error with " + BOMName + "::" + nodeType + "::" + day)
        

print('Requesting file...')
rawPage = urllib.request.urlopen(url)
print('File received...')
read = rawPage.read()
tree = etree.XML(read)

print('Updating OpenHAB...')
rawPage = urllib.request.urlopen(urlBase + 'BOMUpdating=ON')

setFWOItem("air_temperature_maximum","BOM_Temp_Max","0","element")
setFWOItem("precipitation_range","BOM_Precipitation_Range","0","element")
setFWOItem("forecast_icon_code","BOM_Forecast_Icon_Code","0","element")
setFWOItem("precis","BOM_Precis","0","text")
setFWOItem("probability_of_precipitation","BOM_Precipitation","0","text")

setFWOItem("air_temperature_maximum","BOM_Temp_Max","1","element")
setFWOItem("air_temperature_minimum","BOM_Temp_Min","1","element")
setFWOItem("precipitation_range","BOM_Precipitation_Range","1","element")
setFWOItem("forecast_icon_code","BOM_Forecast_Icon_Code","1","element")
setFWOItem("precis","BOM_Precis","1","text")
setFWOItem("probability_of_precipitation","BOM_Precipitation","1","text")

rawPage = urllib.request.urlopen(urlBase + 'BOMUpdating=OFF')
print('End')

Items:

Number BOM_Temp_Max_0 "Forecast max today [%.2f °C]" <temperature> (Weather) {mqtt=">[broker:/home/BOM/BOM_Temp_Max_0/state:state:*:default]"}
String BOM_Precipitation_Range_0 "Forecast Rain today [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precipitation_Range_0/state:state:*:default]"}
String BOM_Precipitation_0 "Precip probability today [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precipitation_0/state:state:*:default]"}
Number BOM_Forecast_Icon_Code_0 "Forecast Icon Code today [%d]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Forecast_Icon_Code_0/state:state:*:default]"}
String BOM_Precis_0 "Forecast Condition today [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precis_0/state:state:*:default]"}
Number BOM_Temp_Max_1 "Forecast max tomorrow [%.2f °C]" <temperature> (Weather) {mqtt=">[broker:/home/BOM/BOM_Temp_Max_1/state:state:*:default]"}
Number BOM_Temp_Min_1 "Forecast min tomorrow [%.2f °C]" <temperature> (Weather) {mqtt=">[broker:/home/BOM/BOM_Temp_Min_1/state:state:*:default]"}
String BOM_Precipitation_Range_1 "Forecast Rain tomorrow [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precipitation_Range_1/state:state:*:default]"}
String BOM_Precipitation_1 "Precip probability tomorrow [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precipitation_1/state:state:*:default]"}
Number BOM_Forecast_Icon_Code_1 "Forecast Icon Code tomorrow [%d]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Forecast_Icon_Code_1/state:state:*:default]"}
String BOM_Precis_1 "Forecast Condition tomorrow [%s]" (Weather) {mqtt=">[broker:/home/BOM/BOM_Precis_1/state:state:*:default]"}
Switch BOMUpdating "Data is being retrieved from BOM"

I am pushing out the values to MQTT as they come in so I can pick them up in an ESP8266 driving an analogue weather clock that I am building.

The switch “BOMUpdating” moves to ON when updating and to OFF when finished. This allows rules to be run when all of the weather items have been retrieved and updated.