Australian Bureau of Meteorology integration with OpenHAB

Tutorials & Examples
, ,
122

Hi Kris

I have commented out all loginfos but still get the whole xml file showing in the log

                <element type="precipitation_range">0 to 1 mm</element>
                <element type="air_temperature_minimum" units="Celsius">21</element>
                <element type="air_temperature_maximum" units="Celsius">36</element>
                <text type="precis">Possible shower.</text>
                <text type="probability_of_precipitation">40%</text>
            </forecast-period>
            <forecast-period index="2" start-time-local="2019-02-07T00:00:00+10:30" end-time-local="2019-02-08T00:00:00+10:30" start-time-utc="2019-02-06T13:30:00Z" end-time-utc="2019-02-07T13:30:00Z">
                <element type="forecast_icon_code">3</element>
                <element type="air_temperature_minimum" units="Celsius">21</element>
                <element type="air_temperature_maximum" units="Celsius">37</element>
                <text type="precis">Mostly sunny.</text>
                <text type="probability_of_precipitation">10%</text>
            </forecast-period>
            <forecast-period index="3" start-time-local="2019-02-08T00:00:00+10:30" end-time-local="2019-02-09T00:00:00+10:30" start-time-utc="2019-02-07T13:30:00Z" end-time-utc="2019-02-08T13:30:00Z">
                <element type="forecast_icon_code">3</element>
                <element type="air_temperature_minimum" units="Celsius">15</element>
                <element type="air_temperature_maximum" units="Celsius">29</element>
                <text type="precis">Mostly sunny.</text>
                <text type="probability_of_precipitation">0%</text>
            </forecast-period>
            <forecast-period index="4" start-time-local="2019-02-09T00:00:00+10:30" end-time-local="2019-02-10T00:00:00+10:30" start-time-utc="2019-02-08T13:30:00Z" end-time-utc="2019-02-09T13:30:00Z">
                <element type="forecast_icon_code">1</element>
                <element type="air_temperature_minimum" units="Celsius">10</element>
                <element type="air_temperature_maximum" units="Celsius">28</element>
                <text type="precis">Sunny.</text>
                <text type="probability_of_precipitation">0%</text>
            </forecast-period>
            <forecast-period index="5" start-time-local="2019-02-10T00:00:00+10:30" end-time-local="2019-02-11T00:00:00+10:30" start-time-utc="2019-02-09T13:30:00Z" end-time-utc="2019-02-10T13:30:00Z">
                <element type="forecast_icon_code">1</element>
                <element type="air_temperature_minimum" units="Celsius">11</element>
                <element type="air_temperature_maximum" units="Celsius">30</element>
                <text type="precis">Sunny.</text>
                <text type="probability_of_precipitation">0%</text>
            </forecast-period>
            <forecast-period index="6" start-time-local="2019-02-11T00:00:00+10:30" end-time-local="2019-02-12T00:00:00+10:30" start-time-utc="2019-02-10T13:30:00Z" end-time-utc="2019-02-11T13:30:00Z">
                <element type="forecast_icon_code">3</element>
                <element type="air_temperature_minimum" units="Celsius">14</element>
                <element type="air_temperature_maximum" units="Celsius">32</element>
                <text type="precis">Mostly sunny.</text>
                <text type="probability_of_precipitation">0%</text>
            </forecast-period>
        </area>
    </forecast>
</product>
' to a state type which item 'BOM_Forecast_Icon_Code_6' accepts: [DecimalType, QuantityType, UnDefType].
123

All 30,000 lines of itā€¦ When it does run its the only thing in the whole log fileā€¦

2019-02-05 13:30:02.408 [WARN ] [rthome.model.script.actions.BusEvent] - Cannot convert '<?xml version="1.0" encoding="UTF-8"?>
<product version="1.7" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.bom.gov.au/schema/v1.7/product.xsd">
    <amoc>
        <source>
            <sender>Australian Government Bureau of Meteorology</sender>
            <region>South Australia</region>
            <office>SARO</office>
            <copyright>http://www.bom.gov.au/other/copyright.shtml</copyright>
            <disclaimer>http://www.bom.gov.au/other/disclaimer.shtml</disclaimer>
        </source>
        <identifier>IDS10044</identifier>
        <issue-time-utc>2019-02-04T18:45:00Z</issue-time-utc>
        <issue-time-local tz="CDT">2019-02-05T05:15:00+10:30</issue-time-local>
        <sent-time>2019-02-04T18:45:08Z</sent-time>
        <expiry-time>2019-02-05T18:45:00Z</expiry-time>
        <validity-bgn-time-local tz="CDT">2019-02-05T05:00:00+10:30</validity-bgn-time-local>
        <validity-end-time-local tz="CDT">2019-02-11T23:59:59+10:30</validity-end-time-local>
        <next-routine-issue-time-utc>2019-02-05T05:30:00Z</next-routine-issue-time-utc>
        <next-routine-issue-time-local tz="CDT">2019-02-05T16:00:00+10:30</next-routine-issue-time-local>
        <status>O</status>
        <service>WSP</service>
        <sub-service>FPR</sub-service>
        <product-type>F</product-type>
        <phase>NEW</phase>
    </amoc>
    <forecast>
        <area aac="SA_FA001" description="South Australia" type="region"/>
        <area aac="SA_PW001" description="Adelaide Metropolitan" type="public-district" parent-aac="SA_FA001"/>
        <area aac="SA_PT001" description="Adelaide" type="location" parent-aac="SA_PW001">
            <forecast-period index="0" start-time-local="2019-02-05T05:00:00+10:30" end-time-local="2019-02-06T00:00:00+10:30" start-time-utc="2019-02-04T18:30:00Z" end-time-utc="2019-02-05T13:30:00Z">
                <element type="forecast_icon_code">3</element>
                <element type="air_temperature_maximum" units="Celsius">32</element>
                <text type="precis">Mostly sunny.</text>
                <text type="probability_of_precipitation">0%</text>
            </forecast-period>
124

Can you provide a

log:display

from the console, that certainly doesnā€™t happen to me with the rule above.

125

sorry iā€™m on windows 10, do you mean a copy of the openhab.log file? i did paste part of the log file above. Australian Bureau of Meteorology integration with OpenHAB - #122 by menzy

I have also checked my persistence & it looks like itā€™s not persisting anything in the forecast rule. the observation rule is working great.

126

here is my complete rule file

import java.io.InputStreamReader;
import java.nio.CharBuffer;
import java.net.URL;
import java.lang.StringBuilder;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

rule "Get weather forecast data from BOM"
when
	Time cron "0 */5 * ? * *" // update every 10 minutes
//	Time cron "0 0 */6 ? * *" // update every 6 hours
then    
	//logInfo("BOM data","reading forecast data")
	val URL url = new URL("ftp://ftp.bom.gov.au/anon/gen/fwo/IDS10044.xml");
    val InputStreamReader instream = new InputStreamReader(url.openStream())
	var StringBuilder sb = new StringBuilder();
	var CharBuffer cb = CharBuffer.allocate(1024);

	var int read = instream.read ( cb );
	while(read != -1) {
    	sb.append(cb.array(), 0, read )
		cb.clear
		read = instream.read( cb )
	}
	instream.close();

	val String bom_data = sb.toString();	

	for(var int i=0; i<7; i++)
	{
		//logInfo("BOM data","day index="+i)
		val String start_time = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/@start-time-local", bom_data)
		//logInfo("BOM data","start_time="+start_time)
		postUpdate("BOM_Date_"+i, start_time)

		val String air_temperature_maximum = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/element[@type='air_temperature_maximum']/text()", bom_data)
		//logInfo("BOM data","air_temperature_maximum="+air_temperature_maximum)
		postUpdate("BOM_Temp_Max_"+i, air_temperature_maximum)

		val String air_temperature_minimum = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/element[@type='air_temperature_minimum']/text()", bom_data)
		//logInfo("BOM data","air_temperature_minimum="+air_temperature_minimum)
		postUpdate("BOM_Temp_Min_"+i, air_temperature_minimum)

		val String precipitation_range = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/element[@type='precipitation_range']/text()", bom_data)
		//logInfo("BOM data","precipitation_range="+precipitation_range)
		postUpdate("BOM_Precipitation_Range_"+i, precipitation_range)

		val String forecast_icon_code = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/element[@type='forecast_icon_code']/text()", bom_data)
		//logInfo("BOM data","forecast_icon_code="+forecast_icon_code)
		postUpdate("BOM_Forecast_Icon_Code_"+i, forecast_icon_code)

		var String precis = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/text[@type='precis']/text()", bom_data)
		//logInfo("BOM data","precis="+precis)
		postUpdate("BOM_Precis_"+i, precis)

		val String probability_of_precipitation = transform ("XPATH", "//forecast/area[@aac='SA_PT013']/forecast-period[@index='"+i+"']/text[@type='probability_of_precipitation']/text()", bom_data)
		//logInfo("BOM data","probability_of_precipitation="+probability_of_precipitation)
		postUpdate("BOM_Precipitation_"+i, probability_of_precipitation)

		val String forecast = transform ("XPATH", "//forecast/area[@aac='SA_PW001']/forecast-period[@index='"+i+"']/text[@type='forecast']/text()", bom_data)
		//logInfo("BOM data","forecast="+forecast)
		postUpdate("BOM_Forecast_"+i, forecast)
		var String wind_range = "";
		var Pattern pattern = Pattern.compile("([0-9]+ to [0-9]+ km/h)");
		var Matcher matcher = pattern.matcher(forecast);
		if (matcher.find())
		{
    		wind_range = matcher.group(1);
		}
		else if (forecast.indexOf("Light winds")>=0)
		{
			wind_range = "Light winds"
		}
		//logInfo("BOM data","wind_range_"+i+"="+wind_range)
		postUpdate("BOM_Wind_Range_"+i, wind_range)
	}
end

rule "Get weather observation data from BOM"
when
	Time cron "0 */10 * ? * *" // update every 10 minutes
then    
	//logInfo("BOM data","reading observation data")
	val URL url = new URL("http://www.bom.gov.au/fwo/IDS60801/IDS60801.95676.json");
    val InputStreamReader instream = new InputStreamReader(url.openStream())
	var StringBuilder sb = new StringBuilder();
	var CharBuffer cb = CharBuffer.allocate(1024);

	var int read = instream.read ( cb );
	while(read != -1) {
    	sb.append(cb.array(), 0, read )
		cb.clear
		read = instream.read( cb )
	}
	instream.close();

	val String bom_data = sb.toString();	

	var String time = transform("JSONPATH", "$.observations.data[0].local_date_time", bom_data)
	time = time.substring(time.indexOf('/')+1,time.length())
	//logInfo("BOM data","observation_time="+time)
	postUpdate("BOM_Observation_Time", time)

	val air_temp = transform("JSONPATH", "$.observations.data[0].air_temp", bom_data)
	//logInfo("BOM data","air_temp="+air_temp)
	postUpdate("BOM_AirTemp", air_temp)
	val apparent_t = transform("JSONPATH", "$.observations.data[0].apparent_t", bom_data)
	//logInfo("BOM data","apparent_t="+apparent_t)
	postUpdate("BOM_ApparentTemp", apparent_t)
	val wind_dir = transform("JSONPATH", "$.observations.data[0].wind_dir", bom_data)
	//logInfo("BOM data","wind_dir="+wind_dir)
	postUpdate("BOM_WindDir", wind_dir)
	val wind_spd = transform("JSONPATH", "$.observations.data[0].wind_spd_kmh", bom_data)
	//logInfo("BOM data","wind_spd="+wind_spd)
	postUpdate("BOM_WindSpeed", wind_spd)
	val rain = transform("JSONPATH", "$.observations.data[0].rain_trace", bom_data)
	//logInfo("BOM data","rain="+rain)
	postUpdate("BOM_Rain_Amount", rain)
	val rel_hum = transform("JSONPATH", "$.observations.data[0].rel_hum", bom_data)
	//logInfo("BOM data","rel_hum="+rel_hum)
	postUpdate("BOM_Relative_Humidity", rel_hum)
	
end
127

No, i mean the Karaf Console. Load that up and look at your log settings

128

@menzy the ā€œ[rthome.model.script.actions.BusEvent] - Cannot convertā€ warning is related to the JSONPath and XPath Transformations. This warning is dumping the XML.

Do you have these transformations installed in the add-ons?

1 Like
129

Nice I didnt even see that log file! ha

130

Hi Guys

Thanks I was missing XPath Transformations. It is now working as expected!

131

@JamesC have you had any issues with this? the console reports the number fine, 26 in this case but I get this error

20:30:01.748 [WARN ] [arthome.model.script.actions.BusEvent] - Cannot convert '' to a state type which item 'BOM_Temp_Min_0' accepts: [DecimalType, QuantityType, UnDefType].
132

Hi guys,

I have created a binding for this. For those who are interested please go to the following link for details https://github.com/tomitan100/org.openhab.binding.bom.

To install this binding you will need to install Eclipse IoT Market add-on first under MISC in Paper UI. The binding, called ā€œAustralian BOM Weather Forecast Bindingā€, should then appear in Bindings page for you to install.

I have not tested all Australian states/regions/stations. Please let me know if there is any issue.

Cheers,
Tom

2 Likes
133

Hi Tom,

Set mine up using Terrey Hills but it seems the Observation element is throwing a stack trace.

ftp://ftp.bom.gov.au/anon/gen/fwo/IDN60920.xml
Weather station ID : 94759

2019-02-14 16:31:32.886 [INFO ] [nhab.binding.bom.internal.BomHandler] - Processing forecast data from FTP path: ftp://ftp.bom.gov.au/anon/gen/fwo/IDN10064.xml, area ID: NSW_PT139
2019-02-14 16:31:33.595 [INFO ] [nhab.binding.bom.internal.BomHandler] - Completed processing forecast data.
2019-02-14 16:31:34.445 [WARN ] [mmon.WrappedScheduledExecutorService] - Scheduled runnable ended with an exception:
java.lang.NumberFormatException: null
at java.math.BigDecimal.(BigDecimal.java:494) ~[?:?]
at java.math.BigDecimal.(BigDecimal.java:383) ~[?:?]
at java.math.BigDecimal.(BigDecimal.java:806) ~[?:?]
at java.math.BigDecimal.valueOf(BigDecimal.java:1274) ~[?:?]
at org.eclipse.smarthome.core.library.types.DecimalType.(DecimalType.java:53) ~[?:?]
at org.openhab.binding.bom.internal.BomHandler.updateChannelState(BomHandler.java:351) ~[?:?]
at org.openhab.binding.bom.internal.BomHandler.lambda$2(BomHandler.java:164) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:?]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:?]
at org.openhab.binding.bom.internal.BomHandler.refreshObservation(BomHandler.java:146) ~[?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
at java.lang.Thread.run(Thread.java:748) [?:?]

If I use the first station defined in the file (94768) it works, similar to your Perth example, so not sure if itā€™s the values from my station or the fact itā€™s further down the file.

Regards,
Nick

134

Hi Nick,

I have fixed this. Your weather station does not have atmospheric pressure data and the code was not handling the exception correctly. Also there seems to be very limited forecast data for the area you chose. I will confirm this comparing with BOM site later today.

Please uninstall and then reinstall. You might have to remove and re-add your thing, and possibly restart.

Regards,
Tom

135

Hi Nick,

I think you want to use ftp://ftp.bom.gov.au/anon/gen/fwo/IDN11060.xml for the forecast instead. You will get 8 days of forecasts but there is one slight problem I see. You will not get the forecast texts as they are in IDN10064. I will have to work out how the XML is referenced.

Regards,
Tom

136

Ahh I see, when I saw the limited data, I assumed it derived it from the parent-aac but looking at the normal bom website, at the bottom is says : Product derived from IDN11060 and IDN10064

137

Good observation. Looks like you do actually need both for a complete forecast. I have modified the code to derive from both.

If you reinstall the binding now you will notice the configuration screen has changed. You will need to enter both precis forecast product ID and city/town forecast product ID.

See screenshot below for what you need to enter for Terrey Hills.

configuration
configuration.png1153Ɨ803 46.5 KB

138

Thanks! Looks good, will let you know if anything else crops up. Pity you cannot easily see the Area ID on the BOM website as you can pull the other values from it and saves jumping into the various xml. I assume they must have some table that links your area to the closest station (and all itā€™s various IDā€™s and files)?

139

Really great work on this @tomitan. Thank you for your efforts

The binding works quite well and I look forward to creating a nice habpanel for the data.

I did want to bring to your attention a number of WARNings I experienced in the logs (a snippet is below):

11:41:09.077 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#dewPoint although the handler was already disposed.
11:41:09.079 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#relativeHumidity although the handler was already disposed.
11:41:09.081 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#pressure although the handler was already disposed.
11:41:09.082 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#windDirection although the handler was already disposed.
11:41:09.084 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#windDirectionDegrees although the handler was already disposed.
11:41:09.085 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#windSpeedKmh although the handler was already disposed.
11:41:09.087 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#windSpeedKnots although the handler was already disposed.
11:41:09.088 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler of thing bom:weather:884f09fe tried updating channel day1#rainfall although the handler was already disposed.
11:41:09.089 [WARN ] [e.core.thing.binding.BaseThingHandler] - Handler BomHandler tried updating the thing status although the handler was already disposed.

This did not occur initially and only occurred after I had started to add items (by text file). A search of the forums produced the link below. I admit I did disable/enable the binding several times within a few minutes via Paper UI which may have caused the issue.
https://community.openhab.org/t/handler-disposed/49168/13

Not sure if that helps any.

Just wanted to say thanks again for taking the time to produce something like this. It is really appreciated.

140

Probably worth pasting your item file contentsā€¦

141

Yes they would have some mapping tables/files. Probably not available to the public. I couldnā€™t find anything in the product catalog but if you do find something let me know.