OH3 XPath

Hi all
Amongst a number of problems I have come across on migration (so much so I’ve gone back to 2.5 for now) is the inability to use XPATH.
I use ti to scrape XML data for both current weather from my weather station and weather forecast from the Met Office
Item file definition

Number OU_Outside_Temperature “Outside Temperature: [%.1f °C]” (WH1080, gTemperature, gWeather) [“CurrentTemperature”] {http="<[http://192.168.0.1/weather/wfrog-current.xml:60000:XPATH(/current/th1/temp/text())]"}

Firstly I cant see how I can use XPATH on the main GUI in OH3 and secondly this never worked when used with OH3.
Does it work??

Just to check: have you got the XPATH Transformation Service installed?

Note that your example is using the V1 HTTP binding, which no longer works in OH3. You will have to use the V3 binding, and re-configure your Items and now Things. In general, v1 bindings will no longer work in OH3.

Ah I see that’s what the problem is then - http binding - yes XPATH installed

So Create thing based on http binding

URL being

http://192.168.0.1/weather/wfrog-current.xml

Then add channel with XPATH configuration

/[name()=‘current’]/[name()=‘th1’]/*[name()=‘temp’]/text()

And link to appropriate item

Presumably the example you’ve given is for your weather station - how do you get the Met Office data? I’m currently doing this, but always interested in seeing how others go about it…

Like this using xpath on an item

Number T_Max_1 “Temperature: [%.1f °C]” (gForecast, gTemperature, gWeather) [“Temperature”] {http="<[http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/351492?res=daily&key=xxx:600000:XPATH(number((//Rep/@Dm)[1]))]"}

And then you process this forecast data in a rule?

Generally I just display it on basic UI - I use MAP to transform weather types and visibility to readable forms - and have icons for each type…well mostly, not quite got round sorting them all out

1 Like

Rather than re-invent the wheel - how have you pushed info to Openhab items from a python script using datapoint?

I think it might be me who re-invented the wheel! I’ll show you what I’ve done, but be warned: I’m not a programmer, nor do I work in IT: everything I’ve done is hacked together from other examples.

I am on openHAB 2.5.10.

I use the Jython Helper Libraries, installed as per this post.

I have installed the datapoint-python project.

In $OPENHAB_CONF/lib/python/configuration.py I have the following, defining my key:

datapoint_key = "xxxxxxxx-yyyy-zzzz-xxxx-yyyy"

In $OPENHAB_CONF/lib/python/personal/personal_functions.py I have the following. This function returns a boolean True or False depending on whether it is likely to rain at the specified site_id, time and date:

from datetime import datetime, timedelta
from core.actions import LogAction
import datapoint
import configuration
reload(configuration)
from configuration import datapoint_key

def will_it_rain(site_id, time=None, date=0):

    rain_flag = False

    # Create connection to DataPoint with your API key
    conn = datapoint.connection(api_key=datapoint_key)

    # Get a forecast for site_id with 3 hourly timesteps
    forecast = conn.get_forecast_for_site(site_id, "3hourly")

    if time == None:
        LogAction.logInfo("will_it_rain","Checking for rain all day")
        # Loop through all the timesteps in day date
        for timestep in forecast.days[date].timesteps:
            # Check to see if the chance of rain is more than 30% at any point
            if timestep.precipitation.value > 30:
                LogAction.logInfo("will_it_rain","Rain chance {}% at {}".format(timestep.precipitation.value,timestep.date))
                rain_flag = True
    else:
        LogAction.logInfo("will_it_rain","Checking for rain at a specific time")
        # Get the difference in time between now and the requested forecast time
        delta = time - datetime.now()

        # Check that the difference in time is into the future. If not, assume the request is for now
        if delta.total_seconds() > 0:
            if forecast.future(in_seconds=delta.total_seconds()).precipitation.value > 30:
                LogAction.logInfo("will_it_rain","Rain chance {}% at {}".format(forecast.future(in_seconds=delta.total_seconds()).precipitation.value,forecast.future(in_seconds=delta.total_seconds()).date))
                rain_flag = True
        else:
            if forecast.now().precipitation.value > 30:
                LogAction.logInfo("will_it_rain","Rain chance {}% at {}".format(forecast.future(in_seconds=delta.total_seconds()).precipitation.value,forecast.future(in_seconds=delta.total_seconds()).date))
                rain_flag = True
    
    return rain_flag

From my Jython rules within openHAB, I can then call:

will_it_rain(351492, date=0)

which will return whether it will rain at the site 351492 at any point today (date 0). Or

import datetime as dt

tomorrow_at_8 = dt.datetime.combine(dt.date.today() + dt.timedelta(days=1), dt.time(8,0))

will_it_rain(351492, time=tomorrow_at_8)

which will return whether it will rain at the site 351492 at 8am tomorrow morning. Or

from datetime import datetime

will_it_rain(351492, time=datetime.now())

which will return whether it will rain at the site 351492 now.

That’s all I’m doing with the DataPoint data at the moment. Not pretty whatsoever, and I am certainly considering moving to using the new HTTP binding when I migrate to OH3 - the fewer rules the better!

I’ve just been playing around with the new OH3 HTTP Binding, and I think I’ll migrate to using the Binding rather than my complicated setup I have on OH2.

Here’s the YAML code for HTTP Thing and Channel which reads the daily forecast, and extracts today’s maximum temperature. The data will refresh every 15 minutes (900 seconds) - DataPoint allows 100k per day, so this is more than safe!

You will need to have the JSONPATH Transformation Service installed (I used DataPoint’s JSON output rather than the XML output)

UID: http:url:DataPointForecast
label: HTTP URL Thing
thingTypeUID: http:url
configuration:
  authMode: BASIC
  ignoreSSLErrors: true
  baseURL: http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/351492?res=daily&key=xxxxxxxxxxxxxxxxxxxx
  refresh: 900
  commandMethod: GET
  timeout: 3000
  bufferSize: 2048
channels:
  - id: Forecast_Today_Temp_Max
    channelTypeUID: http:number
    label: Forecast Today Temperature Maximum
    description: ""
    configuration:
      mode: READONLY
      stateTransformation: JSONPATH:$.SiteRep.DV.Location.Period[0].Rep[0].Dm

I then created an Item to hold this data:

The YAML for the State Description field is as follows, formatting as an integer:

value: " "
config:
  pattern: "%d"

That looks interesting - I’ll have a look tomorrow. This is what I’m trying to do. I suspect JSON will be easier now than XPATH

I’ve been playing around some more with this. Whilst it was fairly intuitive using the UI, I found it quite slow. It is also not so easy to share configurations of the Items (and Links, which would come as a separate JSON) from the UI, so tonight I’ve had a play with using Things and Items files and it was a breeze!

At the moment I’m just playing around with daily forecasts, grabbing the temperature, feels-like temperature, precipitation and weather type for today and the next 4 days.

Things

Replace SITEID and APIKEY

Thing http:url:datapointforecast "Data Point Forecast" [
	baseURL = "http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/SITEID?res=daily&key=APIKEY",
	refresh = "900",
	ignoreSSLErrors = "true"
]
{
	Channels:
		Type number : forecast_today_temperature_maximum "Forecast Today Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[0].Rep[0].Dm"
		]
		Type number : forecast_today_temperature_feels_like_maximum "Forecast Today Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[0].Rep[0].FDm"
		]
		Type number : forecast_today_precipitation "Forecast Today Precipitation" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[0].Rep[0].PPd"
		]
		Type string : forecast_today_weather_type "Forecast Today Weather Type" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[0].Rep[0].W∩MAP:weathertype.map"
		]
		Type number : forecast_tomorrow_temperature_maximum "Forecast Tomorrow Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[1].Rep[0].Dm"
		]
		Type number : forecast_tomorrow_temperature_feels_like_maximum "Forecast Tomorrow Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[1].Rep[0].FDm"
		]
		Type number : forecast_tomorrow_precipitation "Forecast Tomorrow Precipitation" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[1].Rep[0].PPd"
		]
		Type string : forecast_tomorrow_weather_type "Forecast Tomorrow Weather Type" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[1].Rep[0].W∩MAP:weathertype.map"
		]
		Type number : forecast_day3_temperature_maximum "Forecast Day3 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[2].Rep[0].Dm"
		]
		Type number : forecast_day3_temperature_feels_like_maximum "Forecast Day3 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[2].Rep[0].FDm"
		]
		Type number : forecast_day3_precipitation "Forecast Day3 Precipitation" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[2].Rep[0].PPd"
		]
		Type string : forecast_day3_weather_type "Forecast Day3 Weather Type" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[2].Rep[0].W∩MAP:weathertype.map"
		]
		Type number : forecast_day4_temperature_maximum "Forecast Day4 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[3].Rep[0].Dm"
		]
		Type number : forecast_day4_temperature_feels_like_maximum "Forecast Day4 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[3].Rep[0].FDm"
		]
		Type number : forecast_day4_precipitation "Forecast Day4 Precipitation" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[3].Rep[0].PPd"
		]
		Type string : forecast_day4_weather_type "Forecast Day4 Weather Type" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[3].Rep[0].W∩MAP:weathertype.map"
		]
		Type number : forecast_day5_temperature_maximum "Forecast Day5 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[4].Rep[0].Dm"
		]
		Type number : forecast_day5_temperature_feels_like_maximum "Forecast Day5 Temperature Maximum" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[4].Rep[0].FDm"
		]
		Type number : forecast_day5_precipitation "Forecast Day5 Precipitation" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[4].Rep[0].PPd"
		]
		Type string : forecast_day5_weather_type "Forecast Day5 Weather Type" [
			mode = "READONLY",
			stateTransformation = "JSONPATH:$.SiteRep.DV.Location.Period[4].Rep[0].W∩MAP:weathertype.map"
		]
}

Items

These include semantic notation. Today’s forecast should appear in your model inside a Met Office equipment, outdoors.

//OUTDOOR GROUP
Group gOutdoor "Outdoor" <outdoor> ["Outdoor"]

//EQUIPMENT
Group gMetOfficeForecast "Met Office Forecast []" <sensor> (gOutdoor) ["Sensor"]

//TODAY
Number nForecastTodayTemperatureMaximum "Maximum Temperature [%d°C]" <temperature> (gMetOfficeForecast) ["Temperature", "Measurement"] { channel="http:url:datapointforecast:forecast_today_temperature_maximum" }
Number nForecastTodayTemperatureFeelsLikeMaximum "Feels like [%d°C]" <temperature> (gMetOfficeForecast) ["Temperature", "Measurement"] { channel="http:url:datapointforecast:forecast_today_temperature_feels_like_maximum" }
Number nForecastTodayPrecipitation "Rain [%d%%]" <rain> (gMetOfficeForecast) ["Rain", "Measurement"] { channel="http:url:datapointforecast:forecast_today_precipitation" }
String strForecastTodayWeatherType "Forecast" <status> (gMetOfficeForecast) ["Status"] { channel="http:url:datapointforecast:forecast_today_weather_type" }

//TOMORROW
Number nForecastTomorrowTemperatureMaximum "Forecast Tomorrow Temperature Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_tomorrow_temperature_maximum" }
Number nForecastTomorrowTemperatureFeelsLikeMaximum "Forecast Tomorrow Temperature Feels Like Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_tomorrow_temperature_feels_like_maximum" }
Number nForecastTomorrowPrecipitation "Forecast Tomorrow Precipitation [%d%%]" <rain> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_tomorrow_precipitation" }
String strForecastTomorrowWeatherType "Forecast Tomorrow Weather Type" <status> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_tomorrow_weather_type" }

//DAY3
Number nForecastDay3TemperatureMaximum "Forecast Day3 Temperature Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day3_temperature_maximum" }
Number nForecastDay3TemperatureFeelsLikeMaximum "Forecast Day3 Temperature Feels Like Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day3_temperature_feels_like_maximum" }
Number nForecastDay3Precipitation "Forecast Day3 Precipitation [%d%%]" <rain> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day3_precipitation" }
String strForecastDay3WeatherType "Forecast Day3 Weather Type" <status> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day3_weather_type" }

//DAY4
Number nForecastDay4TemperatureMaximum "Forecast Day4 Temperature Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day4_temperature_maximum" }
Number nForecastDay4TemperatureFeelsLikeMaximum "Forecast Day4 Temperature Feels Like Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day4_temperature_feels_like_maximum" }
Number nForecastDay4Precipitation "Forecast Day4 Precipitation [%d%%]" <rain> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day4_precipitation" }
String strForecastDay4WeatherType "Forecast Day4 Weather Type" <status> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day4_weather_type" }

//DAY5
Number nForecastDay5TemperatureMaximum "Forecast Day5 Temperature Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day5_temperature_maximum" }
Number nForecastDay5TemperatureFeelsLikeMaximum "Forecast Day5 Temperature Feels Like Maximum [%d°C]" <temperature> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day5_temperature_feels_like_maximum" }
Number nForecastDay5Precipitation "Forecast Day5 Precipitation [%d%%]" <rain> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day5_precipitation" }
String strForecastDay5WeatherType "Forecast Day5 Weather Type" <status> (gMetOfficeForecast) { channel="http:url:datapointforecast:forecast_day5_weather_type" }

Transforms

Ensure that the JSONPATH and MAP transforms are installed. I use the MAP transform to convert the weather type number into a description (from page 10 of the DataPoint reference PDF)

Save the below into a new file called weathertype.map into the transform folder.

NA=Not available
0=Clear night
1=Sunny day
2=Partly cloudy (night)
3=Partly cloudy (day)
4=Not used
5=Mist
6=Fog
7=Cloudy
8=Overcast
9=Light rain shower (night)
10=Light rain shower (day)
11=Drizzle
12=Light rain
13=Heavy rain shower (night)
14=Heavy rain shower (day)
15=Heavy rain
16=Sleet shower (night)
17=Sleet shower (day)
18=Sleet
19=Hail shower (night)
20=Hail shower (day)
21=Hail
22=Light snow shower (night)
23=Light snow shower (day)
24=Light snow
25=Heavy snow shower (night)
26=Heavy snow shower (day)
27=Heavy snow
28=Thunder shower (night)
29=Thunder shower (day)
30=Thunder
1 Like

I started migration yesterday - and yes using UI is very intensive.
I had a few other problems on migration I needed to resolve.
I can much more easily adapt what you have done above esp as my items are already defined in file - just needs a change to add the channel.
At some point datapoint is being turned off. Need work on a solution for that

They’ve been threatening that for well over a year, but it will be replaced by the unknown DataHub.

Yup - registered for datahub - cant access anything though.
Had a few problems with your code - my openhab didnt like MAP on the channel - so I stuck it on the sitemap…or the item now all is well

Interesting.

Is the MAP Transformation Service installed - presumably yes, because it works on an Item? Someone yesterday had an issue with a Transformation Service not being recognised, but uninstalling and reinstalling the Transformation Service worked!

Both are installed - Jsonpath is giving me the info
MAP is working when used elsewhere just not as you have used it

Hi, when migrating to OH3 my previous approach using XPATH failed. I learned in this thread this is due to the novel http binding. However, I am not able to change this code snippet from OH 2.5 to OH 3 compatible code.
String MyStatus “Status [%s]” {http="<[http://user:pw@192.168.0.167/api.html?part=redstats&label=smr:30000:XPATH(//reader/@status)]"}

Unfortunately, I am also not able to format it ithe proper way for this forum as well. Anyway, any hint greatly appreciated.

Please show us what you have tried so far!

Sorry, if I had any clue I would try it.