Decoding the OWL Intuition XML scheme for electricity monitoring

Tags: #<Tag:0x00007f618c7a8ba0> #<Tag:0x00007f618c7a8ab0> #<Tag:0x00007f618c7a89c0>

I am trying to think about how to capture and display the details of my electricity monitoring system, teh OWL Intuition. It outputs a UDP string which I can capture through an Item;

String TestSensor       "TestState [%s]"        (gUnit1)        { udp="<[192.168.0.1:*:XPATH(/electricity id/timestamp/signal rssi/battery level/chan id/curr units)]" }

It produces a string into the item that is shown below;

 <electricity id='12345678912C'><timestamp>1499337272</timestamp><signal rssi='-42' lqi='0'/><battery level='100%'/><chan id='0'><curr units='w'>402.00</curr><day units='wh'>4688.50</day></chan><chan id='1'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan><chan id='2'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan></electricity> to <electricity id='44371900585C'><timestamp>1499337332</timestamp><signal rssi='-42' lqi='0'/><battery level='100%'/><chan id='0'><curr units='w'>402.00</curr><day units='wh'>4695.20</day></chan><chan id='1'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan><chan id='2'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan></electricity>

I have been reading up on the XLST construct and think I need to use that in some fashion but I just cannot get it to do anything. Do I set this formatting up through this binding at the item or in a rule to do the processing of the string?

I am still a bit of a novice at this but I am eager to learn and improve!

Ultimately I want to get his into a HABPanel widget to show the electricity consumption and into my influxDB persistence so I can track over time but wanted to work on getting the information in the right format first!

Many thanks,

Martin.

The XPATH you are already trying to use is probably the best approach.

The only problem is the path you are trying to use is incorrect. XML is hierarchical. When you see an open tag (e.g. <electricity>) everything that occurs after that until the closing tag (e.g. </electricity>) is inside it. It is a lot like folders/directory structure on your hardrive on your computer.

The xpath is like the path to a folder on your hard drive.

Furthermore, element names cannot have a space. The stuff after the space is an attribute (e.g. “id=12345678912C” is an attribute on the “electricity” element. In xpath you reference an attribute using different syntax.

I have no idea what information you need form this XML. If you only need one piece of information then you can use the XPATH to retrieve just that piece of data. For example, to get the value of curr from chan 0:

XPATH(//electricity/chan[@id=0]/curr)

I highly recommend going through the XML and XPATH tutorials at W3 schools.

However, if there is more than one piece of information you need from this one message, you will need to create a Rule and extract each piece of information individually from the message and postUpdate or sendCommand them. You can still use XPATH and the transform action

TestSensorChan0.postUpdate(transform("XPATH", "//electricity/chan[@id=0]/curr", TestSensor.state.toString))

rikoshak,

That is great and thank you for the steer towards the tutorials I will get on those now. I will have a play with the XPATH to extract each bit of info I need.

Again thanks and I will report back success or not.

Many thanks

Martin.

Rik,

I have modified the TestSensor Item entry as follows and no dice I am afraid. I am still getting the whole string captured against the string variable in the item.

String TestSensor	"TestState [%s]"	(gUnit1)	{udp="<[192.168.0.1:*:XPATH(//electricity/chan[@id=0]/curr)]"}

Also tried the rule based approach and it throws exceptions. I checked the XPATH syntax in my code to make sure I hadnt mis typed etc. on this free checker (https://www.freeformatter.com/xpath-tester.html#ad-output) and that come sout good so there is something a miss with the openhab2 transform end a config I am missing.

XPATH is installed via PaperUI not got any .cfg file for it as far as I can see.

Cheers,

Martin.

Not all xpath implementations are always the same. There might be a problem with the way it handles the attribute. Search StackOverflow for “xpath select attribute” and you will find lots of postings with several different variations.

Always always post the exceptions. We have no idea what could be going on without knowing what they are.

Ok apologies on that one. Set the log to DEBUG and here we are;

2017-07-07 15:41:32.603 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'XPATH': transformation throws exceptions
2017-07-07 15:41:32.584 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'XPATH': transformation throws exceptions
2017-07-07 15:41:32.581 [INFO ] [eclipse.smarthome.model.script.debug] - The data string is.....<electricity id='12345678987C'><timestamp>1499438492</timestamp><signal rssi='-41' lqi='0'/><battery level='100%'/><chan id='0'><curr units='w'>402.00</curr><day units='wh'>6237.05</day></chan><chan id='1'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan><chan id='2'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan></electricity>
2017-07-07 15:41:32.609 [INFO ] [eclipse.smarthome.model.script.debug] - XPATH data is ...... <electricity id='12345678987C'><timestamp>1499438492</timestamp><signal rssi='-41' lqi='0'/><battery level='100%'/><chan id='0'><curr units='w'>402.00</curr><day units='wh'>6237.05</day></chan><chan id='1'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan><chan id='2'><curr units='w'>0.00</curr><day units='wh'>0.00</day></chan></electricity>
2017-07-07 15:41:32.575 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'XPATH': transformation throws exceptions
2017-07-07 15:41:32.610 [DEBUG] [org.eclipse.jetty.io.ChannelEndPoint] - flushed 5724 SelectChannelEndPoint@161b90d{/192.168.1.5:60683<->8080,Open,in,out,-,W,7/30000,HttpConnection}{io=0,kio=0,kro=1}
2017-07-07 15:41:32.614 [DEBUG] [.eclipse.jetty.server.HttpConnection] - org.eclipse.jetty.server.HttpConnection$SendCallback@1def4eb[PROCESSING][i=null,cb=Blocker@11085ba{null}] generate: DONE (null,[p=5716,l=5716,c=32768,r=0],false)@COMMITTED

Is that enough or do you need some more to help out?

Cheers,

Martin.

Well, that wasn’t as informative as I thought it might be. But it is clear it doesn’t like

I’m going to guess that either it doesn’t like the way you are matching on the attribute.

Got some more info in these exceptions after an openhab2 update.

org.openhab.core.transform.TransformationException: transformation throws exceptions
        at org.openhab.core.transform.TransformationHelper$TransformationServiceDelegate.transform(TransformationHelper.java:62)[194:org.openhab.core.compat1x:2.1.0]
        at org.openhab.binding.tcp.protocol.internal.UDPBinding.transformResponse(UDPBinding.java:263)[193:org.openhab.binding.tcp:1.10.0]
        at org.openhab.binding.tcp.protocol.internal.UDPBinding.parseBuffer(UDPBinding.java:159)[193:org.openhab.binding.tcp:1.10.0]
        at org.openhab.binding.tcp.AbstractDatagramChannelBinding.parseChanneledBuffer(AbstractDatagramChannelBinding.java:1014)[193:org.openhab.binding.tcp:1.10.0]
        at org.openhab.binding.tcp.AbstractDatagramChannelBinding.execute(AbstractDatagramChannelBinding.java:1551)[193:org.openhab.binding.tcp:1.10.0]
        at org.openhab.core.binding.AbstractActiveBinding$BindingActiveService.execute(AbstractActiveBinding.java:157)[194:org.openhab.core.compat1x:2.1.0]
        at org.openhab.core.service.AbstractActiveService$RefreshThread.run(AbstractActiveService.java:169)[194:org.openhab.core.compat1x:2.1.0]

Dont know if this throws any more light on it?

just to be on the safe side: do you have the xpath transform installed (it is not the same as xslt)?

with kind regards,
Patrik

Yes I do. I uninstalled and reinstalled using karafe as it armed to throw an error the first time. Certainly appears as active in karafe bundle list.

I still think it doesn’t like that syntax for accessing the attribute. Not all implementations of xpath handle attributes the same.

Hi mrichmond007,

did you manage to find a solution to this in the end? I have the same issue with my Owl intuition PV. I can read the udp packet and the string but when I try to extract an element it just returns the whole string whatever I do. works when I cut and paste into an online formatter so I think the syntax is correct.

interestingly, did remove all xpath code references and just left the udp:xxx.xxx.xxx.xxx address and still get the full string returned. out of ideas!

I didn’t get a solution within openHAB in the end. I did get something working with a python script that posted into the influx DB directly. I was going to have another go at this after my updates and a re-install. I am not filled with confidence now based on your experiences.

I did exactly the same as you and had the entire string return no matter what I did.

Will let you know if I get any further.

M.

Glad its not just me then, even though I must add am coming at this from a very low level of programming ability, you could say non existent. As I said, like you nothing I do changes the output. One thing I did notice is that when I intercept the udp packet using wireshark the xml tags in the string look incomplete. I have split it down and I get


 <solar id='4xxxxxxxxxx3'>
<timestamp>1506550723</timestamp>
 
 <current>
 <generating units='w'>0.00</generating>
 <exporting units='w'>0.00</exporting>
 </current>
 
 <day>
 <generated units='wh'>6364.61</generated>
 <exported units='wh'>2340.79</exported>
 </day>
 
 </solar>

I am probably wrong but to my untrained eye (from the examples I have read) it looks like the generating, exporting, generated and exported elements which I am trying to extract are all missing closing > where you would expect to see them. My hunch is that this means the xml is technically not formatted correctly and therefore invalid and this is preventing the xml string being read and transformed correctly by the XPATH commands perhaps.

1 Like

I’m finishing my binding for my weather station. Then I’ll work on this binding.

It should not be too difficult.

1 Like

Thanks Gabriel,

Getting nowhere with this sadly :frowning:

1 Like

Not sure if this is any help or not but I have taken a slightly different route. I have written a python script that receives the multicast messages. Using xmltodict it parses them and then I send them to openhab using mqtt. This is probably the long way round.

I have it working for current power, current solar, day power usage, day solar generation, current heating stats and water stats.

The next thing I need to figure out it how to start setting th heating controls.

1 Like

Hi, I put this down for a while, mainly as I was getting nowhere, but now the nights are getting darker I have got back on it, Still looking for a way of establishing the exporting reading from my Owl so I can hopefully create a rule to identify when I am exporting over a certain amount so it will then turn an electric heater so I put the free electricity to good use. Would be really Interested to understand how you have managed to use MQTT to generate values or if anyone else has managed to get any further with this?

1 Like

Hi, I bought the Sonoff Pow 2 and setup the tasmota firmware. (~12$).
It was easier than trying to adapt my owl device.