Regex help needed (Serial binding)

Hi, I get the following string from my meter reader:

/ESY5Q3DA1004 V3.03

1-0:0.0.0*255(0273020613676)
1-0:1.8.0*255(00011138.1052974*kWh)
1-0:21.7.255*255(000176.89*W)
1-0:41.7.255*255(000079.48*W)
1-0:61.7.255*255(000169.45*W)
1-0:1.7.255*255(000425.82*W)
1-0:96.5.5*255(82)
0-0:96.1.255*255(1ESY1206003676)
!

Obviously I need the numbers in brackets and I have a working regex from my perl script:

m/1-0:1\.8\.0\*255\((.*)\*kWh/

I’d like to use the serial binding instead, but I can’t get the regex to work.

Can somebody please help me with a working regex?

I tried

REGEX(1-0:1\\.8\\.0\\*255\\((.*)\\*kWh)

but log says it always returns the full string from serial which is obviously not convertable to a number.

You need to escape the “-” and I don’t think you need to double escape. I tried this in a Regex tester and it seemed to work:

1\-0:1\.8\.0\*255\((.*)\*kWh\)

But my experience with Regex in openHAB is a little limited so I might be wrong about the double escaping.

I tried escaping the “-” without change.
Also the double escape seems to be needed because the complete binding config is a string.

It is really strange that the regex obviously returns the complete string. If there would be an error in the regex, it would be more likely that it returns nothing, isn’t it?

Maybe I should add some logging to the serial binding to test further…

It is a string of characters yes, but it isn’t a String (i.e. a java.lang.String) so you don’t have to trick the interpreter to ignore the escapes by double escaping. If you were defining this String inside of code (e.g. in rules or Java code) you would probably have to double escape. But since this is a file read in by code single escapes is all you need.

The “-” character is a special character in regular expressions, usually when the items are in brackets (e.g. [0-9]). The Regex tester I used was for JavaScript, not the library used by openHAB, so I don’t know for sure if openHAB can handle it unescaped. The JavaScript Regex does required it be escaped. The “:” is a similar special character in some contexts (e.g. [:upper:]). It worked without escaping it for JavaScript, perhaps it needs to be escaped for openHAB though.

You can bump up the logging on the serial binding using the logback.xml. See the example at the bottom of the zwave binding wiki page for an example of how to bump up the logging and shunt the statements to its own file. However, I think the bindings that do use transforms use openHAB’s transformation service as opposed to implementing their own, so you might have better luck switching to the logback-debug.xml file which will bump up the logging in openHAB core.

Finally, you might simplify your regex. Since the line you care about is the only one with “kWh” on it this one would work too.

.*\((.*)\*kWh\)

It lets you skip over dealing with whether you need to escape the “-” or “:” entirely.

Is the data received exactly like you described in the first posting? If so, the Regex transformation service expects a pattern that matches the entire string (multiline, in this case). From the regex transformation service source code…

Matcher matcher = Pattern.compile("^" + regExpression + "$", Pattern.DOTALL).matcher(source.trim());

It appears your pattern is intended to match a specific line in the string. You’d need to add “.*” before and after the pattern for it to match.

Although the pattern languages vary a bit between programming languages, you shouldn’t need to escape a “-” or “:” outside of a character class specification in Java or any regex language I know about. (I tried it in Javascript using the Chrome developer tool console and it worked without escapes.) However, I agree with the suggestion to use the simplest regex that will work with your data.

This worked for me on the data you provided: .*\((.*)\*kWh.*

Given the relatively large string, you might need to watch for fragmentation. Depending on how the data is written, the binding might see part of the string before it has all been written to the serial port by the meter. The serial binding debug logging will show whether this is happening or not.

Thank you for your answers.

Thing with start and end is mentioned in the wiki, but I didn’t understand exactly what it means.

The simplified example from rich & steve does work, but I really have to escape the backslashes, otherwise I get a parsing error (NullPointerException) on instancing the item.

Last thing is that I also need the numbers from lines ending with *W (that is the current usage of the 3 phases and sum of them). The last (sum) would be sufficient.

I’ll try to get it working, hope I get it with the new knowledge. :smile:

Edit: got it with regex:
REGEX(.*1-0:1\\.7\\.255\\*255\\((.*)\\*W.*)

Another question regarding this topic: is it possible to limit the update rate somehow?
Seems as if the meter sends out the data constantly, ending in updates about every second.

On the one hand this is really annoying in the classic UI, if I scroll down, the value is updated, site reloaded and scrolled to top.

On the other hand, I really doesn’t need this data that often and would like to save some resources (cpu time etc.)

I don’t think so, not without implementing something to yourself. I didn’t see anything in the Serial binding wiki that leads me to believe you can adjust it so it will return data at a different rate from what it is being published as. You could implement something in your rule with a proxy Item. Put the proxy Item on your sitemap and then have a rule that triggers on your socket item that updates the value of the Proxy once every 60 seconds or something like that. It won’t save any resources but it will limit the UI problem.

The OP implied a question about why the REGEX service (Is that the right word? Three days new to OpenHAB…) returns the whole string with an erroneous REGEX.

I’d like to expand on that question and ask, Why does OpenHAB update the item with the whole the string if the REGEX fails a match, and is there a way around it? (To the OP’s implied question, as best I can tell, a faulty REGEX which can’t be parsed causes a parse error in the openhab.log file.)

My problematic use case involves the use of a Grid Insight AMRUSB-1 utility meter data receiver and my net meter, which reports as if it had three serial numbers. The strings look like this:

$UMSCM,12346668,8,41369*62
$UMSCM,12346669,8,18905*6F
$UMSCM,12346670,8,22463*63

My matches look like this:

String UtilityMeter68 "Power Meter 68"    { serial="/dev/ttyACM0,REGEX(.*68,[0-9],([0-9]*).*)" }
String UtilityMeter69 "Power Meter 69"    { serial="/dev/ttyACM0,REGEX(.*69,[0-9],([0-9]*).*)" }
String UtilityMeter70 "Power Meter 70"    { serial="/dev/ttyACM0,REGEX(.*70,[0-9],([0-9]*).*)" }

which returns the last numeric group, meter reading (kWh*100), which I want to display for now, then maybe store someday.

Anyway, when one of the strings above arrives from the Grid Insight, all three attempt the match, and the one which matches is modified correctly. The other two, however, are updated incorrectly with the entire string, like so:

2015-12-31 20:50:14 - UtilityMeter70 state updated to $UMSCM,12346668,8,41369*62
2015-12-31 20:50:14 - UtilityMeter69 state updated to $UMSCM,12346668,8,41369*62
2015-12-31 20:50:14 - UtilityMeter68 state updated to 41369

I haven’t done any further processing with the data, but I imagine I’ll have to do something pretty major to update the sitemap with only the correctly-updated value because of what I perceive to be a not-great behavior of the serial REGEX service.

Any clues, anybody, on how to get around this problem?

Thanks,
Bill

Thank you, good idea, I’m doing it this way for now.

It’s really weird why the regex returns the whole string, maybe I should have a look at the source what’s happening there. I’m also not really sure what value were posted to the item if regex didn’t work. Because the string then can not be parsed to a number. So if this value will be ignored, it shouldn’t be a problem (just a missed value). But if it writes error/undefined/null to it, maybe this is a problem.
Will have to look further on this topic and I’m still thinking about “forking” the serial binding to an own binding suiting my purposes…

You could try using the Javascript transformation service instead. You could either use a Javascript regex to extract the data or use a different parsing technique. If there is no match then throw a TransformationException which will cause the item to not be updated. This is a hack and will result in a error message in the log file but doesn’t appear to have any other side effects (based on looking at the SerialDevice code). Again, this is not ideal. Maybe someone else can think of a more elegant solution.

The short answer is that the original implementer had a choice to make (and by original implementer I don’t know if it is the original transform developer or the author of the REGEX library they use) when a REGEX doesn’t match. They could have returned null, empty string, or the whole string. Without any other requirements all three of these are valid and reasonable approaches.Personally I probably would have chosen empty string or null instead but I wasn’t there when it was implemented so I don’t know the reasoning for going with the full string.

The way around it is to write a rule that parses out your values using your own logic instead.

The great power of openHAB is if you run into something that doesn’t work the way you want it to you can usually get around it in a rule or and external script. I believe @sipvoip has posted a rule he uses to do just this so if you search through his postings you should find an example.

OI wonder if you need to make your regex a little more defined. I don’t know whether the REGEX is greedy or not and because the number “69” occurs more than once in your return string I’m not sure how the engine will handle the “*.69”. Could be the cause of your problem, could be a red herring.

Other alternatives are to simply not update an item if the transformation fails or support binding configuration options to specify what happens when a transformation fails.

In this case, it’s the REGEX transformation service that returns the full string if the regular expression doesn’t match. The SerialDevice takes that returned string and updates the associated item state. The SerialDevice will not update the item if a TransformationException is thrown from the transformation service.

Solution, found.

Thanks for all the feedback. Indeed, OpenHAB is open enough to allow workarounds, and this is the one I made:

In .items:

String UtilityMeterReading "Power Meter Reading" { serial="/dev/ttyACM0" }
Number UtilityMeter68 "Power Meter 68" {}
Number UtilityMeter69 "Power Meter 69" {}
Number UtilityMeter70 "Power Meter 70" {}

In .rules:

/**
 * This rule updates UtilityMeter68/69/70 based when UtilityMeterReading changes
 **/
rule "Update Power Meter Readings"
	when
		Item UtilityMeterReading received update
	then

		logDebug("UPMR","UPMR got triggered with state "+UtilityMeterReading.state.toString)

		var String[] reading = UtilityMeterReading.state.toString.split("[,*]")

  		logDebug("UPMR", "Meter ID " + reading.get(1))
  		logDebug("UPMR", "Reading " + reading.get(3))
  		var Number kWh = new Integer(reading.get(3))
  		
  		switch (reading.get(1)) {
  			case "12346670" : postUpdate(UtilityMeter70, kWh)
  			case "12346669" : postUpdate(UtilityMeter69, kWh)
  			case "12346668" : postUpdate(UtilityMeter68, kWh)
  		}
  		
end		

(I will change received update to changed when I run this for reals.)

Pretty clever, if I do say so myself.

Anywho, again, thanks for all your help!

/Bill

3 Likes

Hello,

I fixed this issue (https://github.com/openhab/openhab/issues/3784) and created a PR.

@Maniac

Can you use the serial binding to read out the ir-readers ( i have the ir-readers from udo - volkszähler)? I have a ehz power meter from EHM.

I have a small programm wich parses the sml code and gave me nearly the same output which you got. Bu t i think only with the serial binding it isn´t working?

I have to start this programm:

./sml_server /dev/ttyUSB0

And then i get this on the linux console:

1-0:1.8.0*255#9395.504#
1-0:2.8.0*255#42287.215#
1-0:1.8.1*255#9395.504#
1-0:2.8.1*255#42287.215#
1-0:16.7.0*255#0.506#

How can i send this to the linux machine and how can i read out the answer?

I have a simple IR-Reader (don’t know the model anymore). It is connected to the linux machine, my openHab is running on. So my binding directly opens the serial port, reads out the text (it get’s send by the meter automatically) and parses it.

Don’t know if the binding will be usable by others.

you get plain text out of your power meter without need to parse?

Yes, if I open the serial port with correct settings, it sends plain text (like the one in my first posting).