[SOLVED] Why doesn't my REGEX work the way it does on the test sites

Add-ons Other
1

I am using the REGEX addon and trying to take some mqtt input and clean it up however, I am able to get what I want on the test site Regexr test site but when I put it in my item file it gives now out output. I understand that the REGEX addon adds ^$ to beginning and end of the REGEX (not sure why) but I have accounted for this in my REGEX test. Here’s the string:

:{"actual": 12.92, "target": 0.0}

and here’s the regex that I used

^.*(actual)(": )(\d\d.\d\d)(, ")(target)(": )(\d.\d)\}$

I substituted for $1 $3 $5 $7 and get the output of

actual 12.92 target 0.0

This REGEX is not exactly what I want but if I can make it work with just this then I can build from there.
When I put that in my item:

String Carlos "Carlos [:%s]" {mqtt="<[mosquitto:carlos/temperature/bed:state:REGEX (s/.*(actual)(": )(\d\d.\d\d)(, ")(target)(": )(\d.\d)\}/$1 $3 $5 $7/g)"}

I get no output and in the log it just says it cannot retrieve item.
I have tried many iterations the best I could was get a random part of the text to show up. I am sure I am missing something with the way the REGEX works inside openhab 2.4 I might be missing some escapes or other things. I have looked at the documentation but it seems to explain that if it works in one of the testers it should work inside openhab, clearly that is not the case. - Any guidance is greatly appreciated.

2

I don’t know if this is the cause of the problem but I’ll offer it anyway.

Is the actual temperature always two digits and the target temperature one digit? Because if the number of digits can vary, your regex expression won’t capture them.

This regex expression will handle numbers with one or more digits and negative numbers (1.2, 12.45, -3.56, 32.3, etc).

^:\{"(actual)": ((|-)[0-9]{1,}.[0-9]{1,}), "(target)": ((|-)[0-9]{1,}.[0-9]{1,})\}$

If you know the numbers will never be negative then here is a simplified version:

^:\{"(actual)": ([0-9]{1,}.[0-9]{1,}), "(target)": ([0-9]{1,}.[0-9]{1,})\}$

I like using this site to test regex expressions: https://regex101.com/

3

Thanks for the reply, in fact, I made sure the result was always that to test the REGEX so it would work however, as I said this was not my final REGEX solution but I can’t get any of it work successfully. I do appreciate your solution and will implement something similar but first I have to get the thing to work at all. - I will try again and let you know the outcome.

4

OK, so this is not the production version of the regex expression but just enough to get this special test case to work.

I don’t have any practical experience with using OpenHAB’s REGEX transformation but, based on looking at other people’s examples (and the documentation), I noticed one difference. You have a space after the word REGEX. All examples I’ve seen do not use a space but maybe it’s irrelevant.

state:REGEX (s/.*(actual)(

Does the log show any warnings or error messages?

5

I did try it with and without the space, same outcome. - The log doesn’t have anything significant just that it cannot retrieve the item.

6

Oh! I just noticed this example in the documentation. It says the backslash has to be escaped. Notice “\d” is entered as “\\d” (note the double backslash).

String  Temperature_str "Temperature [REGEX(.*=(\\d*.\\d*).*):%s °C]" {...}

If your regex is trying to match the double-quotation mark (") then it appears you’ll also need to escape its meaning as well.

1 Like
7

That’s what I think is going on. The escapes and the addition by the system of ^ and $ to the request make it complicated to know what will work best. (As if REGEX is not complicated enough LOL)

1 Like
8

It’s the old ‘string within a string’ situation and, I agree, it definitely makes reading the regex expression all the more challenging!

I’ve used another system for years and the same situation occurs when appending strings containing single and double quotes. You have to escape their meaning otherwise the string concatenation fails miserably. However, it’s an extra order of challenge when applied to a regex expression!

For documentation purposes, you may wish to include the original regex expression (before all its special characters are escaped for use with the REGEX Transformation) as a comment in the Items file.

// regex ".*=(\d*.\d*).*"
String  Temperature_str "Temperature [REGEX(.*=(\\d*.\\d*).*):%s °C]" {...}

It’s a convenience in case, in the future, you want to decipher what the regex expression is doing! :slight_smile:

9

Good idea on the comments, I will solve this and update the post with the answer. Until then, it’s a bit trial and error.

10

Here is a Regex expression I have working on OH 2.3: String Leaf_Oppdatert “Tid oppdatert [%s]” (gLeaf) {mqtt="<[mosquitto:leaf/status/last_updated:state:REGEX(s/(.{4})(.{1})(.{2})(.{1})(.{2})(.{1})/$1-$3-$5T/g)]"}

11

Thanks, I will try to extrapolate to my desired REGEX. I noticed no space after REGEX :wink:

12

I went to the source to check the format of the mqtt payload and found that this was a good one:

{"_timestamp": 1517190629, "actual": 42.1, "target": 65.0}

So I went to https://regex101.com/ and tested it and sure enough it works. I refactored for ‘’ escape characters and

I used this

String Carlos "Carlos [%s]" {mqtt="<[mosquitto:carlos/temperature/bed:state:REGEX(s/(.{28})(actual)(.{3})(\\d*.\\d*)(.{3})(target)(.{3})(\\d*.\\d*)(.{1})/$2-$4 $6-$8/g)]"}

And now it’s actually printing the REGEX(s/…blah blah)…wait, how can it do that? I checked and made sure the REGEX transformation is installed. I am a bit baffled.

UPDATE without changing anything I did a refresh and it worked!!! - Now I have two 3D printers so I set up the other one with the same REGEX and blow me down with a feather, it worked straight away. - Thank you guys for your help and ideas. I think I am off to the races now. - I will go through the other mqtt payloads and set up REGEX transformations accordingly.

13

I’m glad to hear it worked!

I have a question about your regex expression. However, before I get to it, I noticed your data is in JSON format. Perhaps you should consider using JSONPATH Transformation instead of REGEX. JSONPATH is far more legible than regex expressions. For example, $.target would get the target value. Easy as that!

My question is about your use of 9 capture groups in your regular expression. Why?

Here’s your regex expression:

(.{28})(actual)(.{3})(\\d*.\\d*)(.{3})(target)(.{3})(\\d*.\\d*)(.{1})

It contains 9 capture groups but you only use 4 of them (second, fourth, sixth, and eighth; $2, $4, $6, $8).

The following expression means match 28 characters whatever they may be

.{28}

When you surround it in parentheses, it becomes a capture group.

(.{28})

It becomes the first capture group but you never use $1 so why make it a capture group?

Here’s another observation. You’re using this expression in a capture group to get a floating point value.

(\d*.\d*)

However the period (.) in that expression means to match any character not just a literal period (decimal point in a floating point value). The expression will match 43.2, 432, and even 43Z3. If that’s what you want then OK. If it’s not, then you have to escape the period in order to make the expression match a literal period.

(\d*\.\d*)

Or for OpenHAB’s REGEX, it would look like this:

 (\\d*\\.\\d*)

In addition, the asterisk means zero or more. So \d* means zero or more digits. If you want it to match one or more digits then use a plus sign instead of an asterisk:

(\d+\.\d+)

If the labels, actual and target, never change then you only need to get the floating point values with just 2 capture groups.

.+actual.+ (\d+.\d+).+target.+ (\d+\.\d+)}

For OpenHAB’s REGEX, it will look like this. You don’t need the global “g” option because the expression’s first match will contain both values.

s/.+actual.+ (\\d+\\.\\d+).+target.+ (\\d+\\.\\d+)}/actual-$1 target-$2/
1 Like
14

Thanks, I created more groups than I needed for debugging purposes, I wanted to be able to isolate the text parts. I like your solution and the very clear explanation. I am going to clean it up, I have 4 more that I need to work with so I will do them all as a final effort. As long as I can get it to work consistently I will be very happy. I have never used JSONPATH so that will take a little bit of investigation but it sounds like it could be easier than the REGEX going forward. I will take a look but for now, I have a very readable result.
Now if I could just get my zwave thermostat to accept the setpoint :wink:

15

I may be wrong but I think this is how easy it would be to use JSONPATH.

String Actual_Temp "Actual [%s]" {mqtt="<[mosquitto:carlos/temperature/bed:state:JSONPATH($.actual)]"}
String Target_Temp "Target [%s]" {mqtt="<[mosquitto:carlos/temperature/bed:state:JSONPATH($.target)]"}
16

I used your template for one of the other feeds and it really is that simple. - Very nice. I will do the rest of them using the JSONPATH transformation. Thanks for pointing me to this one.

17

That’s awesome stuff.
I always struggle with OH REGEX vs. online tester as well.
Thanks.