MQTT Regex Filter

Setup, Configuration and Use Runtime
, ,
1

Hello openHab community.

Seems like my brain it too limited when it comes to apply regex to filter incoming MQTT messages.
I’d really appreciate your help on this!

Here’s my setup and what I’m trying to achieve:

A device (Shelly Pro 4PM) is sending data/status changes (power meter values) via MQTT.
It only sends data that has changed (different to many other shelly devices MQTT),
Example: if a power measure on one of it’s for power sensors changed it only sends this single dataset.

This is an example MQTT message (excerpt) for a changed power measurement on sensor with ID/name “switch:2”:

{
  "src": "shellypro4pm-c8f09e87eec8",
  "dst": "shellypro4pm-c8f09e87eec8/events",
  "method": "NotifyStatus",
  "params": {
    "ts": 1695195688.41,
    "switch:2": {
      "id": 2,
      "apower": 36.1
    }
  }
}

In my openHab MQTT thing file I have a channel representig power-measurement for switch:2 (and additional channels for switch:0, switch:1 and switch:3).

Here’s my question;
how do I apply a filter for my channel switch:2 to only consider MQTT messages that are for switch:2 and represend power-measurements (key-word: “apower”)?
Basically I only want to consider MQTT messages containing the strings switch:2 as well as apower .

I did a lot of research in this and other forums and tried several variants I’ve found without success such as:

Type number: power2   "Verbrauch 2" 	[stateTopic="shellypro4pm-c8f09e87eec8/events/rpc", transformationPattern="REGEX:(.*switch:2.*)∩REGEX:.*apower.*)∩JSONPATH:$.params.switch:2.apower"]

rersult: no value being received on this channel any longer.

Without any regex filter (only JOSANPATH) the value is being correctly received on the channel (but I get all these fail-warnings in the log for MQTT-messages which do not contain the JSON elements specified in the JSON-path of the channel (JSONPATH:$.params.switch:2.apower" which I want to avoid).

Any help is highly appreciated!

Thanks and kind regards,
Ralph…

P.S.: I do not want to use the shelly binding first because I had several problems with it after upgrading to OH 4 especially with Generation 2 / PRO devices (stability) and second because I do have several other devices useing MQTT in my system (consistency).

2

Try remove all the parentheses (you’re missing one anyway in the second regex). I don’t remember exactly how OH regex handles new lines, but if your original input is actually formatted as shown then the line breaks are probably causing your first regex (.*switch:2.*) to just return the single line with switch in it which means your second regex is returning no matches because apower is on a separate line.

You could also try: [\S\s]*switch:2[\S\s]* and see if that bypasses the newline problem.

Either way, your best chance to debug this is just to do it one step at a time. Create a string item attached to this channel and add just the switch regex to make sure you are only getting payloads withe the right switch info. Then chain the apower regex to make sure that is still returning a full and correct json before apply the JSONpath. Once your string item return the correct string, then you can just copy paste the transform over to your number item.

1 Like
3

It’s really messed up unfortunately. It doesn’t match lines at all. I understand why it works the way it does but that fact really trips up those who have even a passing experience with REGEX.

The rules of thumb with OH REGEX are:

  1. the expression must match the entire String, even if it’s multiple lines
  2. Only the first matching Group gets returned. Subsequent Groups are ignored.
  3. Way back when the MQTT 2.x binding was first created, REGEX was modified to deviate from the default behavior to make it work as a filter. Most transformations throw an error and return the full string if there is no match. REGEX returns null.

Given the above, in order to use REGEX as a filter the expression must match the full document passed to it. So REGEX:(.*switch:2.*) is good. If the string contains “switch:2” the full document will be passed on to the next transform. The second one is not, as you’ve identified, it’s missing an open parens.

@syntacrsc I think if you fix the opening paren problem you’ll be good. If not, definitely follow Justin’s advice and break it down.

Given the JSON posted you can achieve the filter in a single REGEX. Assuming the order remains the same (which is not guaranteed in JSON) the following should work:

REGEX:(.*"switch:2".*"apower".*)

If the order changes you’d have to use look aheads and it’s going to get real ugly really fast so keeping them separate is going to be the easiest.

REGEX :(.*"switch:2".*)∩REGEX:(.*"apower".*)
1 Like
4

Hi @JustinG, hi @rlkoshak.
Thanks for your input.

I’ve changed the REGEX, proceeded as Justin suggested, found those two both working well and applied the new knowledge to my MQTT channels:

REGEX:(.*\"switch:2\".*)∩REGEX:(.*\"apower\".*)
REGEX:(.*\"switch:2\".*\"apower\".*)

The channels in my thing file now looks like this and all works well:

Type number: power1   "Verbrauch 1"   [stateTopic="shellypro4pm-c8f09e87eec8/events/rpc", transformationPattern="REGEX:(.*\"switch:0\".*\"apower\".*)∩JSONPATH:$.params.switch:0.apower"] 			
Type number: power2   "Verbrauch 2"   [stateTopic="shellypro4pm-c8f09e87eec8/events/rpc", transformationPattern="REGEX:(.*\"switch:1\".*\"apower\".*)∩JSONPATH:$.params.switch:1.apower"]
Type number: power3   "Verbrauch 3"   [stateTopic="shellypro4pm-c8f09e87eec8/events/rpc", transformationPattern="REGEX:(.*\"switch:2\".*\"apower\".*)∩JSONPATH:$.params.switch:2.apower"]
Type number: power4   "Verbrauch 4"   [stateTopic="shellypro4pm-c8f09e87eec8/events/rpc", transformationPattern="REGEX:(.*\"switch:3\".*\"apower\".*)∩JSONPATH:$.params.switch:3.apower"]

Thanks a lot for your help (again)!
Ralph…