MQTT 2.5 M1+ How to implement the equivalent to MQTT1 REGEX filters

@hafniumzinc Yes I’ve it:

openhab> list | grep -i reg
277 │ Active │  75 │ 2.5.7                   │ openHAB Add-ons :: Bundles :: Transformation Service :: RegEx

And I use it in anothers places, like this:

{ udp="<[192.168.1.65:*:'REGEX(.*<chan id=\'0\'>.*<day units=\'wh\'>(.*)</day></chan><chan id='1'>.*)']" }

But you have to spell it right. You asked for REGEXP, it told you you haven’t got one.

1 Like

The most difficult mistake is always the most obvious :wink:

I’m am still having trouble with this…

Functionally, it all seems to work, but logs are spammed out :confused:

REGEX:(.*D3F9EE.*)∩JSONPATH:$.RfReceived.Data

as in;

image

But still get log errors for channels getting;

tele/rf_bridge/RESULT

when just one message is sent;

2020-09-29 21:10:45.008 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*4065D4.*) not found!
2020-09-29 21:10:45.009 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OnOffValue': No enum constant org.eclipse.smarthome.core.library.types.OnOffType.4065D8
2020-09-29 21:10:45.010 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*4065D1.*) not found!
2020-09-29 21:10:45.010 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OnOffValue': No enum constant org.eclipse.smarthome.core.library.types.OnOffType.4065D8
2020-09-29 21:10:45.011 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*D3F9EE.*) not found!
2020-09-29 21:10:45.011 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OnOffValue': No enum constant org.eclipse.smarthome.core.library.types.OnOffType.4065D8
2020-09-29 21:10:45.012 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*E4CF0[AE].*) not found!
2020-09-29 21:10:45.012 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OpenCloseValue': No enum constant org.eclipse.smarthome.core.library.types.OpenClosedType.4065D8
2020-09-29 21:10:45.013 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*D3F9E6.*) not found!
2020-09-29 21:10:45.013 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OnOffValue': No enum constant org.eclipse.smarthome.core.library.types.OnOffType.4065D8
2020-09-29 21:10:45.014 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*E46E0[AE].*) not found!
2020-09-29 21:10:45.015 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OpenCloseValue': No enum constant org.eclipse.smarthome.core.library.types.OpenClosedType.4065D8
2020-09-29 21:10:45.015 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service REGEX for pattern (.*E3BA0[AE].*) not found!
2020-09-29 21:10:45.015 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '4065D8' not supported by type 'OpenCloseValue': No enum constant org.eclipse.smarthome.core.library.types.OpenClosedType.4065D8
Transformation service REGEX for pattern (.*4065D4.*) not found!

You need to install the REGEX transformation service! Add-ons -> Transformation

1 Like

Thank you, that sorted it. :slight_smile:

Just a quick note that this works in the UI with openHAB 3.1.0 Release version.
I was seeing warnings in the log for an item linked to the Brightness channel of an MQTT Dimmer Thing (Tasmota firmware on the dimmer). The state topic only returned the dimmer brightness when it was changed, so I was getting warnings when the power was turned on and off. I added the REGEX transformation and then chained the REGEX to the JSONPATH in the Incoming Value Transformation field (you have to select advanced options in the UI to see that field) in the channel configuration for the Brightness channel…

REGEX:(.*Dimmer.*)∩JSONPATH:$.Dimmer

Thanks for the tip!

Sorry to bring that old discussion up again, but I am facing a problem here (most probably it is my lack of knowledge).

(Running OH 3.3.0 as a VM with Openhabian / Debian on Proxmox)

I am using a MQTT Smartmeter Device that reads the power consumption from the grid. Unfortunately, the meter does provide only a single value per packet, still they are all using the same format:

{"Time":"2022-08-29T18:44:01","Z1":{"76_7_0":-4.04}}

The value after Z1 changes, so 76_7_0 could be 1_8_1 or 2_8_0 f.e., hence I needed to implement the REGEX function.

Now, when defining the MQTT Thing in a text file, the expression looks like that:

Thing topic Zaehler @ "MQTT" {
    Channels:
        Type number : z1Zaehlerstand       "Zählerstand"                [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*1_8_0.*)∩JSONPATH:$.Z1.1_8_0"] 
        Type number : z1BezugTarif1        "Bezug Tarif 1"              [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*1_8_1.*)∩JSONPATH:$.Z1.1_8_1"]
        Type number : z1BezugTarif2        "Bezug Tarif 2"              [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*1_8_2.*)∩JSONPATH:$.Z1.1_8_2"]
        Type number : z1Einspeisung        "Einspeisung"                [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*2_8_0.*)∩JSONPATH:$.Z1.2_8_0"]        
        Type number : z1Leistung           "Aktuelle Gesamtleistung"    [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*16_7_0.*)∩JSONPATH:$.Z1.16_7_0"]        
        Type number : z1LeistungL1         "Aktuelle Leistung L1"       [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*36_7_0.*)∩JSONPATH:$.Z1.36_7_0"]        
        Type number : z1LeistungL2         "Aktuelle Leistung L2"       [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*56_7_0.*)∩JSONPATH:$.Z1.56_7_0"]        
        Type number : z1LeistungL3         "Aktuelle Leistung L3"       [stateTopic="tele/Zaehler/SENSOR", transformationPattern="REGEX:(.*76_7_0.*)∩JSONPATH:$.Z1.76_7_0"]        
    }

Without the REGEX, the well know error is to be seen in the logs. With the REGEX, ALL outcome is filtered and no packed is passed through at all.

So I defined the things in the UI, using the same parameters:

UID: mqtt:topic:6296256670:Zaehler
label: Generic MQTT Thing
thingTypeUID: mqtt:topic
configuration: {}
bridgeUID: mqtt:broker:jarvis
channels:
  - id: z1Leistung
    channelTypeUID: mqtt:number
    label: Einspeisung / Bezug (Zähler)
    description: ""
    configuration:
      stateTopic: tele/Zaehler/SENSOR
      transformationPattern: REGEX:(.*16_7_0.*)∩JSONPATH:$.Z1.16_7_0
      unit: W
  - id: z1Zaehlerstand
    channelTypeUID: mqtt:number
    label: Zählerstand
    description: ""
    configuration:
      stateTopic: tele/Zaehler/SENSOR
      transformationPattern: REGEX:(.*1_8_0.*)∩JSONPATH:$.Z1.1_8_0
      unit: Wh
  - id: z1Einspeisung
    channelTypeUID: mqtt:number
    label: Einspeisung
    description: ""
    configuration:
      stateTopic: tele/Zaehler/SENSOR
      transformationPattern: REGEX:(.*2_8_0.*)∩JSONPATH:$.Z1.2_8_0
      unit: Wh

Which works like a charm. Now, call me old fashioned, but I would like to stay with the text files. Any idea why it does work only with the UI version ?

Looking at the docs for the binding, the REGEX should work just fine (unless I made a mistake with the syntax, that is…)

Query for the JSON using the REST API for the UI defined Things.
Replace the UI defined things with the text defined Things.
Query for those and compare the JSON.

If there are differences you have your answer and you might have to file an issue but you’ll at least have something to go on. If not, you’ll have to file an issue on the binding anyway, only there will be less to go on.

Hm, what do you mean by “query for…” ? If I take away the REGEX from the text files, the output is the same as with the UI defined things, e.g. the values are read correct. So to me the JSONPATH seems to work just fine in both situations. It is just that with the REGEX Expressions in the text definition, no packed at all seems to pass. But I am not sure if the text-based syntax is fully correct in regards to the REGEX part. Unfortunately, the documentation is quite slim here…

Yayks… ok, got it. Need to understand that stuff first and then bungler my way through… :joy:

I’m giving up…no clue how this works. So I will stick with the UI things then.

Don’t ignore my full reply though. There is likely something wrong and an issue needs to be filed.

I didn’t. It is just, that I have no clue how the API calls work (not a developer), hence I won’t be able to file much detail in an issue report. At least that was what my gut feeling was…

Anyways, filed an issue now: [mqtt] REGEX Filter seems to not work in text based configurations, in UI based ones it does. · Issue #13346 · openhab/openhab-addons · GitHub

2 Likes

btw, doesnt seem to work anymore on OH4. ChannelStateTransformation also logs a warn if regex doesnt match

It still works for me on 4.1.1 release. Show your config.