[SOLVED] MQTT 2.4 error when migrating from MQTT1

rswennen · January 20, 2019, 11:11am

I’m trying to migrate to the new MQTT 2.4 binding but struggle.

My .things file is as follows:

Bridge mqtt:broker:flukso2 [ host="10.0.1.120", secure=false ]
    {
      Thing topic fluksosensors {
       Channels:
         Type number : newfluksowatt "Watt"   [ stateTopic="/sensor/8e345fa8179478ce5716514b46373af3/gauge:state:REGEX(.*,(.*),.*)"]
        }
    }

The item is configured like this

Number Watt "Verbruik [%.0f W]" <energy> (gPower) {mqtt="mqtt:topic:broker:fluksosensors:newfluksowatt"}

My old MQTT1 config used to be

Number Watt "Verbruik [%.0f W]" <energy> (gPower) {mqtt="<[flukso:/sensor/8e345fa8179478ce5716514b46373af3/gauge:state:REGEX(.*,(.*),.*)]"}

I assume I’m doing somethig wrong with the regex but can’t find any examples on how to do that with the new binding.

This is the error I get in the logs

2019-01-20 11:03:07.178 [ERROR] [el.item.internal.GenericItemProvider] - Binding configuration of type 'mqtt' of item 'Watt' could not be parsed correctly.

org.eclipse.smarthome.model.item.BindingConfigParseException: Invalid mqtt binding configuration 'mqtt:topic:broker:fluksosensors:newfluksowatt' for item Watt

	at org.openhab.core.binding.internal.BindingConfigReaderDelegate.processBindingConfiguration(BindingConfigReaderDelegate.java:51) ~[?:?]

	at org.eclipse.smarthome.model.item.internal.GenericItemProvider.internalDispatchBindings(GenericItemProvider.java:397) ~[?:?]

	at org.eclipse.smarthome.model.item.internal.GenericItemProvider.internalDispatchBindings(GenericItemProvider.java:366) ~[?:?]

	at org.eclipse.smarthome.model.item.internal.GenericItemProvider.processBindingConfigsFromModel(GenericItemProvider.java:229) ~[?:?]

	at org.eclipse.smarthome.model.item.internal.GenericItemProvider.modelChanged(GenericItemProvider.java:432) ~[?:?]

	at org.eclipse.smarthome.model.core.internal.ModelRepositoryImpl.notifyListeners(ModelRepositoryImpl.java:301) ~[?:?]

	at org.eclipse.smarthome.model.core.internal.ModelRepositoryImpl.addOrRefreshModel(ModelRepositoryImpl.java:139) ~[?:?]

	at org.eclipse.smarthome.model.core.internal.folder.FolderObserver.checkFile(FolderObserver.java:227) ~[?:?]

	at org.eclipse.smarthome.model.core.internal.folder.FolderObserver.processWatchEvent(FolderObserver.java:291) ~[?:?]

	at org.eclipse.smarthome.core.service.WatchQueueReader.lambda$3(WatchQueueReader.java:323) ~[?:?]

	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]

	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]

	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:?]

	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:?]

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]

	at java.lang.Thread.run(Thread.java:748) [?:?]

Caused by: org.openhab.model.item.binding.BindingConfigParseException: Invalid mqtt binding configuration 'mqtt:topic:broker:fluksosensors:newfluksowatt' for item Watt

	at org.openhab.binding.mqtt.internal.MqttItemConfig.<init>(MqttItemConfig.java:67) ~[?:?]

	at org.openhab.binding.mqtt.internal.MqttGenericBindingProvider.processBindingConfiguration(MqttGenericBindingProvider.java:58) ~[?:?]

	at org.openhab.core.binding.internal.BindingConfigReaderDelegate.processBindingConfiguration(BindingConfigReaderDelegate.java:49) ~[?:?]

	... 16 more



"

vzorglub · January 20, 2019, 11:23am

The transformation needs to be specified in another parameter
I recommend the you use the paperUI to define your things, simpler and clearer

Bridge mqtt:broker:flukso2 [ host="10.0.1.120", secure=false ]
    {
      Thing topic fluksosensors "fluksosensors" {
       Channels:
         Type number : newfluksowatt "Watt"   [ stateTopic="/sensor/8e345fa8179478ce5716514b46373af3/gauge", transformationPattern="REGEX:(.*,(.*),.*)" ]
        }
    }

opus · January 20, 2019, 11:44am

Additionally check PaperUI for the correct channel syntax in your item definition. Your broker thing is named “flukso2”, however you are using the name" broker" in the channel statement.

rswennen · January 20, 2019, 12:38pm

@opus thanks

is it possible having the MQTT1 and 2 next to each other conflict.
when configuring the link in the Paper UI it work, the log mentions

2019-01-20 12:33:40.968 [.ItemChannelLinkAddedEvent] - Link 'Watt3-mqtt:topic:flukso2:fluksosensors:newfluksowatt' has been added.

when I put the following in the item file it doesn’t work

String Watt3 "Watt3 string" {mqtt="mqtt:topic:flukso2:fluksosensors:newfluksowatt"}

while I use exactly the same syntax. Strangely I also get an error at startup right after MQTT1 initialisation that seems to indicate the MQTT1 binding tries to interprete my MQTT2.4 item definition

2019-01-20 12:30:10.751 [INFO ] [penhab.io.transport.mqtt.MqttService] - MQTT Service initialization completed.

2019-01-20 12:30:10.754 [INFO ] [t.mqtt.internal.MqttBrokerConnection] - Starting MQTT broker connection 'synology'

2019-01-20 12:30:11.025 [INFO ] [t.mqtt.internal.MqttBrokerConnection] - Starting MQTT broker connection 'flukso'

2019-01-20 12:30:11.089 [ERROR] [el.item.internal.GenericItemProvider] - Binding configuration of type 'mqtt' of item 'Watt3' could not be parsed correctly.

org.eclipse.smarthome.model.item.BindingConfigParseException: Invalid mqtt binding configuration 'mqtt:topic:flukso2:fluksosensors:newfluksowatt' for item Watt3

	at org.openhab.core.binding.internal.BindingConfigReaderDelegate.processBindingConfiguration(BindingConfigReaderDelegate.java:51) ~[?:?]

	at org.eclipse.smarthome.model.item.internal.GenericItemProvider.internalDispatchBindings(GenericItemProvider.java:397) [129:org.eclipse.smarthome.model.item:0.10.0.oh240]

vzorglub · January 20, 2019, 12:40pm

That thing string looks wrong.
Check it’s online in the paperUI and copy the channel name from there

rswennen · January 20, 2019, 12:44pm

The thing is correct IMO but I think I have an error in the item definition, it needs to have channel in it instead of mqtt=

pfff not easy this new binding. I know UI is easier, I’m just worried about disaster recovery. with file based config I can easily create backups and get back up and running in case of disaster

vzorglub · January 20, 2019, 1:01pm

Then back up your OH config. The whole thing.
I do it regularly:

**************************************
**              BACKUP              **
**************************************

Running this script will store all of the configurables made in paperUI or through textual configuration. It will attempt to backup the list of installed addons and bindings, and the settings for those bindings too.

cd $OPENHAB_HOME or cd /usr/share/openhab2
sudo ./runtime/bin/backup                            ## Creates a backup file in the default location with a timestamp.
sudo ./runtime/bin/backup /opt/backups/myconfig.zip  ## Creates a backup file in the specified location.
sudo ./runtime/bin/backup --help                     ## Brings up the help information

USE:
sudo ./runtime/bin/backup /etc/openhab2/backup.zip

The default path to store the zip files in is in $OPENHAB_HOME/backups/ (/var/lib/openhab2/backups for apt/yum users). You can change this default path by specifying the $OPENHAB_BACKUPS environment variable (again use /etc/default/openhab2 if you’re using apt/yum)

**************************************
**              RESTORE             **
**************************************

The restore script restores a zip file created using process above and uses it to completely replace your openHAB configuration. Any newer changes that weren’t in the backup get removed.

*Note: This will not backup the runtime folders. If you’re trying to repair openHAB, it’s best to download a new instance, and then try the restore script to load your configuration.

cd $OPENHAB_HOME
./runtime/bin/restore /opt/backups/myconfig.zip  ## Restores the openHAB config from the specified zipfile
./runtime/bin/restore --help                     ## Brings up the help information

opus · January 20, 2019, 4:12pm

As you already found out, I did miss that you did not state “channel=…” in the definition option. Stupid me!
And no, mqtt1 and mqtt2 do not conflict!
Additionally there is nothing special on the items and things setup for mqtt2, ALL is just the openHAB2 syntax, which you are not used to!

rswennen · January 20, 2019, 4:25pm

@opus I continue struggling with the regex part. Even in the Paper UI I don’t get that working. Any suggestion on how to add the regex part ?
The full input is [1548001548,2902,“W”]. I need the middle bit with the old binding I did this by saying

Number Watt "Verbruik [%.0f W]" <energy> (gPower) {mqtt="<[flukso:/sensor/8e345fa8179478ce5716514b46373af3/gauge:state:REGEX(.*,(.*),.*)]"}

vzorglub · January 20, 2019, 4:28pm

Click SHOW MORE
and you should we the transformation pattern entry field
In it, put REGEX:yourtransformationstringwithoutquotes

rswennen · January 20, 2019, 4:32pm

@vzorglub like this ? It make the channel go offline

vzorglub · January 20, 2019, 4:38pm

Yes exactly like that.

Odd. Try adding a set of bracket: REGEX:((.*,(.*),*))

rswennen · January 20, 2019, 4:44pm

No doesn’t change, also tried quotes but nothing, drives me crazy, without the transformation and as a string thing I got it all working. Now it’s just the regex and making it a number

vzorglub · January 20, 2019, 4:52pm

Is the REGEX transform installed?

rswennen · January 20, 2019, 4:54pm

yes it is and the transformation works flawless with the MQTT v1 binding.

vzorglub · January 20, 2019, 4:56pm

I have not used the REGEX transformation on MQTT payload
@David_Graeff, any characters that need escaping here? Or anything special for the REGEX transformation?

David_Graeff · January 20, 2019, 4:59pm

Other way round, please remove those additional pairs of brackets. Should be .*,(.*),* only. The first argument matcher is used, as far as I know, so the middle part in this case.

rswennen · January 20, 2019, 9:17pm

David, Thanks. I will try this and report back (but won’t be today anymore). Once I have it working I’ll share and maybe it’s a nice example to add in the documentation, I’m sure there are other people trying to convert MQTT items with transformations.

David_Graeff · January 20, 2019, 10:10pm

This is already explained in the documentation, including a Regex transformation example xD

rswennen · January 21, 2019, 6:10pm

David,

thanks. That solved it. There is mention of transformations including REGEX, I agree but having had an example in the .things file of a channel with transformations would have helped me personally.

In my case it’s probably a combination of me not being familiar enough with regex to immediately see the what it does, trying to migrate from the old to new (and therefore trying to copy paste the regex bit) and in parrallel trying to figure out the details of the things, channels, profiles etc.

Combine this with wanting to do it the file way (at least for now) that made it slightly more challenging.

Anyhow, thanks a lot for the help and for those trying to migrate MQTTv1 bindings to the new MQTT with REGEX transformations, here is my final things file, (or for those having a FLUKSO sensor (https://www.flukso.net/) and want to get that one set up). The main tip for those migrating is the old syntax required the REGEX to be between brackets while the new thing syntax doesn’t.

Bridge mqtt:broker:flukso [ host="10.0.1.120", secure=false ]
    {
      Thing topic fluksosensors {
       Channels:
         Type number : watt "Watt" [ stateTopic="/sensor/8e345fa8179478ce5716514b46373af3/gauge", transformationPattern="REGEX:.*,(.*),.*"]
        }
    }