MQTT 2.4 documentation

mqtt-binding
Tags: #<Tag:0x00007fe052322aa0>

(Garrett Porter) #1

I’m continuing my changeover to MQTT 2.4 and I’ve noticed a few problems with the documentation. I’m getting ready to submit a PR to correct a couple of things but I’d like to be sure I am making the CORRECT changes.

First - I’m planning to add a generic Item config (showing channel structure) but there are conflicting items in the current document.

Switch Kitchen_Light "Kitchen Light" { channel="mqtt:topic:myUnsecureBroker:mything:lamp" }

This item shows

channel="<BINDING>:topic:<BROKER_NAME>:<THING_NAME>:<CHANNE_LNAME>"

BUT lower in the document I see this format:

Switch ExampleItem "Heatpump Power" { channel="mqtt:myUnsecureBroker:topic:mything:heatpumpChannel" }

Which appears to be

channel="<BINDING>:<BROKER_NAME>:topic:<THING_NAME>:<CHANNEL_NAME>"

Is either of these correct / incorrect? I’ve configured mine like the first example - but what is the correct channel structure?

Second - the example of MQTT1 -> MQTT2 appears to be wrong

Bridge mqtt:broker:myUnsecureBroker [ host="192.168.0.42", secure=false ]
{
    Thing mqtt:topic:mything {
    Channels:
        Type switch : heatpumpChannel "Heatpump Power" [ stateTopic="heatpump", commandTopic="heatpump/set" transformationPattern="JSONPATH:$.power" ]
    }
}

The stateTopic in the heatpumpChannel is missing the /state

I’m planning to make a pull request with these things added/corrected. Some input on the channel structure would be great.


(Garrett Porter) #2

Proposed addition under "demo.items"

Generic configuration item:

<ITEM-TYPE> <ITEM-NAME> "<FRIENDLY-NAME>" { channel="mqtt:topic:<BROKER-NAME>:<THING-NAME>:<CHANNEL-NAME>" }

(Garrett Porter) #3

PR created. Link:


(Rich Koshak) #4

Hurray! :fireworks:

It might depend on whether one is incoming and one is outgoing. There is also publish Channels that you can make on the Broker Thing.

I’d recommend creating one of each type of Thing and seeing the format of the Channel IDs for each one. The developer only uses PaperUI so we can assume that anything done through PaperUI will generate valid Channel IDs.

I have no idea about this one. I myself have only created Things through PaperUI thus far and have no plans to move to .things files.

But there is not rule that says a topic must have /state as part of it. But if the MQTT 1.x example has that as part of the topic I would imagine it should be there in the MQTT 2.x Thing definition too.


(Garrett Porter) #5

Regarding the channel structure - it appears to be written two ways in the examples in the documentation - I’ll fiddle with it for a while tomorrow. Maybe both work? If both work I’ll leave my example of the structure but also revert the heat pump to its previous state with the different order. I’m a bit surprised that @David_Graeff doesn’t know the answer to this (see PR discussion) - I would have thought that was part of writing this new binding.

There’s definitely no rule like this. I wouldn’t expect it to be such.

This is the reason I added /state in the 2.x example.

Another note- Rich and David you’ve both mentioned that you are not using .things or .items files. I’m going through this because I’m in the upgrade process to 2.4 and am trying to get everything moved to .things and .items files instead of using Paper UI.

Am I doing it wrong?


(Vincent Regaud) #6

No, not really, but I don’t use things files if I can avoid them too.
Their syntax is not straightforward and they are easy to get wrong.
The PaperUI present you with an interface that allows you do everything the text file can and with the paperUI you can’t get the syntax wrong. On most bindings the paperUI also allows for autodiscovery.

Things, in general only have to be setup once. So although it may seem cumbersome to click and click and click and fill in fields, it only has to be done once. The paperUI will then present you with a fully configured things with channels. And you can copy and paste the channel from paperUI to your items file without this channel syntax confusion.


(Rich Koshak) #7

Just .things. I do use .items files, for now.

Long term, perhaps. For now, probably not. There has been tons of discussion about this in multiple threads.

At a minimum, the format and way that .things, .items, et al are created and used will have to change at some point, probably as part of the move to OH 3.

Like what Vincent says, Things created through PaperUI always have the correct format. They can be automatically discovered meaning I don’t have to create them in the first place. And the JSONDB is text so I can do almost anything that people seem to want to do with .things files (backup, migrate to another OH instance, check into source control, edit by hand).

PaperUI is not feature complete when it comes to Items (tags, profiles, etc) and it is pretty clunky right now so I don’t use it for Items yet. But the new PaperUI Replacement Study means I’ll probably move my Items to UI based when that goes live. I’m already working to figure out the Next Gen Rules Engine which will move my Rules out of .rules files. Slowly, I’ll eventually have everything configured through the UI.


(Scott Karns) #8

Maybe this is helpful. I do use .things and .items files. My experience with the PaperUI has not been consistently good. I have yet to change over to the MQTT 2.4 binding on my production system, but I am testing it on a test system, and it is working for me.

My mqtt.things file:

Bridge  mqtt:broker:mqtt_broker [
        host="127.0.0.1",
        secure=false,
        port=1883,
        qos=0,
        retain=false,
        clientid="Oh2MqttClient",
        keep_alive_time=30000,
        reconnect_time=60000,
        username="myUser",
        password="myPassword"
        ] {
        Thing   topic  wifi_sw03        "Sonoff Switch 3" {
                Channels:
                        Type switch : command "Power Control" [
                                commandTopic="cmd/wifi-sw03/POWER",
                                stateTopic="stat/wifi-sw03/RESULT",
                                transformationPattern="JSONPATH:$.POWER",
                                retained=true,
                                on="ON",
                                off="OFF"
                                ]
                        Type string : updated "Time" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Time"
                                ]
                        Type number : uptime "Switch Uptime" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Uptime"
                                ]
                        Type number : vcc "VCC" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Vcc"
                                ]
                        Type switch : state "Power State" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.POWER",
                                retained=true,
                                on="ON",
                                off="OFF"
                                ]
                        Type number : ap "Wifi AP" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Wifi.AP"
                                ]
                        Type string : ssid "Wifi SSID" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Wifi.SSId"
                                ]
                        Type number : rssi "Wifi RSSI" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Wifi.RSSI"
                                ]
                        Type string : mac "Wifi AP MAC" [
                                stateTopic="tele/wifi-sw03/STATE",
                                transformationPattern="JSONPATH:$.Wifi.APMac"
                                ]
        }
}

My mqtt.items file:

*********************************** Groups ***********************************/
// mqtt.items:
//
// ItemType itemName ["label text"] [<iconname>] [(group1, group2, ...)] [{bindingconfig}]
//
/******************************************************************************/

Group   gSonoffSwitches "Sonoff Switches"
Group   gSonoffStatuses "Sonoff Statuses"

/************************************ Items ***********************************/
//
// Lighting Test Room
//
Switch                  WifiSw03        "WiFi Switch 3"         (gSonoffSwitches)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:command",channel="mqtt:topic:mqtt_broker:wifi_sw03:state" }
String                  WifiSw03_Updt   "Updated [%s]"          (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:updated" }
Number:Time             WifiSw03_Uptm   "Uptime [%d h]"         (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:uptime" }
Number                  WifiSw03_Vcc    "VCC [%.2f V]"          (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:vcc" }
Number                  WifiSw03_AP     "AP Index [%d]"         (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:ap" }
String                  WifiSw03_SSID   "WiFi SSID [%s]"        (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:ssid" }
Number                  WifiSw03_RSSI   "WiFi RSSI [%d%%]"      (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:rssi" }
String                  WifiSw03_MAC    "AP MAC [%s]"           (gSonoffStatuses)       { channel="mqtt:topic:mqtt_broker:wifi_sw03:mac" }

Note that I am running a customized version of the Sonoff-Tasmota firmware on the Sonoff I use for testing – the command prefix is set to cmd rather than cmnd.


(Garrett Porter) #9

Thank you for this!

It supports what I have found looking through other examples and is in line with the changes I’m working on in the documentation. Always good to have another data point.


(David Graeff) #10

Thanks @crxporter for making the documentation consistent.
I don’t know how often the website rebuilds the documentation (could you help out, Yannick? @ysc).

Cheers, David


(Jerome Luckenbach) #11

I can trigger a website build.
But addon repo merges should trigger that anyway within a short period. :slight_smile:


(Yannick Schaus) #12

You can check when the website was last deployed here: https://app.netlify.com/sites/openhab-website/deploys
(you need to login with GitHub, I think any account will do).


(YvesHanoulle) #13

you need to login with GitHub

Nope, I can see when the last deploys were without being logged in…


(Garrett Porter) #14

As I continue digging into this I am becoming more convinced that none of us really knows all of the many ways to configure MQTT…

We had a pull request just this morning as someone was trying a new combination of bridge/broker that unfortunately they did not specify. See here.

I suspect that menaceone had a combo between broker and thing config files that just didn’t work out.

Going through the forums, I see this post which appears to be the source of the heat pump example. There is a combo of bridge {} and separate bridge/thing files in this one.

I’m about to log in to my vpn to see if I can test the method of defining broker and Generic MQTT channels separately… stay tuned for more documentation updates if I figure anything out. Please comment if you have all of the answers (or if you’ve tested any of this)


(Garrett Porter) #15

The following options all work:

Option 1 configure Broker as a bridge and items as things “under” that bridge.

Things file looks like:

Bridge mqtt:broker:WorkBroker "Work Broker" [ host="localhost", port="1883", secure=false, username="openhabian", password="ohmqtt", clientID="WORKOPENHAB24" ]
{
    Thing topic WorkSonoff "Work Sonoff" @ "Home" {
    Channels:
        Type switch : WorkLight "Work Light" [ stateTopic="stat/christmas3/POWER", commandTopic="cmnd/christmas3/POWER" ]
        Type switch : WorkLightTele "Work Tele" [ stateTopic="tele/christmas3/STATE", transformationPattern="JSONPATH:$.POWER" ]
    }
}

Option 2 configure Broker as a “thing” and generic other MQTT stuff as things.

Things file looks like:

Thing mqtt:broker:WorkBroker "Work Broker" [ host="localhost", port="1883", secure=false, username="openhabian", password="ohmqtt", clientID="WORKOPENHAB24" ]

Thing mqtt:topic:WorkBroker:WorkSonoff "Work Sonoff" (mqtt:broker:WorkBroker) @ "Home" {
    Channels:
        Type switch : WorkLight "Work Light" [ stateTopic="stat/christmas3/POWER", commandTopic="cmnd/christmas3/POWER" ]
        Type switch : WorkLightTele "Work Tele" [ stateTopic="tele/christmas3/STATE", transformationPattern="JSONPATH:$.POWER" ]
}

Option 3 same as option 2 - but broker is called “Bridge” instead of “Thing”

Things file looks like:

Bridge mqtt:broker:WorkBroker "Work Broker" [ host="localhost", port="1883", secure=false, username="openhabian", password="ohmqtt", clientID="WORKOPENHAB24" ]

Thing mqtt:topic:WorkBroker:WorkSonoff "Work Sonoff" (mqtt:broker:WorkBroker) @ "Home" {
    Channels:
        Type switch : WorkLight "Work Light" [ stateTopic="stat/christmas3/POWER", commandTopic="cmnd/christmas3/POWER" ]
        Type switch : WorkLightTele "Work Tele" [ stateTopic="tele/christmas3/STATE", transformationPattern="JSONPATH:$.POWER" ]
}

I’m beginning to see where the confusion comes from in the docs - everything depends on those { } after the bridge configuration. Surprisingly - the .items file that works with all of these is the same. It looks like:

Switch SW_WorkLight "Work Light Switch" { channel="mqtt:topic:WorkBroker:WorkSonoff:WorkLight", channel="mqtt:topic:WorkBroker:WorkSonoff:WorkLightTele" }

Interestingly - it doesn’t seem to matter between option 2 and 3 whether I type “bridge”, “thing”, or just nothing at all (start the line with mqtt:broker:...) - the broker connection is still working properly. The documentation for the MQTT 2.4 broker indeed shows it without bridge or thing, just starting mqtt:broker:...

I prefer Option 1 and plan to set up my system as such. I also see the benefit in the other approach (let’s call it non-bridged) if someone has many MQTT things in their house and would like to keep them separated into separate files but under the same broker. I think next I will put together a PR for the documentation showing these configuration options…

Does anyone have an example about what things look like with a functioning broker configured in paper ui with things and items files for defining their various channels?

Edit: new PR here.


[SOLVED] OH2.3 with Knx-Binding: Can using 2 Knx Bridges in two .things-files cause Problems?
Next generation design: A Paper UI replacement proposal
Configuration of openHAB
(Rich Koshak) #16

I think option 2 would be considered non - standard. The broker serves the function of a bridge. A bridge is a thing that serves as a gateway through which related Things communicate.

Not really surprising as the Thing and Channel IDs are not changing in the three different approaches.

I think what you are seeing with the confusion is that none of this. things file stuff is really implemented by the binding of I understand correctly. There is a separate part of OH that parses the file and passes the Things to the binding.

what version are you using with? there is a known bug in 2.4 that requires a restart of OH after changes to the .things file which is fixed in 2.5 m1. Mahe sure you are not seeing this bug which could exclaim the behavior.


(Jürgen Baginski) #17

I going to make the picture a bit more complicated:
I saw that the required channel syntax might be differing. When creating the things via PaperUI the brokername is not needed in the channel definition ( as can be seen on PaperUI).My guess is that Options2 and 3 could use this other channel syntax since the broker to which the items are connected is set in the thing definition. But I can only guess that one ATM.


(David Graeff) #18

This is clearly an indication that the current .thing files are crap. Just saying.

But to be honest I’m that close to just remove all the textual examples from the mqtt binding. That’s more than half of the documentation now and with every PR that section is growing. (Alternatively I add pictured Paper UI explanations on top of that. But I wonder who is reading the resulting 30 pages doc then!)

Why does this poor bindings documentation need to take the fall for everybody who is not reading the official .thing-file syntax documentation. I know that the MQTT binding was the first (to be correct second) binding that has shown the nested bridge { thing { channels } } syntax. But that syntax existed forever.


(Garrett Porter) #19

I was expecting to see a difference here because of the confusion with how the channel structure shows up a bit differently in some places there’s been a question of whether it is mqtt:topic:<broker-name>… or mqtt:<broker-name>:topic... - I thought this might have explained that one.

I am using 2.4 stable as loaded on Monday of this week from fresh openhabian on a pi. That could explain the differences between option 2 and 3 I suppose. Really there seem to be 2 options - one with a separate bridge thing and one with the MQTT things and channels defined as a set { } of broker things.

Can you provide some clear screenshots on this? Maybe I’ll delete my .things files after work today to try it out.

I think your second part of this hits the nail on the head. the MQTT binding is one of the first. It’s also the most popular binding of openhab if I understand correctly. It seems everyone around here has their chosen hardware (knx, zwave, zigbee, insteon…) plus a few MQTT things.

The other big issue I realized is that with other bindings - the thing definition is basically “this is where to find my thing” and channels are all put together by default. The official document on .things files has 3 examples: network, astro, and ntp. Each of these just need to have an IP address or location and magically there is a set of channels to use in your items.

MQTT on the other hand is a whole new beast. We have tasmota, espeasy, espurna, the home assistant standard, the homie standard, and then anyone who just felt like adding the pubsubclient.h to their arduino sketch. In my case I have several tasmota and a few of my own creations. Eventually I’m hoping to write new firmware for all of them to use homie with discovery but step 0 in that was to understand how the MQTT binding works and that’s proving to be quite the project. The thing with MQTT is that the channels aren’t automatic or default. They are highly customizable. We each have our own list of very different channels that we think should go with our things.

I like it. But I also like the .things files. I don’t want to go through and enter each of my items into a web form (paper ui) because I make tons of typos as I go and I end up with 4 sonoff switches that are each configured differently instead of a nice little text file where I can copy-paste a line and rename it to work again… You have heard the arguments, this is nothing new. I do hope your next-gen paper ui catches on and becomes the thing of the future. I’m not excited to redo my whole setup again at that point. Oh wait - yes I am. This is my favorite hobby after all.

Do it. Let people read the forums if they want to struggle through things file setup. I have no objection. I was on a campaign to get the documentation fixed up because that’s always been my starting point to configure a binding. If the binding said “here’s how to do everything in paper ui” with some examples… I would probably just configure it in paper ui!


(Vincent Regaud) #20

Amen to that!! Well said