MQTT structural guidelines

[leif (Leif Claesson)]

• Platform information:
  • Hardware: Raspberry Pi 3 B+, 120gb mSATA SSD in a USB enclosure
  • OS: Raspbian Stretch
  • Java Runtime Environment: 1.8.0_212
  • openHAB version: 2.4.0 release build. Jython 2.7.0

Howdy!

Experience level is as follows:
C++ programmer: 20+ years
Arduino/ESP8266: 2 months
openHAB: 3 weeks
JSR223/Jython: 2 weeks
MQTT: 1 day

So. I have a whole bunch of wall-mounted 433mhz remotes, like this one:

I’ve written some code to make the most of them. For example, my code can reliably count pulses to multiply functionality, for example push the same button 3 times for action number 3.

My weekend project this time was to bring my kitchen ceiling fans into the home automation system so that I can use these wall-mounted buttons rather than the fiddly infrared remote that came with the fans.

After first unsuccessfully trying to follow a hopelessly confusing openHAB + ESP8266 + MQTT tutorial, I decided to break up the problem into smaller parts, and learn to understand MQTT outside of openHAB, first using MQTT.fx and then MQTT Explorer to monitor messages.

With my limited understanding of topics and payloads, I set it up with one topic per button, like so:

Pressing the left button yields: topic [payload]

home/mcu/LivingRoomMCU/433mhz/KitchenFan1 [0]

Middle button:
home/mcu/LivingRoomMCU/433mhz/KitchenFan2 [0]

Holding the middle button in for more than half a second yields:

home/mcu/LivingRoomMCU/433mhz/KitchenFan2 [1]

It seemed like a good idea.

Then, once I tried to bring openHAB into the picture, it seemed too “piecemeal” to have to create a separate channel AND a separate item for each.freaking.button (since each button had its own topic), so I looked around for other solutions and found:

Number temperature "temp [%.1f]" {mqtt="<[publicweatherservice:london-city/temperature:state:default]"}

Perfect, I thought! Defining them in an file is okay, I can live with one line per button… Or better yet I can use wildcards. And then I spent a couple of hours trying to get it to do anything at all (no error messages in the console either) until I understood that this was the Openhab 1.x way of doing things, and will not work in Openhab 2.x.

You can of course subscribe to more than one topic in an OpenHAB 2.x MQTT channel, by using wildcards… but then you won’t know what topic caused the channel to be triggered.
I found this workaround which will probably work… but now it’s starting to feel really clunky.

I mean, it is my understanding that MQTT is supposed to be flexible and lightweight.

If I have to subscribe to every single topic and filter in a script, that’s not going to be very efficient.

I could split my topic and payload diffently… for example:

home/mcu/LivingRoomMCU/433mhz/KitchenFan [Button1_Pressed]
home/mcu/LivingRoomMCU/433mhz/KitchenFan [Button2_Held]

But, it seems like a rather arbitrary limitation to not get the topic as part of the event in a normal channel subscription.

It’s probably just me trying to use it in a weird way.

So, would you please enlighten me, how would you do it? What is the recommended way?
How would you structure a remote with multiple buttons where each button has multiple functions into MQTT topics + payloads, for it to work well with openHAB?

How do the people who designed the MQTT binding for openHAB 2.x use it, to have made the design decisions they made?

Would appreciate any insight/help. Thanks in advance.

[JGKK (Johannes)]

Can you make it so that all buttons publish to one topic? So you differentiate all events by their payload? So eg 1 press left = payload 0, two presses left is payload 1 and so on? If so you could create a generic mqtt thing with a text value channel subscribed to this one topic. Than you link this channel to a String item which has the expire binding attached to it set to expire to OFF after one second.
This way you can base your rules on a item state changed to event directly for triggering openhab stuff when pressing a button.
Make sure your buttons publish non retained messages for this to work.
This is all theory.
I use an approach where I don’t actually use the expire binding but do it on the esp8266 side. When you press a button it publishes short pressed / long pressed and than publishes OFF after a few ms. This way I don’t need the expire binding but otherwise it’s the same.
Maybe this helps, Johannes

[leif (Leif Claesson)]

Thanks, Johannes. I’ll do exactly that then.
Indeed I don’t use the expire binding for this either, I do all the work in the ESP8266 as well.

Here’s some example console output from the ESP generating the MQTT messages.
I have a few different button modes in the ESP8266 code, buttons are defined with one mode per button.

“home/mcu/LivingRoomMCU/KitchenFan” is the topic, and “KitchenFan1*push” is the text payload. There is no “released” message. “push” arrives as soon as you push it, and then also “long” if held for more than half a second. This is the remote that has three buttons.

MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan1*push (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan1*long (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan2*push (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan2*long (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan3*push (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan3*long (volatile)

Another mode is tally, which counts the number of pushes and yields one message of the tally (total count), or 0 if long-pressed. The “count” messages (while counting) are useful to for example play a “ding” in a nearby speaker as you’re pushing.
This is a remote that only has one button, pushed four times.

MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan*count1 (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan*count2 (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan*count3 (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan*count4 (volatile)
MQTT publish [home/mcu/LivingRoomMCU/KitchenFan]: KitchenFan*tally4 (volatile)

And “volatile” is of course the opposite of retain.

So yeah, this will work great.

Still seems like kind of a limitation in openHAB though that it’s difficult handling multiple topics… but it’s probably just me.

I wonder if I should release my button handling code as an arduino library, could be useful for someone.

[JGKK (Johannes)]

It’s a limitation of the mqtt binding as it doesn’t offer a button type channel were you could use the channel triggered event directly in rules. I think this is do to mqtt buttons being a pretty niche requirement.
When talking a bout hierarchy I think unleast you are trying to implement something like the homie standard it’s pretty much up to you and what makes sense to you.
Even every commercial company has its own idea of what’s the best topic /device structure in their mqtt APIs.
Johannes

[rlkoshak (Rich Koshak)]

@leif

Then, once I tried to bring openHAB into the picture, it seemed too “piecemeal” to have to create a separate channel AND a separate item for each.freaking.button

This is not an unusual approach. A Thing represents a device. Each sensor or actuator on the device is represented by it’s own Channel. From OH’s perspective, the button is an individual sensor so each button would normally get it’s own Channel.

However, there is nothing that says you need to link each Channel to it’s own Item. It is perfectly normal and very useful to link multiple Channels to the same Item.

Furthermore, because these buttons generate events instead of states, an even better approach would be to create the Channel on the Broker Thing and then you can trigger Rules directly on the Channel, no need for Items at all. This is the more appropriate approach for instantaneous events (e.g. look at the Astro binding’s sunrise/sunset event Channels). In this case, a wild card subscription on a single Channel and a Rule to parse out the topic and the message would be a pretty nice approach.

@leif

until I understood that this was the Openhab 1.x way of doing things, and will not work in Openhab 2.x.

Not strictly true. There are OH 1.x version binding and there are OH 2.x version bindings. OH 1.x version bindings can still be installed and used on OH 2.x. In the case of MQTT, you can even have both bindings installed at the same time. You may need to enable “Show legacy 1.x version bindings” in PaperUI. Also be aware that you can not use PaperUI for anything else beyond installation of OH 1.x bindings. Everything needs to be done in config files.

@leif

but then you won’t know what topic caused the channel to be triggered .

Not true. If you use the event Channels on the Broker thing above, then the topic that received the message is embedded in the receivedEvent implicit variable. See MQTT - Bindings | openHAB

• publishTrigger : This channel is triggered when a value is published to the configured MQTT topic on this broker connection. The event payload (in receivedEvent ) will be the received MQTT topic and its value, separated by the hash character ( # ).

@leif

I found this workaround which will probably work… but now it’s starting to feel really clunky.

As you probably have guessed, it isn’t a work around. This is the designed behavior of the binding.

@leif

If I have to subscribe to every single topic and filter in a script, that’s not going to be very efficient.

You don’t. You subscribe using a wild card subscription. You need but the one Channel on the Broker Thing. Do not create a separate Thing for this. Then your Rule can very simply separate the topic from the message and process the event as necessary.

@leif

But, it seems like a rather arbitrary limitation to not get the topic as part of the event in a normal channel subscription.

The “normal” approach is that you have a Thing. The Thing represents a device. That device has one or more controls or sensor readings. Each is represented by a Channel. Channels get linked to Items. By the time you get to the Item, the Item shouldn’t know the MQTT topic. All pother things being equal, it shouldn’t even know that it is linked to an MQTT Channel in the first place. So there shouldn’t be any information about the MQTT topic in the Item. One of the main points of Items is to normalize out all the differences between all the different technologies.

It’s not arbitrary. It is a well thought out and intentional design decision.

And, as I discuss above, you can get to exactly what you are after using a publishTrigger Channel on the Broker Thing and a Channel trigger on a Rule.

@leif

So, would you please enlighten me, how would you do it? What is the recommended way?
How would you structure a remote with multiple buttons where each button has multiple functions into MQTT topics + payloads, for it to work well with openHAB?

Described above, if I were to use your current topic structure.

If I were doing it myself, I’d probably use the Homie standard and do whatever makes sense for that so my device can be automatically discovered by OH. Or I would do what Johannes describe and use just the one topic and distinguish between button presses in the message body. In general I prefer to implement as much of the timing type stuff on the device instead of OH.

@JGKK

It’s a limitation of the mqtt binding as it doesn’t offer a button type channel were you could use the channel triggered event directly in rules. I think this is do to mqtt buttons being a pretty niche requirement.

But it does. See above.

[leif (Leif Claesson)]

@rlkoshak

However, there is nothing that says you need to link each Channel to it’s own Item. It is perfectly normal and very useful to link multiple Channels to the same Item.

This is a good point. I didn’t think about that. If I’ve added multiple channels to a think, and then linking multiple channels to one item, the rule that is triggered by that item will know which channel triggered it, even if the channels are on a “Generic MQTT Thing” as opposed to on the broker? That is indeed another good way of doing it of which I was not aware.

Rather than repeat things how about we continue in this thread where we’ve touched many of the same issues? I kind of only meant this thread to be a “simple” question, it probably gets confusing for others trying to follow now that it’s split into two threads.

[rossko57]

@leif

the rule that is triggered by that item will know which channel triggered it,

No. You tell your rule what to trigger on.
when Item someItem changed
for example, it does what you tell it and triggers when the Item changes. Whether that change was caused by this channel, or that channel, or a rule, or UI, is not what you asked for.

When using a channel event trigger like
when Channel "astro:sun:local:set#event"
then it’s a channel event, and no Item is involved here.
Note that only some channels are event types, most are state types. To trigger from a state change/update you need an Item.

If you need to distinguish what caused an Item state change/update you need either separate Items, or a list of possible causes to examine in code.