[MQTT][Homie]Behavior when trying to control a device that is offline

Setup, Configuration and Use Items & Sitemaps
1

OpenHAB 2.5.0 S1628 on Raspberry Pi

I may be misunderstanding how this is supposed to work – wouldn’t be the first time. Please bear with me.

OpenHAB sends a message on the topic home/deviceid/node/property/set = value
…and the device responds and confirms by setting home/deviceid/node/property = value

So, what’s supposed to happen when openHAB sends a command to a device that is offline (whether lost or disconnected)?

The device then will NOT confirm the command, and NOT update the base property topic.

So, why is it that the following happens when you try to turn the switch ON:

mqtt_homie300_item predicted to become ON # yes, that is a fair prediction
mqtt_homie300_item changed from OFF to ON # NO it could not possibly have, the device isn’t even powered.

Shouldn’t the item change its state only when the device has confirmed the state change? Isn’t that the whole point of the “pessimistic feedback” design pattern which Homie is designed around?

2

If you do not like what autoupdate does for any particular Item, you may disable it. This is an openHAB feature originally intended to make a slicker UI experience. It’s not always appropriate and so is configurable.

Autoupdate’s predictions CAN be influenced by any of the potentially multiple bindings linked to an Item, but this would not be sensible for MQTT because…

MQTT is by definition a publish-into-the-void scheme. There is no possibility of acknowledgement built in. You do not know if twelve or none are subscribed to your messages, by design.

If openHAB could not publish, say the broker was down, then you might expect different behaviour.

Luckily openHAB provides tools that you can use to overcome the limitations of MQTT. You can use rules and timers to flag missing expected responses and so on.

3

MQTT is publish-into-the-void, but Homie has confirmation built into the standard, by the device publishing the new state of any property to a different topic than the /set topic where it accepts commands. Is that not part of the openHAB binding?

1 Like
4

But isn’t that what the differentiation between state and command topic is there for and why the two are separate topics? To have exactly this back channel for a command confirmation?


It even says that the state topic represents the state of the thing channel. So it shouldn’t change unleast the new state gets published there as a response when the command gets published to the command topic as they are two distinct topics.
Johannes

1 Like
5

The state change in this case is carried out by autoupdate. This is a “value added” feature of openHAB operating on the Item, nothing to do with MQTT. Where you do not want this feature, you can turn it off.

2 Likes
6

You are 100% correct, sir! Thank you.

image
image.png1793×1150 55.2 KB

Not the clearest description but I get it now. :slight_smile:

1 Like
7

I don’t see anywhere in the homie spec that an acknowledgement must be sent for every message received at a device, nor any kind of spec about how long that should take - essential info for this purpose.

What would you expect to happen if you sent On to a light that was already On? In my view, no response at all is fine and meets homie spec. Nothing changed.

In fact I don’t see anything in homie spec requiring instant publishing of any status. What would we do with a temperature sensor, publish each 0.001 change or each second or each hour?

I understand well what you want but the MQTT spec does not provide for it, nor does the homie discovery mechanism.

8

What about this?

Homie is state-based. You don’t tell your smartlight to turn on , but you tell it to put its power state to on . This especially fits well with MQTT, because of retained message.

For example, a kitchen-light device exposing a light node would subscribe to homie/kitchen-light/light/power/set and it would receive:

homie/kitchen-light/light/power/set ← "true"

The device would then turn on the light, and update its power state. This provides pessimistic feedback, which is important for home automation.

homie/kitchen-light/light/power → "true"

From here:
https://homieiot.github.io/specification/

9

What about it? It’s an illustration. Show me where the spec requires a response to every “command”.
I cite again my “what if its already on” example.

Homie is a discovery spec, using an existing transport that cannot do what you want by itself.
You might like homie to be enhanced to a more fully featured protocol, and why not.

10

I seriously suggest writing your own rules for an “acknowledge” function, to see what’s required.

We’ve done ‘already on’ part.

I think homie would need additions in the shape of describing which outbound topic to associate with what inbound topic. Or perhaps a naming convention could cover that.

Homie obviously cannot supply any info about how long to allow for round trip. It might be able to describe it’s own internal delays. Perhaps you could invent a process, ping essentially, that allowed something we might designate as a master to discover how long a round trip should take.

Need to decide what response is acceptable, light on you’d expect on back. But say a blind, first response to up might be 35%. Maybe anything, who cares. Or do you care if the response to on is off?

The case of multiple masters controlling one device is probably not a big problem, given the nature of MQTT, but maybe masters will need to deal with unsolicited “acks”.

I’m guessing the homie spec does rule out multiple devices listening on same topic. MQTT does not, but the homie id should be unique right?

If implementing a kind of “ack” you might consider a “nak” as well.
Cases
“I can’t do that it’s nonsense” - sending 50% to an on/off topic
“I can’t do that just now” - sending color to a light that is currently off.
The device could just stay silent. You may or may not want to differentiate broken device from nonsense command to good device. Many decisions to codify.

11

I’m not sure the best place to put this but I think this is the best category since the issue in the topic has to do with autoupdate.

There are acks, but it is between the broker and the client, not between clients, and then only if using QOS 1 or 2.

The specific behavior you are seeing has to do with autoupdate. All Items by default have autoupdate turned on. When autoupdate is turned on OH “predicts” what the new state for the Item will be based on a given command.

If you turn off autoupdate, the state of the Item will not update until something explicitly tells it to. In this case when the device pubishes on home/deviceid/node/property.

I don’t know what a Homie Thing looks like. With a Generic Thing you can set up a single Channel that pub/sub to both of these topics. Then you can link that Channel to an Item with autoupdate set to off and you will get the behavior I think you are after. If Homie uses two different Channels, one for the command and one for the status then you would link both Channels to the same Item and set autoupdate to off.

One nice thing the binding does is set all your linked Items to UNDEF when the connection to the Broker is lost. So that is one part of the “how do I tell if the device is reachable”. Unfortunately the other part is going to take some Rules. I do the following (can post the rules if asked):

  • Item linked to a Channel subscribed to the LWT topic
  • When OFFLINE (or whatever Homie uses) is published, all the Items linked to that Thing get set to UNDEF using a Rule. The Rule can be made generic using Groups.

I believe Homie uses retained messages for LWT so OH doesn’t necessarily need to be online and connected to the broker to get the LWT message and know the device is offline.

I don’t think that’s necessary here. It isn’t necessary for the old MQTT 1.x style configs like

{ mqtt="<[broker:testing/mqtt/topic:state:default], >[broker:testing/mqtt/back-topic:command:*:default]" autoupdate=false }

From what I know about how Homie is described to work, yes you are correct that every command does not necessarily mean that an ack gets published to the property topic. And maybe the working in Leif’s OP is slightly misleading.

The set topic is where you tell the device you want it to achieve a given state (it might already be in that state). The property topic gets published to every time the device changes it’s state. If every state change doesn’t get populated to the property topic, it’s a pretty lousy standard (OK, for stuff like sensors an update every so often, but thinking about something like a Switch, it has to publish the state changes).

So if I configure commands to go to the set topic, turn off autoupdate, and subscribe to the property topic, doesn’t that do what @leif is asking for? The Item only changes state on a command when the device reports that it has changed state?

As rossko57 has said, the behavior you are seeing is outside of the binding. By default openHAB turns on autoupdate which causes OH (or the binding) to predict what the state will become in response to the command. For example, a Dimmer Item will receive an INCREASE command and autoupdate may predict the Item to become 55.

autoupdate is turned on by default because it is actually the minority of technologues (at least when autoupdate was created) that provides feedback when they’ve been commanded. Heck, even older (before 2016) Zwave switches didn’t report their state back except for those from two vendors because of a patent. Homie and most MQTT type implementations are in the minority in that they report their state updates.

For most technologies supported by OH, pessimistic feed back is impossible because there is no feedback. OH has to assume that the command was reacted upon. So that is the default.

1 Like
12

Thanks Rich,

Holy crap, everything with home automation is a rabbit hole! I reiterate my previous thought, no way will home automation even be truly mainstream.

Auto-update makes perfect sense now, thanks for the explanation.

My homie devices re-publish every accepted command, whether the state changes or not, and ignore invalid commands. But, rossko is right – the standard doesn’t explicitly mandate that, I just followed what I perceived was the spirit of the spec.

Man, I should have just left well enough alone and stuck with plain MQTT :slight_smile:.

13

after building out a Homie 3 python library… plain old mqtt maybe better - depending on where the problems lie. My understanding is the Java and Python libraries are not based on any common code?

14

Home automation is a world of walled gardens with dozens of incompatible APIs and specifications. And the number of those “standards” keeps growing year by year.

standards

Homie is a perfect example of this comic. That doesn’t mean it isn’t a good standard or that it doesn’t have merit. But rarely do standards like these replace existing standards. So systems like OH have a tough job bridging between them all.

Anyway, much of what you are dealing with in this specific case is more related to:

  • specific openHAB behavior which you may or may not see in other home automation systems (i.e. autoupdate)
  • using a standard that has not yet gained wide adoption and therefore might not be as well defined as it should
  • for the most part, if you are messing with MQTT you are messing with DIY electronics or third party firmware flashed to a commercial device (Shelly is the only tech I know of that supports MQTT out of the box) so this already is going to narrow down the audience to only those users who are reasonably technically proficient and motivated.

Mainstream users would never deal with this sort of stuff. They’s be buying commercial stuff using higher level protocols and APIs so a lot of the low lever stuff you are dealing with here would be hidden from the users.

In a sense, MQTT is a relatively low level communications protocol and it is not, by itself, sufficient for home automation use. It’s akin to opening a TCP socket and writing raw packets or like Xbee communication which underlays Zigbee. One needs to define a higher level protocol on top of it to be useful. Homie is an attempt at building such a higher level protocol. If you are working at a lower level and you define your own protocol (e.g. Generic MQTT Things using self defined topics) you are largely doing the same as Homie, just in a custom fashion.

Is this on github anywhere? I wanted to experiment with Homie on sensorReporter but the last time I looked into it there wasn’t a complete Python library for Homie. It’s also going to require a pretty big restructuring of the code so time is a problem as well.

This goes to both of you, have you filed any issues on the Homie project to discuss some of the problems/limitations you’ve encountered implementing the standard? I’m sure they would benefit from such feedback.

15

@rlkoshak

The Python Homie 3 library I build is here https://github.com/mjcumming/HomieV3, you can install from pypi using PIP

The module has several ready to use common devices. Documentation is scant :frowning:

A quick example is below.

import time

from homie.device_dimmer import Device_Dimmer

mqtt_settings = {
    'MQTT_BROKER' : 'QueenMQTT',
    'MQTT_PORT' : 1883,
}

class My_Dimmer(Device_Dimmer):

    def set_dimmer(self,percent):
        print('Received MQTT message to set the dimmer to {}. Must replace this method'.format(percent))
        super().set_dimmer(percent)        

try:

    dimmer = My_Dimmer(name = 'Test Dimmer',mqtt_settings=mqtt_settings)
    
    while True:
        dimmer.update_dimmer(0)
        time.sleep(5)
        dimmer.update_dimmer(50)
        time.sleep(5)
        dimmer.update_dimmer(100)
        time.sleep(5)

except (KeyboardInterrupt, SystemExit):
    print("Quitting.")
2 Likes