Not in every case. Let’s say I have a garage opener that tries on receipt of an mqtt message. If I use retain then the door will open every time the mqtt client reconnects to the broker.
I don’t think the Garage Door example proves your point. In fact, it reinforces the need for using retain=true. Here’s why:
Example with retain=false.
Publisher: GarageDoor=>Open
Broker: Passes the topic’s value on to all subscribers.
Subscriber: GarageDoor opens.
— The connection between subscriber GarageDoor and Broker is lost. —
Publisher: GarageDoor=>Close
Broker: Passes the topic’s value on to all subscribers.
Subscriber: Garage Door remains open because it lost its connection and does not receive the topic’s value.
— Connection between subscriber GarageDoor and Broker is re-established. —
Subscriber: GarageDoor remains open because retain=false and the Broker has nothing cached to report.
Example with retain=true.
Publisher: GarageDoor=>Open
Broker: Stores topic’s value and passes it on to all subscribers.
Subscriber: GarageDoor opens.
— The connection between subscriber GarageDoor and Broker is lost. —
Publisher: GarageDoor=>Close
Broker: Stores topic’s value and passes it on to all subscribers.
Subscriber: Garage Door remains open because it lost its connection and does not receive the topic.
— Connection between subscriber GarageDoor and Broker is re-established. —
Subscriber: GarageDoor closes because retain=true and the Broker reports the cached value (“Close”).
If no new values are published while the Garage Door has lost its connection, then it doesn’t matter if retain is true or false. Upon reconnection, the Garage Door gets the last retained value, which is the state its currently in, or if retain=false then no value at all and remains in its current state. Same difference.
Consider another case, let’s say you have thermometer and that reports periodically. Now let’s assume the thermometer broke and isn’t reporting any more. I’d it correct the way you will your rule to keep the stale reading or would a NULL be more appropriate to indicate there to a problem and maybe do something else.
Ha! Trick question!
If you set retain=false, even when the temperature sensor is connected and functioning properly, a subscriber will get no value upon connection to the Broker until the sensor reports a value. So the initial Temperature upon connection is zilch until the sensor publishes a value. It can produce an undesirable effect in, say, a UI where the Temperature is unknown until the sensor publishes a value. It’s mitigated by having a sensor report frequently (like temperature every few seconds) but not for security sensors that report on events only. Overall, it’s not an effective strategy to use retain=false to serve as some sort of indirect indicator of sensor health. The advantages of retain=true outweigh it.
haha… this is fun…
I’ll respond separately to each person’s responses.
H102,
I’ll get this setup in a bit.
my question is, can I direct the backup to a network share?
I have a Synology 8 disk raided system that I hope will not lose data… so far so good for the past 4 years (crossing fingers)
I can create an SMB or NFS share, etc to allow access from the openhabian.
Taras,
I’ve only somewhat thought the Garage Door scenario through.
But I’m hoping that I can use just a simple button press, maybe labeled “Press” or “Open” ( a mapping thingy?) that will send the initiation of the publishing topic “pushgaragebutton”. This will have the Other Pi act on seeing that topic publicized and initiate the flow I already have setup and working in Node-Red.
I also have a contact sensor currently placed on the garage door which I HOPE to incorporate into this whole garage door fiasco. Maybe when it sees that the sensor is open (0) it will toggle the display on the Sitemap showing it to be “open.” That will in turn place the map state to Open… maybe? That way I can possibly clear the topic and set to Null… again, but… haven’t quite thought about this deeply.
But hopefully, this can be done, that when the contact for the door is Open, the label will show “Close”
Then, at the press of this button, it will initiate the publishing of the topic again.
Even after a glitch or restart, etc… the topic is not being published in any other way so should not, in theory, publish that topic.
Your thoughts?
I’m hoping to start looking at this today… BUT, it’s my Wife’s BD today. I better spend some time with her or else! lol
Thanks again
I’m sure there is a way to use your existing setup as a backup but I’m still learning. Haven’t reached that grade yet…LOL Just don’t put off the backup, even if ya gotta do it manually a few times.
A bit of searching the post and most likely you’ll find exactly what you need to know. Yes you can and I advise using SMB share. Get VSCode software and use the openhab extension to assist in coding. It’s also an easy way, compared to ssh, to access and edit a file.
Oh Yeah… I just learned about Visual Studio Code about a month ago after taking an Azure Automation class.
I’ll see if I installed the SSH plugin or extension for that.
I’ll let you know how that backup pans-out… sometime :)
Good Deal!!
You should think about marking this post as solved. Also, click the heart icon when you get or read advise that is helpful. Kinda helps to verify what’s said is good advise.
Door doesn’t close when I told it to. Other checks recognize that the door didn’t close when I told it to. I press the button again. Door closes.
Note, if you are not using something to monitor the state of the door (open, moving, closed) there is no safe way to operate the door remotely. So if you press the closed button and the door doesn’t move you should know.
I’m not home.
Power goes out in the neighborhood.
Power comes back on and garage reconnects to broker.
Lasts retained message is to open the door (for what ever the reason, let’s say the door was open when the power went out). Door opens.
Now I’m not home and my garage door is open. Or even worse, I’m not home but my wife and son are and the door is opening in it’s own scaring the crap out of everyone.
This actually happened to me more than once. Far better to have to manually resend a command again than to have doors automatically opening because of network or power issues.
Further more, heavy equipment like a garage door should never ever just start moving on its own. That is a huge safety problem.
The subscriber will receive the last retained message even if that reading was from an hour, a day, a week, or a year ago. So if my sensor stopped reporting last week all of the subscribers are receiving stale and not useful data. In this case, IMHP, it’s far better to treat the reading as undefined until a new reading comes in then to just default to reading in the most recent reading, no matter how old that reading is.
But that is really besides the point. The main point is there are lots and lots of use cases where retain=true is not a correct option to employ.
I started with retain=true. I have no retain=true left in my configuration. You will never convince me that it is the correct or only good choice in all, or even in most cases. I’ve been burned far too many times already.
It has its uses but to say it should be used all the time is simply wrong. If it were the only “good” setting then there wouldn’t be a retain option at all, that just be the behavior of the messages. But sometimes the correct use is to let the messages be ephemeral.
Is that always an undesirable effect all the time? Clearly not and that is my point. Sometimes there are used cases where seeing - in the sitemap is desiresble.
I never said it was. What I am saying is there are some use cases where old messages can be worse than leaving the Item NULL until a new reading comes in.
There is no one size fits all seater in this stuff. Everyone needs to evaluate their particular use case and chose the best settings on a case by case basis.
A possible approach that I use for a sensor/controller that manages its state in MQTT is to implement a topic like groundfloor/garagedoor/state and use a “last will”. For the example of the door, there are (at least) three states: open, closed, offline. The first two states are set by the sensor/controller and the last one is set by the MQTT broker (based on the lastWillMessage) when it loses connectivity to the door controller for any reason. In this way, any end-user software that controls the door can determine that it’s offline. When the door comes back online, the first thing it should do is reset the state to “open” or “closed”. In all cases, the message should be retained. As stated before, there are situations where not retaining the message may be logically correct (e.g. temperature).
HTH
Pete
I’m not home.
Power goes out in the neighborhood.
Power comes back on and garage reconnects to broker.
Lasts retained message is to open the door (for what ever the reason, let’s say the door was open when the power went out). Door opens.
You state the door will open because retain=true … but the example states it’s already open during the power loss. Therefore, no change in the door’s state; it’s open before and after the power loss, regardless of the retain flag’s mode.
Generally speaking, the door’s actual state is the last topic value it received. So when the connection returns, the door remains in its state regardless of the retain flag’s setting. The example I provided shows what happens when a new topic value is published during the connection outage. Now when the garage door reconnects it will not get the new topic value if retain=false (doors stays open even though it was commanded to close). On the other hand, if retain=true the door will receive the command upon reconnect.
…the door is opening in it’s own scaring the crap out of everyone.
This happens to me regularly. I’m working in the garage and my wife arrives home and activates the garage door from around the corner (i.e. without seeing that I already have the door open). Yep, unfailingly sets my heart to race!
FWIW, I have automation logic check the garage door’s state when the security system is ‘armed away’. If we drive away without closing the garage door, after a short grace period, it will attempt to close it. An announcement is played in the garage stating the door will close in 10 seconds. It makes up to two attempts to close the door. If it is successful, it sends a "courtesy email’ to let us know it took care of our oversight. If it fails twice, it notifies us via email that it tried but failed to close the door (soon to be changed from email to SMS text).
The subscriber will receive the last retained message even if that reading was from an hour, a day, a week, or a year ago.
I agree that in this situation, retain=true appears to conceal the fact that the sensor has stopped reporting. However, using retain=false to correct it is not the optimal solution. This mode produces the effect I described, namely no value is received by a re-connected subscriber until the sensor publishes one. My outdoor temperature and light sensors report every 10 minutes. That means, worst case, the UI on my phone will show no sensors values for up to 10 minutes (it was this very situation that made me learn about the retain feature).
A better way to handle disconnected devices is to use another MQTT feature, designed for this very purpose. Peter describes it in his post below and it’s the Last Will and Testament. When the Broker detects the temperature sensor has disconnected, it can publishes the sensor’s Last Will which can be something descriptive like “UNAVAILABLE” or “SENSOR_LOSS”. This is the feature’s purpose, to inform subscribers that a publisher has disconnected.
It has its uses but to say it should be used all the time is simply wrong. If it were the only “good” setting then there wouldn’t be a retain option at all, that just be the behavior of the messages.
MQTT continues to evolve. Its original incarnation lacked the retain feature. It was added to handle precisely the kind of situations I’ve described.
Is that always an undesirable effect all the time [no topic value upon initial connection]? Clearly not and that is my point. Sometimes there are used cases where seeing - in the sitemap is desiresble.
The lack of a topic value is not a definitive indicator. It can mean the broker simply has nothing to report (because retain=false) or the sensor has disconnected. You don’t know which condition is in effect. You’d have to wait for the sensor’s reporting period to expire (if it even has one) to know which condition is responsible for the lack of a value. If the goal is to have a value that definitely indicates sensor loss, then the Last Will feature serves that purpose.
My garage door is activated by shorting the wires in its wall-remote for 1 second using a relay. That’s all it takes to activate the motor. However, that action can either open or close the door (i.e. it’s a toggle). You don’t know which will action happen unless you also know the door’s current state. If the door is open then the toggle action will make it close and if closed then it’ll open.
You appear to have the same two devices, namely a means to toggle the door and a sensor to report the door’s current state.
I currently use a different home automation system, so I can’t give you instructions specific to OpenHAB (at least not until I migrate to OpenHAB). Anyway, make the UI’s toggle button display a label that is opposite to the door’s current state. Then you need to determine the average time it takes for the door to close (time it). Add 5-10 seconds or so and use it as the period for a ‘watchdog timer’. A what? A timer that starts counting the moment you command the door to close.
When the door sensor indicates the door is closed, it cancels the watchdog timer. If the timer runs to completion, that means the door failed to close. Now you have to decide what the watchdog should do in this situation. I have it try again (don’t forget that it’s a toggle; the first command will make the door return to full-open, then the second command will make it close). In my case, after two failed attempts, it emails me that it tried but failed to close the door.
Looks like I need to read up on Last Will and Testament AND the watch Dog timer.
These both look to both be very useful for this kind of thing.
Thanks fellers!
To be precise, a watchdog timer is a strategy and not a feature specific to OpenHAB (or other product). It’s a very simple way to handle ‘something gone wrong’. Basically, it’s nothing more than a timer that, upon expiration, executes some code.
The code is designed to handle ‘something gone wrong’. When everything works correctly, the timer never runs to expiration, and never executes its code, because it’s cancelled by ‘something gone right’. In the example above, when the garage door closes, it cancels the timer. If the door fails to close, the timer runs its course and executes the code to handle the failure situation.
When something goes wrong, the watchdog handles it.
Watchdog timers can be used with heartbeats. I’ve used this strategy in many of the drivers I’ve written in order to detect when a connection has failed and attempt to heal it. A heartbeat is just some sort of command sent to a connected device on a regular interval (like every 15 seconds). A prompt reply (acknowledgement of receipt) is expected from the device. Each transmitted heartbeat also (re)starts a watchdog timer. If no response is received within the allotted interval (15s), the timer executes its code which is typically to log the error and reset the connected port (i.e. heal the connection). Actually, I usually allow for at least one failure and only call it an error if two consecutive heartbeats don’t elicit a response.
If there was no Last Will feature available, you could probably use a watchdog timer to monitor an MQTT topic. For example, let’s say you know for a fact that a temperature sensor always reports within a period never exceeding 15 minutes. Set the timer to run for 16 minutes and if it runs to expiration, it can notify you that the sensor may have malfunctioned. This can easily be done in Node-Red and undoubtedly with OpenHAB.
Watchdogs (and heartbeats) are part of your toolkit to ensure that when things don’t work as planned, Plan B springs into action and attempts to handle the failure.
A way to handle this. Not always a better way. Not always even a desirable way.
We can go back and forth on this for many posts. But I already know you will not convinced me. I sent months using retain=true for everything. It caused far more problems then it solved in my use cases.
I’m fully aware of LWT and use it regularly. I’m not talking about detecting a device has failed, though LWT implies that the only failure that occurs is that the device loses connection with the broker.
I have not and will never argue that retain=false shouldn’t be used. But saying it is there only valid option is simply not true.
Yes, but your situation isn’t there only use of MQTT. There are tons of use cases and systems models where it makes no sense.
But in all of those cases, from OH’s perspective, the state of the Item may properly be undefined (i.e. NULL) until a new value is posted.
Motion sensors, push button actuators, stuff like that.
When the only tool you have is a hammer, all problems look like nails.
That isn’t the goal. And as I said, LWT only indicates loss of connection to the broker for a given time. The goal is to let the state of Items remain undefined until a new value is posted. retain=false makes that impossible without a whole lot of extra work.
I’ll close with a few concrete examples from my setup.
I chose to model my garage door opener to work just like the physical remote. When the controller received a message it simulates a button press on the door (it’s physically wired to the wall button with a relay). Receipt of the ON message is interpreted as the command to press the button. There is no other message. retain=false would mean the button would be pressed every time there is a reconnection to the broker.
I control me house fan based on the temperature readings from 4 different sensors. I do not want to turn the fan on based on any reading more than 10 minutes old. If a sensor hasn’t reported for over ten minutes that Item should be NULL because as far as the algorithm is concerned that is the indicator that the value shouldn’t be used. Oh, and not all of the sensors are MQTT and using retain=false let’s the MQTT sensors behave the same as the Nest and Zwave sensors which also remain NULL until they’re is a new reading. Oh, and if a LWT is receive from over of them then all the Items that get proposed from that device are set to NULL as well.
This is one I’ve seen but is not mine. Let’s say I have a mix of zwave and DIY motion sensors. The Zwave ones just report ON when motion is detected, never OFF after a timeout. In order to have the two sets of motion sensors behave the same, retain=false has to be used. Otherwise every time OH connects to the broker it will think the MQTT morning sensors went off.
In all three cases above could I use retain=true? Absolutely. Is it more work? Absolutely. Does retain=true actually provide any value in these use cases? No, it just adds more work.
An alternative implementation for Timers using the Expire binding. I can’t remember if the Motion Sensor timer DP includes an example.
Use the Expire binding to set the Item to NULL or some other value to indicate an error.
There is also the following DP which applies that concept to generically since most technologies OH interfaces with do not support LWT type capabilities.
I need to rewrite this DP so all questions if you use it and have problems. If all you have is MQTT, the logging becomes way easier because of the LWT.
We can go back and forth on this for many posts. But I already know you will not convinced me.
To be precise, my comments are to clarify the application and benefits of the Retain feature and not to change your opinion. Combined with other MQTT features, the Retain feature makes for a robust system and does not rely on the ambiguous NULL state.
But saying it is there only valid option is simply not true.
Only valid option? I’ve reviewed my posts and I never said that. In fact, you misquoted me. “A better way” was the beginning of a sentence about the Last Will feature.
I chose to model my garage door opener to work just like the physical remote. When the controller received a message it simulates a button press on the door (it’s physically wired to the wall button with a relay). Receipt of the ON message is interpreted as the command to press the button. There is no other message. retain=false would mean the button would be pressed every time there is a reconnection to the broker.
Is it possible there’s a step missing here? The relay contacts have to close, to activate the motor, then open to return to the normal, dormant state (i.e. don’t leave the wires shorted). In my home, the relay is modelled with a 1-second delay to create a momentary closure. Therefore the door’s MQTT value is ON then followed immediately by an OFF value. The retained value will be OFF, so the door wouldn’t toggle its state upon a reconnection.
I control me house fan based on the temperature readings from 4 different sensors. I do not want to turn the fan on based on any reading more than 10 minutes old. If a sensor hasn’t reported for over ten minutes that Item should be NULL because as far as the algorithm is concerned that is the indicator that the value shouldn’t be used.
I’m assuming the temperature sensor normally reports frequently and 10 minutes is well beyond its normal reporting interval. The example implies a 10-minute timer is needed (otherwise how would one know the value’s age) in order to indicate the temperature reading is now stale possibly due to failure.
If the timer is part of the code handling the fan, then the retain flag’s mode is of no consequence, the sensor’s value is simply ignored if older than 10 minutes. This is fine if the fan is the temperature sensor’s sole subscriber.
If there are many subscribers to the temperature sensor’s topic, my preference would be to have the timer operate independently (of the fan and all other devices) and function as a watchdog for the temperature sensor. Upon its expiration it would publish to the fan’s topic a value like “STALE” or “FAILURE” or “-273.15” (an out-of-range value like absolute zero if we want to keep things numeric). Now all subscribers of the temperature sensor are apprised of its true condition and can react accordingly. Retain=true works well in this situation because upon connection a client is apprised of the temperature sensor’s true state, namely “FAILURE”. If retain=false, a freshly connected subscriber gets “NULL” which is ambiguous and might mean sensor failure or simply because you connected in the middle of a functioning sensor’s reporting interval.
FWIW, Moquette calls itself an ‘MQTT service dispatcher’. It monitors MQTT traffic and when a certain topic value occurs (or does not occur) it can spring into action. Monitoring the frequency of sensor updates would be right up its alley. If a sensor failed to report within its allotted interval, Moquette would publish “FAILURE” to the sensor’s topic (and log the event and send you a notification and whatever else you may want). Without question, this kind of sensor ‘watchdog’ can be implemented in any HA system (or Node-Red). However, in a complex system, where there may be many different kinds of subscribers, it may be useful to have the ‘watchdog’ functionality centralized in an independent service like Moquette. Just one more tool in the MQTT toolkit.
Let’s say I have a mix of zwave and DIY motion sensors. The Zwave ones just report ON when motion is detected, never OFF after a timeout. In order to have the two sets of motion sensors behave the same, retain=false has to be used. Otherwise every time OH connects to the broker it will think the MQTT morning sensors went off.
This category of sensor can also be handled correctly with retain=true. I’ve seen this issue discussed on the Home Assistant forum, regarding certain battery-operated sensors (that may take shortcuts to ostensibly reduce battery drain). For sensors that only report ON, never OFF, they implement a timer to expire the ON-state and return it to its quiescent state, namely OFF. That gives it operational parity with other motion sensors and makes it comply with HASS’s Binary Sensor platform.
If I were modelling this kind of sensor in my HA system, I’d do the same by including a user-adjustable timeout in its device driver (aka binding) to make it comply with my HA system’s standard-model of motion sensor behavior. I haven’t had to do because I use hard-wired motion sensors (via an alarm panel) that follow the standard-model.
In all three cases above could I use retain=true? Absolutely. Is it more work? Absolutely. Does retain=true actually provide any value in these use cases? No, it just adds more work.
I’ve had the opposite experience (it provides value, saves time and effort) and I hope others discover and reap the same benefits for themselves.
But NULL isn’t ambiguous. It unambiguously means the Item doesn’t have a state. That isn’t ambiguous. The Item doesn’t have a useful state.
If you are certain the connection between the client and broker will always be intact (perhaps both are on the same computer) then retain=false is a viable option (however, it doesn’t make anything noticeably faster and the saved overhead is negligible).
If there is a possibility of interrupting the connection between client and broker, then retain=true is the better option. It avoids the situation with retain=false where the connection is dropped, the topic’s value changes in the interim, then the client reconnects and now reports NULL (or the old value).
So yes, I agree, you didn’t say it is the only option. But you did say, unless I’m misunderstanding what you said, that unless you are running the broker and client on the same machine then retain=true is always the better choice.
And all I’m arguing is that it is not always the better choice in that situation. One needs to choose the right retain flag for how they are using the data posted to the MQTT.
And if that wasn’t what you meant then we are arguing about nothing as we agree.
That is implemented by the code that receives the MQTT message. It receives the message and in response turns on the relay for half a second then turns it off. I didn’t mention it as it is irrelevant to the retain=true discussion.
I usually try to put that sort of smarts into the device itself, not in the central controller. I shouldn’t have to know or care about the inner workings of the device that opens the garage door in OH. OH just sends a message to trigger the opener and the device wired to the button handles the rest.
Put another way, one message to MQTT to trigger the opener, the device that subscribes to that message implements the momentary button press.
If you are implementing this sort of behavior centrally then yes, I agree, retain=true is probably a really good idea. But that isn’t the only way to model/implement a garage door opener and in my approach, retain=false is the better option.
If I were to implement a full state machine on my device and instead of just triggering the button in response to an MQTT message, then retain=true would also probably be a good option.
My point isn’t that retain=true isn’t a good option. My point is that it isn’t the only good option in all cases.
You mentioned above that you haven’t transitioned to OH yet and so probably don’t have a lot of experience with how OH Rules and the other parts of OH work together. In order to know the age of the state of an Item one needs to configure persistence and write Rules to check the timestamp of the last update to the Item. And if retain=true then that would have to be written in every Rule that should not be making decisions based on data that is beyond a certain date.
Or, you can use the Expire binding and allow Items to be initialized to NULL (i.e. do not use persistence with restoreOnStartup) and in those Rules that care about the age of the Item just use a simple if(MyItem.state == NULL) return; or what ever is appropriate. Rules are much simpler and easier to write and dependent on fewer parts of OH (i.e. no need for a properly configured persistence) if we use NULL states for the purpose they were intended.
OH is the subscriber. I have four temperature Items and one fan Switch. I do not want the fan to turn on based on old temperature data. So by utilizing NULL to represent where I don’t have recent enough reading then I can write a very simple:
val upstairsTemp = gUpstairs_Temps.members.filter[t | t != NULL].map[ state as Number ].reduce[ max, t | if(t > max) t else max ]
if(upstairsTemp > Basement_Temp.state) Fan.sendCommand(ON) else Fan.sendCommand(OFF)
Note, my real code has more logic than this.
But if I used retain=true and I’d have to set up persistence (which I have so that isn’t really a hardship) but the code becomes more complicated.
val upstairsTemp = gUpstairs_Temps.members.filter[ t | t.lastUpdate != null ].filter[ t | t.lastUpdate.isAfter(now.minusMinutes(10).millis) ].map[ state as Number ].reduce[ max, t | if(t > max) t else max ]
if(upstairsTemp > Basement_Temp.state) Fan.sendCommand(ON) else Fan.sendCommand(OFF)
I’ve got two filters instead of one, I have to depend on an external service (i.e. database), I have to understand that for this to work I need to be using the proper persistence strategy (i.e. must use everyUpdate), I have to mess with time comparisons which is not super intuitive in OH, and so on.
Or NULL? That is the purpose of NULL states in Items. NULL indicates that the Item doesn’t have a state. You can invent a whole bunch of “invalid” states or use what is built in.
But please note that I’m talking about OH. This is an OH forum after all. And OH is designed to be a central bridge between various home automation technologies. And in this particular case the temp sensors are MQTT and the fan is controlled by the Nest. So OH implements a Rule to interpret the states of the Items that represent the temperatures and issues the commands to the Nest to turn ON/OFF the fan.
If you are operating in an environment where everything is MQTT, I suspect there are more use cases where retain=true is useful and makes things simpler. But in most of my use cases where pretty much every case I’m using MQTT (except for monitoring the health and status of my sensors which is handled completely separately, I don’t use NULL in the temperature sensor Items to indicate a failure or error) I have one publisher (OH or the device) and one subscriber (either OH or the device) because the sensor readings cause actions on a device that isn’t MQTT or the device is actuated from something that isn’t MQTT.
This is one place where I disagree. NULL is not ambiguous unless you are trying to use the state of the Temperature Item to also represent the failure state of the sensor. That would be an anti-pattern in OH IMHO. The failure state of the device should be represented using a separate Item. So the NULL unambiguously means that the Item doesn’t have a state yet. You’d need to look at the device status Item to determine if there is an error with the sensor.
Sounds like a useful took that many would find useful. I bet a tutorial on how to set it up and use it with OH would be a well received tutorial.
But I wouldn’t really have a use for it in my environment.
And one could argue that an equally valid approach is to implement the MQTT sensors to behave the same as the Zwave ones. It’s opposite sides to the same coin. But I’d argue that setting retain=false and implementing the Rules to handle all the motion sensors as they only send the ON and never OFF would be less work in OH.
And maybe that is the crux of my argument. Yes retain=true has many uses. But using retain=true everywhere in OH will cause more work in the long run. And many if not most of the benefits of retain=true have a way to achieve within OH itself, which lets one expand those benefits to all the supported technologies, not just MQTT.
Which doesn’t really help the rest of the users of OH. And when you are talking about something like the Zwave binding that works with hundreds of devices, it really isn’t tenable to create special code for all the little quirks of various specific makes and models of devices.
But it is easy enough to set up a watchdog timer using the Expire binding to make. And that is another one of my points. There are lots of ways to model home automation so a blanket statement that retain=true is always a better option just doesn’t follow.
Perhaps this is one area where the way OH is set up makes that not the case. And I’m not and haven’t argued that retain=true shouldn’t be used or that it doesn’t provide benefit. I’m only arguing that there are lots of cases where it can cause problems or extra unnecessary work and the user needs to understand what they are doing to choose the best option for their specific use case. I really don’t think it’s that controversial of a statement.
But NULL isn’t ambiguous. It unambiguously means the Item doesn’t have a state. That isn’t ambiguous. The Item doesn’t have a useful state.
Perhaps I should’ve clarified this point earlier because maybe OpenHAB’s handling of ‘no value’ (NULL) is different from other systems. When OpenHAB connects to an MQTT Broker and subscribes to a topic that was published with retain=false, what value does it report? NULL?
Because if it does report NULL, the actual topic value may very well be non-NULL, it’s just that OpenHAB won’t receive it until the topic value changes. Therefore, at the moment of connection, NULL is not a definitive indicator of anything. At that moment it can mean ‘nothing to report because retain=false’ or ‘nothing to report because the publisher has failed to report anything’; it’s ambiguous.
Or will OpenHAB report some other value upon initial connection?
Yes. All Items are initialized to NULL (not to be confused with null). I hate the overloaded name. But NULL is a special state that all Items are initialized with to indicated the Item has an undefined state, or no state if you will. What is really happening is OH subscribes to the topic, there is no message so nothing happens to the Item and the Item remains NULL until such time that a message is published, or the Item is commanded or updated through some other means (Rule, restoreOnStartup, manual interaction, some other binding, etc).
If you have persistence with restoreOnStartup configured for that Item then the Item starts with the state of NULL and then gets populated with the most recent value stored in the database. In a lot of ways, restoreOnStartup is the same as retain=false, except that it applies to all configured OH Items, not just MQTT Items.
From OH’s perspective, in some use cases, there is no practical difference between those two cases. Both mean the same thing. There has been no message posted since subscribing to the topic. Therefore the Item doesn’t have a state and remains NULL until such time a message is posted. And that is perfectly acceptable in many use cases.
There are other use cases where you can’t lose a message or the fact that a message was posted when OH was disconnected is important. But there are just as many if not more where that doesn’t matter. That’s my point.
Choose the retain setting that makes sense on a case by case basis.
And it is important to realize that the NULL state for the Item does not and is not intended to indicate there is a problem with the sensor or the connection or anything. That sort of thing should be tracked with completely separate Items. The temp Items only represent the temperature. And if the temperature Item is NULL it just means that you do not have a valid temperature value that you can use right now.
If you want to detect whether the sensor is online, through some other Item or set of Items you would use the LWT, a watchdog, or something like that to discover and react to the fact that the the sensor isn’t reporting. But the Rules that handle the temperatures only care whether or not there is a valid temperature to use or not. It doesn’t care why it doesn’t have a valid temperature.
If it has to care why, then of course that is achievable. And perhaps doing something that utilizes the retain=true makes sense in that case.
But in my particular case, it is far less work to let the temperature fan Rule use those sensors that have a valid number, ignore those that do not have a valid temp reading, and have a separate set of Rules that handles detection and alerting or whatever for problems with the sensor. The first Rule doesn’t care why the temperature value doesn’t have a valid value. It could be OH just started up and is awaiting a new posting to the MQTT topic. It could be that it has been over 10 minutes since the sensor reported a reading. It may be that the sensor is known to be down. The Rule doesn’t care. It makes no difference to that Rule. So, ok, NULL is ambiguous (in the MQTT case), but that ambiguity doesn’t matter.
Perhaps you meant to say restoreOnStartup is the same as retain=true? When true it behaves like OpenHAB’s persistence with restoreOnStartup (stored values assigned upon connection).
The persistence analogy is useful. Retain=true tells the broker to store the topic’s value and give it to subscribers upon connection. Without it, like OpenHAB without restoreOnStartup, all values are initially unknown until they change state. The parallel makes the concept easier to grasp for people new to MQTT’s Retain feature.
It would be interesting to learn from people who use persistence w/restoreOnStartup, what percentage of their devices are initialized on startup and which ones are excluded.
Not that it matters but the system I use has default persistence (restoreOnStartup) for everything. I do like OpenHAB’s fine-grained control (per Item basis) but I can’t say I’ve experienced a problem with full-coverage persistence in the many years (10+) it has been operating.
I agree with your statement whole-heartedly. The only reason I brought it up is because you appeared to say the opposite earlier in the thread (i.e. NULL potentially indicating a problem). It might indeed be a problem or, if retain=false, just no value supplied (upon connection) until the sensor changes state. I definitely agree with you that it’s preferable to use some other mechanism to monitor the sensor’s health and report when defective.
I used to restoreOnStartup everything. I now restoreOnStartup only a select few Items that:
represent a setting like offsets, flags, etc
represent data that remains valid for a very long time
cannot/should not be recalculated in a System started Rule
Everything else remains NULL until a reading comes in and my Rules do something different if they don’t have all the states they need (for example, if all the temps are NULL my fan controller exits and will be retriggered as new readings come in).
I didn’t mean that statement to mean that I’m using the NULL to detect that the sensor has failed. I used that as one example of a reason why the sensor reading could be NULL. My Rule doesn’t care why the reading is NULL, it only cares whether there is a state that is recent enough to be usable.
That is the purpose of NULL states in Items. NULL indicates that the Item doesn’t have a state. You can invent a whole bunch of “invalid” states or use what is built in.