Typically this is something that can easily be done using MQTT and that would also be the protocol of choice, even when using esphome. Maybe there is an easy to use MQTT library you can include in your project?
@Spaceman_Spiff I respect your personal and subjective opinion, but the ESPHome developers argue the opposite
If we want to discuss this I’ll gladly make my argument but maybe a mod can split this topic so we don’t hijack this thread.
First let’s assume that I’m already running an MQTT broker because I have other services that integrate through MQTT. That makes the “One less single point of failure” argument obsolete. MQTT is also quite stable and has very low latency. I’m seeing >10k messages per sec with an unoptimized python library on my hardware. These last two points are not quantified and thus are a statement without proof. So I’ll not give “Stability” and “Low Latency”. The protocol buffers are much more efficient with 1/10 of the size of a message which is a +1 for native api. “One click configuration”, too if one would use this. But since I created the esphome yaml I know which topics are reported and I know when I make a change so it’s easier for me to just modify the mqtt.things file while modifing the esphome.yaml and skip the discovery altogether. I deploy both files simultaneously.
So that’s still +1 for native api.
However when things don’t work I can always start e.g. MqttExplorer to look at the raw messages that the device sends. That makes debugging much easier. Also your binding does not allow static channel definition and relies on dynamic channel creation like e.g. the Shelly Binding (last time I checked).
I’ve had very bad experience with this as there are rare cases where this does not properly work and e.g. the channels are missing after an openHAB restart or until discovery is complete or if discovery is not working properly the channels are missing altogether.
Since I know what channels should be there why do I need auto discovery and why can’t I describe it in a file so the device configuration is static?
I’ve had very hard to track down errors, device that suddenly did not work after restart and many issues with this and I value the stability, the deterministic and proper startup way more than e,g, an optimized message size.
To be fair these experiences have been with dynamic channel generation in general and not with your binding. Maybe everything works flawlessly there (for you 
), but I’ve been more than burned and I’m not taking any changes because of the sporadity of these issues.
That’s one of the lessons I learned the hard way:
If channel creation is dynamic - you better panic
1 Like
We’ve had this dicussion before, and the only reason I’m reacting is your subjective claim being presented as the generally accepted opinion.
No, this just makes your MQTT broker an even more critical single point of failure.
A redundant broker would take down the risk.
If you are using item files, that would be the place where you write down the expected channel names. It is just one layer less of redirection (in your case Device -> MQTT Thing -> Item channel link - in my case Device -> Item channel link).
I fully agree that the current user experience with regards to broken channel links is a disaster (we discussed this here the last time: ESPHome binding for the Native API - #61 by Spaceman_Spiff)
IMHO what could improve the situaiton is:
- Make broken link detection a first class citizen in MainUI - it should be highly visible if a link is broken - and it should be easy to fix for the user. AFAIK currently only available in the openhab-cli. The only thing worse than a broken link is not being aware of it.
- Whenever a thing goes offline or is disabled, all linked items should change to UNDEF - with the possible exception of items linked to more than one channel (I have none). This is done in some bindings, but should be handled on a framework level. There might be situations where this is undesireable, but if so we need to make this configurable.
- A binding should be able to specify it’s expected channel update interval, and the OH framework should then react if the binding fails to update in time. This is partially impletemented by some bindings in a custom manner, but I think the overall stability of an OH installation would benefit from a common solution.
I’m fairly certain that both you, me and many other contributers realize that their smarthome setup has become mission critical, and fault detection and mitigation is highly important to ensure stability. I know there are developers on this forum that has real experience in building industrial control systems that could provide better suggstions on how to improve upon in this topic.
Regards
Arne
You are technically right.
What I meant is that the amount of single points of failure points for the whole system stay the same in this case.
Correct.
But the difference is that MQTT Thing -> Item channel is statically configured and will always be there while Device -> Item channel link depends on some very complex initialization logic.
So I can be (almost) 100% sure that when the device sends the command through MQTT the item will update accordingly but that’s not something I can say for the second implementation.
After our last discussion I implemented a warning in HABApp which does exactly that.
Yesterday I did a restart and got 72 (!) warnings (Z-Wave, Shellies and a floating Thing I left as a reminder because I still have some todos left).
I’ve not yet moved all shellies to MQTT so once again here is a roller shutter that is missing roller shutter channel 
[2024-02-23 15:07:53,431] [ HABApp.openhab.links] WARNING | Channel "roller#control" on thing "shelly:shelly25-roller:BBBBBB" does not exist! (link between item "CmdShutterOffice" and channel "shelly:shelly25-roller:BBBBBB:roller#control")
That’s the tricky part. Because what can I do now? The only thing I can do is either remove and re add the thing and hope it’ll work or restart openHAB.
Imho the only clean solution is to add special discovery steps to the thing status and allow the user to use the result of a successful initialization and skip this discovery. I’ve created an issue with some Ideas and I’d love to hear your thoughts about them.
Good idea - I think there should be a possibility to add a grace time period.
Hm - I think this is covered pretty good by expire. Such a functionality would have to respect a per device per value user specific report interval and this complexity is easier to set on the item.
For bindings/devies which are not configurable this makes sense.
Or maybe just a simple reportInterval config switch on the thing which will check if any channel receives a value and the sets all to UNDEF?
I absolutely agree and I think we both want more or less the same thing:
In the end we all want our smarthome to just work™
I agree to the first 3 steps under the “Improvement to things”, but I don’t have much opinion on the latter 3 . I agree that configuration properties on each thing should be standardized and not stored in custom files like the zwave binding does.
IMHO the OH framework is a bit too “happy-go-lucky” in terms that it does not tell you very much when functionality is broken. A few examples:
- Send a command to an item with a broken/non existent channel link
- Send a command to an item linked to a thing channel where the thing is offline
OH gladly both accepts the command and sends it to /dev/null without informing the user or even logging a warning.
I’m fairly certain that this has been discussed at least 10 times before, but not implmented due to some cornercase where it doesn’t make sense.
I’d love if you implemented that warning in openhab core so that every user would get the benefit of it 
Regarding making it easy for the user to “fix”; this could be things like;
- Notify the user about the problem (the most important step!)
- Tell the user about the thing status - and also show some statisitics. Does it go offline often? → maybe there is a problem with the thing/communication/cloud service?
- Does it happen to all channels - or just a few?
- Has the number of channels on the thing gone down? → indication of trouble (like you experience with the Shelly things)
To a certain degree - yes. But:
- The user needs to add this information to every item (feasible for a small installation)
- Which value to use? What is the update interval for the thing? This may not be visible to the user and only obtainable by analyzing the binding source code)
Yes the value would need to be overridable - but I think the binding should provide the default value in a dynamic fashion to the framework as the reportInterval may depend on some configurable polling frequency etc.
You don’t need to setup a channel link. So this is only a valid issue if there is/was a link which is now broken or no longer existent without user interaction (one could remove a link intentionally)
At least with knx you can successfully send commands to channels which are part of offline Things.
It’s even possible to get channel updates from knx although the thing is marked as offline.
While one could argue, that the online/offline state is wrong and has to be fixed there may be other addons as well which have the same issue.
In fact, openHAB must always send the command and there is no guaranty to get an affirmation from a device.
But yes, it may be helpful to at least give some hint that the command was issued to a channel which is marked as offline.
The MQTT binding used to do this but was forced to take it out. I don’t recall the reason but I think among some of the maintainers UNDEF isn’t expected to be used for this. I would expect some pushback on this proposal.
I reminded the behavior in a rule for awhile but found it didn’t add anything to my situational awareness and took it out and replaced it with the more genetic Threshold Alert rule template.
Not necessarily. I’ve implemented approaches using a single status Item per equipment and a timestamp profile. Link the most reliable updated Channel to the Item and expire it to NULL when it doesn’t update. Using the semantic model it is easy to get the status of the equipment in the UI or in Rules.
None of this is an argument for our against any change to OH.
Threshold Alert rule →
So you are suggesting to use NULL instead of UNDEF? Or that we invent a set of new UndefType values such as THING_DISCONNECTED and THING_FROZEN?
I would assume the primary question for a user is; “Is the value valid/real/updated?”. Secondary is; “If not, why so?”.
Yes, this is doable for a small installation, but throw in a double or triple digit number of equipments and it might feel like an unnecessary burden compared to a generic built-in solution that can be overriden whenever the default behaviour isn’t suitable for a particular set of equipments/things.
IMHO Any user provided rule (including expire mechanism) that sets states to UndefType.SOMETHING reatroactively after a certain amount of time is IMHO to be considered a safetynet to a faulty, generic implementation.
Rgds
In jruby you could do
if !MyItem.thing.online?
...
end
One could easily write a rule that would set all {or some) linked items to undef, null, or whatever value desired whenever a(or any) thing goes offline.
Yes sure one can solve some of the issues by writing rules. I’ve done so in the past myself to compensate, but I would argue that common problems should be solved in a common way by the framework, and thus make it more accessible to less tech-savvy users.
And with the KNX example provided by @Udo_Hartmann in an earlier reply, a rule writer might experience some unexpected behaviour.
One of the beauty of openhab is that it doesn’t force you to do something a certain way. I prefer not to set my items to UNDEF whenever the things go offline, because the item’s current state is valuable to me.
Here’s a few lines to make all items go UNDEF 5 seconds after their thing is no longer online. If someone wants this behaviour, simply create a similar and very simple rule to gain that behaviour on their system.
rule "Make linked items UNDEF when things are no longer online" do
trigger "core.ThingStatusChangeTrigger", thingUID: "*", previousStatus: "ONLINE"
run do |event|
after 5.seconds do
next if event.thing.online?
event.thing.channels.each do |channel|
channel.items.each { |item| item.update UNDEF }
end
end
end
end
Are you sure it should not be NULL? See [mqtt] Treat incoming empty string as NULL for most types by ccutrer · Pull Request #16307 · openhab/openhab-addons · GitHub
While I agree with your sentiment against dynamic channels, one would think that the esphome binding got the channel details directly from the esphome device, and that information should be exactly the same as configured in your esphome yaml. The only downside is that this adds a tiny bit of delay on the initial connection.
So I would love to see the esphome binding allow static channel configuration method and when it is done statically, skip this interrogation step. I don’t know the ins and outs of the binding to understand / appreciate the challenges this may present though, and since it is not my binding, I have very little say in it.
For the time being, I am perfectly happy using mqtt + manual channel definition for my esphome devices. I generate my things + items programmatically, so I just need one template for each device type, and this is done the same way with the esphome configuration.
One thing about the ESPHome API I have been meaning to investigate is the ability for one esphome device to directly issue a command to another, bypassing mqtt and openhab. This would be handy for a switch to control a light, when openhab is down. However, AFAIK this is not supported.
In the other topic,
What about a utility that queries the ESPHome (using API or mqtt discovery), and generate the .things file.
Not sure at all. And from the linked discussion there seems to be confusion. If you look at the official bindings source code, there is no clear answer as well as both NULL and UNDEF are being used. Personally I am happy with either.
But as I wrote here;
Yes, but keep in mind that a single ESPHome component (ie climate) may provide multiple entities/channels, and retrieving those names from the esphome yaml would probably require parsing C++ code as well.
State updates from the device are done using a unique 32 bit key, and not by transmitting the full entity/channel name. The key is provided during the interrogation step. I’m not sure whether they are static across a reboot, but I’m pretty sure they won’t be static across a flashing process (ie version upgrade or modification).
So you cannot skip the interrogation step.
As I wrote in an earlier post:
Wouldn’t a “A python yaml to items generator” solve it?
Do you mean that if the device doesn’t actually provide all statically configured channels, the thing should refuse to go online? Opposed to some centralized logic checking for broken links every X seconds and informing you about the problem - and that applies for all kinds of thing types?
Wouldn’t a general tool be even more awesome? See Adjust width of F7 dialog + include clipboard icon - #5 by splatch
(Specific: In order to interrogate an ESPHome device you would have to provide credentials in most cases)
1 Like
Oh I wasn’t suggesting to use the esphome yaml to generate the channel details. On the contrary, my point was that the channels provided by the binding as a result of its interrogation should be exactly the same as the yaml, so there should be no reason to define it statically, in theory, if we trust the binding to work correctly.
Which yaml file are you referring to? I’m not going to make a second (wild) guess
Cheers
I was referring to the esphome yaml, and by that I don’t mean literally just what’s in the yaml, but the resulting esphome entities that would’ve otherwise been broadcasted through mqtt.
Having said that, the main reason I didn’t pursue using the esphome binding was because my esphome configs involve sending custom mqtt messages to communicate. I need (and tried) to refactor it into standard components, but there are some things that aren’t possible. I did ask the esphome folks about how to get around it but I was told that mqtt is the only way to do what I did. Hence I’m still using mqtt for now. It has been a while now, and I don’t remember the details exactly. I’d like to revisit it again one day.
I’m not suggesting anything. Just warning you that MQTT used to set the Item states to UNDEF when the Thing went offline and the maintainers made them take that behavior out as a misuse of the UNDEF state.
Honestly the reasoning never made sense to me and I personally continue to “misuse” UNDEF in this way. But I don’t need to get the maintainers to approve my personal rules.
And since switching to a status Item I really don’t want my items going to UNDEF just because the Thing went offline.
Why does everyone always assume the worst case when it comes to something like this?
If one is using managed Items it’s a simple script (6-10 lines at most, probably less in jRuby) to find all the equipment and add a status Item to them complete with links and profile config (I’ll post a rule template on request). Then you need one rule (template already posted) to detect when it goes offline and do something about it.
Me too now which is why I added the status Item.
Another approach is to use the Thing Status Reporting rule template to set the status Item or all the linked items to UNDEF too if you want.
I do that for a couple of Things with a status Item.