OH3.1 MQTT Homie Support

leif · December 4, 2021, 4:37pm

Howdy!

I’ve used OH2.4 and 2.5 for close to 3 years, finally starting a new installation with OH3.1.

I have made a lot of Homie devices at this point – I use it for everything – and after playing with OH3.1 for just a few minutes, I can with confidence say that Homie support in OH3.1 is VASTLY improved compared to OH2.x. Very nice job!

It is much more responsive, much more stable, and it no longer randomly sees certain properties as completely the wrong type.

I have two questions / issues already, though.

First is related to adding / removing properties from an existing device. I’m seeing inconsistent behavior.

Homie devices consist of Nodes and Properties.

One time, when I added a new property, it instantly appeared in the channels list of the thing.
When I then deleted that property from my homie device, I could not get rid of it from the channels, openHAB kept it even though it no longer existed.
Then I renamed a node and added another property. At that point the previous zombie item disappeared, and the new node/property is not detected either.

Perhaps this is not the most common use case, but without a “refresh” button on the item, I’m not sure how to work around these issues when they occur.
Edit: I just noticed that deleting and re-adding the thing actually retains linked items! Okay it’s not great but it is a functioning workaround for now.

Second is the channel identifiers presented in openHAB.
The MQTT binding presents it as node#propertyname, which is great. This was the same in OH2.5 too. But, there is a major regression that I can see.

Here’s what it looks like in OH2.5:

And here is what it looks like in OH3.1:

For some reason hyphens and underscores are displayed as the hexadecimal character code rather than the character itself!

This is not pretty and it makes it difficult to read. Surely this cannot be the intent? Hyphen and Underscore are both part of the standard 7-bit ASCII set from the 1960s, I think it probably safe to use them

I’m actually more worried about this than the previous issue, because wouldn’t fixing this in a future version invalidate linked items?

As a developer of numerous devices based on my own Homie library it may be that I’m setting the record for number of Homie devices in one house, as well as the size of those Homie devices. The Master Bedroom relay controller is a homie device with 64 individual switches! In any case I would be happy to help out with testing of this openHAB feature.

liaskt · December 4, 2021, 10:52pm

This seems to be by design.

If you look at the code here, the UIDUtils.encode() function seems to be converting everything other than [A-Za-z0-9] to _ + byteToHex(). The reason is that there is a symmetric UIDUtils.decode() function to get back the original string. Therefore the underscore is used as a delimiter before the Hex characters and therefore it has to be converted to Hex (to avoid delimiter collision) when it appears in the input string.

Therefore an input string such as couch-downlights is converted to couch_2Ddownlights (where the 2D is the hyphen -, and the underscore _ before 2D is the delimeter). You cannot escape from this conversion even if you use underscore yourself: couch_downlights will be converted to couch_5Fdownlights (5F is the hex value of the underscore). The only way to avoid it is to use CamelCase in your MQTT topics and sensor names (if this is under your control).

Since this behavior is part of openhab core (and not the addon), it also behaves this way for other bindings as well, such as the MQTT Homeassistant autodiscovery one (I had the same issue as you and that’s why I looked at the code).

liaskt · December 4, 2021, 11:08pm

I also had the same issue and had to manually remove the channel definition from JSONDB.

I had to first stop OpenHAB, then find and remove the zombie channel in the thing defintions file in JSONDB, and finally start OpenHAB again.

Make sure you keep a backup of the json file first, editing json files manually may lead to syntax errors and break your installation.

leif · December 5, 2021, 3:31am

Thank you @liaskt. It is not the answer I wanted but it is clear. Ugh. SoI’mForcedToUseCamelCase!! Ugh!! Okay. Fine.

The names are under my control – all my homie devices are my own code. I will add an option to my Homie Library to automatically use CamelCase to sidestep this whole issue. But seriously, ugh.

Okay, I’m glad I’m not the only one, but that’s really not good. That’s definitely a bug that could be fixed. Is there a more direct way to report bugs than posting in the forum?

liaskt · December 5, 2021, 3:54am

I guess the proper way would be to open an issue at github.
I am not sure if this is a bug in the core or in the binding. My guess would be in the MQTT binding, so maybe the bug report should go to the openhab-addons repo. But I am really new to OH, someone else may know better.

leif · December 5, 2021, 4:56am

Thank you! I did that now, here:

github.com/openhab/openhab-addons

MQTT/Homie auto-discovery of changes does not always work

opened 04:29AM - 05 Dec 21 UTC

leifclaesson

bug

[org.openhab.binding.mqtt.homie] I have large expandable Relay controller board…s which present themselves as Homie devices, and thus those Homie devices are not static, if I add another relay output, the Homie device gets a new node or property which it did not have before. My issue is: When a Homie device gets a new Node or Property, openhab only **sometimes** gets a new channel. When a Homie device _loses_ a property, openhab **never** removes the channel, it keeps showing the channel that no longer exists. I do not know how to determine if the issue is in the binding or in the core. ## Expected Behavior When the topology of a Homie device changes, these changes are published in the $nodes and $properties topic of said device. I think openHAB should automatically respond to these changes and add/remove channels to match the Homie device. ## Current Behavior Zombie channels are left in their place in OH. ## Possible Solution If automatically adding/removing channels is not desirable/feasible, it would be great to have a "Refresh" button on the Homie _thing_ to force re-building it from the current reported topology of the Homie device. ## Steps to Reproduce (for Bugs) I would be happy to -- but what do you need? The developers of the MQTT / Homie binding certainly already have their own homie devices. Should I provide an ESP8266 Arduino sketch which reproduces the issue? I would be happy to code one up if that would be useful. ## Context I'm building a smart home with entirely my own hardware. I've designed and built my own ESP32 and ESP8266-based relay-boards that control all the lights in the house, and communicate via MQTT/Homie, and want to control them with openHAB. This is directly related to the following [openhab community topic](https://community.openhab.org/t/oh3-1-mqtt-homie-support/129597) ## Your Environment Docker openhab/openhab:3.1.0 on raspios-bullseye-arm64 accessed from Firefox 94.0.2 on Windows 10

Let’s see what happens.

leif · December 5, 2021, 6:03am

On second thought, that’s not going to work for me. I have a huge established network already, and I made sure that the naming convention strictly follows what was established as allowed in Openhab 2.x, namely that hyphens are OK. Even if changing my entire MQTT topology was not a prohibitive task, CamelCase would not be a replacement because it does not allow for capitalized acronyms… such as HTTPServer.

That’s not even looking at all the definitions properties, those are just the devices. Not all of them are current but a lot of them are.

I do see the code and agree with your assessment, my question is why this arbitrary limitation is being imposed – this was not an issue in openHAB 2.x.

Hey, here’s a thought, though.

Since there is a decode function…
Isn’t this a gui bug?
It’s perfectly fine for openHAB to be internally storing the strings any way it wishes (it could be using morse code for all I care) but when displaying it in the GUI shouldn’t it be going through the decode() function first?

liaskt · December 5, 2021, 7:25pm

I guess only the developers can explain the reasoning why the UIDs require a symmetric decode() function and why they didn’t choose a different delimiter in encode() so we could at least use underscore _ in our channel names.

I wouldn’t categorize this as a GUI bug. The issue is that you now have to use these weird channel names in other places such as links in .items files, in API calls etc. If the GUI displayed them in a different way than their internal representation it would probably cause more confusion.