MQTT 2.x why the easy way when you can have it complicated

Add-ons Bindings
1

Hello

I use openhab for few years now i am on the way to update my system.

Openhab for me is a dream … i love the write my own configs / rules / scripts so i can integrate everything that i want to integrate … in the way i want :slight_smile:

So for me Openhab is a real flexible system where you can do nearly everything but you have to invest some (little) time to learn how if you dont wont to spend this time and have fancy simple clicking ui it will be better to buy some out of the box system …

At the moment i switch from Mqtt 1.x to Mqtt2.x binding and wonder why it get so complicated (to get fancy ui integration) and dont forward the clean&easy way.

What did i mean:

Get an Item to communicate over MQTT:

MQTT 1.x:

  1. Config the MQTT Broker once:
   mqtt1.url=tcp://localhost:1883
   mqtt1.clientId=openhab
   mqtt1.user=openhabian
   mqtt1.pwd=xxx
  1. Get the Item / Items / Things etc. MQTT ready:
Switch LAB1   "Light"   <light> (Licht, LAB) {mqtt=">[mqtt1:Licht:command:on:LAB1ON],>[mqtt1:Licht:command:off:LAB1OFF]"}

Done ! nice easy and clean in my point of view …

MQTT 2.x:

  1. Make a broker Thing (Bridge) (once):
Bridge mqtt:broker:WorkBroker "Work Broker" [ host="localhost", port="1883", secure=false, username="openhabian", password="ohmqtt", clientID="WORKOPENHAB24" ]
  1. Make Topic/Channel thing:
Thing mqtt:topic:WorkBroker:WorkSonoff "Work Sonoff" (mqtt:broker:WorkBroker)  {
    Channels:
        Type switch : WorkLight "Work Light" [ stateTopic="stat/worklight/POWER", commandTopic="cmnd/worklight/POWER" ]
        Type switch : WorkLightTele "Work Tele" [ stateTopic="tele/worklight/STATE", transformationPattern="JSONPATH:$.POWER" ]
}
  1. Make the item mqtt ready:
Switch Kitchen_Light "Kitchen Light" { channel="mqtt:topic:myUnsecureBroker:mything:lamp" }

So my questions:
Why i have to create a thing for channels when i only want to get a item to communicate over mqtt ? In the 1.x version it was only a one line code …

I dont want to expose the channel things in a sitemap for security why i have to / i am forced to ?

Example:

If i write the thing code like:

Thing mqtt:topic:WorkBroker:WorkSonoff "Work Sonoff" (mqtt:broker:WorkBroker) @ "mysitemap" {

The thing is exposed to the sitemap “mysitemap” this is ok.

When i write the code like:

Thing mqtt:topic:WorkBroker:WorkSonoff "Work Sonoff" (mqtt:broker:WorkBroker) {

The thing should not to be exposed to any sitemap but oh will create a new sitemap “Home” with the thing on it !?!?

I think this is realy bad … lets say i have a placeholder item this item should communicate over mqtt but not shown in a sitemap because i only use it in a rule/script as a trigger … why i should expose this to an sitemap ? or why i am forced to expose it ?

Thanks
Patrick

2 Likes
2

It’s not just UI integration. It also supports automatic discovery. And it follows the OH 2 architecture instead of the OH 1 architecture. That means Things and Channels, not binding configs on an Item.

Because in OH 2 the architecture is deliberately designed with new concepts to facilitate automatic discovery. Automatic discovery in not possible with the OH 1 way of creating a binding config on an Item.

Please read the Concepts section of the OH 2 docs for a full discussion. A tl;dr is each device is represented by a Thing. Each sensor reading or actuator on the Thing has a Channel. Channels get exposed to the rest of OH by linking them to an Item.

For many/most technologies this results in a significant simplification of setup and configuration. For MQTT, if you are working with devices/APIs that follow one of the supported standards (Home Assistant, Homie, Tasmota I think) then the MQTT Things will be automatically discovered and created for you. It’s only if you have to create the Generic MQTT Things yourself that it gets complicated.

If you don’t want to do this, then continue to use the MQTT 1 binding.

OH generates a sample sitemap that includes all the Channels that are Linked to an Item. It’s supposed to be a simplification for new users. I think automatically created things like this causes more problems than it solves.

But you really shouldn’t be relying on any automatically generated sitemap anyway. You should be writing your own sitemap with only those Items you want to be controlled. If you set your created sitemap as the default, I don’t think that the automatically created one gets advertised. At least I don’t see it listed anywhere in my setup.

If you are worried about the exposure yourself, realize that ALL your Items and ALL your Things are exposed through the REST API. OH 2 doesn’t have any authentication or authorization.

3

No. the @ "mysitemap" part is, where the Thing gets listed within Paper UI control. But Paper UI control is not to be used as an UI.
A sitemap is “items only”, you have to define all items that should be shown at the sitemap.

4

Except that there is an automatically generated sitemap that is really just the sitemap equivalent to the Control tab in PaperUI which is what I think OP is referring to.

5

Well, the automatically generated sitemap is named _default, I’m not aware of any use of the @ parameter in question of this sitemap. :slight_smile:

6

Sorry for the late answer … to much to do on the weekend :frowning:

I know the OH2 architecture and i think it is a good thing.

But in my point of view a thing is a collection of items or better a collection of functions that i will promote to the user why make a collection if i have only one item …

I know a single switch is also a thing in the oh2 architecture but i am a fan of the straight way why configure a thing and an item for only one bulb …

Also i think to offer both ways should not be that problem because when you configure a mqtt broker (brige) thing it should not matter if the item communicates over a seperate channel thing with the broker thing … or if the item communicates directly with the broker thing und a given channel in the item it self …

But yes this is my point of view …

The problem with the automatic generated sitemap for all mqqt things still exists.

Security was the wrong word lets say i have an item that i will not expose to or show the user because its only a placeholder item or trigger for an rule or script that not should be changed by an user.

With oh2 und mqtt 1.x binding it was no problem because the mqtt communication was done on the item.

With oh2 and mqtt 2.x binding a sitemap will be generated no matter if i write it with or without @ “mysitemap”

Here i write it without @:

1
1.JPG813×75 14.9 KB

If you go to Habmin you will sie the new automatic generated HOME Sitemap:

2
2.JPG805×485 24.5 KB

Next point is in the latest documentation of the mqtt 2.x binding all code examples
except for rules are removed … so all new users are “forced” to only use the ui !

All older documentation versions with code demos in it (2.3/2.3) are also removed you get an 404 error !

MQTT binding doku

Without a documentation there is no way to figure out how to do it codebased.

Normaly in oh the codebased sections was primary or lets say in the foreground because the most users want to go depper and write/implement their own stuff.

So i sadly see or suspect that oh slowly move away from a good codebased system where you can do the things your own way and implement “unknown” equipment by our self with simply write your own code (item/things/scripts/rules) (when you learn a litte bit) to an i dont want do know the background “userfriendly” klicking UI thing :frowning:

Thanks Patrick

1 Like
7

Please be aware that HABmin, although “the” UI in question of zwave, is outdated and deprecated. afaik it will be removed and so will the “automatic sitemaps”.
In fact, those sitemaps are unusable in most cases anyway, as the widgets do not fit (default gives a wrong widget). Please forget about HABmin for other use cases than zwave.

In question of removed old versions, I guess that’s an error. Please feel free to open an issue on github, so at least some of the maintainers should explain this step.

The plan for OH3 is, to get rid of different ways to store configuration, but to have the ability to configure in a text via UI. I’m pretty sure there will be also the option to do it remotely or with versioning (with git).

8

And it will continue to persist until someone makes a change to the code to either make the automatically generated sitemap no longer be generated or give one the parameters to configure what appears there.

Then you should define your own sitemap and present that to your user that doesn’t contain the Items you don’t want to be shown.

Correct because the @ is only used by PaperUI to tell it what tab the Item appears on in the Control tab.

That was a choice by the binding developer. You can file an issue on the binding to have it readded but I don’t have much confidence in successfully getting it restored. There are some good reasons why he remove them, mainly being that he was being held responsible for documenting things that the binding itself isn’t responsible for defining. Why should the MQTT binding have to document how Things are defined in .things files when that is all controlled elsewhere. Couple that with the fact that OH 3 will no longer support .things files using that syntax… I don’t necessarily agree with the decision but I’ve no power to force anyone to do what they don’t want to do.

MQTT 2 binding didn’t exist in OH 2.3 or 2.2. There never were any documents for those versions because there was no binding.

There are three threads with well over 300 postings each. This is NOT the direction that OH is going in. But the current syntax for the .things/.items et al are not sustainable and will go away, to be replaced with a different format and a different way of importing/exporting the configs to text files.

And PaperUI will also be completely replaced.There is already two pretty promising prototypes in existence.

I’m not positive that the automatic sitemaps are coming from Habmin. Though, like you note that one is called _default so maybe Habmin is creating things on it’s own.

There are plans to merge the zwave stuff into PaperUI (or whatever replaces it) so that Habmin can finally be laid to rest.

Absolutely. And there will be a way to migrate from one approach to the other without needing to rewrite everything from scratch.

1 Like
9

I have moved them to a separate file, that is linked from the main document. Unfortunately the website generator doesn’t support that yet and stripped all graphics and links. See also Define more files than just readme.md for addons · Issue #678 · openhab/openhab-core · GitHub.

The reason is simple: Textual configuration via xtend files is only one way of using openHAB. And it is quite frankly not my preferred way. For all my bindings and reviews of new bindings and bindings that I migrate to the new buildsystem I’m moving those xtend examples to a separate file.

The original idea of the documentation/add-on page was to introduce the product and the features of the product that the bindings supports. The xtend text examples were not intended to dominate these files.

A think can be seen as a collection of Channels. Thinks have nothing to do with Items in this architecture. You can even compile an openHAB that has no Items at all and is purely Things and Channels. Items are your variables, the state keeping objects in this concept.

10

Hmmmm. There is a link on the Zwave docs to a sub-page. I wonder how Chris does it.

I think this is a fantastic idea. :+1:

It appears that it only shows those Channels that are linked to an Item. Channels withoutlLinks are not shown, at least the last time I looked at it. Otherwise there would be dozens of entries just for a single Astro Thing.

11

Thanks to everyone !

I just wanted to show points that could be better solved (From my viewpoint)

  • Let items directly communicate with the mqtt broker thing.
  • Get rid of the automatic sitemaps

I dont want do start a principled discussion *Sorry for that !

I have my own sitemaps and yes it is no “problem” that now i have to write topic/channel thing and ignore the automatic generated sitemap.

But i think have more ways open (see points above) is never bad …

I also cant say much to Paper Ui / Habmin etc. because normaly i dont use them very often. (Just write my config in the config files)

@ OH3
I am lucky that i only use 2.x Bindings (now mqtt 2 also) and all my self written not officialy supported bindings are also 2.x compatible.

The decision removing the compatibility layer will prevent many users from using the new openhab or they will switch to other systems.

Backward compatibility is ever a big big point !

Ok i agree it is very bad that some binding providers dont take care of their bindings …

But what should do a simple user without the knowledge to write a new one but have 1.x bindings in use ?

It will cost OH much device compatibility and so the users that use this devices.

I also agree that ui and code configs should be at the same place !

But please dont force the users to write the code in the UI it will be a pain :frowning:

Just make a export/import function or something like that.

Thanks Patrick

1 Like
12

Everything after this point has been extensively discussed in other threads and are off topic here.

All I can say is if you think removing the compatibility layer is a mistake, are you willing to step up and maintain it? It would be fantastic if we could keep it but if no one is willing to maintain it we can’t.

Run an instance of OH1 or OH2 and federate that with your OH3 instance.

There will be detailed tutorials written.

No one is. The replacement to PaperUI will be made such that for those who want to it’s much less painful. But there will also be an import/export function so you can use external text configs if desired.

I only respond because I don’t want future readers who missed the lengthy threads from being mislead.

14

I totally agree with that !

You’re right !
So i made a post at the OH3 topic …
Post on OH3 topic

15

It’s not misleading. None of those user who see value in it have stepped up and said “I’ll do it!” None of them have submitted a PR. None have sent a PM to Kai asking “how can I help?” The last time things went awry with the compatibility layer no one helped Kai solve the problem.

Kai doesn’t want to maintain it any more. No one has volunteered to take over the work. If you decide to take it on (I completely understand why you wouldn’t and don’t blame you for not), or find someone who is willing to then my point stands.

There may be all these people who find value in it, but none of them are stepping up to actually do the work to save it. There are only complaints.

So to use your own argument, no one seems to value the compatibility layer. No one seems willing to make the changes to the core and/or the bindings to make them natively supported instead of through the compatibility layer. So how much do they really value it?

I’m not saying that. I’m saying just the opposite. It’s an open source project. Anyone can contribute. No one has volunteered to do so for this. No one is stopping it. I have little doubt that if someone volunteered to make OH 1 bindings actually work in OH 3 the efforts would be accepted and appreciated. I eagerly await the PRs. I’ll champion the changes if needed.

I’ve said this or something like this, sometimes rudely, but the answer is always crickets. Lots of people want to complain and insist how big of a mistake is being made. No one is stepping up to do the work to move in some other direction.

In my personal opinion, I still think removing the compatibility layer is a mistake. I think abandoning the OH 1.x bindings in OH 3 is a mistake. I’m ambivalent about the differences between MQTT 1 and MQTT 2 bindings. There are things I like about both of them.

But I also see that we can’t force anyone to do work on an FOSS project that they are not willing to work on. None of the current developers are willing to work on this. So someone needs to step up and work on it, or we have to accept that the value people see in the OH 1.x bindings must not be that high or else we would have at least one developer opening issues and submitting PRs to preserve them.

3 Likes
16

As I use openHAB since version 1.0:

I saw much progress for openHAB in question of stability and variety (i.e. number of bindings).
Then, with openHAB2, there was a complete redesign in how to work with openhAB. ok, benefit was autodiscovery, really nice feature.
Downside was, there was a certain amount of bindings which did not work as reliable as in openHAB 1.x.

I had to wait more than two years for an (almost [!]) working replacement for knx1.

Meanwhile, openHAB was split in two pieces with a really huge amount of work and stagnation in progress.
The Backend was changed to karaf (can’t even remember the name of the old OSGI Console)
Some time later, complete redesign for documentation.

For the record: knx is one of the really important bindings, at least in europe.
knx2 is in a current version (OH2.5 #1502) and there is most certainly a bug which causes connection loss. Although knx2 should reconnect, this never happens, one has to restart the bindinng to force reconnect. But as this bug is not solid replicable…

Now, there is the re merge of openHAB and Eclipse Smarthome, and even more, a complete change to another build system.
So no time to improve bindings…

In fact, I’m still using openHAB1.8.0 for knx, as it’s rock stable, uptime for more than two months is completely normal, I have only to restart the system if changing the configuration. Restart duration is less than half the time as for openHAB2.5 (even if installing no binding at all)

It’s very sad, but I doubt that openHAB 3 will do it better :frowning:

I’m willing to accept changes, but it’s really frustrating to see no progress in question of stability and variety, but losing of features.

And altough I did some programming in the past, I have to admit that I don’t understand how all this stuff works, when reading the source code (yes, sometimes I try to figure out something) I’m absolutely sure I can’t help the slightest bit.

6 Likes
17

By now I have read almost all parts of openHAB source code. There is a big consistency problem with parts that are OH1 (the entire item management for instance) and newer parts.

Everything tries to go via the eventbus. But not as native objects but as strings. Those strings are at different points within core intercepted and parsed to objects again. Absolutely inefficient.

The usage of xtend which was initially only for items and sitemaps files was extended to everything else. But that is a super slow solution when it comes to initial startup and reindexing on file change.

Karaf is fun to have, but it provides no feedback during feature installation apart from “I have started” and “I’m done”. That’s not enough if you have half a dozen different dependencies for a feature and an unstable internet connection.

Using XML for binding and thing descriptions feels like the 90ties. It eats up start up time as well.

The build system before the change was ancient and very very eclipse technology specific.

But there is light. The new build system does a really good job. I have an XML less implementation for binding development in proposal and xtend hopefully gets entirely optional in the future (I have removed it in my personal setup). Startup time including new gen rules and about 100 items is 5 seconds.

The binding developer documentation is very corporative (coming from Esh) and I blame it for having not so well written OH2 bindings or instabilities like with knx2.

For example it recommends to use the core thread scheduler pool for every thread related task. But at the same time there are only 10 default threads in that pool. Those are used up fast especially with background discoveries and the core will feel unresponsive from time to time. In OH1 every binding would just spawn its own threads.

I hope that I come to write a better binding developer documentation (a real story from a to z) after the migration.

My thoughts so far.

Cheers,
David

5 Likes
18

It was Eclipse IIRC.

19

Another way of looking at it: Some group of people made the 1.x architecture and individual bindings, and never assumed the odd possibility that developers would become so bored of what they had created that they would take it all away from the users. I sympathise with the users, because their assumption that functionality — good, great functionality — would be taken away from them, is a completely reasonable assumption.

It’s a peculiar supposition that continuing to support existing functionality is the burden of previous developers, but that current developers have full licence to break existing functionality without full, honest, complete, compatible replacements. What does such a model mean for actual users?

Great! We agree.

Well of course, but I think this is a false argument presented by others in bad faith. If this project is truly open and provides true value to users, then any attempt to destroy existing functionality is being promoted by people who are actively disinterested in users, and more interested in how this project represents a personal hobby horse.

There are things that are distinct and valuable about both approaches, but absolutely not one at the expense of the other. If the 2.x binding architecture had fully encompassed all the value of the 1.x architecture, that would have been great, but it didn’t. The 2.x added autodiscovery, but actually lost a number of strengths of the 1.x model, and as a consequence it is critically important that both models continue. To apply religious zeal instead of simple product management will be to kill off the relevance of the whole effort.

This is a serious problem, where developers would rather see the product enter limited or no relevance instead of address its historical strengths. You might want to have a talk with “current developers” to help support the project’s future viability by being less religious and more pragmatic.

20

Would you have someone dictate what developers are allowed to work on with their freely volunteered time?

Once someone contributes to an open source project, are they bound to now and forever support that code now and forever?

From what I’ve seen, with the exception of David, it’s more a plea for help. There are too few developers contributing to the project. Cutting out the functionality, as you call it, is a desperate move to reduce the work load down to what can be managed by the current set of developers. Perhaps some bad decisions were made and choices were made in the architecture of OH 2 but we can play the blame game and harbor resentment or we can figure out a way forward. I honestly believe it is a matter of abandoning the OH 1.x bindings in OH 3 or abandoning OH in its entirety. Sure, it would be a slow death, but it would be a death none the less.

They are aware of my opinions and I am aware of theirs. Perhaps that is why I see that these moves are not driven by religious zeal so much as a desperate need to reduce the work load. I’ll still maintain that the only way to preserve the bindings is for more developers to step up and contribute. Otherwise OH has to shrink in some ways to reduce the work load.