I at least have come up with a number of lessons learned based on what happened with the MQTT 2 binding. I’m surprised this hasn’t happened before but I suppose it has been a long time since such a critical binding received a new 2.x version.
Some lessons I’ve learned and some suggestions that may make the transition a little smoother include:
Somehow make the binding available to some testers before it appears in the list in PaperUI or introduce some way to make it harder to find and/or make trying it out just a little more scary for new users. This will give the support users on this forum a chance to play with the binding first, prepare some transition postings and docs, test out how it’s used and find pitfalls that average users may fall into, etc. I’m less concerned about finding bugs (though that is important) and more concerned with giving us a chance to provide support so we can take some of the pressure off of the developer. I know @David_Graeff has some good ideas on how to address this.
Do not replace the 1.x version of the binding on upgrade even if they have show legacy 1.x bindings disabled. This is a breaking change that doesn’t have to be breaking, or certainly it doesn’t need to be quite so disruptive. I think we can and should find a less disruptive way to encourage the move to the new 2.x version of the binding. Maybe a warning in the logs and something at the bottom of PaperUi or the like.
This may not be remotely feasible, but what if there were a tool as part of the apt installer or openHABian or openhab-cli that scans through a user’s config/installed bundles and spits out the known breaking changes so a user knows what they are going to encounter when they attempt the upgrade. @Benjy, does this seem reasonable? One of the things users seem to hate the most is surprise. Maybe if we can avoid the surprise a little better people will be less angry.
I’ll bring over @idkpmiller’s good recommendation to create an Upgrade category in the forum. We can have the discussions in this thread and pre-populate it with tutorials and the like.
Thoughts? Ideas? @vzorglub, @Dim I know you might have some additional ideas.
Let’s say we all agree these or others are good ideas. How do we go about making them happen?
Actually, this is the most important part. But at the same time we should remove old tutorials or add a banner on top that a legacy binding is used. So that new users do not even attempt to use legacy bindings.
I suggest something even more detailed, possibly in the form of a form with text fields and checkboxes that would provide a uniform way to specify the system configuration that was used for a given configuration/tutorial, to include OH version & release date, binding version & release date.
I can remember being burned a number of times when I first started with OH because I didn’t yet know enough to discern what system configuration clues to look for, e.g., OH1 binding on an OH1 system vs. OH1 binding on an OH2 system, …
I can’t tell if you are agreeing with me or saying that removing the old binding is the important part. Ignore the following if you agree with me.
So this is how that played out from the user’s perspective for MQTT 2.
Hurrah! New OH Release!
Read the release notes. Hmmmm, a new MQTT 2.4 binding. I’ll have to look into that.
Look in breaking changes. Nope, none of my bindings are listed. I’m good to go! (NOTE, there is no mention ANYWHERE in the release notes that their MQTT 1 binding will be automatically replaced which IS a breaking change).
Upgrade is complete. Hey, all of my MQTT Items are broken! Where did my MQTT binding go!? It’s not even in the list to reinstall it!
Automatically replacing a working binding without giving the user’s enough information to know this could happen is HUGELY disruptive. User’s do not like surprises and this is a case where even a diligent user who doesn’t spend too much time on the forum would not have known was coming.
We can’t just swap out to the new 2.x binding like that. It really pisses off the users. Not every user is going to be in a position where they can immediately move to the new binding and to not give them the choice during the upgrade is IMHO kind of jerky move. This was exacerbated this time around by the fact that there isn’t anything in the release notes that I could find to tell users that this would happen.
Someday soon we will be in a place where this won’t be that big of a deal. But until then we need to make the transition easier for the users or we will have to face
every time.
I don’t like the idea of removing them, but a warning or banner across the top is a good idea. Though not everyone is that diligent about returning to old tutorials and postings they have made to provide this type of update. I don’t think we can rely on that happening reliably anywhere but in the docs. The Exec 1 binding does have a warning like this. We need to add a similar warning to the MQTT 1 binding README.
I’m not too worried about new users. By default show legacy 1.x bindings is turned off and in the docs you must check a box to even find the README. I think that may provide enough extra barriers in place to discourage new users from using the legacy bindings. I’m more concerned about the less technical or involved users becoming surprised and angry when all their stuff breaks.
There were some people this week, new to openHAB, that were saying this and that doesn’t work with my items file but I followed that tutorial to the word.
So as long as those old tutorials are not banner-ed even new users will use them.
There’s no way (AFAIK) to so prevent an installation based on user query. But the Linux packages as well as the manual packages already state breaking changes after the install.
I’m not sure if this is what you’re referring to, but I know HASS users have a nifty tool called a “Configuration Validator” that will run a scan through their config (YAML) and validate it against the schema, preventing them from restarting into a broken system. This might be harder to implement on openHAB, since our file-based configuration is less tightly defined, but I could see how such a validator could have picked up on the fact that the user’s .items file was using a MQTTv1 configuration for the item definitions, and warned them (at install time even) that this configuration was invalid per the MQTTv2 binding that was just installed.
That might’ve saved a few groans & growls and nasty forum posts
Maybe just something to compare versions of bundles installed to the list of breaking changes. It was just an idea that may not go anywhere. Thanks for looking into it.
There sure are a lot of tutorials that involve MQTT on the forum and note many of them are wikis. Do you have any idea or a plan for how one would go about adding the banner?
And of course, MQTT is just one such binding. What of all the rest.
I don’t disagree with the idea but I don’t see how to get there from here. But I don’t know everything so maybe there is a solution I’m not thinking of.
That wasn’t what I was thinking (see my response to Benjy above for what I was thinking) but something like that might fit the bill. My only problem with it is that is post facto. I won’t know there is a problem or that I have a ton of work to do until after I upgrade. My idea is to provide some way to give the users information prior to upgrading some clue about what they are in for during this upgrade.
I kind of stopped following the KNX postings. Did the problems ever settle down on their own or do you find a lot of KNX users still confusing KNX 1 and KNX 2 or trying to apply 1 configs to the 2 binding?
Maybe this is a problem that will peter out on its own and we just have to power through it until things settle.
Comparing the experiences of the two bindings can be informative.
In my opinion, most of the problems never surfaced due to several posts from the Snapshot era, prior to Go-Live of KNXv2 (one example: KNX1 to KNX2 Migration Steps).
It is (critically) important to have some tutorials/guides/docs/migration scenarios developed prior to Go-Live of a 2.x binding that replaces a 1.x legacy one (your arguments in your first bullet of the OP).
As the time passes, the forum will develop more of that, so the problems will be easier to handle (users will be able to find more info and solve the issues by themselves).
I think especially with the mqtt binding the users were quite surprised because (and that happened to me as well) the “upgrade system” in openhabian-config does not clearly tell that oh is upgraded as well here.
And the warnings during Upgrade for e.g. Z-Wave binding come way too late during the upgrade progress.
If the “upgrade system” section in openhabian-config would say something like “this will also upgrade openhab to release x.y. you are using binding xyz. Please read the compatibility notes carefully before upgrading: …”
