openHAB reliability and upgrade experience

[tcgerhard]

@rlkoshak

The docs are complete and make perfect sense to the one who wrote them. When a user comes along and doesn’t understand them or interprets them in different and unexpected ways it is because they are trying to do something and failing. The docs are not helping them.

You’ve nailed it. I’ve a few times decided that the thing I was struggling with could use better documentation and I was going to draft something and see how it goes. I generally run into either feeling like I don’t have a firm enough grasp to do an effective job, or, the information that I found hard to discover is in an unexpected-to-me place, and writing what I had in mind would create redundancy and probably a maintenance problem. Usually, my attention span wanes when I accomplish whatever I set out to do, and I end up moving on to something else. Next time I’m in that scenario, I’ll try to get further along.

And to add, more often than not, I’m searching this forum for what I can’t sort out in the docs, and when that leads me to a solution, there’s a solid chance that you and a handful of others are the author of the post that got me there. @digitaldan and @nadahar come to mind, and I know there are others. I’m planning to do my persistence audit and take a run at upgrading to 5.1 when I have the house to myself for a couple days… maybe that’ll trigger something.

[Nadahar (Nadar)]

@Nadahar

@rlkoshak

This was recently added to the docs.

Bundle Management | openHAB

That’s really useful, but there’s one thing lacking that I wish was there:

bundle:list -l 156

When using the -l argument, it will list the “location”, which is the “source address” of the current installed bundle, e.g.:

> 156 │ Active │  80 │ 5.1.0.202512050249    │ mvn:org.openhab.core.bundles/org.openhab.core/5.1.0-SNAPSHOT

By copy/pasting this before making a change, it’s very easy to restore the original bundle, because that “address” is all that’s needed as an argument for bundle:install/bunde:update.

I am currently trying to juggle way more oranges than I can handle, so I just can’t do this now - at the same time, I’m pretty sure that this will be forgotten. If somebody wants to do a little bit of contributing, this would be a great opportunity

The one think to keep in mind here is that the location/“address” replaces the bundle description that you usually use to grep to find the right bundle. The documentation should therefore make sure that the bundle ID is “figured out” beforehand, and then bundle:list -l <bundle ID> is used to find the location/“address”. You can run bundle:list -l without a bundle ID, but you will just get a lot of bundles listed that you don’t know what is. Try it out in the Karaf console yourself, and you’ll see what I mean.

[Spaceman_Spiff (Sebastian)]

@Mherwege

The proper procedure is to first make a backup, then run the upgrade using the proper script.. If it doesn’t work out, restore your backup.

The backup is the old complete installation which is just shut down.

@Mherwege

And of course, when copying over cache and tmp have not been cleared, often leading to issues as well, and leaving the impression you need to do that manually every time something goes wrong. It is a last resort. In normal situations where it is needed, it is done for you when doing it the way it was intended.

But since this data resides in userdata it’s really implied that it needs to be copied over.
The thought process is: I don’t want to loose my userdata - better copy that.

@Mherwege

For file based configurations, there never is an automatic upgrade.

That implies I shouldn’t need to run the upgrade tool at all if I use file based configuration.

[rlkoshak (Rich Koshak)]

@tcgerhard

I’ll try to get further along.

And all for help if you need it. All of my significant contributions I’ve made to the docs I first posted to the forum and asked for review. But just beware that you are not obligated to take every suggestion.

@Nadahar

at the same time, I’m pretty sure that this will be forgotten

There’s already a pr open working in this party if the docs. I’ve suggested this be added as part of that pr. If it doesn’t, I’ll add it after that pr gets merged.

@Spaceman_Spiff

That implies I shouldn’t need to run the upgrade tool at all if I use file based configuration.

For clarity, the upgrade script does more than invoke the upgradeTool. Running upgradeTool is just one thing among several that the script does. It’s important not to cooperate the two.

Just because you don’t need to run upgradeTool because you have an all text for based config doesn’t mean you didn’t need to run the upgrade script.

[Nadahar (Nadar)]

@rlkoshak

There’s already a pr open working in this party if the docs. I’ve suggested this be added as part of that pr. If it doesn’t, I’ll add it after that pr gets merged.

@rlkoshak

Just because you don’t need to run upgradeTool because you have an all text for based config doesn’t mean you didn’t need to run the upgrade script.

But, the upgrade script is for package managed versions only? Or Linux only? Or does it apply to every installation?

[Spaceman_Spiff (Sebastian)]

@Nadahar

Or does it apply to every installation?

It seems my installation doesn’t have it:

It’s just start and start_debug.
Searching for upgrade yields only the upgradetool.jar

---

@rlkoshak

Just because you don’t need to run upgradeTool because you have an all text for based config doesn’t mean you didn’t need to run the upgrade script.

Then this information should be bold in bright letters on the openHAB download page.
“If you copy old userdata you need to run the update script!”
Because I was not aware of it and I suspect many users aren’t.

---

Nonetheless:
I still find it a questionable choice and an extremely bad user experience.

I’ve personally fallen into this trap more than once and I helped multiple people out with the same issue.

[sihui]

@Spaceman_Spiff

Searching for upgrade yields only the upgradetool.jar

Should be in runtime/bin/:
update

[Nadahar (Nadar)]

I find an update.bat in a Windows installation that looks like this:

@ECHO off
SETLOCAL

IF "%1"=="?" GOTO printArgs
IF "%1"=="\?" GOTO printArgs
IF "%1"=="/?" GOTO printArgs

SET uargs=
IF NOT [%1]==[] ( SET uargs=%uargs% -OHVersion %~1% )

CD %~dp0
powershell -ExecutionPolicy Bypass -command "& { . .\update.ps1; Update-openHAB %uargs% }"
SET LEVEL=%ERRORLEVEL%

if %LEVEL% LSS 0 (
    PAUSE
    EXIT /B %LEVEL%
)
EXIT /B 0

:printArgs
ECHO Usage: update.bat {OHVersion}
ECHO OHVersion (required) - The version you want to update (4.0, 4.1, etc)
ECHO.
ECHO Example to update to openHAB version 4.0.0 stable:
ECHO    update.bat 4.0.0
ECHO.
ECHO Example to update to openHAB version 4.0.0 snapshot:
ECHO    update.bat 4.0.0-SNAPSHOT
ECHO.
ECHO Example to update to openHAB version 4.0.0 milestone 1:
ECHO    update.bat 4.0.0-M1
ECHO.
EXIT /B -1

However, it’s very unclear to me what it actually does. From the sound of the “documentation” in the script, it seems to update OH itself - but that can’t be.

edit: Looking in update.ps1, it does actually appear to try to download a new version and do an update. I certainly wouldn’t dare to run this on a production system - but in any case, it seems to be something different from just something that does some magic to the userdata/configuration.

[rlkoshak (Rich Koshak)]

@Nadahar

Or does it apply to every installation?

Every manual installation. For Windows there’s a .bat version of the script.

For package installed, openHABian, Homebrew (I assume) and Docker, the script is invoked automatically. For manual installs one needs to invoke it manually. It’s a manual install after all.

This isn’t something new. It’s worked like this since at least OH 2.0. I can’t remember how OH 1 handled it, it may go back even further. The upgrade script was introduced in OH 2.1 and the upgrade sections for all of the supported platforms where it’s needed has included invoking the script as a step for manual installs at least since OH 2.5.

I try not to be a RTFM type of person, but if you are using a manual installation method and don’t read the manual upgrade instructions I don’t have a whole lot of sympathy.

That’s not too say I don’t think it could be done better. It indeed would be great if even manual installations didn’t need to run this script manually.

But if you are surprised by this behavior at this late stage, I don’t have a whole lot of sympathy.

It’s been discussed in countless threads over the years.

@Spaceman_Spiff

Searching for upgrade yields only the upgradetool.jar

It should be in runtime/bin.

@Spaceman_Spiff

Then this information should be bold in bright letters on the openHAB download page.

There is a while section discussing it is each on the installation docs. But I’m sure aprc adding it to the download page would be accepted. The challenge is, like I said above, you only really need to worry about it if you have a manual installation. All the package managers do this step for you during the upgrade process.

@Nadahar

From the sound of the “documentation” in the script, it seems to update OH itself - but that can’t be.

Because Windows only really supports manual installations (Chocolatey was supported for awhile but I think it was abandoned), the Windows scripts always did a little more and actually downloaded the latest zip file and unzipped it for you instead of requiring one to download the zip file yourself.

But I thought that was a powershell script, not a bat file, and that the PS1 script involved the bat file.

@Nadahar

something that does some magic to the userdata/configuration.

To be clear, the sorts of things the upgrade script does is clear the cache and update some of the files in usersata/etc as necessary in addition to involving upgradeTool. Nothing more.

[sihui]

@Nadahar

I find an update.bat in a Windows installation

An update file is also in a manual linux installation and it works fine.
I started using it with 5.x. and had no problems so far.
It updates the version and it checks my jsondb.

[Nadahar (Nadar)]

@rlkoshak

But I thought that was a powershell script, not a bat file, and that the PS1 script involved the bat file.

From what I can see, the BAT file just launches the PS1 file - which means that it will probably fail if run from a command prompt instead of powershell. But, regardless, most use powershell I assume.

I don’t mean to dig into the Windows installation here, that wasn’t the point, I just wanted to know if running the update script is a universal operation. From my browsing of the powershell script, it doesn’t quite seem like there’s an option to just do the cache clearing and other file operations without downloading/extracting - but perhaps if you run it with the currently installed version as an argument.

If you don’t use the script like that, but makes a new installation yourself and then move over configuration/data, I don’t know exactly what would be the right procedure to “apply the update script” to get all the other magic to take place on the data you just moved. But, I didn’t read the documentation for a manual installation either, maybe it’s all explained there.

What I do see is a file userdata_sysfiles.lst, which is a list of files that I assume are either to be deleted or not deleted. The necessity of this file existing perhaps shows exactly why it’s problematic that files are “mixed” they way they are.

edit: The documentation doesn’t really say much about how to use it at all, except to update to a new version using the script to do everything.

[Spaceman_Spiff (Sebastian)]

@rlkoshak

I try not to be a RTFM type of person, but if you are using a manual installation method and don’t read the manual upgrade instructions I don’t have a whole lot of sympathy.

You must have misread my earlier posts:
I am not upgrading an installation!
I want to copy data from an existing installation into a new installation keeping the old installation with data in place.
And that is currently very hard.

But as it seems I am using openHAB wrong (“you’re holding it wrong!”) everything should better stay as it is.
Where would we end up if things simply worked with copy and paste?
Preposterous!!

Someone should also fix the windows docs because they are definately lying to me:

[florian-h05 (Florian H.)]

@Spaceman_Spiff

Then this information should be bold in bright letters on the openHAB download page.
“If you copy old userdata you need to run the update script!”

The Linux docs for manual installation tell you how to upgrade manual unstallations:

From version 2.1.0, openHAB is distributed with the update script included. This script should be called from within openHAB’s root directory. Assuming the openHAB directory is in /opt/openhab, simply run the following commands to update to the next major version of openHAB:

cd /opt/openhab
sudo runtime/bin/update

(openHAB on Linux | openHAB)

This update script also runs the upgradetool: openhab-distro/distributions/openhab/src/main/resources/bin/update at c603838d425e8ed83501f2d871a9922474da9f2d · openhab/openhab-distro · GitHub

@Spaceman_Spiff

I am not upgrading an installation!
I want to copy data from an existing installation into a new installation keeping the old installation with data in place.
And that is currently very hard.

If you copy data from installation A to installation B and B is newer you are doing an upgrade, you just keep the old version in place. => You need to run the update script.

@Spaceman_Spiff

Someone should also fix the windows docs because they are definately lying to me:

There’s an “Edit this page on GitHub” button down below that page …

[Spaceman_Spiff (Sebastian)]

@florian-h05

If you copy data from installation A to installation B and B is newer you are doing an upgrade, you just keep the old version in place. => You need to run the update script.

But I am only copying user configuration. That’s not semantically an upgrade.
That’s a migration or an import at best.

The fact that there are multiple workarounds to this shows that this is and has been identified as a problem. The docker image for example is much more user friendly in this regard because it allows just this: copy paste of configuration and userdata.
But why do the docker image and and the distro both implement a solution which also differs in details (and the distro differs between linux and windows)?
Shouldn’t there be a unified solution implemented exactly once?

And maybe the solution shouldn’t be yet another tool but rather move the cache data out of userdata and make e.g. it version dependent.

@florian-h05

There’s an “Edit this page on GitHub” button down below that page …

“Something is very hard to achieve and behaves unexpected.”
"You should read the docks - dummy!
“The docs say this should work.”
“Then you better edit them so they don’t say this any more!”

[hmerk (Hans-Jörg Merk)]

@Spaceman_Spiff

Something is very hard to achieve and behaves unexpected.”
"You should read the docks - dummy!
“The docs say this should work.”
“Then you better edit them so they don’t say this any more!”

This is an absolute inapropriate comment.

@florian-h05 answered to your post which was about someone needs to change the docs.

You can be that someone yourself !

[florian-h05 (Florian H.)]

@Spaceman_Spiff

The docker image for example is much more user friendly in this regard because it allows just this: copy paste of configuration and userdata.

The logic from the Docker image may be reused by the Distro Linux script, I think both use Unixoid shell scripts for that.
Porting that to windows might be difficult though due to all its shell differences, and Windows being quite unpopular against developers for running applications.

[Spaceman_Spiff (Sebastian)]

@hmerk

This is an absolute inapropriate comment.

I disagree. That was an exaggerated summary of excerpts of the posts by @rlkoshak and @florian-h05

@hmerk

You can be that someone yourself !

I’ve already created a PR before you made that post. What else do you want me to do?

@florian-h05

Porting that to windows might be difficult though due to all its shell differences, and Windows being quite unpopular against developers for running applications.

Maybe instead of these scripts a launcher application which does these things might be the better solution?
The launcher could also serve the webpage and inform the user.

[hmerk (Hans-Jörg Merk)]

@Spaceman_Spiff

I disagree. That was an exaggerated summary of excerpts of the posts by @rlkoshak and @florian-h05

You might disagree, but I and perhaps others too understood it different, therefore my comment !

[florian-h05 (Florian H.)]

@Spaceman_Spiff

Maybe instead of these scripts a launcher application which does these things might be the better solution?
The launcher could also serve the webpage and inform the user.

In which language? You have to deal with permissions, file ownership, moving files etc. That’s a good fit for scripting IMO.

[ErikDB (Erik)]

Maybe this is another subbranch, but one ‘user experience issue’ I have with openHAB, is that I only notice something is broken, when I actually experience some expected end result not working.

I recently added an ‘error overview’, with which I hope to be better able to intercept problems.

But having something out of the box would probably help a lot of users. Maybe an option to get notifications would be an even bigger win.

That way, a lot more problems will be noticed, hopefully reported and even more hopefully fixed. And - again hopefully - that would reduce frustrations because some real world thing unexpectedly doesn’t work.