How shall that work with a home automation? A second RPi just for testing? What devices (things) to use? Shall I run a bunch of e.g. Shelly’s just for a test environment? What shall they do, especially on metering? I have no second home for real testing. All devices (things) I have in my home automation are there for a purpose, not just for fun and we like to rely on it …
Only thing I can do is trying to follow what is going to be changed and then decide on date of release to upgrade or not. This time I missed obviously something changed on / for Shelly binding, whatever it is … on other OH releases in the past I did wait a while, realizing changes on bindings I use could cause some work to my setup.
This is actually a very good point in my view, and I’m grateful to those that do test snapshots, but I think we’ve seen time and time again, that it’s really way too few for us to rely on that the lack of negative feedback means that all is OK. In this particular case, the problem has been reported, and it has been worked on for a while, but wasn’t ready in time for 5.2.0.
More generally, it should be said that you don’t need a second RPi for testing. You can run a test installation at a computer, and you can copy over “most” of the configuration without that leading to problems. What you don’t want to copy over is rules etc. that sends commands in response to certain conditions, that’s not nice to run in parallel. But, everything that just “passively listens”, shouldn’t cause issues if two installations are run in parallel.
This is what I do when I want to test something, and in the rare case where I need to be able to do things that won’t work in parallel, I’ll stop a particular Think at the production system for a while, and enable it on the test system, letting me test it that way.
But, it’s not perfect, and it’s not without some effort.
I make sure I have a a backup and upgrade to the latest snapshot.
If something breaks I gather information, report it to the forum and/or GitHub, and revert back to the previously working version. If it works, I let it run until the next upgrade. It doesn’t take much more than that.
I do this frequently when I have the time. When I don’t I stick to the milestones.
No second installation required. No second set of hardware. Nothing fancy like a staged testing and production environment or separate hardware or anything like that. All you need is the ability to quickly revert back to a working version quickly if something breaks. You do not necessarily need more than this, and being able to quickly revert to a working version is something you should have in place anyway.
This would be the minimial setup required to test. Of course there are all sorts of more complicated ways you can test too.
But no matter how it’s done, if users do not help test the software stuff is going to get past the testing and into the release. Each and every instance of OH that is running in the world is different with a unique configuration. If you don’t help with the testing of your unique configuration, there is a higher likelihood that something will break when you upgrade because you are the only person in the world who has your unique openHAB configuration.
This is not the developers being lazy. They don’t want stuff to break any more than you do. But there are more ways to configure openHAB with combinations of add-ons, not even considering UI configurations, Things, and Rules, than there are stars in the universe. OH only has a few dozen active contributors at any given time. Without a lot more users running at least the milestones and reporting problems issues and regressions will forever be slipping through.
And I will note that I am personally going through this right now. I really didn’t have time to keep up with all the developments of OH that went into OH 5.2 and it turns out one change that I didn’t follow ended up really being a breaking change for me. I’m not mad about it. It’s my fault. In fact I knew there was potentially a problem when I saw exceptions in the logs one day, but I never had time to look into it and I assumed it was a problem I caused, not a change in OH. Once I finally did have time, 5.2 was already released. But this isn’t the developer’s fault. They didn’t know because no one reported it.
And they are currently at work to try to make this not a breaking change in the end. So it’s all good. But if I were able to test and report before, or others were testing and reporting, this could have been handled before the 5.2 release.
Interesting. I’m currently on vacation; when I’m back, I’ll take a look at how this reverting works.
I assume you actively scan the (all) logs when this Snapshot version is running?
Maybe it’s not a bad idea to have a dedicated tutorial topic about this, or even provide something about it in the docs? I assume the former is less work. (Maybe this post could be ‘extracted’ to a new topic.)
If you are on Docker, it’s super easy. Just take a copy of your openHAB folders and upgrade the container. If it doesn’t work, downgrade the container (I recommend using Docker Compose for that).
Restore the original contents of the openHAB folders just to be safe against the case that something changed (e.g. upgrade tool).
The procedure is completely different depending on how you installed OH. It wouldn’t one tutorial, with l it would be somewhere north of six:
openHABian with SD card mirroring
openHANian with Amanda
openHABian without SD card mirroring
apt, yum, etc installs
manual install on Windows
manual install on OSX
manual install on Linux
Docker
Synology
FreeBSD
And I think I’ve forgotten one or two.
These will all be different. But for any system you can’t afford to be down for extended periods of time you must come up with a recovery plan. That’s completely independent of testing.
For openHABian without Amanda or SD card mirroring, at a high level the steps would be:
run openhab-cli backup and put the backup somewhere safe
in openhabian-conf choose the version to install between milestone or snapshot. If you want a specific version you need to set the version to install from the command line using sudo apt openHAB hold <version>
sudo apt update && sufdo apt upgrade or upgrade through openhabian-config
Watch the logs for errors
To revert:
stop openHAB
restore the backup you took in step I using openhab-cli
hold the atp package version to the version that the backup was taken from
run apt upgrade again to go back to the working version.
The steps are always the same: backup, upgrade. To restore it’s always restore backup and downgrade. But the specifics and paths are always different based on how and where OH is installed.
Now this is not specific to openHAB. You should know how to do this for any service which you cannot afford to be offline for extended periods of time.
What to look for in the logs? You know what you’re logs usually look like better than I do. Look for errors and anything out of the ordinary.
What do I do? I run in Docker and I deploy everything using Ansible. My playbooks detect when the Docker imaged has changed and automatically takes a restic backup (restic is my current backup solution of choice). Then it starts the new container.
If I have problems, I’ll stop OH, move the folders OH is using to an archive and run my playbooks again, after setting the specific older version of OH I want to revert to. Now they playbook sees that there isn’t already an openHAB home folder so it restores the most recent backup before pulling the specified docker image and running the older version of OH that worked.
It’s basically the same thing @florian-h05 describes except automated using Ansible.
I still think this is an area where some kind of documentation would be very helpful. Even if you can’t explain the exact commands that must be run, there should be an explanation of the steps. I think many users get into “jams” with this because they just can’t figure out what they are supposed to do.
It doesn’t really help to say that it’s “their responsibility”. Sure, you can argue that, but if they can’t figure it out, what should they do then? Abandon OH?
I honestly don’t remember what is safe and not to do myself, and I find that piecing this information together by searching the forum is tricky, it’s hard to know if the information you find is accurate, still valid, or incomplete. And, it’s hard to know what to search for. This is one of the reasons why I simply don’t touch my production instance unless I have to.
I do these things too seldom to remember all the “rules” for what is and isn’t safe to do, so if I knew that there was some place I could consult and get the information I needed when I needed it, the threshold for doing it would be much lower.
I think that if somebody could find the time/energy to put something like this together, it could be very valuable, and prevent a lot of frustration and other “situations” users get themselves into.
This is documented in many of the installation instructions already. I’ve not read them all so can’t say if it is documented in them all, but several do. It was definitely in openHABian’s docs last time I looked.
It amazes me how few search the docs first instead of searching the forum our asking AI first.
I find that the docs search often doesn’t get me the information I want. It’s hard to guess the correct term to search.
Asking AI is an option, but might lead you down the wrong path as easily as the right one. So, you can’t really trust that the answer keeps you safe.
The backup section in openHABian is very difficult to follow, and I doubt very many know what Amanda is doing. I have it running, but I fear that if I ever need to retrieve something from there, that in itself will be a big undertaking, and certainly not something I’m willing to dive into just to try an upgrade.
I was thinking more of doing some kind of “backup” outside the normal backup operations, that you can easily reapply if things go wrong. You’d need to know exactly what to back up, how to safely copy all the information you need, and how to reapply it after downgrading again.
IMHO using docker for testing (even if not how you run OH) is the simplest for testing with only one device. My only variation is that I copy the current OH files (addons, userdata, conf) into another folder (openhab-test) with the docker-compose.yml. When you have a WAF window of an hour, shutdown the stable (docker compose down, sudo systemctl stop openhab, or whatever) and start openhab-test with docker compose up -d. An hour gives you time for a restart and to check out the logs. Then shut it down and restart your production system. That’s about 5-6 hours over 6 months, plus maybe another 5 hours writing/reading forum on the milestone topic.
The RAM overhead of docker (not running) is less than 100 MB on my trixie OS Pi. With production down and test up total RAM should be about the same. Also delete the test folders if you wish as you will want a new copy for the next milestone.
EDIT: not a recent issue, but docker comes with all the prerequisites (including java version), so if that changed between stable and milestones, no issue.
For those not familiar with the nomenclature, like me, I think WAF is Wife Acceptance Factor. That might not be that large for everybody, so I’m questioning if WAP (Wife Absence Period) might be more suitable.
I just don’t like Docker - I don’t trust it (not in an “evil sense”, just that I’m not sure I know exactly what it will do). I have a Docker instance on my production system for firing up the ZWave JS thing if I need it, but docker is otherwise thoroughly disabled when I don’t need it. Because I don’t know what it will do “on its own”. I think the way you interface with Docker is quite terrible, the CLI commands make no sense to me, and autocomplete seem to be lacking (as far as I can remember). Only the fact that you have two different ways to do things with our without compose, the fact that they keep breaking the compose syntax etc. makes for a terrible user experience, unless you use it regularly. I don’t, which makes using too cumbersome. If you use it regularly, you know all these things, and I’m sure don’t even notice the inconvenience and inconsistency.
My impression is that many run OH using openHABian without Docker, so there should be an easy, safe, way for them to test as well.
I don’t feel the same way about Docker. Yes there are issues, but as regular they are minor to me. I opted for the CLI as a minimum impact (basically about 5-10 commands to learn (and stay current with ). Basically, I burned the PiLite O/S image and added Docker. The only thing I added on metal was Samba. Once you start altering metal apps during testing, you open the whole backup/restore quagmire. I just get any feedback on the milestone for the maintainers and delete the container/files.
as well as in the Linux and openHABian docs. On the other platforms without openhab-cli (Windows, Docker, etc) there is a backup.sh and backup.bat in the $OH_HOME/runtime/bin folder (note the scripts are the same ones openhab-cli calls. This is discussed in the various installation docs (e.g Windows | openHAB).
sudo openhab-cli restore <path to backup file>
Or if you do SD card mirroring, stop the Pi, remove the SD card and swap it with the mirror and boot.
Pretty much nothing. It’s like an old Unix chroot only it isolates more than just the file system and there’s an api built in. But if you want containers with out the daemon and extra stuff use Portainer. All the isolation, no daemon, and there’s a web based UI if you want it. Or don’t use either at all. No one here is trying to convince you personally to do so.
All the links I’ve posted so far where found by searching the term “backup”.The term “restore” returns the same links.
That might be in this situation, but there is something iffy with the docs search. I searched for “thread dump” a couple of hours ago, to find the docs for how to make one. It just lists stuff that’s completely irrelevant, so I have up and had to browse my way there from memory and the side menu. And, this isn’t the first time. So, I generally don’t use the docs search, because the success rate is so low. That doesn’t mean that it never find anything.
Yes. All the information is probably out there somewhere, what I think would be needed is to gather just the information that is relevant for doing an upgrade with the option to back out of things don’t work out, in one place. It’s just too hard to find the various pieces of information when you need it, in my opinion. It’s just a suggestion to lower the threshold for people to take part in testing, it’s not a “demand” on my part.
I’m not saying that “it can’t be controlled” or anything like that. But it does a little to much without involving me at times, downloading, updating, starting and whatnot without involving me sufficiently. Of course I could learn what it does when by reading about it. But, if you have no desire to educate yourself on it, and just want to use it occasionally, not really knowing what it will do, isn’t a good feeling on systems you care about.
This wasn’t about me personally either, I use Docker on some VMs without “problems”. I don’t run OH in it on my production system, and has no desire to.
My original point here is that I think there are lots of people that don’t use Docker, and asking them to just use that if they want to upgrade, isn’t a trivial thing to ask. OpenHABian comes standard without Docker, and I think I remember that it has been said that it is even “discouraged” to install it on openHABian. So, if we want people to participate in testing, I don’t think “using Docker” is relevant for those that don’t already run OH using Docker.
And how do you deal with the fact that it’s different based on where and how you’ve installed? openHABian has no fewer than three options to choose from. That’s why it’s mentioned in Getting Started, and covered in detail in the installation docs for the specific platform and way you’ve installed it.
That’s going to be one mega doc when it’s done. But hey, it’s in the docs but not in the docs “good enough”, so let’s create more separate docs! That’ll fix it.
No one is telling anyone they need to use Docker to test an upgrade. Some have mentioning how choosing to do so makes testing an upgrade a lot simpler, and they are not wrong. But I’ve also pointed out all the other ways it can be done too when not using Docker, at least for openHABian.
Once more for the people in the back who can’t hear:
backup OH
upgrade OH
Look for writers errors, no errors you are done
errors found, report to the forum and/or GitHub
stop and restore OH from the backup created in 1
install the version of OH you were previously running
That’s the steps for every OH install.
But the specifics on how you do that is different for just about every way OH can be installed. If you are on Linux/openHABian installed using apt/yum/etc you’d use openhab-cli. If on a manual install on Linux or Mac you’d use $OH_HOME/runtime/backup.sh. If on Windows you is use $OH_HOMR/runtime/backup.bat. Docker just tar up your mounted volumes. If Synology . Probably the same you’d do for Docker. If FreeBSD , probably there is openhab-cli or call backup.sh . Do we still support Qnap ?
And that’s the case for each and every one of those steps. That’s why it’s documented in the only place where OS/platform specific information is kept. Put this all into one doc and if it becomes a choose your own adventure knightmare where 3/4 of the content isn’t relevant to you.
So to find the specific steps For how you have installed OH, refer to the docs for how you installed OH to begin with and if there’s are more than one option like openHABian offers, choose the one approach of the three options you took for backup and restore. Or just use openhab-cli regardless.
Note, an interested backup is no backup at all. If you have any of backup configured for any system, if you don’t understand how to restore from that backup and do not periodically test that the backups are working, you may as well not have backups.
It’s not necessarily irrelevant either. It’s one of the options, that’s all. It’s an option many of us find comes with many benefits and it provides a super low risk way to test out an OH version outside of messing with the production insurance.
But nowhere above is it mentioned as the only option. It’s not even pushed as the best option for everyone. All I see are some people showing how they do it with Docker and they are happy with how easy it is. openhab-cli is pretty damn easy to use too though.
I know the “mantra” thank you, but it’s not entirely that simple. Amanda is basically a full “enterprise backup solution” with no UI at all, and what I find a very “hostile” CLI. It’s not something you’ll know how to manage unless you use if often. That doesn’t mean that letting it run is completely pointless, I’m confident that I’ll figure out how to restore the data, if they’re there and if I invest the time/effort to do so. But, to me, it’s the last option I’d use. I also use SD card mirroring, and I keep my own backup of all the configuration using Git. So, it’s really just there in case everything else has gone wrong, as a last “emergency solution” that might be able to provide something.
When it comes to regularly restoring, I just shake my head. I doubt anybody actually do that. Yes, it’s useful to test the restore process for a system you rely on, because there can be many “surprises” with backup systems, especially now that everything think that it should force encryption on you. So, if you haven’t verified that your data can be restored, you might not actually have a backup, you just think you do. But, doing “regular restore” is BS. You must be able to put some faith in the verify, the general error reporting etc. If everything is good, you must be allowed to think that you have a backup, as long as you know that the restore procedure itself works.
I use tape library backup, and verifying takes almost as long as the backup, and it leads to double the wear of the tapes. So, if I can’t put any faith in what it reports, what’s the point?
You know what you’re logs usually look like better than I do. Look for errors and anything out of the ordinary.
). Basically, I burned the PiLite O/S image and added Docker. The only thing I added on metal was Samba. Once you start altering metal apps during testing, you open the whole backup/restore quagmire. I just get any feedback on the milestone for the maintainers and delete the container/files.