One more info, in case it would help.
I almost succeded once to upgrade (but i did so many things to try to fix - out of desesperation - that I did not move to it as my production environment, but I still have it running as a test environment) - at that time the latest was 3.3.0.M2.
I have the exact same problem on that version i.e. I can’t upgrade it any more to anything. I have the same undecipherable bunch of errors that are different depending on the number of restarts I attempt etc…
How long after that first start after the upgrade do you wait? Maybe it needs to download somethings and you are not waiting long enough.
How old is the SD card this is running on? Weird behaviors like this often point to a corrupted file system or the SD card is failing.
I wait usually 5 minutes. I’ll try to wait more, I’m pretty sure I did it at some point, but I’ll try again.
I don’t use SD cards other than for the boot partition, the system is installed on a mounted partitions from a NAS.
Well, that’s somewhat of an unusual configuration and not directly supported by openHABian. Or put another way, you aren’t really running openHABian, you are running openHABian with some additional stuff you’ve changed. Given that, our ability to help is hampered because we don’t know what else might be different and we can’t make assumptions we otherwise would be able to make. To a large extent, we can’t know if the problem is something related to OH or it’s a problem you introduced by deviating from openHABian.
I assume you are not running with ZRAM enabled. There’s no point in doing so if you’re running off of a remotely mounted file system.
Are you sure everything that needs to be mounted is mounted and read/writable?
If possible, what I would do is run an experiment. First create a fresh install of 3.1 (no configs) and run an upgrade. If that fails we know there is nothing related to your configs causing the problem and it’s definitely a problem caused by things you’ve done to openHABian. We don’t know what you’ve done so if that’s the case 
If that does work, again start from a clean and fresh install of 3.1 and this time restore a backup and make sure all is well. Then run the upgrade. If only now it doesn’t work than you know the problem is somewhere in your configs and we can maybe narrow down those until we find the problem.
I understand.
I reason I say it is an openhabian config is because I install first on SD card - stricto sensu openhabian -, then i rsync the system partition on the NAS and then restart booting from this new file system. No changes no nothing other than copying and repointing to the copy…
Everything else on the raspberries works and upgrades with no apparent problems, for already some years.
I’ll look at the tests you propose and will see where to go from there.
What i find painful when reinstalling from scratch is everything that is not covered by Openhab backup and restore (grafana, influxdb, my ser2net config to access the Zwave dongle on a different raspberry and a few other things like this…). But i will probably have to go for it…
Thank you for the time you took to look at my case.
Do the experiments on a new SD card. Once you figure out what works and what doesn’t work you can then decide whether you can salvage the unworking SD card/config or need to start over from scratch.
Maybe you already know and I do not know if it is relevant: there are some hardlinks in the openhabian image, that need to be treated properly by the rsync.
Hi, thanks a lot for chiming in… do you know what those hardlinks (or at least example of them)? I’ll check if that could be the issue…
I’m doing rsync -Phax --numeric-ids to perform the copies…
/srv are bind mounts, then the zram directories, but I am not an expert.
Ok, i don’t use zram, so that should be good.
Do you what /srv is used for ? I don’t use Samba either… And i never go to this /srv folder.
So, I started by doing, on both my production env (3.1.0-M3-1) and my test environment (3.3.0.M2) a full purge of openhab, checked that all relevant directories had been cleaned, and re-installed openhab 3.3.0, fresh.
I did the basic username config to check that it was basically ok and then restored my backup.
After giving the sensation that it was going to work, I ended up on both configs with the same kind of random java errors.
Tomorrow I will check your exact steps, Rich.
Ok, sorry for getting back to this so late. Last weekend, I did rebuild a full 3.3.0 configuration on a SD card, I was able to load my backup openhab config and to recover my backup Influxdb database…
And it was successful.
Then, I moved the / directory to the NAS (using rsync as above shown), and all was good. Including installation of the bindings etc…
But, I think there is still a problem, as for nearly all bindings I try to install after having moved to the network system partition I get a significant bunch of java errors and ultimately I need to uninstall them (and then recover a properly working openhab).
I’m nearly sure that when I’ll upgrade to the soem next version of openhab, I’ll have the same problems… At this stage, my conclusion is : when I move the openhabian system to the NAS, there is something that breaks openhab, not to the point that it stops working, as I have it working like this for a long time, but to the point that it is not able to add bindings, or to upgrade etc…
So I need to study this more. @rlkoshak I know what I’m trying to do is not exactly what openhabian is supporting, so I’ll understand if you can’t help. But I’m hoping someone could have a similar config and be able to share some experience. This can’t be that different from the SSD based configs?
@Lionello_Marrelli , thanks for the link you shared, I don’t think what is mentionned could apply to me, as in my case the system works very well after I move the / entirely to the NAS. Do you think I am missing something?
What is strange is that e.g. neither grafana, nor influxdb, nor any other Raspeberry OS seems to have the same problem, as they all update without problem and work just fine over time, even with / on the NAS.
I rsync the / config using -H option to preserve the hard links, so I would hope that this works well.
One thing I noticed is that the problems displayed in the log are very often related to “bundleFiles” in the cache. I don’t know if this can help locating the possible challenge?
Thanks a lot in advance for any help…
Given how permissions are handled through NFS and SAMBA yes, it is significantly different from an SSD based config. But unless the errors are related to file permissions 
. It’s not an approach I’ve ever run with for openHAB. When I’ve tried to run something from an NFS mounted folder in the past (e.g. PostgreSQL) I had nothing but trouble.
Keep in mind that you can’t do hard links across different file systems. When you move to the NAS you have moved to a different file system so definitely pay attention to those. You might have one that is pointing somewhere that isn’t being rsynced properly. That’s just a guess though.
Thank you Rich as usual for your inputs.
For the sake of clarity, i should add that i move the entire system folder starting from /. And I use rsync to perform the move.
Then I forget about the file system on the SD card, I don’t keep any type of sync between the NFS mounted file system and the SD card.
And I have no hybrid config : all system files are on the NAS.
Also I don’t use Samba, i edit the files i need to edit directly on the system using vi or using VS Code via ssh.
One thing I’m thinking about testing is openhab installed straight on Raspi OS config without openhabian and see if there is the same challenges.
I also noticed that openhabian script to move FS to SSD is using rsync -avx options.
I’ll try rebuilding my setup with this instead of rsync -PHax like am doing so far and we’ll we if there is an improvement.
And I changed the subject of the thread to reflect the real topic, better than before.
No. openHABian does not support moving any of its partitions to SSD or NFS.
What you saw is not “the” script to move to SSD but part of the SD mirroring feature.
Noone -including myself- knows what are all the places one would have to look at and change things in for this to work so noone will have this in mind when developing, or testing against.
Not to mention the amount of work you (and we on the forum) are wasting trying this.
That’s the reason these modifications aren’t supported. That’s why you shouldn’t be doing this.
So why are you so insistent on making it work with your NAS ?
Even if you manage to get it to work, you will keep being at risk that things may change at any time with an openHABian or Linux package update, breaking your installation.
Your current setup is already less resilient than a proper new openHABian install would be, one reason because it depends on the NAS to be available in addition to your RPi (and I bet you do not have a spare NAS, do you ?).
NAS mounting has a number of more drawbacks. I know well, I myself had used such a setup in early times myself and openHABian as it is today is an evolutionary improvement over it.
I suggest you reinstall from scratch to a new SD and use the builtin mechanisms.
That is: setup SD mirroring.
You could change the directory you run your persistence database in to a NAS mounted one (remember to eventually take it off ZRAM /etc/ztab), but please do not touch the root partition, or swap.
Thanks @mstormi for taking the time to answer. I know you already told me this was not the recommended way. I will understand perfectly that you do not waste time to answer my questions.
I do find advantages and easyness with the NFS mounted system partition : switching from one config to the other, easily save the entire config, try something and then come back with a just a mv command on the NAS and a reboot, without manipulating SD cards - as my raspberries are in the basement.
Also, I had the feeling that Raspberry OS was evolving in the direction of allowing more things from the network e.g. network boots are now supported out of the box… I thought I had read a lot on the topic and did not encounter recommendations (out of openhabian context for not using this type of config).
As for the “move root to USB”, you are right I looked at it too fast and it was the SD mirroring script. I looked at the move_root2usb() function and it is rsync -axv that is used.
But at this stage I really need to run more tests as I was not patient enough to properly test all scenarios as per Rich suggestions to understand where exactly my problem is coming from.
Just to be complete, there are lots of ways to achieve all of these that do not involve a NAS and NFS mounting. The fact that your RPis are not easily accessed is irrelevant. One approach could be to use VSCode and the RemoteSSH add-ons which make it as if you are working on that remote machine. I think it supports RPis now. With that you work on that machine as if you were sitting at it, including getting a terminal, copying files around and all that sort of thing. And that’s just one approach. Others can include sharing the config folders from the RPi instead of to the RPi. Yet another approach could use git or some other source control software.
The end goal of openHABian is first and foremost reliability. And as Marcus states, the more moving parts you have, the less reliable your overall system becomes. With an NFS mounted file system you greatly increase the unreliability and end up with a bunch of additional problems and limitations for your trouble (what if the NAS goes down? file permissions and ownership, what if the NAS is down when the RPi boots? etc.). About the only thing that the NAS provides from a reliability perspective is it limits SD card wear out, and the ZRAM configuration manages that by itself with fewer moving parts.
Personally, while not doing this for OH, I used to run a number of services off of an NFS mount. I’ve disabled them all except for Plex and Nextcloud as being more trouble than they are worth (I’ve only kept Plex and NC on the NFS mount in part though to store the big files that don’t change much), settling on backup and restore scripts and stuff like that to keep things safe and reliable.
Hello Rich, sure I use all those SSH, VS code + remote SSH addon etc…, of course.
I was referring to quickly switch from one test config to the prod config, for example, without having to switch SD cards.
Thank you for the other considerations.