I just came across an overlay fs as a solution for flash memory corruption / wear (https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt). And yet there is nothing about it on this forum - at least I couldn’t find any posts. The basic approach is to make the flash memory as read only “lower” mount and all writes direct to in memory overlay “upper” mount. Additionally the “upper” writable mount could be synced into flash memory periodically or at shutdown. This approach solves the problem of data corruption on power loss and flash wear by the system design and it feels to me like it is THE WAY to solve those problems instead of buying large industrial grade sd card and attaching UPS. The OpenWRT project uses this approach and I believe that the majority of commercial products that require large file system on the flash memory are using this approach. I can’t think of any product that require graceful shutdown or changing flash memory every year. So if You (I mean the OH community :)) agree with me on that, maybe OpenHABian should also take this approach ? What do You think ?
Except that for most OH systems there is a good deal of information/data that has to be reliably persisted to storage. To get that you either have to sync to the SD card so frequently that the overlay doesn’t buy you much or run the risk of significant data loss.
Take the following for example. Lets say you have a number of Items saved in MapDB with restoreOnStartup because you need these Items to be set to their previous state when OH starts up. Now lets say you sync the overlay to the SD card every hour. Now for whatever reason, the RPi lost power one minute before the next sync. When OH comes back up it will get the restored states from almost an hour ago, which could cause all sorts of problems.
Is it possible to make something like overlayfs work with OH? Absolutely. But it does impose lots of limitations on how one uses OH so I can’t agree that it is “THE WAY.”
I don’t think I’ve seen this as a recommended solution on this forum very often. Usually, the approaches are a combination of one or more of:
a good backup and restore procedure which is a must no matter what you are running OH on
running the RPi off of an SDD or HDD (Pidrives can be had for $40 which is not much more than a good SD card)
putting logs into a tmpfs or on a USB thumb drive
using a phone charging power brick for battery backup (an RPi will run for a good long time on a 10000 mAh phone charging battery)
That makes a lot of sense as systems like OpenWRT and the like do not have the same need to persist data like OH does. A typical OpenWRT system will operate with few changes that need to be persisted. dumping the updates to the storage once a day (and perhaps immediately on a config change) makes a lot of sense. If the device loses power you will probably only lose some logs. A typical OH system with persistence is constantly saving data to disk and if you lose any or most of that data when the RPi loses power it can have a significant negative impact.
Like I said, you can set up OH to work in a system like this:
you can’t rely on logging being persisted on a reboot or loss of power
you cannot use OH persistence that is hosted on the RPi itself which eliminates all of the embedded databases (MapDB, RRD4J, SQLite, etc).
you probably should not rely on restoreOnStatup at all
See above
I think it is a great option but it should not be the default on openHABian. You need to fully understand the compromises involved with using overlayfs and frankly, if you know enough to understand that then you have the knowledge and skills to do overlayfs on your own without openHABian. Given that openHABian is intended for a beginner audience, I’m not sure I would offer it as an option. Of course, you can file an issue and let the people who actually maintain openHABian chime in.
I cannot agree that this is “THE WAY” either.
Honestly I didn’t catch how this is compensating “by design” for the risk of flash wearout/powerfail-on-write leading to SD corruption unless you don’t want to sync at all at some point in time.
And all data is lost if the crash happens inbetween the write and a sync.
That’s ok for expendable data such as logs (see link below), but it is not acceptable for essential data such as config and persistence data. And to continously store persistence data updates isn’t quite the same league in terms of the number of writes as a couple of config files to get updated every few weeks as you would typically encounter this in OpenWRT.
FWIW, I suggested to change the openHABian filesystem layout (here) as a prerequisite but we still haven’t agreed on this to make it the standard in openHABian.
Once there, you can choose to mount any type of FS such as a second and third USB stick for your logs and persistence, or any SSD, HDD or NAS mount.
That way, you can have a different/medium as the sync target so if writing during a power loss causes corruption, your root FS won’t be affected. But still you cannot guarantee integrity of that data.
And FWIW, there’s a number of tweaks you can apply to openhabian today (see this post).
You could e.g. deploy the commit=XXX option. If combined with separate mounts as in my suggestion, that’ll effectively do the same as your proposal.
