Reducing SD card writes with openhabian

It may not be worth all of the trouble now that SanDisk > Western Digital is now making MicroSD cards exactly for HDD replacement applications, including fast, high-endurance SLC, plus a wear-leveling controller all in one MicroSD package. About $25 per 16 GB.

https://www.mouser.com/ProductDetail/467-SDSDQED-032G-XI

Promises to be making. Most cards already do wear-levelling BTW.
If you want to rely in that, feel free.
For everybody else thereā€™s now ZRAM available in openHABian.

Thereā€™s a lot more to HDD-class flash storage than Wear leveling, e.g. SLC among others differences in controller logic. I consider this cheap marginal improvement in availability no matter your setup.

There are better options out there. OpenHabian if you have RPi ā€¦ while if you are on Odroid or something similar, check Armbian | openHAB

All reasonable SD card optimisations comes out of the box, ZRAM logging, delayed writing ā€¦ are present in the system for around 8 years now. For most demanding cases system can also be switched to fully (virtual/memory file system) read only mode (Ubuntu variants) which means absolutely no writings (damaging) to the SD card. Optimisations boost up the system since it doesnā€™t use slow media so much and also prolong their life.

SD cards were improved in past decade, but they eventually die. Also most expensive ones.

Panasonic also makes industrial cards. See https://na.industrial.panasonic.com/products/storage-media/industrial-sd-cards/lineup/industrial-grade-2d-nand

They wonā€™t last forever (just like you and me :wink:), but they might be helpful for some users.

It is interesting that people always concentrate on card wear out and miss a very important piece of having a stable setup. That is if the power fails in the middle of writing a block of data it can corrupt the entire block which can contain operating system files when the write was only for a log file. It is like playing Russian roulette when the power fails. If you delay the writes it only changes the odds. Stopping the writes is how all devices like routers work that still use flash and just keep on working.

If you use a UPS it does not solve the issue fully but it does help until the battery goes flat.

I have a pi clone setup on my tv that gets the power cut every time the tv is turned on and off, it is still working years later despite the thousands of dirty shutdowns.

Spending more on a flash drive does not solve the issue of corruption.

This thread is now very old and outdated as I donā€™t use openhabian I have no idea how often it writes, you can use the methods in this thread to find out, led flashing on your Pi, the Linux command to watch in real-time the writes.

The Panasonic does mention ā€œpower fail robustnessā€. Not sure exactly what that does, but it might help.

I use a battery backup with the apcupsd daemon to shut down the system cleanly when the battery runs out. That works very well.

ZRAM in openHABian uses OverlayFS since OH cache is written at OH start and itā€™s huge (300+ MB) so we donā€™t want to keep that in a Piā€™s RAM.

@mstormi
I should write a big THANK YOU for having put ZRAM and any other features into openhabian to address this. Any improvement is a step in the right direction and helps to stop people blaming Openhab for being unstable when that is not the case.

Yes I see that as the biggest issue.

  1. UPS drains completely and the power is lost, luckily we escaped corruption when this happened.
  2. The power comes back on with a flat UPS and the system powers back on and then cuts in and out 30 times over the next 10 minutes. This happens a few times a year at my place.
  3. This time when Linux/Openhab is booting it is writing like crazy and the odds are very high that corruption will occur. We donā€™t escape corruption this time and the UPS has failed to help.
  4. We have an automated backup system that then backs up this corrupted data.
  5. People then complain that they have worn out 3 cards and/or Openhab is unstable when there is nothing wrong with the card and it can still be used after a re-image.

There are ways to setup a UPS to stop the above happening.

Okay, Iā€™ll bite. How do you do that? I have my UPS set to turn off at 50% battery, so it would likely survive the scenario you list as it could cover the next 10 minutes of on/off, but if that happened to be longer, my system would try to shutdown and restart frequently as well. It would be interesting if the UPS had some sort of turn on level so it wouldnā€™t turn on until the battery charge was at least at a specific level. Then we could have some hysteresis. But I donā€™t see that in the cheap UPS units. And you have made me wonder what the apcupsd software will do if the system starts up and the battery is already too low. Presumably it starts an immediate shutdown, but I donā€™t see anything documented about it. It appears that most of the software assumes a power outage followed by good power with no additional outages, but that isnā€™t what happens in my area.

There are plenty of articles about this topic which is not new or unique to Openhab, a quick google will return many written by people that are smarter then me. I only wish to get people thinking about the issue instead of blaming Openhab or flash based cards. If you know about it then you can put a plan into action to deal with the issue. A UPS helps reduce the risk and by having one you are smart.

Same here, my UPS logs at least 40 issues a week with the local power. I choose to accept some risk and to use backups by keeping a uSD card that is ready to go and can be swapped out by anyone. This brings up the second point I try to push and that is having a good backup that is not made from potentially a corrupted source.

SSDā€™s usually have extra power stored for this exact reason and auto sensing to know the power has failed and they can complete the write they are performing before the power is lost without corrupting the rest of the data that was in the same block. The price of SSD is now very low.

This good points mainly fit to the openHAB 2 with PI. Are there some tutorial for a general installation?

Read about ZRAM and the documentation for this feature in the openhabian docs.

I am not sure this will work on a ā€œgeneralā€ linux. I guess it depends on some extras that needs to be compiled into kernel and tools in rootfs, right? And it requires 1GB RAM, so my board has only 512MB.
So, alhough this might be a perfect solution for a fixed image like openhabian, it might not work on others, like my debian-iot for beagle bone black with 4.19 kernel.

That is why I look for an updated general solution.