Sd-card corruption?


(cschneider111) #21

I also used Raspi 2 and 3 in the past for openhab and had lots of problems. Almost a year ago I switched to Odroid C2 which uses emmc instead of sd cards. The linux system runs very stable since then. I still have openhab crashes sometimes but it is a big difference between a java application crashing and the whole OS being broken.


(E. Gerland) #22

Thanks for your detailed explanation. Now I understand the SD, USB, SSD differences - That totally makes sense.

I see, that it’s not that simple to make the right decision to go away from RPi.

To summarize there are some very valid points in this thread and in some other threads adressing the same issue (SD-card problems).

Pro Pi:

  • Cheap
  • Big community and experience (OH and raspi itself)
  • tools to simplify things such as openhabian (thanks @ThomDietrich - that helped a lot)
  • easy to backup (I really like the sd-card image backup option, because I use to play around a lot and sometimes mess up the system :wink:
  • low power consumption / footprint
  • flexible for further expansions.
  • (other hw will have it’s own issues)
  • barebone setups do not provide such easy backup options like a complete SD-card backup

Contra Pi:

  • SD-card corruption (which is really annoying when not at home and you don’t know which SD-card backup from the past is already corrupt and which is not).

It’s actually that easy - I don’t see any other contras, but I was so annoyed that I did not see the wood for the trees.

So my (preliminary?) approach to solve / address my issue will look like:

  • Stay with RPi (start with an older backup image of my runtime system to reduce efforts against starting from scratch) - close observation required) - otherwise start from scratch with brand new SD-card
  • use my NAS for the persistence (Yes, @mstormi, @rlkoshak, I will keep it running 24/7 from now :wink: )
  • redirect my /var/log to the USB drive
  • will check out various backup functions (Amanda seems to be a good option).

So thanks a lot for all your input.


(cschneider111) #23

Persistence on the nas sounds like an interesting solution. Let us know how it goes for you.


(Markus Storm) #24

Put persistence db AND logs on NAS, you won’t need the USB drive any longer.

PS: and since I believe you’re running ZWave, too, get your replacement procedure ready in case your controller breaks down. See also the introductory section of the Amanda readme.


(Skinah) #25

cschneider111: The odriod range are great as they support h265 where the rpix does not in Kodi. Been looking for a while but the ability to use ready made images for the rpi just keeps me from making the change.

NCO: I’m choosing to send logs to a ramdrive and have not choosen if I’ll use a NAS or a USB drive directly in the pi for the persistence yet.

You can see how I am doing it here…


(E. Gerland) #26

Thanks, I will consider migrating the logs as well.

I have read the Amanda introduction about HW redundancy and actually never considered to prepare a second set.
This will be the next project though :wink:


(Daniel Malmgren) #27

Well, of course the type of interface doesn’t affect the reliability, but there are more differences between the cards than just the interface. For example many of them seem to have wear leveling, which sd cards seldom (or never?) has. I have read multiple report from people having big problems with trashed sd cards and who have then changed to eMMC cards and after that having no corruption problems at all.


( ) #28

Thanks! However just to be clear: This is not really a “Pro Pi” as you can use openHABian elsewhere as well :slight_smile:


(E. Gerland) #29

Touché
That’s true, but I did not try it on any other hardware.
On my RPi it works flawlessly - so that’s good :wink:


(E. Gerland) #30

One more (possibly stupid) question.
If the SD-card is corrupt, does this mean the card itself is ready for trash or just the data?

When I setup the system from scratch I wonder if formatting would re-animate SD-card after corruption.


(Markus Storm) #31

Well it means you have hit a (eventually intermittent) HW problem. Reformatting might or might not spot that as a bad sector and might or might not exclude it from future use.

Bottom line is, you can’t be sure. I would not reuse that card in a Pi, but you can still make use of it in less write intensive applications like say a camera.


(Skinah) #32

99% of the time it is only the data that is corrupt and flashing a new/backup image will work fine. I have personally worn out multiple USB sticks, but I was working full time doing beta testing for an embedded system and hammered them big time in ways they were not designed for. Of course it may have been from another fault as they were low quality no name brands. Never seen issues with brand name industrial grade flash.

Flash the SD and if it works then great, if not try flashing another SD card with the same image and if that one works then you know the other needs to be trashed after first checking that the image is not larger than the space on the SD card. Two ‘identical’ cards even same brand, size and model can have slightly different amounts of space so using a linux program like gparted / parted to reduce the partitions down helps to create smaller images and helps when you want to write the image to another card. Take an image before playing with parted if new to it :slight_smile: By reducing the partition size your backup images also take up less HDD space to store them.


(csowada) #33

I run an Odroid U3 for three years now with an emmc. And now my emmc is broken. At the moment I run it temporary with an sd card and it is much slower than before. :frowning:
But now I will switch to an Odroid HC1 - a cheap ARM computer with a SATA connector. I hope this helps to prevent corrupt cards.


(Markus Storm) #34

Another hint is to move your swapfile to USB or NAS. Swapping (or paging, to be more precise) happens all the time and goes to /var/swap in default Raspian. Configure a new location in /etc/dphys-swapfile.
Use dphys-swapfile swapon to activate and swapon -s to validate the outcome.


(E. Gerland) #35

Damn it - I feared that someone would say somthing like this :wink:
Thanks for the confirmation.


(E. Gerland) #36

Hmmm… might be an option as well.
I will check out which suggestion will work for me best. :slight_smile:


(E. Gerland) #37

All,

to get of frequently appearing issues with openhab (cron not working properly, persistence hangups) I decided to eliminate sd-card corruption issues (maybe causing the issues above?!)

To do so I just purchased an element14 desktop housing for the pi including a ssd:
https://www.element14.com/community/docs/DOC-83477/l/desktop-computer-kit-with-expansion-board-that-can-turn-a-raspberry-pi-into-a-real-desktop-pc

and installed openhabian 1.3 from scratch.

Just a short (maybe stupid) question:

@ThomDietrich and others
If I did a backup using openhabian-config on my NAS and assuming that one or the other file might be corrupt already - can I import the corruption back into my brand new setup?

Does this mean, I need to configure everything again?


Best hardware platform
(Rich Koshak) #38

Everything in conf can be verified by simply opening the file with a text editor, or even using less on the command line on the Pi itself and scan through them for anything that looks out of place. Even better would be to use VSCode or ESH Designer.

You can do the same for the contents of the /var/lib/openhab2/jsondb files. I don’t know if there is an online JSON checker but I would be surprised if there isn’t.

I believe those are the vital files. If those are all OK then you only need make settings and install bindings again to start from scratch which isn’t that big of an effort. I have to do the same pretty much every time I update my Docker image and it rarely takes more than a few minutes.


(E. Gerland) #39

Thanks, Rich,

I will check the files you mentioned manually.
I am really keen on seeing my system run for months without any issues with the new ssd approach…


(Micael ) #40

I’m not running Raspian, but I would never enable swap on RPi. Do you really need it in an OH environment?
I would have every partition read-only except one, if really needed (database log e.g.). And I would make the RPI backed up by a simple battery solution at least.

I don’t even enable swap on my desktops (the last 15 years). Guess I’m a bit damaged… :smile: