How to: The method I use to backup my PI and C2 based Openhab


(Skinah) #1

There are a number of ways to backup a SD card including Amanda, I’m sure all of them are good as I have not tried them all out yet, but I thought I would share my backup system not just the software I use and also the reasons why, in case you find it useful. I recently found a program called Paragon Hard Disk Manager (PHDM) which is free for home use, after you register it for free. The program is great for two reasons.

  1. You can restore the image created on a 64gb card back to a smaller 8gb SD card just by ticking a resize box in the program. No need to use the linux program “parted” or “gparted” to resize partitions. More on this topic below…
  2. It creates images that save space when most of your SD card is empty. Unlike Win32diskimager that creates very large images, PHDM creates a 2gb image for my Openhab backup even with a 128gb card. Larger cards have faster read and write specs.

I wanted to explain why I chose this method and some topics which not all users will be familiar with as this particular method/system is different.

Some things to consider about your backup method are:

  • Have a hardware backup as well as a software backup: What happens if your Openhab hardware gets stolen or the hardware physically breaks. Is your backup files stored on that same device? I like the cheap raspberry PIx and SD cards because I wish to always keep a second one around setup ready to go. A full hardware backup ready to go, not just a software backup. Downtime should be able to be fixed in under a minute plus booting time by any family member.
  • Never backup a system that has been running a long time: Bit rot, soft error (and many other terms you can google for more info to help you sleep) all work to destroy data over time as computers do make mistakes which then accumulate. Sure you can buy ECC ram, and use methods like parity hard drives to error check creating a ‘better’ servers to run Openhab on, but then the cost of having a HARDWARE backup goes up. The point I am trying to make here is, I do not like to make backups of a system that has been running for any length of time. What is the point of backing up if the files already contain corruption?
  • Use a UPS: Dirty shutdowns are when you interrupt the power when the system is running. Never pull the USB plug unless the server is shutdown correctly and also protect your system from this with a UPS. Don’t forget what happens when the UPS runs out of battery life and then the power comes back on and fails in and out. This kind of thing can quickly destroy data on flash based systems as during the boot up period the SD card under goes a lot of WRITES and you never want the power failing when linux is booting as the chance for corruption is very high at that time.
  • Learn 1 system that works for all cases: If you use multiple raspberry PIs or Odroid clones for multiple linux projects, this method will work for them all without the need to re-learn how to use Amanda for openhab and FooBackup for this X project. 1 Method works for all and the system is very simple.
  • Automatic daily backups should not be relied upon: They can be used in addition to this method to capture small daily config changes, but if they are corrupt and you never use them to find this out, the time when you actually need the backup you will get a big shock.

Now for the details of the backup system

I keep 2 to 3 SD cards which are used for:

  1. The MASTER card which is setup and then backed up once working. This card lives in the backup hardware PI ready to go, but not running to prevent corruption from accumulating.
  2. The BACKUP card which is what I use to run my system with and proves the backup file is working.
  3. An optional TEST card. If I need to experiment or trial a new upgrade (newer is not always better) I put the BACKUP card aside and work with the TEST card whilst I am home, then when I go to work the BACKUP card can go back in until I find some time to continue with the TEST card. Often I find I skip the test card as I can easily make cards from my backup files at any time.

When I need to make a change to my setup, I will load the MASTER card, make the change which I have already tested and then I create a new BACKUP file and card which is then used. The cycle keeps following this process and the MASTER card is never used or left running. Never throw old backup files away.

How to create the Image file:

  • Download, install and register the Paragon Hard Disk Manager. It is free for home use if you register.
  • Insert your SD card into a usb reader on a windows based machine with above software.
  • Follow the wizard step by step to create an Image file of the entire SD card.
  • The backup file of my large cards is only taking up 1-2gb on my hard drive because this software does not store every bit that is empty space and compression is used.

How to load/write the Image file onto a smaller SD card:

  • Use Paragon Hard Disk Manager as above but follow the wizard to restore the image.
  • Tick two boxes, one to resize and the other to change the disks unique ID. This is at the top right corner of one of the restore steps.
  • If an attempt fails to work, then the key is to restore the BOOT (first partition) first, and then restore the second partition (larger one) afterwards running the wizard twice to restore only a partition and not the whole disk. By doing this in two steps it prevents the BOOT from being resized and the card should then work. If you don’t use PHDM you can resize the larger partition before taking the image but I recommend taking an image first before playing with the linux program ‘parted/gparted’ to do that.

Why does a restored image not work to an identical SD card?

When SD cards are made in the factory, they do tests on the newly made cards and then mark any bad sectors that fail these tests. They get marked in a way so you can not use the failed areas and then they sell these cards instead of throwing them out because a small area of the flash has a minor issue. Because of this you can and probably will have two cards that are meant to be the same, ie same brand, size and model, yet software will tell you they are not the same size.


Clone installation openhabian and write it to another SD
Does open hab require raspberry pie or any other device to operate?
Help with restoring a partition Amanda [SOLVED]
Install (2.3.0-1) on RPI (apt-get install) Not working
(Danny mullen) #2

Thanks for sharing. I am happy that you found a solution that works for you! Also, happy you shared your knowledge.

I am going to look up the paragon software, I had never heard of it. Sound like a good tool to backup my wifes computer.

I like a full backup solution. I am a big proponent of backing up the full disc vs just data files. I have found that if you only backup data files, there comes a day when you need that obscure operating system file for something and don’t have it. Also, like that you use the backup card as your primary as this proves the backup is good as you stated!


(Markus Storm) #3

Yes thanks for sharing. Always good to hear about working solutions.

Reminding you and readers, though, that taking a backup that way will require you to take down your openHAB server.

You might not be aware but Amanda can backup and restore partitions as well (in addition to doing selective directory/file-level backups), and it can do that while your server and openHAB remain alive. Another advantage is its builtin versioning.
I attached a card writer, too, so just like you, I create BACKUP and TEST SD cards, but can do that right from my running Pi.

PS: I went the fdisk/gparted route to shrink my / partition so it’ll fit onto every SD card to have 4GB or more.
Luckily I just had to do that once.


Clone installation openhabian and write it to another SD
(Kim Andersen) #4

Just a note…
I don´t see Paragon Hard Disk Manager beeing free. Whats free is their Backup & Recovery program located in their free download section. But these are two difference programs.

I use Win32diskmanager. Though this is not good, as it can´t shrink the image size. My old OpenHab system was running a 16GB SD, which ended up in a 128GB SSD when I did the change.
Now when using Win32diskmanager I end up with an 128Gb image file :slight_smile:

But it still works as an backup though.


(Holger) #5

hi, i use “raspibackup”, google for it. It can start/stop services, shrink the image and can restore it. i use a ssd 16gb for my openHAB and the image is a tar with around 6GB, can be restored on disks/sd-card with > 6GB, and i tried many restores :wink:
you can unpacking the tar and grab files if you need to restore single files.
try it out!
Holger


(Skinah) #6

The backup and recovery is now discontinued as it is built into the PHDM, they combined 4 programs into the one so you can see what else they do whilst using the free part of the program and then pay to unlock the extra stuff. It is free for a single computer and home use, all you do is register. Give it a go and your backup files will drop greatly.

Sounds great but does not suit me and the way I prefer to do backups from a non running sd card.


(Kim Andersen) #7

Okay, good to know… It´s not that obvious from their docs.


(Skinah) #8

Yes their website is not the best. I thought the same until I dug further.


(Marian Tudor) #9

Hello,

I want to thank you for your article about how to use Amanda to backup Openhab(ian).   

Until now I was able to configure and perform a backup of some directory but I don’t know how to do it for entire SD Card (Backup raw SD card). I don’t want to use AWS S3 solution for this scenario so I was wandering If you can tell me how to use other local media (usb stick/ HDD/NAS) as a location for backup.

Thank you in advance,
Marian

P.S I try to add /dev/mmcblk0 - my internal SD reader in /etc/amanda/openhab-<config>/disklist but I don’t know is the right way.


(Markus Storm) #10

Yeah that’s the right way.


(Marian Tudor) #11

Thank you very much !