How to: The method I use to backup my PI and C2 based Openhab


(Skinah) #1

There are a number of ways to backup a SD card or you can use the Amanda in Openhabian to backup your system, I’m sure all of them are good as I have not tried them all out yet, but I thought I would share my backup system and why I use it in case you find it useful. I recently found a program called Paragon Hard Disk Manager which is free for home use and makes it much simpler to backup any storage device including SD cards from a windows based machine. The program is great if you are not good with linux for two reasons.

  1. You can restore the image back to a smaller SD card without using the linux program “parted” or “gparted”. More on this topic below…
  2. It saves space when most of your SD card is empty, it will not store a 128gb backup file for a 128gb SD card. Win32diskimager creates very large images, and PHDM is easier to use then DD under linux.

First up I wanted to discuss why I choose this method and some topics which not all users will be familiar with which explains why I use this particular method/system.
Some things to consider about your backup method are:

  • What happens if your system gets stolen or the hardware physically breaks. I like the cheap raspberry PIx and SD cards because I wish to always keep a second one around setup ready to go. A full hardware backup ready to go, not just a software backup.
  • Bit rot, soft error (and many other terms you can google for more info to help you sleep) all work to destroy data over time as computers do make mistakes which accumulate. Sure you can buy ECC ram and better servers to run Openhab on, but then the cost of having a HARDWARE backup to cover you against hardware failure or theft goes up. For many reasons I wont go into I do not like to make backups of a system that has been running for any length of time. No reason why you can not run more than 1 backup system if you wish.
  • Dirty shutdowns are when you interrupt the power when the system is running. Never pull the USB plug unless the server is shutdown correctly and also protect your system from this with a UPS. Don’t forget what happens when the UPS runs out of battery life and then the power comes back on and fails in and out. This kind of thing can quickly destroy data on flash based systems as during the boot up period the SD card under goes a lot of WRITES and you never want the power failing when linux is booting as the chance for corruption is very high at that time.

You need a known good system with no corruption before making a backup, otherwise you backup the already corrupt data. This may sound obvious, but if you do not know the causes of corruption and how it happens it is easy to un knowingly be doing exactly that. These are the reasons for the following system, also this same system works on other projects that I use like Kodi, keeping the system the same means life is simplified…

I keep 3 SD cards which are used for:

  1. The MASTER card which is setup and then backed up once working. This card lives in the backup hardware PI ready to go, but not running to prevent corruption.
  2. The BACKUP card which is what I run my system with and proves the backup file is working which can be used to create any number of new BACKUP cards.
  3. A TEST card. If I need to experiment or trial a new upgrade (newer is not always better) I put the BACKUP card aside and work with the TEST card, this way I can keep the system running whilst I do tests on another PI or just swap the cards out.

When I need to make a change to my setup, I will load the MASTER card, make the change and then create a new BACKUP file and card which is then used. The cycle keeps following this process and the MASTER card is never used or left running. Never throw old backup files away.

How to create the Image file:

  • Download and install Paragon Hard Disk Manager
  • Insert your SD card into a usb reader on a windows based machine with above software.
  • Follow the wizard step by step to create an Image file of the entire SD card.
  • The backup file of my large cards is only taking up 1gb on my hard drive because this software does not store every bit that is empty space and compression is used.

How to load/write the Image file onto a smaller SD card:

  • Use Paragon Hard Disk Manager as above but follow the wizard to restore the image.
  • Tick a box to change the disks unique ID, this is at the top right corner of one of the restore steps.
  • If an attempt fails to work, then the key is to restore the BOOT (first partition) first, and then restore the second partition (larger one) afterwards running the wizard twice to restore only a partition and not the whole disk. By doing this in two steps it prevents the BOOT from being resized and the card should then work. You could alternatively resize the larger partition before taking the image but I recommend taking an image first before playing with doing that.

Why does a restored image not work to an identical SD card?

When SD cards are made in the factory, they do tests on the newly made cards and then mark any bad sectors that fail these tests. They get marked in a way so you can not use the failed areas. Because of this you can and probably will have two cards that are meant to be the same, ie same brand, size and model yet software will tell you they are not the same size. You can get around this in a number of ways by reducing the larger partition down in size, or by following the advice in the steps above. Paragon Hard Disk Manager can do either of these two things for you if you have a windows machine, or you can use parted/gparted under linux to resize a partition BEFORE taking the backup image.


Clone installation openhabian and write it to another SD
Does open hab require raspberry pie or any other device to operate?
Install (2.3.0-1) on RPI (apt-get install) Not working
(Danny mullen) #2

Thanks for sharing. I am happy that you found a solution that works for you! Also, happy you shared your knowledge.

I am going to look up the paragon software, I had never heard of it. Sound like a good tool to backup my wifes computer.

I like a full backup solution. I am a big proponent of backing up the full disc vs just data files. I have found that if you only backup data files, there comes a day when you need that obscure operating system file for something and don’t have it. Also, like that you use the backup card as your primary as this proves the backup is good as you stated!


(Markus Storm) #3

Yes thanks for sharing. Always good to hear about working solutions.

Reminding you and readers, though, that taking a backup that way will require you to take down your openHAB server.

You might not be aware but Amanda can backup and restore partitions as well (in addition to doing selective directory/file-level backups), and it can do that while your server and openHAB remain alive. Another advantage is its builtin versioning.
I attached a card writer, too, so just like you, I create BACKUP and TEST SD cards, but can do that right from my running Pi.

PS: I went the fdisk/gparted route to shrink my / partition so it’ll fit onto every SD card to have 4GB or more.
Luckily I just had to do that once.


Clone installation openhabian and write it to another SD
(Kim Andersen) #4

Just a note…
I don´t see Paragon Hard Disk Manager beeing free. Whats free is their Backup & Recovery program located in their free download section. But these are two difference programs.

I use Win32diskmanager. Though this is not good, as it can´t shrink the image size. My old OpenHab system was running a 16GB SD, which ended up in a 128GB SSD when I did the change.
Now when using Win32diskmanager I end up with an 128Gb image file :slight_smile:

But it still works as an backup though.


(Holger) #5

hi, i use “raspibackup”, google for it. It can start/stop services, shrink the image and can restore it. i use a ssd 16gb for my openHAB and the image is a tar with around 6GB, can be restored on disks/sd-card with > 6GB, and i tried many restores :wink:
you can unpacking the tar and grab files if you need to restore single files.
try it out!
Holger


(Skinah) #6

The backup and recovery is now discontinued as it is built into the PHDM, they combined 4 programs into the one so you can see what else they do whilst using the free part of the program and then pay to unlock the extra stuff. It is free for a single computer and home use, all you do is register. Give it a go and your backup files will drop greatly.

Sounds great but does not suit me and the way I prefer to do backups from a non running sd card.


(Kim Andersen) #7

Okay, good to know… It´s not that obvious from their docs.


(Skinah) #8

Yes their website is not the best. I thought the same until I dug further.


(Marian Tudor) #9

Hello,

I want to thank you for your article about how to use Amanda to backup Openhab(ian).   

Until now I was able to configure and perform a backup of some directory but I don’t know how to do it for entire SD Card (Backup raw SD card). I don’t want to use AWS S3 solution for this scenario so I was wandering If you can tell me how to use other local media (usb stick/ HDD/NAS) as a location for backup.

Thank you in advance,
Marian

P.S I try to add /dev/mmcblk0 - my internal SD reader in /etc/amanda/openhab-<config>/disklist but I don’t know is the right way.


(Markus Storm) #10

Yeah that’s the right way.


(Marian Tudor) #11

Thank you very much !