Recommended way to backup/restore OH2 configurations and things?

[aksnet (Andreas Kuhn)]

Hi Rich,

my PIs and NAS share the same Network. I just had not the passion yet to learn how to map my NAS drive :). I guess Linux will not become my best friend, so every new thing i have to look it up how to do it and sometimes things are different by the distros.
But the Forum here already helped a lot and your thoughts made change my mind and i will see how to map the drive manually and perform backups on the NAS then.
I think MOUNT will be the one command i need to look closer at, cause my nas is not running 24/7 and will be WOLed whenever i need it i have to map and unmap the share for backups.

Thanks
Andi

[Benjy (Ben Clark)]

Hi all, I wanted to direct people that were interested in a PR for openHAB that contains a backup and restore script for Linux and MacOS to be included in future versions of openHAB. It hasn’t been merged yet as it needs people to test it.

Backup and restore scripts for Linux/MacOS

The scripts should work with the snapshot versions.

[Austris_V]

@aksnet, I wonder, do u also have the other side of script - for restore? Thank You!

[sihui]

@Austris_V

for restore?

openhab/openhab-distro/blob/main/distributions/openhab/src/main/resources/bin/restore

[aksnet (Andreas Kuhn)]

@Austris_V

@aksnet, I wonder, do u also have the other side of script - for restore? Thank You!

Hi @Austris_V, i have not yet needed the restore, but let me see what i can come up with during the day…
I already have some in preparation, but need to check if this should work.
I will follow up later on this.

Andreas

[aksnet (Andreas Kuhn)]

@Austris_V, so this is my actual backup script, fully working so far:

#!/bin/bash
# stop openhab instance (here: systemd service)
echo "+-+-+-+-+-+-+ Stopping service...+-+-+-+-+-+-+-+"
sudo systemctl stop openhab2.service

# backup current installation with settings
echo "+-+-+-+-+-+-+ timestamp and folder setup +-+-+-+-+-+-+-+-+";
TIMESTAMP="`date +%Y%m%d_%H%M%S`";
sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP";
sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP/influxdb";
sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP/grafana";
#bakdir="$HOME/openhab2-backup/$TIMESTAMP";
#mkdir -p "${bakdir}"

echo "+-+-+-+-+-+-+ OH backup +-+-+-+-+-+-+-+-+-+-+"
#cp -arv /etc/openhab2 "${bakdir}/conf"
#cp -arv /var/lib/openhab2 "${bakdir}/userdata"
#find /var/lib/openhab2 \( -path /var/lib/openhab2/tmp -prune -o -path /var/lib/openhab2/cache -prune \) -o -name '*' -exec cp -arv {} "${bakdir}/userdata"  \;
sudo cp -arv "/etc/openhab2" "/media/usb/openhab2-backup-$TIMESTAMP/conf";
sudo cp -arv "/var/lib/openhab2" "/media/usb/openhab2-backup-$TIMESTAMP/userdata";

echo "+-+-+-+-+-+-+ Grafana backup +-+-+-+-+-+-+-+-+"
sudo systemctl stop grafana-server
sudo cp -arv "/etc/grafana/grafana.ini" "/media/usb/openhab2-backup-$TIMESTAMP/grafana/grafana.ini";
sudo cp -arv "/var/lib/grafana/grafana.db" "/media/usb/openhab2-backup-$TIMESTAMP/grafana/grafana.db";

echo "+-+-+-+-+-+-+ Influxdb backup +-+-+-+-+-+-+-+"
sudo cp -arv "/etc/influxdb/influxdb.conf" "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/influxdb.conf"
sudo influxd backup "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/metastore/"
sudo influxd backup -database openhab_db "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/db/"

# restart openhab instance
echo "+-+-+-+-+-+-+ Starting service...+-+-+-+-+-+-+-+-+"
sudo systemctl start openhab2.service
sudo systemctl start grafana-server

echo "+-+-+-+-+-+-+ Clear cache and tmp folder +-+-+-+-+-+-+-+-+"
sudo rm -rf "/media/usb/openhab2-backup-$TIMESTAMP/userdata/cache"
sudo rm -rf "/media/usb/openhab2-backup-$TIMESTAMP/userdata/tmp"

# Packen
echo "+-+-+-+-+-+-+ Pack Backup Folder into tar.gz +-+-+-+-+-+-+-+-+"
tar cfvz /media/usb/openhab2-backup-$TIMESTAMP.tar.gz /media/usb/openhab2-backup-$TIMESTAMP

# Entpacken-> tar xfvz archiv.tar.gz

# Folder size
# sudo ls -1d */ | sudo xargs -I{} du {} -sh && sudo du -sh
sudo df -h /media/usb/; sudo du -sh -- /media/usb/*

For restore this SHOULD work, i have not tested yet and unfortunately don’t have the time to do so in the next couple days. So if you can test it and let us know would be perfect.
I did care also of the influxdb restore which requires a few more steps than the backup.

The $TIMESTAMP has to be replaced yet manually in the script before execution. This could be done better in a future version of the script. If you have any ideas, let me know.

#!/bin/bash
# stop openhab instance (here: systemd service)
echo "+-+-+-+-+-+-+ Stopping service...+-+-+-+-+-+-+-+"
sudo systemctl stop openhab2.service

# restore current installation with settings
#echo "+-+-+-+-+-+-+ timestamp and folder setup +-+-+-+-+-+-+-+-+";
#TIMESTAMP="`date +%Y%m%d_%H%M%S`";
#sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP";
#sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP/influxdb";
#sudo mkdir  "/media/usb/openhab2-backup-$TIMESTAMP/grafana";
#bakdir="$HOME/openhab2-backup/$TIMESTAMP";
#mkdir -p "${bakdir}"

# Sample
# $TIMESTAMP -> 20171130_054320
# search and replace it via editor

echo "+-+-+-+-+-+-+ OH restore +-+-+-+-+-+-+-+-+-+-+"
#cp -arv /etc/openhab2 "${bakdir}/conf"
#cp -arv /var/lib/openhab2 "${bakdir}/userdata"
#find /var/lib/openhab2 \( -path /var/lib/openhab2/tmp -prune -o -path /var/lib/openhab2/cache -prune \) -o -name '*' -exec cp -arv {} "${bakdir}/userdata"  \;
sudo cp -arv "/media/usb/openhab2-backup-$TIMESTAMP/conf" "/etc/openhab2";
sudo cp -arv "/media/usb/openhab2-backup-$TIMESTAMP/userdata" "/var/lib/openhab2";

echo "+-+-+-+-+-+-+ Grafana backup +-+-+-+-+-+-+-+-+"
sudo systemctl stop grafana-server
sudo cp -arv "/media/usb/openhab2-backup-$TIMESTAMP/grafana/grafana.ini" "/etc/grafana/grafana.ini";
sudo cp -arv "/media/usb/openhab2-backup-$TIMESTAMP/grafana/grafana.db" "/var/lib/grafana/grafana.db";

# https://docs.influxdata.com/influxdb/v1.3/administration/backup_and_restore/
echo "+-+-+-+-+-+-+ Influxdb backup +-+-+-+-+-+-+-+"

service influxdb stop

sudo cp -arv "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/influxdb.conf" "/etc/influxdb/influxdb.conf"
sudo influxd restore -metadir /var/lib/influxdb/meta "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/metastore/"
sudo influxd restore -database openhab_db -datadir /var/lib/influxdb/data "/media/usb/openhab2-backup-$TIMESTAMP/influxdb/db/"

service influxdb start

# restart openhab instance
echo "+-+-+-+-+-+-+ Starting service...+-+-+-+-+-+-+-+-+"
sudo systemctl start openhab2.service
sudo systemctl start grafana-server

#echo "+-+-+-+-+-+-+ Clear cache and tmp folder +-+-+-+-+-+-+-+-+"
#sudo rm -rf "/media/usb/openhab2-backup-$TIMESTAMP/userdata/cache"
#sudo rm -rf "/media/usb/openhab2-backup-$TIMESTAMP/userdata/tmp"

# Packen
#echo "+-+-+-+-+-+-+ Pack Backup Folder into tar.gz +-+-+-+-+-+-+-+-+"
#tar cfvz /media/usb/openhab2-backup-$TIMESTAMP.tar.gz /media/usb/openhab2-backup-$TIMESTAMP

# Entpacken-> tar xfvz archiv.tar.gz

# Folder size
# sudo ls -1d */ | sudo xargs -I{} du {} -sh && sudo du -sh
#sudo df -h /media/usb/; sudo du -sh -- /media/usb/*

[Austris_V]

Just to verify - it will backup also bindings and additions (e.g. zwave, expire, transformations; influx, graffana) and it’s config?
e.g. if I have it backed-up and then run restore on top of fresh openhab from openhabian - it shall recreate it all?

or I rather shall have a image ready with all the sw staff installed and the can use restore for all the configuration?

[Austris_V]

@aksnet

So if you can test it and let us know would be perfect.

will give it a try tonight. also, just to verify - your restore shall work on fresh openhabian instance? or I shall, as a minimum, install the influxdb and grafana first and the restore takes care of the content (config and data)? Thanks!

[aksnet (Andreas Kuhn)]

Hi,
it should backup everything.
And yes at a fresh install you have to install grafana and influxdb first and then restore.

BR
Andreas

[Benjy (Ben Clark)]

The two scripts backup and restore the configuration of everything inside openHAB. If you restored the backup on top of a fresh installation then it will reload the configuration exactly as it was. This also means that the bindings you had previously would be reinstalled on the next launch.

In practical terms this means that if you backed up a configuration from openHAB 2.1.0, then restored it on top of a new openHAB 2.2.0 instance. It would load the configuration and install the 2.2.0 version of the bindings.

It will also restore cross platform. The same scripts work for manual or automatic installs on any MacOS or Linux OS. Of course if you do this you’d need to reconfigure which port is used by the zwave dongles etc and the default permissions of the openHAB user.

My recommendation would be to call these openHAB scripts with the scripts for the other programs that @aksnet has posted.

[Austris_V]

@aksnet

s. So if you can test it and let us know would be perfect.

I guess I was little lucky:) Openhab was half running, but could not logon through putty (connections refused; even after few restarts). And my previously made backup, which I thought is on flash, apparently was not there. ha

Found ext2explore and used to access the sd from pc; Following your backup script, manually copied all the mentioned files to safe location (did not do proper influxdb backup though). And then repeated the same - manual copy following your restore script. had to add few sudo in front of service starts. had to fix permissions at the end (because was copying through win?). had to remember to copy jars from addons. but other than that all up and running. such a good feeling. and no more enhancements before I try full backup-restore process and become sure I have it fixed
Thank you!

And will eventually also uprgade to snapshot (again) so I have the built in scripts, too.

[homeJoe (Jochen Breh)]

Hi all,
I’m very new to OpenHab and I just want to give you my opinion on this discussion.
My background is technical and in a former life I was even a software engineer

First, the PaperUI helped me to get a jump start and I made first successful experiences. Therefore, I think it is needed for a first introduction.
Second, to understand the entire concept of openhab is really hard. In particular, making the step from PaperUI to the configuration via the eclipse designer was very hard for me. It seemed to me as two different concepts with now relation to each other
Btw, I love text file configuration, but only if I understand it

Third, backup us a real issue for me, because in my trail phase I sometime crashes my Pi or I want to go back to a previous state.
Now, as I understand the config stuff, I can backup this, but what about the PaperUI detected stuff? Best for me would be a twofold approach. 1. an export/import function for the PaperUI DB (its not important to get it human readable/editable but it must be restore-able) and 2. saving the text files (which is available today).

And last but not least, I really like and appreciate what you guys have done here. Now I need to go back to my system to learn more about it

Thanks for reading.

[sihui]

@homeJoe

but it must be restore-able

It’s there, fully automatic . You just need to copy it to another drive to be more safe:

[rlkoshak (Rich Koshak)]

Depending on what version of OH you are running, there is a backup and restore script in /usr/share/openhab2/bin (I think that is where it is). If you are running openHABian, there is a full system backup setup and configured for you as well (Amanda is what it is called I think) that you can run from openhabian-config.

You can look at that backup and restore script and see what gets backups and restored. I’m pretty sure that is is the entire contents of /etc/openhab2, /var/lib/openhab2/jsondb, and perhaps /var/lib/openhab2/persistence.

@homeJoe

but what about the PaperUI detected stuff?

It is all in /var/lib/openhab2/jsondb. Note that this is a human readable file.

[Benjy (Ben Clark)]

@rlkoshak

there is a backup and restore script in /usr/share/openhab2/bin (I think that is where it is)

From the latest stable version of openHAB (2.2) it would be easier for apt/yum/openHABian user to use the command:

sudo openhab-cli backup

Which places a file in /var/lib/openhab2/backups, which can then be restored using:

sudo openhab-cli restore /var/lib/openhab2/backups/filename.zip

This backs up both text config from /etc/openhab and PaperUI config from /var/lib/openhab/

[homeJoe (Jochen Breh)]

Thanks a lot for your immediate responses !
This is what I was looking for.

Since I’m on the latest OH version, I will try “sudo openhab-cli backup” once I finished my current step.

[onkelbobby (Ben)]

Hi all,

I’m not very familiar to Linux and have various problems to fully understand the Amanda setup therefore I’m looking for a more simple option to manually generate a FULL system backup

Is this command

sudo openhab-cli backup

generating a FULL backup of the entire system incl. all things, items, sitemaps, rules, bindings, and personal settings? can it be be used to restore openHAB on an all new openhabian installation on a new SD card?

If I follow the instructions written in the OH documentation
https://docs.openhab.org/installation/linux.html#backup-and-restore a zip file is generated containing a ‘config’ and a ‘userdata’ folder. is this enough restoring my system on a new openhabian installation?

Up to now I was using:

# backup current installation with settings
TIMESTAMP=`date +%Y%m%d_%H%M%S`;
mkdir  ~/openhab2-backup-$TIMESTAMP
cp -arv /etc/openhab2 ~/openhab2-backup-$TIMESTAMP/conf
cp -arv /var/lib/openhab2 ~/openhab2-backup-$TIMESTAMP/userdata

# restart openhab instance
sudo systemctl start openhab2.service

I’m still not sure if this is enough for a full system restore in case of SD card crash. Maybe you can help me a little understanding what I need to do for getting a working backup solution.

Is there any alternative way to simply clone the entire SD card?

Thanks for your help,
Ben

[Benjy (Ben Clark)]

all things, items, sitemaps, rules, bindings, and personal settings

The backup command for openhab will backup all these things EXCEPT manual .jar files that you’ve added yourself (if any). You’d simply need to add these back in afterwards. You may also need to add the openHAB user to the dialout and tty groups if you did this previously.

It won’t backup anything else related to Linux.

sudo openhab-cli restore <filename> can restore that file on a new system as long as it has openHAB (and Java 8) installed.

But I wouldn’t take my word for it. Generate a backup zipfile, get a new SD card and try it, that way if the restore isn’t as sucessful as you’d like, we can help with that.

@onkelbobby

Is there any alternative way to simply clone the entire SD card?

How to Clone Your Raspberry Pi SD Card for Super Easy Reinstallations

[onkelbobby (Ben)]

Hi Benjy,

Thanks for your reply. I’ll get a new SD card and give it a try within the next days.
I know the Win32 Disk Imager as a clone tool but I’m confused since there are 2 partitions on my openhabian SD card. for that reason I was asking if there is a way to clone the entire SD card including all partitions.

[michel53 (Micha Mueller)]

There is a way to clone the whole card with all partitions via linux dd command.
I use it from time to time to have an SD card backup in case the card fails.

sudo dd if=/dev/mmcblk0 of=/media/hd1/image_sd_2018_01_08.img

Backup to image file: (mmcblk0 is the whole SD-card, with both Partitions mmcblk0p1+mmcblk0p2); this results in an image file that has the size of the card.
I have an external HDD mounted at /media/hd1.
This takes 1…1.5h for a 16GB SD card on a RPI3.