Docs on how to backup openHAB

Development Documentation
96

Hi Marcus,
I think I agree with you. Backup is hard and complex, as you point out in your excellent document In that document you also point out that what risks to take is a personal choice to which there is no one answer.
I just want to point out that, to me, it feels that there is a discrepancy between building a GUI enabling people to make use of the excellent stability of OpenHab, without knowledge on eg programming, and the current options to backup your work. With the emphasis on work not e.g. data persistence. If one uses your openhabian, you have several options to backup and restore. If not your on your own in the wild, often without sufficient knowledge. The basics are available: using backup-cli you can backup and restore your system.
So I created items and rules and if I hit the switch from within the GUI, it creates a restore point. To keep track of the changes I make in the system, I have a word file in the openhab configuration folder. If I want to read what “version” I want to restore to, I read the word file inside of the backup .zips.
I know this is not a full backup and restore by far, but it is sufficient for me. (for now)

If I’m suggesting to incorporate such restore point (restore being not equal to backup) into the GUI it is just that: a suggestion. I fully understand the work that’s being done, is done so by people spending their free time and I really appreciate the hard work and the stability of OH. It is because I appreciate the work I wouldn’t find it fair to withhold my experiences, as a suggestion

97

I totally disagree with this.

Backup is a simple task itself. The issue is, the restoring of the backup. And specially if you restore to a different system/different version as the backup was made for.

Thats why its important to distinquiss between the backup procedures as well as the restoring procedures.

A simple way way to backup a computer system, is to shut down the system, and do an image copy of its data on the storing device (normally an harddrive). And the exact opposit for the restoring procedure… Thats easy, simple and will work anytime.
The disadvantage is, this procedure takes time, and its far from flexible, no automatic. And it will only get you back to the time of when the backup was created. But it will work!

In my opinion, this is a minimum backup procedure which should be build-in to any system. Even though it may require you´ll have to add a second storage device for the backup.

Backing up a system and restoring to another system (even a different version of the same system) often requires somekind of hassle, all depending on, how much and what have changed since the backup. This is where a backup procedure really become complex.
Its not impossible, but there can be many things to consider to make it realiable.

And a backup of one type of system on a specific hardware, can become an issue trying to restore to the very same system (same version) but different hardware.

It all depend on, which kind of backup you would like (or maybe pay for).

1 Like
98

Not necessarily, backup can be very difficult to understand, particularly for new users.

That is a manual backup solution, anyone can do it yes, but it is not what is being discussed here. In this we are discussing how to create a much more user friendly and intuitive backup solution, perhaps even accessible through the web UI.

99

I know. But when I read someone mentioning backup beeing complex, I simply have to disagree.
Automatic running backup procedure can be highly complexed, specielly if it should be schedule while the system is running (ie, without loosing data).

1 Like
100

still dint had a chance to test OH3, so its enough /userdata/config folder and userdata/jsondb for versioning control ? And how to share config with other users when part is for example in userdata/jsondb ? All GUI is nice in OH3, but with OH2 file configs for me looked much easier when you need to share some part of items and rules…

3 Likes
101

2 posts were split to a new topic: SD backup

102

@Kim_Andersen may I ask you what the status of your approach is? I read this thread with with great interest.

103

Thank you for the quick and dirty on how to back up!

I ran the command from putty and transferred it with WinSCP. I am thinking of doing a simple bash script to run then a cron job to schedule and transfer.

Just a note: The command changes for openhab3 from sudo openhab-cli backup-full to sudo openhab-cli backup --full

Again, thank you so much!

1 Like
104

Sorry for the very very late reply :cry:
The status is, I ran into some personal issues which meant I had to give up openhab for some time. Hopefully I will be back very soon now, (I bought a new house, and will move into it very soon). From there I will start again from scratch with OH3, (still need to look into it as I left with OH2) and ofcouse the backup solution.

105

Hmm, yes I know… I could also do it…

But hmm, there is a pretty easy way, to get a proper documentation for the openhabian bckup functions into the official documentation… It is mentioned nearly everywhere in the docs… Follow the contribute lin, goto github, fork the project, do your changes to the docs, that you feel to be right and helpful. Then do a pullrequest on your changes and voila, if it survives review, there is a new useful doc for everyone.

And yes the documentation really still is at least confusing, and maybe does not tell what u really need to do. Including at some point it even says don’t do that, better do this on one side and on the next it says, oh exactly do that but never do this… :slight_smile:

Especially when it comes to raw backup of the SD Card. And yes it should be written by someone familiar with Linux, especially debian and the concept posix conform operating systems use to view disks… Which in general means dd is the right way to get a plain copy of something. And yes it has some flaws, as it really is plain… And no using somethig with a gui is no solution for a intended headless system like openHABian…

And yes, a proper and fast to restore backup is essential here… Because the cli backup is a good thing, but misses a lot of things u may have and running on your pi… e.g. it does not safe your config rgarding the local gpios… It does not safe the setup of Mosquitto… It does not safe the configuration of tuya-mqtt and which fine fancy things u may have running on the same device for your smarthome…

And tbh Amanda is also a bit questonable solution.It’s pretty very good as long as your system is somehow running und u may wanna go back to a point before (or in the future if you had to go back to an older copy of your SD Card in order to get things running again). But the the somehow described, ohh i have a 32 GB SD Card and I clone it to a 64 GB and have amanda backing up to a third partition, forcing the replacement SD to be 64 GB at least is !!!not!!! a good idea for several reasons.

Having Amanda clone to a NAS instead of a local attached drive is a good idea, but (because of how amanda works) causes some issues… Can’t use a NAS via CIFS (samba, typical NAS for redmond orginating operation system)… even can’t use a NFS mounted drive, when the NAS doesn’t allow to turn of root_sqaush or all_squash… And there are many NAS outside that do it for several reasons… Even the “professional” NAS from the data storage experts from San Jose do all_squash and if u turn it off by connecting with the console of it, after a rebbot it restores the “settings” in /etc/fstab that they think is right to proper operate… So Amanda is a bit of issue, if u don’t have a second posix style operationg system, if it comes to failure of your system and u wanna restore it now… How nice is a plain SD Card then, to just restart the system with a maybe somewhat older state and then restore from a Amanda backup… But there is also definitely something missing in the openhabian-config tool :slight_smile:

I can do the cli export with it and restore from tht zip file with it.
I can setup regular SD mirroring and remove the mirror… (ok, there i don’t need restore, as i just change the slot of the SD)
I can RAW copy the SD and even do a manual Rsync… (also no need for restore from that, as i just can use that SD-Card to boot a pi)
But how to trigger a restore of a amanda backup??? :slight_smile:

And yes it also would be nice, if somewhere for experts would be described, what exactly the options 50, 53, 54, and 55 do… For the case one wants to do that same thing on a non-openhabian installation of openHAB :slight_smile:

1 Like
106

I appreciate the robust discussion of this topic.

I have recovered from Pi SD failure in two ways: SD card mirroring, which worked flawlessly and was easy for a newbie to set up, and restoring manual back ups made (and restored) using the openhabian cli. The only problems with the latter are that you need to have made a back up recently, and you have to remember what other programs you may have added (e.g. NUT for the UPS Binding). The advantage of Backup/Restore approach is you have a nice, clean openhabian on the Pi, without some of historical artifacts from earlier versions.

I spent hours reading Amanda documentation, but still get to a point in the openhabian setup where it asks me for information about disk space that I don’t know how to answer. I’m sure its there somewhere in the docs, but I couldn’t figure it out.

2 Likes
107

I don’t remember to be asked about the system “memory”, but for sure it asks about the diskspace u want to reserve for it… And the answer to this question according to the actual documentation of openHAB is “strange”… Or lets say it suggests some totally different values… There is mentioned:

  • 3x SD Card size, resulting in some 100 GB for me
  • SD Card size
  • SD Card size + 1GB
  • aprox 5 GB

I have no idea what is right, but I did set the 100GB as i am using a old mechanical 256GB drive and it shall not be used for anything else anyways :slight_smile: But I think it doesn’t make any sense, as I said I don’t backup the complete SD, as I do by a manual SD mirroring on a !!! same sized !!! SD. And yes I may change to automatic mirror, once i have a small enough SD Card reader at hand :slight_smile:

108

(post deleted by author)

109

Great, so why not contribute to help making our docs better instead of writing such a post.

This is not helpfull at all.

110

It’s insulting and disrespectful at best.

111

In the interest of civility, I deleted my “offensive” post.

HOWEVER - if you read the entire thread -

Person 1 (a Sustaining Member) asked for some very simple assistance and made some very interesting suggestions.

Person 2 (A Maintaining Member) has, over the course of the thread, belittled Person 1 and others for “clearly not understanding concepts or documentation and the reason for documentation”

Over the course of the thread, many others jumped in agreeing with Person 1 and holding a civil discourse with Person1.

From the git-go, after Person 1asked their questions and made their comments, the entire thread should have been a discussion of:

  1. How do we explain the difficulties
  2. Maybe we are not as clear as we could be? Maybe we DO need to improve the docs (or backup?)

Although many (later) did just this, I was and still am exceptionally equally offended by Person 2’s overall handling of their input - and Person 2 (because they donated $$?) was not helpful at all in comments but Person 2 was not “taken to task” for not being helpful.

If OH is intended to be something that is useful and understandable by advanced developers only and not hobbyists who hope to learn, then advertise it that way right from the first statement. I do not think that is the intent. Everybody here should be treated with respect (And I apologize for MY outburst) but the early conversation in this thread was not useful at all and would drive most hobbyists away from wanting to even say anything again.

Sorry - I AM disappointed in a large portion of this thread’s “tone”. And if you all would prefer I go find another hobby?

1 Like
112

Ok, guys,
please calm down all and let’s come back on topic.

I did not read the complete topic and I am not going to, so I will focus on the topic itself.

Might be that some parts of the docs are not completely clear to everybody, cause we are mostly not really used to write technical documentation.
If we have somebody here who is able to do such stuff, thats fine.

I would strongly recommend to summarize what is unclear and how this could be documented in a better way. This will help us all to improve our docs.

1 Like
113

Now, the useful response.

Sorry to those who disagree. But, I agree 100% with Kim’s original post that started the thread.

I understand fully the “difficulties” of backup. And what has to be backed up. Our global software has the same issue. Not only do we have to “back up” (in the more traditional sense) a certain set of directories, we have to “back up” certain areas of the registry. And worse yet? Users are allowed to place their own program files wherever they want.

Restore? Well - indeed, a nightmare if you are not using the same version. We have to know how to “migrate” many settings. Yuck.

But, in the end? For our product to be a reliable product for the end user, we had to - write a backup routine. That dealt with most of the above. And what we can’t deal with? We added to our - documentation.

So - our backup program:

  • Backs up a set of directories as well as several registry key areas. It is all saved (in our case) in a zip file. (The registry is stored in XML to help restore to a different “version”).
  • Restores by placing the directories in the new location (may be same version, may not be) and puts the registry back where it goes (same version or not) AND… modifies settings as required for the new version.
    *Must be locale aware (*some EU is a bit of an issue as we have to be “,” vs “.” aware. Asian can REALLY not be fun).

Our documentation states (reduced here):

  • How to run the Backup and the Restore programs.
  • How best to move from an old PC to a new PC (in particular if you are upgrading versions - the best is to either upgrade and transfer, or install the old version on the new PC, transfer, then upgrade)
  • A general discussion on what we can’t backup (we don’t know where you place your files) and how you, as an end user should deal with that.

Whether complex or not for OH, I think the above would directly address Kim’s original statement request. And I agree with Kim. I think that this is critical to a production program (even for hobbyists).

User documentation needs to be careful to address the “how to do this”. Users don’t care about the underlying details.
Developer documentation clearly must deal with the details!

My opinion here in general for several areas is that many documentation items are meant for users, but are written by developers and have much more technical detail than what the user needs. It took me years to learn to take off my developer hat and:

  • Write docs from a user perspective
  • Write docs that are clear in any cultural context (what works for the US does not necessarily work for the EU or Asia and vice-versa. Although it is possible to make it clear to all). And – not use local references that are meaningless in other cultures or, in some case, offensive in other cultures!
1 Like
114

Note: Installation. I struggled a bit on my first attempt by reading the documentation. And a few out here were kind enough to explain my issues and put me on the right path. And then, going back and reading the documentation, the documentation is not (entirely) off base. I do have a couple of very minor tweaks (as in two or three sentences overall) to add that would have cleared up all my confusion.

116

Please do make those changes and submit a PR. There is a link at the bottom of every page in the docs that will take you to where you can make edits and submit a PR. After a review the changes will almost certainly be accepted and merged. See [Wiki] How to contribute to the openHAB Documentation for an excellent guide.

The docs are the way they are because those who have volunteered to write them did the best that they could. I’m not sure there are any certainly not many contributors who have actual training in technical writing. So if you have suggestion or contributions, please dive in and create a PR. It’s not hard and working together we can make the docs better.

As for the rest, all I’ll say is we all have bad days and some days are worse than others. There are also language translation issues (the majority of users on the forum do not have English as their first language) as well a cultural issues and what comes across as rude to some may be standard to others.

Regardless, please don’t resurrect a years old thread to argue with what someone said or how they said it. We’ll never get anything done if we have to rehash and defend ourselves because we said something years ago in a less than kind way.

You’ll notice that a number of threads automatically get marked as closed after a certain amount of time of inactivity for this purpose (and because it’s easier for everyone to open a new thread rather an reopening an old one). Unfortunately when we made that change we couldn’t apply it retroactively. So for posts made in the last year or so they auto close but for posts older than that like this one that they remain open.

1 Like