Recommended way to backup/restore OH2 configurations and things?

Why do some of us like the text files? For me:

  • You have an easy overview what is defined (items, rules…). No magic settings frustrates when debugging - the error must be in some of those config files.
  • You can copy+past: No 20 Buttons click and text input fields for add a view items
  • simple backup and use old versions by saving/replace the text content

This could also be possible inside Paper UI with „source view“ editing or something like this. Therefore I din’t need the text files.
Maybe isn’t there a way for „source view“ in the UI? E.g. in a XML structure with DB field name and value (if needed between CDATA sections).

And… keep it simple. I thought about frustration when you didn’t know an error is from a wrong setting somewhere in the UI or in some text file, or just a problem e.g. with keep text files an DB in sync.

1 Like

Why not use some standardized db instead of a binary one? There are a lot of good options out there (i.e. NoSQL, etc). That way “text” users can easily modify the db, and non-users don’t need to.

It’s interesting…I’m a web developer, and more than capable of figuring out how to do text configs, but far prefer using the UI (except for rules). That said, I would really like it if I could tweak by directly in the database. I think of it as being very similar to what many web apps do: have a UI front-end with something like mysql (not suggesting that as the solution…would be a poor choice in this case)backing it. I can always hop into mysql and work directly there if I need to access the data.

And then for those who are migrating from OH1, have a way of importing the text files.

I hope its OK to add a little of my own thoughts. I think the current use of the Text Files is still a very important feature, at least until OH2 develops further along and perhaps there is some type of standardization in the bindings and item formatting.

I’m still relatively knew to OpenHab, but I do use the Kodi (formerly XBMC) and the Chamberlain MyQ bindings. And currently those bindings don’t necessarily generate items when you install them. I’m sure there are probably a bunch of other binding similar that are really customized specifically to one’s use and don’t necessarily create an item file.

So how do you back your up that information and items for these non typical bindings?

I think shorty707’s comment gets close to what I think could be a viable solution to making text files and other confuration mechanisms coexistst, as Kai Kreuzer puts it.

A key issue in this discussion so far has been the fact that config files are read-only (and I think they should be), which makes overriding anything they specify difficult to do, if you don’t want to either confuse your users and at the same time make the system accessible.
Hence, the two (read-only config files + dynamic configurations) must be unified with some finesse.

I’d like to see a tiered system, where OH2’s 1st place to look for config files is the conf/ directory.
The 2nd place to look, would be any form of dynamic configuration that was done through the (web) UI/karaf shell. This 2nd tier could also be a way for OH2 to override whatever is defined in conf/.

In order to not confuse power users, there should be a facility to prevent such overrides. E.g. by adding a final: true flag to a configuration file’s section, users can stop OH2 from ever creating any other representation of this section.
At the same time, the (web) UI should contain a section that clearly lists all configuration bits and pieces - regardless of whether the plugins involved support this kind of introspection - and for each of those display where it was loaded from, and whether it was marked final. On top of that, a button should also be added to each, that erases any overriding definition and reverts the respective config back to what is defined in the text file.

Lastly, a reliable & universal config export facility can help power users and beginners alike to understand in what ways changing the settings in the (web) UI would affect their textual configs. This feature could look like a button next to each config item in the aforementioned list, which opens an exported view of said config item, including all overriding properties that were created from the (web) UI.

I would like to extend this logic not just to the configuration of plugins, things and sitemaps, but also to the plugins themselves, meaning the binaries that power each plugin.

As you can tell form my comment, I think most of the issues brought forward in this discussion can be managed by providing visibility into what OH2 is doing, as well as means to explicitly & interactively control that.

Interesting topic. I ended up here trying to figure out how to manage an OH2 installation with a configuration management tools, in my case Ansible.
My goal was to be able to keep all the OH configuration, including Things, Items, Rules, Sitemaps, and also stuff like log level etc in a central location, and in case I want to deploy a new instance of Openhab (for instance for redundancy or if the hardware breaks down).
So I really want every part of the configuration to be in some sort of text format, like json, xml or something else.
How am I supposed to be able to keep my configuration in a central place and version controlled if some config is in a dynamic DB or the OSGI framework “somewhere”?

That also make it much more easy to troubleshoot in my opinion.

Why not make Paper UI optional (or maybe that is the plan)? So that we who love text files could continue just using that. And to attract a broader user base offer full functionality through the Paper UI.
But I believe mixing the two is going to lead to confusion.

And for the rules engine/language, to make that easier why not build on that Blockly concept in Habmin? I havent used it myself, but I guess that would really be easy to understand for most people.
A better documentation of the Xtend language used (specific to OH, with examples) could also help.

Thats just my 5 cents.
TL;DR: How do I manage the OH2 configuration with Ansible or some other configuration management tool? And to version control it?
If Paper UI means mapdb, then make it optional.
Blockly for rules + documentation, why not?

1 Like

This is not just the plan, this is how it is! You can use OH2 exactly the way as OH1, i.e. you can define all details of your setup on the “conf” folder and that one should be all you need to backup, if you want to reconstruct your instance (with a few exceptions like the logging configuration, which is still at a different place).

The Paper UI approach is meant for people who do not want to deal with text files, but prefer UIs.
Technically, it is not really feasible to make the UIs editing the files in “conf”, so that these two ways would be more or less doing the same. I agree that this is confusing and we have to work on solutions that make this easily understandable and usable for people. For all the tech-savvy people, my clear advice for the moment is to stick to the textual configuration - you do not need the Paper UI at all for that.

3 Likes

Hi all

I understand one way to do backup is to
1 put all configurations in conf/ directory, and copy conf/ for backup
according to:
Recommended way to backup/restore OH2 configurations and things?

However, am I correct to say, the way to backup can be
2 If somehow UI(e.g. Habmin2) has been used, both conf/ and userdata/ should be copied for backup?

  • Is this way of backup is totally impossible?
  • Or instead of userdata/, some subdir should be copied?

Thank you very much.

All, I have started looking deeper in the problems that exist with backup/restore/upgrade and you find the current state at https://github.com/openhab/openhab-distro/issues/299 - please feel invited to help on it, be it by clever ideas how to solve problems or by testing PRs that mean to address issues in this area.

After reading through most of the previous answers I want to emphasize this statement. Seems like there is a very common misconception among new openHAB2 users.

PaperUI is not needed to accomplish what you want to do with openHAB2! After having my initial difficulties I completely abandoned PaperUI from my personal workflow and am 100% relying on conf files in my OH2 setup. The PaperUI is just one way of interacting with the openHAB 2 core system. Configure your bindings in addons.cfg, services in the corresponding *.cfg files, your things in the things folder, items in the items folder, rules in the rules folder, … you get it :wink:

My personal feeling is, that people first interacting with openHAB2 are drawn to the interesting new PaperUI feature and get misleading expectations. We are trying to solve this problem in the documentation, which is supposed to grow strong in the next two months. Please anyone with clear input on what the documentation should describe, take part in the discussion at https://github.com/openhab/openhab-docs/issues/110

In another thread I already proposed my idea of an approach to unify the “classic” configuration and the database. Here’s the short version:


I want to vote towards the database approach in combination with what I and others mentioned before:

  • Thing/item discovery and adoption through PaperUI
  • Thing/item handling and modification though PaperUI
  • Increase of UI functionality over time
  • While there are not all functions available in PaperUI, mainly for the tech-savvy user:
  • Direct database access through some existing frontend like phpmyadmin (not offered for mapdb? mapdb alternative an option?)
  • More powerful options on the karaf console (random example: smarthome:items 5 rename 'Bathlight')
  • RAW-view edit through PaperUI (compare http://docs.grafana.org/datasources/influxdb -> “RAW”)
  • Export and import of all relevant data from the database to a human-readable file format like xml (as a last resort)

My proposal supports three things:

  1. A clear transition from not-ideal items files to a more suited and system-managed database
  2. Providing the right tools to aid the tech-savvy users to adopt to the new system while not being restricted to a UI
  3. Motivate tech-savvy users to use AND contribute towards a powerful PaperUI
10 Likes

So as the next Beta is just around the corner and im always tempted to update to the latest and greatest. If im already on a snapshot with the new json db. and want to create a Backup . Would this process work now? 1. Backup conf/ userdata. 2. Update via apt get ( yes im on raspberry). 3.Install all my bindings. 4. restore conf/ userdata? Or is this something which will be introduced in a future Build and i will at least have to setup everything once again?

This was the state in May 2016. In the meanwhile the jsondb was introduced, which stores the configuration in text files too. Should I derive .items and .things files from it in order to upgrade to a new openhab2 version without loosing my configuration?

Upgrades no longer wipe out your config and Jsondb is human editable so if you prefer to do stuff in PaperUI there is nothing stopping you.

That sounds great! Since #299 isn’t closed yet I thought this is still in work. Do you know if the new upgrade mechanism works for apt/deb-based installations as well?

Yes. And the upgrade mechanism didn’t change so much as the location and format of the data files changed. There are still a couple of minor outstanding issues, such as the branding you see when you log into karaf not updating to show the build number but for the most part it is safe to proceed.

Thanks, Rich.

Can i still install whole openhab2 into one single folder like the manual install on OH1?

So i can only copy the whole folder to my backup-drive and all will be good. Now with OH2 storing the installation into many folders in different directorys, i have to backup many folders manually.

But it think the apt-get update/upgrade will only work, if i use OH2 with the given installation folders on many different places? If i use OH1 all-in-one folder i will have to update OH2 manually by simply copy the new files to the install-place and overwrite the old files?

That shouldn’t be a problem. That’s how every Linux software works. Backup is as simple as this.
I’d recommend the package repository based installation in general as a few things are managed for you, makes it easier for the end user.

It is a really funny thread, can someone not create an add-ons which create a portable zip file with all stuff hidden from user?

A download button, and upload button is more than enough (copying files in background like here http://docs.openhab.org/installation/linux.html#backup-and-restore)

regards

5 Likes

I made a backup (most of my config was done through the paper ui) as per http://docs.openhab.org/installation/linux.html#backup-and-restore and the resulting files were over 90MB compressed. lulz – that’s just embarrassing. I certainly hope there’s a better solution on the way. I strongly believe having the UIs store data in something human-editable would be a big improvement, for configuration purposes. These binary databases can’t be diffed, can’t be meaningfully stashed into version control, can’t be manipulated outside the UI… and it’s the recommended method for new users. Having the system r/w text files, or json files, or xml files, or… really just about anything else seems like it’d be a big improvement. I worry that OH2 is overengineered at every turn, like many enterprise java apps these days. Forcing a one-time upgrade and edit of OH1 text files into OH2 files (probably via some kind of tool, if needed) seems easy enough, but I’m sure this was debated already long before I started trying OH2. Oh well.

But seriously – 100MB of “configuration data” in a backup? It seems like the vast majority is in the ‘cache’ dir. Can that be omitted? What about ‘tmp’? Maybe update the backup script? Or will things go poorly if ‘cache’ isn’t restored properly, for example?

That has already been done month ago:

You may safely delete it.

All your installed bindings are stored in tmp. That directory will be created again after upgrading, so can be deleted, too.