Recommended way to backup/restore OH2 configurations and things?

Hi there, my first post :slightly_smiling:

As @martin_klimke said, Ui’s often mean more work then simple editing text files.
Also I dint’t like a mix of settings in DB and text files.
Best for me: All config in text files (as in OH1) and let the Ui edit those files. So you can make changes within the Ui and in the text files. Not possible with a little regex magic?

4 Likes

@Kai can you enlight us a little … See #13

I would also agree that this is a good approach. In HABmin, for the ‘HABmin specific configuration’ (like chart configuration, floorplans etc) I write everything out using XML. This provides a fully flexible format that’s srtuctured and easily extensible, but still editable in a text file. It would however be slower than mapdb, but as always, there’s a compromise and it’s probably not an issue…

I would guess that it’s probably possible to replace the mapdb implementation with something else (I believe the architecture supports this although I’ve not looked at it myself). If so, something structured like XML might work quite well.

(just my 2c worth :wink:)

… For things added from habmin2 is it possible to have a switch… “Add to XML” … “Add to mapdb” as a user choice? :slight_smile:

No - that won’t work.

I think it will be possible though to replace mapdb with an XML version. I would guess it’s probably not too hard, but I might be completely wrong as I’ve not looked at it!

(Ok, I just looked at the code, and I’m pretty sure this should be possible - it’s a service, so changing to a different system shouldn’t be an issue, and making this read/write XML should be (relatively!) ‘simple’ :wink: )

1 Like

Would it be possible to backup the userdata folder as a whole and overwrite the userdata folder that comes with the distribution?

Well, in general a full backup comprises the conf and the userdata folder, that’s the idea. Note that the distribution does not come with any userdata folder at all, so this does not get overwritten either.

That is, if a thing is setup in a textual file, it should be taken as it is defined and Paper UI should not discover a new copy of this thing.

But should it also not be possible for the Paper UI to edit the Thing? Or have a binding add dynamically properties to it, which it read from the device (e.g. the Tesla binding adds the info that you have a sunroof)?

Currently, if I am right, you will have two instances for the same thing in this case ?

No, it a thing with the same UID already exists (being defined in a things-file), you will not see it in the inbox.

let the Ui edit those files. So you can make changes within the Ui and in the text files.

There are a few reasons against this:

  • the conf folder is meant to be read-only and only the system admin should have write-access to it. This is something I would want to keep, because if you open it up for the runtime to access it, the user has no idea of what stuff is changed in there without his knowledge. Well, we could say that the runtime copies this stuff to userdata and edits it there, so it would be a kind of import mechanism for static files.
  • The current files are parsed through Xtext. While I agree that the Designer is meant to be a nice tool, I also see that most people are actually using vi or emacs instead. And having Xtext as a part of the runtime has the problem that it is quite heavy purely for handling configuration files and that there do not seem to be people in the openHAB community, who can deal with it, so that simple bugs stay open forever. For these reasons, I would only want to keep the existing files and their infrastructure for OH1 compatibility, but not really build on top of it to also allow editing them through Xtext (what Xtext in general supports).
  • Not using Xtext for writing files would probably mean that a lot of formatting, comments, etc. get lost within the files.
  • The Xtext files are not meant to be a “db replacement”. Doing edit operations through the REST API can cause quite some heavy db activity as it is not done as bulk, but in many small transactions. Even the MapDB implementation seems to be sometimes too slow to handle it properly on a RasPi.
  • By far not all configuration is handled through openHAB/Xtext itself. If you install a new Karaf feature on the console, there is no way to have that serialized into the “addons.cfg”. Likewise all changes about logging configuration or other services. You can do a hell of a lot in Karaf and all such settings are stored somewhere in the OSGi internals according to good OSGi manners…

For these reasons, I think the best approach is to go with the possibility of having the “old” files as a kind of import and treating the database as THE point of reference.

I think it will be possible though to replace mapdb with an XML version.

Yes, ESH allows plugging in different storage implementations. We are actually serializing all stuff to JSON, so this would be my preferred format.
But you need to be aware that you will end up implementing a complete database - with consistency checks, recovery, internal caches, indexes, query optimizations, etc. I am not too sure that this is the way to go.
An alternative might be to accept the binary (and thus efficient) format of MapDB and rather provide an “export to text” mechanism. This could be e.g. triggered automatically once in a while or upon a “backup” request.

FTR, you might also want to read this discussion here: https://www.eclipse.org/forums/index.php?t=msg&th=1072808&goto=1718043&#msg_1718043

3 Likes

Wouldn’t it be possible to have a simple “Show config as plain text” option in the PaperUI, so the configuration can be copied and pasted to the files in the conf dir? This way you can tinker in the UI and persist the configuration when you think it’s finished.

What is a “config”, if you drag&drop stuff in the UI and silently in the background create an item that has a link to a newly created thing that you use within a rule? I think when allowing UIs to do stuff, you need a database and cannot think of everything as an isolated “config file” any more.

By that “if paper UI can edit things that are defined by a text file” would mean
A) paper UI edits are stored in map DB. And the whole thing is a mixture of mapdb and textfile?
B) paper UI does edit Text files only for these things?

Sorry I wasn’t really clear about what I meant: I wasn’t talking about one “config”. It would simply be helpful to export the configuration for a single service, just like I can define the parameters when I click “configure” in the PaperUI.

But maybe I simply don’t fully understand the way openHAB 2 works - my current problem is the following: I have an installation with RWE-Smarthome items (which is a v1 binding) and some hue lightbulbs. The rwesmarthome binding dumps a sample configuration to the log on initialization, so I can copy and paste this to my items file - however I have to adjust the names of course.

I’d like to do the same with the hue bulbs which are configured as things in the mapDB so I can group them together etc. in one place.

interesting… so mapdb to be replaced is already in the works?
any rough time plan for this to be in the beta?

By that “if paper UI can edit things that are defined by a text file” would mean

C) paper UI edits are stored in map DB and they overrule the text file (which can be removed/deleted/regarded as an “old import”

so I can group them together etc. in one place.

You can already now define a single item file, which mixes items and groups items that are linked to rwe and to hue at the same time. Whether the thing definition is in a file or in the db does not really matter there.

interesting… so mapdb to be replaced is already in the works?
any rough time plan for this to be in the beta?

No, these were just ideas being discussed.
As mentioned above: I think mapdb is quite efficient and I would rather suggest thinking about an export/import mechanism to text files. There is no time plan, since I do not know anyone working on this.

I think the problem is when you define the thing in a file after the discovery and save in mapdb.
I could try to reproduce the issue if necessary.

What was making openHAB strong and the way to go for me, was that everything was visible in text files. There was nothing hidden somewhere in a binary configuration I was not able to see completely (but potentially overruling my textual definitions).
The only thing that was not so perfect (and still is with openHAB2) was the Designer. It was only partially able to codecomplete the XTEND syntax which was making some easy things unnecessary complicated for less technical users (type casting for instance).

In my first tests with openHAB2 I was faced with the disadvantage that those constellations could lead to. Was was using the Astro binding. First the new one, with limited functionality (right now). Later the 1.9 version with my old Item definitions. I was failing to make the Items work even if the binding was loaded and seems to work (cron for the next day was in the log etc.)
No matter what I did (removing the addon and re-add it etc). No chance. Sooner or later I just deleted the mapdb (and maybe some other stuff) re-added the 1.9 binding and I worked perfectly.

But there was no chance for me to understand why.

So from my point of view. Items, Things and Rules should remain Text only. And if there is an option to edit them via Paper UI it should change them there. Or if needed create a overrule file to show in text that there are modification done in the ui which are active or something…
Having two places in parallel where one might overrule the other but I do not see if and why is scaring me …

2 Likes

@Lars:
Exactly the same happened to me and I could not agree more to your conclusion.

I do not really understand the benefit of the whole Paper UI.
Creating items was never a big deal and the autodiscovery is only nice to have.

The problems I am always struggling with is the convoluted and poorly documented DSL of the rule files. Designer is mitigating this disadvantage quite a bit. Without designer I would have stopped using openhab.

For me the first priority to improve the user experience would be to fix this tool and make it useable also for reasonable file sizes and remove other bugs.

For ‘advanced users’ - yes - that’s true. But many people spend a lot of time getting all the binding strings sorted. This is especially true of bindings like z-wave which are quite complicated to configure. The number of questions we get on configuring devices is huge - with the OH2 features, once this is sorted for a device, everyone benefits by reduced configuration.

Clearly there are different ways this could be done, and different people have different views - a lot of people do want something more ‘point and shoot’ while some clearly don’t (as is clear from this thread). Personally, I like the point and shoot of a nice GUI, but I also do like the option of being able to ‘get back to basics’ and edit the configuration easily if something goes wrong, or we want to paste a load of similar configurations quickly…

ESH, is a flexible framework so it’s possible to do ‘anything’ of course - if enough people want a feature, then it should be possible to make it happen…

(just my thoughts :wink:)

@chris
Please do understand me wrong I do not hate the paper ui. But it would not be my first priority to improve the ease of use. My priority would be everything that makes the creation of rules less complicated due to the complexity of the DSL and the class definitions.

1 Like

I fully agree and that’s why I tend to having the two modes co-exist: All textual as people are used to do it with OH1 for power users and a “I am not a programmer and I just want the stuff to work” simply mode for beginners (with possibly reduced possibilities). The question is really where to draw the line best and how to enable beginners becoming power users. I think an export of discovered things to a thing file is definitely something that makes sense and should not be too hard to implement. But reflecting any possible configuration in the shell or a UI in a nicely structured and formatted config file is not really feasible.

Note that there are no priorities of the project itself. It evolves by people contributing code to it. And the majority of contributions to ESH in the past two years is about setup and configuration of the system through UIs.

@Kai.
I know that what is developed is what the developers like.
This is clear. User are not paying so they do not have a right to talk about priorities.
It is all about developers having fun.

I have got the message.

I know that what is developed is what the developers like.

No, you got it completely wrong here. Note that a majority of contributions to ESH come from DT and they invest money in having developers implement what their users want. And guess what: DT customers/users do not want to configure their smart home through text files!

2 Likes