HowTo: Manage OpenHab 2 configurations

[Dim (Angelos)]

1. Intro
This tutorial is based on an apt-get installation of OH2 in Linux and is for advanced/manual configuration of OpenHab 2 and for troubleshooting purposes. The normal way of working with OpenHab 2 configurations is via the Paper UI and the config files in /etc/openhab2.

2. Basics
OpenHab 2 is based on the Eclipse SmartHome framework and is fully written in Java. It uses the Apache Karaf (a lightweight and modular container, powered by OSGi) to store many information, including all the configurations (items, bindings, sitemaps, etc).

Karaf remembers the configs for the bindings that you installed and started by storing them in your userdata directory (/var/lib/openhab2) in a mapDB database format (?) and/or the /usr/share/openhab2/runtime/karaf/etc directory.

3. File Based Configurations
In some cases, you may need to modify the configuration files within your /etc/openhab2/ folder in order to manually work with bindings, items, things, etc. These config files are acting as read-only and OpenHab 2 copies the info into it’s internal database/directories and you can edit them there via various UIs. You may not be able to access the stored info directly with text editing tools.

Example:
In order to configure the KNX Binding, you need to modify the file: /etc/openhab2/services/knx.cfg
After you store the configuration in the knx.cfg file, OpenHab will copy the info and maintain it in its internal mapDB/dirs.
Any changes that you may do later on in the knx.cfg file will be re-read and the info will be again stored in the OH2 mapDB/dirs, overwriting the existing parameters.
You could actually delete the /etc/openhab2/services/knx.cfg after OH2 has loaded up the KNX Binding configuration and all would work fine since OH2 will read the Binding config from it’s already stored info. This is not recommended since you should keep your config files in place for future reference.

4. Working with OH2 stored configs
In order to display, change, and/or delete already stored configurations within the internal OH2 mapDB/Dirs, you can use the Apache Karaf Console:

sudo ssh openhab@localhost -p 8101

with password: habopen

You will be presented with the Karaf console prompt:

                          __  _____    ____      
  ____  ____  ___  ____  / / / /   |  / __ )     
 / __ \/ __ \/ _ \/ __ \/ /_/ / /| | / __  | 
/ /_/ / /_/ /  __/ / / / __  / ___ |/ /_/ /      
\____/ .___/\___/_/ /_/_/ /_/_/  |_/_____/     
    /_/                        2.0.0-SNAPSHOT
                               Build #558   

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown openHAB.

openhab>

Some useful Karaf console commands:
a) Show all existing configs: config:list
b) Show a specific config for a binding: config:list "(service.pid=org.openhab.knx)"
c) Show all bindings: feature:list
d) Show the versions of installed bindings: bundle:list
e) Delete an existing config: config:delete org.openhab.knx
f) Delete only one property from an existing config: config:property-delete -p org.openhab.knx localIp
g) Set a property for an existing config: config:property-set -p org.openhab.knx localIp 172.16.13.101

Warning: Changes made in the configs via the Karaf console will not be recorded in your /etc/openhab2/ config files and these files will be re-read by OH2 at startup, overwriting the OH2 stored config parameters.

5. Backup & Restore of your OH2 configs
5.1 Backup

systemctl stop openhab2
TIMESTAMP=`date +%Y%m%d_%H%M%S`;
mkdir /backup
mkdir  /backup/openhab2-backup-$TIMESTAMP
cp -arv /etc/openhab2 /backup/openhab2-backup-$TIMESTAMP/conf
cp -arv /var/lib/openhab2 /backup/openhab2-backup-$TIMESTAMP/userdata
systemctl start openhab2

5.2 Restore

systemctl stop openhab2
cp -arv /backup/openhab2-backup-YYYMMDD_HHMMSS/conf/* /etc/openhab2/
cp -arv /backup/openhab2-backup-YYYMMDD_HHMMSS/userdata/* /var/lib/openhab2/
systemctl start openhab2

BR,
Dim

References:
i) Recommended way to backup/restore OH2 configurations and things?
ii) What is effect of -Dosgi.clean in karaf?
iii) Migration from OH 1.x to OH 2 Tutorial
iv) http://karaf.apache.org/manual/latest/#_shell_console_basics

[watou (John Cocula)]

Great tutorial, thanks!

Any changes that you may do later on in the knx.cfg file will be re-read and the info will be again stored in the OH2 mapDB/dirs, overwriting the existing parameters.

What happens when you remove a configuration key from the .cfg file (like home.port for example) – does it also get removed from the internal configuration? I believe the answer is no, which can lead to configuration problems. If this is correct, I would recommend either 1) opening an issue to correct the bug, or short of that 2) further steps in the documentation/tutorial describing manual steps to properly clean the internal configuration. This issue could be resolved in openHAB 1.x by restarting the runtime, which used the osgi.clean=true system property to delete all old configurations, but I’m unaware of an equivalent in openHAB 2/ESH.

[Dim (Angelos)]

What happens when you remove a configuration key from the .cfg file (like home.port for example) – does it also get removed from the internal configuration? I believe the answer is no, which can lead to configuration problems.

You are right… It does NOT get removed… See an example here: KNX Binding Configuration Changes cause Exception and another here: MQTT config under OH2
I haven’t opened an issue yet… Maybe I should

This issue could be resolved in openHAB 1.x by restarting the runtime, which used the osgi.clean=true system property to delete all old configurations, but I’m unaware of an equivalent in openHAB 2/ESH.

In theory, there are 3 ways to do the same in OH2:

1. Start Karaf with the clean option: /usr/share/openhab2/start.sh clean (which basically executes: /usr/share/openhab2/runtime/karaf/bin/karaf clean and performs the clean option: rm -Rf "$KARAF_DATA/**")
   or
2. Modify: /usr/share/openhab2/runtime/karaf/etc/system.properties and set: karaf.clean.all = true and/or karaf.clean.cache = true
3. Shutdown the Karaf console with the option (system:shutdown --clean-all)

I tried all 3 methods but I couldn’t get them to work properly

Also… there is another issue here…

In OH2, the KARAF_BASE directory is set to the same as KARAF_DATA (set in /usr/share/openhab2/runtime/karaf/bin/oh2_dir_layout)
If you delete the KARAF_DATA to clean the internal stuff, you will have problems because you will delete the KARAF_BASE also. The recommendation is to have different dirs for these 2.

[watou (John Cocula)]

There was also another issue related to the Ecobee binding, where stale configuration was the cause.

I haven’t opened an issue yet… Maybe I should

Dimitris, you are clearly the knowledgable person to properly articulate these points in a new issue report! I would be very happy if you did.

[rlkoshak (Rich Koshak)]

The normal way of working with OpenHab 2 configurations is via the Paper UI and the config files in /etc/openhab2.

I wouldn’t say this. PaperUI is great but it isn’t really complete. For example, you can’t set configuration parameters on zwave devices nor can you create sitemaps and such. A number of users, to include Kai, do not use PaperUI at all.

I’d say “an alternative way of working …”

in a mapDB database format (?)

This is correct.

Great write up!

[mbuchberger1967 (markus buchberger)]

Hi,
not sure if this is the right track.

I installed openhab2 some days ago. I started writing my config files for sitemaps, things, items,etc. but somehow I still got data in my system that probablly belongs to a _default site that was created on startup somehow while playing around.

I cannot see the things or items in the UIs, but I still see the site in HABmin and the inconsisten data of those things/items causes a lot of exceptions during startup.

How can I reset my openhab2 installation to a clean state?
All tips here in the first post do not work and I get errors…

Thanks for your help.
Markus

[Udo_Hartmann]

If all of your configuration is in files (no configuration with paper UI beside addon installation), you could simply backup these files and make a new installation (don’t choose demo!).

However, you could use the REST API or karaf console to get rid of these outworn elements (you will have to find them manually…)

[Dim (Angelos)]

Hi @mbuchberger1967,

There has been a major change since I wrote the original post with regards to how OH2 stores the configs. See here:

Replacement of mapdb by jsondb

All, I feel sorry to tell you that the distro build #624 brings another breaking change, but as in the past, I am certain that it is for the better! There has been a long discussion around the problem that the Things that are created through the Paper UI are stored in a binary database (the mapdb, data to be found under userdata/mapdb), which is not readable or manually editable (e.g. for fixing or cleaning). @chris came up with a nice new implementation of a storage service (luckily, ESH is…

Many of the commands in the original post still work but they are more related to bindings and their configurations (not so much for items, sitemaps, things, etc)

I would go the way that @Udo_Hartmann recommends (keep your customized configuration files and reinstall)

Another way would be: Karaf console, then:

smarthome:links clear
smarthome:items clear
smarthome:things clear

This will clear out whatever OH2 has stored internally. Make sure that you have removed the _default files, keep your files in the configuration directories and OH2 will re-read them and populate the internal (json) database.

[ThomDietrich]

Hey @Dim, @Udo_Hartmann, @rlkoshak and @watou,
that’s quite an interesting discussion you’ve got going on here. Would you mind to rephrase whatever is fitting for a) the FAQs or b) a docs article? We need to start organizing information, especially with the upcoming attention for the openHAB 2.0 GA release…

[MarsLoucetios (John Patterson)]

is it really necessary to ssh into the karaf console to change configuations? openhab is installed on the system im working on so should i not be able to access these files directly with a text editor?

[mstormi (Markus Storm)]

Err … you edit files from within Karaf ? Didn’t even know that’s possible …
Don’t confuse that with the most common way which is to SSH into the machine. Or share the config directory via network to a desktop computer and edit it there, preferrably using VS code extensions nowadays.

[karld (KD)]

Is there a way to dump the config done from paperUI, so I can use it in the items/service/thing configuration files ?

This is because I prefer to have a stored in file copy of everything. paperUI is great when starting, but having a config file version is easier to recover (for me)

[karld (KD)]

For instance I did setup my rfxCOM this morning and did setup all the rollershutters.

Things were created in paperUI (easier for syntax), but items and sitemap done in config files. I’d like to port all my “things” from paperUI to config files (as I understand, those will be read at restart or at any new start, so also after an upgrade)

[rlkoshak (Rich Koshak)]

There is no automated tool to export Things from the JSONDB into .things files. Furthermore, not all bindings support Things defined in .things files yet. So depending on what bindings you have, it may not be possible to go all text config. Those bindings that do support .things files will have the syntax defined in their README.

For the Zwave binding, support for .things files only exists in the development branch.

Finally, if you were not aware, everything you do in PaperUI is saved in a text file of sorts. In /var/lib/openhab2/jsondb there are several JSON formatted text files which is where all your Things and Items that you create in PaperUI get saved. It is text-based and human readable. And it might be possible for an enterprising individual to create a script to convert the JSONDB to .things files, but as each binding has its own parameters and syntax to some degree it might require special code for each binding to make it work.

[metin]

Regarding the questions of configuring things in json db vs. things in files, may I ask for a general clarification:

Which way is the way openHAB is finally aiming at?

From what I understood, configuration files are read and then copied to the database. Also, the REST API provides read and write access to the database. So I assume that the database is where it all should eventually end, am I right?

[rlkoshak (Rich Koshak)]

Which way is the way openHAB is finally aiming at?

Both will remain supported for the foreseeable future. However, JSONDB will become the default and recommended approach at some point. There are still a number of features missing from PaperUI to make this possible though.

Form what I understood, configuration files are read and then copied to the database.

This is not correct. OH reads the Items from the .items file and reads the Items from JSONDB and stores them internally in memory. It does not copy the Items from the .items files to the DB. The two are kept separately.

[Dim (Angelos)]

However, JSONDB will become the default and recommended approach at some point.

I believe the same.

Don’t laugh (i did this in 3 seconds)… here is my understanding (this is oversimplified):

All config inputs (PaperUI, $OPENHAB_CONF, REST, etc) interact with the ConfigAdmin which stores data in HashMaps
Some of that data end up in the jsonDB (not all) and some other in Userdata

for example:
/var/lib/openhab2/config/org/openhab/addons.config contains the parameters that we set in /etc/openhab2/services/addons.cfg (this file is parsed, and entered in ConfigAdmin with config pid: org.openhab.addons)

I would also like to have everything being dumped from the OSGi ConfigAdmin into jsonDB

Of course, it is much more complicated than I described above (with Managed Items, etc) and I am reading more on the topic. There are many nice resources out there: example: http://karaf.apache.org/manual/latest/#_configuration_3

[metin]

This is not correct. OH reads the Items from the .items file and reads the Items from JSONDB and stores them internally in memory. It does not copy the Items from the .items files to the DB. The two are kept separately.

Thanks for clarifying. I had read somewhere that config files are copied and that, therefore, items which were removed from the config file, are not deleted from the internal database and need to be deleted from it manually.

[Dim (Angelos)]

I had read somewhere that config files are copied and that, therefore, items which were removed from the config file, are not deleted from the internal database and need to be deleted from it manually.

this is true for configuration (*.cfg files) related parameters (not for items)

If you remove an item from a flat *.items file, it should be removed also from OH2.
Items coming from flat files are not stored in an another destination (in $OPENHAB_USERDATA or jsonDB or somewhere else) after they have been loaded by OH2 in memory. Same applies for Things in flat files as well.

There is a workaround the stale configs issue: place the config pid as the first line in your *.cfg
This will overwrite any existing stuff in the ConfigAdmin and as a result, clean the output *.config file from old stuff and place only the new parameters coming from the *.cfg file

example for addons.cfg:

pid:org.openhab.addons
package = expert
ui = ...
voice =....
so forth

This workaround for the stale configs applies to all configurations (*.cfg files) that have a config PID… e.g. mqtt, influxdb, mail, etc, etc… mostly stuff in /etc/openhab2/services/

This is because the ConfigAdmin will read both *.cfg and *.config files and load all parameters.
Stuff in *.cfg take precedence but if you remove (not change) a parameter from *.cfg, it does not get removed from the *.config

[rlkoshak (Rich Koshak)]

This is the first I’m really seeing this explained. A quick tutorial post would be great so it is easier to find and link to when the stale config parameters issue pops up on the forum.