All my things disappeared! How do I get them back?

Tutorials & Examples Solutions
1

Scope

This solution is limited to PaperUI defined Things. If you are using .things files this document will not apply to you.

Where are the Things stored?

In a manual installed instance of OH are stored in a file in

$OPENHAB_HOME/userdata/jsondb

For installed instances of OH (e.g. openHABian, apt installed, yum installed) they are stored in a file in

/var/lib/openhab2/jsondb

There may be several files in this folder. It should be obvious which one stores the Things, but if your Things have disappeared, you will want to do the steps described below for all the files.

Why did the Things disappear?

It’s hard to say. They may be corrupted. OH periodically writes out to the JSONDB files and if OH is killed (e.g. loss of power to the machine) before it completes a write then the file will be incomplete and likely malformed. If you are running off of an SD card, loss of power instead of cleanly shutting down can corrupt files. In rare circumstances, an SD card wears out causing writes to no longer take or completely take which can result in corrupted files.

How to recover?

Periodically OH makes a backup of the JSONDB files. You will find these in the backup folder under the jsondb folder. Each backup is prepended with a timestamp in epoch (i.e. number of milliseconds since Jan. 1, 1970 00:00:00). The most recent backup will be the file with the largest number in front of it and the most recent modification date.

To recover your Things (Items, Links, Rules, etc.):

  1. Stop openHAB. openHAB does not read these files except when starting so if you modify them while OH is running, your changes will likely become overwritten instead of applying.

  2. Copy the most recent backup of all the files from backup to the jsondb folder, removing the timestamp from the front of them. It is a good idea to recover all your files so you have a complete backup of the state of your JSONDB at a given point in time. Don’t just recover the Things file.

  3. Start openHAB running again. Watch the logs for errors.

If there were no errors in the logs, check in PaperUI to see if your Things are back. If not, you may need to restore one backup prior to the most recent one.

3 Likes
2

@Benjy, do you think this might be something worth adding to openhab-cli?

1 Like
3

If it’s something that happens more than rarely, perhaps even better to add to the distribution’s restore script!

4

Thanks @rlkoshak! I just wish I had read this a few days earlier… :smile:

5

It happens enough that I’ve written a separate tutorial. I see this posted about at least once a week.

Because the JSONDB is backed up separately automatically, does it make sense to have to restore the whole backup when all that is corrupt is the JSONDB? The rest of the config may be fine and the JSONDB backups will almost always be more recent than the backup taken from oepnhab-cli. Maybe there can be an option to just restore the JSONDB files?

6

Yeah that’s where I was coming from, I would vote for building it into openhab’s restore scripts rather than just make it a openhab-cli feature for apt/yum users. E.g.

/opt/openhab2/runtime/bin/restore --jsondb

And powershell alternative for the Windows users.

1 Like
7

I just encountered this issue and your tutorial saved my life :wink:

I remember, that I had this issue a few times (3-4 times in more than 5 years), but it is still very annoying.

How about a script extracting the data from the things json file and create .thing files?
If that’s even possible / reasonable.
I personally prefer text files for rules, items and such, like a lot of others - I guess.

8

The new UI that will be released with OH 3 will have an import wizard that you can use to import Items and Things from text configs.

9

Thanks - sounds good.

and an export wizzard as well? :slight_smile:

10

I doubt there will be an exporter to .items and .things file formats. But I’ve seen some screenshots where you can get at a YAML version of rules. I would not be surprised if there is something similar for Items and Things. You can download and play with the 3.0 snapshot and explore the UI as it exists right now to see what is there and what appears to be planned for.

Personally, based on my experience with the HestiaPi, the stability and performance gains achieved by running everything out of the JSONDB text configs are worth even putting up with PaperUI and doing through through the REST API. There are no startup problems of any sort. No runtime errors. No Items unexpectedly going to NULL when changing a Rule. On the RPi 0W I took the boot time down from >30 minutes to almost exactly 10 minutes. In those rare cases where OH crashes while writing out to the JSONDB there is always a backup to restore to (unlike .items files which can be corrupted by a loss of power and there is no automatic backup there unless you set it up yourself). OH simply works soooo much better I just can’t see any advantage to using .things and .items files. When I move to OH 3, I’m going to be 100% in JSONDB. I’ll still have all my stuff checked into git and all the workflow stuff that everyone claims as advantages of text based configs and thus implying is not possible with JSONDB.

But I’m more willing to bend my personal workflow to match what best works for the tools I’m using rather than trying to impose my preferences on the tools. Many others are not willing to do so and I’ve no doubt that .items files and .things files will remain supported, or there will be a tool to translate them to a new text file format.

2 Likes
11

Thanks Rich,
Maybe you are right.
The first time I lost all my things I had to do everything from scratch and that’s extremely annoying.
For me (as a non programmer) it just feels better and seems to be more transparent if I have all my OH configs as text files on my PC, change them every now and then and then just copy the changed files over to my RPi with SCP.
Maybe I will learn myself that there are better ways - so let’s wait for OH 3 :wink:

12

Thanks for the great tutorial! It completely saved my ass from restoring everthyng from scratch…

13

This happened to me this morning. I should be able to restore from one of my SD backups tonight.

14

Thanks a ton @rlkoshak! Just happened to me and saved me from restoring a full backup.

15

I experienced this issue now the third time…is there something I can do to prevent this? For example can I turn on some mount options or so?

16

There are a whole lot of variables but in general this happens when openHAB stops/crashes when it is in the middle of a write. So the first thing to do is ensure your machine never just loses power. You might need to deploy and UPS. Always make sure OH closes down nicely, no matter how long it takes. Never just kill the service.

17

Okay I see… last time this happened OpenHab had a heap error…I then stopped the docker container and all things were gone…how can I shut down docker nicely when it hung up? I would also be for a CLI function for restoring :slight_smile:

18

Well, if OH crashed with an error it’s already too late. OH isn’t going to be able to write out the files any more even if shut down nicely. In other cases use docker stop.

19

Okay I see… interesting is, that only the things database gets corrupted…how is it for the others? Did other databases also fail for you?

Thanks @rlkoshak for the explanation :slight_smile: Another method of keeping things from that Event would be to define them manually over text…

Is there a way to export all things to Text? Or would I have to redefine them? The textual configuration is much more durable because it endours all kinds of shutdown…am I right here?

20

Any one of them can fail. It’s just the luck of the draw as to which DB is being written to when OH crashes.

The best way is yo figure out and correct the crash in the first place.

You will have to redefine them.

They bring along their own problems and limitations. I personally don’t recommend them but others use them.