All Things are gone after restarting openHAB


I still use Ubuntu 20.04 and openHAB 3.2.0-1. After restarting openHAB I have no more Things. They are gone. All bindings are installed correctly. And all the items are there too. But logically, without Things I can’t operate any devices.

Off course I restarted openHAB multiple times. I also deleted the content in /var/lib/openhab/tmp and /var/lib/openhab/cache after stopping openHAB.

If I open the org.openhab.core.thing.Thing.json file (in the /var/lib/openhab/jsondb directory) with my editor, I can also see all missing things. But sadly if openHAB is started there are no things available. That’s confusing.

I didn’t find anything in the log files that could indicate an error. On the other hand, I don’t know what to look for. The log files are also extremely large. Of course, I could miss something.

Has anyone here had a similar problem and perhaps knows a solution? So far I have only found a solution by deleting tmp and cache or if you have a backup, how to restore the file described above with the Things. But this is apparently not the problem here.

Thanks in advance.

You need to share some logs

I think I should share a bit more knowledge.

The following file locations are used:

File location
openHAB application /usr/share/openhab
Additional add-on files /usr/share/openhab/addons
Site configuration /etc/openhab
Log files /var/log/openhab
Backups folder /var/lib/openhab/backups
Service configuration /etc/default/openhab

I have also checked that everything is assigned accordingly for openhab:openhab.

What I carried out was:

sudo systemctl stop openhab
sudo rm -r /var/lib/openhab/tmp/*
sudo rm -r /var/lib/openhab/cache/*
sudo systemctl start openhab
sudo systemctl restart openhab

I also stopped openHAB, and moved away the org.openhab.core.thing.Thing.json. Then I have done the steps above. Then I moved back the org.openhab.core.thing.Thing.json file. I thought maybe for reinitializing this could help.

I also restarted like described above openHAB and moved also configuration files. My thought was that maybe there was an error and before reading the Things the system was overloaded. But it seems this could not be the error.

Also I have changed the file locations in /etc/default/openhab. But this could not be the problem because I never changed using the default paths. Off course I set the default paths. And also it could not be the problem because all bindings are installed after restarting openHAB. So using as example the /usr/share/openhab/addons or /var/lib/openhab/... paths are working.

At least I have reinstalled openHAB with sudo apt install --reinstall openhab and restarted it like described above.

It’s not as if all the Things have disappeared. I have stored a few very few things in the openHAB configuration folder in text form. These are also read in correctly when the system is restarted. Only all Things, and that should be more than 90% of my Things that I created via the UI, are missing.

Attached are the log files for the events. The other one was too big, I have to upload a map separately.

events.log (300.3 KB)

Here the openhab.log. It is zipped and you can download it here:

Log file shows

2023-03-09 14:32:37.037 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110157103--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:32:37.176 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110146877--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:32:37.193 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110145179--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:32:37.231 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110143340--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:32:37.234 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110141664--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:40:22.219 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110157103--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:40:22.221 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110146877--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:40:22.223 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110145179--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:40:22.234 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110143340--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.
2023-03-09 14:40:22.236 [WARN ] [] - Json storage file at '/var/lib/openhab/jsondb/backup/1678110141664--org.openhab.core.thing.Thing.json' is empty - ignoring corrupt file.

So you moved back org.openhab.core.thing.Thing.json while openhab already was running …?
That may be the reason why your file is corrupted and things disappeared then.
I don’t know what the OH does when it starts and does not find a thing json file. I would expect it to create an empty one but what happens in case it is overwritten with one containing the previous collection of things. It may be overwritten again with content from runtime and then your things are gone.

That’s not causing the error. The error exists before I have done that.

I saw that the JSON file was not complete. So there is information missing. With the description of a thing, the file breaks off. Among various backup files, I then took one from half a year ago and replaced this file. Accordingly, I have significantly fewer Things. At least that’s what I thought. In reality, even reading in the backup file does not work. I stopped openHAB and deleted the cache before replacing the file. Ergo, there must be another error somewhere.

Probably your files are on ZRAM and if there is no storage left such corruption can occur.

If you upgrade to OH 3.3 or newer there is a fix to prevent such corruption:

I found out that the data was lost because the server crashed. So the json files were not written completely accordingly.

I have restored the org.openhab.core.thing.Thing.json. I have compared with a backup file what was still missing in the corrupt file. This works a bit better than the old backup, which was still designed for openHAB 2.

However, I still have a problem even though I stopped openHAB, cleared the cache and started it again.

The first problem is:

15:32:01.890 [ERROR] [org.apache.cxf.jaxrs.utils.JAXRSUtils] - Problem with writing the data, class, ContentType: application/json
15:32:01.900 [ERROR] [.internal.JSONResponseExceptionMapper] - Unexpected exception occurred while processing REST request.
java.lang.IllegalArgumentException: Last segment must not be blank: [zwave, device, a61d9ee8, node2, ]
        at org.openhab.core.common.AbstractUID.<init>( ~[?:?]
        at org.openhab.core.common.AbstractUID.<init>( ~[?:?]
        at org.openhab.core.thing.UID.<init>( ~[?:?]
        at org.openhab.core.thing.ThingUID.<init>( ~[?:?]
        at org.openhab.core.thing.ChannelUID.getThingUID( ~[?:?]
        at org.openhab.core.thing.internal.ThingRegistryImpl.getChannel( ~[?:?]
        at org.openhab.core.thing.internal.ChannelStateDescriptionProvider.getStateDescription( ~[?:?]
        at org.openhab.core.thing.internal.ChannelStateDescriptionProvider.getStateDescriptionFragment( ~[?:?]
        at org.openhab.core.internal.service.StateDescriptionServiceImpl.getMergedStateDescriptionFragments( ~[?:?]
        at org.openhab.core.internal.service.StateDescriptionServiceImpl.getStateDescription( ~[?:?]
        at org.openhab.core.items.GenericItem.getStateDescription( ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at$2( ~[?:?]
        at$3$1.accept( ~[?:?]
        at java.util.HashMap$KeySpliterator.tryAdvance( ~[?:?]
        at$WrappingSpliterator.lambda$initPartialTraversalState$0( ~[?:?]
        at$AbstractWrappingSpliterator.fillBuffer( ~[?:?]
        at$AbstractWrappingSpliterator.doAdvance( ~[?:?]
        at$WrappingSpliterator.tryAdvance( ~[?:?]
        at java.util.Spliterators$1Adapter.hasNext( ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody( ~[bundleFile:3.4.5]
        at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.serializeMessage( ~[bundleFile:3.4.5]
        at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.processResponse( ~[bundleFile:3.4.5]
        at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.handleMessage( ~[bundleFile:3.4.5]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept( ~[bundleFile:3.4.5]
        at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage( ~[bundleFile:3.4.5]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.ChainInitiationObserver.onMessage( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.ServletController.invokeDestination( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.ServletController.invoke( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.ServletController.invoke( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest( ~[bundleFile:3.4.5]
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet( ~[bundleFile:3.4.5]
        at javax.servlet.http.HttpServlet.service( ~[bundleFile:3.1.0]
        at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service( ~[bundleFile:3.4.5]
        at org.eclipse.jetty.servlet.ServletHolder.handle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle( ~[bundleFile:9.4.46.v20220331]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle( ~[bundleFile:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle( ~[bundleFile:9.4.46.v20220331]
        at ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle( ~[bundleFile:9.4.46.v20220331]
        at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle( ~[bundleFile:?]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.servlet.ServletHandler.doScope( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.session.SessionHandler.doScope( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle( ~[bundleFile:9.4.46.v20220331]
        at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle( ~[bundleFile:?]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.Server.handle( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1( ~[bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.HttpChannel.dispatch( [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.HttpChannel.handle( [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.server.HttpConnection.onFillable( [bundleFile:9.4.46.v20220331]
        at$ReadCallback.succeeded( [bundleFile:9.4.46.v20220331]
        at [bundleFile:9.4.46.v20220331]
        at$ [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask( [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce( [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce( [bundleFile:9.4.46.v20220331]
        at [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( [bundleFile:9.4.46.v20220331]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$ [bundleFile:9.4.46.v20220331]
        at [?:?]

And then I have multiple times problems with the error Cannot inform the listener "org.openhab.core.thing.internal.ChannelLinkNotifier@XXXXX" about the "ADDED" event: Last segment must not be blank: [zwave, device, YYYYY, node2, ] java.lang.IllegalArgumentException: Last segment must not be blank.

The Z-Wave devices were also the ones listed last in the org.openhab.core.thing.Thing.json file.

The org.openhab.core.items.Item.json file is empty.

Which makes me wonder. Is it not possible to simply re-read everything? I don’t know now where there is another mistake. All bindings are installed. Things are also all present. Ultimately, this means that there is an error in the channels between Things and Items or in the Items.

The REST API still says: SyntaxError: JSON.parse: expects ',' or '}' after the property value in the object in row 1 column 1375599 of the JSON data.

Nah. A file is never corrupted because of ZRAM. Don’t put up rumours, please.
It can be empty if you ran out of (ZRAM) space when writing or it can be outdated because it was recovered on (re)boot so you’re β€˜set back’ in time to when it was written back to disk/SD last time.
But it’ll be just as corrupted as it was when it was saved for the last time (read: not).
And by the way the jsondb directory isn’t on ZRAM.
And by the way that’ll be on openHABian on Raspi only. But OP said he’s using Ubuntu.

I missed that but Ubuntu is a nice OS and I also like using it especially on very new hardware! :slight_smile:

1 Like

I today have the same behaviour as written here. I am on latest OH 3.4.4 Release build, I had a power cut and now all org.openhab.core.thing.Thing files in the jsondb folder and the backup folder underneath are empty with 0kb.

Lost all my items now. :frowning:
How can that happen? I’ll check whether I can find a backup to use…

Device is a raspberry Pi.


it happened to me to i’m using OH 3.4.4 too, any ideas how to fix?

I have the same problem. For me, all things defined in a things file disappear after oh or system restart, and reapper if the file is touched or saved again when oh is running.

Tried all above suggested fixes, nothing works.
Log files give no indication of any issue.

I even upgraded to 4.10 from 4.0.3 to check it that solved the issue. It didn’t.

Any pointers to further debug the issue?

Found something in the logs

2023-10-20 18:11:05.025 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Adding files in '/etc/openhab/things' to the model
2023-10-20 18:11:05.029 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Missing parser for 'things' extension, added ignored path: /etc/openhab/things/mqtt.things
2023-10-20 18:11:05.034 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Missing parser for 'things' extension, added ignored path: /etc/openhab/things/gpstrackers.things
2023-10-20 18:11:05.037 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Missing parser for 'things' extension, added ignored path: /etc/openhab/things/things.things
2023-10-20 18:11:05.043 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Missing parser for 'things' extension, added ignored path: /etc/openhab/things/telegram.things

Above entries are way above the entry for parser invocation.

2023-10-20 18:11:09.012 [DEBUG] [org.openhab.core.model.core.internal.folder.FolderObserver                      ] - Adding parser for 'things' extension

So whats seems to be happening is that the by the time the parser is initialised various files have been added to the β€œignored path”.

How does one fix the issue - seems parsers need to get loaded ahead in the startup sequence ?

It’s normal and it should process the ignored paths once the parser is added.

When the parser is added and it completes processing the ignored things paths it should log:

Finished processing ignored paths for β€˜things’ extension. X ignored paths remain

Thanks, I get log entries for rules and scripts, but not for things. That probably explains why the others load but not things.

Final entry is

Finished processing ignored paths for β€˜rules’ extension. 7 ignored paths remain

How to debug further ?

The easiest is if you post the complete logging so we can have a look. :slight_smile:

done :slight_smile:

I have enabled trace only on some components, please let me know if you need levels increased for others.

Logger                                                          β”‚ Level
*                                                               β”‚ WARN
ROOT                                                            β”‚ WARN
e.internal.SseItemStatesEventBuilder                            β”‚ WARN
javax.jmdns                                                     β”‚ WARN
javax.mail                                                      β”‚ WARN
openhab.event                                                   β”‚ WARN
openhab.event.AddonEvent                                        β”‚ WARN
openhab.event.ChannelDescriptionChangedEvent                    β”‚ WARN
openhab.event.InboxUpdatedEvent                                 β”‚ WARN
openhab.event.ItemAddedEvent                                    β”‚ WARN
openhab.event.ItemChannelLinkAddedEvent                         β”‚ WARN
openhab.event.ItemChannelLinkRemovedEvent                       β”‚ WARN
openhab.event.ItemRemovedEvent                                  β”‚ WARN
openhab.event.ItemStateEvent                                    β”‚ WARN
openhab.event.RuleAddedEvent                                    β”‚ WARN
openhab.event.RuleRemovedEvent                                  β”‚ WARN
openhab.event.RuleStatusInfoEvent                               β”‚ WARN
openhab.event.StartlevelEvent                                   β”‚ INFO
openhab.event.ThingAddedEvent                                   β”‚ INFO
openhab.event.ThingRemovedEvent                                 β”‚ INFO
openhab.event.ThingStatusInfoEvent                              β”‚ INFO
openhab.event.ThingUpdatedEvent                                 β”‚ INFO
org.apache.cxf.jaxrs.sse.SseEventSinkImpl                       β”‚ WARN
org.apache.karaf.jaas.modules.audit                             β”‚ WARN
org.apache.karaf.kar.internal.KarServiceImpl                    β”‚ WARN                             β”‚ WARN                                  β”‚ WARN
org.apache.sshd                                                 β”‚ WARN
org.eclipse.lsp4j                                               β”‚ WARN
org.jupnp                                                       β”‚ WARN
org.openhab                                                     β”‚ INFO
org.openhab.binding.generic                                     β”‚ WARN
org.openhab.binding.homeassistant                               β”‚ WARN
org.openhab.binding.mqtt                                        β”‚ INFO
org.openhab.binding.mqtt.generic                                β”‚ TRACE
org.openhab.binding.mqtt.homeassistant                          β”‚ WARN β”‚ WARN
org.openhab.core.model.core.internal.ModelRepositoryImp         β”‚ TRACE
org.openhab.core.model.core.internal.folder.FolderObserver      β”‚ TRACE
org.ops4j.pax.url.mvn.internal.AetherBasedResolver              β”‚ WARN
org.ops4j.pax.web.pax-web-runtime                               β”‚ WARN
penhab.event.ItemStateEvent                                     β”‚ WARN

openhab.log (137.4 KB)

1 Like

Thanks for the logging. It seems to confirm one hypothesis I had for why the issue might occur. I think what happens is that parsers are added by other threads during activation. In that case it doesn’t process the ignored paths so I’ll create a fix for that scenario. During my own testing this never happened… probably because I’ve tested using faster hardware.