Openhab broken after system upgrade

Hi,
I have been running openhabian on a raspberry pi for years with no major issues so far. After making a backup today I thought it would be a good thing to do a system upgrade – including an OH upgrade from 4.2.0 to 4.2.2. After a couple hours, I came to realize that the system was broken: I could still access the WebUI but no sitemaps, rules or items were detected (those are file-based) and my things showed as “Not Yet Ready”. I tried to stop openhab, clear the cache and then start it again, to no avail. When I do so, the logs are swarmed with a lot of errors, way to many to be shared here in their totality, so here are just a few:

[ERROR] [internal.PersistenceRuntimeActivator] - bundle org.openhab.core.model.persistence.runtime:4.2.2 (204)[org.openhab.core.model.persistence.runtime.internal.PersistenceRuntimeActivator(25)] : The activate method has thrown an exception
java.lang.LinkageError: Error reading class bytes: com.google.common.collect.ImmutableSet
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:631) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClassImpl(ClasspathManager.java:607) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClassImpl(ClasspathManager.java:587) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClass(ClasspathManager.java:566) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.ModuleClassLoader.findLocalClass(ModuleClassLoader.java:335) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:397) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.sources.SingleSourcePackage.loadClass(SingleSourcePackage.java:41) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.BundleLoader.findClass0(BundleLoader.java:479) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:416) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:168) ~[org.eclipse.osgi-3.18.0.jar:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
	at com.google.inject.internal.util.SourceProvider.<clinit>(SourceProvider.java:41) ~[?:?]
	at com.google.inject.internal.Errors.<init>(Errors.java:111) ~[?:?]
	at com.google.inject.internal.InternalInjectorCreator.<init>(InternalInjectorCreator.java:65) ~[?:?]
	at com.google.inject.Guice.createInjector(Guice.java:87) ~[?:?]
	at com.google.inject.Guice.createInjector(Guice.java:69) ~[?:?]
	at com.google.inject.Guice.createInjector(Guice.java:59) ~[?:?]
	at org.eclipse.xtext.common.TerminalsStandaloneSetupGenerated.createInjector(TerminalsStandaloneSetupGenerated.java:45) ~[?:?]
	at org.eclipse.xtext.common.TerminalsStandaloneSetupGenerated.createInjectorAndDoEMFRegistration(TerminalsStandaloneSetupGenerated.java:39) ~[?:?]
	at org.eclipse.xtext.common.TerminalsStandaloneSetup.doSetup(TerminalsStandaloneSetup.java:18) ~[?:?]
	at org.openhab.core.model.persistence.PersistenceStandaloneSetupGenerated.createInjectorAndDoEMFRegistration(PersistenceStandaloneSetupGenerated.java:21) ~[?:?]
	at org.openhab.core.model.persistence.PersistenceStandaloneSetup.doSetup(PersistenceStandaloneSetup.java:26) ~[?:?]
	at org.openhab.core.model.persistence.runtime.internal.PersistenceRuntimeActivator.activate(PersistenceRuntimeActivator.java:29) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:569) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:245) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:687) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:531) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317) ~[?:?]
	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307) ~[?:?]
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:354) ~[?:?]
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) ~[?:?]
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:1002) ~[?:?]
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:975) ~[?:?]
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:776) ~[?:?]
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:674) ~[?:?]
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:437) ~[?:?]
	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:671) ~[?:?]
	at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:310) ~[?:?]
	at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:593) ~[?:?]
	at org.apache.felix.scr.impl.Activator.access$200(Activator.java:74) ~[?:?]
	at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:460) ~[?:?]
	at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196) ~[?:?]
	at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169) ~[?:?]
	at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:49) ~[?:?]
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:488) ~[osgi.core-8.0.0.jar:?]
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:420) ~[osgi.core-8.0.0.jar:?]
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232) ~[osgi.core-8.0.0.jar:?]
	at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:450) ~[osgi.core-8.0.0.jar:?]
	at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:949) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEventPrivileged(EquinoxEventPublisher.java:229) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:138) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:130) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor.publishModuleEvent(EquinoxContainerAdaptor.java:217) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.container.Module.publishEvent(Module.java:499) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.container.Module.start(Module.java:486) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:445) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:464) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.apache.karaf.features.internal.service.BundleInstallSupportImpl.startBundle(BundleInstallSupportImpl.java:165) ~[?:?]
	at org.apache.karaf.features.internal.service.FeaturesServiceImpl.startBundle(FeaturesServiceImpl.java:1160) ~[?:?]
	at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:1041) ~[?:?]
	at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1069) ~[?:?]
	at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:1004) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.io.IOException: /var/lib/openhab/cache/org.eclipse.osgi/38/0/bundleFile
	at org.eclipse.osgi.storage.bundlefile.CloseableBundleFile$BundleEntryInputStream.enrichExceptionWithBaseFile(CloseableBundleFile.java:570) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.storage.bundlefile.CloseableBundleFile$BundleEntryInputStream.read(CloseableBundleFile.java:534) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.storage.StorageUtil.getBytes(StorageUtil.java:172) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.storage.bundlefile.BundleEntry.getBytes(BundleEntry.java:98) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:627) ~[org.eclipse.osgi-3.18.0.jar:?]
	... 71 more
Caused by: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature)
	at java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:928) ~[?:?]
	at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:939) ~[?:?]
	at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:456) ~[?:?]
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) ~[?:?]
	at java.io.FilterInputStream.read(FilterInputStream.java:132) ~[?:?]
	at org.eclipse.osgi.storage.bundlefile.CloseableBundleFile$BundleEntryInputStream.read(CloseableBundleFile.java:532) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.storage.StorageUtil.getBytes(StorageUtil.java:172) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.storage.bundlefile.BundleEntry.getBytes(BundleEntry.java:98) ~[org.eclipse.osgi-3.18.0.jar:?]
	at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:627) ~[org.eclipse.osgi-3.18.0.jar:?]
	... 71 more

This is the very first entry logged in openhab.log after a reboot. There’s two more errors even before Starting openHAB 4.2.2 (Release Build) is logged.

[ERROR] [e.model.rule.scoping.RulesClassCache] - bundle org.openhab.core.model.rule:4.2.2 (205)[org.openhab.core.model.rule.scoping.RulesClassCache(27)] : Error during instantiation of the implementation object
[ERROR] [ntime.internal.ThingRuntimeActivator] - bundle org.openhab.core.model.thing.runtime:4.2.2 (216)[org.openhab.core.model.thing.runtime.internal.ThingRuntimeActivator(32)] : The activate method has thrown an exception

I can share more informations about these two but I’m trying not to be (too) overwhelming.

After that, the same warning keeps repeating, dozens of times per seconds:

[WARN ] [mmon.WrappedScheduledExecutorService] - Scheduled runnable ended with an exception: 
java.lang.LinkageError: Error reading class bytes: com.google.common.collect.Iterables
	at org.eclipse.xtext.xbase.lib.IterableExtensions.filter(IterableExtensions.java:274) ~[?:?]
	at org.openhab.core.model.thing.internal.GenericThingProvider.handleXmlThingTypesLoaded(GenericThingProvider.java:902) ~[?:?]
	at org.openhab.core.model.thing.internal.GenericThingProvider.onReadyMarkerAdded(GenericThingProvider.java:886) ~[?:?]
	at org.openhab.core.internal.service.ReadyServiceImpl.lambda$0(ReadyServiceImpl.java:51) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179) ~[?:?]
	at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1850) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?]
	at org.openhab.core.internal.service.ReadyServiceImpl.notifyTrackers(ReadyServiceImpl.java:75) ~[?:?]
	at org.openhab.core.internal.service.ReadyServiceImpl.markReady(ReadyServiceImpl.java:51) ~[?:?]
	at org.openhab.core.config.core.xml.osgi.XmlDocumentBundleTracker.registerReadyMarker(XmlDocumentBundleTracker.java:417) ~[?:?]
	at org.openhab.core.config.core.xml.osgi.XmlDocumentBundleTracker.finishBundle(XmlDocumentBundleTracker.java:360) ~[?:?]
	at org.openhab.core.config.core.xml.osgi.XmlDocumentBundleTracker.processBundle(XmlDocumentBundleTracker.java:383) ~[?:?]
	at org.openhab.core.config.core.xml.osgi.XmlDocumentBundleTracker$2.run(XmlDocumentBundleTracker.java:345) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.io.IOException: /var/lib/openhab/cache/org.eclipse.osgi/38/0/bundleFile
	... 26 more

I also get a bunch of

[WARN ] [.core.internal.folder.FolderObserver] - bundle org.openhab.core.model.core:4.2.2 (197)[org.openhab.core.model.core.internal.folder.FolderObserver(38)] : Could not get service from ref {org.openhab.core.model.core.ModelParser}={service.id=157, service.bundleid=204, service.scope=bundle, osgi.ds.satisfying.condition.target=(osgi.condition.id=true), component.name=org.openhab.core.model.persistence.runtime.internal.PersistenceRuntimeActivator, component.id=25}
[ERROR] [ntime.internal.ThingRuntimeActivator] - bundle org.openhab.core.model.thing.runtime:4.2.2 (216)[org.openhab.core.model.thing.runtime.internal.ThingRuntimeActivator(32)] : The activate method has thrown an exception
java.lang.NoClassDefFoundError: Could not initialize class com.google.inject.internal.util.SourceProvider

I cannot make any sense out of any of it. I suppose that something I updated is no longer compatible with openhab, but I have no idea what or how to figure it out. My guess was java but the version looks just fine:

$ java -version
openjdk version "17.0.12" 2024-07-16
OpenJDK Runtime Environment (build 17.0.12+7-Raspbian-2deb11u1rpt1)
OpenJDK Client VM (build 17.0.12+7-Raspbian-2deb11u1rpt1, mixed mode, emulated-client)

I uninstalled every bundle, deleted every config file just to be sure it wasn’t coming from some syntax error but it did nothing (which honestly wasn’t that surprising since the bug seems to appear even before openhab starts). Restoring my backup from this afternoon doesn’t help either.

What confuses me the most is that openhab seems to be running fine up to some degree: I can access both the log viewer and the webUI, manage the bundles from there, but not the things, access BasicUI, but no sitemaps will show…

I have found nothing either on the forum or the rest of the internet which could give me a clue as to what is going on. I’m at my wit’s end. If anyone as any idea, it will be greatly appreciated.

Thanks,

If you use file based config/rules, did you move all rules and other files out of the config folder? So that it is empty?

Did you clean and purge the cache? Remove the temp and cache folder?

Finally can you share the log file from the start until it goes ‘haywire’ +1 minute or so?

Hi Isiepel and thank you for your help!

did you move all rules and other files out of the config folder?

Yes I did, but it didn’t change a thing.

Did you clean and purge the cache? Remove the temp and cache folder?

Yes I did, to no avail.

Finally can you share the log file from the start until it goes ‘haywire’ +1 minute or so?

I apparently cannot, since “new users can not upload attachments”, sorry. Just so you know, that file’s weight is almost 1MB. Also, it immediately goes haywire, that first error I shared earlier is the very first line of the log file.

Update: OpenHab seems to be working again. I think what did the trick was simply rebooting. Trying to do so via command line failed (the server never went back online) and I had to physically unplug the RPi for it to turn back on, but after doing so I don’t get anymore errors during the setup and the system has been stable for the past couple minutes.

I did a couple of other things before rebooting and I don’t know what’s been useful, so I’ll list them here just in case: reinstalled Java, entirely removed and reinstalled openhab, used openhabian-config to fix permissions and apply various tweaks, switched off every device / script / software susceptible to make several call to openhab every second.

I’ll never know what went wrong but at least now it works.

Totally forgot to ask if you tried turn it off and on again :joy: